Maker Pro
Maker Pro

What Nyquist Didn't Say

T

Tim Wescott

Jan 1, 1970
0
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
 
J

Joerg

Jan 1, 1970
0
Hello Tim,

I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

Very nice. Should be distributed to universities so the kids learn some
real stuff.

Re "3.2 Nyquist and Signal Content": If Nyquist would have listened to
some of today's content (digital radio etc.) he'd have said that it
ain't worth sampling it :)

I like the wording "line in the sand". Isn't that how Archimedes started
studying his circles? They didn't need any white board with the smelly
marker pens.
 
J

John Larkin

Jan 1, 1970
0
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.


Pretty good. My only quibble is the various statements about what
Nyquist said and didn't say.

http://en.wikipedia.org/wiki/Nyquist-Shannon_sampling_theorem#Historical_background


"Exactly how, when, or why Nyquist had his name attached to the
sampling theorem remains obscure. The first known use of the term
Nyquist sampling theorem is in a 1965 book[4]. It had been called the
Shannon Sampling Theorem as early as 1954[5], but also just the
sampling theorem by several other books in the early 1950s."

It was actually Shannon (among others) that did the sampling theorem;
Nyquist made an observation. Your bibliography doesn't cite either of
them. It's probably correct to use "Nyquist rate" but not "Nyquist
theorem."

John
 
J

Joerg

Jan 1, 1970
0
Hello John,
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.


Pretty good. My only quibble is the various statements about what
Nyquist said and didn't say.

http://en.wikipedia.org/wiki/Nyquist-Shannon_sampling_theorem#Historical_background


"Exactly how, when, or why Nyquist had his name attached to the
sampling theorem remains obscure. The first known use of the term
Nyquist sampling theorem is in a 1965 book[4]. It had been called the
Shannon Sampling Theorem as early as 1954[5], but also just the
sampling theorem by several other books in the early 1950s."

It was actually Shannon (among others) that did the sampling theorem;
Nyquist made an observation. Your bibliography doesn't cite either of
them. It's probably correct to use "Nyquist rate" but not "Nyquist
theorem."

Nyquist published his paper about the minimum required sample rate in
1928. Shannon was a kid of 12 years back then. The paper wasn't about
ADCs or sampling in today's sense but about how many pulses per second
could be passed through a telegraph channel of a given bandwidth.
 
J

Jim Stewart

Jan 1, 1970
0
Tim said:
If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

"2.1 Aliasing

By ignoring anything that goes on between samples the sampling process
throws away information about the original signal. This information
loss must be taken into account during system design."

This seems like something of an oversimplification. If the orginal
signal is naturally or otherwise bandwidth-limited to well below
2x the sample rate, there may not be any useful information available
to throw away and the loss may not have to be taken into account
during system design.
 
T

Tim Wescott

Jan 1, 1970
0
John said:
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.



Pretty good. My only quibble is the various statements about what
Nyquist said and didn't say.

http://en.wikipedia.org/wiki/Nyquist-Shannon_sampling_theorem#Historical_background


"Exactly how, when, or why Nyquist had his name attached to the
sampling theorem remains obscure. The first known use of the term
Nyquist sampling theorem is in a 1965 book[4]. It had been called the
Shannon Sampling Theorem as early as 1954[5], but also just the
sampling theorem by several other books in the early 1950s."

It was actually Shannon (among others) that did the sampling theorem;
Nyquist made an observation. Your bibliography doesn't cite either of
them. It's probably correct to use "Nyquist rate" but not "Nyquist
theorem."

John
Thanks John, that's a good point. I'll probably leave the titles intact
because the paper is a reaction to all the posts that have the words
"Nyquist says" followed by something _wrong_ -- but I should put in a
disclaimer, or something.

I'm going to go off and do some web searching; in the mean time do you
have any URLs that point to the seminal papers?

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
 
T

Tim Wescott

Jan 1, 1970
0
Jim said:
"2.1 Aliasing

By ignoring anything that goes on between samples the sampling process
throws away information about the original signal. This information
loss must be taken into account during system design."

This seems like something of an oversimplification. If the orginal
signal is naturally or otherwise bandwidth-limited to well below
2x the sample rate, there may not be any useful information available
to throw away and the loss may not have to be taken into account
during system design.

I can squirm out of that objection:

If you have the continuous-time signal, then you _know_ there's nothing
of note above Fs/2. If you don't have the continuous-time signal, then
you _can't_ know there's nothing of note above Fs/2, unless the sampled
signal train comes with a Certificate of Limited Bandwidth.

Later on in the article I talk about signals that are, indeed,
sufficiently bandlimited by their nature, and the fact that you probably
don't want to do any explicit anti-alias filtering in such a case.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
 
J

John Larkin

Jan 1, 1970
0
John said:
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.



Pretty good. My only quibble is the various statements about what
Nyquist said and didn't say.

http://en.wikipedia.org/wiki/Nyquist-Shannon_sampling_theorem#Historical_background


"Exactly how, when, or why Nyquist had his name attached to the
sampling theorem remains obscure. The first known use of the term
Nyquist sampling theorem is in a 1965 book[4]. It had been called the
Shannon Sampling Theorem as early as 1954[5], but also just the
sampling theorem by several other books in the early 1950s."

It was actually Shannon (among others) that did the sampling theorem;
Nyquist made an observation. Your bibliography doesn't cite either of
them. It's probably correct to use "Nyquist rate" but not "Nyquist
theorem."

John
Thanks John, that's a good point. I'll probably leave the titles intact
because the paper is a reaction to all the posts that have the words
"Nyquist says" followed by something _wrong_ -- but I should put in a
disclaimer, or something.

I'm going to go off and do some web searching; in the mean time do you
have any URLs that point to the seminal papers?

The wiki article has links to Nyquist's 1928 paper and to Shannon's
1949 paper. Few EEs that I've met have ever read either of them, and
have absorbed "the Nyquist theorem" mostly by hearsay, which I guess
is your point.

John
 
J

John Larkin

Jan 1, 1970
0
I like the wording "line in the sand". Isn't that how Archimedes started
studying his circles? They didn't need any white board with the smelly
marker pens.

Don't you go knocking whiteboards. A big board with a nice fresh set
of markers will multiply my IQ by about 1.3 or so. Sand doesn't work
nearly as well.

Oh well, back to the drawing board, literally.

John
 
J

Jonathan Kirwan

Jan 1, 1970
0
<snip>
The wiki article has links to Nyquist's 1928 paper and to Shannon's
1949 paper. Few EEs that I've met have ever read either of them, and
have absorbed "the Nyquist theorem" mostly by hearsay, which I guess
is your point.

I've read Shannon's paper, thorougly. At least, these: The Bell
System Technical Journal, Vol. 27, pp. 379–423, 623–656, July,
October, 1948. Note that this is NOT 1949. But it was, in fact, what
made me understand Boltzmann much better than before and allowed me to
better access the underlying meaning of macro concepts such as
temperature and entropy (which have no micro-meaning.)

Hamming, Shannon, and Golay all worked together in the same place, if
I recall, around that time. Marcel J. E. Golay, 1949, and a little
(short) paper called, "Notes on Digital Coding" which came out in
1949. (I think he was pushed into it by Hamming and Shannon.)

Jon
 
J

John Herman

Jan 1, 1970
0
In the first sentence of your papere, you say that if the signal is band
limited to fo or less, then a sample frequency of 2fo or more is adequate to
contain completely all the information necessary to recreate the signal. My
understanding from water cooler conservation is that the signal bandwidth must
be strictly less than fo for a sample frequency of 2fo and that the signal
must be infinite in extent to allow perfect reconstruction.

IMHO. I could be wrong. I have never seen a formal proof just some
handwaving.
 
M

Major Misunderstanding

Jan 1, 1970
0
Tim Wescott said:
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html

Exccept it was Shannon that said it...
 
J

Jonathan Kirwan

Jan 1, 1970
0
I've read Shannon's paper, thorougly. At least, these: The Bell
System Technical Journal, Vol. 27, pp. 379–423, 623–656, July,
October, 1948. Note that this is NOT 1949. But it was, in fact, what
made me understand Boltzmann much better than before and allowed me to
better access the underlying meaning of macro concepts such as
temperature and entropy (which have no micro-meaning.)

Hamming, Shannon, and Golay all worked together in the same place, if
I recall, around that time. Marcel J. E. Golay, 1949, and a little
(short) paper called, "Notes on Digital Coding" which came out in
1949. (I think he was pushed into it by Hamming and Shannon.)

Some more info and a correction.

First, the correction.

Golay didn't actually with Shannon at Bell labs. His first paper, the
one I mentioned called "Notes on Digital Coding" was actually
published in the Correspondence section of Proc. I.R.E., 37, 657
(1949) was written while he was at the Signal Corps Engineering
Laboratories in Fort Monmoth, N.J.

Now, the additional info.

Golay's 1949 paper is supplemented by two more papers he wrote:
"Binary Coding" I.R.E. Trans. Inform. Theory, PGIT-4, 23-28 (1954) --
written also from Fort Monmoth, NJ; and "Notes on the Penny-Weighing
Problem, Lossless Symbol Coding with Nonprimes, etc.," I.R.E Trans.
Inform. Theory, IT-4, 103-109 (1958) -- written from Philco
Corporation when in Philadelphia, Pa.

Twelve of Shannon's papers (1948 through 1967) are conveniently
collected in the anthology, "Key Papers in the Development of
Information Theory," edited by David Slepian (IEEE Press, 1974.)

Shannon referenced Golay's 1949 paper in the book "The Mathematical
Theory of Communication" (written with Warren Weaver, Univ. Illinois
Press, 1949.) This book contains a slightly rewritten version of
Shannon's first 1948 papers together with a popular-level paper by
Weaver.

Shannon describes the Hamming-7 code in his 1948 papers in section 17,
attributed to Hamming there, but since there is no reference to a
specific paper by Hamming I suspect this reference must have been via
personal communication with Hamming. (Golay also refers to the
Hamming-7 code in Shannon's first paper.)

The first paper by Hamming is "Error Detecting and Error Correcting
Codes" Bell System Tech. J., 29, 147-160 (1950.) Note this is
actually _after_ Shannon's reference to Hamming's code. The
anthology, "Algebraic Coding Theory: History and Development," edited
by Ian F. Blake (Dowden, Hutchinson & Ross, 1973) includes this paper.

Blake says in his introduction to the first 9 papers in his anthology:

"The first nontrivial example of an error-correcting code appears,
appropriately enough, in the classical paper of Shannon in 1948. This
code would today be called the (7,4) Hamming code, containing 16 = 2^4
codewords of length 7, and its construction was credited to Hamming by
Shannon. Golay gives a construction that generalizes this code over
GF (p), p a prime number, of length (p^n -1)/(p - 1) for some positive
integer n. Hamming also obtained the same generalization of his
example of codes of length (2^n - 1) over GF(2) and investigates their
structure and decoding in some depth. The codes of both Golay and
Hamming are now designated as Hamming codes. The interest of Golay
was in perfect codes, which have also been called lossless, or
close-packed, codes. Since he mentions the binary repetition codes
and gives explicit constructions for his remarkable (23,11) binary and
(11,6) ternary codes, it is not stretching a point to say that in the
first paper written specifically on error-correcting codes, a paper
that occupied, in its entirety, only half a journal page, Golay found
essentially all the linear perfect codes which are known today. ...
The multiple error-correcting perfect codes of Golay, now called Golay
codes, have inspired enough papers to fill a separate volume."

By the way, the Hamming-7 code can be used to generate the E7 root
lattice, which corresponds to the E7 Lie algebra. And the Hamming-8
code similarly generates the E8 root lattice, corresponding to the E8
Lie algebra. Heterotic superstring theory has an E8 x E8 symmetry
which is needed for anomaly cancellation. It is nifty that the very
first error-correcting codes of Golay and Hamming play such a profound
role in modern superstring theory.

An interesting supplemental work is from J. H. Conway and N. J. A.
Sloane, "Sphere Packings, Lattices and Groups" from Springer-Verlag,
1988. But probably the best ever book on algebraic coding theory is:
"The Theory of Error-correcting Codes" by F. J. MacWilliams and N. J.
A. Sloane, North-Holland Publishing Co., 1977.

Jon
 
J

jasen

Jan 1, 1970
0
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

No expert, but figure 14 could possibly be improved by explicitly showing
the rising edge of the pulse, maybe start the time axis at -0.5 or start
the pulse at +0.5

The "contact us" link appears to be broken too.
 
X

xray

Jan 1, 1970
0
Some more info and a correction.

You guys are good.

I doubt I'll ever read all that myself, but I do appreciate the work and
motivation it took to take the time to care and find it all.

I'll only say that I hope all of you all keep your enthuiasm to dig,
find the best answer, and share.

It's why I hang out here, read, and snipe occasionally.

As Martha said, even back before she became a capitalistic criminal,
"It's a good thing."
 
R

Rune Allnor

Jan 1, 1970
0
Tim Wescott skrev:
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

Tim,

I really like your style of writing. Accessible, and yet with no hint
at the "for dummies" craze.

I hadn't really thought of the Signal-to-Aliased-Energy ratio as a
metric for signal quality before. It will be a lot easier to discuss
the
quality of sampled systems with that as a tool. Very good
presentation of the both the concept and the effect. I whish
it was me who thought of that sequence of figures 6-9...
oh well.

A couple of very subjective comments:

- There is something about the typography of the page that annoys me.
I can't really see if there is an open line between paragraphs inside a

section; if there is, you may want to make it clearer

- The footnote indicators are WAY too small to be useful, or even seen.


- I can't see why you need the maths paragraph ("To understand
aliasing...", eqs. (2),(3)) at all in section 2.1. I think that what
you
say there is covered by the plain-text parts of the section. Lay-men
might find that maths paragraph scary, throwing them off your article,
while technicians know it already.

- Similar comments apply for eq. (4). I can't see that it is strictly
necessary, the explanation is in the text anyway.

- Eqs. (5)-(7) seem to be necessary, but are the only big equations
in the article. Hmm... that makes me wonder...

As far as I can tell, you are 99.9% at the point where this is an
article a non-engineer layman can read and understand. If you
find a way to get rid of equations 1-7, maybe even eq. 8, you
ought to be there.

Impressive work!

Rune
 
Tim said:
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html

Very good. I would change this phrase:
"This will allow you to use a more open anti-aliasing filter."
By "more open", I gather you mean a larger transition ratio, which
lowers the Q of the resonators.

Note that the Bessel filter will ring at higher orders. I don't have my
copy of Zverev handy, but I think the Bessel rings at 4th order and
higher. The Gaussian filter doesn't ring at any order. The key is to
look at the impulse response of the filter. If it ever goes negative,
then filter will ring.

You might want to go into the inverse sinc response requirements in the
recontruction filter.
 
P

Paul Keinanen

Jan 1, 1970
0
As far as I can tell, you are 99.9% at the point where this is an
article a non-engineer layman can read and understand. If you
find a way to get rid of equations 1-7, maybe even eq. 8, you
ought to be there.

Since understanding the formula _notation_ is not essential for
understanding most of the rest of the text, I would suggest moving the
formulas into separate boxes and moving these boxes out of the direct
text flow e.g. into a box in the right margin of the paper.
Impressive work!

Definitively !

The problem with quite a few text dealing with sampling is that they
are written by mathematicians for the mathematicians. However, these
days, most people using various DSP algorithms are programmers, not
mathematicians, thus the terse mathematical notation can be hard to
understand to them, especially without too much numerical analysis
background.

Instead of using the terse mathematical notation, it might be more
productive for most readers to publish e.g. a Fortran/Algol/C
algorithm.

Paul
 
M

Mike Monett

Jan 1, 1970
0
Note that the Bessel filter will ring at higher orders. I don't have my
copy of Zverev handy, but I think the Bessel rings at 4th order and
higher.

Are you sure about that? Here is a 9th order 1 MHz Bessel for LTspice.
(Save as a CKT file)

* UTS Mike Monett
* Converted From Micro Cap Source file to LTspice
*
C1 0 1 248.3PF
C2 0 2 1200.0PF
C3 0 3 2007.3PF
C4 0 4 2749.9PF
C5 0 Vout 7209.4PF
L1 1 2 1.8UH
L2 2 3 4.1UH
L3 3 4 5.9UH
L4 4 Vout 8.6UH
R1 Vin 1 50
R5 0 Vout 50
V1 Vin 0 DC 0 PULSE (0 1 0 0 0 2.5e-006 5e-006)
..TRAN 1e-008 10u 0 1n UIC
..PRINT TRAN V(VOUT) V(VIN)
..PLOT TRAN V(VOUT) V(VIN)
..PROBE
..END
;$SpiceType=SPICE3

It doesn't ring.

Regards,

Mike Monett

Antiviral, Antibacterial Silver Solution:
http://silversol.freewebpage.org/index.htm
SPICE Analysis of Crystal Oscillators:
http://silversol.freewebpage.org/spice/xtal/clapp.htm
Noise-Rejecting Wideband Sampler:
http://www3.sympatico.ca/add.automation/sampler/intro.htm
 
Top