Maker Pro
Maker Pro

What Nyquist Didn't Say

F

Fred Bloggs

Jan 1, 1970
0
Tim said:
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.

So I decided to write a short little article to make it all clear.

It's a little longer than 'short', and it took me way longer than I
thought it would, but at least it's done and hopefully it's clear.

You can see it at
http://www.wescottdesign.com/articles/Sampling/sampling.html.

If you're new to this stuff, I hope it helps. If you're an expert and
you have the time, please feel free to read it and send me comments or
post them here.

I just had to stop reading about half way through. The writing style and
organization is structurally incoherent and comes across as desultory
gibberish with intolerably sloppy terminology and perspective. On the
upside, it should be somewhat beneficial to the low caliber types who
are confused by the subject matter.
 
Tim,

I like the article and totally agree that there's a lot of smoke and
mirrors
and misuse of sampling theorem ideas. I would definitely point
newcomers to your description.


Please don't take the following as flaming.
But I just can't resist throwing in a couple of my favourite pet
peeves :-(

I echo the sentiments of an earlier comment about

fs >= 2fmax

versus

fs > 2fmax

If you look at e.g. Oppenheim & Wilsky, Signals & Systems pp 519 they
clearly state in terms of proper greater than. However, if you look at
Shannon&Weaver "The Mathematical Theory of Communication" P 86 "Band
limited ensembles of functions" it's less clear (verbage rather than
math): they say (more or less) "let f(t) contain no frequencies over W.
Then you can sample at 2W". This would imply the geater-than or equal,
which I suspect is incorrect.

In fact, in your article you point out that you cannot sample a sine
wave at exactly 2*f unless you know the phase, which clearly
contradicts
the ">=" interpretation. (Actually you need the phase and the
amplitude).

I'm not entirely sure of the history but I suspect that the
band-limiting can actually be traced much father back to Fourier
analysis in mid 1800's.

In the same vein, I've thought that one could do a neat write-up of how
to reconstruct from the sampling of sin(2*pi*f*s) at a frequency
approaching (but not equal to) 2f. If you describe how
the almost-aliased-to-DC low frequency samples (regardless of phase)
get
turned back into a full amplitude signal through the magic of sinx/x
reconstruction, it would be very cool. The samples would (where phase
approached one side) be almost perfect (+1,-1,+1,-1) but as the phase
drifted the other way, the envelope would attenuate (+0.0001, -0.0001,
etc). Through the magic of summing an infinite series, you get your
full-size plain old sine wave back. But that might be more math than
the scope of this article.

I also would propose a change to sec 3.3 about repetitive signals.
There is really no difference between this example and 3.4 - band
limited signals! The only reason that the powerline sampling in sec 3.3
works is that you're talking about a bandlimited "modulation" signal
that's modulating a 60Hz carrier! So it's not really the case that
"sometimes you can sample slower than the Nyquist Thm would
indicate"...
this is a little misleading. I'd introduce it more like "some classes
of signals, which appear to be fast, are actually low bandwidth,
they're just a slowly
modulated carrier, so you get a low Fs". (I'm not being precise but I
hope
you get my drift).

Cheers,

- Kenn
 
H

Helmut Sennewald

Jan 1, 1970
0
Mike Monett said:
Are you sure about that? Here is a 9th order 1 MHz Bessel for LTspice.
(Save as a CKT file)

* UTS Mike Monett
* Converted From Micro Cap Source file to LTspice
*
C1 0 1 248.3PF
C2 0 2 1200.0PF
C3 0 3 2007.3PF
C4 0 4 2749.9PF
C5 0 Vout 7209.4PF
L1 1 2 1.8UH
L2 2 3 4.1UH
L3 3 4 5.9UH
L4 4 Vout 8.6UH
R1 Vin 1 50
R5 0 Vout 50
V1 Vin 0 DC 0 PULSE (0 1 0 0 0 2.5e-006 5e-006)
.TRAN 1e-008 10u 0 1n UIC
.PRINT TRAN V(VOUT) V(VIN)
.PLOT TRAN V(VOUT) V(VIN)
.PROBE
.END
;$SpiceType=SPICE3

It doesn't ring.

Regards,

Mike Monett


Hello Mike,

It seems you have overlooked that LTspice assumes a certain rise and fall
time
if you specify t_rise=0 or t_fall=0.

V1 Vin 0 DC 0 PULSE (0 1 0 0 0 2.5e-006 5e-006)

LTspice assumes in this case trise = fall = 10%*Twidth = 250ns

This is the reason why you see no overshoot.

Now we use fast edges.

V1 Vin 0 DC 0 PULSE (0 1 0 1n 1n 2.5e-006 5e-006)

You will see about 1.6x% overshoot. By the way, I am not sure
how precise the choosen component values are to get the ideal Bessel
filter response. Bessel filters have indeed overshoot in the step response.

Best regards,
Helmut
 
M

Mike Monett

Jan 1, 1970
0
Hello Mike,
It seems you have overlooked that LTspice assumes a certain rise
and fall time if you specify t_rise=0 or t_fall=0.
V1 Vin 0 DC 0 PULSE (0 1 0 0 0 2.5e-006 5e-006)
LTspice assumes in this case trise = fall = 10%*Twidth = 250ns
This is the reason why you see no overshoot.
Now we use fast edges.
V1 Vin 0 DC 0 PULSE (0 1 0 1n 1n 2.5e-006 5e-006)
You will see about 1.6x% overshoot. By the way, I am not sure how
precise the choosen component values are to get the ideal Bessel
filter response. Bessel filters have indeed overshoot in the step
response.
Best regards,
Helmut

Hi Helmut,

Thanks for taking a look at this, and for pointing out that LTspice
changes the rise and fall times if you do not specify the values. I
am still learning how LTspice works, and these unexpected variations
from other SPICE programs can be major pitfalls.

In this case, the amount of overshoot is very small, and I'm not
sure if the 1.6% is not caused by the way the component values are
rounded.

I use this filter program for general filter design. The component
values were calculated for the theoretical values and the inductors
were rounded to 2 significant digits. This reflects the difficulty
in obtaining precision inductors, particularly at VHF and UHF.

The capacitors were rounded to tenths of a pf. This is because the
program is used mostly for work at VHF and UHF, where the caps are
much smaller. It may well be that using the correct theoretical
values for the components may reduce or eliminate the small amount
of overshoot you measured.

However, there is still the practical matter of obtaining inductors
and capacitors with the needed precision, and any practical filter
will have imperfect components. I have found the Bessel and
Equiripple to be much more tolerant of errors in component values
than other filter types, and still give good performance.

If we are concerned about such small values of overshoot, we need to
repeat this using the correct theoretical values for the components.
Even so, an overshoot of 1.6% is not in the same category as the
overshoot from Butterworth or other non-linear phase filters, which
may have ten percent or more, depending on the sharpness of the
cutoff. In practise, the Bessel and Equiripple are considered linear
phase filters, and have negligible overshoot.

For example, a Butterworth or other nonlinear group delay filter may
cause excessive timing errors when used to filter digital data. This
can be minimized by changing to a Bessel or Equiripple filter.

So practically speaking, the Bessel and Equiripple distinguish
themselves from other filters by the constant group delay through
the filter bandpass, and the low or non-existant overshoot.

Regards,

Mike Monett

Antiviral, Antibacterial Silver Solution:
http://silversol.freewebpage.org/index.htm
SPICE Analysis of Crystal Oscillators:
http://silversol.freewebpage.org/spice/xtal/clapp.htm
Noise-Rejecting Wideband Sampler:
http://www3.sympatico.ca/add.automation/sampler/intro.htm
 
D

David Ashley

Jan 1, 1970
0
Tim said:
I've seen a lot of posts over the last year or so that indicate a lack
of understanding of the implications of the Nyquist theory, and just
where the Nyquist rate fits into the design of sampled systems.
...

It is just me or did anyone else get a strong sense of deja-vu?

-Dave
 
T

Tim Wescott

Jan 1, 1970
0
Fred said:
I just had to stop reading about half way through. The writing style and
organization is structurally incoherent and comes across as desultory
gibberish with intolerably sloppy terminology and perspective. On the
upside, it should be somewhat beneficial to the low caliber types who
are confused by the subject matter.
Finally! Someone who sees my writing the way I do! I think we're in
the minority, though.

I don't think of the intended audience as "low caliber", though -- just
people who've had a different educational background, and whose
back-brains don't light up with equations when they see a fountain, or a
near miss on the freeway.

I have to say that this paper rather turned into a monster while I was
writing it. I thought it was going to be between 2000 and 3000 words,
with almost no math and very little real work. Instead it's about 5500
words, and I've got about a man-week into all those pretty charts and
graphs (I should publish an appendix with "the making of..." along with
all the math underneath).

As a reaction to this I haven't done my usual stage of letting it rest
and getting back to it -- I was afraid I'd never do the "getting back to
it" step. Instead I've put it outside without giving it time to get
it's coat and boots on. If I can figure out how to tighten it up I
certainly will -- assuming that I don't run away screaming at the
thought of doing even _more_ work on it.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
 
T

Tim Wescott

Jan 1, 1970
0
David said:
It is just me or did anyone else get a strong sense of deja-vu?

-Dave
Yes, I did write essentially that at the head of a posting soliciting
suggestions for the article -- and you all helped.

(note to self -- add an acknowledgments section)

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google? See http://cfaj.freeshell.org/google/

"Applied Control Theory for Embedded Systems" came out in April.
See details at http://www.wescottdesign.com/actfes/actfes.html
 
B

Ban

Jan 1, 1970
0
Mike said:
Are you sure about that? Here is a 9th order 1 MHz Bessel for LTspice.
(Save as a CKT file)

Bessel filters do have a small overshoot, which decreases at higher orders.
2nd .43%
4th .84%
6th .64%
8th .34%
10th .06%
note the *very low* value for higher orders. In fact the frequency response
gets hardly better then.
The Gauss filter has indeed zero ringing, but a much larger transition band.
 
Mike said:
Hi Helmut,

Thanks for taking a look at this, and for pointing out that LTspice
changes the rise and fall times if you do not specify the values. I
am still learning how LTspice works, and these unexpected variations
from other SPICE programs can be major pitfalls.

In this case, the amount of overshoot is very small, and I'm not
sure if the 1.6% is not caused by the way the component values are
rounded.

I use this filter program for general filter design. The component
values were calculated for the theoretical values and the inductors
were rounded to 2 significant digits. This reflects the difficulty
in obtaining precision inductors, particularly at VHF and UHF.

The capacitors were rounded to tenths of a pf. This is because the
program is used mostly for work at VHF and UHF, where the caps are
much smaller. It may well be that using the correct theoretical
values for the components may reduce or eliminate the small amount
of overshoot you measured.

However, there is still the practical matter of obtaining inductors
and capacitors with the needed precision, and any practical filter
will have imperfect components. I have found the Bessel and
Equiripple to be much more tolerant of errors in component values
than other filter types, and still give good performance.

If we are concerned about such small values of overshoot, we need to
repeat this using the correct theoretical values for the components.
Even so, an overshoot of 1.6% is not in the same category as the
overshoot from Butterworth or other non-linear phase filters, which
may have ten percent or more, depending on the sharpness of the
cutoff. In practise, the Bessel and Equiripple are considered linear
phase filters, and have negligible overshoot.

For example, a Butterworth or other nonlinear group delay filter may
cause excessive timing errors when used to filter digital data. This
can be minimized by changing to a Bessel or Equiripple filter.

So practically speaking, the Bessel and Equiripple distinguish
themselves from other filters by the constant group delay through
the filter bandpass, and the low or non-existant overshoot.

Regards,

Mike Monett

Antiviral, Antibacterial Silver Solution:
http://silversol.freewebpage.org/index.htm
SPICE Analysis of Crystal Oscillators:
http://silversol.freewebpage.org/spice/xtal/clapp.htm
Noise-Rejecting Wideband Sampler:
http://www3.sympatico.ca/add.automation/sampler/intro.htm

You really need to look at theory versus what spice predicts. The input
is an ideal step. As you do the convolution, the negative region of the
impulse response subtracts from the result, making the signal drop in
value, then the postive regions make the result increase in value,
hence ringing.. The Gaussian impulse response is always positive, hence
the convolution output can't decrease as time increases.

Linear phase alone is not enough to stop ringing.

The discrete time situation is easier to understand, especially if you
consider a finite impulse response filter. The response of a FIR filter
to a step input is the the sum of the tap coefficients from one to N.
That is, the first output is the first tap. The second output is the
sum of the first two taps, etc. If no tap is negative, then the output
always rises, hence no ringing.

No circuits were simulated in writing this post. ;-)
 
M

Mike Monett

Jan 1, 1970
0
Ban said:
Bessel filters do have a small overshoot, which decreases at higher
orders. 2nd .43%
4th .84%
6th .64%
8th .34%
10th .06%
note the *very low* value for higher orders. In fact the frequency
response gets hardly better then.
The Gauss filter has indeed zero ringing, but a much larger transition
band.

Ban,

Thanks for the clarification. In most references, the Bessel is considered
to have low, negligible, or no overshoot, especially when compared to
Butterworth, Chebyshev and other types of filters. Your numbers confirm
this.

In practise, it is difficult to obtain the exact component values needed
for the theoretical performance. Not only are the values non-standard, but
it may be difficult to get the inductor "Q" values used in most
calculations. So we can assume there will be some deviation from the
theoretical performance, and the overshoot will probably increase slightly.
However, it is still low enough to be difficult to measure, and the terms
"low", "neglible" or "no overshoot" are quite descriptive.

Regards,

Mike Monett

Antiviral, Antibacterial Silver Solution:
http://silversol.freewebpage.org/index.htm
SPICE Analysis of Crystal Oscillators:
http://silversol.freewebpage.org/spice/xtal/clapp.htm
Noise-Rejecting Wideband Sampler:
http://www3.sympatico.ca/add.automation/sampler/intro.htm
 
M

Mike Monett

Jan 1, 1970
0
You really need to look at theory versus what spice predicts. The
input is an ideal step. As you do the convolution, the negative
region of the impulse response subtracts from the result, making
the signal drop in value, then the postive regions make the result
increase in value, hence ringing.. The Gaussian impulse response
is always positive, hence the convolution output can't decrease as
time increases.
Linear phase alone is not enough to stop ringing.
The discrete time situation is easier to understand, especially if
you consider a finite impulse response filter. The response of a
FIR filter to a step input is the the sum of the tap coefficients
from one to N.
That is, the first output is the first tap. The second output is
the sum of the first two taps, etc. If no tap is negative, then
the output always rises, hence no ringing.

Please see Ban's post and my reply. Most references describe the
Bessel as having low, negligible, or no overshoot. Ban's numbers
show a theoretical overshoot of less than 1%, decreasing with higher
order. In any practical LCR filter, the Bessel will have such low
overshoot as to be difficult or impossible to measure. So it doesn't
make sense to split hairs between <1% and zero overshoot.

DSP is a quite different story, and you can realize filters that are
impossible in real life. But it would be difficult to implement DSP
at higher frequencies.

Regards,

Mike Monett

Antiviral, Antibacterial Silver Solution:
http://silversol.freewebpage.org/index.htm
SPICE Analysis of Crystal Oscillators:
http://silversol.freewebpage.org/spice/xtal/clapp.htm
Noise-Rejecting Wideband Sampler:
http://www3.sympatico.ca/add.automation/sampler/intro.htm
 
B

Ban

Jan 1, 1970
0
You really need to look at theory versus what spice predicts. The
input is an ideal step. As you do the convolution, the negative
region of the impulse response subtracts from the result, making the
signal drop in value, then the postive regions make the result
increase in value, hence ringing.. The Gaussian impulse response is
always positive, hence the convolution output can't decrease as time
increases.

Linear phase alone is not enough to stop ringing.

The discrete time situation is easier to understand, especially if you
consider a finite impulse response filter. The response of a FIR
filter to a step input is the the sum of the tap coefficients from
one to N. That is, the first output is the first tap. The second
output is the sum of the first two taps, etc. If no tap is negative,
then the output always rises, hence no ringing.

No circuits were simulated in writing this post. ;-)

There are differences between analog and digital filters. Digital means an
approximation of the desired characteristic in the passband, but the poles
and zeros are modified to compensate for the modulation effects. aliasing is
always happening, but it can be reduced below the noise floor with the
analog input filter.
Linear phase has a very undesirable side effect, it rings *before and after*
the step, supposed to be more audible.

And Audio is a very forgiving métier, because the ears themselves function
as reconstruction filters, suppressing all those high frequency artifacts..
If you want to display the signal with a digital scope or ECG, you better
start with 5 to 10 times the upper frequency rolloff, look at the specs
there, nobody even considers Nyquist adequate, exept programmers having no
idea of reality.
 
J

John E. Hadstate

Jan 1, 1970
0
Tim Wescott said:
I've seen a lot of posts over the last year or so that
indicate a lack of understanding of the implications of
the Nyquist theory, and just where the Nyquist rate fits
into the design of sampled systems.

Nyquist has a lot to say about how far you *can* go and
little to say about how far you *should* go. Those of us
who are forced to push the Nyquist limits may have a better
appreciation of this.

For example, compare the frequency response of a 5-pole IIR
Butterworth low-pass filter designed to roll off at 1 KHz.
with one designed to roll off at 10 KHz., both filters
having been designed around a sampling frequency of 50 KHz.
Things start to get nasty above Fs/10 and the harder you
push Nyquist, the worse it gets.

Nyquist says you can put a sharp +/- 500 KHz analog filter
around a 21.4 MHz IF and sample it at 2.14 MHz. What
Nyquist doesn't say is what that does to your
signal-to-noise ratio. TANSTAAFL!

Nyquist is often used by slick, hand-waving charlatans to
over-sell their capabilities without having a clue about the
real trade-offs involved. (And, I'll bet Nyquist didn't say
*that*, either.)
 
V

Vladimir Vassilevsky

Jan 1, 1970
0
Ban wrote:

And Audio is a very forgiving métier, because the ears themselves function
as reconstruction filters, suppressing all those high frequency artifacts..

And audio is a very unforgiving matter, because the distortions of the
audio amp are quickly growing towards the high frequencies. You won't
hear the high frq artifacts as they are, but you will very well hear the
result of the nonlinear distortion of those.

Vladimir Vassilevsky

DSP and Mixed Signal Design Consultant

http://www.abvolt.com
 
R

Robert Latest

Jan 1, 1970
0
["Followup-To:" header set to sci.electronics.design.]
On Fri, 29 Sep 2006 13:55:32 -0700,
in Msg. said:
Don't you go knocking whiteboards. A big board with a nice fresh set
of markers will multiply my IQ by about 1.3 or so.

But only with the smelly markers. The water-based stuff doesn't do it
for me.

robert
 
R

Robert Latest

Jan 1, 1970
0
["Followup-To:" header set to sci.electronics.design.]
On Fri, 29 Sep 2006 22:35:46 GMT,
in Msg. said:
In the first sentence of your papere, you say that if the signal is band
limited to fo or less, then a sample frequency of 2fo or more is adequate to
contain completely all the information necessary to recreate the signal. My
understanding from water cooler conservation is that the signal bandwidth must
be strictly less than fo for a sample frequency of 2fo and that the signal
must be infinite in extent to allow perfect reconstruction.

I see it that way as well. A pure sine signal of exactly f_0, sampled at
exactly 2f_0, might come out as all-zero, or as a pulse train with
alternating polarity and an amplitude of anything between 0 and V_p.
There is not enough information to reconstruct the signal in this case.

However, if you sample the same signal at 2f_0+f_e (f_e being very
small, think epsilon), it will come out as a pulse train with
alternating polarity and the amplitude modulated by a beat frequency
f_e. This would seem to be not enough information to reconstruct the
signal, but in fact that's not true: A f_0 sine is the only possible
input signal because the modulated pulse train contains frequency
components greater than f_0 which couldn't have been in the input signal
(which, as a prerequsitite to Nyquist, is brick-walled at f_0).

So, the way I see it is that the sampling frequency must be strictly
greater than the highest frequency in the input.

Thanks, Tim, for a great write-up on the subject!

robert
 
R

Robert Latest

Jan 1, 1970
0
["Followup-To:" header set to sci.electronics.design.]
On Sat, 30 Sep 2006 09:22:43 -0700,
in Msg. said:
I have to say that this paper rather turned into a monster while I was
writing it. I thought it was going to be between 2000 and 3000 words,
with almost no math and very little real work. Instead it's about 5500
words, and I've got about a man-week into all those pretty charts and
graphs (I should publish an appendix with "the making of..." along with
all the math underneath).

As a reaction to this I haven't done my usual stage of letting it rest
and getting back to it -- I was afraid I'd never do the "getting back to
it" step. Instead I've put it outside without giving it time to get
it's coat and boots on. If I can figure out how to tighten it up I
certainly will -- assuming that I don't run away screaming at the
thought of doing even _more_ work on it.

Let me tell you that this article is so high-class from an educational
point of view that it /deserves/ coat and boots. Next time anybody comes
up to me and wants something explained about Nyquist I can just point
them at your page, and I'm sure many others will do likewise.

The only thing I'd want is the whole thing as a pretty, nicely printable
PDF document. But don't listen to me.

robert
 
Top