Flash retention in uC at higher temps, experience?

[Joerg]

Hello All,

After some problems a client saw I was treated to my own dose of what is
likely flash loss: The uC in our mailbox door has become erratic. I
installed it about three months ago and half of the day it receives a
good pelting from the sun. First it began not recognizing some keys,
then it started doing weird stuff like lock cycling. Things it wasn't
meant to ever do. Batteries, contacts and such look ok, reset didn't
help, so that's not it.

TI has an app note about the topic:
http://focus.ti.com/lit/an/slaa392/slaa392.pdf

Figure 1 looks scary above the 80C range. Later they presented another
test with a different bake cycle which makes things look better but who
knows.

What is you experience with respect to flash errors on uC that are
exposed to elevated temperatures as most outdoors applications are?

 

[Jonathan Kirwan]

Hello All,

After some problems a client saw I was treated to my own dose of what is
likely flash loss: The uC in our mailbox door has become erratic. I
installed it about three months ago and half of the day it receives a
good pelting from the sun. First it began not recognizing some keys,
then it started doing weird stuff like lock cycling. Things it wasn't
meant to ever do. Batteries, contacts and such look ok, reset didn't
help, so that's not it.

TI has an app note about the topic:
http://focus.ti.com/lit/an/slaa392/slaa392.pdf

Figure 1 looks scary above the 80C range. Later they presented another
test with a different bake cycle which makes things look better but who
knows.

What is you experience with respect to flash errors on uC that are
exposed to elevated temperatures as most outdoors applications are?

I only have some small experience here with the MSP430. It seems to
operate at 140C at 3V and 3.3V for the several-hour long tests I've
done. But some bad experiences in storing data into the flash at that
temp and even at 3.3V and higher. But I didn't need the darn thing to
survive all that long, either.

I haven't read, for understanding, the data sheet you mentioned. I
just downloaded it, though, and thanks for pointing it up. I think it
wasn't around when I looked a few years back and I'm glad that you
pointed it up for me.

Your obvious solution is to move north about a thousand miles.

Jon

 

[John Tserkezis]

What is you experience with respect to flash errors on uC that are
exposed to elevated temperatures as most outdoors applications are?

Our boxes never see direct sun exposure on the semis, but they're encased in
an aluminium box, then in a black plastic 'pelican' waterproof case. Here in
Australia, it gets quite warmish over summer, and we routinely see the insides
at 50C (122F) or more (as far as the measurement of the internal temp sensor
goes anyway).

Our flash chips are quite ancient, and although they don't hold any critical
information, it's critical enough that you're going to notice if it chucks a
wobbly.

Although we get some with corrupt flash data, it's relatively rare.

 

[Joerg]

I only have some small experience here with the MSP430. It seems to
operate at 140C at 3V and 3.3V for the several-hour long tests I've
done. But some bad experiences in storing data into the flash at that
temp and even at 3.3V and higher. But I didn't need the darn thing to
survive all that long, either.

Wow, problems within hours at 140C? Not cool :-(

I haven't read, for understanding, the data sheet you mentioned. I
just downloaded it, though, and thanks for pointing it up. I think it
wasn't around when I looked a few years back and I'm glad that you
pointed it up for me.

Your obvious solution is to move north about a thousand miles.

My wife would absolutely not do that.

 

[Joerg]

Our boxes never see direct sun exposure on the semis, but they're
encased in an aluminium box, then in a black plastic 'pelican'
waterproof case. Here in Australia, it gets quite warmish over summer,
and we routinely see the insides at 50C (122F) or more (as far as the
measurement of the internal temp sensor goes anyway).

50C is easy, we even get that as an ambient temperature around here. But
in summer stuff inside of enclosures can easily reach 80C or more. Worst
case for 10h/day all summer.

Our flash chips are quite ancient, and although they don't hold any
critical information, it's critical enough that you're going to notice
if it chucks a wobbly.

Chucks a wobbly, cool. Learned a new expression!

Although we get some with corrupt flash data, it's relatively rare.

Depends on what "relatively rare" means ;-)

 

[Joerg]

One of my clients makes products that regularly go above 80C, and that
has never been an issue to my knowledge (all the service guys I used to
hang with have either left or been promoted, so I don't have that
immediate knowledge one gets by being service's life line). Certainly
the to-do if it were recognized would have bubbled down to me.

Well, in my case it kind of has bubbled down to me now ;-)

OTOH, all the parts are very carefully selected to work over the
industrial temperature range; if your mailbox thingie is designed with
commercial temp range parts all bets are off (and it may be something
else that's happening, too).

It's not just this mailbox but also some MSP430 apps at a client. They
predate my involvement there and we are pretty much stuck with that for
a while. We are seeing a distinct pattern where those inside smaller
boxes fail more often than those in larger and more airy enclosures.
This stuff is used in the south where summers are quite toasty.

 

[Joerg]

Have you tried cooking them on purpose? Even if you don't have a "real"
environmental chamber, you can do a lot with an insulated box and a heat
gun (including catching the lab on fire (ask me how I know!), but that
usually entertains the technicians).

That I haven't tried yet. A friend of mine (chemical engineer) did
something similar. After an extended hospital stay he casually mentioned
that his lab was now a black hole and that the door was gone.

I'd toss some in an oven for an extended high-temperature test, to see
where they failed.

I'd also check the heat rise against ambient in both small and large
enclosures.

That's what I suggested to the client, to place some USB temp loggers in
there.

If they get any solar load, I'd throw up my hands and call a mechanical
engineer who's good with thermodynamics, but that's just because I
(usually) know where my competence ends.

Some of the flash parts that I have seen used to success have been from
TI, but they aren't TMS430s, and they _are_ industrial temperature range
parts.

I wish the MSP430 was available in automotive but it ain't. At least not
according to the TI rep.

One could add some code so it re-flashes itself once in a while, that's
another option here. However, the device cannot lose power during that
brief period or it'll be really toast. So that would require large
enough electrolytics and some pre-regulator voltage monitoring. The
latter would require a relayout, bigger ECO and all that. Not a VP of
Engineering's favorite path, usually.

 

[Joerg]

I have found strong R.F. and EMF's in close proximity have caused the
flash to get corrupted in PIC's. I'm sure it effects others.

What do you mean by strong? VHF radio close by? Cell phone? If yes, do
you remember how close?

 

[Jamie]

Have you tried cooking them on purpose? Even if you don't have a "real"
environmental chamber, you can do a lot with an insulated box and a heat
gun (including catching the lab on fire (ask me how I know!), but that
usually entertains the technicians).

I'd toss some in an oven for an extended high-temperature test, to see
where they failed.

I'd also check the heat rise against ambient in both small and large
enclosures.

If they get any solar load, I'd throw up my hands and call a mechanical
engineer who's good with thermodynamics, but that's just because I
(usually) know where my competence ends.

Some of the flash parts that I have seen used to success have been from
TI, but they aren't TMS430s, and they _are_ industrial temperature range
parts.

I have found strong R.F. and EMF's in close proximity have caused the
flash to get corrupted in PIC's. I'm sure it effects others.



http://webpages.charter.net/jamie_5"

 

[Jamie]

What do you mean by strong? VHF radio close by? Cell phone? If yes, do
you remember how close?

100 W VHF transmitter ~ 6 feet away.

Another problem I have heard of but never experienced and
that is police microwave radar. I guess for some uC's there
are circuit paths that match the wave length and can cause
unwanted effects in the memory.

 

[James Arthur]

That I haven't tried yet. A friend of mine (chemical engineer) did
something similar. After an extended hospital stay he casually mentioned
that his lab was now a black hole and that the door was gone.



That's what I suggested to the client, to place some USB temp loggers in
there.



I wish the MSP430 was available in automotive but it ain't. At least not
according to the TI rep.

One could add some code so it re-flashes itself once in a while, that's
another option here.

I was going to suggest that. But, you ask, "How often?"

A few wild troubleshooting ideas:

One thing you might do is vary the device's Vdd while reading it, thus
changing the read threshold and alerting you to marginal cells before
they fail.

With a heat gun and a device reader you might actually document the
bleed-down of the device's memory cells vs: temperature & derive a
definitive lifetime projection.

Another idea: can you control the programming timing? If so, you could
weakly program a test pattern + CRC in the part. The part could then
test the area itself, detecting impending failures.


Cheers,
James Arthur

 

[Jonathan Kirwan]

Wow, problems within hours at 140C? Not cool :-(

No. No problems, at all. Just that I didn't run them for more than
about 5 hours at a time. Same one ran for weeks, though, at periodic
elevated temperatures. I was just collecting data from a rotating hot
surface and wanted to just stick the whole contraption there while it
stored a few bits of data into RAM. The battery was the problem.

My wife would absolutely not do that.

Oregon is absolutely beautiful! I've got pileated woodpeckers, 4
kinds of squirrels including a flying squirrel (nocturnal), peafowl,
chickens, guinea hens, turkeys and so on -- tall 60-80 year old firs,
two kinds of ferns, rhododendrons that bloom in succession around the
place, and it looks like a lush rain-forest national forest when you
walk the paths on the property. Lots of acres, 5000 sq ft home, 1/4
mile driveway to the house, view of the mountains, 5 minutes to a
hospital and 20 minutes to the PDX international airport, a 17 mile
well-maintained walking and horse trail that goes from 1/2 mile away
from my home to the Willamette River in Portland, and it cost me $330k
in 2002. Prices are still low, too. Next door has been on the block
for 2 years, is a million dollar home (tax appraisal price) with about
4500 sq ft and 5 acres, and is being offered at $599k now. I'm told
they'd accept under $500k. Neighbors are wonderful, too.

3' of fantastic soils, 45" of rain a year nice and evenly distributed
all year 'round in a constant drizzle, and everything grows where you
throw the seed, no digging needed. What could be better?

Jon

 

[Paul Keinanen]

One could add some code so it re-flashes itself once in a while, that's
another option here. However, the device cannot lose power during that
brief period or it'll be really toast. So that would require large
enough electrolytics and some pre-regulator voltage monitoring. The
latter would require a relayout, bigger ECO and all that. Not a VP of
Engineering's favorite path, usually.

In a high radiation environment usually some ECC bits are stored in
each sector and with frequent sector reads you can detect if there are
flipped bits, correct the errors using the ECC and write back the
sector.

In this way, you know when to do the writeback and you can minimize
the (limited) number of reflashing for a specific sector on the flash.

I assume this would also work with the data retention time problem at
elevated temperatures.

For a program written in assembly, it should not be a problem to
allocate a number of bytes for the ECC bits at the end of sector and
use a branch instruction to skip them, but this might be an issue, if
an HLL is used.

Paul

 

[Frank Buss]

Jonathan Kirwan wrote:

[...]

3' of fantastic soils, 45" of rain a year nice and evenly distributed
all year 'round in a constant drizzle, and everything grows where you
throw the seed, no digging needed. What could be better?

Sounds great if you want to become a farmer I'm living near Cologne
centre, but a quiet back road near the Rhein. Just a rented flat and not
acres of grass and bushes around it, but supermarkets, stores, pubs,
restaurants, theaters, cinemas, parks etc., all within walking distance.

 

[Frank Buss]

One could add some code so it re-flashes itself once in a while, that's
another option here. However, the device cannot lose power during that
brief period or it'll be really toast. So that would require large
enough electrolytics and some pre-regulator voltage monitoring. The
latter would require a relayout, bigger ECO and all that. Not a VP of
Engineering's favorite path, usually.

If the flash is big enough, you could copy the program to an unused memory
area and back again the next cycle. Then the only critial moment is writing
the switchover. But should be no problem, if you can monitor the supply
voltage and if you can guarantee, that it remains over the flash limit for
the time needed to write the switchover bytes.

What do you think of MRAM for critial applications? It's a bit expensive,
but they claim data retention of minimum 20 years and unlimited number of
writes. It is even used in a satellite:

http://www.eetimes.com/showArticle.jhtml?articleID=206900226
http://www.freescale.com/files/microcontrollers/doc/data_sheet/MR2A16A.pdf

 

[Frank Buss]

For a program written in assembly, it should not be a problem to
allocate a number of bytes for the ECC bits at the end of sector and
use a branch instruction to skip them, but this might be an issue, if
an HLL is used.

You can collect all ECC bits at the end of the flash.

 

[John Tserkezis]

50C is easy, we even get that as an ambient temperature around here. But
in summer stuff inside of enclosures can easily reach 80C or more. Worst
case for 10h/day all summer.

For us, it was a non-issue above 40C, because we didn't rate the measurement
electronics to be within specification above that point. We actually could
have gone higher, but the built-in LCD 1x16 display wasn't rated above 40. It
was a nice round number so we left it at that.

Chucks a wobbly, cool. Learned a new expression!

It's an oldish Australian term. I don't know how wide spread it is, but
everyone here who I've mentioned it to knows what it implies.

Depends on what "relatively rare" means ;-)

Of the dozen or so broken boxes I used to see every week, we'd get one with
trashed flash data every month or so.
I have no idea how many boxes were in service in the outside world. I'm
guessing many hundreds, but I really have no idea as I was only involved in
the service side, not sales or other areas that would give me an idea of what
was already sold.

 

[TheM]

What do you think of MRAM for critial applications? It's a bit expensive,
but they claim data retention of minimum 20 years and unlimited number of
writes. It is even used in a satellite:

I thought it was limited number of writes. But for certain packages (serial) it practically means
unlimited as the protocol limits number of reads sufficiently.

M

 

[TheM]

100 W VHF transmitter ~ 6 feet away.

I would guess the part went haywire and started overwriting the flash
itself by some randomly executed code. Possibly the fuses were not set or
were even ignored.

Mark

 

[Frank Buss]

I thought it was limited number of writes. But for certain packages (serial) it practically means
unlimited as the protocol limits number of reads sufficiently.

The Freescale chip has fast parallel access (35ns), like a slow SDRAM, and
this page says "Unlimited Write and Read Cycles":

http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=4MBIT&nodeId=015424