New approach to dead Boom / SB3 (Classic) / Transporter

[JoeMuc2009]

Hi all,

just wanted to let you know that I had some interesting experiences recently. It started with a shock when once again I had done SMD capacitor replacement on multiple SB3s and the dropout rate was extreme. Five separate devices, all would boot but crash and reboot once they were commanded to start playback. Three of them "died" somewhere between the moment I detached the CPU board to do the capacitor replacement and the moment I put it back together to test. Three! They would do what every dead SB3 does, show a very dim TOSLINK, no connectivity, no display. And that despite the fact I hardly ever touched the CPU board at all.
I still can't explain how it got to this three times in a row (even five times in a row earlier which had me so depressed I would almost quit). I wore a grounding wrist strap all the time. I discharged the capacitors before replugging the CPU board. Used current limiting for the first startup to ensure that a short won't blow anything up. Used an IR camera to look for hotspots during powerup. So that was a bit awkward. The devices were sent to me for repair and shortly after the repair they would fail and be much worse than they initially were. One of them had run a firmware update, then restarted and was normal for a short period of time before it failed. Which brought me to an idea.
Just out of curiosity I extracted the Flash EEPROM from one of them and put it in my reader. Compared to a known-working SB3 there were a lot of differences, at least in the first blocks and, expectedly, where the configuration is held. But I would not assume that the bootloader or whatever is read first from the Flash is very different between identical devices. So I attempted to flash the working image to the EEPROM that was suspected corrupt, with erase first and eventual verification of course to ensure that it isn't the chip itself that is at fault. Then soldered the chip back in and, what do you know, two out of three SB3s were recovered! A Boom PCB is under repair currently, I'll try the same thing there as the hardware arrangement around the CPU is similar to that of the SB3. It looks like what I used to call "CPU death" actually isn't the CPU but the Flash memory for some reason. I don't know why it happens. It should only be written to during configuration and during firmware updates, but something during the repair seems to cause a partial corruption. It's a pity that the EEPROM needs to be desoldered and put back in place as this can only be done once or twice before the board gets damaged. But it's way better than attempting to reflow the CPU which I never succeeded at, and most of the time it might not even be the component at fault. The EEPROM is not easy to handle thanks to its 0.5mm (or so) pin pitch but way easier than the BGA stuff under the CPU.
I have too little experience yet to document this or proclaim it as one of the first measures to fix, and due to the complexity of the operation it should rather be considered a last stand, but still. There is a lot of new hope for the stack of failed SB3s and Booms I have around, and I'll let you know how I fare with it.

Cheers,
Joe

[philippe_44]

Hi all,

just wanted to let you know that I had some interesting experiences recently. It started with a shock when once again I had done SMD capacitor replacement on multiple SB3s and the dropout rate was extreme. Five separate devices, all would boot but crash and reboot once they were commanded to start playback. Three of them "died" somewhere between the moment I detached the CPU board to do the capacitor replacement and the moment I put it back together to test. Three! They would do what every dead SB3 does, show a very dim TOSLINK, no connectivity, no display. And that despite the fact I hardly ever touched the CPU board at all.
I still can't explain how it got to this three times in a row (even five times in a row earlier which had me so depressed I would almost quit). I wore a grounding wrist strap all the time. I discharged the capacitors before replugging the CPU board. Used current limiting for the first startup to ensure that a short won't blow anything up. Used an IR camera to look for hotspots during powerup. So that was a bit awkward. The devices were sent to me for repair and shortly after the repair they would fail and be much worse than they initially were. One of them had run a firmware update, then restarted and was normal for a short period of time before it failed. Which brought me to an idea.
Just out of curiosity I extracted the Flash EEPROM from one of them and put it in my reader. Compared to a known-working SB3 there were a lot of differences, at least in the first blocks and, expectedly, where the configuration is held. But I would not assume that the bootloader or whatever is read first from the Flash is very different between identical devices. So I attempted to flash the working image to the EEPROM that was suspected corrupt, with erase first and eventual verification of course to ensure that it isn't the chip itself that is at fault. Then soldered the chip back in and, what do you know, two out of three SB3s were recovered! A Boom PCB is under repair currently, I'll try the same thing there as the hardware arrangement around the CPU is similar to that of the SB3. It looks like what I used to call "CPU death" actually isn't the CPU but the Flash memory for some reason. I don't know why it happens. It should only be written to during configuration and during firmware updates, but something during the repair seems to cause a partial corruption. It's a pity that the EEPROM needs to be desoldered and put back in place as this can only be done once or twice before the board gets damaged. But it's way better than attempting to reflow the CPU which I never succeeded at, and most of the time it might not even be the component at fault. The EEPROM is not easy to handle thanks to its 0.5mm (or so) pin pitch but way easier than the BGA stuff under the CPU.
I have too little experience yet to document this or proclaim it as one of the first measures to fix, and due to the complexity of the operation it should rather be considered a last stand, but still. There is a lot of new hope for the stack of failed SB3s and Booms I have around, and I'll let you know how I fare with it.

Cheers,
Joe

That's excellent news! I wrongly thought that you already compared flash content in the past. I don't have my programmer here but I'll try as soon as I can get it

[alfista]

Curious, what's the model number of the device?

[JoeMuc2009]

Curious, what's the model number of the device?

You mean the programmer I'm using? That's a MiniPro TL866 (CS), a rather cheap but really capable device, along with a stack of adaptors for TSOP48 chips that I probably bought from AliExpress, and still it was about 40 USD. The pins of the chip need to be squeaky clean in order for it to work but it's easy to find out if a pin does not have good contact. The chip ID can not be retrieved in that case. I had ideal results in flooding the row of pins with tacky flux, then wiping over the bottom of each pin from the inside out and frequently cleaning all solder from the soldering iron tip. That leaves them shiny and consistent.
Here's a photo of the setup:



Cheers,
Joe

[alfista]

Sorry for being vague, shouldn't have formulated a question before my first cup of coffee. I was thinking of the model of the memory device.
But from your answer I can deduce roughly what kind of Flash it is. Flash of that generation usually have pretty much no data retention problems, but heat can accelerate the process so maybe using a hotplate for soldering could potentially be an issue.
Never had a problem with data loss in normal circumstances, some 15 years ago I came across a few memories where single bits had been flipped, but that was on outdoor equipment where we suspected lightning could have been involved.

[JoeMuc2009]

Sorry for being vague, shouldn't have formulated a question before my first cup of coffee. I was thinking of the model of the memory device.
But from your answer I can deduce roughly what kind of Flash it is. Flash of that generation usually have pretty much no data retention problems, but heat can accelerate the process so maybe using a hotplate for soldering could potentially be an issue.
Never had a problem with data loss in normal circumstances, some 15 years ago I came across a few memories where single bits had been flipped, but that was on outdoor equipment where we suspected lightning could have been involved.

Ah, all right. So what Logitech used here, as far as I found on my bench, is either

Spansion S29AL016D90TFI02

or

AMD AM29LV160DB (-90EC)

Both are 16 megabit CMOS 3V 48-pin TSOP and probably interchangeable. Infineon has another model that might also be compatible but I have never checked, so you are on your own trying, the model is Cypress S29AL016J.

What I think might be the reason is not the memory chip itself or its exposure to outside influence during the repair but rather a malfunction somewhere in the OS. I mean the Squeezeboxes are capable of downloading and flashing their EEPROM themselves, and store configuration data dynamically (e.g. the Wi-Fi SSID, the WPA password, the LMS server's host name if an LMS is being used, and some local config like display-related stuff). The part that is dedicated to writing to the Flash EEPROM for these purposes might be kicking in at some unexpected moment and go rogue. Just a tiny fraction of that would be enough to brick the device.
Which also means this issue might come back at any time. What is interesting though is that a device bricking itself is happening only after years of flawless operation. I would not suggest that the Squeezeboxes were timebombed though. Logitech does not have anything better to offer nowadays so it wouldn't serve them to have the previous generation of devices kill itself. Could be just a glitch somewhere in the firmware that was overlooked.

Another theory is that the Flash is written too frequently so the flash cells get weaker. Unlike SSDs with SMART capabilities, these chips don't monitor themselves and there is no way of knowing how reliable the memory cells are. Also, there is no load balancing to ensure that all cells are used so they all age at the same rate. There may be test equipment to find out detail but I guess nobody has something at hand that can evaluate the quality of a Flash chip, and not destroy it any further along the way.
In case of an aged chip I would expect that reflashing has issues, for instance, verifying a freshly written image should result in some errors if cells don't accept the new data as they should. But maybe the cells recover (at least for a time) when they are erased and then completely reflashed. It's too early now to say that reflashing the original EEPROM is the cure forever. If the chip is aged, the device might just be on the edge of dying again soon. I will keep watching this. At least this might explain why the devices fail this way only after more than 10 years.
I believe the Squeezeboxes write to Flash whenever something is changed about the basic configuration (which requires to go to the Settings menu or even to the setup menu), and also when there are changes about display brightness, "now playing" info screen, volume etc. Of course these are all assumptions but if I were the developer dedicated to storing dynamic configuration values to Flash memory, I would try to do that as rarely as possible, for instance when the device changes mode from on to standby. In that moment, I would collect all config data that is relevant for permanent storage, compare it to the values being in the config storage already, and only writing the difference. But who knows if the Squeezebox firmware is actually that smart. Unfortunately, the address where the config is held is always the same so, assuming there is no wear balancing, the same Flash cells are written over and over again, which might actually be a problem. Maybe the designers didn't care much about Flash memory wearing at all. If so, how could they have known that these systems get so popular and will be in use for so long?
I remember Tesla was in the news about Flash memory in their cars which ages rapidly because of the permanent camera surveillance and everything being recorded. It seems that they forgot that Flash isn't lasting forever, and over-used it heavily. But that's a different amount of data and bandwidth. Squeezeboxes are doing nothing in comparison. Still, it's a thought.
From that perspective it might be best to use entirely new chips instead of reusing the original ones...

But let's keep discussing here. It's very interesting to include your opinions on this, guys. I'm far from being an expert here.

[alfista]

Too lazy to dig up the datasheets for those memories but I suspect they are rated at 10,000 or even 100,000 erase cycles. I feel it is unlikely wear should be an issue when used in this application.
Since the erase operation is rather slow, sometimes you don't erase and overwrite when using this kind of device to store small data structures such as player settings, instead you invalidate one setting entry and then append a new. This way you normally don't waste an erase cycle each time you change a setting. Just speculating, have no idea if any of this is how it's done in the SB.

And for more speculation. Perhaps some power glitch could be derailing the state machines inside the Flash that perform the low level erase/write operations, leaving stuff half erased/written? Especially since the failures were triggered when playback was about to start. Now, given that this happened after you had replaced a bunch of capacitors that were past their prime the voltages should be cleaner than before, there's a lot to contradict this theory, but maybe it's still worth looking for ripple or spikes on the voltages. Anecdotally, I've seen some weird issues in some consumer electronics when I replaced electrolytic caps of questionable pedigree with the best stuff I could get my hands on, seems like the design was tuned for the low budget components.

[Glenn2]

Anecdotally, I've seen some weird issues in some consumer electronics when I replaced electrolytic caps of questionable pedigree with the best stuff I could get my hands on, seems like the design was tuned for the low budget components.

Absolutely. Some voltage regulator circuits require that you use low ESR or ultra-low ESR capacitors, while some require that you DON'T. Often you need a mixture of types. Many linear regulators will oscillate if you stick an ultra-low ESR capacitor across their outputs. Who knows what Logitech used originally and if they only used one type across the board. Does anybody know the manufacturer and type of the capacitors? And did Logitech change this when they took over.

My (Logitech) SB3 is still hanging on and in daily use. For half of its life it's been plugged into a remote controlled mains socket, so that it's only powered up when I'm actively using it. This should help preserve both the electrolytics and the VFD.

As a side note I did have to replace the caps in the audio section a few years ago when I realised that the analogue outputs didn't work anymore. That's was a bit OCD of me really as I don't even use them, I connect it to an external DAC! The caps between the internal DAC chip and the op-amp were back-to-front but they did agree with the silk screen. The op-amp input is biased to a higher voltage than the DAC output so the positive side of the caps should face the op-amp, which they didn't in my unit. Interestingly one of the caps between the op-amp and the RCA sockets was also open circuit yet those are not the wrong way around. I replaced all 4 signal path capacitors and normal sound was restored. It doesn't fill me with confidence about the state of all the other caps on the board, but any temptation to replace those whilst it is still working is receding fast after hearing these horror stories!

[JoeMuc2009]

Absolutely. Some voltage regulator circuits require that you use low ESR or ultra-low ESR capacitors, while some require that you DON'T. Often you need a mixture of types. Many linear regulators will oscillate if you stick an ultra-low ESR capacitor across their outputs. Who knows what Logitech used originally and if they only used one type across the board. Does anybody know the manufacturer and type of the capacitors? And did Logitech change this when they took over.

My (Logitech) SB3 is still hanging on and in daily use. For half of its life it's been plugged into a remote controlled mains socket, so that it's only powered up when I'm actively using it. This should help preserve both the electrolytics and the VFD.

As a side note I did have to replace the caps in the audio section a few years ago when I realised that the analogue outputs didn't work anymore. That's was a bit OCD of me really as I don't even use them, I connect it to an external DAC! The caps between the internal DAC chip and the op-amp were back-to-front but they did agree with the silk screen. The op-amp input is biased to a higher voltage than the DAC output so the positive side of the caps should face the op-amp, which they didn't in my unit. Interestingly one of the caps between the op-amp and the RCA sockets was also open circuit yet those are not the wrong way around. I replaced all 4 signal path capacitors and normal sound was restored. It doesn't fill me with confidence about the state of all the other caps on the board, but any temptation to replace those whilst it is still working is receding fast after hearing these horror stories!

Logitech used Panasonic, and all Slimdevices SB3s I have seen (which are quite a lot) also used them. Not sure about the series though, I have found FC, FK, FP, FT, and HB, and it seems to vary a bit from lot to lot. Certainly not the worst choice they could have made, still if they are reversed that's a good explanation for their failure. I have not a lot of knowledge around these things, which of the capacitors are reversed in your opinion? I can say for sure that the silk screen has been the same forever, and it could point towards a design fault that the capacitors are reversed in the silk screen as well as in the pick-and-place process.
Even if you are not using your DAC, the Xilinx chip is still talking to it, and if the DAC power supply is unstable thanks to dead capacitors, that will bring the entire system down and have it do a reboot. Hence, no matter what output you are using, the DAC needs to be happy so things are working out.
It certainly helps a lot to power the SB3 only when needed, as you say, for the display as well as the capacitors. It's a bit less flexible but should be a concern, now that we are thinking a lot more about our consumption bills.

[Glenn2]

I managed to dig out an old thread, complete with contributions from me on the last page.

It is C16 and C20 apparently.

https://forums.slimdevices.com/showthread.php?71914-SB3-volume-output-lower-on-left-than-right-channel

[JoeMuc2009]

I managed to dig out an old thread, complete with contributions from me on the last page.

It is C16 and C20 apparently.

https://forums.slimdevices.com/showt...-right-channel

Here is something that confirms this theory, watch how the C16 and C20 places look. Both caps leaked (C16 a lot more than C20) whereas all others at least didn't spill:



So what is the recommendation here? Replace the caps with a bridge, a ceramic capacitor, a tantalum...? Could a bipolar electrolytic cap do this job?

[Glenn2]

Damn I typed a long reply and it disappeared. I would use an electrolytic but just fit the other way around. You can use a non-polar electrolytic if that makes you nervous. I wouldn't use a ceramic or tantalum.

[JoeMuc2009]

Damn I typed a long reply and it disappeared. I would use an electrolytic but just fit the other way around. You can use a non-polar electrolytic if that makes you nervous. I wouldn't use a ceramic or tantalum.

I found your reply in the mail that was forwarded from the subscription system:

These are signal coupling capacitors, the analogue audio passes right through them. I would personally not use ceramic or tantalum for audio coupling. Also tantalums have been known to go short circuit when they fail which could put DC back into the DAC chip and kill it.

The best thing audio-wise is a film capacitor if you can find a 10uF one that fits, but that might be tricky, and may be overkill.
Perhaps just stick with an electrolytic but fit them the other way around. If that makes you nervous use a non-polarised electrolytic.

It might be interesting, once all other caps are done, to power it up without these capacitors and test the voltages on each pad. It would be good to know for certain what the DC levels really are.

Thank you so much, this really helps. I think I'm going to go with a bipolar electrolytic, incidentally I happend to find 10V/16µF in my parts store, so why not. Will check if there is any effect on the audio spectrum but I wouldn't expect a huge difference. Voltage measurements without caps in place will also follow.

Cheers,
Joe

[JoeMuc2009]

Hey all,

just wanted to let you know that the bipolar electrolytics are doing a great job. I have compared to an SB3 with regular capacitors, having both play a 1kHz sine wave in sync, and there is no measurable difference between them.
Unfortunately, I forgot to measure voltages across the capacitor pads before actually putting the capacitors back in. Sorry about that. Will do that with the next repair as the device that was under repair recently is already going back to its owner.

Cheers,
Joe

[philippe_44]

Hey all,

just wanted to let you know that the bipolar electrolytics are doing a great job. I have compared to an SB3 with regular capacitors, having both play a 1kHz sine wave in sync, and there is no measurable difference between them.
Unfortunately, I forgot to measure voltages across the capacitor pads before actually putting the capacitors back in. Sorry about that. Will do that with the next repair as the device that was under repair recently is already going back to its owner.

Cheers,
Joe

Hi Joe - I missed the original question but why not replacing them by same original electrolytics?