PDA

View Full Version : Rasp Pi Problem cant solve



alastaid
2015-11-26, 04:50
Hi,

New to all of this and very pleased with where I have got to so far. So, have a multi room setup using LMS 7.9, I have 6 clients running on Pi B+ all work fine except one, which pauses the playback roughly for about 10 seconds every two or three minutes (but can vary). I have changed all the components (except pi), reflashed the card, used a different build (both a full Jessie build and a piCore using squeezelite), but it still does it. When it stops playing, it has lost contact with the server, as it doesn't respond to pause commands etc, then eventually picks up again. I turned on logging and got the following:

[11:28:18.437050] _output_frames:146 track start sample rate: 44100 replay_gain: 0
[11:28:18.449100] _output_frames:174 fade start reached
[11:28:18.449250] _output_frames:207 fade complete
[11:28:27.972800] _output_frames:69 skip 31619 of 31619 frames //** player playing ok upto this point
[11:30:56.073461] decode_flush:190 decode flush
[11:30:56.073606] output_flush:423 flush output buffer
[11:30:56.108243] codec_open:218 codec open: 'f'
[11:30:56.108463] stream_sock:384 connecting to 192.168.2.11:80
[11:30:56.110163] stream_sock:413 header: GET /stream.mp3?player=b8:27:eb:8c:54:c8 HTTP/1.0


[11:30:56.227881] stream_thread:176 headers: len: 118
HTTP/1.1 200 OK
Server: Logitech Media Server (7.9.0 - 1448035497)
Connection: close
Content-Type: audio/x-flac


[11:30:56.364898] write_cb:116 setting track_start
[11:30:56.518068] _output_frames:146 track start sample rate: 44100 replay_gain: 0 ///**player picks up again
tc@Kitchen:/tmp$


Looking at this, it appears to have lost network connectivity, this is using a wired network, and watching it while it hangs, the link light stays constant and the data light still flashes (have tried different switch ports and cables)

Have I got it right that the pi loses network connectivity, and if so, have I got a faulty pi as I cant think what else to try?

Thanks

Alastair

Man in a van
2015-11-26, 05:04
Hi,

New to all of this and very pleased with where I have got to so far. So, have a multi room setup using LMS 7.9, I have 6 clients running on Pi B+ all work fine except one, which pauses the playback roughly for about 10 seconds every two or three minutes (but can vary). I have changed all the components (except pi), reflashed the card, used a different build (both a full Jessie build and a piCore using squeezelite), but it still does it. When it stops playing, it has lost contact with the server, as it doesn't respond to pause commands etc, then eventually picks up again. I turned on logging and got the following:

[11:28:18.437050] _output_frames:146 track start sample rate: 44100 replay_gain: 0
[11:28:18.449100] _output_frames:174 fade start reached
[11:28:18.449250] _output_frames:207 fade complete
[11:28:27.972800] _output_frames:69 skip 31619 of 31619 frames //** player playing ok upto this point
[11:30:56.073461] decode_flush:190 decode flush
[11:30:56.073606] output_flush:423 flush output buffer
[11:30:56.108243] codec_open:218 codec open: 'f'
[11:30:56.108463] stream_sock:384 connecting to 192.168.2.11:80
[11:30:56.110163] stream_sock:413 header: GET /stream.mp3?player=b8:27:eb:8c:54:c8 HTTP/1.0


[11:30:56.227881] stream_thread:176 headers: len: 118
HTTP/1.1 200 OK
Server: Logitech Media Server (7.9.0 - 1448035497)
Connection: close
Content-Type: audio/x-flac


[11:30:56.364898] write_cb:116 setting track_start
[11:30:56.518068] _output_frames:146 track start sample rate: 44100 replay_gain: 0 ///**player picks up again
tc@Kitchen:/tmp$


Looking at this, it appears to have lost network connectivity, this is using a wired network, and watching it while it hangs, the link light stays constant and the data light still flashes (have tried different switch ports and cables)

Have I got it right that the pi loses network connectivity, and if so, have I got a faulty pi as I cant think what else to try?

Thanks

Alastair

Hi Alastair and welcome.

I can't help with the code as I'm NVQ 3 Code Numpty.

I would suggest that you swap the pi with one that is working correctly , but in a different location. See if the problem follows the pi.

Also check that each pi has its own unique MAC address.

atb

Ronnie

DJanGo
2015-11-26, 05:04
Hi & welcome

change the power source from the one that always fails to another (from another Pi that didnt fails)


Have I got it right that the pi loses network connectivity, and if so, have I got a faulty pi as I cant think what else to try?
Can you ping /ssh to it when it fails playing?

bpa
2015-11-26, 05:10
Look at interface TCP and IP statistics to see if there is a burst of errors/retries or retransmissions.

Faulty connector etc can cause intermittent errors.
Try a USB ethernet (or even wifi) adaptor to eliminate the onboard Ethernet part of the Pi.

alastaid
2015-11-26, 05:19
Hi,

Thanks for the quick responses, so I have swapped with known good power supplies, and no change. It does move around with the specific pi board.

Tried it again, with a continuous ping from another machine, and the player stopped playing, but the ping stayed perfect and consistent in response time during the whole "outage". So this blows the pi board theory and looks like the OS N/W stack is happy, so I am completely stumped now.

Anyone got anything else to try?

Thanks

Alastair

bpa
2015-11-26, 05:26
Hi,

Thanks for the quick responses, so I have swapped with known good power supplies, and no change. It does move around with the specific pi board.

Tried it again, with a continuous ping from another machine, and the player stopped playing, but the ping stayed perfect and consistent in response time during the whole "outage". So this blows the pi board theory and looks like the OS N/W stack is happy, so I am completely stumped now.

Anyone got anything else to try?

Thanks

Alastair

Enable Squeezelite logging (e.g. -d slimproto=info ) initially try functional areas separately so slimproto first to see if actually network related , then output to see if audio out is blocking somehow, then stream.

alastaid
2015-11-26, 09:06
Ok,

Started on the logging front.

slimproto had nothing

stream had the following:

[15:49:15.497148] stream_thread:249 end of stream
[15:51:39.440958] stream_sock:384 connecting to 192.168.2.11:80
[15:51:39.443271] stream_sock:413 header: GET /stream.mp3?player=b8:27:eb:8c:54:c8 HTTP/1.0


[15:51:39.496815] stream_thread:176 headers: len: 116
HTTP/1.1 200 OK
Server: Logitech Media Server (7.9.0 - 1448035497)
Connection: close
Content-Type: audio/mpeg


[15:54:13.886698] stream_sock:384 connecting to 192.168.2.11:80
[15:54:13.888166] stream_sock:413 header: GET /stream.mp3?player=b8:27:eb:8c:54:c8 HTTP/1.0

the dropout was between 15:51:39 & 15:54:13 the 15:54:13 event is the stream picking back up, it drops out for around 10 secs, but there is no event posted when it loses the stream.

and output the following:

/mnt/mmcblk0p2/tce/squeezelite-armv6hf -n Kitchen -o sysdefault:CARD=ALSA -a 80:::0 -s 192.168.2.11 -d output=info -f /tmp/outputinfo
[15:55:39.224704] output_init_alsa:817 init output
[15:55:39.225231] output_init_alsa:846 requested alsa_buffer: 80 alsa_period: 4 format: any mmap: 0
[15:55:39.265996] output_init_common:410 supported rates: 384000 352800 192000 176400 96000 88200 48000 44100 32000 24000 22500 16000 12000 11025 8000
[15:55:39.277752] output_init_alsa:862 memory locked
[15:55:39.278770] output_thread:638 open output device: sysdefault:CARD=ALSA
[15:55:39.278919] alsa_open:355 opening device at: 44100
[15:55:39.286085] alsa_open:406 opened device sysdefault:CARD=ALSA using format: S32_LE sample rate: 44100 mmap: 0
[15:55:39.288298] alsa_open:485 buffer: 80 period: 4 -> buffer size: 3536 period size: 884
[15:55:39.436100] output_flush:423 flush output buffer
[15:55:40.185595] _output_frames:146 track start sample rate: 44100 replay_gain: 0
[15:55:41.004934] output_flush:423 flush output buffer
[15:55:41.445447] _output_frames:146 track start sample rate: 44100 replay_gain: 0
[15:55:51.825429] _output_frames:69 skip 108353 of 108353 frames
tc@Kitchen:/tmp$ cat outputinfo

/mnt/mmcblk0p2/tce/squeezelite-armv6hf -n Kitchen -o sysdefault:CARD=ALSA -a 80:::0 -s 192.168.2.11 -d output=info -f /tmp/outputinfo
[15:55:39.224704] output_init_alsa:817 init output
[15:55:39.225231] output_init_alsa:846 requested alsa_buffer: 80 alsa_period: 4 format: any mmap: 0
[15:55:39.265996] output_init_common:410 supported rates: 384000 352800 192000 176400 96000 88200 48000 44100 32000 24000 22500 16000 12000 11025 8000
[15:55:39.277752] output_init_alsa:862 memory locked
[15:55:39.278770] output_thread:638 open output device: sysdefault:CARD=ALSA
[15:55:39.278919] alsa_open:355 opening device at: 44100
[15:55:39.286085] alsa_open:406 opened device sysdefault:CARD=ALSA using format: S32_LE sample rate: 44100 mmap: 0
[15:55:39.288298] alsa_open:485 buffer: 80 period: 4 -> buffer size: 3536 period size: 884
[15:55:39.436100] output_flush:423 flush output buffer
[15:55:40.185595] _output_frames:146 track start sample rate: 44100 replay_gain: 0
[15:55:41.004934] output_flush:423 flush output buffer
[15:55:41.445447] _output_frames:146 track start sample rate: 44100 replay_gain: 0
[15:55:51.825429] _output_frames:69 skip 108353 of 108353 frames
[15:58:26.013875] output_flush:423 flush output buffer
[15:58:26.375596] _output_frames:146 track start sample rate: 44100 replay_gain: 0
tc@Kitchen:/tmp$

you can see the dropout was between 15:55:51 & 15:58:26

It looks like it just looses contact with the server stream? Yet you can ping it continuously with no error during the "dropout"

Mnyb
2015-11-26, 09:17
Do you ping from the sever ?

alastaid
2015-11-26, 09:19
No it was another machine, but will try from the server

bpa
2015-11-26, 09:22
It looks like it just looses contact with the server stream? Yet you can ping it continuously with no error during the "dropout"

If it loses contatc then another GET would be issued - is there a second GET ?
Otherwise it is LMS that has stopped sending ?
Possibilties are many - one possibility is that TCP ack from Pi has been lost and so LMS has sent all the data it can - LMS will wait until TCP timeout before reseting link and then sending again wit another GET. Netstats on LMS will show if timeout are happening.

I think wireshark on thePi ( I presume it is available) would be the best bet - change network adaptor (use USB or Wifi) on PI would be another test.

alastaid
2015-11-26, 09:28
just tried pinging it from the server, and it pings fine, starting to think I should just send the pi back, as to test my sanity, I just swapped the sd card into another pi with the same network cable and power supply (just using the headphone socket to eliminate the DAC), and it worked perfectly. It has to be the pi surely?

bpa
2015-11-26, 09:35
just tried pinging it from the server, and it pings fine, starting to think I should just send the pi back, as to test my sanity, I just swapped the sd card into another pi with the same network cable and power supply (just using the headphone socket to eliminate the DAC), and it worked perfectly. It has to be the pi surely?

I've seen a network interface (not on a pi) which had problem with large packets (CRC errors) yet received small "ping type" packets OK (bad clock h/w got out of sync with data if network frame was large) - that is why wireshark would be definitive but using another network adaptor is an alternative way of isolating the problem. If this is a new Pi - send it back. Network stats will indicate these sort of problems

Jeff07971
2015-11-26, 09:39
I've seen a network interface (not on a pi) which had problem with large packets (CRC errors) yet received small "ping type" packets OK (bad clock h/w got out of sync with data if network frame was large) - that is why wireshark would be definitive but using another network adaptor is an alternative way of isolating the problem. If this is a new Pi - send it back. Network stats will indicate these sort of problems

Also have you tried a different port on your switch ?

I have had single ports go down in a similar way

DJanGo
2015-11-26, 13:19
I just swapped the sd card into another pi with the same network cable and power supply (just using the headphone socket to eliminate the DAC), and it worked perfectly. It has to be the pi surely?

May i didnt understand what you've done:confused:
Did you change two RPi vice versa and used the same power supply?

That didnt proove nothing when you didnt use the same DAC cause the USB Dac needs Power.
Try the same again with the same DAC thats "not" working on the "Other" Pi.

d6jg
2015-11-26, 13:23
Are you using a USB DAC on the non working Pi? I can't see the DAC type mentioned before.
The Pi Ethernet port & USB do share the same bus and it is known to cause problems with certain hardware.

alastaid
2015-11-27, 01:50
Sorry for not being clear, to try and eliminate as much as I could, I have two identical Pi's, the working one and non working one, no DAC just using the headphone for testing, same SD card. So try it in the working pi all fine, then swap the same sd card to the non working pi, use the same power supply, same Ethernet cable, same switch port as before and it drops out every time.

So this eliminates the SD card, OS, switch port, Ethernet cable, no DAC used, only leaves the pi?

Whilst I have been thinking about it, when I started building my rig, I do remember one of the blog posts saying I should upgrade the firmware of the pi, and to be honest I cannot remember if I did them all, so I will go through this proves on the non working one, see if I missed one, only thing I could think of? Hopefully get round to it on Saturday.

Greg Erskine
2015-11-27, 02:01
Whilst I have been thinking about it, when I started building my rig, I do remember one of the blog posts saying I should upgrade the firmware of the pi, and to be honest I cannot remember if I did them all, so I will go through this proves on the non working one, see if I missed one, only thing I could think of?

hi alastaid,

There is no firmware on the Raspberry Pi itself. All the software is on the SD card.

regards
Greg

d6jg
2015-11-27, 03:25
There is firmware on a Pi - it is the equivalent to a PCs BIOS - why its called firmware and not BIOS I am not sure.

Greg Erskine
2015-11-27, 03:57
There is firmware on a Pi - it is the equivalent to a PCs BIOS - why its called firmware and not BIOS I am not sure.

The point I am trying to make is there is no firmware/boot software on the actual Raspberry Pi PCB itself, it is all loaded from the SD card when booting.

If I understand correctly, the OP is testing with 2 RPIs with one SD card. On one Pi it works and one doesn't. He thinks the firmware is upgraded on one but not the other. I don't think this is possible using the same SD card.

When I was a boy, firmware was software permanently loaded onto hardware, ie ROMs or EPROMs. Now its all RAM with battery or capacitor backup and writeable. The terms and meanings tend to change and get fuzzy over time.

regards
Greg

Julf
2015-11-27, 06:22
There is firmware on a Pi - it is the equivalent to a PCs BIOS - why its called firmware and not BIOS I am not sure.

Because "firmware" is the generic term for semi-permanently installed control software held in non-volatile memory. One specific kind of firmware is a boot loader - the firmware that allows the device to boot from a secondary device. On a PC, that boot loader is called "BIOS".

The Raspberry pi has a simple first stage bootloader in ROM - that bootloader reads the second stage bootloader from the SD card.

Julf
2015-11-27, 06:26
The point I am trying to make is there is no firmware/boot software on the actual Raspberry Pi PCB itself, it is all loaded from the SD card when booting.

No, there is a first stage boot loader in ROM. It is needed to load software from the SD card.


When I was a boy, firmware was software permanently loaded onto hardware, ie ROMs or EPROMs. Now its all RAM with battery or capacitor backup and writeable.

These days non-volatile RAM is mostly flash memory, not (battery- or capacitor-backed) SRAM.

Greg Erskine
2015-11-27, 13:12
No, there is a first stage boot loader in ROM. It is needed to load software from the SD card.

Yes, there must be some minimal code to get things started. How do you update the first stage boot loader?


These days non-volatile RAM is mostly flash memory, not (battery- or capacitor-backed) SRAM.

Good. I am going to remove all those batteries and super caps from my computers now! ;)

regards
Greg

Julf
2015-11-27, 13:37
Yes, there must be some minimal code to get things started. How do you update the first stage boot loader?

Why would you need to do that? All it does is load the second stage boot program from SD.


Good. I am going to remove all those batteries and super caps from my computers now! ;)

"Proper" computers are a different thing (I see the raspberry pi as an embedded device, not a computer). A desktop PC has a battery-powered clock, and usually set-up parameters that need to be maintained, even if the BIOS is in flash. Embedded computers, such as the RPI, don't use batteries or super caps.

DJanGo
2015-11-27, 13:49
So this eliminates the SD card, OS, switch port, Ethernet cable, no DAC used, only leaves the pi?

Depends on how do you use squeezelite..

If you dont use a fake mac address for the squeezeliteplayer some strange playersettings in your lms may also involved.

Do you use a fake Mac Address for squeezelite?
If no - stop your logitechmediaserver and edit in the server.prefs from lms the whole "chapter"
(start something like _client:CA:FE:04:DJ:GO:) -<the mac address from that Pi
and restart the server / software not the Hardware.


If yes - get another Rpi and bring that back

Greg Erskine
2015-11-27, 14:08
Why would you need to do that? All it does is load the second stage boot program from SD..

I don't. I wanted you to confirm it can't be upgraded and ALL "user accessible" software is on the SD card? Yes?


"Proper" computers are a different thing (I see the raspberry pi as an embedded device, not a computer). A desktop PC has a battery-powered clock, and usually set-up parameters that need to be maintained, even if the BIOS is in flash. Embedded computers, such as the RPI, don't use batteries or super caps.

Yes you are right, I agree. I stopped playing with PCs 15 years ago and have forgotten a few things. I just see them as a box now. ;)

regards
Greg

alastaid
2015-11-28, 03:56
OK, so I maybe wrong with the term "firmware", but I ran the rpi-update as per here: http://www.devils-heaven.com/raspberry-pi-firmware-update/

which did appear, at least, to be doing something on the board as opposed to the card, but I could be wrong, I cant even find out where I read that you need to do this when using Pi as a squeezelite client. Anyway, will run it on the non working pi to see if it makes a difference.

Jeff07971
2015-11-28, 04:17
OK, so I maybe wrong with the term "firmware", but I ran the rpi-update as per here: http://www.devils-heaven.com/raspberry-pi-firmware-update/

which did appear, at least, to be doing something on the board as opposed to the card, but I could be wrong, I cant even find out where I read that you need to do this when using Pi as a squeezelite client. Anyway, will run it on the non working pi to see if it makes a difference.

I don't think this is doing what you think it does
This appears to be updating firmware files on the Raspian (Debian) system.
These firmwares are pushed to certain devices which need them when connected.

Actually just looked into this a bit further, they are firmware "in software" most of them are firmware drivers for the kernel ie modules

Taking one at random : https://github.com/raspberrypi/firmware/commit/d56bf185f0f9e1cc2d9090970a2db1f6ebe76bb2

alastaid
2015-11-28, 05:07
Django, looks like you are on the money!!!! Many thanks, I am not using fake macs, removed the section for the suspect pi in the server.prefs and bingo it works, very pleased.

I assume at sometime I (or one of my kids!) has been in and started playing with the settings on the player and caused the issue.

Anyway, its fixed, I can move on to more fettling :-), thanks again to all on the forum. Also it does make sense as I couldn't see how it could possibly be the pi, but I was stumped as to where else to look.

Julf
2015-11-28, 05:07
OK, so I maybe wrong with the term "firmware", but I ran the rpi-update as per here: http://www.devils-heaven.com/raspberry-pi-firmware-update/

which did appear, at least, to be doing something on the board as opposed to the card, but I could be wrong, I cant even find out where I read that you need to do this when using Pi as a squeezelite client. Anyway, will run it on the non working pi to see if it makes a difference.

Yes, the first stage bootloader firmware is update-able, but usually there is little need to do it unless you need to boot from exotic hardware or directly over the net.

Julf
2015-11-28, 05:11
I don't think this is doing what you think it does
This appears to be updating firmware files on the Raspian (Debian) system.
These firmwares are pushed to certain devices which need them when connected.

Actually just looked into this a bit further, they are firmware "in software" most of them are firmware drivers for the kernel ie modules

Taking one at random : https://github.com/raspberrypi/firmware/commit/d56bf185f0f9e1cc2d9090970a2db1f6ebe76bb2

Take a look at the package "raspberrypi-bootloader". The boot loader needs drivers for the basic hardware and the devices you want to boot from (SD devices, wlan etc.), and both the boot loader itself and the drivers can be updated.

Julf
2015-11-28, 05:14
I don't. I wanted you to confirm it can't be upgraded and ALL "user accessible" software is on the SD card? Yes?

Not quite. See my previous reply about "raspberrypi-bootloader".


Yes you are right, I agree. I stopped playing with PCs 15 years ago and have forgotten a few things. I just see them as a box now. ;)

Likewise, but then I got involved in "embedded devices" - something that used to mean an 8-bit microcontroller, but these days means something that runs rings around a 15-year-old PC both in terms of performance and complexity :)

utgg
2015-11-28, 11:14
Yes, the first stage bootloader firmware is update-able, but usually there is little need to do it unless you need to boot from exotic hardware or directly over the net.

No it's not. It is in embedded rom.

Julf
2015-11-28, 12:31
No it's not. It is in embedded rom.

Yes and no. The RPi designers did quite some cost-cutting measures. The first-stage bootloader, in RPi terminology, is indeed in ROM, and can't be updated, but the only thing that that code can do is load the second stage (bootcode.bin) from the boot partition on the SD card - so a RPi can not boot from the network or USB stick. Not only that, but neither the first nor the second stage actually run on the main CPU - they run on the GPU (graphics processor). The second stage loads the third-stage bootloader (start.elf) from the SD card.

So the second stage bootloader on a RPi corresponds to a traditional basic bootloader, and *can* be updated. The third stage corresponds to a PC BIOS, and has actual hardware settings (config.txt), and can also be updated.

As the fourth stage, the actual kernel (kernel.img) gets loaded, and is the first code to actually run on the main CPU.

Anyway, yes, all the user-upgradeable parts live on the SD card, and the stuff on the main board can not be updated.