PDA

View Full Version : Streaming WAV files to LMS over http question



RipNCode
2019-04-26, 10:29
Hi All,

I'm not sure this question is in the right place, but as best as I can tell, this is a general question.

I recently set up a LMS server, and I love it. I use OpenHAB for home automation, and have run into a curious situation that I'm hoping someone here can help with.

OpenHAB as a binding for supporting squeezelite players. It works well for controlling the system and playing anything that's on LMS. However, it also has the option to set a squeezlite player as an audio sink. This is very useful in my situation as I can have LMS stream audio over my entire home and OpenHAB can interrupt the audio and broadcast a message as needed (ie, "someone is at the door") and then resume the stream. Great system. However, when the broadcast is in WAV format, LMS just pauses the stream with no audio output. No problem with MP3, FLAC, or other formats. Since I'd like to have the automation system utilize text-to-speech, and that forces WAV files to be produced, it renders the system unusable. If these files are already in LMS as WAV files, there's no problem. It has something to do with sending the WAV files over http for transcoding.

I have had a Home Assistant automation system user confirms this happens with Home Assistant as well, so it's not just an OpenHAB problem.

In a perfect world, I'd have a text-to-speech system (in this case, Pico TTS) read a file and output the stream in WAV to the appropriate squeezelite player. However, I'm not sure what's happening in the stream that's causing this. I'm hoping someone here can point me in the right direction to find a reasonable solution to this situation. Any help is appreciated.

Paul Webster
2019-04-26, 10:37
Can you provide some more info about how the file/stream is passed to LMS?

RipNCode
2019-04-26, 10:56
Hi Paul,

It's not long before I'm out of my depth here, but in brief, it's via the Eclipse Smarthome Java console, Karaf in OpenHAB, and directly via Python (or I'm led to believe), in Home Assistant (I have no direct experience with Home Assistant). In both cases (very different automation systems), all file types except WAV seem to work. I can verify this directly with sending MP3 files in OpenHAB that work as anticipated. When WAV files are sent, the UI on LMS show a "Notification" and play no audio at all (though after the duration of the wav file sent has expired, whatever audio was playing before resumes). In the same case with an MP3, the "Notification" shows up and audio is output as expected (then whatever was streaming before streams as usual).

The best I can probably offer are some links to the github files that process the audio for OpenHAB. I'm more than happy to find out more to help you, but I'm not even sure what you may need at this point.

FWIW, if I stream the same audio directly to ALSA audio output, it works. The issue seems to be entirely bound to sending WAV to LMS for processing.

Link to Github OpenHAB Audio.java class (https://github.com/eclipse/smarthome/blob/master/bundles/model/org.eclipse.smarthome.model.script/src/org/eclipse/smarthome/model/script/actions/Audio.java)

bpa
2019-04-26, 11:04
However, when the broadcast is in WAV format, LMS just pauses the stream with no audio output. No problem with MP3, FLAC, or other formats.

LMS doesn't recognise the stream and IIRC streamign WAV is not part of standard distribution as there is no "de-facto" standard for streaming WAV so for your stream a new audio type and associated handler may be required.

First, LMS needs info to determine the audio format. This info is usually in the Mime of the stream in the HTTP Content-Type header.
Examples

MP3 audio/mpeg,audio/mp3,audio/mp3s,audio/x-mpeg,audio/mpeg3,audio/mpg
AAC over ADTS audio/aac,audio/aacp
AAC over MPEG4 audio/m4a,audio/x-m4a,audio/mp4

RipNCode
2019-04-26, 11:15
LMS doesn't recognise the stream and IIRC streamign WAV is not part of standard distribution as there is no "de-facto" standard for streaming WAV so for your stream a new audio type and associated handler may be required.

First, LMS needs info to determine the audio format. This info is usually in the Mime of the stream in the HTTP Content-Type header.
Examples

MP3 audio/mpeg,audio/mp3,audio/mp3s,audio/x-mpeg,audio/mpeg3,audio/mpg
AAC over ADTS audio/aac,audio/aacp
AAC over MPEG4 audio/m4a,audio/x-m4a,audio/mp4

Thanks, bpa. That helps a lot. It's probably the core of what's wrong. I assume LMS would probably need an appropriate plug-in that allows it to recognize the correctly defined audio type?

Sorry for the newbie questions, but I'm still getting a handle on this...

RipNCode
2019-04-26, 11:20
I also replied to Paul, but the post didn't show up. I pasted it below for reference.


Hi Paul,

It's not long before I'm out of my depth here, but in brief, it's via the Eclipse Smarthome Java console, Karaf in OpenHAB, and directly via Python (or I'm led to believe), in Home Assistant (I have no direct experience with Home Assistant). In both cases (very different automation systems), all file types except WAV seem to work. I can verify this directly with sending MP3 files in OpenHAB that work as anticipated. When WAV files are sent, the UI on LMS show a "Notification" and play no audio at all (though after the duration of the wav file sent has expired, whatever audio was playing before resumes). In the same case with an MP3, the "Notification" shows up and audio is output as expected (then whatever was streaming before streams as usual).

The best I can probably offer are some links to the github files that process the audio for OpenHAB. I'm more than happy to find out more to help you, but I'm not even sure what you may need at this point.

FWIW, if I stream the same audio directly to ALSA audio output, it works. The issue seems to be entirely bound to sending WAV to LMS for processing.

Link to Github OpenHAB Audio.java class (https://github.com/eclipse/smarthome/blob/master/bundles/model/org.eclipse.smarthome.model.script/src/org/eclipse/smarthome/model/script/actions/Audio.java)

bpa
2019-04-26, 11:21
Thanks, bpa. That helps a lot. It's probably the core of what's wrong. I assume LMS would probably need an appropriate plug-in that allows it to recognize the correctly defined audio type?

Sorry for the newbie questions, but I'm still getting a handle on this...

Not necessarily a plugin as something might already cover it but the stream has to be set up so it is unambiguously a WAV stream. LMS also has to know is it is a "Live" stream (i.e. infinite) or a file (e.g. WAV podcast like) with a defined length (WAV header has a max size of 2Gb so "Live" stream has to break WAV headers). The transport protocol also needs to be defined (e.g. MPEg2 over HTTP, MPEG4 over HTTP, plain http). Side issues of metadata also come into view.

bpa
2019-04-26, 11:35
Since the stream already exists - it is a case of describing the current format and then advice can be given.

If you're not sure how to describe then play it with mplayer with "-v" verbose and copy the output to a post. It will give all the header and mime and other stream format info necessary to play the stream.

Years ago - KEXP used to have a uncompressed stream and there various hacks to play it - all of which are now out of date.

For example - one thread of many
https://forums.slimdevices.com/showthread.php?35028-Connect-to-KEXP-Uncompressed

RipNCode
2019-04-26, 11:52
Since the stream already exists - it is a case of describing the current format and then advice can be given.

If you're not sure how to describe then play it with mplayer with "-v" verbose and copy the output to a post. It will give all the header and mime and other stream format info necessary to play the stream.

Years ago - KEXP used to have a uncompressed stream and there various hacks to play it - all of which are now out of date.

For example - one thread of many
https://forums.slimdevices.com/showthread.php?35028-Connect-to-KEXP-Uncompressed

Thanks, bpa. Below is a sample created with Pico TTS and with output from mplayer. In theory, the format would always be the same (PCM 16-bit signed), but the length would vary as Pico converts various text files to audio.


Playing test.wav.
get_path('sub/') -> '/home/rip/.mplayer/sub/'
[file] File size is 136492 bytes
STREAM: [file] test.wav
STREAM: Description: File
STREAM: Author: Albeu
STREAM: Comment: based on the code from ??? (probably Arpi)
libavformat version 58.20.100 (internal)
Configuration: --enable-gpl --enable-postproc
LAVF_check: WAV / WAVE (Waveform Audio)
Checking for YUV4MPEG2
ASF_check: not ASF guid!
Checking for REAL
Checking for SMJPEG
Searching demuxer type for filename test.wav ext: .wav
Trying demuxer 17 based on filename extension
==> Found audio stream: 0
======= WAVE Format =======
Format Tag: 1 (0x1)
Channels: 1
Samplerate: 16000
avg byte/sec: 32000
Block align: 2
bits/sample: 16
cbSize: 0
================================================== ========================
demux_audio: audio data 0x2C - 0x2152C
Audio only file format detected.
Load subtitles in ./
get_path('sub/') -> '/home/rip/.mplayer/sub/'
================================================== ========================
Opening audio decoder: [pcm] Uncompressed PCM audio decoder
dec_audio: Allocating 2048 + 131072 = 133120 bytes for output buffer.
AUDIO: 16000 Hz, 1 ch, s16le, 256.0 kbit/100.00% (ratio: 32000->32000)
Selected audio codec: [pcm] afm: pcm (Uncompressed PCM)
================================================== ========================
Building audio filter chain for 16000Hz/1ch/s16le -> 0Hz/0ch/??...
[libaf] Adding filter dummy
[dummy] Was reinitialized: 16000Hz/1ch/s16le
[dummy] Was reinitialized: 16000Hz/1ch/s16le
Trying preferred audio driver 'alsa', options '[none]'
alsa-init: requested format: 16000 Hz, 1 channels, 9
alsa-init: using ALSA 1.1.7
alsa-init: setup for 1/2 channel(s)
alsa-init: using device default
alsa-init: opening device in blocking mode
[AO_ALSA] alsa-lib: pcm_dmix.c:1063:(snd_pcm_dmix_open) unable to create IPC semaphore
[AO_ALSA] Playback open error: Permission denied
Failed to initialize audio driver 'alsa'
Trying every known audio driver...
ao2: 16000 Hz 1 chans s16le
audio_setup: using '/dev/dsp' dsp device
audio_setup: using '/dev/mixer' mixer device
audio_setup: using 'pcm' mixer device
[AO OSS] audio_setup: Can't open audio device /dev/dsp: Device or resource busy
alsa-init: requested format: 16000 Hz, 1 channels, 9
alsa-init: using ALSA 1.1.7
alsa-init: setup for 1/2 channel(s)
alsa-init: using device default
alsa-init: opening device in blocking mode
[AO_ALSA] alsa-lib: pcm_dmix.c:1063:(snd_pcm_dmix_open) unable to create IPC semaphore
[AO_ALSA] Playback open error: Permission denied
AO: [pulse] 16000Hz 1ch s16le (2 bytes per sample)
AO: Description: PulseAudio audio output
AO: Author: Lennart Poettering
Building audio filter chain for 16000Hz/1ch/s16le -> 16000Hz/1ch/s16le...
[dummy] Was reinitialized: 16000Hz/1ch/s16le
[dummy] Was reinitialized: 16000Hz/1ch/s16le
Video: no video
Freeing 0 unused video chunks.
Starting playback...
Increasing filtered audio buffer size from 0 to 9024
A: 4.0 (03.9) of 4.0 (04.0) 0.0%
ds_fill_buffer: EOF reached (stream: audio)
ds_fill_buffer: EOF reached (stream: audio)
A: 4.0 (04.0) of 4.0 (04.0) 0.0%
ds_fill_buffer: EOF reached (stream: audio)
A: 4.1 (04.0) of 4.0 (04.0) 0.0%
EOF code: 1

Uninit audio filters...
[libaf] Removing filter dummy
Uninit audio: pcm
vo: x11 uninit called but X11 not initialized..

Exiting... (End of file)

d6jg
2019-04-26, 12:08
Another approach may be to take the wav input into liquidsoap and then stream it via icecast to LMS as an mp3.

RipNCode
2019-04-26, 12:44
Another approach may be to take the wav input into liquidsoap and then stream it via icecast to LMS as an mp3.

I've posted another post with the details, but again, it's not showing up. I'll try again in a few minutes as it may be that my posts are still being moderated. If this goes through without moderation, I'll assume the last post was somehow lost...

Since most of the automation is running on an RPi3+, I'm trying to minimize the overhead. Since I anticipate most of the Pico TTS outputs to be small files, I can script lame to convert it to MP3 and then stream it. Ideally, if it's possible to get LMS to recognize the Pico TTS-generated wav files, it would save some overhead and act as a template to do allow other possible wav files to be streamed. I'm just in over my head right now, so if I get a working example, it's off to the races for lots of automation folks to leverage it with LMS in the future...

RipNCode
2019-04-26, 12:46
double post - sorry. I reposted the mplayer output twice. It's on the prior page.

bpa
2019-04-26, 13:01
I've posted another post with the details, but again, it's not showing up. I'll try again in a few minutes as it may be that my posts are still being moderated. If this goes through without moderation, I'll assume the last post was somehow lost....

Post with links from new users get moderated

Just get the output from mplayer and include it in the post.

RipNCode
2019-04-26, 13:23
Post with links from new users get moderated

Just get the output from mplayer and include it in the post.

In case you didn't see that is showed up on the prior page:


Playing test.wav.
get_path('sub/') -> '/home/rip/.mplayer/sub/'
[file] File size is 136492 bytes
STREAM: [file] test.wav
STREAM: Description: File
STREAM: Author: Albeu
STREAM: Comment: based on the code from ??? (probably Arpi)
libavformat version 58.20.100 (internal)
Configuration: --enable-gpl --enable-postproc
LAVF_check: WAV / WAVE (Waveform Audio)
Checking for YUV4MPEG2
ASF_check: not ASF guid!
Checking for REAL
Checking for SMJPEG
Searching demuxer type for filename test.wav ext: .wav
Trying demuxer 17 based on filename extension
==> Found audio stream: 0
======= WAVE Format =======
Format Tag: 1 (0x1)
Channels: 1
Samplerate: 16000
avg byte/sec: 32000
Block align: 2
bits/sample: 16
cbSize: 0
================================================== ========================
demux_audio: audio data 0x2C - 0x2152C
Audio only file format detected.
Load subtitles in ./
get_path('sub/') -> '/home/rip/.mplayer/sub/'
================================================== ========================
Opening audio decoder: [pcm] Uncompressed PCM audio decoder
dec_audio: Allocating 2048 + 131072 = 133120 bytes for output buffer.
AUDIO: 16000 Hz, 1 ch, s16le, 256.0 kbit/100.00% (ratio: 32000->32000)
Selected audio codec: [pcm] afm: pcm (Uncompressed PCM)
================================================== ========================
Building audio filter chain for 16000Hz/1ch/s16le -> 0Hz/0ch/??...
[libaf] Adding filter dummy
[dummy] Was reinitialized: 16000Hz/1ch/s16le
[dummy] Was reinitialized: 16000Hz/1ch/s16le
Trying preferred audio driver 'alsa', options '[none]'
alsa-init: requested format: 16000 Hz, 1 channels, 9
alsa-init: using ALSA 1.1.7
alsa-init: setup for 1/2 channel(s)
alsa-init: using device default
alsa-init: opening device in blocking mode
[AO_ALSA] alsa-lib: pcm_dmix.c:1063snd_pcm_dmix_open) unable to create IPC semaphore
[AO_ALSA] Playback open error: Permission denied
Failed to initialize audio driver 'alsa'
Trying every known audio driver...
ao2: 16000 Hz 1 chans s16le
audio_setup: using '/dev/dsp' dsp device
audio_setup: using '/dev/mixer' mixer device
audio_setup: using 'pcm' mixer device
[AO OSS] audio_setup: Can't open audio device /dev/dsp: Device or resource busy
alsa-init: requested format: 16000 Hz, 1 channels, 9
alsa-init: using ALSA 1.1.7
alsa-init: setup for 1/2 channel(s)
alsa-init: using device default
alsa-init: opening device in blocking mode
[AO_ALSA] alsa-lib: pcm_dmix.c:1063snd_pcm_dmix_open) unable to create IPC semaphore
[AO_ALSA] Playback open error: Permission denied
AO: [pulse] 16000Hz 1ch s16le (2 bytes per sample)
AO: Description: PulseAudio audio output
AO: Author: Lennart Poettering
Building audio filter chain for 16000Hz/1ch/s16le -> 16000Hz/1ch/s16le...
[dummy] Was reinitialized: 16000Hz/1ch/s16le
[dummy] Was reinitialized: 16000Hz/1ch/s16le
Video: no video
Freeing 0 unused video chunks.
Starting playback...
Increasing filtered audio buffer size from 0 to 9024
A: 4.0 (03.9) of 4.0 (04.0) 0.0%
ds_fill_buffer: EOF reached (stream: audio)
ds_fill_buffer: EOF reached (stream: audio)
A: 4.0 (04.0) of 4.0 (04.0) 0.0%
ds_fill_buffer: EOF reached (stream: audio)
A: 4.1 (04.0) of 4.0 (04.0) 0.0%
EOF code: 1

Uninit audio filters...
[libaf] Removing filter dummy
Uninit audio: pcm
vo: x11 uninit called but X11 not initialized..

Exiting... (End of file)

d6jg
2019-04-26, 14:12
In the middle there are lots of “permission denied” errors when trying to open for playback

RipNCode
2019-04-26, 14:31
In the middle there are lots of “permission denied” errors when trying to open for playback

True, but it's a bit of a misnomer. I generated the file on the Pi with Pico TTS and downloaded it to my admin machine to play it back, just to get the data from mplayer. The Pi doesn't have audio output directly-only a connection to the LMS via the integrated audio sink. So the errors being thrown are because my admin machine isn't set up for this. I just did it to make getting the verbose data a bit quicker. As I mentioned, all other audio playback is not a problem. Hopefully, when done, Pico TTS will not even create a file - it will just stream the audio out directly. Unfortunately, it only does so as a wav...

bpa
2019-04-26, 14:38
Playing test.wav.
get_path('sub/') -> '/home/rip/.mplayer/sub/'
[file] File size is 136492 bytes
STREAM: [file] test.wav
STREAM: Description: File
STREAM: Author: Albeu
STREAM: Comment: based on the code from ??? (probably Arpi)
libavformat version 58.20.100 (internal)
Configuration: --enable-gpl --enable-postproc
LAVF_check: WAV / WAVE (Waveform Audio)
Checking for YUV4MPEG2
ASF_check: not ASF guid!
Checking for REAL
Checking for SMJPEG
Searching demuxer type for filename test.wav ext: .wav
Trying demuxer 17 based on filename extension
==> Found audio stream: 0
======= WAVE Format =======
Format Tag: 1 (0x1)
Channels: 1
Samplerate: 16000
avg byte/sec: 32000
Block align: 2
bits/sample: 16


Pity there is no URL as mplayer says it guessed the format based on the file name !! You did use an URL ? Can you post the command you used so the URL & filename can be examined also the full log.

A couple of other problem - 1 channel and 16kHz a sample rate - neither s these are supported by SB players - so LMS will have trasncode the stream into 2 channel and at least 32kHz.


THis is a fragment the sort of output that is neede to determine how the wav is being streamed (i.e. all the HTTP stuff)


mplayer -v -playlist http://www.somafm.com/groovesalad.pls
Filename for url is now http://www.somafm.com/groovesalad.pls
Filename for url is now http://www.somafm.com/groovesalad.pls
STREAM_HTTP(1), URL: http://www.somafm.com/groovesalad.pls
Resolving www.somafm.com for AF_INET6...

Couldn't resolve name for AF_INET6: www.somafm.com
Resolving www.somafm.com for AF_INET...
Connecting to server www.somafm.com[74.217.192.206]: 80...

--- HTTP DEBUG HEADER --- START ---
protocol: [HTTP/1.1]
http minor version: [1]
uri: [(null)]
method: [(null)]
status code: [301]
reason phrase: [Moved Permanently]
body size: [194]
Fields:
0 - Server: nginx/1.10.3 (Ubuntu)
1 - Date: Fri, 26 Apr 2019 21:40:01 GMT
2 - Content-Type: text/html
3 - Content-Length: 194
4 - Connection: close
5 - Location: http://somafm.com/groovesalad.pls
--- HTTP DEBUG HEADER --- END ---
Filename for url is now http://somafm.com/groovesalad.pls
Resolving somafm.com for AF_INET6...

Couldn't resolve name for AF_INET6: somafm.com
Resolving somafm.com for AF_INET...
Connecting to server somafm.com[74.217.192.208]: 80...

--- HTTP DEBUG HEADER --- START ---
protocol: [HTTP/1.1]
http minor version: [1]
uri: [(null)]
method: [(null)]
status code: [200]
reason phrase: [OK]
body size: [511]
Fields:
0 - Server: nginx/1.10.3 (Ubuntu)
1 - Date: Fri, 26 Apr 2019 21:40:02 GMT
2 - Content-Type: audio/x-scpls
3 - Content-Length: 511
4 - Connection: close
5 - Expires: Fri, 26 Apr 2019 21:40:12 GMT
6 - Cache-Control: max-age=10
7 - X-Proxy-Cache: MISS
8 - X-Proxy-Cache: MISS
9 - Access-Control-Allow-Origin: *
10 - Accept-Ranges: bytes
--- HTTP DEBUG HEADER --- END ---
Content-Length: [511]
Content-Type: [audio/x-scpls]
Cache size set to 320 KBytes

RipNCode
2019-04-26, 14:49
Pity there is no URL as mplayer says it guessed the format based on the file name !! You did use an URL ? Can you post the command you used so the URL & filename can be examined.

A couple of other problem - 1 channel and 16kHz a sample rate - neither s these are supported by SB players - so LMS will have trasncode the stream into 2 channel and at least 32kHz.

The URL is probably encapsulated in the java binding. I'd have to figure out to pull that out as it's not obvious. In this case, I made Pico create a file to pull the relevant metadata.

pico2wave -w "test.wav" "Let's hope this audio file works this time!"

From there, I took test.wav (which doesn't stream with any audio output) and ran it on a local laptop with mplayer as a file to pull the data. I also used lame to convert test.wav to test.mp3, and that does stream correctly to the SB player. The trick I'm going to run into is that I should be able to have a TTS system dynamically (no file creation) convert strings to speech and stream them on the fly.

I think the issue of SB players not supporting 1 channel/16kHz is probably the issue at hand. I don't believe I can make the Pico output anything else, so this may be a dead end if it's not possible to get a player to work with the current output. Maybe it's time to go back to the drawing board on this design...

bpa
2019-04-26, 23:35
The URL is probably encapsulated in the java binding. I'd have to figure out to pull that out as it's not obvious. In this case, I made Pico create a file to pull the relevant metadata.

pico2wave -w "test.wav" "Let's hope this audio file works this time!"

From there, I took test.wav (which doesn't stream with any audio output) and ran it on a local laptop with mplayer as a file to pull the data. I also used lame to convert test.wav to test.mp3, and that does stream correctly to the SB player. The trick I'm going to run into is that I should be able to have a TTS system dynamically (no file creation) convert strings to speech and stream them on the fly.

I'm sorry I don't understand the problem. You say LMS can't play a http stream WAV LMS but yet you cannot show mplayer playing the same URL.

Playing the file does not help with how a file is streamed over http

The http handling is half the issue with playing on LMS but it the issue that needs most work.


I think the issue of SB players not supporting 1 channel/16kHz is probably the issue at hand. I don't believe I can make the Pico output anything else, so this may be a dead end if it's not possible to get a player to work with the current output. Maybe it's time to go back to the drawing board on this design...

Transcoding from 1 channel 16Khz to anything is dead easy - once an URL and mplayer output has been given. There have been a number of users who have got TTS working with LMS players for announcements. A recent one is https://forums.slimdevices.com/showthread.php?109096-Announce-LMS-Announce-play-announcements-notifications but I think their approach may not work for you.

Paul Webster
2019-04-26, 23:49
Thread over on OpenHAB shows a bit more about how things work ... and also hints at a possible workaround.
i.e. move from PicoTTS to Google Cloud Voice (whatever that is)

https://community.openhab.org/t/solved-using-squeezebox-as-audio-sink-for-tts/67652