PDA

View Full Version : Mesauring/Spectral analysis of streaming services



CommanderROR
2014-08-06, 05:08
Hi everyone.

I've been struggling with the problem of sound quality in the various streaming services for quite some time now, and I'd really like to see some "real" measurements.
My question: Is somebody here who can pipe the digital stream from Wimp Hifi/Spotify/Google Play Music through some kind of analysis software to show where the different services cut off frequencies and if/how they limit dynamic range and so on?

I'd like to do this myself, because I currently have active subsriptions for these three services (Wimp is only Premium right now, but I can upgrade that for the test) but I don't know how and probably don't have the right Hardware/Software.

A list of reference tracks that are available for all services would be the smallest problem.


Any ideas? Or has this been done already?

Thanks.

pippin
2014-08-06, 06:29
I haven't done spectral analysis, just listening tests but my impression is that they are pretty similar.
Most services seem to get the normal CD mastering from the record labels with all their limitations, especially WRT dynamic compression.
Qobuz look to me to be the only ones who have significantly different material.
Spotify seem to do some re-encoding to defer the dynamic compression to the client (make it optional) but they still sound mostly the same to me.

CommanderROR
2014-08-06, 09:04
Might be.
I found that "audacity" is a useful tool (and free) on thr PC. It can capture the Audio from the soundcard and record it. I did some very brief testing and saw interesting results already.
I'll do some more tests.
If you have an active Qobuz account you can also help if you like!

I'm using the track "Till Eulenspiegels Lustige Streiche" in the recording conducted by Semyon Bychkov as my first sample. Will post some preliminary findings if I manage to get something useful.

dafiend
2014-08-06, 09:57
Spotify seem to do some re-encoding to defer the dynamic compression to the client (make it optional) but they still sound mostly the same to me.
Spotify gives you an option "volume normalization" which is something akin to ReplayGain or Apple's SoundCheck. The volume change is done by the client on the decoded PCM audio. There's no re-encoding going on.

--

Basically, I think everybody is pretty much getting the same masters. Yes, I'm sure there are exceptions here and there just as you see them occasionally in download stores (e.g., where one store sells the 1999 remaster and another the 2008 remaster). But for the most part, they simply serve you the same stuff. Only difference is in the lossy (or lossless) codec used for streaming.

And the streaming services are pretty upfront about which codecs (and codec profile and bitrates) they use in which scenario. Many comparative reviews of streaming services provide tables with that information, too. I dout there's much insight to be gained from analyzing the streams.

CommanderROR
2014-08-06, 10:39
I did a short run with Audacity and analized the first part of the file (as much as audacity allows in one go).
The results are interesting, especially with Spotify Options for normalization turned off (the results are in the file Spotify_HQ.txt). The results are stored as .txt files sadly, which is sort of stupid. I'll see about screenshots some other time.

Google Music has a pretty drastic cutoff at 20000Hz, Wimp (AAC) and Spotify (ogg) carry on beyond that for a bit. Spotify with Normalization turned on makes everything louder.

Sadly my recording method introduces quite a lot of background noise (no idea why, it's a bass hum) but the results are still not too bad. i'll do some more work on this when I get around to it.

@dafiend
The streaming servides do give us info on the codec and bitrate, but the interesting bits are in the setings of the encoder. they can make all the difference...

UPDATE: I added Deezer. It seems to have the same cutoff as Google Play Music

dafiend
2014-08-06, 13:55
@dafiend
The streaming servides do give us info on the codec and bitrate, but the interesting bits are in the setings of the encoder. they can make all the difference...
Honestly, I don't get the impression you really know what you are talking about. From the OP, it isn't entirely clear you even understand the difference between audio compression and dynamic range compression.

Then there's your plan to judge audio quality by doing spectral analysis. (A terrible idea, you listen with your ears, not your eyes. Due to masking, it's likely you cannot hear when content above 16 kHz is removed. But in any case, only a double-blind listening test will tell.)

CommanderROR
2014-08-06, 14:14
@dafiend

Of course I don't know what I'm doing. I'm a musician, not a sound engineer. I do know the difference between Dynamic Compression and audio compression though...

What I'm trying to see is, what the different encoders and encoder settings do to the music. 320kbit/s mp3 does not always sound the same. There is a reason for that. ogg and aac codecs are different animals altogether. Sadly, there are few tests out there that actually look at the data properly. They mostly just say "good quality because it has a high bitrate". That's nonsense.

And trusting my ears? Seriously? Human hearing is highly subjective. I saw one decent test (cannot where it was) that got things right in a way, they listened to tracks from Google Music and Spotify and found severe differences in quality, but not in favour of one service over the other, it was different from genre to genre, from track to track. They tested that with their ears which is fine, but I'd like to see the data to prove that.
I actually tried Play Music vs. Spotify and think I could hear the difference, unlike my comparison between Wimp Hifi and Spotify. It's hard to be sure though...

Let's look at classical music where Google failed according to that test. If the instruments sound thin, if precision is missing, or maybe warmth, then that shows in the overtones and their strength. I know this from testing instruments (violins) where I could very clearly see the "sound colour" in the spectral analysis of the recordings I made back then. My subjective impression matched what I saw in the "colours" .

I'm trying to apply that to the quality of the codecs used by Google, Spotify and others, however, like you said, I don't know what I'm doing, and don't have the tools to do it properly. That's why I was hoping somebody here had better equipment and more know-how.
I do believe spectral analysis is the way to go though, because it shows more than the maximum cutoff frequence. It shows the "editing" the codecs do when they do their psychoacoustic compression. That's the data I'm after.
Best case, an identical snippet from a certain recording would be captured, analyzed, and then the differences visualized, with overlay to show exactly where they differ. But that's beyond my simple means. I do see marked differences between Spotify, Wimp and the two mp3 encoded participants, Deezer and Google Music. But my recording/capture is too flawed (the lowlevel hum for example that is introduced somewhere during capture) to really analyze it properly.

pippin
2014-08-06, 15:31
Spotify gives you an option "volume normalization" which is something akin to ReplayGain or Apple's SoundCheck. The volume change is done by the client on the decoded PCM audio. There's no re-encoding going on.


Not "is", "was". When Spotify introduced that feature they obviously re-encoded all the music files in their library, the made a big fuzz around how much computing power you need and how long it takes to encode so many millions of audio files. They obviously get raw input files in PCM or FLAC something and do the ogg-encoding themselves.

I don't know what exactly they changed before and after the introduction of the client-side volume normalization, maybe before that they had done some volume normalization on the files on their side and they just removed it or something.

All I can tell is that Spotify streams without volume normalization sound different from most others. I tried WiMP, Deezer, Rhapsody, Spotify, Qobuz and MOG at the time. Deezer, WiMP (both FLAC and MP3) and MOG were indistinguishable to me, Qobuz and Spotify sounded different as did Rhapsody with Rhapsody being the most easily identifiable one, they were the only ones sounding clearly inferior. With Qobuz and, say, Deezer I am sometimes unsure which one I like better even though they sound different; this is even more so with Spotify, the difference between Spotify and the bulk of services is also smaller than the difference between Qobuz and other services, with Qobuz you can clearly hear they use different mastering on a lot of tracks, not just different encoding.

But that's listening only, no spectral analysis.

JohnB
2014-08-06, 16:38
Just a thought - I think it is important to specify whether the Spotify streaming is the 160 kbps free or the 320 kbps Premium (though even if you have Premium Spotify you might have to change the settings to get 320 kbps - I forget). It doesn't help that, some years ago, even with Premium Spotify you *used* to get streaming of the albums at 160 kbps, though they did a major overhaul a few years ago and things greatly improved, so hopefully it now delivers what it says on the can. (The only way I used to be definitely able to tell the bit rate was to monitor the internet bandwidth being used!)

Also whether the Qobuz is being streamed as mp3 or FLAC.

pippin
2014-08-06, 16:41
Spotify was at 320 Kbps, Qobuz was FLAC.

JohnB
2014-08-06, 17:03
By the way, what Spotify calls "Volume Normalization" is actually quite severe dynamic range compression (not normalization as I understand the term). Its effect on classical music is absolutely dreadful.

pippin
2014-08-06, 17:25
Yes it is. But the good thing is: because they do it on the client side instead of applying all that dynamic compression to the actual stream you can turn it off.
Record labels often want the dynamic compression because it's louder so sounds "better" at low volumes and on cheap speakers and this way Spotify can give it to them without killing the original quality.

Mnyb
2014-08-07, 11:31
By the way, what Spotify calls "Volume Normalization" is actually quite severe dynamic range compression (not normalization as I understand the term). Its effect on classical music is absolutely dreadful.

If you are very sensitive to these kind off artefacts try decode mp3 streams on the server instead of the Squeezebox , the non floating point implementation on the not so powerfull player has some limitation .

This is a real but small effect that can be heard on some tracks, not the kind of imagined differences people get with different lossles formats (which can be proven to decode to exactly the same result on a squeezebox ) .

I can not distinguish a properly made LAME encoded 320kbps from the source mp3 , you can have them with full frequncy response 20-20Khz none of that droping off at 14k

My other not so well substantiated observations :

I can on some material probably hear the mp3 decoder in the squeezebox even at 320kBps , but not if lame on the server does it .

Spotify .

You can turn of the dynamic compression in Triodes plugin ,very good .

Quality varies some stuff are not 320kBps whatever they say not good.

The good 320kBps Spotify are still not on par with a homemade 320kBps lame mp3 with all settings to the best ,even if they use ogg ? they are probably using the least cpu intesive settings they can get away with for their server load or something, not so great

Gandhi
2014-08-07, 13:40
Hi everyone.

I've been struggling with the problem of sound quality in the various streaming services for quite some time now, and I'd really like to see some "real" measurements.

One way of finding differences between music streams, saved as files, would be to use null tests with e.g. Audio DiffMaker (http://www.libinst.com/Audio%20DiffMaker.htm).

Using this, you could 'subtract' one music file from another and save the result to a difference file. You could then listen to the difference file or load it in e.g. Audacity to look at it's spectrum.

CommanderROR
2014-08-07, 15:27
@Ghandi

Thanks! That sounds like what I'm trying to do. However, I'll have to find a way to get a proper capture first. Capturing from Soundcard (Soundblaster Recon 3D PCI-E) gives me unwanted noise for some reason. Maybe if I connect one of my PiCorePlayer units (perhaps the one with the Hifiberry Digi card) to the digital input and record from there... Anyway, I'll attempt that sometime, will be away from home (holiday) for a while.

Btw... Where in Triodes app can I deactivate normalization? I never saw that option...

The files (txt) I posted are the Frequency analysis btw. It does give somw useful data, but not much. Looking at the audio spectrum im audacity is interesting, it shows Wimp (AAC) and Spotify (ogg) with noticeable differences to the mp3 encoders, I'm hoping to get a deeper understandung why with further testing.
One good thing already came of this: I discocered Deezer and am currently testing their SB integration. It's not perfect, but not bad either. Will see where it takes me next... 😁

garym
2014-08-07, 16:09
Btw... Where in Triodes app can I deactivate normalization? I never saw that option...


in LMS > Settings > Plugin tab, next to spotify plugin, select settings, then untick volume normalization option, just below "stream quality" option.

CommanderROR
2014-08-08, 01:19
Sorry, but that option doesn't exist in the latest version (2.3.7) and I never saw it in the older versions either. Below the stream quality I've got the option to deactivate cache.

CommanderROR
2014-08-08, 02:04
Ok, new attempt.

I just did another set of recordings with the same Track, this time connecting my PiCorePlayer Pi with Hifiberry Digi to the Optical in of my Soundcard. Now I have distortion free files to work with. I'll see what I can make of this.

garym
2014-08-08, 04:03
Sorry, but that option doesn't exist in the latest version (2.3.7) and I never saw it in the older versions either. Below the stream quality I've got the option to deactivate cache.

I'm using 2.3.7 version (latest) as well. I've attached a screen shot. (perhaps the plugin is smart enough to only show you the volume normalization option if one has turned on the "volume normalizaton/replay gain" use in the player settings under audio, which I have (SMART GAIN)).
16230

CommanderROR
2014-08-08, 04:20
Ah, that's it then. I don't use normalization at all (audio section), so the option does not appear in my Spotify settings. Mystery solved... ;)

CommanderROR
2014-08-18, 01:13
Sadly I'll have to give up on this. I have useable recordings and found the perfect software, Audio Diffmaker, but I simply can't align the files perfectly, so I don't get the required results. I'd have to create audio snippets that start at the exact same moment, and end at the exact same moment, and that isn't possible if you click start and stop by hand. I'm sure there is a way to do this, but I don't know how.
Maybe I'll figure it out, but I'm not too optimistic.

Julf
2014-08-18, 02:32
Sadly I'll have to give up on this. I have useable recordings and found the perfect software, Audio Diffmaker, but I simply can't align the files perfectly, so I don't get the required results. I'd have to create audio snippets that start at the exact same moment, and end at the exact same moment, and that isn't possible if you click start and stop by hand. I'm sure there is a way to do this, but I don't know how.
Maybe I'll figure it out, but I'm not too optimistic.

Doesn't Audio Diffmaker have a function for aligning the samples?

Mnyb
2014-08-18, 03:18
I supect you also want to try different samples from different genres and more popular and more obscure music withon these genres .

I have a feeling that most streaming services does not really offer thier best quality on 100% of the tracks , they are probably getting there , but reasonably they want to keep most people happy . So i supect that wimp begun with the most popular tracks when they released wimp hifi , yes they did release this option before thier full library actually was aviable as flac

Gandhi
2014-08-18, 11:54
Sadly I'll have to give up on this. I have useable recordings and found the perfect software, Audio Diffmaker, but I simply can't align the files perfectly, so I don't get the required results. I'd have to create audio snippets that start at the exact same moment, and end at the exact same moment, and that isn't possible if you click start and stop by hand. I'm sure there is a way to do this, but I don't know how.
Maybe I'll figure it out, but I'm not too optimistic.

Audio Diffmaker has a built-in function for Time Alignment. It's enabled by default and is found under Settings -> Advanced. The default parameters for this function seems very relaxed, so should not be a problem.

Gain Alignment is also enabled by default.

But one setting that might be important is *not* enabled by default; Sample Rate correction, compensation for sample rate drift. Did you try that one?

CommanderROR
2014-08-18, 13:36
Thanks for all your advice.
I am aware of the features Diffmaker offers, however, they dont' really seem to work for me. Automatically aligning doesn't help, and the sample adjustment makes thebprogram crash every time. I tried shortening my samples to 30 seconds, then it works, bit the result is not realistic. I get a diff file, but it contains an almost complete sound, just a little tinny. The differences are not that large, so somethingbis wrong.
I did a diff with two identical files for testing, and that worked perfectly, so the problem is with my recordings somehow...