Stereophile Article: MP3 vs AAC vs FLAC vs CD

[Nikhil]

Article from Stereophile about the age old argument, with lots of graphs included.

http://www.stereophile.com/features/308mp3cd/

[opaqueice]

Article from Stereophile about the age old argument, with lots of graphs included.

http://www.stereophile.com/features/308mp3cd/

While the graphs are kind of interesting, they'd be much more so if he'd actually explain a bit more about why they look like that. It's not like it's a mystery.

Then there are posturings like this:

Given the bigger bit budget at 320kbps, the AAC codec produces a result that may well be indistinguishable from CD for some listeners some of the time with some music.

So he's claiming that by looking at a graph of this particular test-tone he can tell whether or not these effects are audible on music. And yet he didn't bother to do a listening test - in the time it took him to prepare that article, he could have performed a whole series. That would have been interesting, because what most people really want to know is which format to use and at what level of compression. This kind of vague cursory analysis and unsubstantiated statements isn't useful.

Frankly, I very much doubt he would have been able to hear the differences between 320 MP3 or AAC and CD (at least not without lots of practice and choosing the test track very carefully), and after all the whining Stereophile has been doing recently that wouldn't look very good.

[Nikhil]

While the graphs are kind of interesting, they'd be much more so if he'd actually explain a bit more about why they look like that. It's not like it's a mystery.

Indeed

Frankly, I very much doubt he would have been able to hear the differences between 320 MP3 or AAC and CD (at least not without lots of practice and choosing the test track very carefully),

That's something I have been wanting to ask for a while. Just to be on the safe side (future proof and archival value), and since storage is becoming cheaper by the day, I have been using FLAC for quite a few years now. I can certainly spot the limitations of 128 kbps mp3s and very early in this game, I was very unhappy even with 256kbps encoded mp3s. But in the several years since I bothered to test, the encoding has obviously got a lot better. Are there any particular tracks that can showcase the limitations of todays best mp3 encodings (ie, 320 kbps or high VBR mp3s?)

[adamslim]

So he's claiming that by looking at a graph of this particular test-tone he can tell whether or not these effects are audible on music. And yet he didn't bother to do a listening test - in the time it took him to prepare that article, he could have performed a whole series. That would have been interesting, because what most people really want to know is which format to use and at what level of compression. This kind of vague cursory analysis and unsubstantiated statements isn't useful.

Frankly, I very much doubt he would have been able to hear the differences between 320 MP3 or AAC and CD (at least not without lots of practice and choosing the test track very carefully), and after all the whining Stereophile has been doing recently that wouldn't look very good.

Yes, a typical JA article. This is the man who won't listen to a SET amp as he considers it a tone control, IIRC.

It would be so easy for them to do a detailed blind test using a high resolution system; they could even highlight any changes in the results generated by the listeners getting familiar with the MP3/AAC artefacts. That would have been interesting! The vague 'ooh look at those charts' tone of the article is at best pointless, at worst misleading.

Moral: don't expect investigative reporting in the hifi press!

[opaqueice]

That's something I have been wanting to ask for a while. Just to be on the safe side (future proof and archival value), and since storage is becoming cheaper by the day, I have been using FLAC for quite a few years now. I can certainly spot the limitations of 128 kbps mp3s and very early in this game, I was very unhappy even with 256kbps encoded mp3s. But in the several years since I bothered to test, the encoding has obviously got a lot better. Are there any particular tracks that can showcase the limitations of todays best mp3 encodings (ie, 320 kbps or high VBR mp3s?)

I too use FLAC, and for the same reasons haven't paid that much attention to lossy formats. However my impression is that the MP3 codec has improved considerably in recent years. One obvious thing is to use VBR (variable bit rate) rather than CBR. AFAIK VBR is always superior, and yet most MP3s don't use it.

As for 320, my understanding is that if you know the weak points of the particular MP3 encoder you're using you can find of construct a track that will reveal them pretty easily. I've never seen a positive ABX result for 320 MP3, but I wouldn't be surprised if they exist.

Actually I'd say a good bet is a strongly asymmetric test tone. You can hear absolute polarity inversion with that pretty easily, and I think all these lossy algorithms completely destroy phase coherency, so that should manifest itself as a subtle change in the pitch of asymmetric tones. Never tried it though.

The best place I know of to ask questions like that is the hydrogenaudio forums - I think some of the designers of these codecs hang out there.

[RGibran]

Then there are posturings like this:

Quote:
Originally Posted by John Atkinson
Given the bigger bit budget at 320kbps, the AAC codec produces a result that may well be indistinguishable from CD for some listeners some of the time with some music. [/quote]

i]That[/i] would have been interesting, because what most people really want to know is which format to use and at what level of compression. This kind of vague cursory analysis and unsubstantiated statements isn't useful.

Geez, isn't that what he said?

Frankly, I very much doubt he would have been able to hear the differences between 320 MP3 or AAC and CD (at least not without lots of practice and choosing the test track very carefully), and after all the whining Stereophile has been doing recently that wouldn't look very good.

Who's posturing now?

I too use FLAC, and for the same reasons haven't paid that much attention to lossy formats. "HOWEVER MY IMPRESSION" is that the MP3 codec has improved considerably in recent years. One obvious thing is to use VBR (variable bit rate) rather than CBR. "AFAIK" VBR is always superior, and yet most MP3s don't use it.

As for 320, "MY UNDERSTANDING" is that "IF" you know the weak points of the particular MP3 encoder you're using you can find of construct a track that will reveal them pretty easily. I've never seen a positive ABX result for 320 MP3, but I wouldn't be surprised "IF" they exist.

Actually I'd say "A GOOD BET" is a strongly asymmetric test tone. You can hear absolute polarity inversion with that pretty easily, and "I THINK" all these lossy algorithms completely destroy phase coherency, so that "SHOULD" manifest itself as a subtle change in the pitch of asymmetric tones. "NEVER TRIED IT THOUGH".

Thanks for the chuckles.

RG

[ralphpnj]

Actually that particular Stereophile article is a direct result of a recent thread on the Stereophile forum. (http://forum.stereophile.com/forum/s...fpart=all&vc=1)
Note: I'm "jazzfan" on the Stereophile forum)

If you take the time to read/scan through the thread you'll see that JA takes all these things very seriously and does respond to forum posts. As for the lack of comparison testing I can only guess that since JA and Stereophile take the position that files made with lossy compression are do not offer true high end sound quality, then why bother to run all those tests. And while I completely agree that in this day and age of cheap storage there is no reason to use a lossy codec, it would still be nice to know which lossy codec to use in the event that one has a need for it (think iPod).

[opaqueice]

Thanks for the chuckles.

Glad to oblige.

As for the lack of comparison testing I can only guess that since JA and Stereophile take the position that files made with lossy compression are do not offer true high end sound quality, then why bother to run all those tests.

But then why bother with the graphs, or the article at all, for that matter?

And while I completely agree that in this day and age of cheap storage there is no reason to use a lossy codec, it would still be nice to know which lossy codec to use in the event that one has a need for it (think iPod).

Yeah, it would be nice. And this is one case where subjective assessments are all you need. No one doubts that 128 kbps MP3 is easily distinguishable from redbook, so the question becomes which codec and settings are the least objectionable.

But the issue is fast become irrelevant in any case - the latest HD-based ipods have 160GB of memory, and the flash-based ones are getting up there fast. I suppose the next debate will be over the necessity of hi-res formats.

[pfarrell]

Stereophile Article: MP3 vs AAC vsFLAC vs CD

opaqueice wrote:
>> And while I completely agree that in this day and age of cheap storage
>> there is no reason to use a lossy codec, it would still be nice to know
>> which lossy codec to use in the event that one has a need for it (think
>> iPod).
>
> But the issue is fast become irrelevant in any case - the latest
> HD-based ipods have 160GB of memory, and the flash-based ones are
> getting up there fast. I suppose the next debate will be over the
> necessity of hi-res formats.

The question was moderately interesting when folks used "56KB" modems to
download stuff. But its really moot now.

In any environment that you will do serious listening to the music, you
don't need to compress the hell out of it. So don't. If you are riding a
New York subway, or in any car short of a Rolls Royce or big Mercedes,
it doesn't matter.

Store your files long term in something like FLAC, transcode them to
suit your PMP to something like 386KB VBR, and be done. Further
optimization is silly.

All IMHO, but this is audiophiles....

[ralphpnj]

Could someone please explain just where all the voodoo science is in the following statement (taken from the article in question):

As a CD plays, the two channels of audio data (not including overhead) are pulled off the disc at a rate of just over 1400 kilobits per second. A typical MP3 plays at less than a tenth that rate, at 128kbps. To achieve that massive reduction in data, the MP3 coder splits the continuous musical waveform into discrete time chunks and, using Transform analysis, examines the spectral content of each chunk. Assumptions are made by the codec's designers, on the basis of psychoacoustic theory, about what information can be safely discarded. Quiet sounds with a similar spectrum to loud sounds in the same time window are discarded, as are quiet sounds that are immediately followed or preceded by loud sounds. And, as I wrote in the February 2008 "As We See It," because the music must be broken into chunks for the codec to do its work, transient information can get smeared across chunk boundaries.

Seems to all be based on good sound reasoning and hard scientific facts not voodoo or magic. The only issue is whether or not one believes that these changes to the musical waveform are audible and if they are indeed audible then under what conditions will they be audible. After that is only a matter of determining whether or not those conditions where the changes to the waveform are audible are important to one's normal listening situation.

[darrenyeats]

You've said it yourself. The crucial bit is audibility, and that is the bit not addressed in the article and just assumed to be true.

The listening test referenced was pretty weak and it makes me wonder. If the same test method (ONE play, missing pertinent technical details - e.g. downsampling/dithering for red book - apparently they "looked at each other" which is taken to mean they all had identical impressions) had been used by "the skeptics" how conclusive would it be seen as?

If anyone can show me some real evidence that CD isn't transparent, I will change my opinion of course.
Darren

[smc2911]

If you take the time to read/scan through the thread you'll see that JA takes all these things very seriously and does respond to forum posts. As for the lack of comparison testing I can only guess that since JA and Stereophile take the position that files made with lossy compression are do not offer true high end sound quality, then why bother to run all those tests. And while I completely agree that in this day and age of cheap storage there is no reason to use a lossy codec, it would still be nice to know which lossy codec to use in the event that one has a need for it (think iPod).

I'd tend to agree with this observation. I think that the starting point (which is rather similar to my own position at home) is that since you can have lossless, why bother with anything else? As for the rest, it looks like he was having fun playing with graphs. However, there is one point no-one has mentioned that I would take issue with:

Something I have rarely seen discussed is the fact is that because all compressed file formats, both lossless and lossy, effectively have zero data redundancy, they are much more vulnerable than uncompressed files to bit errors in transmission.

As I understand it (please correct me if I'm wrong) SB uses TCP/IP to stream, which means that it is error free. The only problem you can run into is that if your bandwidth is so terrible that your buffer runs out, but this would be more likely to happen with an uncompressed format (although unlikely even then). So, long live FLAC!

[darrenyeats]

So, long live FLAC!

Well, quite. I believe CD and FLAC are all we need in terms of wide format acceptance, freedom from DRM and sound quality. Although high rate MP3 "might be" transparent my opinion is so what.
Darren

[opaqueice]

Could someone please explain just where all the voodoo science is in the following statement (taken from the article in question):



Seems to all be based on good sound reasoning and hard scientific facts not voodoo or magic. The only issue is whether or not one believes that these changes to the musical waveform are audible and if they are indeed audible then under what conditions will they be audible. After that is only a matter of determining whether or not those conditions where the changes to the waveform are audible are important to one's normal listening situation.

Where in the world did that come from? Who said anything about voodoo or magic?

There are some problems with that passage, though. For example this:

As a CD plays, the two channels of audio data (not including overhead) are pulled off the disc at a rate of just over 1400 kilobits per second. A typical MP3 plays at less than a tenth that rate, at 128kbps.

is misleading to the point of being just wrong. MP3 files "play" at exactly the same rate as CDs - for example you could simply decode one into a 16/44.1 WAV file and then play it, and that's actually what at least some (if not all) decoders/players do. That semantic quibble aside, the underlying logic is more seriously wrong, because by precisely the same reasoning you could say "a FLAC file plays at less than half the rate of a CD...", and yet FLAC and WAV playback are identical.

It's also odd that he focuses on time-domain issues there, and yet all his plots are frequency domain. That's part of what I meant by my original comment about the graphs.

[opaqueice]

However, there is one point no-one has mentioned that I would take issue with: As I understand it (please correct me if I'm wrong) SB uses TCP/IP to stream, which means that it is error free. The only problem you can run into is that if your bandwidth is so terrible that your buffer runs out, but this would be more likely to happen with an uncompressed format (although unlikely even then). So, long live FLAC!

Yeah, that's a good point.

Since no one transmits FLAC except over a TCP/IP network, the statement is nonsensical. It's like saying standard uncompressed files are superior to zipped files because zipped files have less redundancy, when the opposite is true - and the whole point!