Wrong transcoding settings to MP3 in default install?

[gorman]

Unless I'm mistaken, there is a problem with the standard settings that LMS used to transcode when streaming to a bitrate limited player.

This is the string that's used, for instance, to transcode from FLAC:

[lame] --silent -q $QUALITY$ $RESAMPLE$ -v $BITRATE$

Now, the quality setting is the one we set under each player settings (0 to 9, 9 being the default that uses a fast psychoacoustic model, good for high bitrates). The problem comes with the -v switch that is used. That is equivalent to use -V 4, which yelds a bitrate of about 165kbps (it varies, considering the -v settings are VBR).

This makes completely worthless the $BITRATE thing, considering it translates to {BITRATE=-B %B}.

So, supposing I set a player to Quality 9 (the default that the interface tells us sounds very good at high bitrates) and limit it to 320kbps... what do I get? In case of a FLAC to be transcoded I think that the normal expectation would be to expect a 320kbps MP3 to be streamed. But if I'm not mistaken, that's not the case. Because the transcoding line would read like this:

lame.exe --silent -q 9 -v -B 320

This encoded a V4 variable bit rate MP3, which has an upper bitrate limit of 320kbps. But V4 is never going to hit that limit, sitting conveniently in its usual 165kbps range. So even though LMS is telling the player that it's sending it a 320kbps MP3, I really don't see how that could be the case.

Wanting to keep the same approach (otherwise I think the changes needed would be more far reaching), the correct line to use in convert.conf should be:

[lame] --silent -q $QUALITY$ $RESAMPLE$ -V 0 $BITRATE$

This would use the V0 preset that has a target bitrate of 245kbps (http://wiki.hydrogenaudio.org/index....te.29_settings), with bitrate peaks at 260kbps. Considering that the current settings go from 256 to 320, it would still make sense to keep them as they are. 256 would strictly enforce the 256kbps limit, while 320 would let the V0 preset more space to "breathe".

I don't think that, given the currently available variables we could have a real 320kbps CBR setting, as that would require the use of the -b 320 switch, that's not compatible with the rest of the bitrates.

All this, as I mentioned at the beginning, unless I'm mistaken and mp3 transcoding happens in a whole different way.

[JJZolx]

It doesn't make any sense to me either.

The default Mp3 transcoding rules began using CBR, then it was switched to ABR, and then changed to -V4 VBR with a bitrate cap. I don't know what the logic was behind the last switch, but it does seem flawed.

I'm not sure how much better using something like "-V0 -B xxx" would be with lower bitrates such as 160. For a given bitrate/quality level, you end up needlessly limiting the bitrate of frames that need it more and you somewhat increase the bitrate of frames that would be given less at that level.

The following is from the LAME documentation. Using -B with ABR/VBR potentially gives you lower quality than even CBR.

-B n Max bitrate

Maximum allowed bitrate when using VBR/ABR.

Using -B is NOT RECOMMENDED. A 128 kbps CBR bitstream, because of the bit reservoir, can actually have frames which use as many bits as a 320 kbps frame. ABR/VBR modes minimize the use of the bit reservoir, and thus need to allow 320 kbps frames to get the same flexibility as CBR streams. This is useful only if you need to circumvent a buggy hardware device with strange bitrate constrains.
Default value is the maximum allowed bitrate for the MPEG versio (i.e. no limit)

Using ABR still makes the most sense. It uses a variable bitrate, so individual frame rates vary according to encoding need, but the overall streamed bitrate is shaped to fit a particular bandwidth level, as set in the LMS player settings. Isn't that the whole idea behind bitrate limiting, rather than just putting a cap on the maximum bitrate of a tiny part of the encoded stream?

You can fix it simply by placing the following rule in custom-convert.conf:

flc mp3 * *
	# FB:{BITRATE=--abr %B}T:{START=--skip=%t}U:{END=--until=%v}D:{RESAMPLE=--resample %D}
	[flac] -dcs $START$ $END$ -- $FILE$ | [lame] --silent -q $QUALITY$ $BITRATE$ $RESAMPLE$ - -

[gorman]

If Cpu power and strict bitrate are not critical, one could also do:

flc mp3 * *
# FB:{BITRATE=--abr %B}T:{START=--skip=%t}U:{END=--until=%v}D:{RESAMPLE=--resample %D}
[flac] -dcs $START$ $END$ -- $FILE$ | [lame] --silent -V $QUALITY$ $RESAMPLE$ - -

Bitrate limiting is not used for encoding (just, I suppose, for triggering the transcode) and one can use the Quality setting to choose among the different VBR presets, that are definitely the most optimized toward quality. The default quality setting for Lame is 5, for the -q switch. I guess one could hardcode the desired one, taking into account his CPU (it's not something that needs to change from player to player).

Would the above work?

[JJZolx]

Yeah, that would work if you want to dictate the VBR level instead of the bitrate. But I think that most people using bitrate limiting really are using it to limit the streaming bitrate. Either to deal with a limited bandwidth internet connection (streaming from their home LMS server), or to else to deal with wifi bandwidth issues.

VBR levels don't actually have a target bitrate. For example, I was doing some testing of LAME recently and found that different versions and different files can produce surprisingly different overall bitrates. These are the resulting bitrates of just a small pert of that test:

Del McCoury - Rambling Boy.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       242   212   192   176   161   144   129   107    94
3.99.5       266   226   193   177   155   130   111   102    89

Dizzy Gillespie - Manteca.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       246   221   201   180   167   152   137   116   101
3.99.5       263   224   196   181   162   139   119   108    94

Ella Fitzgerald - Anything Goes.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       167   144   131   116   107    96    87    72    63
3.99.5       187   159   137   123   109    92    79    71    60

George Baker Selection - Little Green Bag.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       256   233   213   184   170   157   141   121   107
3.99.5       268   235   205   190   171   150   133   120    98

George Jones - She Thinks I Still Care.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       231   198   178   161   148   134   121   105    93
3.99.5       264   212   180   166   147   127   110   102    87

Jack Johnson - Better Together.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       242   215   194   166   152   137   123   105    93
3.99.5       263   227   193   172   152   130   113   103    87

Jimmy Cliff - Ska All Over The World.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       227   207   191   179   166   151   135   114   100
3.99.5       249   218   191   179   160   133   113   105    93

John Coltrane - Giant Steps.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       269   227   203   184   171   156   140   126   113
3.99.5       292   238   201   187   167   145   126   121   102

Johnny Winter - Mojo Boogie.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       256   224   200   171   156   142   131   106   100
3.99.5       284   232   193   170   150   130   114    98    77

Kanye West - Gold Digger.wav
             -V0   -V1   -V2   -V3   -V4   -V5   -V6   -V7   -V8
3.98.4       220   190   172   147   137   125   115    95    87
3.99.5       249   213   178   159   143   125   109    96    77

[gorman]

Really interesting comparison, thanks for sharing.

For me it's just a matter of a couple of my SBs wifi connections not being up to FLAC bitrates. But they can handle V0 bitrates just fine.

In any case, I'll later report this as a bug. No way the default settings do what they're supposed to.

By the way, would you be so kind as to point me to documentation for the capabilities portion of convert.conf? The one where you setup stuff such as $BITRATE.

[JJZolx]

By the way, would you be so kind as to point me to documentation for the capabilities portion of convert.conf? The one where you setup stuff such as $BITRATE.

The only documentation that I know of is in the convert.conf file itself. The top portion of the file is a very large comment explaining how the rules are set up. Then the rules themselves provide examples. If you want to override the rules you shouldn't edit convert.conf, but instead create a file called custom-convert.conf (in the same directory) with any rules that you want to either add or redefine.

[gorman]

Thanks, I knew about the built-in docs, as well as the custom file to be created. But thanks for the help, it's really appreciated.

[gorman]

In any case, one thing to give thought to would be to use Ogg Vorbis to handle transcoding. Vorbis is natively gapless and sounds better than LAME at low bitrates (at least according to general knowledge at Hydrogenaudio).

Too bad that it's pretty impossible to handle transcoding to a different format than MP3 when it needs to be triggered by bitrate limiting.

bugs.slimdevices.com is down???

[gorman]

Posted on Bugzilla: http://bugs.slimdevices.com/show_bug.cgi?id=17953

[gorman]

Nobody voted for this bug... was I wrong?

[JJZolx]

Doesn't matter. Server development is dead.

[gorman]

Fine. But at least people could comment here if I am right or wrong.

Edit:

I am right. I opened the http://localhost:9000/stream.mp3 stream on foobar2000. I confirm that the default transcoding limits results to -V4 preset quality. I advise people to use JJZolx suggested line, that is:

flc mp3 * *
	# FB:{BITRATE=--abr %B}T:{START=--skip=%t}U:{END=--until=%v}D:{RESAMPLE=--resample %D}
	[flac] -dcs $START$ $END$ -- $FILE$ | [lame] --silent -q $QUALITY$ $RESAMPLE$ $BITRATE$ - -
# special case for smart bitrates on mp3 input
mp3 mp3 transcode *
	# IFB:{BITRATE=--abr %B}D:{RESAMPLE=--resample %D}
	[lame] --silent -q $QUALITY$ $BITRATE$ $RESAMPLE$ --mp3input $FILE$ -

for flac and mp3 transcoding (and, basically, whatever else you want to transcode to mp3).

[gorman]

I've actually found out why the mistake happened. It didn't happen. Back then. When it was decided to move to this configuration for transcoding, the -v switch simply indicated to use VBR encoding. In later versions of LAME this changed and it became a substitute for -V 4.

Old LAME (3.89) relevant help section:

VBR options:
  -v              use variable bitrate (VBR) (--vbr-mtrh)
  --vbr-old       use old variable bitrate (VBR) routine
  --vbr-new       use new variable bitrate (VBR) routine
  --vbr-mtrh      a merger of old and new (VBR) routine
  -V n            quality setting for VBR.  default n=4
                  0=high quality,bigger files. 9=smaller files
  -b <bitrate>    specify minimum allowed bitrate, default  32 kbps
  -B <bitrate>    specify maximum allowed bitrate, default 320 kbps

New LAME (3.99.5) relevant help section:

VBR options:
  -V n            quality setting for VBR.  default n=4
                  0=high quality,bigger files. 9=smaller files
  -v              the same as -V 4
  --vbr-old       use old variable bitrate (VBR) routine
  --vbr-new       use new variable bitrate (VBR) routine (default)
  -Y              lets LAME ignore noise in sfb21, like in CBR
  -b <bitrate>    specify minimum allowed bitrate, default  32 kbps
  -B <bitrate>    specify maximum allowed bitrate, default 320 kbps

The sad thing is that Squeezeboxes display the transcoding bitrate that the server communicates, based on server settings, and not the actual bitrate of the file being played back. Hence the problem in discovering this (and why I used foobar2000 streaming to check).

It's also worth noting that in the LAME documentation (http://lame.cvs.sourceforge.net/viewvc/lame/lame/USAGE) it specifically states:

Using -B is NOT RECOMMENDED. A 128 kbps CBR bitstream, because of the
bit reservoir, can actually have frames which use as many bits as a
320 kbps frame. ABR/VBR modes minimize the use of the bit reservoir, and
thus need to allow 320 kbps frames to get the same flexability as CBR
streams. This is useful only if you need to circumvent a buggy hardware
device with strange bitrate constrains.

Also, even by using the --abr <bitrate> option we automatically accept that the -Y switch is used (http://wiki.hydrogenaudio.org/index....=LAME_Y_SWITCH). At least theoretically this could lead to reduced quality for higher frequencies.

One could thus decide to accept a more variable bitrate and make the most out of LAME quality by using these conversion settings:

flc mp3 * *
	# F:{PATH=%f}T:{START=--skip=%t}U:{END=--until=%v}D:{RESAMPLE=--resample %D}
	[flac] -dcs $START$ $END$ -- $FILE$ | [lame] --silent -h -V $QUALITY$ $RESAMPLE$ - -
# special case for smart bitrates on mp3 input
mp3 mp3 transcode *
	# IF:{PATH=%f}D:{RESAMPLE=--resample %D}
	[lame] --silent -h -V $QUALITY$ $RESAMPLE$ --mp3input $FILE$ -

The -h switch is the high quality equivalent of -q (it equals -q 2), -V $QUALITY$ uses the quality you set for each player as the value for the VBR preset to use while encoding (http://wiki.hydrogenaudio.org/index....te.29_settings).
Although, by using this, one loses the option to go to 320kbps, since -V 0 equals ~245 kbps. On the other hand, there's no ABX test proving a difference between -V 0 and -b 320.

[JJZolx]

I've actually found out why the mistake happened. It didn't happen. Back then. When it was decided to move to this configuration for transcoding, the -v switch simply indicated to use VBR encoding. In later versions of LAME this changed and it became a substitute for -V 4.

There was no change to this. The documentation was updated for 3.99.5, but -v has always meant use VBR, which will be encoded at the default VBR quality setting of 4.

The "mistake" was intentional. Boneheaded, but intentional.

[gorman]

There was no change to this. The documentation was updated for 3.99.5, but -v has always meant use VBR, which will be encoded at the default VBR quality setting of 4.

The "mistake" was intentional. Boneheaded, but intentional.

Doh! Well, that's a real mistake then. In any case, I've done several experiments that I added to my previous post. There's two approaches: one that puts a premium to constant bitrate, using ABR as you suggested, and another one where bitrate fluctuates more but the best quality is achieved through the VBR presets.

If a standard one could be chosen, I'd go with yours. Mine is interesting for people willing to experiment.