View Full Version : Suggestions regarding convert.conf and transcoding

2003-11-25, 18:46
I have had a few e-mails recently from people asking if I had any plans
to update my AlienStreams plugin (which transcodes Real Player and
Windows Media streams) for the new version of the server. I am a bit
swamped with work at the moment, but it's on my to-do list. Anyway, now
that the web-based CVS browser is up and running, I decided to take a
peek at the new transcoding framework that we have all been looking
forward to (I'm still running an archaic 4.x version of the server,

convert.conf looks great, but I have a couple of suggestions that I
think might make it easier to write more flexible transcoders.

* First, input data. I assume that all of these transcoders are to
expect their own style of music data on STDIN. Thus the Slim server
handles tranport. But often Real and Windows "streams" are really
just links to pseudo-XML files, which in turn point to the actual
streams. Presumably the Slim server does not know how to grok this
type of indirection. Furthermore, that stream might itself be a
non-HTTP protocol that the Slim server doesn't grok, such as RTSP.

Therefore, it seems to me that some transcoders will need to get a URL
and handle the transport themselves. So should there be a flag in the
convert.conf entry that tells the application whether to expect data
in STDIN, or to expect a newline-terminated URL in STDIN?

* Second, non-musical output. Given that stream players may produce
output data of a visual as well as an auditory kind, it seems a shame
to throw this away. For example, if you are listening to a Real
stream and the audio goes dead, it would be nice to see the
"Rebuffering" message, so that you know it was just a network glitch.
AlienStreams went to considerable trouble to try to display this sort
of visual feedback from the player (since the track details won't be
embedded in the stream as with mp3 streams), and I would like to
have this ability with any new version.

I can think of a couple of ways to provide this facility:

- Provide a socket or fifo or pipe for the transcoder to write to.
This is how AlienStreams worked, but it was a plugin and so it was
easy enough to create a pipe and then fork. But for the new
scheme, I'm not sure how to pass this on to the transcoder.

- Grab the transcoder's STDERR and send that to the devices'
displays. But then where do real error messages go?

Both of the above also mean a lot more code in the server, which would
have to read from the transcoder and put the text on the display.
This can be tricky to debug, as I know from experience.

So I thought about it, and I have another idea, which I think would be
much easier to implement in the server, and would leave all the hard
work up to the people who write transcoders:

- Before launching the transcoder, set an environment variable eg.
SLIM_SERVER equal to the IP address/port of the Slim server, and
another environment variable eg. SLIM_CLIENTS equal to a delimited
list of the IP addresses and ports of all of the clients that will
be playing the stream. Then the trancoder will inherit those
variables, and if it wishes to, can manipulate the displays of the
clients directly by sending commands to the server via TCP. If the
server goes on to launch other transcoders, it just needs to
update the value of SLIM_CLIENTS, and the new transcoder process
will get the new values, while the old transcoder will continue
running happily with the old values.

How do those suggestions seem? I think it would be pretty easy (he says
recklessly) to rewrite AlienStreams to work with the new server, if the
option for URL input and these environment variables are available. I
took a quick look at Source.pm in ViewCVS, and these don't seem too hard
to implement.

Are you doing anything about killing possible subprocesses that may have
been launched by the transcoder? Simply closing the filehandle to which
the transcoder process is connected may not do it (I had problems with
this before). Maybe the server will want to send a particular signal to
the transcoder, which it would catch, and then try to kill all of its
children before exiting. But that's a hypothetical issue, and perhaps
it's best not to try to solve problems before they arrive.

Finally, a question: does <source_format> in convert.conf refer to the
file extension only?


Peter Heslin
2003-11-28, 13:24
On Wed, Nov 26, 2003 at 01:46:37AM +0000, slimp3 (AT) heslin (DOT) eclipse.co.uk wrote:
> convert.conf looks great, but I have a couple of suggestions that I
> think might make it easier to write more flexible transcoders.

I'm responding to my own post, since Warnock's dilemma applies to it :-)

I have done some work on the plan I indicated for Real Audio and Windows
Media transcoding for the new server, and I have made the changes I
think I need in the server, and I'm about halfway through the
external application that will feed in the data.

Before I go any further, I need advice about my implementation. There
are a few places in the server where the assumption is made that the
Slim server itself will handle any HTTP URL. I need a way to override
this, so I have modified the server so that for specified contentTypes,
the URL is simply put on the playlist or played, rather than having the
server contact the remote stream.

So far, so good. The question is where to specify which contentTypes
should be left alone by the server. At first, I added an extra column
to convert.conf, but that is wrong, since this has to be a general
preference, not specific to a particular destination device or format.
That didn't work well, so I just added a new config file, called

It looks like this:
---------------------------------------------------Beginning of file
# This file defines the sort of data that the Slim server should not try to
# stream from the network itself. For these types of data, it should hand that
# job to an auxiliary program (which may be defined in convert.conf).

# These are three-letter identifiers, as defined in types.conf. Only one entry
# is permitted per line.

---------------------------------------------------End of file
Then in types.conf, there is a line like this:

ram rm,ram audio/x-pn-realaudio

And in convert.conf, there are two lines like this:

ram mp3 * *
alienstream $URL$ -

My question is: do we really need another config file? As I have coded
it, there should be no impact on the running of the server if the file
is not present. But it does mean adding more config-file parsing code
to the server.

The other place I considered putting this info is in types.conf, where
there could be a fourth, optional column, where a * (for example) would
indicate that this particular content-type should have all transport
handled by an external program.

Which implemetation would be more a welcome patch: the current version
with transport.conf, or a modification to types.conf, or something else
I haven't thought of?