PDA

View Full Version : Random mix not so random...



Ranokivio
2008-11-25, 00:54
Hi there,

I have a squeezebox since last year and I am quite happy with it. However, I usually like to listen to my music using the random album mix and I always found the random function not very efficient (i.e the same album will play 3 times in a week while 95% of the other albums won't play once). Is there any other people with the same problem here? Is there a way around? It happens on Windows and on Ubuntu (I have a dual boot). I have less than 10000 songs on my computer (i.e less than 2000 albums). I always use the last stable version of SC...
I know "random" means I could have the same album 3 times in a row once but it happens on a regular basis so there is definitely something wrong.
Any ideas?

aevans
2008-11-25, 06:03
I second that. I use my sb3 as an alarm. With SC 7.2 the random mix didn't work so it always defaulted to the 'random 10 tunes' fallback alarm. I found this 100% random and fine. but since upgrading to 7.2.1, the alarm is now fixed but i find i'm waking up to guns and roses every morning! and i really don't have much of their music.

cjbrooker
2009-03-04, 11:40
I always found the random function not very efficient (i.e the same album will play 3 times in a week while 95% of the other albums won't play once).

I have noticed a similar thing.

I listen to playlists on random. There are 100's of songs on the playlist but I still find it can repeat songs from the same album every 15 or so songs.

For example if I hear a song from an album it's almost certain that I will hear another song from the same album in a few tracks time while other songs in the playlist are random.

In the playlist I'm listening to at the moment (2000+ songs) I can see that for the only Jimi Hendrix album I have there are 3 tracks within the space of 20 songs in the 'random' playlist. This happens consistently so isn't as random as it could be. However, I do get all of my albums eventually unlike Ranokivio.

I think it has been this way for some time. There must be a problem with the algorithm that is used for random. I know computers are not good at doing 'random' but I'm sure Squeezecenter could be improved here.

Anyone else noticed this or am I being over sensitive?

EricBergan
2009-03-04, 11:49
I have noticed a similar thing.

In the playlist I'm listening to at the moment (2000+ songs) I can see that for the only Jimi Hendrix album I have there are 3 tracks within the space of 20 songs in the 'random' playlist. This happens consistently so isn't as random as it could be. However, I do get all of my albums eventually unlike Ranokivio.

I think it has been this way for some time. There must be a problem with the algorithm that is used for random. I know computers are not good at doing 'random' but I'm sure Squeezecenter could be improved here.

Anyone else noticed this or am I being over sensitive?

+1

I have playlists with thousands of songs from almost a thousand albums, yet within the space of 50 songs, will get 2 or 3 each from a couple albums.

I've worked on random algorithms, so I do understand that only statistical analysis will really prove whether it's working or not, but my instincts say there is an error in the algorithm.

I also agree its been there for several generations of server.

eric

mrfantasy
2009-03-05, 07:58
Yeah, again, I haven't been able to prove anything statistically significant, but my intuition tells me it's not working. http://forums.slimdevices.com/showthread.php?t=59041

If you look at the Random Mix code it's relying on MySQL's "ORDER BY RANDOM" to return results, so at first blush that would be where the problem is; but there's also a lot of code in there to do the genre matching, duplicate avoidance, and other things that I can't really determine and I suspect there's a side effect coming off that.

I'm curious what the size of your music libraries are. Since it seems there's an effect that only a subset of my library is getting mixed, I wonder if there's a size factor.

EricBergan
2009-03-05, 12:08
I'm curious what the size of your music libraries are. Since it seems there's an effect that only a subset of my library is getting mixed, I wonder if there's a size factor.

543 albums, 6519 songs by 293 artists

My common playlists are ~800, ~1600, ~2500 and ~5800 songs.

eric

cjbrooker
2009-03-05, 13:17
I'm curious what the size of your music libraries are. Since it seems there's an effect that only a subset of my library is getting mixed, I wonder if there's a size factor.

4540 songs, 340 albums, 153 artists.

Playlists of between 600 and 2000 tracks.

mrfantasy
2009-03-05, 14:05
Tried something interesting: Connect to the squeezecenter MySQL instance, and run the query "SELECT * from tracks ORDER BY RAND() LIMIT 10;" and see what you get. I'm getting results that very closely mirror the strange results I get from Random Mix itself, which suggests the problem is, in fact in MySQL's RAND() function. I'm going to pursue that angle a little more.

tipsen
2009-03-05, 16:09
I'm also experiencing seemingly "not-so-random" behaviour from random albums with my squeezebox! Apparently other people have been experiencing the same behaviour when using MySQL from PHP:

http://archives.devshed.com/forums/mysql-99/making-order-by-rand-more-random-1427558.html

As far as I can tell from the MySQL manual there is no other obvious way to do this - it is the "correct" solution... :-(

/Tommy

Chop Chop
2009-03-06, 01:26
Apparently other people have been experiencing the same behaviour when using MySQL from PHP

Looking at that thread, wouldn't it be possible to use the Perl rand() function to seed the random query in MySQL, whenever a random query is done in MySQL? I know it's not elegant, but hopefully it would give a more random selection.

mrfantasy
2009-03-06, 08:26
Theoretically, using RAND() in MySQL uses a different seed every time. The issue is that it's still a pseudo-RNG (mathematical function) that has some entropic qualities but obviously isn't good enough for our purposes. Passing a different seed won't really help in this case--you'll just get a different list of specific tracks with the same overall behavior. Much of the problem, in fact, may be that we're always taking the first n tracks off the list of random tracks, which may have some self-similarities across seed values.

If you believe the Perl rand() function was better, you'd have to use it by using it to get a number from (1..numtracks) and using that track number to add a track as needed. I think the Random Mix plugin could be modified to do this. The question is whether or not the Perl rand() is any better.

My SqueezeCenter is a server running a VIA C7 processor, which has a hardware RNG. I have the Linux modules loaded, and if the Perl functions can take advantage of a hardware RNG I would get really good entropy in that case. (I have seen nothing that indicates MySQL could use it, although theoretically it could be coded in.) I could probably write a module to explicitly use the processor RNG, but it would only be useful to VIA CPU users.

cdmackay
2009-03-06, 08:33
mrfantasy wrote:
> Theoretically, using RAND() in MySQL uses a different seed every time.
> The issue is that it's still a pseudo-RNG (mathematical function) that
> has some entropic qualities but obviously isn't good enough for our
> purposes. Passing a different seed won't really help in this
> case--you'll just get a different list of specific tracks with the same
> overall behavior. Much of the problem, in fact, may be that we're
> always taking the first n tracks off the list of random tracks, which
> may have some self-similarities across seed values.

perhaps SC should randomize the list it gets back from MySQL :)

radish
2009-03-06, 09:14
I'm not sure there's anything particularly wrong with the PRNG in mysql. My stats knowledge isn't good enough to do the mathmatical proof, but I ran some simulations using a very good (i.e. very random) PRNG of a couple of the scenarios people have described in this thread. Here are some results:



In the playlist I'm listening to at the moment (2000+ songs) I can see that for the only Jimi Hendrix album I have there are 3 tracks within the space of 20 songs in the 'random' playlist. This happens consistently so isn't as random as it could be.

Assuming 2000 tracks from 200 albums, every test I ran showed multiple instances of "3 tracks from the same album within the space of 20 songs". In fact, there were usually several instance of 4 tracks from the same album within 20. Now if you're saying that the album in question is always the Jimi Hendrix one every time you reshuffle, then the likleyhood of that happening in a fair system is vanishingly small which would indeed indicate an issue.



I have playlists with thousands of songs from almost a thousand albums, yet within the space of 50 songs, will get 2 or 3 each from a couple albums.

OK, assuming 5000 tracks from 1000 albums, my simulation shows you will always get a good number of instances of 2 or 3 from the same album within 50 tracks. I only saw 1 instance of 4 from the same album within 50 though, so I wouldn't expect to see that often.

Randomness is often very surprising to humans, it's a difficult thing to have an accurate intuition about. I know, for example, that the iTunes random playlist generator is very un-random, it specifically changes the playorder to remove things which people perceive as being "wrong" - even though statistically they're correct. I believe that erland has a plugin which applies similar rules to the SC random mix - preventing multiple tracks from the same album/artist within a certain window. Worth a look if you don't like what comes out by default.

EricBergan
2009-03-06, 11:58
[QUOTE=radish;403743]OK, assuming 5000 tracks from 1000 albums, my simulation shows you will always get a good number of instances of 2 or 3 from the same album within 50 tracks. I only saw 1 instance of 4 from the same album within 50 though, so I wouldn't expect to see that often.

/QUOTE]

I've worked on pseudo random number generators, and I certainly agree that humans can definitely think they see patterns in randomness that aren't there, and that might be the case. But what I'm seeing isn't just 2 or 3 from the same album in a small space, but 2 or 3 from one album, and 2 or 3 from another, and 2 or 3 from a third, and in a small sample set, and it seems to happen pretty frequently. It's almost like its randomly picking a subset of albums, and then picking tracks from them.

Again, might statistically make sense, just has me curious...

eric

callesoroe
2009-03-08, 13:32
Hi there,

I have a squeezebox since last year and I am quite happy with it. However, I usually like to listen to my music using the random album mix and I always found the random function not very efficient (i.e the same album will play 3 times in a week while 95% of the other albums won't play once). Is there any other people with the same problem here? Is there a way around? It happens on Windows and on Ubuntu (I have a dual boot). I have less than 10000 songs on my computer (i.e less than 2000 albums). I always use the last stable version of SC...
I know "random" means I could have the same album 3 times in a row once but it happens on a regular basis so there is definitely something wrong.
Any ideas?

I have just the same problem. This thread is very interesting. Has this problem been reported as a bug??? What does the Logitech Gys say to this problem.

Almost every time I start a random mix, I always get "Robert Cray" in the random mix. Very fine music but it isn't very random. I just have one album with Robert Cray(3 cd collection)

I have at this point stored almost 1100 cd's and is still ripping....

I would really appriciate a better random mix....

cjbrooker
2009-03-11, 11:51
Randomness is often very surprising to humans, it's a difficult thing to have an accurate intuition about. I know, for example, that the iTunes random playlist generator is very un-random, it specifically changes the playorder to remove things which people perceive as being "wrong" - even though statistically they're correct. I believe that erland has a plugin which applies similar rules to the SC random mix - preventing multiple tracks from the same album/artist within a certain window. Worth a look if you don't like what comes out by default.

It would be interesting to know if it is just my perception thinking it's not 'random'.

I guess I can live with it.

Thanks for everyones input.

mrfantasy
2009-03-12, 13:32
There's two reasons I haven't called it a bug: One, I've been unable to quantify the effect in any way that rules out observer bias in a constructive way (although I see similar things to you--I don't own much Timbuk 3 but it shows up an awful lot) and two, I'm 99% sure the issue (that I can't quantify objectively) is in MySQL's "ORDER BY RAND()" using a horrible pseudo-RNG. This is evidenced by doing random queries on the database directly. As a side note, this is why some people think a "clear and rescan" of the database fixes the problem--it gives items in the database new IDs and changes the list of results returned by RAND(). But if you clear and rescan again it only changes the behavior if you've added music.

I've been hacking in my spare time, and I actually have some Perl code that can generate random numbers using Crypt::Random, which can be sourced with a hardware RNG if you have one (which I do) or with /dev/random on Linux which should still be a much better random source than the RAND function in MySQL. The only issues are actually using that random data to pull items from the list of results, the fact that I'm only testing it on Linux, and the actual issue of will it actually produce better output than what we're currently seeing. If I can get the code to work, and it produces desirable behavior, I'll provide it; and someone else can work on the cross-portability issues.



I have just the same problem. This thread is very interesting. Has this problem been reported as a bug??? What does the Logitech Gys say to this problem.

Almost every time I start a random mix, I always get "Robert Cray" in the random mix. Very fine music but it isn't very random. I just have one album with Robert Cray(3 cd collection)

I have at this point stored almost 1100 cd's and is still ripping....

I would really appriciate a better random mix....

Mario
2009-03-20, 01:35
Hi,

I also got this problem some time ago (see this thread: http://forums.slimdevices.com/showthread.php?t=43869). My wife has a 450 tracks playlist and complaints about the repeat of the same during random play.

I've look for a bug and found this: http://bugs.slimdevices.com/show_bug.cgi?id=8310.

I suggest to vote for it (as I just have done)

MrSinatra
2009-03-21, 12:27
i may be way off the mark here...

but if the SC new schema will track files and stats, things like playcount for instance, and last played, it seemes to me random mix should create "pools" and pick from those pools.

meaning that if something just played recently, it shouldn't be cued up next, or if something has played 10 times more than something else, it shouldn't be cued up next.

to me, random mix works best when everything plays once in a pass thru. but of course, such a rule means things aren't truly random. so the question then should be what is the goal? mathematical randomness? or perceived randomness? personally, i'll take the perception over the reality, esp if it means everything gets played once before it is played again.

Goodsounds
2009-03-21, 14:49
i may be way off the mark here...

but if the SC new schema will track files and stats, things like playcount for instance, and last played, it seemes to me random mix should create "pools" and pick from those pools.

meaning that if something just played recently, it shouldn't be cued up next, or if something has played 10 times more than something else, it shouldn't be cued up next.

to me, random mix works best when everything plays once in a pass thru. but of course, such a rule means things aren't truly random. so the question then should be what is the goal? mathematical randomness? or perceived randomness? personally, i'll take the perception over the reality, esp if it means everything gets played once before it is played again.

I think you answered your own question - everything once in a pass is not random. But you can do that now - choose your entire library, then shuffle the songs.

Maybe the complaint is not about "random" (it amuses me that anyone would obsess about how it is done or what the results are). Maybe people are simply saying, "what are other ways of producing an assorted list?" And perhaps your idea would be accomplished by finding a means of answering this question - "Play some songs I haven't played lately". It's almost the opposite of "favorites" (which presumably are played most frequently) but not exactly, because included in the group of songs played infrequently are ones that are disliked. I'm not a coder but I'd think some type of tracking routine that approximates this shouldn't be too hard.

Alternatively, MusicIP produces interesting playlists. Whether random or not, or how done, etc., doesn't concern me, but I do find the results interesting.

MeSue
2009-03-21, 15:14
Maybe the complaint is not about "random" (it amuses me that anyone would obsess about how it is done or what the results are). Maybe people are simply saying, "what are other ways of producing an assorted list?" And perhaps your idea would be accomplished by finding a means of answering this question - "Play some songs I haven't played lately". It's almost the opposite of "favorites" (which presumably are played most frequently) but not exactly, because included in the group of songs played infrequently are ones that are disliked. I'm not a coder but I'd think some type of tracking routine that approximates this shouldn't be too hard.
Erland's plug-ins already do this. I've never used SC's built-in random mix, but I have been very happy with the randomness of my dynamic playlists through the plug-in. I have most of mine set to not repeat a song for X number of days.

frank_from_hh
2009-04-06, 14:44
May be we just don't want true random playlists? From statistics it is known that a true random generator will always show some clustering. This is why several mathematicians have invented quasi-random distributions (e.g. Halton). Their results look random and fill space a lot more equal than true randomness does. To me this sound like what people want for playlist.

cjbrooker
2009-06-16, 08:14
May be we just don't want true random playlists? From statistics it is known that a true random generator will always show some clustering. This is why several mathematicians have invented quasi-random distributions (e.g. Halton). Their results look random and fill space a lot more equal than true randomness does. To me this sound like what people want for playlist.

That's exactly what I'd be looking for. True randomness can be a bit annoying ;-)

MrSinatra
2009-06-16, 12:06
at the station, we use a pgm called "Natural Music 5" to schedule music and so on...

it has many rules you can set up, like search depth (for picking the next song to be scheduled), artist spread, no repeats, etc...

i'm not saying its a model to follow, but a random mix that had some rules as to how to handle the pools of possible selections would be a great, and in fact, amazing feature.

i'd like to be able to set mine so that everything plays once, with maxmium artist spread, with an exception however for doubleshots. thats the kind of thing i mean when i talk about rules.

david_f1976
2009-06-17, 07:02
True randomness can be a bit annoying ;-)

I remember reading an article about all the complaints that Apple had regarding how random their random/shuffle mode was. In the end, despite proving that their shuffle mode was shuffling properly (or as is well as is possible given the constraints of seeds) they ended up changing the programming in how it works to make tracks appear more 'randomly'

I'll post the source if I can find it

raaurora
2009-06-24, 16:26
Hi there,

I have a squeezebox since last year and I am quite happy with it. However, I usually like to listen to my music using the random album mix and I always found the random function not very efficient (i.e the same album will play 3 times in a week while 95% of the other albums won't play once). Is there any other people with the same problem here? Is there a way around? It happens on Windows and on Ubuntu (I have a dual boot). I have less than 10000 songs on my computer (i.e less than 2000 albums). I always use the last stable version of SC...
I know "random" means I could have the same album 3 times in a row once but it happens on a regular basis so there is definitely something wrong.
Any ideas?

I think the problem is the "randomizing" algorithm is somewhat deterministic. Unless something in your library changes, the "random" playlist created is either similar or exactly the same. Before purchasing my duet a year ago, I had an earlier generation product from Netgear (mp101). It had the exact same issue.

Contrast that with my iPod nano. Every time it cycles through my library of music in random mode, it reshuffles a new random playlist which is completely different than the one that just wrapped up.

azadian
2010-04-24, 10:10
I'm finding that the random song mix isn't even close to random. To understand how I can say that, you have to know the structure of my music folder. I have 178 top-level directories, most of which correspond to albums (10-20 songs), some of which correspond to sets of albums (3-8 subdirectories), and one, called singles, which has 261 songs.

When I do random song mix, I estimate that at least half of the songs come from that singles directory. Considering that the singles directory has 261 songs, and in total I have about 4000, it is clear that this random is at least highly peculiar, if not completely broken.

I suspect this is related to the fact that the library statistics shows that I have a total of 319 tracks!

MrSinatra
2010-04-24, 10:41
while i agree with your conclusion, i suspect folder structure has little to nothing to do with the playlist random mix generates. i think it has more to do with artist tags, but its just speculation on my part.

snarlydwarf
2010-04-24, 10:59
I suspect this is related to the fact that the library statistics shows that I have a total of 319 tracks!

I agree. You have something amiss.

You don't say if your scan shows all the tracks/albums... does it?

If it aborted after completing 'singles' but before completing the full scan, well, then, um, that's exactly why: it hasn't scanned your whole library.

Looked at your logs?

EricBergan
2010-04-24, 12:49
while i agree with your conclusion, i suspect folder structure has little to nothing to do with the playlist random mix generates. i think it has more to do with artist tags, but its just speculation on my part.

Since my folder structure is by artist then album, can't really say for sure, but I agree that the random problem seems to be by artist, not album.

MrSinatra
2010-04-24, 17:14
probably some others too:

http://bugs.slimdevices.com/show_bug.cgi?id=6068

http://bugs.slimdevices.com/show_bug.cgi?id=8310

RichieB
2010-06-14, 11:26
I've always found the SBS random lacking some randomness, but when I switched from a Linux PC to a QNAP NAS to run SBS, it became really annoying. I tried the SQL playlist plugin, but it has the same issues. So I am now using the attached randomize perl script. I use it this way from cron:

find /path/to/mp3s -name \*.mp3 | randomize.pl -l 500 -n 10 -d /path/to/sbs/playlists/random -s .m3u
This creates 10 random playlists of 500 songs each every night. I get much better results this way. You just need to remember to start a new random playlist before you reach the 500 song limit.

erland
2010-06-14, 12:42
I've always found the SBS random lacking some randomness, but when I switched from a Linux PC to a QNAP NAS to run SBS, it became really annoying. I tried the SQL playlist plugin, but it has the same issues.

What's the problem with SBS and SQL Playlist ? Are tracks from the same artist played after each other ?

If that's the case, the Custom Skip plugin might be the solution to your problems. Custom Skip used together with Dynamic Playlist/SQL Playlist plugin will make it possible to setup a SQL Playlist where Custom Skip filter makes sure it doesn't play the same artist unless 50 other artists has been played first.

RichieB
2010-06-14, 13:18
What's the problem with SBS and SQL Playlist ? Are tracks from the same artist played after each other ?

Tracks from the same album even. 10 tracks from the same album within 400 "random" songs is not very random. And the same albums over and over, each time I play a "only songs in playlist + random" SQL playlist. The base (m3u) playlist I'm using has 11,000 tracks, so there is really no reason to only pick a few albums from it.

A side effect of using SQL Playlist this way is that that mysqld goes to 100% cpu usage when new tracks are added. Since I run SBS on an Intel Atom, that can last for minutes, making SBS totally unresponsive. I understand SQL Playlist has to work harder than a simple Random Mix, but with my hardware it's not very usable.



If that's the case, the Custom Skip plugin might be the solution to your problems.

I just tried Custom Skip, and it looks like a work around more than a solution. It is now adding 6 tracks at a time, instead of 10. I suppose this means 4 tracks got skipped. Nice, but now the 100% cpu issue occurs even more often.

Thanks for these plugins though, they definitely serve a purpose.

tonyptony
2010-07-03, 19:07
I've complained about the SBS Randomizer in the past also. I like Erland's plugins but agree that it's more of a workaround than an alternative. Can someone just make a "pseudo-random" plugin that does the sort of thing it seems a bunch of us are talking about? Take everything from either the selected genres, decades, artists, or all, randomize by song, artist, or album, and then run it through the "pseudo" part to remove or reorder based on closeness of the non-primary factors. For example, if the randomization is by song, then any closeness by either artist or album would trigger the "pseudo" part to occur.

Yeah, I know it doesn't sound that simple, but with all the talent on this board somebody should be able to put something together.

erland
2010-07-03, 22:21
I've complained about the SBS Randomizer in the past also. I like Erland's plugins but agree that it's more of a workaround than an alternative. Can someone just make a "pseudo-random" plugin that does the sort of thing it seems a bunch of us are talking about? Take everything from either the selected genres, decades, artists, or all, randomize by song, artist, or album, and then run it through the "pseudo" part to remove or reorder based on closeness of the non-primary factors. For example, if the randomization is by song, then any closeness by either artist or album would trigger the "pseudo" part to occur.

Yeah, I know it doesn't sound that simple, but with all the talent on this board somebody should be able to put something together.

Just to understand the problem a bit better. What is the difference with what you are suggesting and SQL Playlist+Dynamic Playlist+Custom Skip ?

Do you want a long playlist instead off one that adds the tracks bit by bit or is there something else ?
I assume that you know that the performance of SBS gets bad with long playlists ?

Custom Skip can make sure the same artist isn't repeated during the next x tracks

tonyptony
2010-07-04, 05:30
Just to understand the problem a bit better. What is the difference with what you are suggesting and SQL Playlist+Dynamic Playlist+Custom Skip ?

Not much? :-) Like I said, your plugins do a great job, so maybe it's just a question of an integrated user interface or something like that. Say, didn't you used to have a Random Playlist plugin a while back? Did SQLP+DP+CS replace that?

On looking at it further, I think it may really just be a question of a single interface for this. For example, if I go into SQL Playlist and select Random Songs in Genre, after I select the desired Genres and the exclusion time for repeats, it seems hitting the Play button then sends the user to Dynamic Playlists. What operations are required here that couldn't be integrated into the previous panel under SQL?

JJZolx
2010-07-04, 11:04
I've complained about the SBS Randomizer in the past also. I like Erland's plugins but agree that it's more of a workaround than an alternative. Can someone just make a "pseudo-random" plugin that does the sort of thing it seems a bunch of us are talking about? Take everything from either the selected genres, decades, artists, or all, randomize by song, artist, or album, and

It's the 'take everything' part that is the pisser. For a genre mix that results in a large number of tracks this can be extremely memory and processor intensive. It's not something that is going to run well, particularly on an NAS or other low powered server.

This is a big reason why the current random mix only pulls a fairly small number of tracks to add to the current playlist and then adds one each time one is played.


then run it through the "pseudo" part to remove or reorder based on closeness of the non-primary factors. For example, if the randomization is by song, then any closeness by either artist or album would trigger the "pseudo" part to occur.

Think about what happens when the genre contains a relatively small number of tracks/albums/artists. The task becomes impossible. As the number of tracks/artists/albums increases it begins to become possible, but can extremely time and processor consuming to find some optimal (non)randomness that satisfies your criteria. Only with a large number of tracks from a mix of artists and albums can it be accomplished satisfactorily.


Yeah, I know it doesn't sound that simple

No, it's not at all simple.

tostka
2012-03-19, 21:00
I've owned either a SliMp3 or a Squeezebox since 2002.
But I've spent the last few months since upgrading from SqueezeServer 7.3 to Logitech Media Server 7.7.1 (BIG MISTAKE), listening to my wife continually complain about the lack of functional randomization from playlists. Every time a we try to randomize a large multi-thousand track genre we end up with what seems to be variations of a very limited number of artists from the main list.

So today I finally snapped, dropped the necessary $50 (to buy _once_, and not see nags or yearly charges - not an option) and bought a license for everything, 'forever' to get SQL Playlist+Dynamic Playlist+Custom Skip.

After a few hours of listening to a Random Songs on genre X pass, randomization is finally functional.
But I resent the hell out of the kind of money we're spending on this hardware, to have to drop another $50 for 3rd-party solutions to get a core feature like track randomization to actually deliver a *randomized* list.
I'm not running on a NAS, I'm running on 4gb of ram with a decent cpu. I have the horsepower to load up a big playlist, and it's a waste when the system tries to optimize playlist size for lowest common denominator.

StridingEdge
2012-10-25, 15:15
I have found that the lack of randomness is more restrictive than other people have reported. I have one playlist that I have imported from Media Jukebox. It comprises of around 2100 tracks. When I play that "randomly" via the Squeezebox it only ever populates the playing list, which is restricted to 500 items (does it always limit a list to 500?), from the first few letters of the alphabet. So, instead of getting a random selection from artists A-Z, I actually get a randomly played selection from artists A-D. However many times I select this playlist, it only ever behaves in this manner - I will never get any music from artists beginning with the letter F onwards - NEVER!

I have not had the device for long enough to know whether this is just an issue with larger playlists but logic suggests so.

Of course there is a way around it, from the listener's perspective. I randomise the playlist within Media Jukebox and then export that as a playlist. I can then just play that list straight from the Squeezebox playlist menu. Each time I want a different order, I can re-jig it on Media Jukebox before exporting the list and then re-scan the playlists; or simply start the playlist from mid-way through. It's not that big a deal to get the required results. As a work-around it's fine but does rather highlight the Squeezebox's inadequacies in this regard.

kidstypike
2012-10-25, 15:29
I have found that the lack of randomness is more restrictive than other people have reported. I have one playlist that I have imported from Media Jukebox. It comprises of around 2100 tracks. When I play that "randomly" via the Squeezebox it only ever populates the playing list, which is restricted to 500 items (does it always limit a list to 500?), from the first few letters of the alphabet. So, instead of getting a random selection from artists A-Z, I actually get a randomly played selection from artists A-D. However many times I select this playlist, it only ever behaves in this manner - I will never get any music from artists beginning with the letter F onwards - NEVER!

I have not had the device for long enough to know whether this is just an issue with larger playlists but logic suggests so.

Of course there is a way around it, from the listener's perspective. I randomise the playlist within Media Jukebox and then export that as a playlist. I can then just play that list straight from the Squeezebox playlist menu. Each time I want a different order, I can re-jig it on Media Jukebox before exporting the list and then re-scan the playlists; or simply start the playlist from mid-way through. It's not that big a deal to get the required results. As a work-around it's fine but does rather highlight the Squeezebox's inadequacies in this regard.

Settings > Advanced > Performance > Maximum Playlist Length.

StridingEdge
2012-10-25, 15:45
Yes, thanks Kidsty, that works now I have upped the limit. However, it does not explain the odd behaviour within the 500 song restriction.

erland
2012-10-25, 17:03
I have found that the lack of randomness is more restrictive than other people have reported. I have one playlist that I have imported from Media Jukebox. It comprises of around 2100 tracks. When I play that "randomly" via the Squeezebox it only ever populates the playing list, which is restricted to 500 items (does it always limit a list to 500?), from the first few letters of the alphabet. So, instead of getting a random selection from artists A-Z, I actually get a randomly played selection from artists A-D. However many times I select this playlist, it only ever behaves in this manner - I will never get any music from artists beginning with the letter F onwards - NEVER!

I have not had the device for long enough to know whether this is just an issue with larger playlists but logic suggests so.

Of course there is a way around it, from the listener's perspective. I randomise the playlist within Media Jukebox and then export that as a playlist. I can then just play that list straight from the Squeezebox playlist menu. Each time I want a different order, I can re-jig it on Media Jukebox before exporting the list and then re-scan the playlists; or simply start the playlist from mid-way through. It's not that big a deal to get the required results. As a work-around it's fine but does rather highlight the Squeezebox's inadequacies in this regard.

There are two solutions for this in addition to the one you mention:

- Change the maximum playlist length in LMS Settings/Advanced/Performance settings section. If you have a reasonably fast computer running LMS this might work good enough, with a slower computer large playlists are going to degrade the performance a bit.

or

- Use the third party Dynamic Playlist plugin, which allows you to play the playlist automatically bit by bit from My Music/Dynamic Playlists menu, there is an option in Dynamic Playlist settings page to play the playlist in random order.

RichieB
2013-10-28, 13:34
I think I've found a solution to this "not so random" problem. I was playing around with /dev/random on my NAS (QNAP TS-459 Pro+), and noticed /proc/sys/kernel/random/entropy_avail was always below 200 bits. Then I stumbled upon http://www.issihosts.com/haveged/ which is a way to increase randomness for Linux kernels. After installing and running this daemon, entropy_avail is always 1024 bits or more. My random mixes now show songs I haven't heard in ages, and I do not see tracks from those ever re-occurring albums any more! So I think this problem (for me at least) had to do with not enough randomness being available on my system.

I've made the haveged software into a ipkg which makes it easy to install if you have a x86/i686 based NAS like me: http://goo.gl/lT0lw4

callesoroe
2013-10-29, 07:55
I think I've found a solution to this "not so random" problem. I was playing around with /dev/random on my NAS (QNAP TS-459 Pro+), and noticed /proc/sys/kernel/random/entropy_avail was always below 200 bits. Then I stumbled upon http://www.issihosts/haveged/ which is a way to increase randomness for Linux kernels. After installing and running this daemon, entropy_avail is always 1024 bits or more. My random mixes now show songs I haven't heard in ages, and I do not see tracks from those ever re-occurring albums any more! So I think this problem (for me at least) had to do with not enough randomness being available on my system.

I've made the haveged software into a ipkg which makes it easy to install if you have a x86/i686 based NAS like me: http://goo.gl/lT0lw4

Can you get this to Work on a ReadyNas Ultra II ??

RichieB
2013-10-29, 09:47
Can you get this to Work on a ReadyNas Ultra II ??
Google tells me the ReadyNas Ultra II has an Intel CPU but I can't find if it supports Optware==ipkg. If it doesn't you can always install gcc and compile it yourself. It was not that hard.

callesoroe
2013-10-29, 23:45
Google tells me the ReadyNas Ultra II has an Intel CPU but I can't find if it supports Optware==ipkg. If it doesn't you can always install gcc and compile it yourself. It was not that hard.

It seems like it does not support ipkg . How du I compile ???? Your link does not work.... ?

steve-g
2013-10-30, 02:51
The correct address is http://www.issihosts.com/haveged/

Be interested in how you get on

callesoroe
2013-10-30, 12:42
The correct address is http://www.issihosts.com/haveged/

Be interested in how you get on

This is quite interesting. But what I don't understand is, that if this is a well known problem, why is the solution not implemented in the operating systems kernel software ????
Unfortunately I can not find any way to get this working on a ReadyNas Ultra. I really am no expert in Linux so hopefully someone else can get this to work. Because it is a fact that
"Random is not so random in LMS"

RichieB
2013-10-30, 15:33
This is quite interesting. But what I don't understand is, that if this is a well known problem, why is the solution not implemented in the operating systems kernel software ????

Well, there is a solution in the Linux kernel: a PRNG (pseudo random number generator). It uses keyboard activity, disk I/O operations, and hardware interrupts as input. The problem is, on a headless system (no keyboard or mouse) the kernel PRNG really doesn't has much input to work with. This results in not enough entropy and repeating (close) tracks by LMS. Haveged (another RNG) uses the timing between system calls in the CPU cache. This simply is a much better choice for systems without a keyboard and heavy I/O usage. Why doesn't the kernel replace it's PRNG with the haveged algorithm? Maybe because the problem is limited to a small number of cheap boxes without RNG on their CPU? (Modern desktop/server class CPUs often have a builtin RNG.) See http://www.issihosts.com/haveged/history.html

Mnyb
2013-10-30, 19:19
Well, there is a solution in the Linux kernel: a PRNG (pseudo random number generator). It uses keyboard activity, disk I/O operations, and hardware interrupts as input. The problem is, on a headless system (no keyboard or mouse) the kernel PRNG really doesn't has much input to work with. This results in not enough entropy and repeating (close) tracks by LMS. Haveged (another RNG) uses the timing between system calls in the CPU cache. This simply is a much better choice for systems without a keyboard and heavy I/O usage. Why doesn't the kernel replace it's PRNG with the haveged algorithm? Maybe because the problem is limited to a small number of cheap boxes without RNG on their CPU? (Modern desktop/server class CPUs often have a builtin RNG.) See http://www.issihosts.com/haveged/history.html

And how many user actually expects their NAS to pull up random numbers series ? That's was not a typical use case , especially with older NAS boxes ( NAS is not A Server )you where supposed to store files on it . but the recent generation has got better CPU and memory and actually becomes more suited to multipurpose media server activity .

Other than music servers there is cryptography and statistical dependent science that need randomness . The scientist may actually opt for the hardware solution with a radioactive source ( DIY version is smoke detector ) all computer randomness is pseudo randomness that is simply not good enough for some science they need real random , but if the pseudo randomness is ok it should be good enough for our music server .

Anyone knows if a typical intell CPU has a good built in RNG ? Otherwise I'm interested in this too , I have not experienced to much problem with this with my headless server , but it could explain some MIP anomalies ?

callesoroe
2013-10-31, 02:04
This is quite interesting. But what I don't understand is, that if this is a well known problem, why is the solution not implemented in the operating systems kernel software ????
Unfortunately I can not find any way to get this working on a ReadyNas Ultra. I really am no expert in Linux so hopefully someone else can get this to work. Because it is a fact that
"Random is not so random in LMS"

I posted in the ReadyNas forum and got a solution. It really seems to work....... :)

https://www.readynas.com/forum/viewtopic.php?f=20&t=73695&p=409561#p409561

RichieB
2013-10-31, 03:59
Anyone knows if a typical intell CPU has a good built in RNG ?
My NAS has an Intel Atom D525. This line of CPUs does not seems to have a hardware RNG. Even if it did, you would need to run a separate rngd process to make use of it.

@callesoroe: awesome that you got it to work on your ReadyNAS. apt-get is much easier than ipkg, there is a lot more software available for it.

slartibartfast
2013-10-31, 07:12
Would haveged be needed on a squeezeplug?

RichieB
2013-10-31, 07:18
It seems that at least some Raspberry Pi's have a hardware RNG: http://vk5tu.livejournal.com/43059.html
SqueezePlug should include the bcm2708_rng kernel module and rng-tools package. If it doesn't, ask the maintainer to add it.

jimbobvfr400
2013-10-31, 07:25
Have any of you guys tried Michael Hergers smartmix plugin. I haven't touched the built in random function since I did. I've not tried it but I wonder if it's possible to create a smartmix recipe that is wide enough to cover all of your local music rather than being more mood or style based. TBH I prefer random play that at least flows nicely rather than completely random (Slayer followed by Kylie doesn't work for me, although Kylie guesting on a Slayer track could be interesting :-) )

Sent from my Nexus 7 using Tapatalk 4

slartibartfast
2013-10-31, 07:26
It seems that at least some Raspberry Pi's have a hardware RNG: http://vk5tu.livejournal.com/43059.html
SqueezePlug should include the bcm2708_rng kernel module and rng-tools package. If it doesn't, ask the maintainer to add it.

My squeezeplug is on a Goflex Net. So assuming it includes rng-tools are you saying it doesn't require haveged?

Mnyb
2013-10-31, 07:27
Have any of you guys tried Michael Hergers smartmix plugin. I haven't touched the built in random function since I did. I've not tried it but I wonder if it's possible to create a smartmix recipe that is wide enough to cover all of your local music rather than being more mood or style based. TBH I prefer random play that at least flows nicely rather than completely random (Slayer followed by Kylie doesn't work for me, although Kylie guesting on a Slayer track could be interesting :-) )

Sent from my Nexus 7 using Tapatalk 4

yes I use that all the time especially the way it adds files after your playlist or album ended .

RichieB
2013-10-31, 07:42
My squeezeplug is on a Goflex Net. So assuming it includes rng-tools are you saying it doesn't require haveged?
http://archlinuxarm.org/platforms/armv5/seagate-goflex-net says:

The GoFlex Net is a device based on the Marvell Sheeva plug computer platform
Wikipedia says the Sheeva plug is based on the Marvell Kirkwood 6281 SoC. I don't think this SoC has a hardware RNG.
Keep in mind that the rng-tools are just a bridge between the hardware RNG and /dev/random. So if your system has a /dev/hwrng device and rngd is configured correctly[1] and running, you do not need haveged.

[1]: some people run rngd with /dev/urandom as input, this is silly because /dev/urandom does not have more entropy than /dev/random, it only does not block. Running rngd this way simply turns /dev/random into /dev/urandom.

RichieB
2013-10-31, 07:48
Have any of you guys tried Michael Hergers smartmix plugin.
I haven't used this plugin, but if it uses randomness of the local LMS server to do (some part of) it might suffer from the same problems reported in this thread.

aubuti
2013-10-31, 08:29
Have any of you guys tried Michael Hergers smartmix plugin. I haven't touched the built in random function since I did. I've not tried it but I wonder if it's possible to create a smartmix recipe that is wide enough to cover all of your local music rather than being more mood or style based. TBH I prefer random play that at least flows nicely rather than completely random (Slayer followed by Kylie doesn't work for me, although Kylie guesting on a Slayer track could be interesting :-) )
Sorry if I'm being dense, but how does one achieve random play that "flows nicely" but "covers all of your local music" and is "not mood or style based"? It sounds to me as if some of those desires are at odds with each other. But maybe I'm just being thick.

One thing that may come close to what I think you are asking for is Spicefly Sugar Cube with MusicIP. It is mood/style based, but only on the most recent track(s). So instead of generating an entire playlist from a single seed (track/album/artist/genre), it is constantly evolving. It's a form of semi-free association that can easily end up with a playlist that has something in track #25 that you never would have guessed from track #1, but actually makes sense if you follow the path from track to track.

I need to play more with Michael's SmartMix plugin. I haven't been that happy with the results I've had so far, but I think the plugin is very capable and I just need to learn how to use it better.

slartibartfast
2013-10-31, 11:18
http://archlinuxarm.org/platforms/armv5/seagate-goflex-net says:

Wikipedia says the Sheeva plug is based on the Marvell Kirkwood 6281 SoC. I don't think this SoC has a hardware RNG.
Keep in mind that the rng-tools are just a bridge between the hardware RNG and /dev/random. So if your system has a /dev/hwrng device and rngd is configured correctly[1] and running, you do not need haveged.

[1]: some people run rngd with /dev/urandom as input, this is silly because /dev/urandom does not have more entropy than /dev/random, it only does not block. Running rngd this way simply turns /dev/random into /dev/urandom.

I checked my available entropy using "# cat /proc/sys/kernel/random/entropy_avail" and found it to be 175. I installed Haveged as described here http://kevin.deldycke.com/2011/08/entropy-debian-squeeze-server/ and now my entropy stays between 1000 and 4000. I had never heard of entropy in this context until I saw this thread but I assume this will have a positive effect.

My version is 1.1 but the latest version is 1.7c which according to http://packages.debian.org/search?suite=jessie&searchon=names&keywords=haveged
is only available for debian jessie & sid and not for debian squeeze. Does that make sense?