Home of the Squeezebox™ & Transporter® network music players.
Page 5 of 6 FirstFirst ... 3456 LastLast
Results 41 to 50 of 52

Thread: Genpuid advice

  1. #41
    Senior Member cparker's Avatar
    Join Date
    Oct 2005
    Location
    Hampshire, United Kingdom
    Posts
    528

    Mysterious MIP

    I couldnt sleep lastnight so spent an hour researching a bit.

    Found this for interest; http://whicken.blogspot.com/ start sending Christmas cards

    You can download the musicbrainz database for free and it has PUIDs in it. These are just the unique IDs though and NOT the analysis.

    I took La Femme d'argent by Air as a test example;

    http://musicbrainz.org/recording/864...0-ad986e646c67

    It has heaps of PUIDs for all the various cuts of it. So you wouldnt necessarily be able to use just a PUID as a match? I had a look inside my version and couldnt see any obvious link up to the PUIDs on the site to the tag in my file, I'm guessing its SHA encrypted? (Guess libofa would answer this?)

    Maybe simpler to match up AcoustIDs or MusicBrainz ID (MBID) to try and find a unique match, and then use that to match up and drop the "MUSICMAGIC DATA" and "MUSICMAGIC FINGERPRINT" tag back to a requesting client?

    I'll try Picard and see what IDs it puts for the track and can we compare and see if there is any match at all


    La Femme d'argent by Air
    ========================
    For this track my MUSICMAGIC DATA tag is;
    AgEARzl3qPyvRbEGgVjkHQAwgAGMk4c4AJ6VKk94s9TMs8HVsg faM6xx9fuxYohai1SAAdecxj0qlsmqgGDgkvQsDDirNspUgAKA AZ44rsbBBIOr6M4v/ZjyiNOB0PPdlfpRk6aw1omqbog/7LCsS79GtyCSQ4NigAHeX9eIFxrk54Dy1OMBWfe8owDJTIF/gJ8=

    MUSICMAGIC FINGERPRINT is bigger than the data though, I was expecting this to be the other way around, unless the fingerprint is actually the analysis and the data is the fingerprint!;
    AQmjLVE3XUM9PIAfVhQCDecOjQy7BwgS+wbdDukJpwMrAqgHiQ JRBdIIrgOnBQsEmwK+Aa0A8AFHAQMA4QDcAM0AlACWAK0AjADL AJUAfABt+MPbt2jVx7j2Jf4uBLEB4glLDDECgAbyAnHs1wDI/xP+8P6nAIgCuwYoAgMAaP2N/dP/0v/wACwAeQB0/+UADQASABQAEQAGAA0AAQAFABAMGVOCG2MOYKnx6CwFTABr/E8BTf5d+h3/ZfO9AI//Bv/A/Aj/TABc+9L/JP4twLX/NwDc/9L/0fwO/9HwPAAR//0ADQAF/wP//f/4//T/wP866jX6iwQM2Kj93QMT+zMI2w1RBuZamwrfRZERHQVwA0wEyQ AwwMELwARzDhgIRQIRAUEAcgBlAG8AiwBJADgAJQA6AEAAIwBP ADUAJwAsBes0GwekgA0mZ/fCAjn5euRy1ory9gEGAZxD7AODAQQAK/T1wKD4VPcK/Pb8JP8SwNf+ZwAP/wD/F/7c/zz/Ov+B/2P/QP9y/zD/X/93/477Q9aZGjkshuYu9bT8hf+l99T3jgPStF4AA0PXFy4DVQAQ9uv/TP0Z+QMD8AFdAjj/z/9lACPwHv+zAGf/7P/8//3/4//f//MAAP/3/wD/5wVHFRXpzNc0wOAT0w91AI0p5VZWGMnYugY3GtAG1ANqA2UXCA NDDX4HSQXcB78FXAL1Ab4BBgFzAUEBKQD1ASYA0QDTAQ4AvwEV AMMArgCiJiQfGA==
    Last edited by cparker; 2011-11-11 at 02:56.
    www.spicefly.com - ** Spicefly SugarCube ** - A hassle free acoustic journey through your music library using MusicIP. Plus the finest MusicIP installation guides, enhanced MIP Interface and SpyGlass MIP the Windows Automated MusicIP Headless Installer.

  2. #42
    Senior Member audiomuze's Avatar
    Join Date
    Oct 2009
    Posts
    884
    As I understand it the PUID only had relevance to enable matching of a song to musicbrainz metadata. Where the finrgerprinting side of things is concerned we should be able to use libofa to generate the fingerprint - which I'm guessing will be the same as the fingerprint already embedded in my tracks.

    One way to check would be to run a few tracks (that have previously had analysis archived to tag) through libofa and then to compare the resulting fingerprint. If they're the same we're in business and what we'd proposed about a open musicdns database would work. We'd have to have a toolset for:

    - community db to store and host fingerprint and analysis data
    - trawling an existing collection and picking up the fingerprint and analysis tags for submission to the community db
    - calculating fingerprinting new tracks and checking against the community db - if there's a match, return the analysis tag and write it to the file or to a text file for the user to import via a tagger
    - analysis (genpuid and/or musicip mixer) to be used in the event there's no match in the community db.

    libofa seems to contain everything needed to do the fingerprint generation - see developer's guide.
    digital groove | Vivere DAC MKI | ATC SCA2 | ATC SCM100ASLT

    Linux finally gets a great audio tagger: puddletag - now packaged in most Linux distributions.

  3. #43
    Member B1tbull's Avatar
    Join Date
    May 2009
    Location
    Belgium
    Posts
    93

    Question PUID : is it needed for music mixes

    Greetings,

    First of all , many thanks to CParker for the wonderful Sugarcube plugin.

    Two questions make me worry a bit for the future of MusicIP as a way of generating those mixes.

    1. Is the PUID tag a requirement for generating music mixes ?

    While most of my FLAC music files got this PUID from running the command line genpuid,
    I noticed that recently genpuid was quite randomly missing the generation of PUID's.
    Other "magic" numbers ( as described in the first post ) are still being generated.
    I assume the PUID is only used to retrieve album/track info from music sources like MusicBrainz - is this correct ?

    2. If the backend server(s) ever disappear, will genpuid still be able to generate those other "magic" numbers ?

    Thanks in advance for shining some light on this.

  4. #44
    Senior Member audiomuze's Avatar
    Join Date
    Oct 2009
    Posts
    884
    Which part of he previous posts didn't answer your concerns?
    digital groove | Vivere DAC MKI | ATC SCA2 | ATC SCM100ASLT

    Linux finally gets a great audio tagger: puddletag - now packaged in most Linux distributions.

  5. #45
    Good stuff guys. I like the idea of leveraging the existing MusicIP "stuff" as much as possible, but at the same time any future plans should involve AcoustID (since it's open-source we won't ever have to go through this kind of situation again).

    In the meantime I am still genpuid'ing away. I am running it in large batches, prior to writing the rest of my track tags, since I don't fully trust genpuid not to mangle existing tags (probably being too paranoid, but I remember reading something or other about some version of MusicIP doing that). What I have found is that if I run genpuid against a single album, it usually finds PUIDs for some number of tracks (oftentimes 1 or 2, but sometimes even more). Whereas if I run it with the -r option against a folder with lots of albums it ends up doing the full analysis on almost every track. Maybe 1 or 2 tracks out of the entire group of 40-100 albums will have a PUID tag.

    Makes me wonder if their server is being DDOS'ed (or otherwise overloaded) and they just haven't noticed or don't care. Depending on how genpuid submits fingerprints for PUID-lookup (looks like it's in batches) it seems to get fouled up and just ends up performing local analysis.

    Which is all fine and dandy since I have a fast quad-core processor and lots of time. Just interesting.

    And if someone ever sets up a third party database I'd be happy to submit data for all my albums (over 700 at this point). I'll keep an eye on this thread. Thanks all.

  6. #46
    Senior Member audiomuze's Avatar
    Join Date
    Oct 2009
    Posts
    884
    Whether we like it or not, the musicdns servers should for all intents and purposes be treated as defunct, which means genpuid is also defunct unless we find a way to spoof the musicdns server by returning a value that genpuid interperets as "the track doesn't exist on musicdns, analyse it locally". PUIDs are completely irrelevant where fingerprinting, analysis and playlisting are concerned, there is no need to discuss them further in the context of the purpose of this thread.
    digital groove | Vivere DAC MKI | ATC SCA2 | ATC SCM100ASLT

    Linux finally gets a great audio tagger: puddletag - now packaged in most Linux distributions.

  7. #47
    Quote Originally Posted by audiomuze View Post
    Whether we like it or not, the musicdns servers should for all intents and purposes be treated as defunct, which means genpuid is also defunct unless we find a way to spoof the musicdns server by returning a value that genpuid interperets as "the track doesn't exist on musicdns, analyse it locally". PUIDs are completely irrelevant where fingerprinting, analysis and playlisting are concerned, there is no need to discuss them further in the context of the purpose of this thread.
    Why do you say genpuid is defunct? It's performing analysis for me just fine. The only time it didn't was when I manually redirected its DNS name to localhost in my hosts file.

    I say as long as their server is still in existence (which it most certainly is) there's no reason not to use genpuid. Doesn't mean that people shouldn't plan for the day when genpuid stops working, but as of this very minute genpuid works just fine (for local analysis). One huge benefit (for me anyway) is that genpuid is easily multi-threaded. I don't know if the same is possible with the MusicIP Mixer app.

    And I agree that PUIDs are meaningless going forward, since their only benefit was to assist in getting previously run analysis blobs and for matching metadata via MusicBrainz. I only commented on it in the context of trying to figure out what's going on with the MusicIP server. Because for all intents and purposes if a track ends up with a PUID tag, its MusicDNS Fingerprint data was successfully matched on the MusicIP server and therefore its MusicDNS Analysis tag was downloaded and not locally generated. Genpuid is still fetching PUIDs for some tracks, just not very many. I'd be curious to know if people who are using MusicIP Mixer for analysis instead are seeing the same.

  8. #48
    Senior Member audiomuze's Avatar
    Join Date
    Oct 2009
    Posts
    884
    You are wrong re PUID's having anything to do with retrieving analysis metadata - this is done by fingerprinting a track and then seeking the fingerprint in the musicdns database - if a match is found the metadata is returned, if not genpuid does analysis locally.

    I've tried on multiple occasions now to use genpuid on a directory tree of albums and it takes days only to achieve little to nothing. Whilst it would appear to be analyzing, it never ends and CPU usage shows it's doing little to no analysis.
    digital groove | Vivere DAC MKI | ATC SCA2 | ATC SCM100ASLT

    Linux finally gets a great audio tagger: puddletag - now packaged in most Linux distributions.

  9. #49
    Quote Originally Posted by audiomuze View Post
    You are wrong re PUID's having anything to do with retrieving analysis metadata - this is done by fingerprinting a track and then seeking the fingerprint in the musicdns database - if a match is found the metadata is returned, if not genpuid does analysis locally.
    What I mean is that seeing a PUID tag added is evidence that genpuid successfully retrieved matched your locally generated fingerprint and retrieved (versus locally generated) the track's analysis tag. I know it's not used to do anything meaningful, but aside from picking through log files, looking to see if a PUID tag was added tells you that the track's analysis was downloaded. That's all.

    I've tried on multiple occasions now to use genpuid on a directory tree of albums and it takes days only to achieve little to nothing. Whilst it would appear to be analyzing, it never ends and CPU usage shows it's doing little to no analysis.
    That's because you are using the linux and/or MacOS binaries of genpuid. If you check my first post in this thread I reported the same issue. Genpuid on Windows works fine, however. May not be an option for you but it is for others who have a Windows machine at their disposal.

    Multithreading is a huge time saver--I have a quad core Xeon chewing through 4 separate instances of genpuid simultaneously.

  10. #50
    Quote Originally Posted by bookemdano View Post
    What I mean is that seeing a PUID tag added is evidence that genpuid successfully retrieved matched your locally generated fingerprint and retrieved (versus locally generated) the track's analysis tag. I know it's not used to do anything meaningful, but aside from picking through log files, looking to see if a PUID tag was added tells you that the track's analysis was downloaded. That's all.



    That's because you are using the linux and/or MacOS binaries of genpuid. If you check my first post in this thread I reported the same issue. Genpuid on Windows works fine, however. May not be an option for you but it is for others who have a Windows machine at their disposal.

    Multithreading is a huge time saver--I have a quad core Xeon chewing through 4 separate instances of genpuid simultaneously.
    I've just started with SugarCube and am going through the process to analyze my library. I've read through many threads and FAQs and I'm still confused about how all this works... ;-)

    I'm on F17 and as far as I can tell genpuid, mmserver and mmmixer all work, but there seems to be some incompatibility between genpuid and mmserver/mmmixer. If I generate a cache file with mmserver/mmmixer it can be read by genpuid for the validation process. After genpuid does the validation the revised cache file is not recognized by mmserver/mmmixer. It gives me a message saying that the cache file was created by a newer release of musicip. From that I gather that mmserver/mmmixer (version 1.8) is somehow older than genpuid (version 1.4).

    If I try to run genpuid without first creating the cache file from mmserver/mmmixer it does work, but if you have a really large library forget it. That is probably the behavior some of you have been reporting in this thread. I tested this theory out by running against just one album versus my entire collection. So, for what ever reason, it appears if you want to run genpuid against a large collection the best way to use genpuid is to first create the cache file (without doing the validation process) with mmserver/mmmixer. Keep in mind though that once the validation is done by genpuid it doesn't appear you can use that cache file with mmserver/mmmixer. The workaround I'm guessing is to use the -archive function to write the analysis/fingerprint to the music file tags. I did another small test and mmserver/mmmixer do recognize the tags which are created by genpuid. You can then just create a new cache file from the files processed by genpuid with mmserver/mmmixer.

    What I found that seems to work the best for me is to run two instances of genpuid against subsets of my collection and store the tags to the files. For this initial run through, I'm not keeping a cache file. I'll generate that once everything is tagged. The command I'm using is: ./genpuid dns_key -r -logex -archive .

    I then modified a short script I found in another FAQ by Izzy to deal with multiple genpuid processes:

    #!/bin/bash
    # Supervisor script for genpuid service (c) by Izzy
    ################################################## ############################

    #================================================= ========[ Configuration ]===
    INTERVAL=5 # How many seconds to sleep between checks
    NICE=10 # Nice level - only positive values 0..20
    SERVER="mipcore" # Name of the executable
    LOGFILE="watch.log" # Where to log the progress
    VERBOSE=1 # Generate visual output? 0|1

    #================================================= =======[ Initialization ]===
    # Make us nicer
    renice $NICE $$ >/dev/null

    # Initialize vars:
    ANA1=0
    FP1=0
    MPID1=0

    #================================================= ==[ Run the supervisor ]===
    while [ 1 -eq 1 ]; do
    ANAL=`cat log.txt |grep Analyzed|wc -l`
    FP=`cat log.txt |grep Fingerprinted|wc -l`
    if [ $FP1 -eq $FP -a $ANA1 -eq $ANAL ]; then
    :
    else
    [ $VERBOSE -eq 1 ] && echo "`date +%F` `date +%X` Fingerprinted: ${FP} Analyzed: ${ANAL}"
    echo "`date +%F` `date +%X` Fingerprinted: ${FP} Analyzed: ${ANAL}">>$LOGFILE
    ANA1=$ANAL
    FP1=$FP
    fi
    MPID=`pidof $SERVER`
    for PID in $MPID
    do
    PIDOUT=`ps --noheader -l -p $PID`
    set -- junk $PIDOUT
    shift
    NL=$8
    if [ "$NL" != "$NICE" ]; then
    renice $NICE $PID >/dev/null
    else
    :
    fi
    done
    sleep $INTERVAL
    done

    Seems a bit convoluted, but so far it appears to be working - but I've yet to get to the stage where I can run all this with SugarCube. Very few files are being found (which I find hard to believe, but whatever.) so it's doing analysis locally, which takes about 3 minutes a track, so it is going to take a long time to finish this. My concern of course is that whatever is being generated by genpuid is actually correct. I've backed up my entire library so I can restore just in case this becomes a fiasco.

    If anyone disagrees with what I have observed, please let me know.

    Is there a way I can somehow check the tags that are being created to ensure they are correct? I see that they are being added to tracks, but have no idea if they are correct or not. That has probably already been discussed somewhere but I haven't been able to find it yet.

    I did find a way to validate the PUID information in the tag file. I used the puddletag program to display the tag information, then used this URL to check if the PUID was known:

    http://musicbrainz.org/ws/1/track/?t...ase-rels&puid=

    Just put the PUID after the =

    So far, every PUID appears valid. I did find a new album which just came out didn't have a PUID tag - which I would expect.

    Any other ideas on how to validate these files are being tagged correctly? Thanks!
    Last edited by gbcox; 2012-06-02 at 19:33. Reason: Additional information

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •