Weird Genpuid Problem

[bookemdano]

I tried to find a freely available song that didn't work for me. I just downloaded the first song from this album: http://www.amazon.com/gp/product/B00...rd_i=678551011 and it didn't work.

So you could try that one (if you can download songs from Amazon).

I assume you downloaded track 1 ("Helplessness Blues" by Fleet Foxes). Nice choice--they're a great band. Anyway, I just downloaded it and ran it through genpuid without any problem whatsoever. Here's the output from the command window:

genpuid 57aae6071e74345f69143baa210bda87 -archive -data -logex -print -r -threads=1 -xml=xmltest.xml "y:\01 - Helplessness Blues (Album).mp3"
GenPUID Version 1.4 - 31 October 2008
Copyright (C) MusicIP Corporation 2006-2008.

Songs to be scanned: 1
43974224 Elapsed time: 2
45940304 Elapsed time: 62
** Stopping Processing

Looking at the id3 tags, there are now three additional tags:

MusicIP Data:
TWlQAQABAAoAUAEBAAAAAKl7qEDf0YQ2uSondIABi3eCQ/Epm0dEhJLa86jJJZ1u+QusBIu3sVKU/YbOgAEBCr2hAv/3CIKp3qnydx9CvJzcw4ACgAE=

MusicMagic Data:
AQEAAAAAqXuoQN/RhDa5Kid0gAGLd4JD8SmbR0SEktrzqMklnW75C6wEi7exUpT9hs6AAQEKvaEC//cIgqneqfJ3H0K8nNzDgAKAAQ==

MusicMagic Fingerprint:
AQJZDskkeU4dH/kW6CdMEIMNzhQ1CooKpRZkDNcNhhP+EIMMwxYdCikWVhnLDvMMoQ0kDTwJjwoZBlMEowQyA3sDdgOgA9sDWgNJAvQClgIf/138FfLqz/3uoeqa7cX/swOg+wz/vv2y/5cPVhw+NgAqQBINJu0QgiMiOoEIuP68+6kFMwSmCYP+Kf45/wX/zf+sAMQAi/90/4b+hv6U/woAuwD4AbbC7QTlYGQo7/lF/+P01wWPDW4F0wM1A+3x9vcYE2z8hv1eDSAEOwYzCAINtAZWA7gCev+vAH7/LAAE/uEAPv/6AEsAU//b/5v/oP/EAAP6XgTSAMjxM/nOB80Rxf/7+kcDqvSOIAM5RLbhJ1Mdac+x6F4PAxGY/0oAKP8eAUoBFQPvARUANv73AN0AUQCn/7X/3AEWAJ4AVP/8AKwNMxeAyystwNg6MQoYwB12Eq3/8/IUDq4ElgnoCUQIC8jK6gP/z+3v//j0vQVpCv8GGQI0BlAD4gGlAb8CpAGiAnwBpgIZAkYB2wHxAVr/1Pjg9S0Yd/q4L1LtHfB5Cav41Ahu76rpU/hA/VArvjEWzePOyt0V7RoXe/D//fD8qvzg/HUEdv0R/pb/kABk/yb/vv5G/zD+9f9F/xn/agGFC2gWqPBOTx0VNa6j+tgXeAh5DM/yLAFD9CEGGP3fBGoPTwn5ChUCC+xpBboBMPht+yT+vQAMAhAANAFRAN0BQAEsAC4ABAARAREAdgEhJyUeGQ==

I have uploaded the resulting mp3 file to mediafire:

http://www.mediafire.com/?0rgpcjk76fvkab1

Maybe someone else here can try the same experiment and see what they get. Otherwise I am about out of ideas other than to maybe try a totally different computer (not sure if you were dual booting linux and Windows on the same machine), try using the DNS key I pasted above and try re-downloading the windows version of genpuid from elsewhere.

Edit: Here's the output from the log file:

[3] Fingerprinting y:\01 - Helplessness Blues (Album).mp3
Mon Nov 21 10:52:52 2011
 [0:02] Fingerprinted: y:\01 - Helplessness Blues (Album).mp3
Error communicating with server (2) : -1
[4] Analyzing y:\01 - Helplessness Blues (Album).mp3
Mon Nov 21 10:54:55 2011
 [1:02] Analyzed: y:\01 - Helplessness Blues (Album).mp3
Mon Nov 21 10:54:57 2011
 ** Stopping Processing

and the xml file:

<?xml version="1.0"?>
-<genpuid xmlns:mip="http://musicip.com/ns/mip-1.0#" songs="1"> -<track status="pending" file="y:\01 - Helplessness Blues (Album).mp3"> <data>AQEAAAAAqXuoQN/RhDa5Kid0gAGLd4JD8SmbR0SEktrzqMklnW75C6wEi7exUpT9hs6AAQEKvaEC//cIgqneqfJ3H0K8nNzDgAKAAQ==</data> <print>AQJZDskkeU4dH/kW6CdMEIMNzhQ1CooKpRZkDNcNhhP+EIMMwxYdCikWVhnLDvMMoQ0kDTwJjwoZBlMEowQyA3sDdgOgA9sDWgNJAvQClgIf/138FfLqz/3uoeqa7cX/swOg+wz/vv2y/5cPVhw+NgAqQBINJu0QgiMiOoEIuP68+6kFMwSmCYP+Kf45/wX/zf+sAMQAi/90/4b+hv6U/woAuwD4AbbC7QTlYGQo7/lF/+P01wWPDW4F0wM1A+3x9vcYE2z8hv1eDSAEOwYzCAINtAZWA7gCev+vAH7/LAAE/uEAPv/6AEsAU//b/5v/oP/EAAP6XgTSAMjxM/nOB80Rxf/7+kcDqvSOIAM5RLbhJ1Mdac+x6F4PAxGY/0oAKP8eAUoBFQPvARUANv73AN0AUQCn/7X/3AEWAJ4AVP/8AKwNMxeAyystwNg6MQoYwB12Eq3/8/IUDq4ElgnoCUQIC8jK6gP/z+3v//j0vQVpCv8GGQI0BlAD4gGlAb8CpAGiAnwBpgIZAkYB2wHxAVr/1Pjg9S0Yd/q4L1LtHfB5Cav41Ahu76rpU/hA/VArvjEWzePOyt0V7RoXe/D//fD8qvzg/HUEdv0R/pb/kABk/yb/vv5G/zD+9f9F/xn/agGFC2gWqPBOTx0VNa6j+tgXeAh5DM/yLAFD9CEGGP3fBGoPTwn5ChUCC+xpBboBMPht+yT+vQAMAhAANAFRAN0BQAEsAC4ABAARAREAdgEhJyUeGQ==</print> </track> </genpuid>

[Henry66]

I did not use genpuid, but just MIP.
This is what I got for "Helplessness Blues" MP3 downloaded from Amazon:

MUSICMAGIC DATA:
AgEAAAAAqXuoQN/RhDa5Kid0gAGLd4JD8SmbR0SEktrzqMklnW75C6wEi7exUpT9hs6AAQEKvaEC//cIgqneqfJ3H0K8nNzDgAKAAaxHqkK1lYLV+rZAv4+ihQ2GXvLWnB9Uv7g875WlNo1Q8gKtu8mDuJqqYoMogzAIIcWRH13rQKPS1kP57/ztsuTkwaSyhG8=

MUSICMAGIC FINGERPRINT:
AQJZDskkeU4dH/kW6CdMEIMNzhQ1CooKpRZkDNcNhhP+EIMMwxYdCikWVhnLDvMMoQ0kDTwJjwoZBlMEowQyA3sDdgOgA9sDWgNJAvQClgIf/138FfLqz/3uoeqa7cX/swOg+wz/vv2y/5cPVhw+NgAqQBINJu0QgiMiOoEIuP68+6kFMwSmCYP+Kf45/wX/zf+sAMQAi/90/4b+hv6U/woAuwD4AbbC7QTlYGQo7/lF/+P01wWPDW4F0wM1A+3x9vcYE2z8hv1eDSAEOwYzCAINtAZWA7gCev+vAH7/LAAE/uEAPv/6AEsAU//b/5v/oP/EAAP6XgTSAMjxM/nOB80Rxf/7+kcDqvSOIAM5RLbhJ1Mdac+x6F4PAxGY/0oAKP8eAUoBFQPvARUANv73AN0AUQCn/7X/3AEWAJ4AVP/8AKwNMxeAyystwNg6MQoYwB12Eq3/8/IUDq4ElgnoCUQIC8jK6gP/z+3v//j0vQVpCv8GGQI0BlAD4gGlAb8CpAGiAnwBpgIZAkYB2wHxAVr/1Pjg9S0Yd/q4L1LtHfB5Cav41Ahu76rpU/hA/VArvjEWzePOyt0V7RoXe/D//fD8qvzg/HUEdv0R/pb/kABk/yb/vv5G/zD+9f9F/xn/agGFC2gWqPBOTx0VNa6j+tgXeAh5DM/yLAFD9CEGGP3fBGoPTwn5ChUCC+xpBboBMPht+yT+vQAMAhAANAFRAN0BQAEsAC4ABAARAREAdgEhJyUeGQ==

My "MusicMagic Fingerprint" is the same as yours. My "MusicMagic Data" is longer than yours, but the first part is almost the same. Don't know what "MusicIP Data" is, but it must be something that only genpuid generates.


I have the same track accurately ripped from CD in FLAC format, and for that I got:

MusicMagic Data:
AgEAAAAAqVqooOEZhELajyE3gAGLboOe8SGbZkSKkl/zt9fGkan36awPlUCxUZmPiD+AAQEgvXICl/D+gyLesfDbHpm2ENzHgAKAAao/p73XtYVO6Z00K4b9h0iB0Phim34/YaAe3yenBJKl/0asMJtmx4WdA4Y3kkriy9cq8RPPLIK01zj+9wvDt7vJToJsgTA=

MusicMagic Fingerprint:
AQJdDsYkfE4iH/gW6CdNEHwNyxQ3Co0KoRZoDNMNgBP6EIQMvBYjCiAWVhnBDvMMng0jDT4JjgoXBlYEpAQ0A3wDdgOjA9wDWgNKAvYClgIh/1j8MfLr0Avuo+qK7cX/swOn+xL/tv2m/5IPXRxNNhAqSxH3Jv4QgSMSOnEIsv7A+7UFMASaCX7+K/45/wX/z/+kAMwAk/96/4T+hv6R/wUAwQD4AdDCxwUOYGgo6flS//X09QWMDVgFzwMoA/fyLPcnEz78kv1iDRUEPwYUB/sNuwZbA78Cd/+yAH7/JgAE/uIARP/5AEsAXv/g/5n/nP/SABX6VwTEAPDxWfnMB7wRtf/a+kkDufSMIBc5Wbb4J10dhM+o6HEO8hGP/0cAKf8jATcBCAP1ASAAP/71ANsAUACu/67/5AEWAJ8ATgAAAH0NKBfdy2QtvNhdMKcYvh2UEsMAKPHlDlwEewnxCZIId8ij6ar/0e3T/8r0eQVNCvEF9wIPBlwD9AGfAbUCtAGiAnsBowIaAlMB3AHvAVL/tfkD9RQYuvqML2bsv/AsCYv5DgiI78rpQfgx/Qcr9DFDzorOi91C7T8XJPE2/fr8ffzO/FoEZP0Q/ov/iABf/y//vf5K/zX+8P88/xr/ZwFtC2QWi/B3TrAVBq7Y+eUXlgetDZfxcQKG8nwC5fqbB6EOTwuBC+sCi+11BX8A7/jX+1f/Dv/hAewAMgFVAOEBOQErADQABwAOARkAeAEiJyUeGQ==

[bookemdano]

Thanks for posting your results Henry66. Incidentally, I also have this track accurately ripped to FLAC and analyzed... here are my tags for it:

ANALYSIS:
MusicMagic DataAgEAAAAAqVqooOEZhELajyE3gAGLboOe8SGbZkSKkl/zt9fGkan36awPlUCxUZmPiD+AAQEgvXICl/D+gyLesfDbHpm2ENzHgAKAAaeDqE7YH4VO6Fct2YY5hp2Bx/gkm3g7hZ9R3ZGnBJAx/qish5jLyDicGIbPjDLfLNYS6rXL7YK00+H+sA9PtwHF4YJsgTA=

FINGERPRINT:
MusicMagic FingerprintAQJdDsYkfE4iH/gW6CdNEHwNyxQ3Co0KoRZoDNMNgBP6EIQMvBYjCiAWVhnBDvMMng0jDT4JjgoXBlYEpAQ0A3wDdgOjA9wDWgNKAvYClgIh/1j8MfLr0Avuo+qK7cX/swOn+xL/tv2m/5IPXRxNNhAqSxH3Jv4QgSMSOnEIsv7A+7UFMASaCX7+K/45/wX/z/+kAMwAk/96/4T+hv6R/wUAwQD4AdDCxwUOYGgo6flS//X09QWMDVgFzwMoA/fyLPcnEz78kv1iDRUEPwYUB/sNuwZbA78Cd/+yAH7/JgAE/uIARP/5AEsAXv/g/5n/nP/SABX6VwTEAPDxWfnMB7wRtf/a+kkDufSMIBc5Wbb4J10dhM+o6HEO8hGP/0cAKf8jATcBCAP1ASAAP/71ANsAUACu/67/5AEWAJ8ATgAAAH0NKBfdy2QtvNhdMKcYvh2UEsMAKPHlDlwEewnxCZIId8ij6ar/0e3T/8r0eQVNCvEF9wIPBlwD9AGfAbUCtAGiAnsBowIaAlMB3AHvAVL/tfkD9RQYuvqML2bsv/AsCYv5DgiI78rpQfgx/Qcr9DFDzorOi91C7T8XJPE2/fr8ffzO/FoEZP0Q/ov/iABf/y//vf5K/zX+8P88/xr/ZwFtC2QWi/B3TrAVBq7Y+eUXlgetDZfxcQKG8nwC5fqbB6EOTwuBC+sCi+11BX8A7/jX+1f/Dv/hAewAMgFVAOEBOQErADQABwAOARkAeAEiJyUeGQ==

plus the MUSICIP_DATA tag which yes is added by genpuid but is apparently meaningless in the current implementation of MusicIP. I think it may have been for future use or it was some kind of diagnostic or something.

Everything I have was analyzed by genpuid 1.4 on Windows so maybe different versions of mipcore account for some of the differences in the actual tag data.

Also, just wondering--your FLAC files get tagged as I pasted above, correct? With the vorbiscomment field called "ANALYSIS" and "FINGERPRINT" and the text of "MusicMagic Data" and "MusicMagic Fingerprint" prepended to the data itself, right? I've noticed only FLAC files are tagged this way (mp3s are tagged as in my previous post) so I assumed this is just a vorbiscomment quirk--maybe due to vorbiscomment field names not allowing spaces?

Wish we had some insight into what all those seemingly random strings mean. Like you said, the strings are similar in some spots but quite different in others--even seemingly from the same exact source data.

Oh well, as long as it works I guess. Anyone know if MusicIP and Sugarcube continue to work properly on 7.7?

[Henry66]

Yes, my FLAC tags are as you describe. I just split out the data to make it easier to compare.

Your FingerPrint for the CD FLAC is exactly the same as mine.

[bookemdano]

Yep and the data dag has many similarities and yet several differences. I'm wondering if two different people, using the same version and platform of mipcore and with the same DNS key (if using genpuid) on the same FLAC track would get identical data tags. If not that points to other variables being encoded in there--like maybe the date and time (and who knows, maybe even IP address?) when the analysis was run.

Just idle curiosity. We'll probably never know the answer.

[slamhound]

Okay, you were right, the difference is linux versus Windows. Although I thought these files were not working in Windows, I just wasn't giving it enough time. Normally (as you know), if a file is analyzed locally, the mipcore process starts up, CPU usage spikes, and then after a long time of this, the file is analyzed. This is still what happens with files that work both on linux and Windows.

For files that don't work, however, genpuid starts, then there is a very long pause with no CPU activity. On linux, that pause never ends, and when it went on for a while on Windows, I assumed it was the same thing and killed genpuid. The difference is that on Windows, after a while, mipcore does start and can analyze the file. For each of these files, you get the "Error communicating with server" message in the log file.

So it seems to me that whatever is causing the "Error communicating with server" message is not arbitrary but is linked with specific characteristics of the song that is being analyzed. In other words, something about a specific file triggers this message. When this happens on linux, the program gets stuck (I never see the message in the log on my linux system); when it happens on Windows, it somehow can continue.

I checked this by taking a song that can be analyzed on linux and when analyzing it on windows, I don't get the "error communicating with server" message. It just starts mipcore and analyzes the data.

So it's still very confusing, but I can work around this by using Windows just to run genpuid.

One other quick question, how do you look at the tags that genpuid creates? I can't find the added tags in picard, puddletag, or easytag; the only thing I can see is a MusicIP Fingerprint in picard. But I can't see the data tag in any of those programs.

[bookemdano]

Glad you got to the bottom of it. The tags are simply ID3 (for mp3 files) and vorbiscomment (for flac files), which are the standard tag formats for those two file types. The tags genpuid creates should be easily visible in any tag editor (but for the record I've been using the freeware app called simply "Tag" on Mac OS).

If you've been using genpuid on Windows and the file is an mp3 file then you should see three tag fields created by genpuid:

MusicMagic Fingerprint
MusicMagic Data
MusicIP Data

All three will have long strings of seemingly random ASCII characters.

If you archived the analysis via MusicIP Mixer then you will only see the first two tags listed. For a FLAC file the fields are similar but named "FINGERPRINT", "ANALYSIS" and "MUSICIP_DATA" instead (with MUSICIP_DATA missing if you used MusicIP Mixer instead of genpuid).

If you used genpuid on linux/MacOS I have no idea what those tags look like--I assume they are the same but I could never get genpuid working properly on those platforms so I can't say for sure.

Oh, and one more thing to watch for, some tracks may show a "MUSICIP_PUID" tag (probably shows as "MusicIP Puid" for mp3 files). If you see that tag then it means for that track genpuid/MusicIP Mixer matched the fingerprint with one on the server and the contents of the "MusicMagic Data" tag was fetched from the server and not generated locally. I have very few files that show a PUID tag but the less files I feed to genpuid en masse the more PUIDs seem to match.

If all you are seeing is the Fingerprint tag then it means that analysis was not completed or not archived. Make sure you're using the -archive switch with genpuid. And for troubleshooting, turn on the -xml option and then genpuid will write the analysis tag to the xml file in addition to the id3/vorbiscomment tag.

Hope that helps.

[raven22]

I have some issues with genpuid as well, but they are on both windows and linux.
In the log i find this message: Error decoding ID3 tag (mismatched BOM)

Other software (including mip server) have no problems with reading the tags from these musicfiles.

Anyone knows what this means?

[bookemdano]

Well the MusicIP server has been rebooted , or something. Because as of this evening it's working perfectly again, performing lookups and matching PUIDs for most everything. I don't know if that has also resolved the hanging issues on linux and macos--I bet it did but I have only tested Windows.

And I have no idea when exactly the server started working again as I hadn't run genpuid for a couple of weeks until tonight.

Who knows how long it will last but if you have any analyzing to do, now would be a good time.

[slamhound]

Yep, working for me on linux, too. However, I'm now having a problem with tagging. If I rip to flac, I usually use picard to get the basic tags. Then I run it through genpuid to add the analysis tags. This always used to work fine. But now when I try it, although I have the archive option set with genpuid, when I used picard to look for the analysis tags (whichever were appropriate for flac), they aren't there. I experimented a little and found that if I use a different program to erase all the tags (or don't use picard in the first place) then genpuid writes the tags correctly. However, if I then use picard to edit the tags, it gets rid of the tags genpuid created. Strange.

I also noticed that the tags are actually still there, but they don't show up when I look for them in picard, and they aren't picked up when I scan them in to the musicmagic server. I know they are there, though, because if I use puddletag and look at extended tags I see them. Even stranger.

So, my genpuid problem is solved, but now I need to figure out this picard problem to finally get the process that I was using for tagging and analyzing back.