PDA

View Full Version : downloading files from LMS



muksuluuri
2018-01-24, 06:01
Hi All,

I decided to register and ask help for a problem I've had for quite some time.

Is there a way to download/copy all files from LMS?

Here's the background as to why I'm asking this. A long time ago my NAS broke and I started the recovery process, which for the most part was a success. I was able to save most of my music. Something went very wrong though, and in LMS I can see most of the folders and files as a double. The first folder contains the music that plays fine and the second only zero byte songs or just some tracks that play for a few seconds. If I browse the files normally on a computer I can only see the zero byte and bad files. This is some sort of file system corruption issue I suppose. Just to make it clear, the only way to see the proper files is via LMS and squeezebox. The file system is ZFS with two mirrored hard drives, if that makes any difference.

Now I'd like to find a way to safely copy the music to another destination, and then delete the original files and rebuild the library.

Naturally I'd like to understand why the double file thing has happened, but the most important thing for me is just to be able to copy and save my music.


Muksuluuri

muksuluuri
2018-01-24, 06:34
Hi,

1: no way
2: did you do a cleanup and rebuild databse from scratch already?
3: What System? There are many of them and each one uses other ways to only copy files larger than x bytes.


Thanks for the quick reply!

Currently squeezebox is the only thing that can see the proper files. I really don't want to rebuild database or anything else, I'm just looking for a way to save my music. If I browse the files in any other way, be it windows, linux, all I can see is the corrupted files. Once I have the files safe I can rebuild database. For the moment I'm not worried about double files, only about saving the music.

Most of my music is ripped from my original CDs. It would be a huge task to rip it all again. Plus there are some rare stuff I only have as mp3s.

My LMS resides on my linux mint box.

Paul Webster
2018-01-24, 06:52
Can you get to command line on the NAS (eg SSH or Telnet)?

KeBul
2018-01-24, 15:38
In LMS if you drill down through your library to an individual track, in the left hand pane, you should get a location for that track, this may provide you with a clue as to the whereabouts of the good working file(s).

Also on my system If I right click on the location I get the option to download the linked file, (also a Linux mount point to a share on a NAS) which works for me. Might be time consuming but if no better methods appear at least you might be able to get your rare mp3's safely off the NAS.

Good lesson learnt here, regarding backups... even a couple of 2.5" external USB hard drives won't break the bank and would give you two separate copies to recover from.

Good luck

Kev

mherger
2018-01-24, 22:46
> Also on my system If I right click on the location I get the option to
> download the linked file, (also a Linux mount point to a share on a NAS)

That's correct. But LMS will only allow you to download a single file at
any time. I don't think this will be a reasonable way to get to the files.

But anyway: if LMS can see the "correct" files, then you should be able
to do so, too. You're better off trying to understand the real problem
and fix it with some tool closer to the OS than LMS.

> Good lesson learnt here, regarding backups... even a couple of 2.5"
> external USB hard drives won't break the bank and would give you two
> separate copies to recover from.

Yes! Backup. Do backups. Really.

--

Michael

muksuluuri
2018-01-29, 02:11
Thank you everyone for your replies. I've been away for a while but now I'm back.

Originally I had a Buffalo NAS which broke, along with two of the four hard drives inside. I was able to mount the two good drives on my desktop pc running linux mint and do recovery using tools and help online. I was able to save almost all of my music. The NAS could survive one disk failure but not two, so I consider myself very lucky to have been able to recover so much.

The situation now is that I have the two working hard drives formatted and as a mirrored zfs, and this is where the music is stored. Something went wrong during the process of copying the files into this zfs drive. And the result is if I browse the files via desktop, ssh into it or anything else I only see broken files. But LMS sees both the good files and the broken files.

I do agree that I should first understand what is going on at the os level. But I don't, and that's why I'm asking for help. And my first priority still is to save the music.

I'm just hoping someone with more experience can guess what's going on and give me something I could try.

mherger
2018-01-29, 03:08
The situation now is that I have the two working hard drives formatted and as a mirrored zfs, and this is where the music is stored. Something went wrong during the process of copying the files into this zfs drive. And the result is if I browse the files via desktop, ssh into it or anything else I only see broken files. But LMS sees both the good files and the broken files.

I think you should try to get a better understanding of how zfs works. This sounds like an issue with the file system, rather than LMS applying some magic. If LMS can see the files, then you can see them, too. But you might want/need to see what is wrong with zfs.

Don't panic. The files obviously are there. Leave the system alone while you figure out the details :-).

d6jg
2018-01-29, 03:18
Where is LMS actually running ?
I have read through the thread and must assume its on a different machine to the desktop and NAS that you refer to.
If I am right have a look at the path that LMS thinks the files are stored at and try shelling onto the NAS directly from the LMS machine using that path and NOT from your desktop.

EDIt. Now I see that LMS is ona Linux mint box. Shell to the NAs from there.

muksuluuri
2018-01-29, 04:08
My NAS broke. The LMS and the library both are on the linux mint machine. Sorry if I wasn't being clear on that.

Another concern is that from the original NAS two HDs broke. Now the music is on ZFS on the remaining two HDs. Which of course is very stupid, because if the first two HDs broke and the two remaining ones are of the same manufacturing batch, there is a risk that they are faulted too and that it is just a matter of time.

So I am a bit nervous and want to get the music into a safe location.

Thanks again guys for your replies.

Greg Erskine
2018-01-29, 14:35
hi muksuluuri,

Normally, if 2 drives fail in a 4 disk array your files are gone. :( Your files are striped across the 4 drives.

I am not familiar with ZPS though.

In a professional environment, monitoring disk health is daily task and disks are replaced when they become unhealthy rather than completely fail. You also have to remember that the disk rebuild process can take some time and if another disk fails in the array during the rebuild process you have a big problem.

I can't see a mechanism where files are available through one application but not available through the operating system.

regards
Greg

garym
2018-01-29, 16:41
This has been noted, but to be crystal clear for anyone coming across this thread it is important to understand that a RAID disk setup is NOT a backup. Google "Is RAID a backup" if you want to read 1000s of horror stories.

muksuluuri
2018-01-30, 01:38
hi muksuluuri,

Normally, if 2 drives fail in a 4 disk array your files are gone. :( Your files are striped across the 4 drives.

I am not familiar with ZPS though.

In a professional environment, monitoring disk health is daily task and disks are replaced when they become unhealthy rather than completely fail. You also have to remember that the disk rebuild process can take some time and if another disk fails in the array during the rebuild process you have a big problem.

I can't see a mechanism where files are available through one application but not available through the operating system.

regards
Greg

Yes I was very lucky to be able to recover so much. As I wrote earlier I got some help online. I could not have done the recovery on my own. By the way, family photos are the most important to us and those were backed up in another location. Nothing was lost there.

The NAS breakage and the recovery is a thing of the past, that's done. My point in telling about that part is I'm afraid the two remaining HDs will fail at some point, because the other two already did.

I can't understand either how my files are available on LMS but not through the os. That's precisely why I'm asking for help.
I can't copy them to a safe location because most of them are zero bytes or just a few seconds long. But via LMS I can see and listen to both the broken ones and the full undamaged songs.

I really appreciate you guys trying to help, but it is starting to look like no-one here has a clue on what has happened to my files. I suppose I will need to search for help in some other forum, maybe a ZFS support or something.

I will be monitoring this thread too every day, in case there is someone who can help me.

mherger
2018-01-30, 01:59
Did you try what d6jg suggested? Can you check where the valid file in LMS is pointing, and where the broken item is pointing? Maybe you then can figure out where to find the working files on your system.

KeBul
2018-01-30, 02:05
Ok so I'll try one more time...

Have you used LMS to drill down to a working track and find the path that LMS is using?

Likewise you could do the same for a non-working track and compare the two paths to see the difference (or not).

To me this is crucial - if LMS can see the working files then you need to find out where LMS is looking.

Kev

Edit - Ha Michael beat me by a few minutes :o)

Edit 2 - you can click on M next to a working track to get the song info, relevant line highlighted:

24449

muksuluuri
2018-01-30, 12:57
You won't get very far if you can't be bothered to do as we ask. :(

Man, I was working on it, posting screenshots. You replied in the middle of the first and second message I was sending.

Anyway I deleted the first one because I figured out what is wrong.
Initially it seemed to me as if the paths were exactly the same. Only after I uploaded the images I noticed the difference.

24457
24458


Naturally I feel very stupid that I didn't see it. I have a folder "Albums" inside the folder "albums".
There is nothing wrong with the file system. From the difference in file length you can see the corrupt file and the proper file. I must have made an error when rsyncing the files back then.

Sorry for taking up your precious time and thanks again for your replies.
I was at work when I was posting here and didn't have access to my machine. As soon as I got the chance I did what you suggested.

KeBul
2018-01-30, 13:41
Cool,

No worries, it can be difficult to respond to all the requests for further info and ideas for troubleshooting when receiving help on forums, of course it's a good thing that many want to help out, the ongoing community support here is great.

Glad you are sorted

Kev