PDA

View Full Version : Q-RAID 1 problems



AlTaiR
2006-09-28, 02:08
Hi there,

I have a QNAP TS-101 with SS6.5 installed on it now. I have an iTank Stardom eSATA housing. Both my TS-101 & the iTank house identical 400GB Seagate drives. When I first got this system & was running SS6.3 I set it up in Q-RAID 1 mode which seemed to work fine though it took forever to sync. I started copying files to the TS-101 over a 100mb link yet the Q-RAID seemed to fall far behind the copy. This made no sense as it's supposed to be up to 3GB/s. [Problem 1]

I finished copying 180GB and a few hours later it caught up. However a couple of days later I had a power outage. When the system came back up it said it need to resync the drive which it tried to do but then failed as it said there wasn't enough free space. I don't understand this as I only had 180GB on a 400GB drive which had been in sync and the only thing that had happened was a reboot. [Problem 2]

I couldn't make it work so I decided to wipe the Q-RAID backup and start again. It took 2&1/2 days to resync now with 180GB on the drive! It finished eventually and seemed to be alright.

I went away for a week and being cautious after the last power outage I turned everything off by first ejecting the Q-RAID drive so all data was saved, then powered it off, then powered the TS-101 off. When I got back from holidays I turned it all back on and again it said it was out of sync and when it tried to sync it failed because it didn't have enough free space. [Problem 3]

I've been in email contact with Qnap in Taiwan and they've been close to useless.

Can anyone help me or explain what's going on?

Thanks,
Bradley.

jimbo_l
2006-09-28, 04:58
Hi,

I'm using Q-RAID too. I haven't seen the specific problems that you mentioned but I have noticed that there is significantly less free space on the exernal disk than there is on the QNAP. I would have expected them to have identical free space.

I have also had a problem where Q-RAID complained that it couldn't sync (I can't remember the exact error), so I had to start the Q-RAID process all over again :-(

I'm not sure whether any of these issues are related to the SlimServer installation or whether they happen on all TS-101s.

See also a post that I made a while ago:


Hi all,

I know this is a bit off-topic, but is anyone else using the QNAP's Q-RAID feature? I recently bought a cheapo external disk enclosure and a 500GB drive (the same model as in the QNAP) in order to use Q-RAID and my initial impressions/thoughts are:

1) I didn't realise that the QNAP had an external (eSATA) socket rather than SATA (they're a different shape) so had to order an SATA to eSATA cable. Doh!

2) Once I received the cable I had to cut down the rubber part of the eSATA end to get it to fit into the QNAP case, which seems to have a recessed connector. Even then it doesn't seem to fit very securely.

3) Once everything was connected there were no problems getting Q-RAID to work - just switch it on in the eSATA Disk page and hit Apply. The disks then chuntered away for a while (several hours) with the progress being shown on the eSATA Disk page.

4) Now that I have the external drive switched on, both disks seems to spend most of their time chuntering away, even when nothing external is accessing the QNAP, which means they don't sleep as much (or maybe at all - I've not been paying much attention). You can 'eject' the drive (from the eSATA Disk page) and switch it off, and it will re-sync when you next turn it on, but that's not an ideal solution.

5) There also seems to be a large discrepancy in the reported free space between the internal and external drives. As I write, the internal drive has 423427MB free versus 378517MB for the external drive. I would have expected them to be the same(?) Should I have formatted it as FAT rather than EXT3? I wouldn't have thought so as I expect the internal drive would be EXT3 as it's Linux-based (I think).

I'd be interested in anyone's thoughts, anyway.

Cheers.

AlTaiR
2006-09-28, 06:18
Hey Jimbo,

I too noticed the massive discrepancy between available sizes on supposedly 'identical' disks. I thought this may be the reason it says it doesn't have enough free space to sync but then that would mean you could never fill the source drive over 40%. Like I said in my earlier posting I had put 180Gb on my source drive but when it finished syncing there was about 320GB on the RAID drive. Surely it doesn't need an equivalent amount of free space as the source drive is using to resync. I would've thought it could just replace it's existing data or update it.

I can understand an additional amount of space being required if parity bits were being set but Q-RAID1 is purely mirroring. How can there be any 'extra' data?

I have no real world experience with drives working under RAID. Can someone more knowledgeable enlighten us?

Thanks,
Bradley.

jimbo_l
2006-09-29, 05:43
I have some real world experience with drives working under RAID. In my case I have hardware RAID on my motherboard (see http://en.wikipedia.org/wiki/RAID#Hardware_vs._software) and the OS sees only one drive to write to; the hardware takes care of mirroring the data.

In the case of the QNAP I would assume that it's using software RAID, so how it mirrors the data on the external drive is anyone's guess.

I would be interested in finding out whether anyone using a TS-101 without SlimServer installed is suffering the same problem, or whether the SlimServer installation is interfering with it somehow.

roamingstudio
2006-09-29, 07:25
Can you see what the extra data is? is it possible that during the backup it is doing a parity check on the data; but then forgetting to delete it? Would it be better to have a simple mirror at push of button (using file copy routines) rather than parity check?

jimbo_l
2006-10-01, 03:07
Can you see what the extra data is? is it possible that during the backup it is doing a parity check on the data; but then forgetting to delete it? Would it be better to have a simple mirror at push of button (using file copy routines) rather than parity check?

All good questions :-)

I can't see what the extra data is unfortunately. I guess if I had Linux installed somewhere I might be able to have a look at the external drive that way, but unfortunately I don't.

Personally I think it would be better to have a simple mirror at the push of a button, yes, as I just want to ensure that I have a backup of my music every now and then; it's not too important to me if I lose (say) the last week's ripped music as I can always re-rip it.

AlTaiR
2006-10-04, 01:40
Jimbo & I can't be the ONLY people out there tring to use Q-RAID 1 with their QNAP/SlimServer box. Can anyone else let us know about their experiences? Is it working perfectly for everyone else & we have dodgy boxes or is there a flaw in the way RAID1 is implemented in the QNAP box?

consciouspnm
2006-10-04, 07:30
I gave up on QRAID as it was taking days to sync. In the end I just plugged the USB drive into my PC and did a one off copy over the network.

mangaratiba
2006-10-07, 10:10
Hi guys,
You are certainly not the only ones that have bad experiences with Q-Raid functionality or I should rather say malfuncionality. For me it started right after filing the brand new Seagate disks with data (no more than 50 GB). Syncing took ages. After day or two the device displayed "Q-Raid failed". I thought something was wrong with eSata interface since QNAP slot was not realy trustworthy. But USB connection failed too. It was truly amazing what I must have done to get any kind of response from QNAP support. After six months still no solution. Their engineers have conected remotely to my device, did some mysterious research of which nature I am not sure, but found nothing. No good way out of where I am now. Returning the product? You must buy a new one first and move you data there hoping Qnap is the only one company selling not fully tested product and giving its customers poor support... Pity since TS-101 is not bad at all. Lot of nice gadgets here. And they keep adding new ones. But Q-RAID was critical functionality for me. I have no extra backup so I can't even imagine loosing all my dauther's pictures for ever for example... Right now I feel like sky diver in amazing sky dive who knows he's got only one parachute on his back...
So there is at least three of us. As soon as I know something on Q-RAID issue I will let you know.

roamingstudio
2006-10-07, 11:09
Do you at least still have the original data on the first (internal) hard disk?

Not a direct help - but can be useful for others...
Looking through the source code of the firmware (downloadable from QNAP) there is the following directory
\hostutil\buildroot\package\raidtools\
Comment from the build line - 'utilities to support old style RAID disks' 'http://people.redhat.com/mingo/raidtools/'
Then there is a load of support in the linux kernel 2.6.12.3 (loads of kernels listed)

Info on QRAID appears in the
\model\TS101\Makefile : cd ${NASWARE_PATH}/NasDriver/nasmgr/app/QRAID1
similarly for
\model\TS201\Makefile : blah blah
There is the command
sudo install -o root -g root -m 644 qraid1.ko ${ROOT_PATH}/lib/modules/misc;

In the \NasLib\config directory is a load of C files for the QRAID1 function - including
\cfg_rsync.c [Line808] int Start_QRAID1(char *dev_name, int progress)
A few lines further on it describes stopping the QRAID1 process by calling
system("/usr/bin/killall qraid1");
system("/sbin/rmmod qraid1");
system("kill -9 `cat /var/run/rsync-qraid1.pid`");
This seems to be the program which is 'QRAID'.

Then under
\NasLib\storage\nasdisk.c is more stuff for QRAID functionality - dated 2002. Copyrighted to ICP electronics.
---------------------
So what does that all mean? Well it would seem that QRAID is just another name for their standard RAID software developed some time ago; building upon rsync-2.6.6 - a standard part of linux. Im not a linux guru so have no idea whether rsync has other issues; but I suspect getting documentation on this would help.

jimbo_l
2006-10-07, 15:14
Im not a linux guru so have no idea whether rsync has other issues; but I suspect getting documentation on this would help.

http://rsync.samba.org/documentation.html

roamingstudio
2006-10-07, 23:40
Thanks Jimbo - I should have added that myself.

rsync is a synchronisation package - it copies files / directories which do not exist; and should delete remote files / directions which have been deleted (on host). It uses memory based hash numbers to determine file differences.

Interesting aspect on the FAQ - Memory Usage
http://rsync.samba.org/FAQ.html
Expect rsync to use 100bytes of memory per file. Thus 800,000 will consume about 80MB of memory. Erm we are currently limited to 64MB - and this will force the use of page files... and lots of disk thrashing.

Looking through the buglist there are some interesting known things.
Bug 3186
Deals with memory useage - seems that RSYNC can generate its file list twice and eat up double RAM. However the fault appeared to be related to linux kernel problems rather than rsync.

Bug 3485 - rsync uses more space in destination even with -S specified
Not resolved; but possible effects include
Different cluster sizes (but when using identical drives it should not be an issue);
Symbolic links being copied as hard links (i.e. 2x directory structure) - there is a -H option which should be called. I wonder if it is not?;
Sparse files (i.e. those inefficently distributed / defraged) are copied across using more bytes than necessary (although this should not happen)
du (disk usage command) incorrectly reads used bytes and clusters after rsync - this is possibly a fault with du and not rsync.

All possible candidates with the original rsync - but dont forget QNAP added functionality as well; athough not that much.

Other "speculation" - how well does QRAID work without SS installed?. If SS is changing the file access attributes; then rsync will attempt to recompile the list of files / hash table in case something has changed. This may occur if you are streaming files; or it may be that SS is discretely doing file scans (which I doubt). Does QRAID work if all streaming is disabled?

jimbo_l
2006-10-08, 02:17
Bug 3485 - rsync uses more space in destination even with -S specified

Does anyone have the means to find out what the extra files are on the external disk? That would be a good starting point.

roamingstudio
2006-10-08, 02:45
People could try drivers to extend Windoze->ext2 / ext3

http://uranus.it.swin.edu.au/~jn/linux/ext2ifs.htm (ext2/ext3 support)
http://www.fs-driver.org/download.html (on a German forum someone claimed it could read USB drives, with ext2 and ext3 formats)
http://www.ghisler.com/plugins.htm (open source plug in for windows commander)

pakman
2007-10-08, 05:00
Hi,

[edit - just noticed that the end of this thread is one year ago today. It is a bit disturbing if this has been going on for a year with no-one fixing it....]

I am having the same problem, and I think that it is caused by SlimServer mounting directories in alternative locations for a chroot-ed environment (using 'mount --bind'). You can see this by telnet or ssh into the NAS, and giving the 'mount' command (on its own, with no options) to list all active mounts.

This causes some directories to be mounted in more than one place in the filesystem. The Q-RAID 1 process (basically a patched rsync) does not recognise that this is going on and makes a separate copy of each directory that is mounted in more than one location. The Qmultimedia directory is one of these, so you get two copies of it on your external disk, which fills up faster than the internal one. It probably also means that restoring from the external disk to a new internal one will apparently succeed, but after that point some things will break (like new media files being added will not be picked up by SlimServer).

I originally posted about this to the QNAP forums at http://forum.qnap.com/phpbb2/viewtopic.php?t=663 , and have also filed a support question with the people who supplied my NAS.

shake-the-disease
2008-03-11, 22:20
Hi,

[edit - just noticed that the end of this thread is one year ago today. It is a bit disturbing if this has been going on for a year with no-one fixing it....]

I am having the same problem, and I think that it is caused by SlimServer mounting directories in alternative locations for a chroot-ed environment (using 'mount --bind'). You can see this by telnet or ssh into the NAS, and giving the 'mount' command (on its own, with no options) to list all active mounts.

This causes some directories to be mounted in more than one place in the filesystem. The Q-RAID 1 process (basically a patched rsync) does not recognise that this is going on and makes a separate copy of each directory that is mounted in more than one location. The Qmultimedia directory is one of these, so you get two copies of it on your external disk, which fills up faster than the internal one. It probably also means that restoring from the external disk to a new internal one will apparently succeed, but after that point some things will break (like new media files being added will not be picked up by SlimServer).

I originally posted about this to the QNAP forums at http://forum.qnap.com/phpbb2/viewtopic.php?t=663 , and have also filed a support question with the people who supplied my NAS.

I'm not sure I want to give up so easily, as I too am in this position of wanting to qraid to an external drive and where my qnap has SS.

I assume if you pulled out the drive from the external enclosure and plugged it into the qnap then the mount --bind command would fail as the directory would already exist?

Couldn't you then just rm the duplicated directory tree and allow the mount --bind to do it's stuff.

This would not get around the problem of chewing up the extra space in the external drive, but really that is not the big issue, the main one is to be able to recover.

Comments?