Backup from QNAP to FreeNAS using RSYNC stuck at 30%

[Ernie]

Hi guys,

I'm new to FreeNAS. I'm currently having a QNAP and I've got an old micro server that i installed FreeNAS in with the latest 9.10 STABLE OS on it. I'm backing my QNAP data to FreeNAS and somehow I'm having this weird issue like file within 10Gigs I can back up no problem. But with files from 30Gigs, it will start the replication but it will get stuck at 30% and sometimes 47% and will just stuck there like forever. Any idea why?


Regards,
Ernie

 

[joeschmuck]

Please provide details on your FreeNAS system per our forum rule. Also, when you say it's "stuck", does that mean that there is no activity on the QNAP and FreeNAS? Honestly it sounds like you have hit a section of lots of small files or you have placed too much data on your FreeNAS (exceeded the 10% free capacity rule).

Provide some more details and someone may be able to lend assistance.

 

[wblock]

I would wonder if the files are corrupted on the QNAP system, or perhaps it's running out of memory on the source or destination system.

 

[joeschmuck]

I would wonder if the files are corrupted on the QNAP system, or perhaps it's running out of memory on the source or destination system.

Maybe @Ernie has already solved the problem, no responses yet.

 

[Ernie]

Hi guys,

So sorry for the late reply busy week indeed. so far i dont see any issue in terms of the files being corrupted. I did a few replication jobs base on the folder sizes and yea i notice that when the folder size more than 30Gb it will get stuck at. 30%. i did a test backing up to another qnap of the same folder and there was no issue. Let me post some more info.

 

[Ernie]

currently I am trying to implement this backup solution using FreeNAS so that i can backup my qnap to FreeNAS. I currently installed FreeNAS on a flash drive attached to my laptop which is a Thinkpad T410s not a micro server yet. the OS is sitting on a flash drive while I use the internal hdd as my mnt drive. Im planning to test it first using my laptop and once all is perfectly working, i will go ahead and format my HP server and install FreeNAS on it.


Laptop Specs: Thinkpad t410s
Build: FreeNAS-9.10.2 (a476f16)
Platform:Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz
Memory: 3862MB
Os Disk: 8GB flash drive
Mnt Drive: 320GB

 

[joeschmuck]

Your RAM is half of what it needs to be. You need 8GB RAM to run FreeNAS without errors.

 

[Ernie]

@joeschmuck do you think that would be the issue? Regards.

 

[Ernie]

@joeschmuck . I can do data transfer via samba to FreeNAS without any issues though. Ive tried copying 50-70gb of data with no issue so yea. a bit of a headache for me.

 

[joeschmuck]

FreeNAS can be a bit of a pain when it doesn't have enough RAM and it will throw out intermittent failures.

I'm not certain that is your problem, but I also question the size of your 320GB drive and the remaining capacity. FreeNAS requires a minimum of 10% free space on the hard drive and if you go below that then special optimizing routines start and writing slows down considerably. Not having enough RAM would compound the issue.

Also, is your hard drive error free?

I could point my finger in several directions but without the machine in front of me, or without having more information, I couldn't tell with any certainty other than the RAM is a problem. Some people can run on 4GB, most cannot.

 

[Ernie]

Yea im pretty sure that my drive is error free. I did scan it with HD sentinel before i put it into my laptop. Ive attached a snapshot here

 

[joeschmuck]

So it's not a hard drive issue. Then it could be two things:
1) Short on RAM
2) The NIC in the laptop could be a Realtek which has been proven (I can personnaly attest to this) to cause data transfer crashes. For me it was for large files. But Realtek is not very well supported.


Cheers!

 

[Ernie]

@joeschmuck Thanks for the info. Btw my HP server (MicroSVR Gen8 G2020T) has a better specs with 4x 6TB and 16GB of ram and will setup raid10 on it. is it possible to partition the whole volume and allocate maybe 100GB as may OS and the rest will be for my Data? Pardon me for being a newbie but im getting a hang of it now.

 

[snaptec]

I'm not sure its the FreeNAS side.
I've got problems with qnaps a couple of times using their built in rsync.
It randomly crashed.
Temporary solution was to mount the qnap through NFS on a Linux Server and let the Linux do the Rsync, then it worked like a charm.

 

[joeschmuck]

is it possible to partition the whole volume and allocate maybe 100GB as may OS and the rest will be for my Data?

Normally I'd just say that is isn't possible but I think that it could warrant an experiment, just not by me. I would not recommend it either even if the experiment worked because it would be a mess when you had to replace a drive with the OS on it due to hardware failure. I would recommend a USB Flash drive if someone was thinking this route and I hate using USB Flash drives as the boot device.