How to rsync from shell to an external USB drive

[Shoop83]

Hello!

I've had a pool go bad to the point I can no longer mount it in the GUI without causing a kernel panic. I need to save the data in the pool if at all possible.

I've been able to mount the pool (BigVault) as read only through the shell, but it is not visible in the GUI and I can't figure out how to get it visible there to share it with my windows machine to copy the data so I can destroy the pool and begin again from scratch on my FreeNAS machine.

A few questions I just can't seem to figure out from the documentation, forums, tutorials, etc...
1. How do I view the file directory in the read-only mounted pool?
2. How do I use rsync to copy the files from my read-only pool to an external usb harddrive? (Is there a better command to use to do this?)
3. Will using rsync copy the files so I can access and use them on a windows machine once the copy is finished?

Thanks

 

[Chris Moore]

3. Will using rsync copy the files so I can access and use them on a windows machine once the copy is finished?

That is entirely dependent on the file system of the USB drive, not rsync. From what I have experienced and from what I have heard mounting NTFS is supposed to be possible, but I had problems with reliability.

2. How do I use rsync to copy the files from my read-only pool to an external usb harddrive?

You could use a command similar to this, changed for your specific path:
rsync -a /mnt/Emily/BigPond/ /mnt/Backup/BigPond/ --no-perms

(Is there a better command to use to do this?)

You might be able to use Midnight Commander. A text mode file manager that is built into FreeNAS. Here is a brief video demo:
https://www.youtube.com/watch?v=lWu2R9cn_b8&t

1. How do I view the file directory in the read-only mounted pool?

If you mounted it from the command line, the mount point may not be where you expect it to be. It might be mounted at root. You can look around for it but you should just cd into it like any other directory.

 

[Shoop83]

Hi Chris, thanks for helping me again. I super appreciate it.

Midnight Commander looks promising. Thanks for the video link showing it in action. Can it access external USB drives?

I followed the advice here: https://forums.freenas.org/index.php?threads/zfs-has-failed-you.11951/

Going down to the line saying:
If you get a panic then repeat the above steps, but try:

# zpool import -o readonly=on -R /mnt poolname

If this works your pool imported read only. Backup your data and life goes on.


I had to alter it to
# zpool import -o readonly=on -fR /mnt poolname to get it to mount.

The part that is causing me headaches is the line "If this works your pool imported read only. Backup your data and life goes on."

I don't know how to backup my data from the shell in a read-only pool.

If Midnight Commander is a good way to go, I'll go that route. There's not enough space on the disks to make a new pool and move the data to it. If there's a better way to backup my data so I can rebuild from scratch I'm all ears.

 

[pro lamer]

What is missing in @Chris Moore 's answer? Have you managed to mount the USB hdd?

Sent from my mobile phone

 

[Chris Moore]

If Midnight Commander is a good way to go, I'll go that route. There's not enough space on the disks to make a new pool and move the data to it. If there's a better way to backup my data so I can rebuild from scratch I'm all ears.

If you can mount the USB drive, it will be a folder on your directory structure that you should be able to copy data to using Midnight Commander (or any other method) and the only thing special about your pool being mounted the way you did is (two thing) first, it may not be mounted where it normally is, which would be under /mnt/<pool name> and second, it can't have any data added to it.
Do you know how much data you have in there to backup? zpool list will tell you how much space is allocated.

 

[Chris Moore]

PS. When I have had to manually mount a pool, it always showed up on the root directory at /<pool name> instead of under /mnt but it should not be difficult to find.

 

[Shoop83]

@pro lamer I haven't yet had a chance to attempt mounting a USB drive. I'm waiting for Amazon to send me a new drive large enough to hold everything. Nothing is necessarily missing from Chris's response, I'm just incredibly ignorant about navigation commands and processes in the shell. It takes me hours to figure out what someone who knows what they're doing can accomplish in a few minutes.

@Chris Moore Thanks for that explanation, that helps a lot. I think I'll be in business once my USB drive arrives so I can attempt this.

Do you know how much data you have in there to backup?

I've got Roughly 6.4 TB of data used with a little over 4 TB free. A large amount of it I have backed up on another externa (it's what originally got copied into my FreeNAS machine) so I don't NEED to save every last file. If I can just copy it all that will be simpler than combing through the files to grab only things that are new since originally transferring files over.

 

[Shoop83]

PS. When I have had to manually mount a pool, it always showed up on the root directory at /<pool name> instead of under /mnt but it should not be difficult to find.

Using Midnight Commander I am able to find my read only pool and see the files in it. That's reassuring. What I'm not yet able to figure out how to do is find the attached external USB. It doesn't seem to be anywhere in the folder structure though I can see the NAS recognize when I plug it in.

Any thoughts on how to find it?

And an oddity... I'm using MC through the shell in the FreeNAS WebGUI but it doesn't seem to like to let me use the mouse to click on anything. For example, the buttons 1-10 across the bottom, I can't click them. All I can do is highlight everything.

 

[pro lamer]

Any thoughts on how to find it?

Can you see it when you run mount without parameters?

If not, what path under /dev/ directory is it assigned when the below happens:

I can see the NAS recognize when I plug it in.

?
When you learn this, you can use the mount command with parameters.

Sent from my mobile phone

 

[Shoop83]

what path under /dev/ directory is it assigned when the below happens

I was able to see that it mounted as /gpt so I modified my command to get it mounted and that seemed to work. I could see in Midnight Commander and navigate to it.

I had to figure out how to SSH into FreeNAS to get Midnight Commander to function properly. Using it through the GUI shell wasn't allowing me to click or use the numbered commands. I was able to SSH into my machine using Bitvise and that let me navigate Midnight Commander properly. When I tried to copy something from my NAS to my external, it told me there was no space. Did some trouble shooting on that for a while and eventually realized that in the Bitvise window I could see the files on my NAS and I could see my Win10 PC file tree.

I ran a test to see if I could download a file from the NAS to my Win10 PC and it worked! So I grabbed my picture folder off the NAS before making any other changes.
Rather than trying to figure out how to modify where Midnight Commander was caching files as it moved them (I think that was the issue with it claiming the destination hard drive was full) I plugged my external hard drive into my Win10 PC and downloaded the contents of my NAS to it. 20 hours later and it looks like it worked.

The download through Bitvise does tell me I have some 3,600 failed files, but the log doesn't list any errors and only has 6 warnings about plexmediaserver files not being present. That's not 3,600 files. I've been spot-checking and have not yet found anything missing. Maybe it didn't bring along thumbnails or something like that. I'm not too overly concerned because I got my pictures saved, that was the folder causing me anxiety. I got my music saved, that was going to be a big p.i.t.a. to rebuild a collection 20 years in the making. Everything else tastes like victory.

Now I plan to rebuild my FreeNAS from scratch using the latest build. Since this saga first started I have replaced the MB and RAM with "safer" ECC hardware and have an SSD instead of a thumbdrive for the system drive. I ran MemTest 86+ 5.01 for a ludicrous 79 hours completing 28 passes with zero errors then ran Breakin Version 4.26.1-b41 for an hour with zero errors. This thing should be stable.

@pro lamer @Chris Moore thank you for all your help. I hope my rebuild goes flawlessly.