Copying between two local disks, best method?

[legisilver]

Hi all, this is my first post, please be kind. *Yes, I've read a reasonable amount of threads but still would like some suggestions on this topic.

Here is my systems specs below...

Physical:
Manufacturer: Supermicro
Model: H8SGL
CPU: 8 CPUs x AMD Opteron(tm) Processor 6320
Mem: 40GB ECC
GPU: NVidia GT240 (no, it can't play Crysis)
1 SSD 240GB (AHCI)
1 SSD 120GB (AHCI)
1 SATA HDD 8TB (AHCI)
1 SATA HDD 8TB (AHCI)
SB700 SATA Controller

Virtual:
Version: TrueNAS-13.0-U3.1
ESXi 7.0-based VM
4vcpu 24GB vram
3 virtual disks (VMDK's):
1) OS (50gb)
2) 1 drive - 1 Dataset (7.2tb) [family share]
3) 1 drive - 1 Dataset (7.2tb) [backup of family share]

*If you need more data on my system I'd be happy to provide it!


Obviously, this is not a mission-critical system. It's just intended to be used as a file share for my home network, mainly used to store family photos and other files like music and whatnot. There is one SMB share that will be used, fileshare.silver.net. We don't have a LOT of data but we do like our data. That said, I'm not looking at investing too much more into this but I'm also not against buying some more 8TB drives, if needed. My mobo supports two more SATA ports that are currently unused.

Ok, here is my question. Before I load up the dataset on the 2nd drive [family share], I want to ensure that I'm backing that data up to the 2nd dataset on a nightly basis. I guess I'm a little ignorant as to what the best method is to do that. My current thought is to simply run an advanced replication task nightly. This replication task would be setup kind of like this, using the (Almost) Full File System Replication...

*Ignore that the destination directory location is not available yet, it will be. I just haven't put the 3rd disk in yet.


So, does this method make the most sense if my goal is to pretty much have a 1 to 1 copy of the family share dataset on the backup vmdk (silvernas_3.vmdk)?


Or would you all suggest something entirely different?

 

[Arwen]

3 virtual disks (VMDK's):
1) OS (50gb)
2) 1 drive - 1 Dataset (7.2tb) [family share]
3) 1 drive - 1 Dataset (7.2tb) [backup of family share]

ZFS dislikes virtual storage. If I am reading this correctly, you are not passing through the disk controller, with attached disks, but VMWare virtual storage. This is a big way to loose data. See this post;

"Absolutely must virtualize FreeNAS!" ... a guide to not completely losing your data.

[---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some specific things known to be problematic ----] [---- 2014/12/24: Note, there is another post discussing how to deploy a small FreeNAS VM instance for basic file...

www.truenas.com

I am no expert, perhaps one of the other forum users with VMWare above TrueNAS will chime in.

 

[joeschmuck]

So I'm not a replication expert so I can't answer that question.

But I can offer some advice about the VMWare TrueNAS server you have setup. I've been virtualizing TrueNAS for many moons on ESXi, it works great. But using virtualize drives is a terrible idea. It's fine to test with but not to put any data on it that you want to keep. It all works great, until it doesn't. Using ZFS on a virtual drive is a horrible idea. Like I said, great for testing a VM out, but that is all.

I highly recommend you pass through the hard drive controller (HBA) to the VM, but I suspect the HBA is part of a motherboard chipset. If you cannot pass through the HBA then you could pass through each individual disk, It works but it's not a popular way to do it. I did this for about half a year while waiting on a new HBA.

 

[legisilver]

Yeah, for some reason my mobo won't let me passthrough an entire HBA. It might be because the OS drive for the TrueNAS is also on that HBA as well as the OS drive for the ESXi server. I used to run the ESXi OS off of a motherboard-based USB slot, and I think I WAS able to passthrough the HBA in the past when doing this. Maybe I'll rebuild the ESXi server to load off of the USB drive again and then see if I can passthrough the HBA.


I'm glad you guys brought this up, it's a really fair point.

With that said, if this doesn't work, are you guys talking about passing through the disk as a raw device mapped drive?

 

[joeschmuck]

With that said, if this doesn't work, are you guys talking about passing through the disk as a raw device mapped drive?

That is what I would suggest. I've done it before and helped others do it, all successfully. But some folks dislike it. The good part about it is you can still take the drives out and put them into a bare metal TrueNAS machine and they work.

But if you can pass the HBA, that is by far the safest and easiest way to do it. And I understand not being able to pass the HBA when ESXi is using it to bootstrap the system, it does not allow it.

Good Luck

 

[ChrisRJ]

Yeah, for some reason my mobo won't let me passthrough an entire HBA. It might be because the OS drive for the TrueNAS is also on that HBA as well as the OS drive for the ESXi server.

That is indeed the reason why you cannot pass through the HBA, it is already in use by ESXi.

Why not connect the disks for ESXi and/or the virtual disks to the SATA ports of the motherboard

As to the criticality of the data: Perhaps there is no money or legal consequences attached, but personally I see family photos etc. as mission-critical.

 

[legisilver]

That is indeed the reason why you cannot pass through the HBA, it is already in use by ESXi.

Why not connect the disks for ESXi and/or the virtual disks to the SATA ports of the motherboard

As to the criticality of the data: Perhaps there is no money or legal consequences attached, but personally I see family photos etc. as mission-critical.

Ha! Well, considering my wife is a pro photographer she would nearly kill me if I lost the data. Anyways, the disks ARE connected to the motherboard. There isn't a separate controller or anything.

 

[ChrisRJ]

Anyways, the disks ARE connected to the motherboard. There isn't a separate controller or anything.

You can only pass through an HBA as a whole. For running TrueNAS on ESXi (or similar) you therefore always need a dedicated HBA or SATA controller (only use the one on the motherboard, don't buy a separate one) that cannot be used for anything else, even if you do not need all its ports for the TrueNAS VM.

 

[legisilver]

Hrmm, now I'm starting to question whether it makes sense to run TrueNAS on ESXi at all. Yes, having a virt layer would be nice for the OS, but nearly every other VM I might need is done in the cloud today, so I have no other NEED for a local hypervisor any longer, save for this TrueNAS OS.

Maybe I'll just go bare metal and save myself the headache.

Thoughts?


*I forgot to mention that I also have TWO of the exact same hardware setup.

 

[jgreco]

TrueNAS on an appropriate hardware platform and with ESXi is expected to be problem-free these days. However, you are still stacking one complex software product on top of another, and if you do not actually need to do this, perhaps it is better if you don't.

 

[joeschmuck]

Maybe I'll just go bare metal and save myself the headache.

Thoughts?

Bare metal does make things a bit easier to manage overall. No worrying if you properly shutdown other VM's or having dependencies that bite you in the rear. Also for your hardware setup, it's probably the smart thing to do.

 

[legisilver]

TrueNAS on an appropriate hardware platform and with ESXi is expected to be problem-free these days. However, you are still stacking one complex software product on top of another, and if you do not actually need to do this, perhaps it is better if you don't.

This is just generally good advice, thanks.

 

[legisilver]

Bare metal does make things a bit easier to manage overall. No worrying if you properly shutdown other VM's or having dependencies that bite you in the rear. Also for your hardware setup, it's probably the smart thing to do.

Yup, I just fired up the second board and realized that although it uses passthrough (my original listed does not) and has IPMI, it's running an older Opty 4130 cpu, so even less powerful and slightly less ram.

The convenience of ESXi is nice and all, but in the end the entire purpose of the station is to be a NAS, not a hypervisor. I use cloud services for all vm-related needs now.

Looks like I'm going back to baremetal, bros!

 

[ChrisRJ]

Bare metal does make things a bit easier to manage overall. No worrying if you properly shutdown other VM's or having dependencies that bite you in the rear. Also for your hardware setup, it's probably the smart thing to do.

As you wrote in another thread today: K.I.S.S. rules