8 2TB drives. How should I configure them?

[Roveer]

Fairly new to FreeNAS, have been playing around for a few months. I have a bunch to read about zfs, vdev etc but I wanted to post a somewhat generic question about how best to utilize the hardware that I have.

I just purchased 8 2TB 7200RM drives that will go on my IT flashed HBA. Running the wizard creates a single ZFS volume (hope I'm using the right terminology) and I then add a share to that for SMB.

Before I start putting any kind of real data on it (going to kick it around a bit to understand how it works), what is the preferred configuration for my 8 drives? The enclosure is maxed out at 8 so I won't be adding anything, but I'm considering a 12 bay enclosure for the future. It would not be the end of the world if I had to re-create, as all of this data is either backups (already existing in 2 other places), or replicated off-site data (existing in 2 other places), so other than time, no great concern for data loss right now.

I've just started reading about zfs, vdevs etc, but confess i don't know much yet.

I keep hearing about multiple vdev's which can allow for more expansion?

Right now array is just under 10TB but would like to see if it's possible to grow if I were to move to 12 bay or would I just re-build on new platform?

A few pointers would be helpful.

Thanks,

Roveer

 

[danb35]

Eight disks for mostly-static backups? A single RAIDZ2 vdev would be my vote.

 

[Arwen]

Yep, a single 8 disk RAID-Z2 vDev would be my vote too.

@Roveer, from the command line you can use zpool status to check out your pool layout.

 

[Roveer]

So this would be good?

pool: DATA
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
DATA ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/76894668-349a-11e8-a90c-000e0c633e4f ONLINE 0 0 0
gptid/77b8641b-349a-11e8-a90c-000e0c633e4f ONLINE 0 0 0
gptid/78f107c1-349a-11e8-a90c-000e0c633e4f ONLINE 0 0 0
gptid/7a1a04c9-349a-11e8-a90c-000e0c633e4f ONLINE 0 0 0
gptid/7b2750e0-349a-11e8-a90c-000e0c633e4f ONLINE 0 0 0
gptid/7cb6dea9-349a-11e8-a90c-000e0c633e4f ONLINE 0 0 0
gptid/7de7f115-349a-11e8-a90c-000e0c633e4f ONLINE 0 0 0
gptid/807e066d-349a-11e8-a90c-000e0c633e4f ONLINE 0 0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da0p2 ONLINE 0 0 0

errors: No known data errors

 

[anmnz]

If you were to move to 16 bays in the future, you could set up an 8-disk RAIDZ2 vdev now, then later you could add 8 disks as a second RAIDZ2 vdev to the pool, and you wouldn't have to recreate anything.

If you're moving to 12 bays soon, you could do the same thing but with just 6 disks in each vdev.

Then again, if you really are trying to fit 10TB of data onto 8 2TB drives right now, your hands are tied -- an 8-disk RAIDZ2 seems like the only safe way to fit it in!

 

[danb35]

So this would be good?

Looks good to me.

 

[Roveer]

If you were to move to 16 bays in the future, you could set up an 8-disk RAIDZ2 vdev now, then later you could add 8 disks as a second RAIDZ2 vdev to the pool, and you wouldn't have to recreate anything.

If you're moving to 12 bays soon, you could do the same thing but with just 6 disks in each vdev.

Then again, if you really are trying to fit 10TB of data onto 8 2TB drives, an 8-disk RAIDZ2 may be the only safe way to fit it in!

Thanks for explaining. This is what I have to do the reading about vdev's and such. Right now my data needs are around 6-7TB without a huge amount of growth, but I do have to be thinking about the future as we always seem to find a way to fill whatever capacity we have and re-copying 9+TB is no menial task.

 

[Jailer]

You can always grow the pool by replacing the 2TB drives with larger ones replacing them one at a time. It would allow you room to grow in the future without having to invest in new enclosure/hardware. If I had my setup to do over again I would have started with 8 drives instead of 6.

 

[Roveer]

You can always grow the pool by replacing the 2TB drives with larger ones replacing them one at a time. It would allow you room to grow in the future without having to invest in new enclosure/hardware. If I had my setup to do over again I would have started with 8 drives instead of 6.

I'm liking this. I've got two possible directions I'm thinking of going.

1st is to stick with my current hardware which works, but I'm not crazy about (external enclosure, server case actually), wired into my ESXi box running FreeNAS. when I need more space I can do as you mention and start addding 3TB or 4TB disks in place of the 2TB's and upgrade in that fashion.

2nd option is to buy a 12Bay R510 LFF and build 3 3 disk vdev's with 2TB drives (would have to get another 2TB drive for a total of 9), then I am guessing I'd have the option of adding a 4th vdev (correct me when I go off the tracks here) to expand AND the added option of replacing 2TB drives with 3 or 4TB drives for further expansion?

So 2 possible ways to expand the array? If this were possible, I'd prefer this as it gets me off the ESXi machine onto it's own stand-alone hardware, and gets rid of that external server enclosure with 8 red SATA cables going into the back of my R510. Whatta we think about that?

Roveer

 

[anmnz]

I think this is the point at which people around here would start asking for detailed hardware specs -- check the forum rules.

3-disk vdevs sound scary. They'd be RAIDZ1, so if a disk dies you are left with no redundancy on the vdev until it's fully replaced. No redundancy on the vdev means a further failure can destroy your entire pool. The 8-disk RAIDZ2 was sounding good, I'd find a way to stick with that.

 

[Jailer]

Yeah I agree with @anmnz on this one.

 

[CraigD]

If you were to move to 16 bays in the future, you could set up an 8-disk RAIDZ2 vdev now, then later you could add 8 disks as a second RAIDZ2 vdev to the pool, and you wouldn't have to recreate anything.

I guess...

Code:

 NAME											STATE	 READ WRITE CKSUM
		WDRed										   ONLINE	   0	 0	 0
		  raidz2-0									  ONLINE	   0	 0	 0
			gptid/8b6e9691-46c3-11e7-b75f-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/8cb509ee-46c3-11e7-b75f-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/8d72e644-46c3-11e7-b75f-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/9039b411-a428-11e7-998b-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/8eef42c2-46c3-11e7-b75f-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/8fcc9b9a-46c3-11e7-b75f-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/9092843c-46c3-11e7-b75f-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/915353b9-46c3-11e7-b75f-0cc47aab6f2a  ONLINE	   0	 0	 0
		  raidz2-1									  ONLINE	   0	 0	 0
			gptid/904f454a-0ca3-11e8-a51e-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/911e7d80-0ca3-11e8-a51e-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/91cafc26-0ca3-11e8-a51e-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/92dda435-0ca3-11e8-a51e-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/93a7628f-0ca3-11e8-a51e-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/946276dc-0ca3-11e8-a51e-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/950d1ac3-0ca3-11e8-a51e-0cc47aab6f2a  ONLINE	   0	 0	 0
			gptid/95b819f8-0ca3-11e8-a51e-0cc47aab6f2a  ONLINE	   0	 0	 0

Have Fun

 

[Arwen]

...
1st is to stick with my current hardware which works, but I'm not crazy about (external enclosure, server case actually), wired into my ESXi box running FreeNAS. when I need more space I can do as you mention and start addding 3TB or 4TB disks in place of the 2TB's and upgrade in that fashion.
...

Please note that a ZFS pool will not grow until the last of the 8 disks is replaced with a larger one. Meaning if you start replacing drives as they fail with larger ones, you don't see the benefit until ALL 8 are replaced. This is why some people create smaller vDevs. Like 6 disk RAID-Z2s. Easier and cheaper to replace 6 drives than 8. But less initial storage.

It's still okay to replace a failed, lower capacity drive with the current cheapest per TeraByte. Even if that means you end up with some 3TBs, 4TBs and 6TBs by the time your last 2TB failed. The pool will simply grow to the smallest of the new disks, (3TB in this example).

 

[Roveer]

I think this is the point at which people around here would start asking for detailed hardware specs -- check the forum rules.

3-disk vdevs sound scary. They'd be RAIDZ1, so if a disk dies you are left with no redundancy on the vdev until it's fully replaced. No redundancy on the vdev means a further failure can destroy your entire pool. The 8-disk RAIDZ2 was sounding good, I'd find a way to stick with that.

Now I'm just not thinking straight. I wouldn't want raidz1 so I'll stay away from that idea. I'll stick with my 8 disk pool for now. I might put it all in a R510 enclosure to get it off of VM.