Avoiding Pitfalls When Expanding Storage For First TIme

[I3lackR0se]

I've been running my FreeNAS server with no horrible issues for a couple years now, and I'm ready to expand my storage. After looking up what information I could, I had a few outstanding questions before I went forward with doing it.

My current build:
Motherboard: Supermicro X10SL7-F-O
CPU: Xeon E3-1241 V3
HDD: Western Digital Red WD40EFRX 4 TB (x6, put into a single Z2 vdev)
Case: Fractal Design Refine R4
RAM: Crucial 16GB Kit (8GBx2) DDR3 DR x8 ECC UDIMM CT2KIT102472BD160B
OS Drives: 64GB ADATA Synchronous NAND SSD(x2)
Cooling: Cougar Vortex PWM 120mm Fan (x4)
PWM Fan Splitter
PSU: SeaSonic G Series SSR-550RM 550W 80 Plus Gold
UPS: Cyberpower CP Office Tower 1500AVRLCD1500VA
OS: FreeNAS 9.10
Server use: Running small Plex and OwnCloud setup for family (no more than 4 concurrent users)

I picked up six Western Digital Red WD80EFZX 256MB Cache 8TB drives, and I am about to pick up another CT2KIT102472BD160B to bring me up to 32GB of RAM. Ideally, I'd like to expand my only existing pool (which has a single vdev made up of six 4TB WD Reds). The three questions I couldn't find reliable answers to:

1) Will 32GB of RAM be enough to operate a 4 concurrent user Plex and OwnCloud setup after I upgrade by 24TB of HDDs to 72TB (or, since it's Z2, 14.55TB usable space to 43.65TB usable space)?

There's a rule of thumb mentioned some-what ubiquitously about 1GB of RAM for 1TB of storage, but I found several other places that indicated that doesn't really hold up at higher storage sizes (but didn't give good indications of what more accurate limits would be).

2) Can I had a vdev of six 8TB drives to a pool with a vdev six 4TB drives, or can the pool only be expanded if the vdevs are identical in size and drive amount? Nothing I read seemed to say this couldn't be done, but I couldn't find anything in the affirmative either.

3) Other than plugging the drives into the mobo and using the Volume Manager to add to extend my existing volume, are there any additional steps I need to take to safely add a new Z2 vdev to my pool?

Thanks for taking the time to read that massive block of text, and sorry in advance if I managed to miss something easily found in the documentation

 

[BigDave]

3) Other than plugging the drives into the mobo and using the Volume Manager to add to extend my existing volume, are there any additional steps I need to take to safely add a new Z2 vdev to my pool?

No, that's pretty much all thereis to it...
BUT, I have two recommendations for you.

1. Read some more about ZFS configuration, there's a ton of questions
already asked and answered here on this subject. The forum's search
feature is your friend.
2. Back up your two years of data before doing anything.

When I wanted more space for data, I replaced each drive (one at a time) with
one of larger capacity until all the drives were replaced. Because my pool was
configured as RAIDz2 this was very little risk and when the last drive is
finished getting replaced, the capacity of the pool expanded! If you proceed
with this method, you will get the increased space without buying a new case
or trying to cool 12 drives in the one you currenty have (Define R4).

Mixing drive capacities between Vdevs is fine but try and match spindle speeds. This is my opinion, and may not considered best practice by some
members...

 

[Stux]

I've been running my FreeNAS server with no horrible issues for a couple years now, and I'm ready to expand my storage. After looking up what information I could, I had a few outstanding questions before I went forward with doing it.

My current build:
Motherboard: Supermicro X10SL7-F-O
CPU: Xeon E3-1241 V3
HDD: Western Digital Red WD40EFRX 4 TB (x6, put into a single Z2 vdev)
Case: Fractal Design Refine R4
RAM: Crucial 16GB Kit (8GBx2) DDR3 DR x8 ECC UDIMM CT2KIT102472BD160B
OS Drives: 64GB ADATA Synchronous NAND SSD(x2)
Cooling: Cougar Vortex PWM 120mm Fan (x4)
PWM Fan Splitter
PSU: SeaSonic G Series SSR-550RM 550W 80 Plus Gold
UPS: Cyberpower CP Office Tower 1500AVRLCD1500VA
OS: FreeNAS 9.10
Server use: Running small Plex and OwnCloud setup for family (no more than 4 concurrent users)

I picked up six Western Digital Red WD80EFZX 256MB Cache 8TB drives, and I am about to pick up another CT2KIT102472BD160B to bring me up to 32GB of RAM.

Was going to suggest the ram upgrade :)

Ideally, I'd like to expand my only existing pool (which has a single vdev made up of six 4TB WD Reds). The three questions I couldn't find reliable answers to:

1) Will 32GB of RAM be enough to operate a 4 concurrent user Plex and OwnCloud setup after I upgrade by 24TB of HDDs to 72TB (or, since it's Z2, 14.55TB usable space to 43.65TB usable space)?

It will have to be enough since that's your system's limit ;)
If should probably be fine though.

There's a rule of thumb mentioned some-what ubiquitously about 1GB of RAM for 1TB of storage, but I found several other places that indicated that doesn't really hold up at higher storage sizes (but didn't give good indications of what more accurate limits would be).

It's much more important what your 're doing rather than how many sectors on your disk are used. So iSCSI, that needs another 8GB, plex, that needs another 8GB. So, 32GB should be good.

Other than the minimum 8GB which is about preventing lockups, these RAM recommendations are about not crippling performance, but non-optimal performance still works.

2) Can I had a vdev of six 8TB drives to a pool with a vdev six 4TB drives, or can the pool only be expanded if the vdevs are identical in size and drive amount? Nothing I read seemed to say this couldn't be done, but I couldn't find anything in the affirmative either.

Yep. No problem, and quite normal. And then one day you replace your 4TB drive vdev with 16TB drives ;)

3) Other than plugging the drives into the mobo and using the Volume Manager to add to extend my existing volume, are there any additional steps I need to take to safely add a new Z2 vdev to my pool?

Follow the HD burnin / testing instructions and burn in your new HDs before adding them to your pool. @Spearfoot has a good script to do it.

Thanks for taking the time to read that massive block of text, and sorry in advance if I managed to miss something easily found in the documentation

:)

 

[I3lackR0se]

No, that's pretty much all thereis to it...
BUT, I have two recommendations for you.

1. Read some more about ZFS configuration, there's a ton of questions
already asked and answered here on this subject. The forum's search
feature is your friend.
2. Back up your two years of data before doing anything.

When I wanted more space for data, I replaced each drive (one at a time) with
one of larger capacity until all the drives were replaced. Because my pool was
configured as RAIDz2 this was very little risk and when the last drive is
finished getting replaced, the capacity of the pool expanded! If you proceed
with this method, you will get the increased space without buying a new case
or trying to cool 12 drives in the one you currenty have (Define R4).

Mixing drive capacities between Vdevs is fine but try and match spindle speeds. This is my opinion, and may not considered best practice by some
members...

I'll do that. I wasn't specifically searching for ZFS configuration, so that might of been a large part of my problem. Really good to know about being able to replace the drives one at a time like that to upgrade, I figured the process to do something in that manner would be more involved than that. I'm not too worried about heat or space in the R4, but I'll double check before I commit to anything and consider doing as you suggest if I run into trouble. Thanks a lot!

Was going to suggest the ram upgrade :)



It will have to be enough since that's your system's limit ;)
If should probably be fine though.



It's much more important what your 're doing rather than how many sectors on your disk are used. So iSCSI, that needs another 8GB, plex, that needs another 8GB. So, 32GB should be good.

Other than the minimum 8GB which is about preventing lockups, these RAM recommendations are about not crippling performance, but non-optimal performance still works.



Yep. No problem, and quite normal. And then one day you replace your 4TB drive vdev with 16TB drives ;)



Follow the HD burnin / testing instructions and burn in your new HDs before adding them to your pool. @Spearfoot has a good script to do it.



:)

Awesome, good to know. I'll definitely be sure to run that on them before adding to the pool. Thank you so much for clearing up those points! I appreciate it alot