SOLVED Single pool with several JBODs?

[blanchet]

Hi,

I would like to create a single ZFS pool that spreads over several JBODs. Each time, I need more space I will buy a new JBODs, create some vdev inside and then extend the pool. Because it is an uncommon setup, I would appreciate some feedback for the project feasibility.

For the moment, the main difficulty I have noticed:

• an issue with one JBODs, while the FreeNAS controller is still running, may destroy the pool.

Nevertheless, but I think that I could overcome this difficulty with redundant JBODs, that imply the following hardware:

• JBODs with dual power supply and 2 serious UPS
• two HBA SAS (LSI-9300-8e) on the FreeNAS controller to setup multipath disks.
• JBODs with a dual expander
• only SAS disks

Do I have forget something ?

 

[Johnny Fartpants]

That all sounds ok but how much space do you need/want?

Would you not be better off with one or two big 60/90 bay JBODs with 12/14TB drives?

How do you plan on backing up the data or is this unit for backup?

 

[jgreco]

This isn't an uncommon setup. It is just uncommon for forum users, who tend to be home users.

Note that multipath implies SAS disks.

 

[HoneyBadger]

Because it is an uncommon setup, I would appreciate some feedback for the project feasibility.

...

Do I have forget something ?

Uncommon but not unusual. Redundant JBODs are the rule for any kind of commercial storage solution.

It looks like you've got the necessary prerequisites for this. Make sure you keep both HBAs on the same firmware, and perform some failure testing in addition to the usual burn-in testing (eg: pull a SAS cable while the system is running)

 

[blanchet]

I would like to start with a SuperMicro 4U 44-bay JBODs divided in 4 or 5 vdevs.
I will use only raidz3 because resilvering 12TB disks take a lot of time.
Every two years, I would like to extend the system with a new JBODs, until I reach about 1 PB.

This system will be used for backup. To secure the data, I plan to use ZFS replication to an identical system located on our disaster recovery site.

Very large JBODs may be interesting to increase density but indeed I do not really like them because:

• they are very heavy, and require a very deep rack
• you have to pull the server to access the disks

According to the above answer, my project is feasible with redundant JBODs and multipath disks.
Thank for your help