how many disks we can attach for single ZFS pool

[venkata]

Dear all,

we are planing to create ZFS pool with 24 disks [4 TB].

Is FreeNAS-9.10.2-U1 version can support 24 disks.

 

[nojohnny101]

FreeNAS certainly can support 24 disks (many more). What you would not want to do is put all of those in a single vdev.

 

[venkata]

FreeNAS certainly can support 24 disks (many more). What you would not want to do is put all of those in a single vdev.

Thank you so much for response.
We are planning to create single vdev only. But while vdev creating time it showing 15 disk numbers only. So that i got doubt on it, whether it can support or not?
pls check attached screen shot.

 

[nojohnny101]

We are planning to create single vdev only.

Don't do that, you'll regret it.

As per forum rules, please post full hardware specs and what version of FreeNAS your'e using.

 

[venkata]

Don't do that, you'll regret it.

As per forum rules, please post full hardware specs and what version of FreeNAS your'e using.

Thank you so much for your response.
As per rules, we have to share hardware specification for any issues. however here i am sharing my setup.
My hardware is:
CPU-E5 2020 V4 - 1numb
Mother board - ASus Z10PAD8
RAM - 32 GB DDR4 ECC
SATA DOM - 16 GB for FreeNAS OS
Hard disks - 4 TB SATA HDD enterprise 120 MB cache X 24 numbers
HBA card - LSI SAS 9305-24i Host Bus Adapter

Is this hardware sufficient for 24 disks single vDev?

 

[nojohnny101]

Your RAM is too low for the amount of raw storage you are planning. I would bump that up to at least 64GB.

I don't have personal experience with HBAs so I will defer on that one. Your CPU is good and you motherboard is fine.

 

[Spearfoot]

Thank you so much for your response.
As per rules, we have to share hardware specification for any issues. however here i am sharing my setup.
My hardware is:
CPU-E5 2020 V4 - 1numb
Mother board - ASus Z10PAD8
RAM - 32 GB DDR4 ECC
SATA DOM - 16 GB for FreeNAS OS
Hard disks - 4 TB SATA HDD enterprise 120 MB cache X 24 numbers
HBA card - LSI SAS 9305-24i Host Bus Adapter

Is this hardware sufficient for 24 disks single vDev?

You don't want to use 24 disks in a single vdev! Is it possible you're confusing a pool/volume with a vdev? A pool is made up of one or more vdevs, where a vdev is made up of one or more disks. A single-disk vdev offers no redundancy, so you want to use either mirrors or RAIDZ1, RAIDZ2, or RAIDZ3.

With 24 disks, you could configure a pool made up of:

1. 4 vdevs, each containing 6 disks in RAIDZ2
2. 3 vdevs, each containing 8 disks in RAIDZ2 or RAIDZ3
3. 12 vdevs, each containing 2 disks in a mirror

...or any number of other designs.

You probably don't want to use that expensive LSI 9305-24i card, either. With an expander backplane, you can connect all 24 drives to a single LSI 9210/9211 or IBM M1015 or Dell H200/H310 -- and save a lot of money.

 

[gpsguy]

How do you plan to use the storage?

For general use consider Spearfoot's example #1 or #2. If you will be using it as an iSCSI target, #3 would be a better choice.

 

[venkata]

You don't want to use 24 disks in a single vdev! Is it possible you're confusing a pool/volume with a vdev? A pool is made up of one or more vdevs, where a vdev is made up of one or more disks. A single-disk vdev offers no redundancy, so you want to use either mirrors or RAIDZ1, RAIDZ2, or RAIDZ3.

With 24 disks, you could configure a pool made up of:

1. 4 vdevs, each containing 6 disks in RAIDZ2
2. 3 vdevs, each containing 8 disks in RAIDZ2 or RAIDZ3
3. 12 vdevs, each containing 2 disks in a mirror

...or any number of other designs.

You probably don't want to use that expensive LSI 9305-24i card, either. With an expander backplane, you can connect all 24 drives to a single LSI 9210/9211 or IBM M1015 or Dell H200/H310 -- and save a lot of money.

Thank you so much for your reply..
1. For our case we have to use 4 number of LSI 9210/9211 cards.. instead of that, we are planing to use single 9305-24i card. Because our 24-bay enclouser having 6 mini SAS ports [back plane].
2. We are planing to use for file sharing services [CIFS/SMB], so we are planing to create 1 vDev containing 24 disks in RAIDZ1 [or RAIDZ2]. I need suggestions for this, weather vDev can support 24 disks or not.

 

[Spearfoot]

Thank you so much for your reply..
1. For our case we have to use 4 number of LSI 9210/9211 cards.. instead of that, we are planing to use single 9305-24i card. Because our 24-bay enclouser having 6 mini SAS ports [back plane].

Yes, I understand. This is a common situation, a prime example being Supermicro's 24-bay 4U chassis that come with a direct-attach backplane with 6 SAS ports, and therefore requiring 3 (not 4) of the LSI 9210/9211 cards.

2. We are planing to use for file sharing services [CIFS/SMB], so we are planing to create 1 vDev containing 24 disks in RAIDZ1 [or RAIDZ2]. I need suggestions for this, weather vDev can support 24 disks or not.

Whether you use 3 of the 9210/9211 or a single 9305; it's a very bad idea to put all 24 drives in a single vdev. That's far too wide! Consider:

1. If a drive fails and you have to replace it, the system will have to read & write every drive in the vdev during resilvering
2. The huge array will have the random IOPS performance of a single disk

For general-purpose file sharing I suggest you create a pool made up either of:

1. 4 vdevs, each containing 6 drives in RAIDZ2
2. 3 vdevs, each containing 8 drives in RAIDZ2
3. 2 vdevs, each containing 12 drives in RAIDZ3

Under no circumstances should you use RAIDZ1 with 4TB drives; RAIDZ1 is not recommended when using drives > 1TB in size.

Good luck!
REFERENCES:

• "ZFS RAIDZ stripe width, or: How I Learned to Stop Worrying and Love RAIDZ"
• "ZFS: you should use mirror vdevs, not RAIDZ"
• "A Closer Look at ZFS, Vdevs, and Performance"