What Performance Can I Expect?

[cmcigas]

Hi, I have used FreeNAS for a while now for home storage and Plex use but recently I was able to take home from work an HP server and a storage shelf. I have 30x 600GB 10K 2.5" drives in these systems total and all in one raid, RaidZ2.

I was wondering if someone knew what the max read/write speeds I would get out of these drives in this raid.

I was able to take home an Arista switch too with four 10Gb ports on it so I was curious if those drives would even write close to 10Gbps with that raid. Read doesn't really matter I guess.

I know it relies on my read speed from my PC to the server as well but I was planning on getting an NVMe SSD around 1800MBps which would be more than enough I believe to read at the speed of 10Gbps across the network.

Thank you, sorry if this is the wrong to place to post this too.

 

[Jessep]

RaidZ2 arrays are not suggested past ~10 drives, ideal is more around 6. If you want more performance I would suggest 5x(6 drive RaidZ2).
With that and enough RAM you would be able to come close to filling a 10Gb pipe, SMB performance might depend on CPU somewhat.

 

[cmcigas]

RaidZ2 arrays are not suggested past ~10 drives, ideal is more around 6. If you want more performance I would suggest 5x(6 drive RaidZ2).
With that and enough RAM you would be able to come close to filling a 10Gb pipe, SMB performance might depend on CPU somewhat.

So I actually never created more than one drive array in FreeNAS, how will Plex work with it? Will I have to have multiple sources in my mount points?

EDIT: To continue on the question above, multiple sources would mean multiple data sets? Which would mean multiple SMB shares correct?

 

[Chris Moore]

So I actually never created more than one drive array in FreeNAS, how will Plex work with it? Will I have to have multiple sources in my mount points?

EDIT: To continue on the question above, multiple sources would mean multiple data sets? Which would mean multiple SMB shares correct?

You can have multiple vdevs in a single storage pool. I think you need to go back and review some resources:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://www.ixsystems.com/community...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://www.ixsystems.com/community/threads/terminology-and-abbreviations-primer.28174/

I was wondering if someone knew what the max read/write speeds I would get out of these drives in this raid.

If you put all 30 drives into a single vdev, that would limit you to roughly the performance of a single drive. This is not the way pools should be created. A pool can be made of one or more vdevs. I have several systems where I work that contain ten vdevs.

 

[Chris Moore]

I was able to take home an Arista switch too with four 10Gb ports on it so I was curious if those drives would even write close to 10Gbps with that raid.

The drives, if laid out in the correct pool configuration, should easily support 10Gb speed.

 

[Chris Moore]

Please review the guidance here: https://www.ixsystems.com/community/threads/forum-guidelines.45124
Then give us as much detail about the hardware you have as possible.

 

[cmcigas]

Please review the guidance here: https://www.ixsystems.com/community/threads/forum-guidelines.45124
Then give us as much detail about the hardware you have as possible.

Thank you, Im going over everything else you posted as well. As for the specs, I have an HP DL380p Gen 8 and a HP Storageworks Gen 6 I believe connected via SAS cable. The DL380p has two E5-2640's with 128GB of RAM. Hard drives are a mix of the Gen 8 and older versions of 600GB 10K 2.5", model of the drive is EG0600JEHMA, 30 of them. My boot drive is a USB stick from Kingston. Controller is HP Smart Array P822 Controller. And my 1Gb NIC is integrated, the 10Gb NIC is an Intel PCIe card. I do not have any info on it though right now. All drives are RAID 0 for the server but are configured at RaidZ2 from FreeNAS

 

[cmcigas]

The drives, if laid out in the correct pool configuration, should easily support 10Gb speed.

Ok so I never knew what the 'Add Data' button meant in the pool creation page before all those links. So from what I read and what I am understanding, I should:

Back up my data off the server
Delete my pool
Create a new pool using separate vdevs by clicking Add Data.

Since I have 30 disks, I am guessing RaidZ2 for each VDev, 5 VDevs 6 disks each?

 

[Chris Moore]

All drives are RAID 0 for the server

Controller is HP Smart Array P822 Controller.

That is going to kill your performance and put your data at risk. You need a SAS HBA, not a hardware RAID card. That way ZFS and FreeNAS can have full, direct access to the drives. Passing them through a RAID controller can cause all kinds of problems.

 

[Chris Moore]

Back up my data off the server
Delete my pool
Create a new pool using separate vdevs by clicking Add Data.

Since I have 30 disks, I am guessing RaidZ2 for each VDev, 5 VDevs 6 disks each?

Yes, I use vdevs of 6 drives. I have a system at work with sixty drives in ten vdevs and it works great. You would end up with five vdevs.
However, before you create the new pool with the new configuration, you should get an HBA, per my previous post.

 

[cmcigas]

Yes, I use vdevs of 6 drives. I have a system at work with sixty drives in ten vdevs and it works great. You would end up with five vdevs.
However, before you create the new pool with the new configuration, you should get an HBA, per my previous post.

Thank you, at least I know what to do now.
Im trying to find one now, just don't feel like spending $100+ right now. Im guessing I need one that is atleast 10Gbps for performance?

EDIT: Also need to find one that has internal and external facing ports because of the HP Storageworks I have connected

 

[Chris Moore]

I like this model controller card, I have four of them personally, not counting the systems at work:
https://www.ebay.com/itm/HP-H220-6G...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162862201664
This seller is very good at flashing them with the correct firmware and testing them for correct operation.

 

[Chris Moore]

I need one that is atleast 10Gbps for performance?

No. The scale of measure is different between network traffic and disk traffic. The card I linked is more than fast enough.

 

[cmcigas]

I like this model controller card, I have four of them personally, not counting the systems at work:
https://www.ebay.com/itm/HP-H220-6G...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162862201664
This seller is very good at flashing them with the correct firmware and testing them for correct operation.

Ah, awesome, thank you, I edited my last comment too late but I need a external facing one as well because of my HP Storageworks shelf too. Would having two HBA cards work? One for internal SAS cables and a second for the SAS cable to the storageworks?

 

[cmcigas]

Looks like this would be good, if I can use two separate cards?
https://www.ebay.com/itm/LSI-9200-8...:USPSPriorityMailSmallFlatRateBox!08817!US!-1

 

[Chris Moore]

Would having two HBA cards work?

Certainly. I have one of the servers at work that has four disk shelves attached on an external controller in addition to the internal drives on a controller integrated in the system board.

 

[Chris Moore]

Depending on the number of cable connections you need, this might even work:
https://www.ebay.com/itm/Genuine-LS...0-IT-mode-for-ZFS-FreeNAS-unRAID/163197898846

 

[cmcigas]

Certainly. I have one of the servers at work that has four disk shelves attached on an external controller in addition to the internal drives on a controller integrated in the system board.

Awesome thank you so much for the help. I may actually have 1 of these cards at home, Ill check that out, and start transferring everything.

I do have one more question kind of related if you dont mind. Since I will have a lower capacity now due to 5 different RaidZ2s, my thought of getting a second shelf is interesting to me too. From the links you provided before, it seems I can make another VDev with different drive sizes right? So I should be able to get a second storage shelf with 3.5" bays with 1TB or higher drives and then add another VDev to my pool? Or am I seeing that wrong? Thanks

 

[Chris Moore]

From the links you provided before, it seems I can make another VDev with different drive sizes right?

Yes. I had one vdev of 2TB drives and one vdev of 4TB drives in my home system for a while when I was migrating to larger drives. The capacity of drives in a vdev is limited to the smallest drive, but all drives do not need to be the same size. Each vdev is independent with regard to redundancy, but if a vdev fails it will take the pool down, so the recommendation is to have all vdevs at the same level of redundancy. Performance also tends to be more consistent if all vdevs have the same number of disks.

So I should be able to get a second storage shelf with 3.5" bays with 1TB or higher drives and then add another VDev to my pool?

Absolutely. That server I have at work has 60 internal drives:

Then it has four more disk shelves attached. You can go wild with the disks. That server has 124 drives connected to it.

 

[cmcigas]

Yes. I had one vdev of 2TB drives and one vdev of 4TB drives in my home system for a while when I was migrating to larger drives. The capacity of drives in a vdev is limited to the smallest drive, but all drives do not need to be the same size. Each vdev is independent with regard to redundancy, but if a vdev fails it will take the pool down, so the recommendation is to have all vdevs at the same level of redundancy. Performance also tends to be more consistent if all vdevs have the same number of disks.

Absolutely. That server I have at work has 60 internal drives:
View attachment 31203
Then it has four more disk shelves attached. You can go wild with the disks. That server has 124 drives connected to it.

Awesome, this great, thanks so much for the help. Once I get this all figured out Ill start looking into another shelf. Thanks!