zllovesuki
Cadet
- Joined
- Aug 13, 2014
- Messages
- 4
Greetings! After many nights of interweb Googling and searching, I'm still indecisive about the configuration.
Currently I have:
Xeon E3-1230 V3
4 x 8GB Crucial 1600MHz ECC CAS 11
ASRock E3C224
Corsair CX430 80+ bronze (thinking of getting a gold, but too much wattage to be efficient)
6 x 2TB WD Red (2 more will be here tomorrow)
Corsair C70 case with 5.25 hard drives bay
Switch:
TP-Link TL-SG2424 (all machines LACP)
Clutch:
APC Smart-UPS 1000VA
Previously I was running ZFS on Linux and serving Netatalk (Time Machine and file access to the whole house) + NFS (XenServer on another E3), I realized that NFS was painfully slow. Yep, sync writes. I added a SanDisk Ultra Plus for ZIL, much better. Yet I will be adding a PCIe SSD for ZIL (should be here Friday).
Then strange things happened. (Spinners') IO utilizations were all about 80% when they are doing Time Machine, and when I tried to stream via Nginx, it stutters (apparently the hard drives couldn't keep up). Network problem is eliminated, CPU was around 50% when 3 of us are doing time machine and streaming, memory was around 24GB.
Now throw that out of the mind, I will be adding two more hard drives to the zpool. Before that, I am waiting for 3 x 3TB Seagate coming for offsite backup. I will back the files to another NAS then rebuild the spool.
Now the real question is, what configuration to go with 8 x 2TB? I will be using FreeNAS (ZoL is dumb: SPL and VFS don't like each other--double caching.)
Workload: Moderate amount of random reads and large amount of sequential reads. Random (sync) writes are out of the mind as I will be adding a ZIL. https://calomel.org/zfs_raid_speed_capacity.html shows that RAIDZ2 is a better choice for a balance of sequential performance and capacity in the case of ~8 hard drives. However I read it off somewhere that I will only have one disk of random read performance if I do a single RAIDZ2/Z. I am thinking of RAID 10 (4 vdevs, each has two disks mirror), RAID 60 (2 vdevs, each has four disks RAIDZ2), or just plain old RAIDZ2. Capacity is not the concern, however, I cannot seem to find a real number concerning about random performance between a stripe of mirrors and RAIDZ/Z2. Data integrity is not a burning concern as I will have 2 x offsite cold backup.
Or, just SAME (Stripe And Mirror Everything)
Thanks guys.
Currently I have:
Xeon E3-1230 V3
4 x 8GB Crucial 1600MHz ECC CAS 11
ASRock E3C224
Corsair CX430 80+ bronze (thinking of getting a gold, but too much wattage to be efficient)
6 x 2TB WD Red (2 more will be here tomorrow)
Corsair C70 case with 5.25 hard drives bay
Switch:
TP-Link TL-SG2424 (all machines LACP)
Clutch:
APC Smart-UPS 1000VA
Code:
[root@Hypervisor ~]# zpool status pool: Storage state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM Storage ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 scsi-SATA_WDC_WD20EFRX-68_WD-WMC4M2912068 ONLINE 0 0 0 scsi-SATA_WDC_WD20EFRX-68_WD-WCC4M3028400 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 scsi-SATA_WDC_WD20EFRX-68_WD-WCC4M3089372 ONLINE 0 0 0 scsi-SATA_WDC_WD20EFRX-68_WD-WCC4M3096188 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 scsi-SATA_WDC_WD20EFRX-68_WD-WCC4M0778730 ONLINE 0 0 0 scsi-SATA_WDC_WD20EFRX-68_WD-WMC4M1695775 ONLINE 0 0 0 logs scsi-SATA_SanDisk_SDSSDHP140874400287-part1 ONLINE 0 0 0 errors: No known data errors
Previously I was running ZFS on Linux and serving Netatalk (Time Machine and file access to the whole house) + NFS (XenServer on another E3), I realized that NFS was painfully slow. Yep, sync writes. I added a SanDisk Ultra Plus for ZIL, much better. Yet I will be adding a PCIe SSD for ZIL (should be here Friday).
Then strange things happened. (Spinners') IO utilizations were all about 80% when they are doing Time Machine, and when I tried to stream via Nginx, it stutters (apparently the hard drives couldn't keep up). Network problem is eliminated, CPU was around 50% when 3 of us are doing time machine and streaming, memory was around 24GB.
Now throw that out of the mind, I will be adding two more hard drives to the zpool. Before that, I am waiting for 3 x 3TB Seagate coming for offsite backup. I will back the files to another NAS then rebuild the spool.
Now the real question is, what configuration to go with 8 x 2TB? I will be using FreeNAS (ZoL is dumb: SPL and VFS don't like each other--double caching.)
Workload: Moderate amount of random reads and large amount of sequential reads. Random (sync) writes are out of the mind as I will be adding a ZIL. https://calomel.org/zfs_raid_speed_capacity.html shows that RAIDZ2 is a better choice for a balance of sequential performance and capacity in the case of ~8 hard drives. However I read it off somewhere that I will only have one disk of random read performance if I do a single RAIDZ2/Z. I am thinking of RAID 10 (4 vdevs, each has two disks mirror), RAID 60 (2 vdevs, each has four disks RAIDZ2), or just plain old RAIDZ2. Capacity is not the concern, however, I cannot seem to find a real number concerning about random performance between a stripe of mirrors and RAIDZ/Z2. Data integrity is not a burning concern as I will have 2 x offsite cold backup.
Or, just SAME (Stripe And Mirror Everything)
Thanks guys.
Last edited: