Hi,
Another new soon to be FreeNAS convert.
I've been running hardware RAID (both 5 and 6) with Windows servers for the past 10 odd years.
Last year moved from hardware RAID over to FlexRaid, but I don't have 100% confidence in it and started to worry about things like bitrot.
Although I'm much more of a Windows person I like a lot of the ZFS features and FreeNAS seemed the most logical choice.
So I've swapped my current hardware (X9SCM, E3-1230, 16GB RAM, Areca 1882 RAID card) for the following;
Chenbro 9U 50 bay case
Supermicro X10SRL-F
E5-1620V3 (and SNK-P0048AP4 cooler)
32GB DIMM (MEM-DR432L-SL01-ER21) - I've actually got 2 of these on the way but want to test performance with one first
1 x M1015 - which needs to be flashed to IT mode
1 x Chenbro CK23601 36 Port SAS Expander
2 x Intel S3500 120GB SSD's for the mirrored OS - I wanted SATADOM's but they aren't easy to find in the UK and the SSD's were cheaper
20 x Hitachi 4TB drives
X540-T2 10GBe NIC
I have a 24 bay xcase 4U case, but I've just got a good deal on a 50 bay Chenbro case, so will be using that instead - it also gives me the expansion I was looking for.
The server is being used purely for hosting media. I have another machine for running, Plex, Sickbeard etc, so I don't plan to run any jails on this machine.
My current thinking is to setup 2 vdevs using 10 x 4TB in RAIDZ2.
Once I've filled that up (this will be filled up by migrating the data off my existing boxes), I'll add another 10 x 5TB RAIDZ2 vdev.
Important things like photos, documents etc are backed up to two other machines and crashplan.
The majority of the space will be filled up with Blu-ray and DVD rips (where the physical disk is the backup).
Therefore I have a few questions;
1) I currently have 1 M1015 and 1 Chenbro expander, but in order to connect all of the bays up I'll need at least one other expander. The Chenbro ones appear impossible to find at the moment. Therefore I appear to have 2 options;
6 x M1015 HBA's
1 x M1015 HBA and 2 x HP SAS Expander
I haven't seen a great deal on this forum with regards to using the HP expander and compatibility (aside from the suggestion that the Chenbro has an LSI chipset and is therefore better).
Can I use the HP expander or is it likely to lead to issues?
Are the 6 HBA's a better option?
2) Buying another server to backup the Blu-ray rips isn't going to be cost effective, but ideally I really won't want to rip them all over again. How safe does the above config look? In reality would it benefit much by using RAIDZ3 instead (that would mess up my 5 vdev plan in a 50 drive case)? I recognise it's a how long is a piece of string question, but I am looking more for confidence levels.
I don't see it mentioned very often but would it be wise to have multiple pools to remove the risk of a catastrophic failure, rather than just a single one with multiple vdevs?
3) How hard and fast is the 80%/90% full rule for vdev's before performance starts to suffer? It seems an awful lot of space to keep free (20TB in a 100TB machine). Does performance drop off a cliff or is it a gradual decline?
Reading through the forum I've seen mention of not getting close to 95% due to not being able to allocate space in order to delete data.
4) Is the plan to start with 2 vdevs and then add new ones when the space is required the best way to go? I would imagine following this path the server would ultimately end up with storage requirments closer to or above 200TB.
5) Is there a preferred method backing up between FreeNAS and Windows machines? I've seen people mention BTSync, rSync, Robocopy on the forum, depending on the direction of the copy.
6) I believe FreeNAS is hardware independent, so if I changed motherboard (if it failed) this wouldn't be an issue?
Similarly, if I start with a 24 bay case, get things up and running, then move to a 50 bay case, is there anything I would need to do? Move drives in a specific order, i.e. they would need to be in the same drive slots when moved?
7) Outside of running regular SMART diagnostics (notified via email) and scrubs, is there any other routine maintenance that should be setup?
8) I need to do more research on this but I believe bad blocks in the norm for drive testing before utilising in production. The first batch I can do before I start using the server, but once it's up and running, how do people then test more drives? Do you use another machine?
Thanks in advance.
Another new soon to be FreeNAS convert.
I've been running hardware RAID (both 5 and 6) with Windows servers for the past 10 odd years.
Last year moved from hardware RAID over to FlexRaid, but I don't have 100% confidence in it and started to worry about things like bitrot.
Although I'm much more of a Windows person I like a lot of the ZFS features and FreeNAS seemed the most logical choice.
So I've swapped my current hardware (X9SCM, E3-1230, 16GB RAM, Areca 1882 RAID card) for the following;
Chenbro 9U 50 bay case
Supermicro X10SRL-F
E5-1620V3 (and SNK-P0048AP4 cooler)
32GB DIMM (MEM-DR432L-SL01-ER21) - I've actually got 2 of these on the way but want to test performance with one first
1 x M1015 - which needs to be flashed to IT mode
1 x Chenbro CK23601 36 Port SAS Expander
2 x Intel S3500 120GB SSD's for the mirrored OS - I wanted SATADOM's but they aren't easy to find in the UK and the SSD's were cheaper
20 x Hitachi 4TB drives
X540-T2 10GBe NIC
I have a 24 bay xcase 4U case, but I've just got a good deal on a 50 bay Chenbro case, so will be using that instead - it also gives me the expansion I was looking for.
The server is being used purely for hosting media. I have another machine for running, Plex, Sickbeard etc, so I don't plan to run any jails on this machine.
My current thinking is to setup 2 vdevs using 10 x 4TB in RAIDZ2.
Once I've filled that up (this will be filled up by migrating the data off my existing boxes), I'll add another 10 x 5TB RAIDZ2 vdev.
Important things like photos, documents etc are backed up to two other machines and crashplan.
The majority of the space will be filled up with Blu-ray and DVD rips (where the physical disk is the backup).
Therefore I have a few questions;
1) I currently have 1 M1015 and 1 Chenbro expander, but in order to connect all of the bays up I'll need at least one other expander. The Chenbro ones appear impossible to find at the moment. Therefore I appear to have 2 options;
6 x M1015 HBA's
1 x M1015 HBA and 2 x HP SAS Expander
I haven't seen a great deal on this forum with regards to using the HP expander and compatibility (aside from the suggestion that the Chenbro has an LSI chipset and is therefore better).
Can I use the HP expander or is it likely to lead to issues?
Are the 6 HBA's a better option?
2) Buying another server to backup the Blu-ray rips isn't going to be cost effective, but ideally I really won't want to rip them all over again. How safe does the above config look? In reality would it benefit much by using RAIDZ3 instead (that would mess up my 5 vdev plan in a 50 drive case)? I recognise it's a how long is a piece of string question, but I am looking more for confidence levels.
I don't see it mentioned very often but would it be wise to have multiple pools to remove the risk of a catastrophic failure, rather than just a single one with multiple vdevs?
3) How hard and fast is the 80%/90% full rule for vdev's before performance starts to suffer? It seems an awful lot of space to keep free (20TB in a 100TB machine). Does performance drop off a cliff or is it a gradual decline?
Reading through the forum I've seen mention of not getting close to 95% due to not being able to allocate space in order to delete data.
4) Is the plan to start with 2 vdevs and then add new ones when the space is required the best way to go? I would imagine following this path the server would ultimately end up with storage requirments closer to or above 200TB.
5) Is there a preferred method backing up between FreeNAS and Windows machines? I've seen people mention BTSync, rSync, Robocopy on the forum, depending on the direction of the copy.
6) I believe FreeNAS is hardware independent, so if I changed motherboard (if it failed) this wouldn't be an issue?
Similarly, if I start with a 24 bay case, get things up and running, then move to a 50 bay case, is there anything I would need to do? Move drives in a specific order, i.e. they would need to be in the same drive slots when moved?
7) Outside of running regular SMART diagnostics (notified via email) and scrubs, is there any other routine maintenance that should be setup?
8) I need to do more research on this but I believe bad blocks in the norm for drive testing before utilising in production. The first batch I can do before I start using the server, but once it's up and running, how do people then test more drives? Do you use another machine?
Thanks in advance.