eexodus
Dabbler
- Joined
- Aug 31, 2016
- Messages
- 39
Background
I'm a PC systems specialist at a university and second-in-command for a department of ~150 users. We're looking to replace our aging 3-2-1 storage solution. Currently we have an old Xserve running OS X which mainly acts as an LDAP server. We don't plan on using FreeNAS to replace it, instead we will likely buy a Dell server running Debian/Ubuntu or openSUSE and OpenLDAP.
Multiple hardware RAID arrays (4 HDD bays) are plugged up to the Xserve over FireWire 800/eSATA. User home directories (infrequently used) and SMB/AFP shares (frequently used) use these arrays which total ~50TB. A second set of arrays are plugged up to the Xserve and act as a primary backup (another ~50TB). A third set of arrays are plugged up to a Mac mini server off-site which acts as a secondary backup (another ~50TB). We plan to replace these arrays with FreeNAS pools; we have access to 10Gb/s so bandwidth between FreeNAS and the Linux server should be good (the pool shares would be mounted as network drives on the Linux server).
In the past, my boss has just added more arrays as needed. It is becoming a headache for a few reasons:
We considered cloud block storage for primary/secondary backups
...but we decided against it. If anyone has experience in this area I'd be extremely interested in any insight. AWS EBS would be $0.045 per GB per month (not including the cost of an AWS EC2 instance required to attach it to) and Google Cloud would be $0.026/GB/mo. Compared to a $10,000 FreeNAS server that will last 5-8 years it just doesn't compare. For example, Google Cloud would cost $1,300/mo for 50 TB/mo. Maybe I'm missing something but cloud block storage doesn't seem "there yet" to replace local storage for small/medium businesses like our department. When compared to pricey SANs or NASs with rock-solid warranties, maybe, but I feel our unique situation of having free electricity, bandwidth, rack space, battery/generator backups, plenty of free time, etc. pushes us towards local storage because it is certainly cheaper to replace a few dead HDDs/PSUs a year vs. monthly payments of $1,300.
The build (and questions)
Keep in mind we will likely be buying three of these builds (primary data, primary backup, secondary off-site backup)
I'm a PC systems specialist at a university and second-in-command for a department of ~150 users. We're looking to replace our aging 3-2-1 storage solution. Currently we have an old Xserve running OS X which mainly acts as an LDAP server. We don't plan on using FreeNAS to replace it, instead we will likely buy a Dell server running Debian/Ubuntu or openSUSE and OpenLDAP.
Multiple hardware RAID arrays (4 HDD bays) are plugged up to the Xserve over FireWire 800/eSATA. User home directories (infrequently used) and SMB/AFP shares (frequently used) use these arrays which total ~50TB. A second set of arrays are plugged up to the Xserve and act as a primary backup (another ~50TB). A third set of arrays are plugged up to a Mac mini server off-site which acts as a secondary backup (another ~50TB). We plan to replace these arrays with FreeNAS pools; we have access to 10Gb/s so bandwidth between FreeNAS and the Linux server should be good (the pool shares would be mounted as network drives on the Linux server).
In the past, my boss has just added more arrays as needed. It is becoming a headache for a few reasons:
- The Xserve has unreliable uptime.
- The arrays don't have redundant PSUs so when one dies the users that have data on it are unable to access it until a new PSU comes in.
- They are hardware RAID (this isn't a big issue beyond unlikely silent data corruption) in RAID 5 (this also isn't a big issue because we have two backups, UPSs, and generators).
- They are expensive to expand. For example, it would cost us $1,726.57 to buy a 24TB Buffalo TeraStation which is $71.94 per TB vs. buying new Western Digital 6TB hard drives to put into the empty bays of our future FreeNAS server at $39.69 per TB.
We considered cloud block storage for primary/secondary backups
...but we decided against it. If anyone has experience in this area I'd be extremely interested in any insight. AWS EBS would be $0.045 per GB per month (not including the cost of an AWS EC2 instance required to attach it to) and Google Cloud would be $0.026/GB/mo. Compared to a $10,000 FreeNAS server that will last 5-8 years it just doesn't compare. For example, Google Cloud would cost $1,300/mo for 50 TB/mo. Maybe I'm missing something but cloud block storage doesn't seem "there yet" to replace local storage for small/medium businesses like our department. When compared to pricey SANs or NASs with rock-solid warranties, maybe, but I feel our unique situation of having free electricity, bandwidth, rack space, battery/generator backups, plenty of free time, etc. pushes us towards local storage because it is certainly cheaper to replace a few dead HDDs/PSUs a year vs. monthly payments of $1,300.
The build (and questions)
Keep in mind we will likely be buying three of these builds (primary data, primary backup, secondary off-site backup)
- Chassis: SuperMicro 6048R-E1CR36L
- Motherboard: SuperMicro X10DRH-iT
- SAS Controller: AOC-S3008L-L8i (supports pass-through IT mode)
- HDDs: 18x Western Digital WD60EFRX 6TB in RAIDZ2 pools of 6 (would add 6 drives at a time later)
- Is this the best setup? Or would larger pools/RAIDZ be OK with our multiple backups? I'm also not sure if I should be afraid of rebuild times.
- RAM: Samsung MEM-DR416L-SL01-ER21 (16 GB) x 4 for a total of 64 GB (might add another 64GB later)
- CPU: 1x Xeon E5-1650 v4 or Xeon E5-2630 v4 (might add a second CPU later)
- I'm leaning more towards the E5-1650 because I know GHz will help SMB, but beyond that I don't know how CPU choice affects FreeNAS besides obvious speed benefits.
- FreeNAS SSDs: does anyone have any recommended enterprise SSDs I could mirror for FreeNAS?