low cost SAN for our School

[R wkgs]

First off i have used FreeNAS since v8 and OpenSolaris before that for my home storage/backup needs so i know enough to be dangerous but not close to a expert either. current set-up is:

motherboard: supermicro X10SL7-F
cpu: e3-1220v3
ram: 16GB EEC
case: 16 bay Supermicro CSE-836E2-R800B
startech reverse sata to sas cables sff-8087 with the lsi 2308 in HBA mode

This has worked fine but I am not really pushing the system with 6 users backing up at the same time or watching a few movies and own cloud in a jail. The school I work at now needs low cost storage and we scored 2 more of these cases for £200 new unused but old stock on ebay :) so if there is a alternative recommendation it must be ATX or E-ATX.

If I recommend the same set-up with 32GB RAM with a intel 4x 1g network card.

Usage would be a few windows shares for shared department files e.g powerpoints, word docs, pdf's, mp3s and video. also a few non critical servers would be moved to ISCSI shares such as our windows deployment server. finally some NFS or SMB shares for archival of ISO installers etc.. plan based on very low remaining budget is to use WD RED drives rather than SAS or 10kSAS as we don’t need colossal IO. No Dedupe as I think 4TBx16 will need more than 32GB in our case as teachers and students move away from paper it is not uncommon to get 1000+ copies of the same pdf even though we keep telling them to put them online.

A) Cope with me stacking the backplanes so 32 drive bays, or would people recommend 2 SANs each with its own motherboard, or a E5 xeon based setup?

B) Pool layout options. I am thinking 1 pool with 4x 4 disk RAIDz1 or 1 pool with 2x 8 disk RAIDz2 as we don’t need the IO of raid10. Next year the second case will be populated giving 8x 4 disk RAIDz1 or 4x 8 disk RAIDz2


pool
|
|¬ [hd1,hd2,hd3,hd4] Raidz1
|
|¬ [hd5,hd6,hd7,hd8] Raidz1
|
|¬ [hd9,hd10,hd11,hd12] Raidz1
|
|¬ [hd13,hd14,hd15,hd16] Raidz1

vs

pool
|
|¬ [hd1,hd2,hd3,hd4,hd5,hd6,hd7,hd8] Raidz2
|
|¬ [hd9,hd10,hd11,hd12,hd13,hd14,hd15,hd16] Raidz2


C) Can anyone see a reason that this set-up and use case for 70 teachers/users would struggle to access there files?

SSD read / write cache would be added net budget year as would the second case or we may use it for replication as backup for the first SAN. may be 10g intel x520.

Sorry for the rambling post but deadline for parts selection is looming and FreeBSD is not often welcome in windows shops so I want to get a sanity check on my plan.

 

[Darren Myers]

remember when using iSCSI dont use over 50% of the pool you are using with iSCSI. I wouldnt do RAIDz1's as others will weigh in, certainly use RAIDz2; i think i would use both chassises with one motherboard/cpu/ram set , i would only use 2 separate servers if i was going to do high availability/failover, or if i wanted to be able to power one down and still keep everything online.

 

[Mr_N]

If your budget allows I'd go:

pool
|
|¬ [hd1,hd2,hd3,hd4,hd5,hd6] Raidz2
|
|¬ [hd7,hd8,hd9,hd10,hd11,hd12] Raidz2
|
|¬ [hd13,hd14,hd15,hd16,hd17,hd18] Raidz2

better IO than using 2 vdevs and better redundancy than z1 (44.5TB with 4TB drives)

 

[R wkgs]

remember when using iSCSI don’t use over 50% of the pool you are using with iSCSI

A good tip I suppose if we just put c:\ for our VMware servers on that iSCSI share and then use mapped drives to windows shares we can use some more space.

If your budget allows I'd go:

pool
|
|¬ [hd1,hd2,hd3,hd4,hd5,hd6] Raidz2
|
|¬ [hd7,hd8,hd9,hd10,hd11,hd12] Raidz2
|
|¬ [hd13,hd14,hd15,hd16,hd17,hd18] Raidz2

better IO than using 2 vdevs and better redundancy than z1 (44.5TB with 4TB drives)

I can either hide 2 more drives inside as only 16 hot swap bays per case or more likely with our budget go for 2x 6 disk RAIDz2 this year and next year once stacked with the second case add another 2x 6 disk RAIDz2.

pool
|
|¬ [hd1,hd2,hd3,hd4,hd5,hd6] Raidz2
|
|¬ [hd7,hd8,hd9,hd10,hd11,hd12] Raidz2
| stacked with second supermicro case
|¬ [hd1,hd2,hd3,hd4,hd5,hd6] Raidz2
|
|¬ [hd7,hd8,hd9,hd10,hd11,hd12] Raidz2

 

[gpsguy]

If you are planning on doing this, I'd suggest striped mirrors instead.

Buy a motherboard that will accept more than 32 GB RAM. For iSCSI, you'll need to start with at least 32GB, maybe even 64GB RAM.

A good tip I suppose if we just put c:\ for our VMware servers on that iSCSI share and then use mapped drives to windows shares we can use some more space.

 

[R wkgs]

If you are planning on doing this, I'd suggest striped mirrors instead.

If we where going to have more than 2 VM's (idle 99% of the time) i would agree. but as teachers will not even use these files every lesson or even all be opening files at the same time, we currently find a synology 4 HD RAID5 is not showing any bottleneck its more about moving what we can off the current equallogic SANs. e.g past students work which we need to keep till the exams officer gives us the all clear to delete. students past years work could also be a candidate again files which are not often accessed.



I would love to go with a Supermicro X9SRH-7F LGA 2011) supports Intel® Xeon® processor E5-2600/1600 but we may only be able to afford 6 HD in a RAIDz2 if i go for that solution. money is the enemy but on the plus side its why the boss is not just buying another Dell equallogic.

X9SRH-7F would give us up to 512GB ram so L2ARC would be a benefit I belive you really need >32GB of ram to make work properly. Dedupe would also be on the table as we have so much duplicated data in the students work areas.

I could always go with a X10 this year and once Freenas has proved itself rebuild with the X9 next summer in the second case. the X10 could then be relegated to offsite replication backups in a new case?

 

[anodos]

I would avoid deduplication. In almost all circumstances, it's cheaper and better to just add more storage.