What is the best way to setup our storage architecture? Need help thinking it...

[Martim Schnack]

We are an innovation company that is looking to create a fast, simple, and easy to use network. We are not IT people, and we don’t want to be. That’s why is so important to us that our network can be managed with ease, performs as fast as possible, and be low maintenance.

We need help establishing our storage architecture (which servers do we need? which software? which brand? which configuration? which drives?) in a way that will fit our current needs, be future proof, and is as close to worry-free as possible.

Goals:

• Needs to be easy to manage and worry-free (almost no maintenance).
• Scale out easily, as we grow.
• High performance
• Reliable
• Easy to manage user permissions, shares...
• Have a simple way for us to get qualified support, when needed.
• Work for Mac and Windows users seamlessly.
• Allow our employees to work directly from the server, as if they were accessing the files directly from their hard-drive.

Storage Needs:

• Virtual Environment - extremely fast storage, with high IOPS - for our VMWare hosts.
• Business Storage - fast storage for our creative users (mac) and standards users (windows) with considerable size and made for accessing files like they were on the user machine.
• Security Cameras - storage for 24 cameras, recording 1920x1080, 10fps around 10 hours per day, retaining the footage for 180 days.
• DAM - database, index, caching… everything our Digital Asset Management solution will need.
• Backup - everything that we own and operate must be backed up on a daily basis (even with hourly increments for some cases)
• Archive - made to archive files that will be read not frequently.
• BitBucket - a big storage so we can dump everything we want to, make sense of it, and move to a better place (or leave it there) - like external drives, old hard-drives, etc.
• Long-Term Backup - a way for us to backup everything in a cheap but reliable media for a long time.

Considerations:

• We are willing to pay more for a solution that has a brand behind and proven track record, if it makes sense.
• Video storage will be dealt with later on. Our current solution works for our current needs. Will need to be revisited in the near future.

Concerns:

• Having servers and hard-drives that were bought and custom build, having no brand to support it and provide any help when needed.
• Every place we read says that FreeNAS should NOT be used in an enterprise environment.
• Should we use SAS or SATA drives?

I am attaching the way we see our storage needs in terms of tiers, showing how the relationship of size x speed x cost is. I am adding also a storage flow that shows how we believe the flow of our data needs to be.

Here is the solution that was initially proposed to us, by another consultants:

Tier 0 - Flash Storage (For VMs)

• Chassis: 1U SuperMicro
• CPU: 2 Intel 4-core Xeon E5-2609v2
• RAM: 128GB
• OS: FreeNAS 9.3
• RAID configuration: 2 2-disk RAID 10 vdevs
• Bays: 8 (4 available)
• Disks: Samsung 850 PRO 1TB
• Usable Storage: 1.5TB
• Max Storage: 27TB (adding 2 more JBODs)
• Price: $6,350.00 ($2,823.33 per TB)

Tier 1 - Business Storage

• Chassis 4U SuperMicro
• CPU: 2 Intel 4-core Xeon E5-2609v2
• RAM: 128GB
• OS: FreeNAS 9.3
• RAID configuration: 3 6-disk RAID Z1 vdevs + 2 hot spares
• Bays: 24 (0 available)
• Disks: WD RE 4TB
• Cache: 2-disk read zil, 2-disk write zil (Samsung 850 PRO 256GB)
• Usable Storage: 40TB
• Max Storage: 2.46PB (adding 8 more JBODs)
• Price: $11,704.00 ($292.60 per TB)

Tier 3 - Backup Storage

• Chassis 4U SuperMicro
• CPU: 2 Intel Xeon E5-1650
• RAM: 256GB
• OS: FreeNAS 9.3
• RAID configuration: 13 2-disk RAID 10 vdevs + 4 hot spares
• Bays: 36 (0 available)
• Disks: WD RE 6TB
• Cache: 2-disk read zil, 3-disk write zil (Samsung 850 PRO 256GB)
• Usable Storage: 57TB
• Max Storage: 475TB (adding 8 more JBODs)
• Price: $24,977.21 ($438.55 per TB)

Tier 4 - Archive Storage

• Chassis 4U SuperMicro
• CPU: 2 Intel Xeon E5-1650
• RAM: 256GB
• OS: FreeNAS 9.3
• RAID configuration: 13 2-disk RAID 10 vdevs + 4 hot spares
• Bays: 36 (0 available)
• Disks: Seagate Archival HDD 8TB
• Cache: 2-disk write zil (Samsung 850 PRO 256GB)
• Usable Storage: 160TB
• Max Storage: 1.37PB (adding 8 more JBODs)
• Price: $17,716.75 ($110.73 per TB)

What do you think about this configuration? Shoot holes in it! Anything that is bad that we are being suggested? Anything that we should be aware of?

What your configuration would be? What is your recommended storage solution? Drives? Servers?



Please share some insights, ideas, recommendations, advice, examples, anything - so frustrated and willing to pay for help.



Storage Tiers : http://cl.ly/image/3J1A0g3W1h1T

Storage Diagram: http://cl.ly/image/2s3D0m430W39

 

[sfcredfox]

You have some real business needs, and are planning on spending some real money.
You also don't want this to be someone's full time job, and have help.

Have you thought about calling iXSystems?

You send them your above list, and they send you a quote and a proposal of support.

I don't work for them, but if you really like the benefits of FreeNAS and I was working for your company, that would be my move.
My other favorite enterprise storage provider is Nimble Storage.
(I'd also be getting quotes from other enterprise storage places like Nimble Storage, and maybe the other usual suspects HP/NetApp/etc.)

I'm also a little curious why you are doing four separate systems instead of maybe two? I'm not saying it's a bad idea, not just sure if you need to unless there's another reason I don't see.

Also, you put twice as much memory in the backup systems as the one for handling VMs? If you want the storage platform to be killer for VMs, make sure you have killer SLOG devices, and loads of RAM, and L2ARC.

 

[zambanini]

martim, freenas is not for you. truenas may.



you need to spent much time and money to understand zfs fully. that is not something you can do within a few weeks.

 

[Robert Trevellyan]

We are not IT people, and we don’t want to be.
...
Every place we read says that FreeNAS should NOT be used in an enterprise environment.

Like @sfcredfox said, these two statements suggest you should be calling iX (and others) for a quote. FreeNAS could do what you want, but you would need to be 'IT people' to get it right and keep it properly fed and watered.

 

[Robert Smith]

With four servers and heavy emphasis on virtualization, you may be a good candidate for Virtual SAN.

The entire requirements need to be taken into the account. Software, the exact types and volumes of data, etc.

One of the advantages of Virtual SAN is that it prioritizes ‘fast’ direct attached storage (DAS) links, where with traditional storage servers everything goes over ‘slow’ IP connections.

(‘fast’ and ‘slow’ are relative terms here)


Also your pyramid structure means what? When any one of the servers goes down then the work stops?

I would restructure it in a way that if any server, or maybe even any two servers, go down, the work continues (albeit with possibly reduced speed) while the IT team scrambles to fix what is broken.

 

[Martim Schnack]

Have you thought about calling iXSystems?

The configuration I posted was made by them. I would like to validate and collect inputs before proceeding.

 

[sfcredfox]

With four servers and heavy emphasis on virtualization, you may be a good candidate for Virtual SAN.

The entire requirements need to be taken into the account. Software, the exact types and volumes of data, etc.

One of the advantages of Virtual SAN is that it prioritizes ‘fast’ direct attached storage (DAS) links, where with traditional storage servers everything goes over ‘slow’ IP connections.

(‘fast’ and ‘slow’ are relative terms here)


Also your pyramid structure means what? When any one of the servers goes down then the work stops?

I would restructure it in a way that if any server, or maybe even any two servers, go down, the work continues (albeit with possibly reduced speed) while the IT team scrambles to fix what is broken.

I forgot about VMware's Virtual SAN. Good call. That might be an option, though I don't know anything past the basic marketing for it. OP should consider that as an option since it supports clustering and can sustain a node failure to some extent.

OP, I think I know how you feel, and I know where you're coming from wanting people to provide some serious feedback on specific things in the proposal you posted in question. I think most people's answers are going to be about the same until they put their consultant hat on an basically type a dissertation about the major pieces to the storage infrastructure for your entire company's IT infrastructure. That's a pretty big job, and companies get paid serious money to have and share that kind of knowledge.

There's a few really super smart cookies on here, but I don't know how much more you'll be able to get out of this support forum.

I definitely think the advice of some of the responses for venders is very good advice, even though I know it's not what you wanted to hear.

If you have some more very specific questions, I bet we could help out some more, but just asking for general 'does this solution work for my company' questions aren't going to get much better from a forum like this, especially if you aren't actually IT focused and playing with and trying out the FreeNAS product a bunch before hand.

That is another recommendation I do have. Since you guys are about to do something serious, make these proposed venders bring in some equipment to demo and actually show you a system and how it responds to certain work loads. It's nice when you can beat on two systems side by side and put them under a realistic workload for your business/environment.

 

[depasseg]

The questions you are asking are valid of this forum. Namely, to look at and provide comments on different systems, with different specs and categorize them as fast faster fastest. It sound's like you got some good info from ixsystems (although I was a little confused about "read" and "write" ZIL's).

That being said, where you really need to focus your question is twofold - usability (you said you aren't and don't want to be IT people, do you know how to configure and manage FreeNAS?) and Sizing (do you have any idea of the current or expected IOPS, Latency, and other storage perfomance needs for your environment?)

I'm left scratching my head wondering why you have an all flash pool for VM's. What are the VM's doing that require an all flash pool? Also, the math on current capacity and max seems off (straight multiplication of base system capacity x 8 jbods), and even if it were correct, can the host system support that many petabytes of storage reliably?