Cluster and Dell vs Supermicro

[titusc]

Hi I'm new to server land and certainly storage. I want to build an expandable storage for my web portal running database, saving and serving files and mails on the storage. Think FreeNAS fits to bill for me to start with small (say 32TB) and grow large (say 1PB if it does get large).

Option 1
Two nodes (Cluster in a Box) connecting to incremental 24 bays or 60 bays JBOD enclosures. Both DataOn and SuperMicro have these Cluster in a Box solutions but I don't see anyone mentioning them here. Is it possible with FreeNAS and if yes any preferable models?

Option 2
Dell 720xd with 24 bays + 2 disks in RAID 1 for the OS. Then connect also incremental external JBOD enclosures. Has anyone use this Dell model?

Also how does the 2 nodes (presumably with different IPs) be able to serve requests at the same time as one unit? How is this achieved? I think Nexanta does this but don't understand how and whether FreeNAS has this.

And should I size the nodes' memory size based on total disk based storage size?

Thanks

 

[cyberjock]

There are some limitations as you go very very big. If performance is important, going with a server that has a 1PB pool/pools will get excessively expensive over having 2 or 3 smaller ones. When people want a single very big server, you pay for it, bigtime. So keep that in mind. Quite often 3 smaller servers can be faster, more reliable, and far cheaper than 1 big server.

As for those enclosures, as long as they are compatible with FreeBSD, you should be able to use them. I'd avoid the Dell unless you can test it before using it. Had lots of problems with proprietary Dell stuff, and Supermicro is far more "just works".

 

[bigphil]

I agree with cyberjock...I would scale out for a system that big...not up. As for the clustering question...are you asking if FreeNAS has the ability to be an active/active pair or active/passive pair? If so, it doesn't currently support this setup. There are some article on setting up FreeBSD with HAST+CARP, but I can certainly say this is not production ready as the failover is messy and nothing like say a NetApp in cluster or 7-mode. Your best bet is replication as that is what is best supported at the time. TrueNAS from ixSystems does support HA clustering, so if you're looking for something like that, you might check with them.

 

[cyberjock]

Clustering is, shall we say "being investigated for development", but I won't even give a guestimate on when or even if it will be integrated. There may be technical reasons why it will ever work that haven't been identified yet.

 

[titusc]

Okay thanks. How do I know which JBOD boxes are compatible? I mean there isn't like a Compatible with Windows 7 type logo for *NIX systems. Are there specific brands of JBODs we know would work for sure?

Also I need some guidance on HBAs. When I look at say a LSI 8 way HBA I thought there will be 8 ports for connecting cables but it doesn't appear to be so in the pic. Is there any tutorials on how HBAs work? Can I daisy chain the JBODs?

 

[cyberjock]

There is a FreeBSD hardware compatibility matrix(it's linked from the FreeNAS manual).

As for HBA's there's many different kinds, and they often have different kinds of connections. My advice is to do lots of research as that question is beyond the scope of FreeNAS. If you can't get your answers from Google, consider contracting out the job. You don't want to get this stuff wrong. It can get very expensive for minor mistakes.

 

[titusc]

There is a FreeBSD hardware compatibility matrix(it's linked from the FreeNAS manual).

Okay I think you are referring to this manual here? On page 12 it says "Since FreeNAS® 9.2.1 is based on FreeBSD 9.2, it supports the same hardware found in the amd64
and i386 sections of the FreeBSD 9.2 Hardware Compatibility List." which brings me to this link here. But this is just a release note for FreeBSD and it mentions nothing about JBODs and disks. So can I just stick buy any JBODs and stick a bunch of SATA disks in?

In terms of contracting out are there any one you can refer me to?

 

[cyberjock]

Not off the top of my head. I do some contracting/consulting, and most people(especially businesses) find my prices very reasonable.

iXsystems sells prebuilt systems too. People have said their prices are reasonable.

JBOD is a feature of the hardware. You'll need to find hardware that supports JBOD and is on that list. Be mindful that hardware that is "supported" isn't necessarily "recommended" with regards to ZFS pools. That's one of many reasons why we have stickies on topics like this one. It's non-trivial to get right, and a waste of money to get it wrong.

 

[Sir.Robin]

I have a comment.

I work with several hardware brands every day. Servers. Storage. Fabric. etc.

With that said, i find the frase "Easy as Dell" cinda funny...
Cuz i never throw more time out the window than when i have to work with servers from Dell.