FreeNAS Multi-Node Support?

[Gregory Smith]

Hello FreeNAS community!

Recently my company retired 2 pair of "SANs" that are essentially vanilla SuperMicro boards, supporting 4 drives each, in 3x1U enclosures linked via a separate 1GbE adapter for the backplane. I took them home to tinker and a friend of mine suggested I take a look at FreeNAS. I have combed through the documentation and there are a lot of things I'm excited to try out with this great open source software!

So my question is complicated because I'm sure I'll word it improperly. This is not my arena of expertise so I'll defer to the community here. In my ideal setup I would involve having all 12 disks from the 3 separate physical servers linked together to create 1 large logical disk. I know this is possible, the question is whether or not FreeNAS supports it. The software that was installed on the "SAN" before hand was absolute garbage and the only reason these devices were retired.

 

[cyberjock]

FreeNAS only supports using disks that are locally connected to the server. Not sure what hardware you are trying to use, but your choice of words with "3 separate physical servers linked together to create 1 large logical disk" sounds like something that several people have wanted to do, but isn't particularly possible with ZFS (which is the file system FreeNAS uses).

 

[Gregory Smith]

FreeNAS only supports using disks that are locally connected to the server. Not sure what hardware you are trying to use, but your choice of words with "3 separate physical servers linked together to create 1 large logical disk" sounds like something that several people have wanted to do, but isn't particularly possible with ZFS (which is the file system FreeNAS uses).

Thank you for the quick reply. The hardware I'm referencing is 3 independent systems all with identical hardware. It was running some horribly unstable, heavily modified linux distro to do essentially what I'm looking to do.

 

[L]

There is a project currently to add freebsd support for gluster. That would provide a single namespace, meaning making a number of systems look like 1 big NAS drive. You can also look at the truenas stuff as it can do more of the multihead support primarily for failover/availability. Either way freenas doesn't have san targets. A lot of the other openzfs distros do have san targets.

Is there a reason you want 1 big drive?? I have seen single pools that are multiple PB is size.

There is tools like alua that lets you have active/active or active/passive failover.

 

[Gregory Smith]

Sounds like my only option, if I choose to use FreeNAS, is to put a hypervisor in-between the physical servers and make one FreeNAS box on top of that.

 

[Gregory Smith]

There is a project currently to add freebsd support for gluster. That would provide a single namespace, meaning making a number of systems look like 1 big NAS drive. You can also look at the truenas stuff as it can do more of the multihead support primarily for failover. Either way freenas doesn't have san targets. A lot of the other openzfs distros do have san targets.

Wonderful, I'll look into this.

Is there a reason you want 1 big drive?? I have seen single pools that are multiple PB is size.

Mainly just trying to keep this a $0 cost project. The blades each come with 4x 1TB drives. By the time I would provision for hard drive redundancy I'm left with 2.6TB usable best case scenario. If it supported clustering them together I could essentially have north of 8TB usable.

 

[L]

I know a ton of products that do that, but they all pricey. RSF-1 is specifically designed for zfs. Open source gluster, lustre .. just google global name space. In most cases with global name space you still have to partition. I know a couple people doing it with hadoop..

If you are looking for a slow science experiment, one thing you can do is create iscsi targets on 2 servers and mount them on the third, create the pool out of all.. might break. might fail, but it would fun to try..

 

[Gregory Smith]

If you are looking for a slow science experiment, one thing you can do is create iscsi targets on 2 servers and mount them on the third, create the pool out of all.. might break. might fail, but it would fun to try..

This might just work. I was thinking something along these lines. The final server would most likely have to be a windows box? Does FreeNAS have an iscscicpl equivalent or ISCSI initiator?

 

[Gregory Smith]

I ended up scrapping the original plan. If the option becomes available in the future I may look at it. I appreciate everyone's help with this.

So the plan for now is...

1. Node 1: 2.67GB usable RAIDZ - will act as primary CIFS share
2. Node 2: 2.67GB usable RAIDZ - will act as PULL target for replication from node 1
3. Node 3: 2.67GB usable RAIDZ - undecided, test platform, cold spares, or offsite rsync for my friends identical setup on Node 1 and vice versa

 

[L]

Probably for the best, especially in a commercial setup.

 

[Micheal]

FreeNAS only supports using disks that are locally connected to the server. Not sure what hardware you are trying to use, but your choice of words with "3 separate physical servers linked together to create 1 large logical disk" sounds like something that several people have wanted to do, but isn't particularly possible with ZFS (which is the file system FreeNAS uses).

I know I'm reviving an old post here, but I had a question that I was hoping you could answer about FreeNAS Multi-Node Support.. kind of.

The Dell PowerEdge C6100 is bad ass hardware that comes with Nodes or Sleds which are their own blades within the server. I was wondering if it was possible to get FreeNAS to use all RAM on say two Nodes? They're still in the same machine/server, but I'm not sure the connection beyond that. I'd appreciate any thoughts you might have, I'm considering buying one of these but would like to use cheaper RAM (smaller DIMM Size of say 4GB each) but still get a bunch of RAM for FreeNAS to use.

http://www.dell.com/downloads/global/products/pedge/en/poweedge-c6100-spec-sheet-100810.pdf

I appreciate it.

Micheal

 

[Ericloewe]

I was wondering if it was possible to get FreeNAS to use all RAM on say two Nodes?

No, of course not, they're separate computers.

 

[Micheal]

Yeah, I decided to go a different direction and get the C2100 instead as the C6100 doesn't even have ECC because it's meant to be more like 4 separate workstations, not servers. Really cool hardware but I really just wanted the 12 x 3.5" bays (the 24 x 2.5" bays are pretty cool too but this is for a "budget" build).

Also, I found this Wikipedia article pretty helpful:
https://en.wikipedia.org/wiki/List_of_Dell_PowerEdge_Servers

Thanks