New FreeNAS File Server SMB vs iSCSI vs NFS (best path for File storage?)

[BigChumpy]

Hi All, I can't quite find the answer to this question. Trying to find out what's the best way to go. Your wisdom is appreciated.

I've had various different NAS boxes in the past with tons of different issues getting SMB and domain permissions to be respected. Thus the idea of setting up an iSCSI target fileserver that just shows up as a windows drive where I can apply user permissions easily, then re-share from the windows server seems attractive to me. Does this make sense? Here's my goals:

Goals:
1. Provide a file server with FreeNAS that respects domain permissions\credentials
2. Provide file level access as rapidly as possible on a 1GB network.

What I've done:
I setup FreeNAS as an iSCSI target and setup the initiator on my WIn2008 Server, sharing out the different folders on the iSCSI share via Windows. All LAN workstations will map drives directly to the Win Server. Does this make sense? Whats the advantages of this, or disadvantages? I've heard that iSCSI is faster since it's block level, but it may not matter since I'm routing through the Windows Server anyway.

Options:

1. Should I just use NFS? Not much experience with that. I do need it to respect the windows permissions without hassle. I heard it's slower on Freenas
2. CIFS...I've used this a bunch, but getting permissions to work with the domain controller was always finnicky.

 

[darkwarrior]

Hi there,

I'm currently using the kind of scheme that you intend to use (iscsi device -> windows server -> shares -> clients) and I'm very happy with it. But that's just me that likes to make experiments
iSCSI is not specifically faster because it's a block device ... You may want to read up on that matter (in my sig is a link to a huge post on this matter). All will depend on the resources that you have.
Side note: you will want to use striped mirrors with iSCSI to get the best random I/Os out of your hardware.

Regarding the other points:
1. NFS could be working as well, but i don't know if the permissions would be handled easier.
And it's not slower on Freenas. And I actually don't know where that idea comes from ... Maybe Sync=On ?
2. CIFS permissions on Freenas is not my thing either, but @m0nkey_ posted a very nice video tutorial on that matter, it might help you wrapping your head around it.

 

[BigChumpy]

Thanks for the great and thorough post. I'll dig into it! Seems like iSCSI is a pretty decent approach for this kind of thing. I'll have a look at what you provoded thanks @darkwarrior. Also I have RAID10 setup on four 3TB drives.... on an R510 with a Dell Perc H200 in IT mode.... seems to work pretty well? thoughts?

 

[darkwarrior]

Then looks like you have the adequate setup already (IT Mode && RAID10 == striped mirrors). Great ;)
Just make sure that you have as much RAM as possible to start with and be ready to add more (or L2ARC, if you're already over 32GB of RAM) if you're not very happy with the performance that you get :D

 

[BigChumpy]

only at 16RAM though.....hmm...maybe needing more? I did a quick test copying a 12GB file:
Windows>WinServerVM (Dell R710 on Xenserver)>iSCSI>FreeNAS = 30-90 MB/s ....all over the map...just bouncing around. (not really a great result)

Any tips for optimizing the transfer speeds? it's 1GBe all the way across
Thanks

 

[darkwarrior]

You're at 1Gbit LAN so you will not be able to get much more than 100MB/s throughput.
But if your transfer speeds are getting as low as 30MB/s, you will really need to add way more RAM and also more spindles (=> VDEVs) to absorb that write activity.

 

[BigChumpy]

@darkwarrior, after diving in to this a bit more, I'm really loving the simplicity of using iSCSI reshared out from the VMServer..... I'm getting great speeds from anything direct connected to iSCSI... (105-133MB/s) but when I go from a workstation to the VMServer to iSCSI , it drops down to 30 MB\s

To restate:
Windows10>SMB>Win2008VM (Dell R710 running Xenserver)>iSCSI>FreeNAS ....30MB\s
Win2008VM (Dell R710 running Xenserver)>iSCSI>FreeNAS...... 105-133 MB\s
Windows10>iSCSI>FreeNAS ...........105-133 MB\s

So clearly it's got something to do with using that WIn2008VM in the middle... that's slowing things down, but can't imagine what it would be. I even put FreeNAS on NIC#2 only, and put the LAN on NIC#1 thinking that somehow it was saturating the one network card and that didn't seem to make a difference.

Any ideas on what might be up here? You think I might need to optimize the NIC's in the Xenservers that are running that VM? Outbound however from that VM to the FreeNAS is fast

thanks

 

[lukeren]

I'm just a newbie in this FreeNAS world, but I've been burned by dodgy NICs before...
Is there maybe a Realtek NIC in of the boxes?