Is FreeNAS right for me?

[HeliosFA]

Hi all,

I'm new to FreeNAS and I'd like to appologise for the length of this post - I seem to have ended up with a rather complex situation and I would like to see if FreeNAS is right for what I am trying to do and if anyone has any experience/tips for how to implement it, or can shout at me if I have something completely wrong :). I've popped this in the N00bs section as it does span a few, and I an a FreeNAS N00b!

Basically, I have three separate locations (soon to be 4) linked across cable WAN connections (30-60Mb bown, 2 to 3 Mb up) by IPSEC VPN where I would like to have access to shared storage.

There are 4 users in total, three of which regularly move between sites and who are getting fed up of either dragging around external hard drives or having slow access to data they don't have on their laptops/local machines. The local clients will mostly be Windows, but some devices are linux-based.

Given the amount of data involved, this storage should have a local instance at each site to provide fast reads and writes.

The storage will be used as a mixture of off-device storage and backup of current work, so I envisage each user having a current work directory, which is subsequently kept in sync with their local device so that they can take the work anywhere, and an "archive" directory which is just used when they are on one of the local networks. There will also be a shared media library syncronised between two of the sites ONLY (the 4th site will likely have the media as well when it joins my little mad house...).

Finally, each node should provide a small area which is NOT synchronised for the storage of OS backup images of local devices which do not move site to site. The two nodes with the media library will probably need to store VM images here as well for local VM servers.

Data wise, we are talking approximately 2 to 3TB of media/installs to be shared between two of the sites ONLY with another several hundred GB (<750GB) of data, mostly PDF/Word/Excel documents and C-Source code but with a smattering of archives, to be synchronised between all sites. The media will be mostly read while the other data will be more of an even balance between read/write.

Some sort of snapshotting/version control on one of the nodes for the current work data would be useful to provide quick restore/rollback.



Hardware wise, I am looking at using a HP Microserver with 2x 750GB drives mirrored for the documents, a 1.5TB and a 2TB unmirrored/unRAIDed for the bulk data at each of the sites with the media repository. These two sites are wired with gigabit networking.

The other site, predominantly used by a single user, will likely have a Via EPIA board (1GHz processor, 1 GB RAM) with a couple of large disks on it to form a software mirror. This user mainly uses 802.11G wireless and the site does not have gigabit networking.

Does this sound about right? am I likely to run into any really bad bottle necks?



To summarise the above, I need a setup which is able to provide the following. Is FreeNAS suitable for this?

1. Efficient, multi-way and extensible synchronisation.
2. Segregated access to data - permission controlls.
3. Seamless, consistent access at each site (e.g. going to \\NAS at any site takes you to the local NAS)
4. Separate sync schedules - e.g. the media can be syncronised daily or weekly, while current documents are synchronised every 15 minutes to 1 hour.

1. From reading the docs and having a play, am I correct in thinking that ZFS replication won't do two-way, let alone 3/4-way, bi-directional sync? and that Rsync has issues with deleting files on multi-way sync?

I have seen that there is a script for getting Bittorrent Sync to work - has anyone used this and how efficient is it?

2. I think this is an integrated part of FreeNAS from the documentation?

3. I have appropriate DNS at each site, so I think I can just give each a unique hostname (e.g. site1_nas, site2_nas, etc.) and at each site, just create another record at each site pointing NAs at the IP of the local NAS. Obviously I will need to keep user accounts, etc. in sync between FreeNAS boxes - I am considering tacking on some form of LDAP as well.

4. This is not so critical but I would like to avoid GBs of new media from hogging the upload in the middle of the day, but still have current documents regularly syncronised.


Before anyone asks, this is for a home environment - just we all seem to move about a lot!

Thanks in advance for any comments/advice/help,
Helios

 

[cyberjock]

WOW. Interesting scenario. To be honest, if person 'a' will always be modifying "his" data set in the same directories, it should be possible to come up with an rsync script to sync with the other sites when hes done. Basically part of his "closeout" for the day would be to intiate the rsync to the other devices.

I don't think ZFS replication or just a plain-jane rsync of the whole server would work.

As for your CPU/RAM, you are far far under the minimum recommended for non-ZFS setups. Most people recommend 4GB minimum for UFS. If you don't have enough RAM expect the system to be unreliable at best.

Wifi is a terrible bottleneck and you should do speed tests and determine the size of data you are using to see if what you want to do is even practical. For example, if you're only getting 1MB/sec, but you are generating 2GB of data per day, then your whole plan is shot down based on the sheer amount of data that you are needing to move around.

I would stick all of your data on ZFS. Just because with all of the data being moved around and you don't want a setup that requires regular maintenance to keep everything in good shape. But, using ZFS will require more $ on hardware. I'm not familiar with that Via EPIC but if its not x86 based it won't work with FreeNAS. Considering its a 1Ghz CPU, I'm thinking its far less powerful than Intel Atoms, which would probably work for your needs.

ZFS doesn't support 3-4 way syncing, and I'm not sure if you could easily tweak it to make it work without having a bunch of different datasets. Depending on the distribution of your data this could make ZFS replication not an option.

I'm not familiar with bittorrent sync, so you are totally on your own with that.

Overall, I think i've given you some stuff to look into as potential pitfalls. What you should do is see if there are any show stoppers. If there aren't and you are determined to make this work, there is no doubt in my mind that you could come up with some custom scripts to do what you want.



In any case, you definitely have your homework cut out for you. Good luck!

 

[HeliosFA]

Thanks for the reply cyberjock - it is an intersting scenario, just a pain in the ass to sort!

You are correct that it will be one user editing his data in one locations (though could be accessed at two). I like the custom rsync idea, though I am not convinced it is suitable for this setup - it needs to be as transparent to the users as possible, e.g. they don't have to do anyting to the current data which is synced between their device and the NAS "cluster" - it just synchronises in the background. I've been doing a little checking and I think a scheduled task run every 15 mins or so calling the Microsoft Sync Toy to sync the client > local NAS could be the way to do the client to NAS sync.

Wifi is a bottle neck, but it is not practical or sensible to carcas that site with Cat 5. It will be one user doing up to 100MBish of changes a day. They are the only user at that site and the Wifi is almost straight down through a wooden floor so not too much attenuation. There may be some scope for upgrading to N on 5.7GHz, I will do a site survey next time I am up there.

The MicroServers will be having a RAM upgrade - probably to 8GB each. I take it that this will be just about adequate for ZFS?

The Epia is an x86 processor, they are supposedly lower power than an atom and they are definitely lower performance by a long shot. An atom board could be on the horizon for that node, it is just I have an EPIA board lying around at the moment and speding more cash on another board and RAM whe I am trying to setup the two big nodes is probably a no-no just now.

What would your thouhgts be on having ZFS on two of the nodes, but another alternative on the smaller node?

One of the big concerns is power consumption, so trying to keep the RAM amounts down and stick to lower-power hardware is important.

 

[cyberjock]

You could mix and match ZFS on some and UFS on others. In all honesty, if those EPIA boards are that much less powerful than Atoms, you may find your servers horribly underpowered. Some people complain about the performance of the "high" end Atoms. So you might want to see if those CPUs can handle that performance. Rsync is VERY CPU heavy. It's also a single threaded application. ZFS is also CPU heavy. If I were a betting man I'd bet your EPIA's won't cut it, but Atoms might be okay for you. But you shouldn't expect Atoms to exceed 30-50MB/sec in best case scenarios. Atoms are low power and they command a lower performance. Of course, that's far far above the upload speeds of your WAN connection, but for someone that is connected by Gb to the server, that may feel very slow.

8GB of RAM might be good for 1-5TB of data on ZFS. It just depends on alot of various things and is something that you should also be mindful that you may need more than 8GB. If you are dropping money on hardware that can't handle more than 8GB of RAM be sure you are ready to accept the risk that if 8GB of RAM isn't going to cut it you are ready to spend the money to upgrade.

As much as you might not want to hear it, you may be forced to either drop the money and hope Atoms(or EPIAs if you really want to go that route.. but I don't recommend it) can cut the mustard or just overbuild your systems to allow for any potential misunderstandings for hardware needs.

I do firmly believe that if you split up your data in such a way that 2 people aren't modifiying the same data at the same time you should be able to come up with some scripts to do the dirty work. Syncing every 15 minutes seems to be pretty frequent and you need to realize that your systems need to be able to process a sync in that time frame, along with upload/download any data that needs to be moved. I'm not sure what your exact need is for this data, but perhaps you should consider a design where instead of having multiple servers with the entire dataset you have a server based program that processes your data remotely and you simply remotely connect to it like a remote desktop.

 

[gpsguy]

Since the Microserver only has 2 slots for RAM, I'd upgrade them to 16Gb.

If you're using a Gen 7 Microserver (N54L/N40L/), even though the HP spec's show them limited to 8Gb, you can get 16Gb with the right RAM. In my N54L I'm using:

Kingston 16GB (2 x 8GB) 240-Pin DDR3 SDRAM DDR3 1333
ECC Unbuffered Server Memory Model KVR1333D3E9SK2/16G

The MicroServers will be having a RAM upgrade - probably to 8GB each. I take it that this will be just about adequate for ZFS?