BUILD Sanity check for new NAS

[no_connection]

Long term FreeNAS user with version 7.2

Now I'm looking to build a proper server for a friend who owns a photo studio. He is currently spreading out data on external drives and several macs and that makes my hair curl.
And it makes accessing the data a hassle as well. Not to mention backups.

After reading up a bit I have made the following list.
Supermicro X10SL7-F
Pobably an Intel Xeon E3-1220v3 unless I should add more features?
32GB of ECC RAM Kingston ValueRAM DDR3 PC12800/1600MHz CL11 ECC 32GB (KVR16E11K4/32)
PSU Corsair HX650v2 or similar, preferably modular.
Fractal Design Arc R2 for 8 HDD bays out of the box. Two more could be added in the 5.25" slots.
Drives would be WD RED 4TB.

I was hoping to use the LSI 2308 for additional SATA ports and I found one post of someone using it with freenas. Will it work with 4TB drives and beyond? Or work at all?
2TB limit was for older controllers right?
*edit* google must be bad again(getting worse every day for finding info), will read the thread about X10SL7-F in this forum. Tomorrow.

I was planning to add drives two and two in z-mirror for ease of upgrade and performance.
If all ports work that would give me 14 drives (if I can cram them into the chassis).

Am I right in assuming that I can just add pair of drives to increase storage for the same share?
Would new data be striped among the mirrored drives increasing performance as I increase spindles (for new data) and automatically balance the drives. Or would a file be written to a single mirror with some sort of load balancing amongst drives?

I'm not using MAC but he is, will it work fine and give good performance?
Time machine will work?
Any tips for when using it with MAC?

 

[jgreco]

0.7.2 is basically what is now NAS4Free. FreeNAS 9 is all new.

The M1015 in IT mode is the MOST compatible and preferred solution for additional SATA ports but I'd expect the 2308 flashed to IT mode to be just dandy -but also 3x the cost. See the LSI sticky.

2TB limit was relevant to old controllers, for LSI, that means pre-6Gbps chipsets are limited. No size concerns on the LSI 6Gbps stuff. Other controllers may vary!

Mirrored drives can be used to extend your pool incrementally, yes. ZFS uses an intelligent algorithm to balance load among vdev's; all thing being equal, it will stripe across them to increase performance. But if you add a mirror vdev to a mirror vdev pool that is 99% full, all new data is going to wind up on the new disks. There is no "rebalancing" done. You can simulate rebalancing through judicious shuffling of data on the pool. But it probably won't result in a big speed difference.

Do note that loss of both disks in a single mirror effectively destroys the ENTIRE pool. The argument has been made that "RAID5 is dead" and mirroring is (for purposes of the point I am making) a degenerate version of RAID5; I would be completely comfortable with your proposed setup as long as there was an independent backup of the data (which you should always have anyways!). Do not mistake redundancy against failure as a substitute for backups.

Time Machine on netatalk has always been a little dicey, and I say that with the deepest respect for the netatalk team. Developing these programs to implement proprietary protocols is a special class of hell only undertaken by the cleverest gurus. TM often goes wonky at the mildest provocation. But netatalk 3 is included in FreeNAS 9.2, which hopefully cures some of the issues.

 

[no_connection]

Thanks.
I played a bit with 8.x versions but never upgraded or put in production. I have yet to play with 9.1 or 9.2 as I have too little memory in the esxi box for playtime.

The LSI 2308 is included on the X10SL7-F board with only about 10-20$ price increase, which seems like a good deal?


Good to confirm that vdev's would work as I thought.

I'm currently trying to convince the guy that backup that prevent data loss is not the same as keeping data integrity, and at the moment I see no other solution than freenas to prevent data corruption. Even if dropbox might prevent their copy from getting corrupt you might still end up with bad data.
Unless macs are magic and don't suffer from data corruption?

 

[jgreco]

Macs are magic once you drink the Steve Jobs koolaid. The 2308 on the Supermicro boards is certainly a good deal. I keep forgetting about it...

 

[JohnK]

A few thoughts.
1. You might want to consider a Pentium or even i3. A Xeon might be an overkill if you are not planning encryption/transcoding/many users.
2. You can get away with a much smaller PSU and once you do go smaller modular is not that important.
3. I would consider going with 6 disk RaidZ2 or 7 disk RaidZ3 for better redundancy.
4. If you are thinking 14 disks long term to use all the connections on the motherboard then buy a chassis that can handle it.

 

[no_connection]

Would a G3420 and 16GB be a good start then?

Did you use the Seasonic G-360? Does it have enough cables?

I guess I can add or make a cabinet for 6 drives cheaper than it would be to buy a 14 drive cabinet.
Bolting on a second ARC R2 with a PSU could also work.

 

[JohnK]

I use the G3420 in my backup NAS. I would have used one in my primary as well if I didn't get the i3 for a great price might move it to my desktop in the long run. Anyway I believe for plain file serving the G3420 is more than enough. The G3420 cannot hand encryption and don't even think about transcoding. From what you wrote I don't believe you need this.
16GB should be enough though you might get that 32GB cheaper per gig. Also as you increase your storage, you will need more ram. I have 8GB in my backup (note you can use 1 stick split off the 4 stick set) and it serves my purpose.
The Seasonic G-360 only has 4 Sata. I use a splitter that adds 4 more. It adds to the cost, but I'm trying to use as low as possible well rated PSU.

Have you considered a ARC XL?
It has space for 8 drives plus a 4 times 5.25. You can easily add a 3 to 5 adapter in there and use one space for a drive. That will give you 14 disks. I like both my Mini and Midi, but I might go bigger if needed one day. Of course my needs limits me to a tower set up and sound is not a problem

 

[no_connection]

Did the G3420 work fine with IPMI or did the HD graphics cause problems?
What performance do you get out of it? Headroom?
Encryption is probably not needed, playing TV-series and movies on iPhone or apple TV (or whatever apple things can do) is desired, so I'm not sure what will put strain on the CPU or if iDevices will play most things. Any apple advice?

ARC XL might be an alternative.

 

[JohnK]

IPMI works great. I am not using the integrated graphics as both of my servers are headless.

The way I measured performance is playing some blue ray files to all my XBMC devices. Processor not taxed at all. I move files to it at 108MB/s no problem. Then again you can put things in perspective and say that I used to do the same using a Buffalo Linkstation and that had a tiny processor. So as long as you do no transcode or encrypt you will be fine.

My wife has an IPAD and she watch some stuff on it, but I do not support it. She is on her own! :) I don't touch Apple. I do play stuff to my Android and I'm having no problems.

What I have heard CyberJock mention is that he streams stuff to his iPhone and transcode some bit rates and thus requires the Xeon. I cannot help though, sorry.

 

[no_connection]

Lets say I would start with 6 disks. Is there any benefit going RAIDZ2 vs two and two mirrored. Except a bit more space?

For data recovery, could I take one disk from a mirror and import it on another machine (If I don't stripe several vdevs together)?

Can I take a disk that is HFS+ formatted, stick it into a SATA port, take a backup to it, then remove it?
Same with USB?

And the other way around, backup USB disk or memory by sticking it in the NAS?

 

[cyberjock]

FreeNAS uses UFS and ZFS natively only. HFS+ isn't supported at all.

 

[hoboville]

Macs are magic once you drink the Steve Jobs koolaid. The 2308 on the Supermicro boards is certainly a good deal. I keep forgetting about it...

Hehe, for a while now, Macs have used PC hardware: CPU, RAM, HDD, chipset (minus bios). Only difference in the OS, which is certified Unix! Macs are PCs!

 

[jgreco]

Hehe, for a while now, Macs have used PC hardware: CPU, RAM, HDD, chipset (minus bios). Only difference in the OS, which is certified Unix! Macs are PCs!

Which displays the upsides of the x86 platform ... quality hardware options exist at reasonable prices ... and the downsides, like most gear not supporting ECC.

 

[no_connection]

Great, it seems like my ESXi box won't live forever, ML350G5 seems to have a PSU fan stuck at full speed which is probably not good.

So I might have to make a new one. And at the same time solve my lack of storage.

Supermicro X10SL7-F with the intent to use the SIL controller in pass-through to freenas.
32GB ECC RAM with 8GB+ to freenas. (depending on what else might be running)
Xeon E3-1220v3 should work?

Would the Xeon E3-1225v3 benefit from the internal graphics when running other VMs? I have read that a PCIe GFX can be added to provide some functions to a VM but would the internal also give this benefit?

I guess the safest way to add disks in a VM setup would be to mirror two and keep pools separate?
So I reiterate my previous question:
For data recovery, could I take one disk from a mirror and import it on another machine (If I don't stripe several vdevs together) in a degraded state and get data?
It also makes it simpler to migrate two disks at the time as finding a MB with eight working ports might be difficult, if I where to use Z2 with eight disks for example.

According to the ESXi guide it should work fine to just swap ESXi USB for FreeNAS USB and have it running if something goes wrong. Or If I want to dedicate the box to freenas later. (with the benefit of having 6 more ports for disks and the full 32GB RAM).

 

[cyberjock]

Would the Xeon E3-1225v3 benefit from the internal graphics when running other VMs? I have read that a PCIe GFX can be added to provide some functions to a VM but would the internal also give this benefit?

That's a question for the ESXi forums. We support FreeNAS here.

I guess the safest way to add disks in a VM setup would be to mirror two and keep pools separate?
So I reiterate my previous question:
For data recovery, could I take one disk from a mirror and import it on another machine (If I don't stripe several vdevs together) in a degraded state and get data?
It also makes it simpler to migrate two disks at the time as finding a MB with eight working ports might be difficult, if I where to use Z2 with eight disks for example.[/quote]

Breaking up pools deliberately is just a bad idea. Don't do it. They are meant to be used together. Use them as such.

 

[jgreco]

Is there a way to have ESXi display something other than DCUI on the console? I might have missed that... but I don't think so!

 

[no_connection]

That's a question for the ESXi forums. We support FreeNAS here.

I'm sure no one uses ESXi around here and only FreeNAS expertise can be found.
Anyway it seems VMware added support (no specific info on what) for several including intel into vSphere/ESXi 5.5 so it *should* be possible to use it as intended.

Breaking up pools deliberately is just a bad idea. Don't do it. They are meant to be used together. Use them as such.

I would start with separate pools with a single vdev in each. That way I could migrate data and just replace the pare if I wanted to. Same with moving them to a different system.
A big pool with several vdevs would be a little harder to move or get running again. I guess you could pluck a disk at the time and then use resilvering for more storage space, but it seems a bit dangerous.

What is the actual downsides to do it that way?
No performance gain due to more spindles, separate storage space 3tb usable each is two points I am aware of.

 

[no_connection]

Is there a way to have ESXi display something other than DCUI on the console? I might have missed that... but I don't think so!

ESXi can present the VM with hardware accelerated graphics(or software for that matter), a single physical GFX can be shared amongst several VMs.
http://www.vxpertise.net/2013/08/whats-new-in-vsphere-5-5/

 

[jgreco]

ESXi can present the VM with hardware accelerated graphics(or software for that matter), a single physical GFX can be shared amongst several VMs.
http://www.vxpertise.net/2013/08/whats-new-in-vsphere-5-5/

I've interpreted that to mean that it can give VM's access to GPU compute resources, not actually to take over the screen. It isn't clear to me that the onboard graphics can be used for that, so I guess if that's what was being asked, my answer is "dunno!"

 

[no_connection]

A screen output could possibly be made using VT-d but that is not my goal.
I simply want some of my VMs to have slightly better graphics available with the option of 3D graphics. For example simulation software with 3D viewport through VNC on a VM. Or just a tiny bit better performance when browsing the web.