Advanced n00b looking for some build guidance

[DingbatCA]

I have been building Linux based NAS for many years, both at the home and professional level. I know the limitation of Linux’s software RAID stack and it is now time for something new; FreeNAS + ZFS!

My build is a little complicated and I would love some guidance on what to do.

• Motherboard SuperMicro X8SI6-F /w onboard LSI 6Gb/s SAS 2008
• Xeon X3470 (4C), 12GB RAM
• 15 port SuperMicro SAS expander backplane
• 7X 5400RPM 2TB WD Green drives, RAID6 (Media/home directories/archives…)
• 2X 240GB SSDs Mirror (VM’s)
• 2X 16GB SLC SSDs, Mirror (Internal, using the onboard Intel SATA controller, OS mirror)

Currently the above hardware stack is running Linux and about 2~5 VMs inside of KVM. The VM’s are important as they run my house. Things like security cameras. My biggest concern is power draw, so I like to keep my big spinning disks powered down, most of the time.

My current plan: Using ESXi 6.0 (VMWare) to virtualize FreeNAS, then passing FreeNAS the LSI controller so it has real access to all the disks. My big question is: Should I cut off a few GB’s of SSD as a caching layer for the spinning disk array? If I only give the FreeNAS VM 4GB (2GB if i can get away with it) of RAM, I know that dedupe with take a performance hit. Is there any way I can place/force the dedupe table onto the SSD mirror in order to skimp on the RAM? I understand I would be trading performance for RAM and adding more wear and tear on the SSDs.

Performance: Currently I can maintain ~120MB/s on single threaded write while doing backups from my Windows PC to the spinning disks over CIFS on 1Gbit networking. I would like to be able to achieve that with my FreeNAS setup, but not required. VMs and power savings is much more important.

Any guidance on how to set this up would be great.

 

[SweetAndLow]

FreeNAS needs 8GB minimum of memory and most home setup should have 16GB of memory. There is a rough estimate 1GB memory for every 1TB of storage with a lower limit of 8GB.

Also if you want to do ssd read caching it doesn't really help until you get around 64GB memory. Just having a cache device will eat up some of your RAM so in low ram environments it actually hurts you more than helps you.

For being an advanced user I would suggest you read way more. There are lots of stickies on the forum that will teach you everything you need to know.

I'll also point out that if you want something that spins down disk, freenas probably isnt' for you. There is almost always some disk activity going on which will spin up the disk. The constant spin down and up of disk will kill them fast. It is best to just keep them running. My system with a xeon 1230, 16GB memory, 6 disk and fans on full only uses 40W at idle. I think that is pretty low power usage for most people.

 

[DingbatCA]

So why is there a 8GB RAM requirement? I read the docs and saw the specs. I know FreeBSD can run well on 256MB of ram, so what in FreeNAS is requiring all the RAM?

I will need dedupe on the SSD's, but the big spinning disks will see very little benefit from it....

 

[SweetAndLow]

hahahaha you need to read the doc's buddy. Dedup will increase your memory needs to very high quantities. Also dedup will not give you what you think it will give you.

Yes it is freebsd but has built in features and has an expectation of certain things. Read this slideshow: https://forums.freenas.org/index.ph...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

If you want to run that low of memory then just run bsd but you will not get the performance you are expecting.

 

[Jailer]

So why is there a 8GB RAM requirement?

In a nutshell, ZFS. Even FreeBSD with ZFS needs a lot of RAM.

 

[DingbatCA]

Thanks for the manual SweetAndLow. Read the whole thing. Just purchased 12GB of RAM to give me 24GB total. I will give all 12GB to FreeNAS. Still pondering how, and if, I should use ZIL and L2ARC.

 

[Jailer]

Still pondering how, and if, I should use ZIL and L2ARC.

If you're not sure if you need, you most likely don't.

 

[DingbatCA]

I will be using FreeNAS to provide storage for a few VM's (iSCSI), but those will live on SSD, so I dont think ZIL, or L2ARC will gain me anything?

 

[Robert Trevellyan]

I dont think ZIL, or L2ARC will gain me anything?

I think you're right.

 

[dpearcefl]

Anybody else see this?

Using ESXi 6.0 (VMWare) to virtualize FreeNAS​

Virtualizing FreeNAS is generally a bad idea.

 

[cyberjock]

Why do you think that *none* of the heavy hitters posted in this thread? :P