BUILD Building a FreeNAS box

[Mark Holtz]

BTW... Yes, the system is running with a NVMe drive, and, as you can see with the screenshot attached, it is identified as nvd0 (as well as my 2TB drive being identified as ada0). The BIOS has been configured accordingly to use that NVMe drive as the boot drive and to ignore everything else. (I can smash F11 during boot-up if I want to boot from USB).

The NVMe drive was the smallest I could get at $100 for 128GB. My guess is that it is also about 8 times bigger than what I really need for a FreeNAS build. Unfortunately, it is also the smallest size NVMe drive I can get. It is probably booting faster than a USB but, more importantly, it is not taking up a SATA port on the motherboard. Yes, I am only using 1 drive now, but, in a few months, that will be changed over and a eight NAS drive setup will be in place. The funny part is that, after bootup but before hooking up the hard drive, I managed to find a very cosmetic bug in the Plugins page.

Does that mean that I'm the first person to use a NVMe in a FreeNAS build? I hope not.

 

[joeschmuck]

Does that mean that I'm the first person to use a NVMe in a FreeNAS build? I hope not.

I don't know if you are but if not, you are likely one of very few who are using one. I'm glad it's working for you. If you run into anything weird because of it, please post all the details.

 

[Mark Holtz]

So, why am I getting an aftermarket cooler for my FreeNAS build? I'm not a fan of the stock coolers from either Intel or AMD, and even a low-end cooler such as the Cooler Master 212EVO, costing $25 after rebate, does a better job of cooling, and fits nicely into the case. There are much bigger and more expensive coolers like the Cooler Master V8 GTS or the be quiet! DARK ROCK PRO 3 Silentwings, but even I say they are a waste of money because you don't overclock a server.

With the installation of the cooler, the case has been closed up, the FreeNAS server placed in the corner, and the build is now in the status of "To Be Continued". The next step is to get the hard drives, and that won't be for a few months. I have what I needed to accomplish for the time being. I will also figure out why my UPS isn't communicating with my server. Right now, though, I have high priority issues.

 

[Mark Holtz]

And, now, I've installed Plex as a media server. This, of course, involves transcoding, which is why I got the fast processor in the first place.

 

[joeschmuck]

And, now, I've installed Plex as a media server. This, of course, involves transcoding, which is why I got the fast processor in the first place.

It's always nice to have the transcoding feature but if your media were already in a compatible format for the playback devices, that would negate needing to transcode on the fly. All my videos are like this so I could use just a DLNA server if I wanted to. Of course I like the Plex interface so I'll stay with it.

 

[Mark Holtz]

And, Newegg had a deal on HGST 5TB drives today. I can live with $35 per TB. Utilizing RAID-Z2, that should give me 27.3 TB / 27939.7 GB of storage. Nice.

 

[joeschmuck]

Yup, saw that this morning too, not a bad deal. Just remember that those drives are 7200 RPM so they will generate more heat so just keep an eye on the drive temperatures (smartctl will report the current and the high temp).

 

[Ceetan]

And, Newegg had a deal on HGST 5TB drives today. I can live with $35 per TB. Utilizing RAID-Z2, that should give me 27.3 TB / 27939.7 GB of storage. Nice.

Any data on heat (drive temp) of the build, Mark?

 

[Mark Holtz]

No idea on the thermals, although they are running cooler than the 2TB I was running in there. I just installed them yesterday afternoon and have it configured for RAIDZ2. It's nice having a massive hard drive like this: :)



This is also allowing me to finally complete some configurations which I held off such as UPS integration. :)

 

[joeschmuck]

That is a lot of storage. Enjoy it.

 

[Ceetan]

If you get any more data on thermals and power consumption, I'd like to know more, as I am planning a similar build.

 

[joeschmuck]

I'd recommend grabbing the thermals form the hard drives after you load them up with some large amount of data and then run a scrub. During the scrub (after an hour of continuous running) check the Smart data for drive temps form all the drives and that will give you a fair indication on how hot the drives get.

 

[Mark Holtz]

I have set up backup jobs for three of my four computers now (the fourth is my personal laptop which is at work, and the backup job is manually run), all of my files have been transferred from my external drives to the NAS system, periodic snapshots have been set up for almost all of the drive shares, media server is back online, and I'm only using 13% of my drive space. Now, to get everything organized.

 

[joeschmuck]

I don't recall how familiar you are with FreeNAS and Snapshots but pay attention to your storage. Your snapshots could eat up all your space in no time. The forums are flooded with postings of people in that situation.

 

[Mark Holtz]

I don't recall how familiar you are with FreeNAS and Snapshots but pay attention to your storage. Your snapshots could eat up all your space in no time. The forums are flooded with postings of people in that situation.

As much as FreeNAS is a good NAS solution, there are some things in the User Experience portion that could use some tweaking. There is a computing adage which goes "when in doubt, go with the defaults", and as you can see below, the defaults are hourly snapshots, Mondays to Fridays, 9:00- 18:00 (9 AM-6 PM), and non-recursive. When I was playing around with this on an "experimental" FreeNAS box, I set it up for the entire poll. Bad idea. Very bad idea. No wonder the disk space was consumed faster than PacMan at an all-you-can-eat buffet.



I've set the snapshots to run as follows in the screenshot below. Steam game backups and torrents are not worthy of snapshots. My biggest worry is my mother downloading something that decides to do a happy dance and scramble all the files on the shares. I may tweak the settings so that it's all 1 day.



Also, one of the 5TB drives suffered "infant mortality" on Sunday with only 61 hours of usage. It has already been received at Newegg, and I am awaiting a replacement to be shipped. The other seven drives are working just fine (thank you). It is set up for RAID-Z2 anyways.

 

[Mark Holtz]

And, the replacement drive arrived yesterday at work, but I didn't install it until this morning. The resilvering process took about four hours for 5 TB of data.

 

[Bidule0hm]

I don't recall how familiar you are with FreeNAS and Snapshots but pay attention to your storage. Your snapshots could eat up all your space in no time. The forums are flooded with postings of people in that situation.

The trick to have frequent snapshots and long retention time at the same time without flooding the free space is to have something like one snapshot every 10 min kept for 1 or 2 days, one every day kept for one week, one every week kept for a month and one every month kept for a year ;)

 

[Mark Holtz]

It's been several months, and I feel that I should write up a followup on this topic. Here are my observations:

• RAM prices have cone up like crazy. The Crucial 64GB Kit (4 x 16GB) DDR4-2133 ECC (UDIMM : CT7983920) that I ordered back in June for $370 is now going for $592 as of this writing.
• There has been no more hard drive failures.
• Because of heat issues from the hard drives, I changed the FreeNAS location to one that has better air circulation.
• Currently sitting at 18.6TB free out of 24.9TB.
• The NVMe drive is at 2.8GB capacity. Not unexpected, but I prefer the reliability over the USB in this situation.
• System load is extremely low. I like this.
• Replacing a 100Mb switch with a 1Gb switch really really makes a huge difference, especially when you are making TrueImage backups.

Know what I know now, what would I change? The only possibility is going with a RAIDZ3 config instead of RAIDZ2. And thats only a maybe.

 

[joeschmuck]

Know what I know now, what would I change? The only possibility is going with a RAIDZ3 config instead of RAIDZ2. And thats only a maybe.

You still have time before you start filling up your pool with so much data that you can't easliy store it off the system.

Currently sitting at 18.6TB free out of 24.9TB.

That is a lot of capacity, hopefully it lasts you a few years.

Replacing a 100Mb switch with a 1Gb switch really really makes a huge difference, especially when you are making TrueImage backups.

Yup, faster connectivity make a huge difference. I'd like to upgrade to 10Gb but the cost of updating my network is cost prohibitive right now.

 

[diedrichg]

• RAM prices have cone up like crazy. The Crucial 64GB Kit (4 x 16GB) DDR4-2133 ECC (UDIMM : CT7983920) that I ordered back in June for $370 is now going for $592 as of this writing.

• 
  And the industry says they are only going to continue to go up!