Intel Atom C3XXX build | Will it FreeNAS ?!

[LimeCrusher]

Dear FreeNAS users,

I recently moved in a home with Ethernet cabling all around so I decided it was time to get serious and start introducing decent NAS services in my household. The temptation evolved to become about a micro server including NAS functionalities plus a few extra things. I discovered FreeNAS and thought it might totally do the job.
At the same time, I discovered the Intel Atom C line of CPUs. I really liked those low-TDP little guys and thought they might be very adapted to a small, efficient, low-power home server. I am interested in building a FreeNAS powered machine with it and I would like your expert opinion on the build I envision. Let's go.

Required Functionalities:

• Basic NAS usage : storage and distribution of photos, movies, music, various documents through Windows-compatible share and NFS. Back-ups for two people laptops (a Windows machine and a Linux machine) and maybe an extra light workstation.
• Plex server for the Ethernet-connected 4k TV. We don't have any 4k files yet though, the TV takes FHD files and upscales them so I'm not sure about the transcoding capabilities required here
• Bittorent client, because why not.
• NextCloud server. I would like to have a few documents (or the entire NAS content) available anywhere around Earth.
• VM capable for running a Linux distribution for light tasks (maybe a MPD server, a Jupyter server ...) or just mess up in general.
  I feel like FreeNAS has all those functionalities, correct me if I'm misguided.

Based on that, I envision the following build around an Intel Atom C3558 and a 4 disks storage.
Proposed Build: Prices indicated in €, because privacy spoiler, that's where I live.

• CPU & Motherboard: ASrock C3558D4I-4L or SupermicroA2SDI-4C-HLN4F (about 280€ each), both embedding an Intel Atom C3558.
• RAM: 2 x 4GB DDR4 PC2400 CL17 ECC unbuffered Kingston ValueRAM (about 60€ each) for a total of 8GB of RAM.
• Boot drive: Samsung 970 EVO NVMe M.2 250GB (about 80€).
• PSU: Seasonic G-360 (about 70€).
• Case: Fractal Node 304 (about 85€).
• Disks: 4 x WD Red 2TB or Seagate Ironwolf 2TB for a total of 8TB and as low as I can get them from any vendor. I would grab 4 x 3TB if I can find a good deal (seen some lately) porting the total to 12TB.

My idea would be to simply bundle all those drives in a single ZFS vdev/pool in RAIDZ1.

Upgrade could be either a 8-cores C3758 embedded CPU (about 120€ more expensive) or 16GB of RAM.
To be fully honest, the final bill here is about 650€ without disks. That's about 100€ more than a Synology DS918+ and makes me wonder: is it really worth it ?

What are your thoughts ? I'd love to have your feedback.

 

[Alex98234]

FreeNAS recommends 8 GB base memory plus 1 GB per TB of storage. A 4 core atom will struggle to transcode 4k files however no transcoding is required if media files are in a native format your TV can process.

 

[kdragon75]

RAM: 2 x 4GB DDR4 PC2400 CL17 ECC unbuffered Kingston ValueRAM (about 60€ each) for a total of 8GB of RAM.

You should add at least 4GB more right off the bat. With Plex and NextCloud you yoill want at least 2GB more plus what you assign to the VM. On the subject of the VM, only assign the
MINIMUM needed to do the job. This likely means 512MB to 1GB and 1 to 2 vCPUs. The VM will run faster with 2 cores than 4. Trust me.

Disks: 4 x WD Red 2TB or Seagate Ironwolf 2TB for a total of 8TB

My idea would be to simply bundle all those drives in a single ZFS vdev/pool in RAIDZ1.

RAIDz1 means you will lose one disk worth of capacity to the parity data. You would then only have 6TB or in reality ~5.4TB of usable space. You will save some with compression (on by default) but dont expect much savings with photos or videos. Binary data, like that of a VM or backups tends to compress well if not already compressed.

plus 1 GB per TB of storage.

This only applies to "active" storage. By this I mean data that is accessed on a regular basis as the recommendation is to allow for a larger ARC (read cache). But if your serving video and bulk backup data there is no point in caching that data. You can in fact disable the ARC for non-metadata on a per dataset level leaving more ARC for things like the photos (if you edit from the NAS) or the Linux VM.

A 4 core atom will struggle to transcode 4k files however no transcoding is required if media files are in a native format your TV can process.

At 2.2GHz this has some truth. Perhaps someone with this CPU and vouch?

 

[Alex98234]

At 2.2GHz this has some truth. Perhaps someone with this CPU and vouch?

I am more concerned with the lack of cores. My 10 core E5-2680 uses 50 to 80% CPU transcoding raw 4k blueray media, especially ones > 60 GB in size.

 

[kdragon75]

My 10 core E5-2680

https://ark.intel.com/products/6458...-2680-20M-Cache-2-70-GHz-8-00-GT-s-Intel-QPI-
That only 6 cores.

 

[Alex98234]

https://ark.intel.com/products/6458...-2680-20M-Cache-2-70-GHz-8-00-GT-s-Intel-QPI-
That only 6 cores.

v2

https://ark.intel.com/products/75277/Intel-Xeon-Processor-E5-2680-v2-25M-Cache-2-80-GHz-

 

[kdragon75]

v2

https://ark.intel.com/products/75277/Intel-Xeon-Processor-E5-2680-v2-25M-Cache-2-80-GHz-

Details matter ;)

 

[Alex98234]

Details matter ;)

They sure do! My bad.

 

[LimeCrusher]

Alex98234, kdragon75, thank you for your feedback. It is much appreciated.
So, let's recap slowly!

Memory-wise!

FreeNAS recommends 8 GB base memory plus 1 GB per TB of storage.

You should add at least 4GB more right off the bat. With Plex and NextCloud you yoill want at least 2GB more plus what you assign to the VM.

Alright, alright! I'll happily throw an extra 60€ to get one more 4GB stick. Might as well just go for a total of 16GB. There's no such thing as too much RAM anyway right?

This only applies to "active" storage. By this I mean data that is accessed on a regular basis as the recommendation is to allow for a larger ARC (read cache). But if your serving video and bulk backup data there is no point in caching that data. You can in fact disable the ARC for non-metadata on a per dataset level leaving more ARC for things like the photos (if you edit from the NAS) or the Linux VM.

This is really interesting. As I understand I have a strong incentive in defining datasets and remove caching for a few ones like video files for instance. This could allow for a lighter RAM usage as less caching will be necessary right?

CPU-wise!

A 4 core atom will struggle to transcode 4k files however no transcoding is required if media files are in a native format your TV can process.

At 2.2GHz this has some truth. Perhaps someone with this CPU and vouch?

So I did my homework and tried to figure out how and when transcoding happens. Let's be accurate with the context first: I intend to run a Plex server on FreeNAS and use the Plex client of my Panasonic TX-49FX623. The Plex client will try to detect if the file format is supported by the TV set and if then try to use Direct Play or Direct Stream. If not, it will have no other option than to transcode.
Given the extensive list of format supported by the TV set, the very ordinariness of most of my files and the format supported by the Plex client, there is a good chance no transcoding will be required. The picture is not totally clear and there is actually a good chance that Plex fails to Direct Play or Direct Stream and feels like it must somehow transcode!
Of course, a good powerful Xeon E3 will guarantee that even if transcoding is needed, enough power will be available. But that probably bumps the total bill beyond what I budgeted :-/

Also, alternative options may exist. If Plex fails, I could probably still browse the NAS from my TV set and read the files directly from the NAS (less user-friendly). Most ISP in France provide you with a nice little internet box thought as a media player that can connect to your NAS. Again, less user-friendly but I've bee doing that for the last ... 8 years or so and have rarely encountered video files I could not play.

I am curious though, a Synology DS918+ pretends it can transcode 2 channels, 30 FPS @ 4K (4096 x 2160) in H.264 (AVC)/H.265 (HEVC). This thing has an Intel Celeron J3455 with a Passmark score of 2146 while the Atom C3558 has a score of 2538. They both have 4 cores and are roughly equivalent (one is desktop-oriented, the other is server-oriented). I would then expect the Atom C3558 to be able to do more or less the same. What do you think ?

To be honest, Ericloewe's Hardware Guide mentioned the legacy Atom C2550 as a realistic option for a light FreeNAS usage. I thought the more recent Atom C3558 might as well work. I could always upgrade to a 8-cores C3758 (as the legacy 2750 is mentioned for heavy usage) but once again, the price bumps :-/

ZFS-wise!

RAIDz1 means you will lose one disk worth of capacity to the parity data. You would then only have 6TB or in reality ~5.4TB of usable space. You will save some with compression (on by default) but don't expect much savings with photos or videos. Binary data, like that of a VM or backups tends to compress well if not already compressed.

Yes, I am aware of that. Nothing's free right? Especially not data integrity. I might actually grab four 3TB disks.

VM-wise!

On the subject of the VM, only assign the MINIMUM needed to do the job. This likely means 512MB to 1GB and 1 to 2 vCPUs. The VM will run faster with 2 cores than 4. Trust me.

I certainly do.

What are your thoughts FreeNAS wizards ?

 

[CraigD]

I really liked those low-TDP little guys and thought they might be very adapted to a small, efficient, low-power home server. I am interested in building a FreeNAS powered machine with it and I would like your expert opinion on the build I envision. Let's go.

Required Functionalities:

• Basic NAS usage : storage and distribution of photos, movies, music, various documents through Windows-compatible share and NFS. Back-ups for two people laptops (a Windows machine and a Linux machine) and maybe an extra light workstation.
• Plex server for the Ethernet-connected 4k TV. We don't have any 4k files yet though, the TV takes FHD files and upscales them so I'm not sure about the transcoding capabilities required here
• Bittorent client, because why not.
• NextCloud server. I would like to have a few documents (or the entire NAS content) available anywhere around Earth.
• VM capable for running a Linux distribution for light tasks (maybe a MPD server, a Jupyter server ...) or just mess up in general.

What are your thoughts ? I'd love to have your feedback.

A CPU uses little power when idle, a low TDP cant go faster when needed...

PLEX is great, as stated above 4K transcoding is beyond your budget

Why not buy used enterprise gear if money is tight?

Have Fun

 

[kdragon75]

There's no such thing as too much RAM anyway right?

Ha, nope!

This is really interesting. As I understand I have a strong incentive in defining datasets and remove caching for a few ones like video files for instance. This could allow for a lighter RAM usage as less caching will be necessary right?

That's the idea. I have yet to do this my self but I am guessing my ARC hit ratio will be much better for my VMs as less will get replaced by movies that are watched once.

The picture is not totally clear and there is actually a good chance that Plex fails to Direct Play or Direct Stream and feels like it must somehow transcode!

If you have the space to spare, Plex can pre transcode or "optimise" for you. There is no requirement to transcode on-the-fly. Hmm can flys transcode video....

I am curious though, a Synology DS918+ pretends it can transcode 2 channels, 30 FPS @ 4K (4096 x 2160) in H.264 (AVC)/H.265 (HEVC). This thing has an Intel Celeron J3455 with a Passmark score of 2146 while the Atom C3558 has a score of 2538. They both have 4 cores and are roughly equivalent (one is desktop-oriented, the other is server-oriented). I would then expect the Atom C3558 to be able to do more or less the same. What do you think ?

This is tricky. I don't know what the Synology uses to transcode. It may have some sort of acceleration somewhere but I doubt it. The other thing is the content of the video its self. A static scene me be very easy to decode and re-encode with fast details motion is much more intense.

Yes, I am aware of that. Nothing's free right? Especially not data integrity. I might actually grab four 3TB disks.

Yeah that was as much for you as anyone else that may read and misinterpret that.

 

[LimeCrusher]

A CPU uses little power when idle, a low TDP can't go faster when needed...

True that!

PLEX is great, as stated above 4K transcoding is beyond your budget

Would you say that Plex is pointless without the guarantee of the capability to do 4k transcoding ?

Why not buy used enterprise gear if money is tight?

I am 100% opened to that option! I'm just not an IT professional so my acquaintance with the Intel Xeon E3 or E5 lines was low (but rapidly growing those last 24 hours!). I see a bunch of used motherboards and CPUs on Ebay and I am certain I could build a decent machine with that, exceeding the performance of a brand new Intel Atom C3558 for a very similar price.
I have been reading Ericloewe's hardware guide (which is really good) to that purpose. If you have any suggestion, I'll be more than happy to check on them!

If you have the space to spare, Plex can pre transcode or "optimise" for you. There is no requirement to transcode on-the-fly. Hmm can flys transcode video....

HERE'S THE GODDAMNED GOOD QUESTION!
Joke aside, I wasn't aware this pre-transcode thing. I'll look it up.

Yeah that was as much for you as anyone else that may read and misinterpret that.

SAY IT AGAIN LOUDER FOR THE FOLKS IN THE BACK!

 

[kdragon75]

A CPU uses little power when idle, a low TDP can't go faster when needed...

This is not always true. Older generations still use a ton of power as idle. Not to mention that full servers are built to be more flexible that power efficient. Im not saying its a bad idea but to get anywhere close to the power consumption of the Atom platform (not just the CPU) you will need an equally new Xeon. I have seen old Xeon X5400 systems idle at 250+ watts with minimal configurations.

 

[CraigD]

@LimeCrusher I've sent you a PM about my love for PLEX

@Chris Moore is the Ebay guru he may be able to help

You may have noticed everyone commenting in this thread has very different CPUs, but we all have massive amounts of RAM

Have Fun

 

[CraigD]

I was where you are now in May 2016
https://forums.freenas.org/index.php?threads/my-freenas-journey-first-9-months.50181/

 

[LimeCrusher]

I was where you are now in May 2016
https://forums.freenas.org/index.php?threads/my-freenas-journey-first-9-months.50181/

Thanks! Your story really help me understand what I am trying to achieve here. I just wish I had a test machine to mess up with ZFS here before jumping in for good.

 

[LimeCrusher]

Update and Temporary Conclusion

Thanks to the feedback provided by @Alex98234 @kdragon75 and @CraigD and also the excellent Hardware Guide put together by @Ericloewe, I have had the opportunity to think a bit more about what I wanted to achieve and the build I proposed above. In the following, I suggest a build update still based on an Intel Atom C3XXX, discuss why it may not be the best idea (as of late 2018) for my use case and what may be the alternative.

Updated Build
This build is meant to exploit an Intel Atom C3558 for a simple home NAS. Here is how I would do it:

• CPU & Motherboard: ASrock C3558D4I-4L or SupermicroA2SDI-4C-HLN4F (about 280€ each), both embedding an Intel Atom C3558.
• RAM: 1 x 16GB DDR4 PC2133 ECC CL15 Unbuffered Hynix (about 159€).
• Boot drive: ... whatever M.2 SSD that is not known to be utter crap? I am very skeptical of the interest of getting Samsung latest consumer SSD at 80€. (so let's say about 40€).
• PSU: Seasonic G-360, because it's gold rated (about 70€).
• Case: Fractal Node 304, because it's nice (about 85€).
• Disks: 4 x 2TB or 4 x 3TB of either WD Red or Seagate Ironwolf depending on your needs and budget.

Now is that build any good ? Good question. Here are my humble take on the pros and cons:
PROS:

• The motherboard are truly great server-grade motherboards! Up to 64GB unbuffered ECC or 128GB of registered ECC 2400MHz DDR4, a PCIE x8 port, five SATA 3 connectors, two mini SAS connectors allowing you to had eight more SATA, four 1Gbps Ethernet ports, a dedicated IPMI Ethernet port. All this on a mini-ITX format!
• The CPU is pretty modern and supports up to twelve SATA, has VT-x, VT-d, EPT, AES-NI.
• The sum probably barely draws more than 20W at peak (16W TDP CPU). That's incredibly low for that much IO. With four disks, you'll probably experience a peak power of ... 150W at best?
• You can easily add RAM by just throwing more sticks of the same.
• You can easily add more disks directly on the motherboard though you may have to change the case and PSU to something more fit.

CONS:

• The Atom C3558 has a Passmark score of about 2500. While you may be able to host a Plex server and transcode a 1080p video with it but probably not much more. What if I start editing 25MB raw picture on my workstation at the same time or sync things with a NextCloud server? And use a bittorrent client at the same time? Is it going to struggle? I don't know. I suspect it may.
• The motherboard has a great connectivity but for 280€ the CPU power feels a bit weak.
• My proposed build only takes advantage of four out of twelve SATA. Sure there's room to grow but am I not missing the strength of the motherboard here?
• The CPU is embedded. You can't replace it with a more powerful one. Your best evolution in the Atom C line is the C3758 which costs about 400€ embedded in the same motherboard. It is pretty expensive to reach a Passmark score of about 5000.

Conclusion (as of late 2018)
My feeling is that an Intel Atom C3558, embedded in this nice motherboard, is totally capable of being the backbone of a medium NAS up to twelve disks while offering a very small footprint and consumption. Please correct me if I'm misguided but I feel like such a medium NAS should be dedicated to file sharing and ABSOLUTELY NOTHING ELSE! No VM, no Plex, no NextCloud, no bittorrent, no nothing. Only storing safely and sharing, period. Am I wrong ? Please, let me know what you think.
This is, in my opinion, not very adapted to the newbie home user than I am. I don't need twelve disks, only four for now. I don't own any other computer to do all of the extra above but a 2011 laptop and I don't plan to buy an always-on, expensive, consumer-grade workstation to turn it into what would finally be used as an over-priced server.
I think I need something more tailored to my needs and after exchanging with @CraigD, I think I can build a cheaper, more powerful, more evolutive NAS based on a popular Supermicro motherboard (thanks again @Ericloewe for your hardware guide, did I thanked you enough for that?) by leveraging the Pentium / Xeon LG1151 socket compatibility. @CraigD has convinced me the Pentium is a relevant entry point and I will soon submit a build to the community.

 

[Evi Vanoost]

If you absolutely want to transcode, I would suggest using the ffmpeg/Intel QSV options. Not all processors has them but some do.

Given your 'budget' of $600, here is a configuration with all you desire (QSV, ECC):
APEX SK-386-C Black ATX Mid Tower Computer Case - $30
NEMIX SuperMicro DDR4 16G ECC - $120
Intel Xeon E3-1225 V6 - $234.99
SUPERMICRO MBD-X11SSM-F-O - $199.99
120G NAND SSD or 2x32G USB - $20 (or free if you know where to look)

FreeNAS doesn't require much for a boot drive, it doesn't read/write to it much after loading the OS. You could use a 16/32GB USB drive (mirrored) and it works, those SuperMicro have USB internally, I've done it for years in large production servers before SATADOM became affordable.

For internal drives, 2TB = $75; 4TB and 3TB = $100. I would suggest going directly to 4x4TB in RAIDZ = 12TB yield.

 

[LimeCrusher]

If you absolutely want to transcode, I would suggest using the ffmpeg/Intel QSV options. Not all processors has them but some do.

Given your 'budget' of $600, here is a configuration with all you desire (QSV, ECC):
APEX SK-386-C Black ATX Mid Tower Computer Case - $30
NEMIX SuperMicro DDR4 16G ECC - $120
Intel Xeon E3-1225 V6 - $234.99
SUPERMICRO MBD-X11SSM-F-O - $199.99
120G NAND SSD or 2x32G USB - $20 (or free if you know where to look)

Hey, thanks for the build suggestion Evi. This is actually pretty close to what I am considering but I will probably start only with an Intel Pentium G4560 for now. The case you suggested is incredibly cheap but I'm afraid I can't find it in my country. I'll have to look for another option.
I've looked up a little bit those Intel Quick Sync (QSV) technology and there seems to be some good news for Plex on FreeNAS:

• Hardware acceleration using QSV seems to be supported on FreeBSD since very recently. It seems to require a Plex subscription though.
• The devs seem to be aware of the situation and seem to be working on it though I have a hard time guessing what's the exact status of that feature currently.

Hopefully this is functional very soon and may generally lighten the CPU workload in situation where transcoding is required. :)

FreeNAS doesn't require much for a boot drive, it doesn't read/write to it much after loading the OS. You could use a 16/32GB USB drive (mirrored) and it works, those SuperMicro have USB internally, I've done it for years in large production servers before SATADOM became affordable.

This is very interesting because it frees up a SATA port. A Supermicro X11SSM-F-O has eight SATA port for instance and using one for a boot SSD is one less available for an array of hard drives. I hoped it had a M.2 port but I'm not sure it is relevant for the market that motherboard is targeted at

For internal drives, 2TB = $75; 4TB and 3TB = $100. I would suggest going directly to 4x4TB in RAIDZ = 12TB yield.

Where I live right now it's more like 2TB = 75€; 3TB = $81; 4TB= 106€. So the 3TB is probably going to be my option. That will be enough capacity to start with.
I have doubts about that RAIDZ1 though ... I have read about the possibility of an URE during resilvering after a disk failure and when I run the Maths I freak out. I need to study this further to figure out if standard mirroring (trading capacity for safety) is not preferable.

 

[Evi Vanoost]

I'm not sure about Plex, but ffmpeg is free (and what most companies like Plex seem to be using on the backend). You may have to compile your own ffmpeg though but I'm pretty sure QSV can be used on *BSD - if nothing else a VM with proper passthrough of the CPU functionality should work as well.

I don't see what you would benefit from a "faster" boot SSD. FreeNAS doesn't store anything besides the configuration on the boot drive, not even a VM (should be on the ZFS volume) and you can/should point logs and stats to your ZFS volumes too so you literally only boot from the drive, most binaries will be in RAM for the uptime of your system.

You could use an M.2 SSD for ZIL or L2ARC but that's not really within your 'budget' anymore and also less flexible since your future system may not have a compatible M.2 SSD slot, I think M.2 SATA will go away in favor of NVMe.