mITX build for NSC-800 case: 10GBe + 8xHDD + Socket CPU + L2ARC

[szydl0]

Hi!
I'm looking for a way to modernize my current build, based on NSC-800 case, consumer mobo and HBA M1015. My goal is as in the topic, to have at the end 10GBe connectivity, SATA ports for 8 HDD, socket CPU and it would be superb if it could have also some spare SATA for L2ARC.
I don't want to use Xeon-D boards, since they are expensive and have fixed CPU.
I want to stay with NSC-800, I like how tiny it is, but it can only fit 17x17 mITX.

My current option is to have AsRock C236 WSI combined with 10GBe card. The CPU would be E3-1240L V5. This way I can have almost all of my requirements. The only sacrifice is the lack of port for L2ARC, but maybe 32GB ARC would be reasonable for current 8x3TB raidz2 drives.

What is your opinion, do you have some better proposition? Thanks!

 

[kdragon75]

SATA for L2ARC.

Don't bother. Without getting into potential memory constraints, ZFS record size, etc.. SATA SSDs are just to slow. You pool with 8 disks will almost certainly be faster than a single SSD on SATA. If you really want an L2ARC, you need to coordinate your ZFS record size, RAM, and a proper NVMe drive.

What will the system be used for? How many users?

 

[szydl0]

The NAS is typical home usage, usually 1-2 users, max. 4 users. The NAS has several jails, i.e. Nextcloud, sometimes reencoding to x265 at low CPU usage, or HTTrack. NAS has mainly multimedia purpose - photo, video, steam library. Some VMs for fun. 10GBe connection needed only to one user.

Good to hear that there is no much gain from L2ARC in my case. I did not see similiar boards with M.2 slot.

 

[kdragon75]

Good to hear that there is no much gain from L2ARC in my case. I did not see similar boards with M.2 slot.

eah you have artificially constrained your hardware by insisting on microATX. You can safely skip the L2ARC though a SLOG would be nice for VMware unless you don't mind your VMs getting corrupted if there is a connectivity/power/any other issue that would cause writes to stop mid processing.

 

[Stux]

Well, I know you said no, but I’d look at a Xeon D.

The latest versions support 8HDs I believe, have 10gbit on board and leave you with a pcie slot that can be used with nvme.

 

[Ender117]

Don't bother. Without getting into potential memory constraints, ZFS record size, etc.. SATA SSDs are just to slow. You pool with 8 disks will almost certainly be faster than a single SSD on SATA. If you really want an L2ARC, you need to coordinate your ZFS record size, RAM, and a proper NVMe drive.

I don't think this is actually true. While the throughput of an sata SSD can be easily matched with HDD vdevs, same cannot be said on IOPS. An SSD can easily do 50k+ IOPS and a 8 disk striped mirrors will have hard time get over even 5k.

 

[szydl0]

Hi! Thanks for all of the responses. The Xeon-D unfortunately is over my budget - 4 core board with 10GBe is around double the cost of C236 WSI + E3-1240L V5.
And separate 10GBe cards are dirty cheap on ebay.

I'm thinking about bifucation PCIE 16x into 10GBe and NVMe - maybe I could manage some little space in my case. However I cannot find information whether it is possible with this board - then probably not. How bifuraction is supported in FreeBSD, are there some known problems?

Regarding VMs, I don't mind when they are not 100% secure in case of power loss etc. They are only for testing and reverting to previous state is not a problem.

 

[szydl0]

I've received info from Asrock that Bifurcation on C236 WSI is possible:

C236 WSI BIOS have "PCIE7 Link Width" item to support x8/x8 Riser card.

 

[szydl0]

Just to let you know - bifurcation works great on this board. I have Mellanox Connectx-3 and NVMe adapter with Corsair MP500 120GB as L2ARC. L2ARC is set to also cache continuous data. I can get 1GB/s transfer on cached data. Non-cached data have 350-400MB/s data rate, but I have wrong ashift set on pool - 512 size sectors instead of 4096.

I can recommend following riser, but it seams out of stock now:
https://www.amazon.com/Express-riser-useful-Mini-ITX-single/dp/B06Y5LBY8G

The rest of specs is as in the original post.

 

[Ender117]

Just to let you know - bifurcation works great on this board. I have Mellanox Connectx-3 and NVMe adapter with Corsair MP500 120GB as L2ARC. L2ARC is set to also cache continuous data. I can get 1GB/s transfer on cached data. Non-cached data have 350-400MB/s data rate, but I have wrong ashift set on pool - 512 size sectors instead of 4096.

I can recommend following riser, but it seams out of stock now:
https://www.amazon.com/Express-riser-useful-Mini-ITX-single/dp/B06Y5LBY8G

The rest of specs is as in the original post.

Why had you got ashift wrong? I thought it should be default to 12?

 

[szydl0]

The pool is around 3-4 years old, it was made badly that time and I lived with it, since I was still caped at 1gbe connection. Now I think is time for rebuild :).