BUILD The right CPU for the task (20 bay solution)

[Allan_M]

Hi everybody

This thread is an extension of my preliminary thread - read more here (prepare for wall of text): https://forums.freenas.org/index.php?threads/20-bay-build-suggestion-opinions.41123/

Glorious1 suggested that I posted a thread here regarding the choice of hardware. To summarize (quoted from original thread):

For specific hardware, I've been looking at the suggestions/advices given on this forum and came up with this: [prices are from a local vendor I'd like to support]

CPU:

• Single core price/performance*: Pentium G3258/3460 ~$65
• Multi core price/performance*: E3-1231v3 ~$235
• 'All-round' price/performance*: i3-4160/4170 ~$105-$110

Motherboard [ both with IPMI]:

• Intel DBS1200V3RPS ~$160
• X10SLL+-F-O ~$200

Memory [on the QVL]:

• Samsung DDR3-1600: ~$8/GB

Harddrive [WD]:

• RED NAS HDD ~$43/TB
• RED Network NAS HDD ~$48/TB
• RED Pro NAS HDD ~$54/TB

*Performance source: Passmark.com

I've set up af spreadsheet, doing the calculations and those three CPUs came out on top - taking into account; single and/or multi-core performance and local prices. I figured, the final setup should also include 2x M1015's and a '4x SATA to SFF8087'-cable. The backplane in the case I've already purchased has five SFF8087 connectors. That's why I'm thinking of buying the SATA to SFF8087 cable.

My primary concern, however, is the choice of CPU. The shares will be a combination of SMB and AFP (not on the same datasets or pools for that matter). I'd like to keep the performance around 60-100* MB/s for sustained read/write - not necessarily random I/O - I'm led to believe this is quite unreasonable with this setup.

I'd like to hear, if I'm way off - considering the possible choices of hardware or some of it could be considered overkill (especially the E3-1231V3 vs. i3 vs. G3xxx)?

* - 108 MB/s should be possible. At least it is, between my two desktop computers (both equipped with SSDs).

 

[AVB]

I have a 10 drive, 30TB, 1Vdev, RaidZ2, Sata II connected pool on my FreeNas box (horror! AMD 6 core 3Ghz, 16 Mb ram) and regularly get 110 MB/s copying to and from 3 different drives (Sata III) on my desktop one of which is an SSD. Both are connected to the same unmanaged TrendNet switch. This is with pretty large files though anywhere from 2-40GB in size. For smaller files it drops to 60-70 MB/s copying to the pool. This is also single user with no other demands on the system.

 

[Allan_M]

I have a 10 drive, 30TB, 1Vdev, RaidZ2, Sata II connected pool on my FreeNas box (horror! AMD 6 core 3Ghz, 16 Mb ram) and regularly get 110 MB/s copying to and from 3 different drives (Sata III) on my desktop one of which is an SSD. Both are connected to the same unmanaged TrendNet switch. This is with pretty large files though anywhere from 2-40GB in size. For smaller files it drops to 60-70 MB/s copying to the pool. This is also single user with no other demands on the system.

Though I do appreciate the perspective, I'm not entirely sure what to make of it. Are you telling me that I need a 6 core AMD (and 16 Mb of RAM? ;)) or just telling me stuff (dis)regarding the CPUs I asked about? :)

 

[AVB]

No, I'm just saying that the speeds you are looking for should be obtainable. I put the processor info in just so you would know a somewhat lower quality build can do it.

Though I do appreciate the perspective, I'm not entirely sure what to make of it. Are you telling me that I need a 6 core AMD (and 16 Mb of RAM? ;)) or just telling me stuff (dis)regarding the CPUs I asked about? :)

 

[Allan_M]

No, I'm just saying that the speeds you are looking for should be obtainable. I put the processor info in just so you would know a somewhat lower quality build can do it.

Makes sense. Like I said, I do appreciate the perspective :)

 

[tvsjr]

SMB is single-threaded, so you'd prefer faster cores over more cores. I don't see you listing 10GbE in your post anywhere... if you're limited to gigabit ethernet, the reality is that any of these processors will likely be fine. When you mentioned copying between two systems with SSDs, you mentioned 108MB/sec - you're being limited by the gigabit ethernet between the systems. If you had 10GbE between them, performance would be far higher.

Personally, I'd go with the Xeon just because I feel they are better suited to a dedicated server-class system, but that's a personal bias. Make sure whatever you select (CPU, motherboard, and chipset) support ECC, and that your memory selection is also ECC.

 

[Allan_M]

SMB is single-threaded, so you'd prefer faster cores over more cores.

I know.

I don't see you listing 10GbE in your post anywhere... if you're limited to gigabit ethernet, the reality is that any of these processors will likely be fine.

Super. I wasn't sure 'cause I couldn't find any specific details about the correlation between CPU single/multi-core performance and SMB - only that it is single-core on a pr. user basis [one user = 1 core, 2 user = 2 cores, and so on]. I just followed the advice from the slideshow; min. 2.5 GHz and ECC support.

When you mentioned copying between two systems with SSDs, you mentioned 108MB/sec - you're being limited by the gigabit ethernet between the systems. If you had 10GbE between them, performance would be far higher.

I know. This information was given preemptively, coupled with when I said I aimed for 60-100 MB/s of sustained performance, before anyone would ask 'well, then, are your equipment even able to deliver those speed? You should know, 100 Mbps is limited to a theoretical 12.5 MB/s...' so on and so forth :)

Personally, I'd go with the Xeon just because I feel they are better suited to a dedicated server-class system, but that's a personal bias.

Sounds like all the personal bias I need, to justify the purchase of a E3 CPU ;)

Make sure whatever you select (CPU, motherboard, and chipset) support ECC, and that your memory selection is also ECC.

I thought this was obvious from the list of hardware suggestions - but it should be ECC-supported through the entire 'chain'. The only component I wasn't entirely sure about was the i3's - but I double and triple checked with various forum posts around the web.

Thank for your weigh in. Do you have any thoughts on the list of hard drives? :)

 

[tvsjr]

WD Red or HGST NAS. No need for WD Red Pro - the additional RPM means more heat for a negligible increase in performance (small enough you'll never "feel" it). Don't buy anything Seagate.

 

[Allan_M]

WD Red or HGST NAS. No need for WD Red Pro - the additional RPM means more heat for a negligible increase in performance (small enough you'll never "feel" it). Don't buy anything Seagate.

Super. Thanks :)

 

[Allan_M]

Update:

I got some of the parts the other day, and have been assembling and trying out the new toy. I'm still waiting for some parts to arrive (two 9211-8i's + a reverse SATA to SFF-8087 cable), but for now I've got the essentials.

The board (X10SLL+-F) is equipped with 6 SATA ports, enough for some preliminary testing and getting to play around with FreeNAS. I dusted of some old 2.5" spinners (3x 320 GB + 3x 120 GB), set up two Z1 pools [just for testing/playing around] and have been running some tests.

My goal was to achieve 60-100 MB/s of throughput and was astonished how 'easy' that was. The attached files are the results from my girlfriends Windows-laptop (WiFi, 100 mbps and 1 gbps). On my Mac Mini, I settled for some runs of BlackMagic Disk Speed Test.

MB/s (up/down)
1 instance; 62.60/61.08
+3 instances; 117.21/109.80

Just for fun, I tried with some files, and the results where almost identical. The CPU didn't go above 15 % of usage (hovered around 5-10 % most of the time).

EDIT: I've also attached an image of the temporary setup.