Board recommendation?

[Halk]

I've got FreeNAS running now but it's not using ECC RAM so I've decided to build a replacement so it's time for new CPU/Board/RAM.

DDR3 ECC RAM seems to be cheap and in a previous thread it was recommended so that seems to be something to aim for.

I've read that some 16GB module can cause problems so I'd like to stick to 8x8GB if I can find a board/cpu combo that will take them.

This server will only run FreeNAS, it'll serve the media files out and will bring in new ones by rsync from a remote server.

It has 8x12TB drives in there. Reading guides suggests 1GB per 1TB of storage but 96GB seems excessive - do I really need that?

I've had quite a few dead ends, wild goose chases etc looking on ebay trying to find a DDR3 board that will take 8 DIMMs and a modest CPU.

I don't think I need much in the way of a CPU.

Can someone with a bit more experience perhaps suggest some kind of starting point for me? As it stands I'm just blinding looking around and not finding a set.

 

[kdragon75]

Reading guides suggests 1GB per 1TB of storage but 96GB seems excessive - do I really need that?

Thanks for taking the time to read the guides. In your case, just serving a few big files to a few people, no. You should be fine with 16GB of RAM.

I don't think I need much in the way of a CPU.

as long as you're not trying to transcode media with Emby or Plex, you should be fine with any quad core.
@Chris Moore usually has some input here. If you don't mind second hand teh Dell R510 can be a great system as long as you don't mind power consumption. With 12 drives your probably in the 250 watt range.

 

[Halk]

At the moment it's 32GB of DDR3 non-ECC, a generic consumer motherboard and an i5-2300 and it seems fine. It only really struggled when I was copying from one array to another using an LSI card, a generic 4 port SATA and the motherboard headers, and it really struggled. Since then load numbers are minimal.

I don't mind second hand, feels like I'd be rescuing obsolete hardware. :)

 

[nemisisak]

I've got FreeNAS running now but it's not using ECC RAM so I've decided to build a replacement so it's time for new CPU/Board/RAM.

I've read that some 16GB module can cause problems so I'd like to stick to 8x8GB if I can find a board/cpu combo that will take them.

16Gb modules shouldn't cause any problems just try to get a good brand and try to ensure no mismatches.

I've got FreeNAS running now but it's not using ECC RAM so I've decided to build a replacement so it's time for new CPU/Board/RAM.
This server will only run FreeNAS, it'll serve the media files out and will bring in new ones by rsync from a remote server.

It has 8x12TB drives in there. Reading guides suggests 1GB per 1TB of storage but 96GB seems excessive - do I really need that?

From reading the forum you may have noticed many people saying ZFS loves RAM. The 1Gb per Tb rule is a rough estimate for your read cache i.e. ARC.

While you could run the system your proposing with around 16Gb RAM you will incur significant performance reduction anytime you try to do just about anything.

You should also consider that you have spent around £3k on storage space so whats an extra £100 of RAM to bring you up to scratch and ensure no issues, especially if one of those sticks derps out and corrupts your data because you're running minimum spec.

I've had quite a few dead ends, wild goose chases etc looking on ebay trying to find a DDR3 board that will take 8 DIMMs and a modest CPU.

I don't think I need much in the way of a CPU.

Can someone with a bit more experience perhaps suggest some kind of starting point for me? As it stands I'm just blinding looking around and not finding a set.

As you are looking for RAM, CPU & motherboard it may be cheaper to source a 2nd hand server from ebay rather than looking for individual parts. You will probably also save your self some money and its much more likely all components included will be in working condition as they would have been tested beforehand.

 

[Halk]

>While you could run the system your proposing with around 16Gb RAM you will incur significant performance reduction anytime you try to do just about anything.

Could you give me more detail on when that might become an issue?

I'm happy to spend any money that's justified. I like the idea of an obsolete server, and you could be right about a second hand server but I want it to have anti-vibration mounts like the Lian Li case I use has so likely I'd want to move it into that case and if it's all propriety....

 

[kdragon75]

will incur significant performance reduction anytime you try to do just about anything

If the disk without cache can fare more that saturate a 1Gb link, and he's only doing sequential IO, please explain the benefit of a huge ARC.

especially if one of those sticks derps out and corrupts your data because you're running minimum spec.

What are you talking about? He's getting ECC. Also there's no such thing as the scrub of death so don't even bring that nonsense up. If a full DIMM dropped of of ANY running system, the system would halt almost instantly. That is unless your running memory mirroring but I have never seen anyone go that far here.

As you are looking for RAM, CPU & motherboard it may be cheaper to source a 2nd hand server from ebay rather than looking for individual parts.

This is true.

ts much more likely all components included will be in working condition as they would have been tested beforehand.

This is a bad joke.

 

[kdragon75]

I want it to have anti-vibration mounts

That's more for if your using cheap drives and to keep noise down. We have racks FULL of drives and no vibration dampers. They run for 5+ years 24/7 and its not an issue.

 

[Halk]

Despite the server being up the loft when it was copying from the old to the new 8 of the drives weren't in the anti vibration mounts and I could hear the rattle through the ceiling so I do want to keep them in that if possible.

 

[kdragon75]

Despite the server being up the loft when it was copying from the old to the new 8 of the drives weren't in the anti vibration mounts and I could hear the rattle through the ceiling so I do want to keep them in that if possible.

Well if you end up with a hot swap case, that wont be an issue. If it is, sticky felt pads are great!

 

[Halk]

It does seem like a long way to go to avoid buying a motherboard ram and a CPU... the case is attractive, functional etc, has good airflow etc so I'm quite reluctant to change it. I'm also wary about buying a pre-built computer.

I'm still, I think, going to be on the lookout for components rather than a server.

 

[kdragon75]

It does seem like a long way to go to avoid buying a motherboard ram and a CPU... the case is attractive, functional etc, has good airflow etc so I'm quite reluctant to change it. I'm also wary about buying a pre-built computer.

I'm still, I think, going to be on the lookout for components rather than a server.

Take a look at the supermicro boards. There usually solid even if a bit picky about RAM. At least some of the older ones were.

 

[Halk]

Yeah it's them I've been trying to look at. However I've not looked since you said 16GB was probably plenty of RAM. That's going to open up lots of boards I suspect. I'll go for 32GB instead but it means 4x8GB will do the job, I'd been looking for boards that had 8 DIMM slots only so far.

Now I can look at boards with 4 slots, which should open things up considerably.

 

[Chris Moore]

I have not reviewed the entire thread, but I usually suggest Supermicro hardware because they have so many components that are interchangeable, which allows a lot of flexibility with the system you want to build. I made a suggestion in this thread that might be of interest. Many of the auctions have expired or sold out, but if you look at the original item I suggested, it will give you some ideas on hardware. I will try to take time to update the links later this evening.

https://forums.freenas.org/index.php?threads/new-freenas-build.64095/#post-459293

 

[Chris Moore]

I updated my previous suggestions with current product links. The system board I suggested is a standard ATX model that you can just as easily put into a tower like this:

Fractal-Design-Define-R5-FD-CA-DEF-R5-BK-Black-Silent-ATX-Midtower-Computer-Case - US $99.99
https://www.ebay.com/itm/263674323506

or into a rack mount chassis like the ones I use. Since it is single socket, it uses less power than a dual socket board of the similar generation and the Registered DDR3 ECC memory it can accept are less costly than the new production DDR4 modules. I picked up two of these boards in the early part of this year to upgrade two of my systems because I wanted more memory and it saved me over $200, each system, on the cost of the upgrade even though the prices were higher for the vintage gear when I bought it. Last I looked, the prices on the new gear were still high, so you might be saving even more. The new processors don't get you much more performance, but they do use a little less power.
Options.

 

[Halk]

Thanks Chris! I wouldn't spend any time updating links, that seems to be all US stuff so I wouldn't be buying the specific auctions, just using them as some kind of guide, as you said, to help me find parts.

A question on the dual CPU boards - do they both need to be populated?

As for the tower case, I already have an EATX that I found on eBay a few years ago.

It's a Lian Li PC-A71B and it has loads of space for drives etc.

 

[Stux]

It has 8x12TB drives in there. Reading guides suggests 1GB per 1TB of storage but 96GB seems excessive - do I really need that?

IMO, this rule is out of date. It’s more important what you plan to do with the system. Ie running VMs or perhaps iSCSI

16GB would probably be okay, and 32GB should be fine.

And then, If you want a larger ARC or L2Arc, you’d consider more ram.

 

[Chris Moore]

A question on the dual CPU boards - do they both need to be populated?

The specifics of how a dual CPU board works depends on the board. I was pretty surprised recently to find a board where adding (or not) a second processor had no effect on the number of PCIe lanes that were available for expansion cards, only on the number of cores available for compute.
In the Supermicro X10 series dual processor boards I have in the servers at work, if you don't populate both sockets, some of the PCIe slots are not working because they are tied to the second processor socket. That difference in design makes a big difference, because in one board, you loose nothing except memory capacity and CPU cores where in the other board you loose available expansion slots.
I don't want to tell you how to use your hardware, but you must take great care in the selection or you can create future pain.
If you don't anticipate the need for a second processor for additional computational cores, you would likely be better off with a single socket board, but you can run a dual socket board with a single socket, however there may be other consequences.

 

[jgreco]

IMO, this rule is out of date.

Not entirely. It's pretty valid at the smaller memory sizes, and back in the day when people would come here with their 4GB RAM systems, it was helpful to point to something that said "8GB minimum." But I've said for years that as you get out past 16GB that the rule softens a bit, and by 32GB you can easily do 2x the storage, probably 3x the storage, as long as you're not doing heavy workloads. This hasn't really changed. For an archival system, I'd expect you can do 100TB of disk on 32GB of RAM without issue. 16GB is definitely kinda tight, and the system will start to thrash if you get a lot of fragmentation.

 

[Chris Moore]

This hasn't really changed. For an archival system, I'd expect you can do 100TB of disk on 32GB of RAM without issue.

I have an archive system (at work) that has 230 TB of disk capacity and using gzip-9 compression, we have over 280 TB of data in it with a reported free space of 100 TB. This system ran fine with 32 GB of RAM. I upgraded it to 128 GB of memory just to try and get a little more speed from it for those occasions when data needs to be accessed, but I can't see that it made a notable change.

 

[Halk]

I'm using lz4 compression for the first time, and I'm getting a ratio of 1.00x - which to me reads that compression is doing exactly nothing. My thoughts are turn it off since there must be some hit somewhere on power consumption/ram/speed.

... I'm starting to get cold feet about this new server.

If I'm storing media - movies, tv shows etc and doing regular scrubs. How essential is ECC?

I did start out looking to replace 2 servers with a single server. And the gear for that was reasonably easy to find since I wanted to have a fairly powerful system to do two things VMs and NAS. I've been warned not to do this and that FreeNAS should have it's own box which has me looking for lower end hardware. Trying to find a lower end Supermicro board and CPU but also having it be able to take 32GB of RAM isn't so easy to find. Performance would likely be on par with the server as it is just now.

So it'd be quite a bit of hassle and literally the only outcome would be ECC RAM.

Is it worth it? I understand ECC RAM is/could be beneficial but in my use case is it really that desirable? If the answer is yes then I will build the server...