Question about the recommended motherboards

[nzd]

Hi,

Im new to freenas and scouting around the netz to read about it and order my first parts.
I came across (and read) the entire guide of hardware requirements and another pdf of slides so far that was about 70 pages.
Now under the hardware requirements I did find some supermicro motherboards that were mentioned and recommended. But my question is about a supermicro motherboard that was not in the list. The reason I ask this is because there was no other motherboard fitting my requirements in that list.
Requirements is minimum of 64GB ram, good intel NIC and DDR3 platform. Because I want so much RAM and DDR4 is extremly expensive compared to DDR3 so thats why I have to go with DDR3 platforms.

Now Im planning either 8 x 8TB drives in raidz2 och 10 x 8TB drives in raidz3 (please give advice here). The 8x8 would be enough for a few years but since freenas cant really be expanded easily Im thinking might as well add another drive plus make it a little bigger. Maybe even 11x8TB with Raidz3.
Because of this ammount anyway Im thinking of getting 128gb of RAM. Even if I would go with 8x8 TB I believie I can and should go with more RAM for better everything.. Right? I mean I get get 16gb ram sticks very cheap at the moment. Altough they are slow 1066mhz: https://www.serversupply.com/MEMORY/PC3-8500/16GB/HYNIX/HMT42GR7BMR4A-G7.htm (these are the specs) , I believie I read that speed does not matther for the RAMs right?

Now because the ammount of DDR3 RAM needed dependent on the storage I have to go with ATX boards. And there were no ATX boards with DDR3 in the recommended list. So I found 2 or 3 boards by myself from supermicro but now I wanna ask you experts if its a bad idea to go with these, or if theres anything else wrong with them, since they are not in the recommended hardware list. Obviously I am a beginner here and have no prior experience from serverbuilds so thats why I need to ask here..
Supermicro X9SRi
X9SRi-F
X9DRL-7F
X9DRL-EF

These are the motherboards I came across that fits my requriements..
Also I have read a little about SLOG and ZIL but Im didnt really grasp it 100% yet. I have a samsung 970 EVO 500GB that I was planning to use in the build some way. But I belivie it cannot be used for SLOG but could it be used for anything else that is actually needed? I belivie if I go with 128GB RAM I dont really need any more cache for normal home use right? So I might as well sell this card..?

Thank you so much for any inputs!

 

[MDD1963]

I'd reserve your 970 EVO as a primary OS disk in a WIndows build....

Use a 32 GB SSD, or a pair of them mirrored, for the FreeNAS OS... (few USB flash drives really handle the constant barrage of writes anymore, or so I've read)

Does your chosen board have an adequate amount of SATA ports (OS drive, plus 8-10 ports as you envisioned), or are you using some sort of HBA card?

 

[nzd]

I'd reserve your 970 EVO as a primary OS disk in a WIndows build....

Use a 32 GB SSD, or a pair of them mirrored, for the FreeNAS OS... (few USB flash drives really handle the constant barrage of writes any more, or so I've read)

Does your chosen board have an adequate amount of SATA ports (OS drive, plus 8-10 ports as you envisioned), or are you using some sort of HBA card?

Hi!
Thanks for your reply.
I already have a 970 evo as my OS disk on my windows machine. I will just sell this one then..
I have not chosen a board yet so that is mostly what this thread was thought to help me with. The four boards I have found that will suite me is not on the hardware recommended list so Im a bit worries that they will not work well.
Supermicro X9SRi
X9SRi-F
X9DRL-7F
X9DRL-EF
These are the boards, all supermicro.
They got mostly SATA2 ports except for one that got an integrated SAS2. But that one is not availible anywhere in europe that I have found. The one that is is the X9SRi-F. It has 2 SATA3 ports that I could use for the OS for example. Then I guess I need to buy 2 LSI HBA cards with 2i each, making it 16 total internal SATA3 drives. I belivie that a single LSI HBA card only has 8i each right..?

 

[Pitfrr]

RAM:
For a home usage, you don't necessary have to follow the 1GB per TB rule. It rather depends on your use cases: what are you going to use your FreeNAS box for: storage only, jails, VM?
Of course if you can afford the RAM (or if it is already available) then go for it!
You could start with 16 (with one stick) or 32 GB and then expand it easily if you have the need for.
I also don't think you should worry about the speed of the RAM, it is speedy enough!

Storage:
RAIDz2 or RAIDz3?
I would say that depends on your data sensitivity...
But again, for a home usage, I'd say RAIDz2 might just be fine.
And your choice of RAIDz level should be independent of your backup strategy (i.e. you still need to backup up your data regardless of RAIDz level).
Personally, it would depend on the hardware: I would maximize the number of disks based on the SATA ports available. If your hardware can manage 12 disks, then I'd go for 11 disks in RAIDz3 and 1 disk (small SSD: 32GB) for the OS.


Mainboard:
The two X9DRL mainboards you listed are dual CPU mainboards. I don't know if you will need dual CPU, so depending on your usage, it might be overkill (and if you use it in a single CPU set-up, you only can populate half or the RAM slots).
The X9DRL-7F integrates a SAS port but I don't know if it can be used as HBA, you should make sure of that and then you would have 8 extra ports for disks!
The X9SRi, single CPU, can apparently have up to 512GB of RAM so this leaves some room for upgrade! And has 10 SATA ports which is quite good (in that case, I might consider using mirror USB flash drives as boot device. Again, for a home usage, I wouldn't mind if the boot devices are not so reliable as long as the system configuration is saved.).

ZIL/SLOG:
If you're planning for a home use with single or few users, you won't need an SLOG or a ZIL.
As @MDD1963 said, I would also keep the 500GB SSD for something else, it would be a waste to use it as boot device (where only 8GB is needed) and you won't benefit from it as SLOG or ZIL.

 

[Pitfrr]

Oh, now I see the difference with the X9SRi and X9SRi-F, I didn't pay attention: it's the IPMI.
Would should go for a mainboard with IPMI, that's a great thing to have to avoid to have to connect to the server each time you do maintenance. Especially if the server is somewhere in a cupboard or garage or somewhere else...

 

[nzd]

RAM:
For a home usage, you don't necessary have to follow the 1GB per TB rule. It rather depends on your use cases: what are you going to use your FreeNAS box for: storage only, jails, VM?
Of course if you can afford the RAM (or if it is already available) then go for it! :)
You could start with 16 (with one stick) or 32 GB and then expand it easily if you have the need for.
I also don't think you should worry about the speed of the RAM, it is speedy enough! :)

Storage:
RAIDz2 or RAIDz3?
I would say that depends on your data sensitivity...
But again, for a home usage, I'd say RAIDz2 might just be fine.
And your choice of RAIDz level should be independent of your backup strategy (i.e. you still need to backup up your data regardless of RAIDz level).
Personally, it would depend on the hardware: I would maximize the number of disks based on the SATA ports available. If your hardware can manage 12 disks, then I'd go for 11 disks in RAIDz3 and 1 disk (small SSD: 32GB) for the OS.


Mainboard:
The two X9DRL mainboards you listed are dual CPU mainboards. I don't know if you will need dual CPU, so depending on your usage, it might be overkill (and if you use it in a single CPU set-up, you only can populate half or the RAM slots).
The X9DRL-7F integrates a SAS port but I don't know if it can be used as HBA, you should make sure of that and then you would have 8 extra ports for disks!
The X9SRi, single CPU, can apparently have up to 512GB of RAM so this leaves some room for upgrade! :) And has 10 SATA ports which is quite good (in that case, I might consider using mirror USB flash drives as boot device. Again, for a home usage, I wouldn't mind if the boot devices are not so reliable as long as the system configuration is saved.).

ZIL/SLOG:
If you're planning for a home use with single or few users, you won't need an SLOG or a ZIL.
As @MDD1963 said, I would also keep the 500GB SSD for something else, it would be a waste to use it as boot device (where only 8GB is needed) and you won't benefit from it as SLOG or ZIL.

Hi. Great info thanks!
Lets see if I can sum it up.
It is for home use as you said, backup, storage, some jails. VM I dont think. Ive read here in the forum guides that is is not recommended to run freenas as production inside something like esxi. So I guess it will just run barebone.
As for z2 or z3. Ive read that it takes a lot of time to resilver the array when a HDD needs to be replaced. Especially if Im gonna use like 10x 8TB drives. So I thought for a while that either 8 drives in z2 would be fine or if Im gonna have a little more storage then 10 drives in z3 would be fine.
I am also gonna have a backup of course, but only on the sensitive data. So if the entire array broke down the important stuff would be backed up always, but theres gonna be some dataloss that will be annoying. Like I ripped every single movie disc I ever owned and will keep doing. And if this breaks down I can still re-ripp them from the discs but that will be a lot of work and time.
Im gonna have this backed up on a synology so as time passes maybe I will add more and more drives to that backup device so that I could maybe eventually backup the entire freenas array.
For the mainboard the way I see it then the best alternative for me is the X9SRi-F which has 8 DIMMs and IPMI.
It does have 10 SATA ports but 8 of them are just SATA2 so Im not sure I wanna use them for my SATA3 storage devices. The HDD's are gonna be WD Reds.
If RAM speed really doesnt matther then I can easily get 128gb of ram no problem. Then I dont need to think about cache devices for my usecase I belivie. And probably SLOGG and ZIL is overkill as well for my home usecase it seems..

Did I understand everything correct? What do you think about these RAMs btw:
HYNIX HMT42GR7BMR4A-G7 16GB (1X16GB) PC3-8500R 1066MHZ QUAD RANK X4 ECC REGISTERED CL7 1.35V DDR3 SDRAM 240-PIN RDIMM MEMORY MODULE FOR SERVER

Thanks!

Oh, now I see the difference with the X9SRi and X9SRi-F, I didn't pay attention: it's the IPMI.
Would should go for a mainboard with IPMI, that's a great thing to have to avoid to have to connect to the server each time you do maintenance. Especially if the server is somewhere in a cupboard or garage or somewhere else...

Right so I think X9SRi-F is what I should go for then.
Please also look through my above answer to the other post and see if you find anything else I should consider.
Thank you!

 

[Pitfrr]

Virtualization:
About FreeNAS and VMware... it is not recommended to virtualize FreeNAS because it's delicate and there are some strict rules to follow (you can also search in the forum) if you want to be sure to get it correctly.
But then with FreeNAS 11, you can also have some VM running in FreeNAS (rather than virtualising FreeNAS). I'm not up-to-date with this (since I'm still using FreeNAS 9.10) but it is worth a look.
So if you go the virtualizing FreeNAS route or having some VM running in FreeNAS, go for the RAM, sure! ;-).
But as said, you can start slowly with let's say 16GB or RAM and FreeNAS bare metal and get use to it. And then later on add more "complexity". The only thing to anticipate is storage since you won't be able to extend your pool easily once created (but for the rest it's not an issue).

RAIDz2/3:
Here again, I'd say for a home use you don't really care about how long the resilver process is. In a professional environment with lots of users, the resilver process might have some impact on the performances but in a home environment, you won't barely notice it.
So it really depends on the number of drive you can afford I'd say.

Mainboard:

It does have 10 SATA ports but 8 of them are just SATA2

Do you mean SATA2 vs SATA3, and you're worried about performances, is that it?
Well, like the difference in RAM speed, I'm not sure you will notice any difference between SATA2 and SATA3 ports in your usage. Because the bottleneck will be your network: SATA2 is 3Gb/s, SATA3 is 6Gb/s and your network most probably 1Gb/s.
And also no issues to mix SATA2 and SATA3 ports.
So in your case, if you chose the X9SRi-F, you could use the 10 SATA ports without needing an extra HBA (that you might add later on if you need to expand with a new volume).
In the end it really depends on how many drive you want to get...

RAM:
About the RAM module (Hynix HMT42GR7BMR4A-G7) I have no advice... It is advised to use a RAM module from the approved list from Supermicro for your choosen mainboard but it is not always possible... There is still a chance that this given module does not work properly. In any case, you should burn in the RAM (using memtest for exemple) before using it (the same goes for the hard drives).

Backup:
It seems a good backup strategy. You might consider to use one extra 8TB drive as external/off-site backup (for really irreplaceable data) that you synchronize once or twice a year.

 

[MDD1963]

Regarding the statements regarding runnig FreeNAS virtualized:

I have been successful running FreeNAS under HyperV, so far, but, did have to disable Hyper-V checkpointing at the start just after creation of the initial VM, as I was plagued with VM checkpointing errors/crashes until simply disabling checkpointing for that VM.

Disks taken offline within Windows admin/storage are then available as FreeNAS passthrough drives, so, it will certainly suffice to at least experiment/tinker with/learn basic admin tasks, etc... (For those curious)

 

[melloa]

I like the X9DRi-LN4F+ for a dual processor board. That's the one I use for my ESXi with FreeNAS running as a VM.

For my back-up FreeNAS running on bare metal, I use the X9SRL-F.

See specs below on my signature for more information.

 

[nzd]

Virtualization:
About FreeNAS and VMware... it is not recommended to virtualize FreeNAS because it's delicate and there are some strict rules to follow (you can also search in the forum) if you want to be sure to get it correctly.
But then with FreeNAS 11, you can also have some VM running in FreeNAS (rather than virtualising FreeNAS). I'm not up-to-date with this (since I'm still using FreeNAS 9.10) but it is worth a look.
So if you go the virtualizing FreeNAS route or having some VM running in FreeNAS, go for the RAM, sure! ;-).
But as said, you can start slowly with let's say 16GB or RAM and FreeNAS bare metal and get use to it. And then later on add more "complexity". The only thing to anticipate is storage since you won't be able to extend your pool easily once created (but for the rest it's not an issue).

RAIDz2/3:
Here again, I'd say for a home use you don't really care about how long the resilver process is. In a professional environment with lots of users, the resilver process might have some impact on the performances but in a home environment, you won't barely notice it.
So it really depends on the number of drive you can afford I'd say.

Mainboard:

Do you mean SATA2 vs SATA3, and you're worried about performances, is that it?
Well, like the difference in RAM speed, I'm not sure you will notice any difference between SATA2 and SATA3 ports in your usage. Because the bottleneck will be your network: SATA2 is 3Gb/s, SATA3 is 6Gb/s and your network most probably 1Gb/s.
And also no issues to mix SATA2 and SATA3 ports.
So in your case, if you chose the X9SRi-F, you could use the 10 SATA ports without needing an extra HBA (that you might add later on if you need to expand with a new volume).
In the end it really depends on how many drive you want to get...

RAM:
About the RAM module (Hynix HMT42GR7BMR4A-G7) I have no advice... It is advised to use a RAM module from the approved list from Supermicro for your choosen mainboard but it is not always possible... There is still a chance that this given module does not work properly. In any case, you should burn in the RAM (using memtest for exemple) before using it (the same goes for the hard drives).

Backup:
It seems a good backup strategy. You might consider to use one extra 8TB drive as external/off-site backup (for really irreplaceable data) that you synchronize once or twice a year.

About the resilver process regarding z2 or z3, Im not worried about the performance hit during resilver. Im worried that more drives brakes down since I dont have enterprise drive, its just regular WD Red drives that might hit problems during the large resilver size that makes the drives unresponsive and therefor drop out of the array. I dont remember what this is called but I know I read about this problem. That 10^x something means that in certain number of blocks there is a chance that drive is unreadable and that it will count as a broken drive for that reason..

Mainboard yes I mean SATA2 vs SATA3 speeds.. But yeah maybe its not a problem to use SATA2 ports for SATA3 drives in a NAS but just feels so bad.. Since its SATA3 drives.. One would have to read more about this. Maybe I could save myself one HBA card..

 

[Pitfrr]

If you burn in your drives, keep them under 40°C and don't power them down every day, there shouldn't be any issue during resilver.
You might have some drive failure but the failure rate should not be alarmingly high.

Yeah, I understand that intellectually it is not satisfying to have SATA3 drives on SATA2 ports... but, unless you have a 10Gb/s home network you shouldn't feel the difference.
In any case, you can still start like that and later on add an HBA card if you feel the performances are not to your satisfaction.

You can start "small" (no HBA, only one stick of RAM, boot device) and (easily) expand later. The only thing that you have to anticipate is the size of your volume, i.e. the number of disks and their configuration (RAIDz)