-
Important Announcement for the TrueNAS Community.
The TrueNAS Community has now been moved. This forum has become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums
So you want some hardware suggestions.
- Thread starter jgreco
- Start date
- Status
So here I am again asking for your help and suggestion...
After few months of practice with freenas 9.1.1 in order to learn how to avoid the most commons mistakes and make it works properly, I was thinking about buying new hardware for a new NAS box.
My current set up is a really small and basic one, made with:
- SAPPHIRE AMD mini-itx E350 APU mobo (with a realtek integrated NIC...)
- 2 x 2TB wd green HDDs (mirrored)
- 2 x 2 GB non-ECC Corsair Ram
Before reading lots of posts on this great forum, i didn't knew anything about the need of using ECC ram and raidZ2 configuration, so the reason to make now a more reliable and properly structured NAS.
After all the reading, I was almost convinced to buy a SUPERMICRO X9SCM-F or an ASROCK E3C226D2I mobo because of the double intel NIC + IPMI, until I understood that Link Aggregation only worth with lots of users and is not instead useful to maximize bandwith with a single user as I wrongly thought so now I'm wondering (even because it's so hard to find these boards here in Italy) if there are "cheaper" boards with just 1 INTEL NIC that works properly with freenas, so that I could invest the saved money in more ECC ram or a better CPU.
Do you have any suggestions?
Thanks
After few months of practice with freenas 9.1.1 in order to learn how to avoid the most commons mistakes and make it works properly, I was thinking about buying new hardware for a new NAS box.
My current set up is a really small and basic one, made with:
- SAPPHIRE AMD mini-itx E350 APU mobo (with a realtek integrated NIC...)
- 2 x 2TB wd green HDDs (mirrored)
- 2 x 2 GB non-ECC Corsair Ram
Before reading lots of posts on this great forum, i didn't knew anything about the need of using ECC ram and raidZ2 configuration, so the reason to make now a more reliable and properly structured NAS.
After all the reading, I was almost convinced to buy a SUPERMICRO X9SCM-F or an ASROCK E3C226D2I mobo because of the double intel NIC + IPMI, until I understood that Link Aggregation only worth with lots of users and is not instead useful to maximize bandwith with a single user as I wrongly thought so now I'm wondering (even because it's so hard to find these boards here in Italy) if there are "cheaper" boards with just 1 INTEL NIC that works properly with freenas, so that I could invest the saved money in more ECC ram or a better CPU.
Do you have any suggestions?
Thanks
jgreco
Resident Grinch
- Joined
- May 29, 2011
- Messages
- 18,680
Intel NIC chips are most often found on serverboards and actually don't add much cost (lots less than an add-in card!). The problem with consumer boards is that Realteks work "well enough" for Windows. You could try seeing if Intel offers any suitable mainboards for desktops, I suppose...
There pretty much are none to choose from that I'm aware of. Mini-ITX comes with significant compromises(less SATA ports, fewer slots for expansion, etc) and pretty much there's no choices. You should be wary of Mini-ITX for a server anyway because you are so limited in expansion with that 1 slot that quite a few people buy it, then 3 months later realize they can't put an Intel NIC and an M1015 controller on the board at the same time(only one slot, remember).
i know this, but if i get a board with already an intel NIC on it, it would be enought for me. Consider that i'm not planning to use more than 4-6 HDD on it.
EDIT
obviously if the gain is just few bucks I'll go directly for the most used boards here in the forum...the problem is just to get them here in italy as i said...
Yeah, I don't know if I've even seen a mini-itx with an Intel NIC on it, regardless of what else you get. Just another compromise trying to go with mini-itx I believe. This is just based on forum experience and not my own searching. I don't search for mini-itx stuff at all because I don't consider it a wise way to spend money.
John M. Długosz
Contributor
- Joined
- Sep 22, 2013
- Messages
- 160
The IPMI is terrific! Don't pick a lower board and lose that too.
The desktop server motherboards are already rather small for ATX, since it doesn't have length for more slots nor a secondary chipset for gamer features. With a bunch of drives, you need a case big enough for that (and airflow) anyway.
jyavenard
Patron
- Joined
- Oct 16, 2013
- Messages
- 361
last year generation, but mini-ITX, ECC RAM support and dual intel gigabit NIC and.... IPMI
http://www.supermicro.com/products/motherboard/Xeon/QM77/X9SPV-M4.cfm
comes with the CPU:
http://ark.intel.com/products/65712
ECC support and AES hardware acceleration
for a whole list of supermicro mini-ITX boards:
http://www.supermicro.com/products/motherboard/Core/index.cfm#FCBGA1023
only one problem with those boards.... they cost an arm :) (maybe two)
http://www.supermicro.com/products/motherboard/Xeon/QM77/X9SPV-M4.cfm
comes with the CPU:
http://ark.intel.com/products/65712
ECC support and AES hardware acceleration
for a whole list of supermicro mini-ITX boards:
http://www.supermicro.com/products/motherboard/Core/index.cfm#FCBGA1023
only one problem with those boards.... they cost an arm :) (maybe two)
...but no-ECC ram...
well i guess the best choice keeps being between:
- SUPERMICRO X9SCM-F
- ASROCK E3C226D2I
- SUPERMICRO X10SLM-F
John M. Długosz
Contributor
- Joined
- Sep 22, 2013
- Messages
- 160
Gag!
The SO-DIMM memory in ECC will probably be pricy too.
But seriously, isn't the regular µATX board small enough at 244mm square? The mini-ITX is only 3 inches smaller.
Great opening post. It sadly makes me feel uncomfortable, but it's still a great opening post.
"Everywhere" i go people keep telling me that FreeNAS in some old hardware they had laying around is a much better alternative then some of the shelf home /small office NAS. It's (a lot) cheaper, it might use slightly more power, but since it basically was free, it's going to take a while before the cost is to the of the shelf NAS advantage.
Now, i've been using a HP DataVault X310, and a ReadyNAS NV+ (gen1), and they still do everything i need them to do. But the space is getting small. Yeah, admittedly the NV+ is a bit slow at torrents, but hey.
So i started looking around, again. Trying to figure out what i got laying around at home. I've been at this before, and never really got of the ground. Well, to be honest, i was looking around for stuff to build 2 things, a new Smoothwall system, and the possible NAS. The firewall is a bit more urgent, so i finally got off my ass. I found:
512-8192 MB DDR2 RAM, in different flavors and speeds.
Intel Core 2 Duo E8400 (65w TDP)
Gigabyte EP45-DS4 motherboard with:
2x Realtek 8111C 10/100/1000 LAN
1 x PCIE x16, 1 x PCIE x8, (Physically a x16 slot) 1 x PCIE x4, 3 x PCIE x1
1 x PCI
AMD Phenom X4 9850 Black Edition (125w TDP)
Gigabyte GA-MA790FX-DQ6 motherboard with:
2x Realtek 8111B 10/100/1000 LAN
2 x PCIE x16, 2 x PCIE x8, 1 x PCIE x1
2 x PCI
AMD Phenom X3 8550 (95w TDP)
Asus M4A785-M
1x Realtek 8112L 10/100/1000 LAN
1 x PCIE x16
1 x PCIE x1
2 x PCI
Integrated ATI Radeon HD4200
Where a GFX was needed i used a Radeon HD5450
All systems were tested with a Intel PRO/1000 PT Dual for a minumun of 3 gigabit ports total.
The C2D i dropped the vCore & clock on, ending up with a 1.2Ghz 2GBram system with 4 gigabitports at 60w idle, and 64w full load. Crazy, only 4 watt between idle and full load, but that will do for the firewall.
So, i got the two other systems. Since i was testing stuff out, i found out they could both be clocked as low as 800'ish MHz, but the X4, which i expected to be a good low volt candidate, wasn't, and the X3 could run at full speed at the X4's minimum voltage, and at 838-1100MHz on 1.075v. Ended up setting it at 1100 on the ASUS M4A785-M for a system total of 75w idle, and 87-88 full load.
At this point i realized the Firewall would be the C2D, and started looking at my 2 AMD systems in a bit of a different light. Which one would be the best NAS ? The ASUS, and it's single meaningful PCI-E slot seamed less attractive then the GA-MA790FX-DQ, so the X3 would have a new home. The slightly "fancier" motherboard drew a couple more watts, but over all it wasn't that bad. Started looking around for controller/RAID cards. Based on my findings, it looked like i was getting a Perc 6/i, leaving room for a H700 in the future as the "next" upgrade, and stick Windows Home Server 2011 on it. It would end up idling at around (now) 85-90w. Full load would be somewhat dependent of the drives of course, but this is the bare minimum.
So, i started comparing with FreeNAS since that was the raved about alternative. It IS free. And it uses cheaper controllers, which is a bonus. But then came the kicker. Where as i already have the motherboard, CPU, and all the memory to make the base of the system for a WHS2011 system, i apparently need all new hardware for FreeNAS to really be a viable alternative. This was my "WTF moment" where i was thinking "but, what about all the system based used hardware people are talking about?" Reading a bit more, i more and more realized that even if i could make it work on "old used hardware" i wasn't going to get away with having dropped the clock, and hence minimizing the power the system use. Scratching my head, and trying to remember. Bone stock, no storage drives, 143w, i think the system hit at full load, and 98 idle. Adding drives to that...
The X310 is at about 65w full load, i could run 3 of them, for 12 drives before the power bill is about equal. The ReadyNAS at 29w idle, and 43w full load is even worse, and unlike the X310 it provides protection against one failed drive. (Where as on the other hand, the x310 only loses whats on the failed drive/drives)
I can compete with these figures on a used system based of WHS2011, but it doesn't seam realistic to try and do so with FreeNAS. Ok, so, i cheat slightly, i already have a 12buck license for WHS2011. But even if i add that to the price of a Perc6/i, and compare it to a ServeRAID M1015, at best it's going to come out equal. I have to push the clocks back up, i might even have to run the X4, thats even worse for the wallet, not to mention the noise, and i'm still using old hardware, a none ECC configuration, with not enough memory, in a way thats not really recommended. Actually, the only system i ever had, that would have been "right" for something like this is a Compaq DL350 G5, and it's idle consumption power was well in to the 230+w.
The obvious issue here seams to be that the FreeNAS system design simply isn't aimed at me, unlike what proponents have been telling me. And the opening post here REALLY made that clear. It's a shame, but it's also saving everyone a lot of grief. I'm going to keep watching this forum, one of these days someone i know, or someone who wants to pay me, might need storage, and at that point they are paying the bills, and buying new stuff regardless, making this a much more valid option.
If there is something i'm missing, that might make my reasoning invalid, feel free to point it out, here, or in a PM. But my main point isn't really about FreeNAS, it's about the opening post. It's a great post, and should be a must read. It might have deterred me, since i feel that i might be better of using a low watt system, with a "smarter" drive sub system to do the math for me, but it's definitely a great post.
B!
"Everywhere" i go people keep telling me that FreeNAS in some old hardware they had laying around is a much better alternative then some of the shelf home /small office NAS. It's (a lot) cheaper, it might use slightly more power, but since it basically was free, it's going to take a while before the cost is to the of the shelf NAS advantage.
Now, i've been using a HP DataVault X310, and a ReadyNAS NV+ (gen1), and they still do everything i need them to do. But the space is getting small. Yeah, admittedly the NV+ is a bit slow at torrents, but hey.
So i started looking around, again. Trying to figure out what i got laying around at home. I've been at this before, and never really got of the ground. Well, to be honest, i was looking around for stuff to build 2 things, a new Smoothwall system, and the possible NAS. The firewall is a bit more urgent, so i finally got off my ass. I found:
512-8192 MB DDR2 RAM, in different flavors and speeds.
Intel Core 2 Duo E8400 (65w TDP)
Gigabyte EP45-DS4 motherboard with:
2x Realtek 8111C 10/100/1000 LAN
1 x PCIE x16, 1 x PCIE x8, (Physically a x16 slot) 1 x PCIE x4, 3 x PCIE x1
1 x PCI
AMD Phenom X4 9850 Black Edition (125w TDP)
Gigabyte GA-MA790FX-DQ6 motherboard with:
2x Realtek 8111B 10/100/1000 LAN
2 x PCIE x16, 2 x PCIE x8, 1 x PCIE x1
2 x PCI
AMD Phenom X3 8550 (95w TDP)
Asus M4A785-M
1x Realtek 8112L 10/100/1000 LAN
1 x PCIE x16
1 x PCIE x1
2 x PCI
Integrated ATI Radeon HD4200
Where a GFX was needed i used a Radeon HD5450
All systems were tested with a Intel PRO/1000 PT Dual for a minumun of 3 gigabit ports total.
The C2D i dropped the vCore & clock on, ending up with a 1.2Ghz 2GBram system with 4 gigabitports at 60w idle, and 64w full load. Crazy, only 4 watt between idle and full load, but that will do for the firewall.
So, i got the two other systems. Since i was testing stuff out, i found out they could both be clocked as low as 800'ish MHz, but the X4, which i expected to be a good low volt candidate, wasn't, and the X3 could run at full speed at the X4's minimum voltage, and at 838-1100MHz on 1.075v. Ended up setting it at 1100 on the ASUS M4A785-M for a system total of 75w idle, and 87-88 full load.
At this point i realized the Firewall would be the C2D, and started looking at my 2 AMD systems in a bit of a different light. Which one would be the best NAS ? The ASUS, and it's single meaningful PCI-E slot seamed less attractive then the GA-MA790FX-DQ, so the X3 would have a new home. The slightly "fancier" motherboard drew a couple more watts, but over all it wasn't that bad. Started looking around for controller/RAID cards. Based on my findings, it looked like i was getting a Perc 6/i, leaving room for a H700 in the future as the "next" upgrade, and stick Windows Home Server 2011 on it. It would end up idling at around (now) 85-90w. Full load would be somewhat dependent of the drives of course, but this is the bare minimum.
So, i started comparing with FreeNAS since that was the raved about alternative. It IS free. And it uses cheaper controllers, which is a bonus. But then came the kicker. Where as i already have the motherboard, CPU, and all the memory to make the base of the system for a WHS2011 system, i apparently need all new hardware for FreeNAS to really be a viable alternative. This was my "WTF moment" where i was thinking "but, what about all the system based used hardware people are talking about?" Reading a bit more, i more and more realized that even if i could make it work on "old used hardware" i wasn't going to get away with having dropped the clock, and hence minimizing the power the system use. Scratching my head, and trying to remember. Bone stock, no storage drives, 143w, i think the system hit at full load, and 98 idle. Adding drives to that...
The X310 is at about 65w full load, i could run 3 of them, for 12 drives before the power bill is about equal. The ReadyNAS at 29w idle, and 43w full load is even worse, and unlike the X310 it provides protection against one failed drive. (Where as on the other hand, the x310 only loses whats on the failed drive/drives)
I can compete with these figures on a used system based of WHS2011, but it doesn't seam realistic to try and do so with FreeNAS. Ok, so, i cheat slightly, i already have a 12buck license for WHS2011. But even if i add that to the price of a Perc6/i, and compare it to a ServeRAID M1015, at best it's going to come out equal. I have to push the clocks back up, i might even have to run the X4, thats even worse for the wallet, not to mention the noise, and i'm still using old hardware, a none ECC configuration, with not enough memory, in a way thats not really recommended. Actually, the only system i ever had, that would have been "right" for something like this is a Compaq DL350 G5, and it's idle consumption power was well in to the 230+w.
The obvious issue here seams to be that the FreeNAS system design simply isn't aimed at me, unlike what proponents have been telling me. And the opening post here REALLY made that clear. It's a shame, but it's also saving everyone a lot of grief. I'm going to keep watching this forum, one of these days someone i know, or someone who wants to pay me, might need storage, and at that point they are paying the bills, and buying new stuff regardless, making this a much more valid option.
If there is something i'm missing, that might make my reasoning invalid, feel free to point it out, here, or in a PM. But my main point isn't really about FreeNAS, it's about the opening post. It's a great post, and should be a must read. It might have deterred me, since i feel that i might be better of using a low watt system, with a "smarter" drive sub system to do the math for me, but it's definitely a great post.
B!
jgreco
Resident Grinch
- Joined
- May 29, 2011
- Messages
- 18,680
This may be in part because FreeNAS 8 (and FreeNAS 9) are completely new beasts; the old FreeNAS .7 which became NAS4Free is likely what they were talking about. You're encouraged to look at it! The new FreeNAS is aimed more at business and enterprise deployments, and started its life from scratch making larger assumptions about available resources like memory. For a new machine, 8GB isn't that much or that expensive. In 2005, though, it was four figures worth of memory.
ZFS is designed to consume large amounts of resources and to do useful things as a result. Doing that with a better-than-average chance of success is what this thread's all about.
As for low watt systems, the trend is towards noticing that there aren't any useful such things. The "faster" CPU's are managing energy these days more aggressively than ever, and in idle, don't really consume that much more energy than a "low watt" solution (unless you move down into SoC-land). When busy, you probably don't mind them actually consuming watts for doing useful work faster.
ZFS is designed to consume large amounts of resources and to do useful things as a result. Doing that with a better-than-average chance of success is what this thread's all about.
As for low watt systems, the trend is towards noticing that there aren't any useful such things. The "faster" CPU's are managing energy these days more aggressively than ever, and in idle, don't really consume that much more energy than a "low watt" solution (unless you move down into SoC-land). When busy, you probably don't mind them actually consuming watts for doing useful work faster.
"older" incarnations of the OS might very well be what makes the difference.
I'm not entirely convinced that "low power" systems don't have their place, as i managed to get the full load watts to less then idle of the stock systems for both the C2D and the X3. I expect both these systems to, for all intents and purposes except on the fly re-encoding of streamed video, do very well with a Perc 6/i & WHS2011. The Perc 6/i supposedly draws 5w, i haven't got one so i cant test it, but getting the checksum calculations for 5w seams near unbeatable. But, were getting of topic (again) and it's my fault. (again)
I'll go look at Nas4Free, i've yet to buy anything, so all my options are so far, open. Thanks for the tip. And, again. The opening post is really good. Thank you for taking the time, and making the effort.
B!
I'm not entirely convinced that "low power" systems don't have their place, as i managed to get the full load watts to less then idle of the stock systems for both the C2D and the X3. I expect both these systems to, for all intents and purposes except on the fly re-encoding of streamed video, do very well with a Perc 6/i & WHS2011. The Perc 6/i supposedly draws 5w, i haven't got one so i cant test it, but getting the checksum calculations for 5w seams near unbeatable. But, were getting of topic (again) and it's my fault. (again)
I'll go look at Nas4Free, i've yet to buy anything, so all my options are so far, open. Thanks for the tip. And, again. The opening post is really good. Thank you for taking the time, and making the effort.
B!
jyavenard
Patron
- Joined
- Oct 16, 2013
- Messages
- 361
Any Broadcom based router (linksys, asus) and their MIPS core or even latest gear with arm core would use a 10th of the power (typically 3W idle, 7-8W at load) and would handle everything you throw at it network wise... 60W idle for a router is crazy.
A asus RT-N56 will have 5 gigabit ports, and in excess of 980Mbit/s routing capacity. I'm fairly certain that this is significantly more than what your C2D with Realtek adapter can achieve (having said that, would need to route at gigabit speed at home?)
Probably will cost you much more in the long run with your old well over-powered PC... Takes a fraction of the space, makes no noise.. The list goes on.
In regards to requirements, personally, I don't see any reasons why two different distributions of essentially the same thing (by that I mean same OS kernel and drivers) would have much different hardware requirement.... The recommendations provided here are just that: recommendations and best practice. Anything less and it is going to have a negative impact on performance. Doesn't mean that you can't use much less..
I had a chuckle yesterday when I created a VMware image of FreeBSD 9.2. VMware default settings for a VMware appliance is 256MB (as a comparison windows 7 is 2GB recommended).
Having said all that, I'm an embedded hardware/software engineer. So I'm used to scraping for resources...
jgreco
Resident Grinch
- Joined
- May 29, 2011
- Messages
- 18,680
I will note that "route" used in this context is usually deceptive, what you mean is "NAT". It is disappointing that CPE manufacturers co-opted a term that means something fairly specific and used it to mean something rather different.
Those of us with multiple network segments may actually be wanting to route. I haven't actually come across a competent low-wattage solution to handle multiple gigE interfaces, small packet traffic, and a dynamic routing protocol without also falling over in some way. I mean, yes you can theoretically set up multiple networks on an OpenWRT system (for example) but the lack of CPU means you aren't moving packets real fast.
Those of us with multiple network segments may actually be wanting to route. I haven't actually come across a competent low-wattage solution to handle multiple gigE interfaces, small packet traffic, and a dynamic routing protocol without also falling over in some way. I mean, yes you can theoretically set up multiple networks on an OpenWRT system (for example) but the lack of CPU means you aren't moving packets real fast.
jyavenard
Patron
- Joined
- Oct 16, 2013
- Messages
- 361
No, by routing I mean routing... I would have used NAT if that's what I was referring to :)
The asus rt-N56 I mentioned above uses an atheros chipset, it has an interesting feature in that almost everything is hardware accelerated and really it does move data as fast as I mentioned.
This isn't supported by openwrt at this stage unfortunately. You get it with tomato firmware but the drivers come as binary blobs.
The other new asus routers with Broadcom also have hardware accelerated routing (and NAT), they aren't as fast but still perform incredibly well without ever needing to see the CPU jump.
Problem is that you can't have QoS on with those. It's something they are currently working on I heard, and asus last firmware already made great progress there.
- Status
