BUILD Intel S1200V3RPL vs Supermicro X10SLH-F

[kroko]

Hello,

This forum has been a really good resource, learned a lot from it. Thank you!

I have question about choice of logicboard and setup in general based on planned usage.

//////////////////////////////////////////////////////
1)
Intel S1200V3RPL http://ark.intel.com/products/71384/Intel-Server-Board-S1200V3RPL
vs
Supermicro X10SLH-F

Where I’m at delivery time for Asrock C226 WS or Asrock E3C226D2I is ~month. As we are running out of space and some of our data even isn't backuped any more this is ASAP. :)
In case of second one I’d have to change RAM, based on a sticky note in this forum.

I can find many Supermicro users here (sometimes feels like Supermicro uuber alles :)), but have not found any discussion on S1200V3RPL.
Many praise Supermicro's IPMI. But Intel has it's counterpart Intel RMM http://www.intel.com/content/www/us/en/server-management/intel-remote-management-module.html which can be attached to this board if needed. Same stuff - low level monitoring and controlling through ethernet.
And that kind of seems the only "difference" between those boards. It feels Intel is a robust alternative.
And as I’m ready-out-of-box computer user I wasn't able in last days (reading 24/7 about ZFS, simulating failed drives in virtualbox etc) to cover that much of information.

Any advice between these two logicboards? Any Intel user here?

//////////////////////////////////////////////////////
2) Setup

Storage:
6x WD60EFRX WD Red 6TB SATA 6Gb/s 64MB

LOBO:
Intel S1200V3RPL + Intel AXXRMM4

CPU:
Intel Pentium Dual Core G3420 (ECC supported)
Core i3 Intel Core i3-4330 3.5Ghz/4MB (ECC supported)
(switching from G3420/G3460 to i3-4330. same Hz, but bigger CPU cache and a touch of multithreading)

RAM:
4x8GB KINGSTON KVR16LE11/8 SERVER MEMORY 8GB PC12800 DDR3/ECC

PSU:
CTG-600-80P CHIEFTEC A-80 PSU 600W

USB:
2x KINGSTON DTDUO3/16GB 16GB (DT microDuo USB3.0)

CASE + COOLER:
Cooler Master computer case Silencio 452 black
ZM-F2 FDB(SF) ZALMAN 92MM FDB CASE FAN

UPS
Eaton 5S1000i via USB NUT



Usage 1:

RAIDZ-2. After ~year will be expanded to 6 HDDs
A second backup machine matching this one so no “mirroring” is done in RAID, but because of backuping.
No encryption
No jails
APF, CIFS, SFTP, FTP shares + one NFS share.
Every day APF and CIFS shares used (5 OSX, 3 MSW machines), NFS to connect to webserver.
1 NIC used for everyday sharing
1 NIC used for mounting NAS on webserver via NFS exclusively
1 NIC (AXXRMM4) for Intel RMM.

Usage 2:
It will be used every day as a studio “data archive” machine. Designing, coding and animations. Thus lots of projects that take gigabytes per project. Once project is ended it goes into archive. Things are reusable, so we use this archive. Containing photoshop, illustrator, aftereffects, storage for raw videos etc. Storing assets for rendering (like texture files libraries). And of course also documents.

Usage 3:
We have webpage development server in our studio, NAS will sit right next to it. Apart from other things it also runs a vhost for owncloud. Owncloud is for project/files we actually currently work on, so much more busy read/write, sharing with clients, uploads from clients.
NAS? well.. the owncloud data dir actually will be physically on NAS, mounted via NFS in local gigabit ethernet (I'm using HP 1410 switches everywhere). I have setup this approach already with a test installation of FreeNAS inside virtualbox, tested access restrictions for server to connect only. For permissions creating matching IDs and GIDs is easy (even so as we run apache vhost using mpm_itk_module). Although I have not run any read/write tests yet, all smooth, all works.
All the “computation” apart from file r/w is done on server machine, not NAS. Server, not NAS, also stores and manages owncloud DB. The link (NFS speed) of course is not the “fastest way r/w wise”, but … we get tons of space for owncloud files and we separate logic from file storage.

Usage 4:
It is backup location (rsync) for development servers code / database.

Given this setup and usage is anything you could comment? :) Ditching Pentium and from a start going i3 CPU? Archive will not be busy, owncloud data stored on NAS however will, but also only for files I/O.

Many thanks in advance,
kroko

 

[kroko]

And what is your take on SATA DOM (some 3rd party for Intel board, for example SATA3 DOM http://www.adata.com/index.php?action=product_feature&cid=10&piid=306&lan=en / for supermicro either their "SuperDOM" or 3rd party)? It eats up one mobo SATA controller port. As I'm planning to go for 6 HDDs in future, and if one SATA port is taken, need for PCIe RAID card in IT mode arises. Any real-life experience/reason why use/give that one SATA port away for OS instead of using USB3 flash stick? Is it "so much better"?

Thank you!

 

[marbus90]

You can't upgrade from 4 to 6 disks whilst keeping the same raidz2. You could consider buying 6x3TB Reds from the start and upgrading to 6TB HDDs later.

Supermicro X10SLL-F is enough for your usage, another option may be the ASRock MT-C224 or just going down the C2750 route with the C2750D4I in a Fractal Node 304 Chassis with Seasonic G-360 PSU. For the Supermicro board I'd pick the Fractal Arc Mini or Define chassis. Rackmount options include Supermicro 822, 823, 825 and 826 series.

 

[kroko]

sure. thats why i have my superduper backup. i was planning to attach 2 new disks and wipe all, rebuild the data from backup (process will require some days in the future). however your suggestion about growing storage through zfs autoexpand is interesting.

it's really hard to kind of plan those all "what ifs". i mean not to plan, but (as you probably see i'm ready to give away considerable budget for this build) where to draw the line knowing that this system will be capable in future for this and that and that's it. will 32GB RAM will be enough in the future. if not, how will i upgrade. how many disks. when will this 1150 socket based system hit 32GB RAM. maybe i should in the very beginning go with a 4 channel RAM mobo. when studio has so much storage - well, you just "grow into it" and "throw data at it". as one guy said - things (here - big archives of data) do not come randomly, they come based on possibilities. if the studio expands it means more owncloud users. questions to myself you know.

considering that i went for six smaller disks, let's say 3TB, and afterwards when needed upgraded all of them. i will have 6x6TB = 36TB. imho, 36TB is still manageable with 32GB of ram with intended usage (although i have no clue how long would regular maintenance tasks (scrubs) take for such system if run say every second weekend and it is not strictly by the formula of 8GB + 1GB per 1TB). please correct me if i'm wrong. but if i were to also utilise those 6x3TB=128TB through some PCIe controller afterwards, then i've hit RAM abilities of the mobo/CPU.
probably what i'm asking here as a new question - isn't 36TB sweet max storage that one could run with 32GB RAM system? basically, if i go with 2 memory channel setup in lobo/CPU then i just have to recognise - this system will be 36TB max and if i need more i will need extra box. if want to go more from the very beginning, then i should look at 2011 socket.

asrock is a no go. i dunno why but here in latvia the shipment time for asus server lobos is insane. official answer - min 1 month. desktop lobos - no probs.

as for X10SLL-F. i was thinking about C226 chipset, all my examples above have that. i know that C226 will cost more than C222/4 and that really i will not saturate SATA3 speeds, but it kind of "feels right" :) another thing is that if in the future owncloud user count increases (total realistic as this big storage will grow our studio bigger :D), thus NFS usage increases to a constant bandwidth max, i might have to use ZIL on separate SSD. and here there will be benefits from C226/SATA3.
or am i wrong, no need for c226 thus giving e a wider range of mobos to choose from?

Fractal is nice, but i don't care for looks. I've looked at all of them. Lian-Li have also nice ones. Just one demand - good cooling Consider that i will mess up the case anyways, as there are very few boxes with kensington lock, i will do my own "locking system".

what about SATA DOM over USB stick? if i went for 6 disk setup from the beginning and SATA DOM is way to go, then i would have to buy a PICe SATA extension card or use X10SL7-F/alternative that has onboard LSI 2308 (but is C222)

thank you!

 

[marbus90]

My rule of thumb for a still speedy ZFS system with 6disk raidz2 vdevs is: raw TB /2 = usable TiB. Out of 6 disks you'll have 4 usable = 24TB, equaling ~22TiB, applying the 80% rule it's ~18TiB usable. At up to 90% it starts to slow down a bit, above 90% you should already have replacement/addon drives on your way. For home use I'd say you could use up to 2x 6disk z2 on 6TB drives since that's at 36TiB usable an acceptable sweet spot. For Enterprise I'd rather plan on 2+GB per TB usable storage, meaning 64-128GB would serve you better for 12 HDDs.

If you want to plan towards expansion anyway, consider starting on a Xeon E5 system. Singleprocessor boards support 128GB RAM, Supermicro offers nice 12 and 36bay barebones with the option to add in 2x2.5" SATA SSDs in rear hotswap bays. For FreeNAS boot you could use the Supermicro SATADOM modules which plug onto the board directly. Not really feasible with Xeon E3 systems however since their standard issue boards would only have 6x SATA which you would fully utilize for the HDDs - Xeon E5 systems sport 10x SATA, of which you could use 2 for the SATADOMs and 2 for SLOG/L2ARC SSDs. I've found that the Xeon E5 systems aren't that much more initially if you look at a Xeon E3 base system. Here are some barebones to which you'll only have to add the Xeon E5-1620 v3 or E5-1650 v3 CPU and 4 or 8x 16GB DIMMs from the Tested Memory list, coolers, cables, trays and controllers would be included:
http://www.supermicro.nl/products/system/2U/5028/SSG-5028R-E1CR12L.cfm 12bay
http://www.supermicro.nl/products/system/4U/5048/SSG-5048R-E1CR36L.cfm 36bay
http://www.supermicro.nl/products/system/4U/6048/SSG-6048R-E1CR36L.cfm 36bay Dual-CPU, here I'd select the E5-2620 v3 CPU - a single would suffice to start with, if you find that you could use more than 128GB you'll need to add the 2nd CPU.

http://www.supermicro.nl/products/nfo/SATADOM.cfm SATADOMs for the listed Supermicro servers

 

[kroko]

thanks!
currently i'm thinking of taking a break in food consumption, and going 6x6TB from the beginning.
it will take much time to fill those 90% of ~18TB usable as per your experience. it will be storage strictly for work (no constant inflow of sitcoms or bluray rips) and although sometimes one project files can take up 100GB of space it's not everyday, ~10 per year. and not snapshotting datasets "every minute".
thing is though that i will have to add some PCIe controller if it turns out i need SSD ZIL for this system. i don't think i will need L2ARC but then again i'm just a beginner in managing ZFS based system.
imho, if in the future i step over this HDD space i would just buy another machine that can either handle bigger capacity (say 12HDDs), move these HDDs there, or just buy another this type/class box and distribute our storage needs among boxes.
about that supermicros you list - realy nice. but this will be above the budget. remember - i have to also buy "backup storage" for everything. the most precious things will be backuped to offsite 3rd party cloud, but "normal backup" will be 2nd box in studio. i think i'm taking already it to the level above our needs (which is a question only i can answer :)), this will be just too much for starters like us.
@marbus90 also kind of shows that the vast majority here is for for supermicros or asrock. intel users are none nada null :) . damn, i really like that intel lobo. when adding AXXRMM4 it costs the same as X10SLH-F. i found some instances of people using S1200V3RPL, so i'll take my chances in PM'ing them :)
of course if anyone has to comment on
- USB vs SATA DOM and
- 1150 CPU choice i'd be grateful :) thank you!

 

[marbus90]

If you talk backup as in backup only storage, not production-replacement spare server, you could be doing fine with a Xeon E3 based X10SL7-F system in a Supermicro 826A chassis. For primary storage I'd go for the Xeon E5's almost instantly. Xeon E3 choice for backup would be either to cheap out on a 1220 v3 or going all-in on a 1241 v3. Backups would also run acceptable on 2x8GB RAM.
Also with that 826A chassis you could look for the Xeon E5 UP mobo and drop in a LSI 9207-8i or -4i4e SAS card to run all 12 ports. At least that one would be optional for a start. ;)

L2ARC isn't really needed, better max out the RAM first.

 

[kroko]

yesterday ordered the spec above (corrected the original post). in spite of this forum i went for intel lobo hehe ^_^ pardon.
i read also about Supermicro IPMI in interweb, how the clients are coded (java, but then some low level os code is thrown in) - for osx user there are no good news. it runs, but with patching and stuff like that. Intel RMM is os agnostic.

yeah, i ordered that i3, not xeon. however i maxed out ram. i hope it goes well as there will be no jails whatsoever. if it turns out that CPU, not network, is bottleneck then swapping CPU for this primary NAS/cloud is not that hard. commmooon, 6x6TB array is enough for a start. ;) anything above is as you say - extra controller through PCIe and Xeon.

thank you, i will take into account your suggestions for backup machine, and yes it will be used only as dust gathering regular rsync destination for this primary machine, nothing elese.

as the webserver has no redundancy (apart for having OS and DB on SSD, all except for owncloud vhost webroots are on 7200rpm though) then it will be backuped regulary to primary NAS. cloud files will reside directly on primary NAS through NFS as described above, not webserver. the cloud will be the only busy place in this NAS system.

 

[Whattteva]

yesterday ordered the spec above (corrected the original post). in spite of this forum i went for intel lobo hehe ^_^ pardon.
i read also about Supermicro IPMI in interweb, how the clients are coded (java, but then some low level os code is thrown in) - for osx user there are no good news. it runs, but with patching and stuff like that. Intel RMM is os agnostic.

I've never actually used their tool on OS X personally, but Java is a platform agnostic language by design and will run on basically anything that has the JRE installed... (their installation boasts that it runs on over 2 billion devices or something like that).

 

[Bidule0hm]

their installation boasts that it runs on over 2 billion devices or something like that

Well, I would say something more like this personally: "their installation bloats over 2 billion devices or something like that"... :p

 

[Whattteva]

Well, I would say something more like this personally: "their installation bloats over 2 billion devices or something like that"... :p

Well, everyone is certainly entitled to their opinions. I'm just a mere messenger who quoted their claim.

 

[Bidule0hm]

Yeah, I know ;) Can't resist to beat on java...

 

[Ericloewe]

I've never actually used their tool on OS X personally, but Java is a platform agnostic language by design and will run on basically anything that has the JRE installed... (their installation boasts that it runs on over 2 billion devices or something like that).

My, aren't you an optimist!:p

 

[Whattteva]

My, aren't you an optimist!:p

Well, I programmed in Java for a good number of years, so the bit of confidence at least has some background. It's a bit outdated though cause I haven't used it in about 4 years, so I'm sure a lot of things have changed.
That being said, I don't see the bit about the cross-platform thing changing as that was one of their core design principles.

 

[Ericloewe]

Well, I programmed in Java for a good number of years, so the bit of confidence at least has some background. It's a bit outdated though cause I haven't used it in about 4 years, so I'm sure a lot of things have changed.
That being said, I don't see the bit about the cross-platform thing changing as that was one of their core design principles.

Well, Supermicro's IPMI application is notorious for not working properly on anything other than Windows... If you can get it to run on Windows.

 

[Whattteva]

I actually haven't run it on Windows, but I've seen other people's screenshots on Windows.
I've personally run it under Linux Mint 17, so I can at least vouch for that one.
Let me run it on OS X and I'll get back to you on that... stay tuned!

 

[Ericloewe]

I actually haven't run it on Windows, but I've seen other people's screenshots on Windows.
I've personally run it under Linux Mint 17, so I can at least vouch for that one.
Let me run it on OS X and I'll get back to you on that... stay tuned!

Note that it's not the viewer downloaded from the server that's problematic. I mean their big standalone application.

 

[Whattteva]

Note that it's not the viewer downloaded from the server that's problematic. I mean their big standalone application.

Yes, that's what I'm talking about. It looks like rather than one big application, it's actually a suite of several tools with IPMIView being the main executable that launches other tools (iKVM for the KVM console) when requested by the main app.

 

[kroko]

I was just discussing my choice of board with a user in PM. He also leans towards Intel so my second take on IPMI topic.
My conclusions on Supermicro's IPMI were based on simple google. Whatever google keyword combination including OSX in it, pain and suffering. I actually also kind of do Java development (lots of processing.org sketches) and nowadays we do have issues with java on OSX in general. Not to mention web Applets in newest OSX versions. As I wrote - it [IPMI ] runs, but with patching and stuff like that. Cool, I set up one computer. But what if I need a fast check from my iPhone, other computer. Intel RMM is in-browser! And I ordered the model with dedicated NIC for that.

 

[Ericloewe]

I was just discussing my choice of board with a user in PM. He also leans towards Intel so my second take on IPMI topic.
My conclusions on Supermicro's IPMI were based on simple google. Whatever google keyword combination including OSX in it, pain and suffering. I actually also kind of do Java development (lots of processing.org sketches) and nowadays we do have issues with java on OSX in general. Not to mention web Applets in newest OSX versions. As I wrote - it [IPMI ] runs, but with patching and stuff like that. Cool, I set up one computer. But what if I need a fast check from my iPhone, other computer. Intel RMM is in-browser! And I ordered the model with dedicated NIC for that.

uhmm, pretty much anything you'd want to do from a phone can be done from Supermicro's IPMI web interface...