Ode to the Dell C2100/FS12-TY

[Spearfoot]

Ahh gotcha, good point.

I heard back from the seller, who claims that 4TB drives do work:

New message from: mrrackables (5,942 Green Star)
yes we have tested 4TB in this and they do work

please let me know if you have any other questions or would like to place an order, thanks

regards
Justin
Reply

Your previous message
You're offering this server with 12 x 4TB disks - are 4TB disks known to actually work with the C2100 FS12-TY?

Thanks!

 

[maglin]

Mine just came in today. Just crammed 144GB of memory in it. Took me a bit to figure out that I had to actually hook up and monitor and keyboard as the IPMI isn't set to DHCP so I couldn't log into it. Once that was done I turned on all the nice power saving features that where turned off for the CPU as well and VT-d :) . Now it's happily crunching numbers as MEMTEST86 runs away.

One question is MEMTEST said it detected the EEC ram but didn't turn on EEC. I turned on EEC but is that the correct way to run MEMTEST on EEC memory? I did some googling and what few things I saw it said it should be on. Here is a screen shot of memtest. I'm probably going to let this run for a few days until I have all the parts to get the array up and running. I haven't updated anything yet.

Another question is regarding Users. I changed the password of root but there is a no user account as well as 2 admin accounts. Is it ok for me to just go ahead and disable those?

And last I have a question about CPU temps. They seem a bit high and all that is running currently is MEMTEST.

 

[Mirfster]

One question is MEMTEST said it detected the EEC ram but didn't turn on EEC.

You mean ECC correct? Issue is that for MemTest to actually be able to test ECC, it requires UEFI. So, from that stand point it is not able to do ECC Error Injections. You should be fine though, it will still stress test the memory.

Another question is regarding Users. I changed the password of root but there is a no user account as well as 2 admin accounts. Is it ok for me to just go ahead and disable those?

Yep, I usually just have User #2 (root) and User #3 (what I configure for myself) set.

And last I have a question about CPU temps. They seem a bit high and all that is running currently is MEMTEST

That does look high for the CPUs you are running "L5630"s right? Per ARK the TCase for them is 63.1°C so you are already above that. I am wondering a couple things (some may be "elementary", but just covering bases):

1. If you got the heatsinks with the copper faceplate and piping
2. If there is thermal paste properly applied
   
3. If the shroud is in place (since this directs the cooling)

I haven't updated anything yet.

Does that mean the BIOS and BMC as well?

 

[Mirfster]

Just recalled that you may have mentioned "L5640" (Hex Core) CPUs in another conversation... If those are the processors then per ARK they have a TCase of 69.4°C. However, they use 60W as opposed to the other which uses 40W. For that I recommend the heatsinks with the copper plat and piping.

But, even if you had the standard heatsinks I would think that is would prevent them from running at or above the TCase.... Something is amiss...

 

[maglin]

I have the L5630's in there. I didn't pull the heat sinks off but they are aluminum fins with sticker on top showing you which direction the airflow is supposed to go.
Sucks about the ECC MEMTEST. EEC ECC it's all the same in my head. LoL. I'm really liking Dell's implementation of IPMI over SuperMicro's. I've never seen the DRAC management interface so I can't compare to say the R server series of network management.

I'll not worry about the temps. I'll just wait until I get the L5640's.

I just wanted to stat testing the RAM. ie. wife came home and wanted to go to dinner. I kept giving her the "I'm almost done" and I just ran out of wife patience. I'll just do the updating tomorrow and then restart doing MEMTESTs until the rest of the parts arrive.

One note is mine didn't come with the inner rails. I have outer rails coming. I'm thinking it might just be cheaper to get another C2100 in a few months with rails and sale that to a family member for their own FreeNAS.

edited because I suck when it comes to grammar.

 

[Raident]

So, you fancy running ESXi on this beauty eh? Yeah 5.5 works fine so what about 6.0? 'Cuz you know "latest and greatest" stuff rulz... Being the savvy person you are you download the ISO and start installing, only your get a freaking PSOD! Quick search leads you to an article titled "ESXi 6.0 drops support for dell C1100/C2100 NOOOOOOOOOO!!"... Now panic sets in... No worries mate! Answer is actually located in that thread, but I will make it easy for you here..

1. To run ESXi 6.0, you actually have to downgrade the BIOS since newer versions actually mucked with VT-D
2. Download the "PEC2100BIOS3B16.exe"
3. Extract it to whatever folder you want to make (which will be copied to your Bootable USB Stick)
4. In that folder make yourself a batch file (like "FX.bat")
5. Edit the file and add this line: Afudos C99Q3B16.ROM /P /B /K /X
   • This is the exact same as what is in the vendor's "F.bat", but with the "/X" switch
6. Copy the folder to your USB
7. Boot up with the USB, change to that directory and launch the "FX.bat"
8. After it is done (never during), reboot and you can install ESXi 6.0 without any issues

*** Just so you know once it is flashed and after you reboot; it may reboot once/twice more (I think mine did since it was applying the "ME" update as well).
My proof:

I'm a bit surprised to learn that so many people had trouble with ESXi 6.0 - I was able to download and (clean) install 6.0 to both a FS12-TY and a CS24-TY with no problems whatsoever using the stock VMware ISO. Clean installs of both 6.0U1 and 6.0U2 later on were smooth sailing as well.

On the other hand, I would never have said "5.5 works fine" - DirectPath I/O (aka VT-d passthrough) was broken to the extent that even HCL devices were hit and miss. And don't even get me started on stupidity of the artificial desktop client limitations, or the terrible performance of the web client that VMware was trying to push everyone towards...

 

[Mirfster]

I'm a bit surprised to learn that so many people had trouble with ESXi 6.0 - I was able to download and (clean) install 6.0 to both a FS12-TY and a CS24-TY with no problems whatsoever using the stock VMware ISO. Clean installs of both 6.0U1 and 6.0U2 later on were smooth sailing as well.

Interesting, by chance what BIOS and BMC firmware do you have? Also, did you have VT-D enabled in the BIOS when you installed?

 

[Mirfster]

Just an FYI... DFW Computers has graciously agreed to a "FreeNAS Turn-Key" configuration. With this unit, not only does one get the system but it is for all purposes ready to go right out of the box! Just slap in your hard drives, configure your BIOS settings, install FreeNAS and get started.

*** Please hold off buying until I have had a chance to purchase and review ***

Some highlights that are included with the system are:

• Memory: 32gb (4x 8gb) DDR3 ECC Registered Memory
  • Those are 8GB sticks, not the cheesy 2GB or 4GB (so you have room to get up to 144 GB)
• Trays: Qty. 12 w/ Screws
  • Yep, no need to worry about actually having to order these separately
  • Keeping in tune with the "FreeNAS Turn-Key" configuration
• RAID / SCSI Controller(s): 9211-8i 6 GB/s Raid Controller
  • Not only is it one of the preferred HBAs, they went so far as to flash it with "LSI 9211-8i Ver. P20"
  • Keeping in tune with the "FreeNAS Turn-Key" configuration
• BIOS Ver. B16
  • So, you can run ESXi 6.0 U2 if desired
• BMC Flashed to Ver. 1.66
  • No need to worry about having to do the flashing yourself to "Quiet that sucker down"
• CPU Heastinks with the copper plate and tubing
  • Future proofing so you can easily go up to the higher processors without worrying
  • *** This is not specifically listed, but was mentioned in my communications with them and will be verified
• Shipping included in price
  • Not sure about outside of the lower 48 states though...
  • Update: I incorrectly selected "Local Pickup"; hence the $0.00 for Shipping. Just discovered that it is $47.54 for Ground Shipping...
• 10% Discount if you use the coupon code of "freenas"
  • Yea!! Saving money...

So, I have went ahead and purchased one of the units and will give it a thorough review. Once I have completed said review, I will post back.

With that being said, if anyone else is wanting to be adventurous, here is the link:
http://www.dfwpartsdepot.com/?produ...n-e5630-2-53ghz-qc-32gb-9211-8i-2x-750w-trays

* Don't forget to use the coupon code of "freenas", if you are not going to wait
** Coupon is only valid at their site, not eBay
*** I do not have any business associations with them and am in no way getting any financial benefits in return ***

Proof of the system I just ordered:

 

[maglin]

So $351 shipped. That is crazy good. Wonder how much more if they throw the rails on it, because that is really all that is missing for installing into a rack.

 

[maglin]

So I got BMC 166 along with BIOS B16 installed. Back to MEMTEST. It is passing with flying colors. Ran for 25 hours without a single error. I'll try BIOS B23 before I install ESXi 6 on it and see what happens. Maybe it was an issue with ESXi 6 when it was released and is now fixed.

 

[Mirfster]

Maybe it was an issue with ESXi 6 when it was released and is now fixed.

Perhaps, I was using ESXi 6.0 U2 but haven't checked for any newer versions. Please post back and tell us how it went. Ever since @Raident stated they did not have any issues, my interest is re-peaked.

 

[maglin]

Interseting enough. BIOS B16 doesn't support DDR3L RAM. I believe the system had B21 on it when I got it and what I was running memtest on until I flashed everything tonight. Every Reboot I was greeted with a "DDR3L at 3DPC not supported". I went to run memtest thinking no big deal and my memory was running at DDR800 instead of DDR1066. So I just flashed to BIOS B23 and not only can the memory run at lower voltages (doesn't help the memory has no heat spreaders on them) but it's back to DDR1066. So I'm hoping for the Best out of ESXi 6 and hope the VT-d stuff will be working properly.

Edit: I just noticed that MEMTEST now can see the cache of the CPU.

 

[Mirfster]

Are you running mixed memory? I think I saw something similar, but got around it by ensuring all the RAM was using the same make/model. Interesting though that B23 would permit it and B16 would not.

 

[Mirfster]

Wonder how much more if they throw the rails on it, because that is really all that is missing for installing into a rack.

I would guess ~$30.00. While some vendors list something like "Rails Included - A $50.00 Value...", I wouldn't pay $50.00 for a complete set of rails personally. The rails are pretty plain and there is nothing available that is like the Dell Rapid Rails or even able to take a Dell Cable Management attachment.

*** Modding a case to properly take Dell Rapid Rails (I *think* the R720 rails were the right length) and a Dell Cable Management is one of the things I want to tackle one day...

 

[maglin]

Are you running mixed memory?

Nope. All Hypinx (sp?) 8GB DDR3 DIMMS. It makes since that they are DDR3L DIMMs since the C2100 was designed for power consumption in mind. I purchased RAM from a C2100 to ensure it would work with no issues. I just think that B16 either doesn't have DDR3L low power memory support or didn't have support for these specific DIMMs. Since the memory doesn't have heat spreaders I really would like to use them with less voltage to reduce heat output. Honestly it wasn't even something that even thought about until I noticed the entries in the event log.

I also have memory throttling turned on just to ensure I don't start cooking these things.

 

[Mirfster]

I have some Samsung "Energy Efficient" RAM, will try to remember and check it out on a system to see if I get similar results.

 

[Ericloewe]

Hypinx (sp?)

Hynix

 

[Raident]

Interesting, by chance what BIOS and BMC firmware do you have? Also, did you have VT-D enabled in the BIOS when you installed?

I'm using BMC 1.82 and I had VT-d enabled when I installed ESXi.

As for the BIOS, is there any way to check without having to reboot the server?

 

[Mirfster]

As for the BIOS, is there any way to check without having to reboot the server?

Yep, via IPMI it should show you the BIOS version; Click [System] - [Component] and it should show you.

 

[Raident]

Yep, via IPMI it should show you the BIOS version; Click [System] - [Component] and it should show you.

Thanks for the tip. I'm running the B23 BIOS.