BUILD High(er) end build review

[rogerswest]

...or at least what I'm thinking is a higher end build.

Host:
Chassis: CSE-825MTQ-R700LPB - SMC 2RU
http://www.supermicro.com/products/chassis/2U/825/SC825MTQ-R700LP.cfm

Main board: X9DRi-LN4F+ - SMC Dual LG2011 24 RDIMM slot http://www.supermicro.com/products/motherboard/Xeon/C600/X9DRi-LN4F_.cfm

Central Processing Unit: E5-2660v2 - 2x Intel 2.2-3.0 GHz 10 Core 20 Thread 25MB cache
http://ark.intel.com/products/75272/Intel-Xeon-Processor-E5-2660-v2-25M-Cache-2_20-GHz
Random Access Memory: (384Gb) KVR13R9D4/16 24x Kingston 16GB Dual Rank ECC DDR3-1333 RDIMM CL9
http://www.kingston.com/dataSheets/KVR13R9D4_16.pdf
Host Bus Adapter: 9300-8e - LSI 12Gb/s 8 port SAS
http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-9300-8e.aspx​

Power Supply: PWS-702A-1R - 2x SMC 700 Watt
http://www.supermicro.com/support/resources/pws/
Operating System Storage: SDSSDRC-032G - 2x SanDisk 32GB SSD using onboard SATAII controller from Main Board and in RAID Level 1 configuration
http://www.sandisk.com/products/ssd/sata/readycache/
SLOG: D2RSTK251S14-0200 - OCZ Deneva 2 R Series 200GB SLC SSD using onboard SATAIII controller from Main Board
http://ocz.com/enterprise/deneva-2-sata-ssd/rs-slc-specifications​

JBOD Arrays:
Chassis: 417E16-RJBOD1 - (264 2.5" bay) 3x SMC 4RU 88 2.5" bay JBOD Chassis
http://www.supermicro.com/products/chassis/4U/417/SC417E16-RJBOD1.cfm

Control Board: CSE-PTJBOD-CB2 - SMC Power Control Board
Backplane: BPN-SAS2-213EL1 - SMC SAS2 Backplane
http://www.supermicro.com/manuals/other/SAS2-213EL.pdf
Backplane: BPN-SAS2-216EL1 - x3 SMC SAS2 Backplane
http://www.supermicro.com/manuals/other/BPN-SAS2-216EL.pdf
Hard Drives: ST600MM0026 - x264 Seagate Savvio SAS 600GB 10,000 RPM 64MB Cache
http://www.seagate.com/internal-har...terprise-performance-10K-hdd/?sku=ST600MM0026​

For those not keeping count, this system will avail 158.4TB of ZFS space via NFS. More or less.

Some obvious minor issues with this build, that I haven't been able to resolve as of yet:

1. E5-266ov2 supports 4 channel RDIMMs, however the SMC board only has 24 dimm slots in 3 channel config, and at reduced bandwidth (1066 instead of the full support for 1866) so the the ram that I'm putting in will be down rated to 1066 from 1333. It's a bit of shame to loose this bit of bandwidth and more so the extra channel but I haven't found a suitable main board alternative yet. At least not one that I've liked much.
2. Denva 2 SSD for Slog is only 200GB, its the best I could find out there with power loss safety and SLC. The array will be home primarily to 10-20mb each PDF and TIFF files, as well as hosting ~80 vmdk's for KVM, these vmdk's will be litte more than minimal Centos7 install. Postgres has its own machines so this array will not be used for RDBMS data.

Constructive critiques welcomed, as well as ideas to solve the above two issues.

Kind Regards,

 

[Ericloewe]

Why not get an Intel SSD? OCZ is the last I'd recommend, much less for an important machine. Intel's S3700 are a popular, dependable choice.

Also, make absolutely sure that RAM is on Supermicro's compatibility list. If it isn't, get something else. Kingston is as of recently known for their shady practices and incompatible memory.

As for the motherboard, you got it wrong: Each CPU has four channels with three DIMMs each. Using three DIMMs per channel does reduce speed to 1066, but more RAM is always better than faster RAM (within reason). You won't lose the fourth channel, though.

 

[rogerswest]

Why not get an Intel SSD? OCZ is the last I'd recommend, much less for an important machine. Intel's S3700 are a popular, dependable choice.

Also, make absolutely sure that RAM is on Supermicro's compatibility list. If it isn't, get something else. Kingston is as of recently known for their shady practices and incompatible memory.

As for the motherboard, you got it wrong: Each CPU has four channels with three DIMMs each. Using three DIMMs per channel does reduce speed to 1066, but more RAM is always better than faster RAM (within reason). You won't lose the fourth channel, though.

Why is OCZ "...the last thing I'd recommend" keeping in mind the Deneva 2 is a very high end device not a consumer level product. Is this just your opinion or is there some empirical evidence suggesting its not a good drive for a SLOG?

Further, in reviewing the specifications of the Intel drive you recommend, it doesn't come anywhere near those of the OCZ. The Intel drive is MLC with endurance of 5 writes per day for 5 years. The OCZ is SLC with "unlimited" endurance. Granted Intel is showing 5k IOPS improvement on random 4Kb reads, the intel also shows 48k IOPS (yes 48,000 not 8,000) reduction for random 4Kb writes. Intel is also listed at 50MB/s less on seq read, and 165MB/s less on seq write (WOW!) Other than the price and available drive size differential, I'm not sure how the Intel drive you reccomend could be better than the OCZ.

 

[Ericloewe]

Why is OCZ "...the last thing I'd recommend" keeping in mind the Deneva 2 is a very high end device not a consumer level product. Is this just your opinion or is there some empirical evidence suggesting its not a good drive for a SLOG?

Further, in reviewing the specifications of the Intel drive you recommend, it doesn't come anywhere near those of the OCZ. The Intel drive is MLC with endurance of 5 writes per day for 5 years. The OCZ is SLC with "unlimited" endurance. Granted Intel is showing 5k IOPS improvement on random 4Kb reads, the intel also shows 48k IOPS (yes 48,000 not 8,000) reduction for random 4Kb writes. Intel is also listed at 50MB/s less on seq read, and 165MB/s less on seq write (WOW!) Other than the price and available drive size differential, I'm not sure how the Intel drive you reccomend could be better than the OCZ.

OCZ has a reputation for terrible, horrible quality. They're also going through a bankruptcy/liquidation process, which means warranties are shaky at best. If you need more performance, Intel has higher-end, PCI-e stuff, too. Additionally, since it's a Sandforce drive, those numbers are only true for compressible data and are probably a sort of pie-in-the-sky scenario. Sandforce is also terrible in performance consistency.

 

[DrKK]

I believe Toshiba bought OCZ in whole, and that deal is done.

I am not sure if the "new" OCZ drives (which I see from time-to-time on Newegg) are good, or not. They seem to have decent reviews when I click...but some of those horrible products OCZ put out last year and the year before are enough to have soured me on the name, regardless of who's driving the bus.

Full disclosure: I have the cheapest of the cheapest OCZ "Basic" (their lowest model) SSD's in one of my boxes, now for at least a couple years, and it has run absolutely flawlessly, with spectactular performance, the whole time. But I think I'm one of the lucky ones.

 

[Ericloewe]

I believe Toshiba bought OCZ in whole, and that deal is done.

I am not sure if the "new" OCZ drives (which I see from time-to-time on Newegg) are good, or not. They seem to have decent reviews when I click...but some of those horrible products OCZ put out last year and the year before are enough to have soured me on the name, regardless of who's driving the bus.

Full disclosure: I have the cheapest of the cheapest OCZ "Basic" (their lowest model) SSD's in one of my boxes, now for at least a couple years, and it has run absolutely flawlessly, with spectactular performance, the whole time. But I think I'm one of the lucky ones.

I believe that the drive in question is from 2011, so around the peak of OCZ's quality problems, from what I remember.

 

[rogerswest]

OCZ has a reputation for terrible, horrible quality. They're also going through a bankruptcy/liquidation process, which means warranties are shaky at best. If you need more performance, Intel has higher-end, PCI-e stuff, too. Additionally, since it's a Sandforce drive, those numbers are only true for compressible data and are probably a sort of pie-in-the-sky scenario. Sandforce is also terrible in performance consistency.

I'm hoping you might have some empirical evidence that you could share.

Toshiba has purchased OCZ, and is honouring all warranty commitments for devices already released.

As Sandforce is owned by LSI, by extension does your claim hold true for LSI products? I believe Seagate is also contemplating the purchase the division in LSI that owns Sandforce as well. Will Seagate also be susceptible to your rhetoric?

If I sound flippant, it probably means I am. I did ask for constructive critique. Other than the part of the memory channels, your position on the drive has really only been one of opinion at this point.

 

[DrKK]

As Sandforce is owned by LSI, by extension does your claim hold true for LSI products? I believe Seagate is also contemplating the purchase the division in LSI that owns Sandforce as well. Will Seagate also be susceptible to your rhetoric?

Incorrect, sir. Sandforce is not owned by LSI. At one point, it briefly was, but it has since changed hands at least twice that I know of (Avago, and then Seagate).

 

[DrKK]

Incorrect, sir. Sandforce is not owned by LSI. At one point, it briefly was, but it has since changed hands at least twice that I know of (Avago, and then Seagate).

The wikipedia appears to bear me out.

 

[rogerswest]

The wikipedia appears to bear me out.

FWIW The deal doesn't close until later this year...

Anyways, if we all could refrain from polluting this thread with irrelevant Corporate M&A information and the like, that probably has little effect my build shown above, it will likely yield a more interesting and beneficial thread for all. Again limiting responses to something constructive, otherwise I might as well have just posted on Stack Overflow.

 

[DrKK]

FWIW The deal doesn't close until later this year...

Anyways, if we all could refrain from polluting this thread with irrelevant Corporate M&A information and the like, that probably has little effect my build shown above, it will likely yield a more interesting and beneficial thread for all. Again limiting responses to something constructive, otherwise I might as well have just posted on Stack Overflow.

Feel free to go to StackOverflow. There is no iteration of this thread that will be "beneficial for all", so I'm not too worried about that.

FWIW: You're the one that brought up corporate custodianship, sir.

Sorry you're not finding us helpful.

 

[rogerswest]

Feel free to go to StackOverflow. There is no iteration of this thread that will be "beneficial for all", so I'm not too worried about that.

FWIW: You're the one that brought up corporate custodianship, sir.

Sorry you're not finding us helpful.

Um, actually you did:

I believe Toshiba bought OCZ in whole, and that deal is done.

Which you posted before my rebuttal to Ericloewe's claim that the warranty on the OCZ drives was "...shaky at best." due to a "a bankruptcy/liquidation process".

I'm sorry too that I'm not finding you helpful. Perhaps we'll just leave this thread for some experienced users to provide constructive and relevant advice instead.

 

[DrKK]

Um, actually you did:

Which you posted before my rebuttal to Ericloewe's claim that the warranty on the OCZ drives was "...shaky at best." due to a "a bankruptcy/liquidation process".

I'm sorry too that I'm not finding you helpful. Perhaps we'll just leave this thread for some experienced users to provide constructive and relevant advice instead.

Yes. Perhaps we'll do that :)