Help choosing storage configuration

[Miguel Nunes]

Hello,

I am in the middle of completing the configuration of a web applications' server, but I am a bit lost on the storage solution. The Motherboard is a AsRockRack EP2C602 D16/4L and, since it doesn't like GPU passthrough that much, I decided to assign it to a develop, preproduction server.

The case has 24 hot swap bays, the board has 2xSATA III + 4xSATA 2 + Marvel 4xSATA3. I am planning to use XenServer (because ESXi free doesn't work with 2 x E5-2650 v1). Bought a SAS expander, late to find out that it doesn't interoperate with Dell's PERC 310. HP P410 was a possibility, till I read there are BIOS complications.

The question is: What would be the best for a Web Apps machine?

Possibility 1: 3 x PERC 310 to control the back plane in 3 x 8 drives blocks.
Possibility 2: Find a controller that supports the expander.
Possibilities 3..oo: Some other choice you find better.

I will be looking forward for your kind help.

Best Regards.

MAN

 

[Chris Moore]

Hello,

I am in the middle of completing the configuration of a web applications' server, but I am a bit lost on the storage solution. The Motherboard is a AsRockRack EP2C602 D16/4L and, since it doesn't like GPU passthrough that much, I decided to assign it to a develop, preproduction server.

The case has 24 hot swap bays, the board has 2xSATA III + 4xSATA 2 + Marvel 4xSATA3. I am planning to use XenServer (because ESXi free doesn't work with 2 x E5-2650 v1). Bought a SAS expander, late to find out that it doesn't interoperate with Dell's PERC 310. HP P410 was a possibility, till I read there are BIOS complications.

The question is: What would be the best for a Web Apps machine?

Possibility 1: 3 x PERC 310 to control the back plane in 3 x 8 drives blocks.
Possibility 2: Find a controller that supports the expander.
Possibilities 3..oo: Some other choice you find better.

I will be looking forward for your kind help.

Best Regards.

MAN

What SAS expander is not working with the Dell H310?

The 3 controller setup will work, if the system board plays nice.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Miguel Nunes]

Hello Chris

Thank you for your reply.

What SAS expander is not working with the Dell H310?

I tested with a HP expander (green pcb). The system doesn't hang, it just says in the PERC H310 console messages that the device should be a end device. According to my research this probably means it doesn't have the code to interact with SAS switches or it is just disabled.

The 3 controller setup will work, if the system board plays nice.

I have tried with two PERCs and the motherboard handles them well. I tried both hooked on the Bootstrap CPU and also one on each CPU and there are no problems. I presume a 3rd controller wouldn't cause any major problems.

About the virtualization solution, my initial plan was to use FreeNAS as NAS and Hypervisor and set the virtual infrastructure on top. A got hinted that this solution would be a bit overkill considering the CPUs in question, so I decided to try ESXi. When I realized the CPU limitatiom, I tried XenServer (very amazon like) and it works. Another possible scenario would be using KVM (very digital ocean aware), that if I am not mistaken was ported to FreeBSD. I am just not sure how updated is KVM for FreeBSD and if I could eventually run OpenStack to manage the infrastructure.

Best Regards,
MAN

 

[Johnnie Black]

I have an green HP expander working with a Perc H310.

 

[Miguel Nunes]

I have an green HP expander working with a Perc H310.

Hi Johnnie

What is the version of the HP expander you are using? Mine is "Regulatory Model Number HSTNM-B107.

What firmware version are you using in the Perc H310?

MAN

 

[Johnnie Black]

Mine is "Regulatory Model Number HSTNM-B107

That doesn't look like the model, I'm not home at the moment but mine is the same as this one, and I'm using firmware 2.08:

https://www.amazon.com/HP-468405-002-EXPANDER-468405-001-487738-001/dp/B00Z9AMKOA

H310 is using firmware p20.00.07, it also worked with previous releases.

 

[Miguel Nunes]

Hi again,

That doesn't look like the model, I'm not home at the moment but mine is the same as this one, and I'm using firmware 2.08:

https://www.amazon.com/HP-468405-002-EXPANDER-468405-001-487738-001/dp/B00Z9AMKOA

Mine looks like this one. I can't find where is the HP product reference is, so I used the regulatory code.
I will check when I am near the card again.

How do I upgrade the firmware of the expander?

What ports do you connect to the controller? I tried with the lower ones. The external is 1C then I have 2C, 3C, 4C, 5C, 6C, 7C and I end up with 8C and 9C for the controller.

Do you use any special cable to connect from the controller to the expander?

H310 is using firmware p20.00.07, it also worked with previous releases.

I will check my firmware version... it doesn't ring me a bell about being p20.xx.xx

I really suspect it might be a expander problem.

I found a post from a Johnnie Black with speed test info, and it seems the performance per disk is really low. When using hdd it might have little impact, but with SSDs it might degrade the performance a lot.

I will be checking the references you gave me soon, and post again.

Thank you for your help.

MN

 

[Miguel Nunes]

The 3 controller setup will work, if the system board plays nice.

Now that you mentioned it, I only had tried a controller hooked on each cpu.

When doing on the same cpu, it tends to delay shutdown on XenServer quite considerably. When the XenServer graphics was replaced by console text I just turned the server off.

Johnnie has made some performance benchmarking and it is clear that with an expander (even dual linked) the performance per drive is going to be 95MB/s at most.

It does seem that the expander is only useful when running Low Bandwidth workloads.

I think in this case, using RAID 5 (even though ZFS is better) could be a way to increase performance and uptime a bit.

I have been trying to spot a all FreeBSD solution but I haven't had much success in finding the proper software.

 

[Johnnie Black]

Mine looks like this one. I can't find where is the HP product reference is, so I used the regulatory code.
I will check when I am near the card again.

I see the same saying on the front, but the part number is on the back, HP P/N 468405-001, or spare P/N 487738-001

How do I upgrade the firmware of the expander?

I didn't upgrade mine, I believe you need an HP controller to do it, but as long as it's 2.00 or higher you should be fine, older versions only link with SATA disks @ SATA1 speed (150MB/s)

What ports do you connect to the controller? I tried with the lower ones. The external is 1C then I have 2C, 3C, 4C, 5C, 6C, 7C and I end up with 8C and 9C for the controller.

You can use whatever ports you want, all ports work as in or out depending on what they are connected to.

Do you use any special cable to connect from the controller to the expander?

No, a standard SFF-8087 to SFF-8087 SAS Cable.


I suspect you may have a problem with the expander.

 

[Miguel Nunes]

I suspect you may have a problem with the expander.

Are you using Dell firmware or cross flashing?

 

[Johnnie Black]

Cross flashed to LSI IT mode, currently firmware p20, though like I said earlier, it also worked with previous releases.

 

[Miguel Nunes]

Cross flashed to LSI IT mode, currently firmware p20, though like I said earlier, it also worked with previous releases.

I will check all the expander ports then.
I would prefer to have the hardware raid available.

Thank you Johnnie

 

[Johnnie Black]

I would prefer to have the hardware raid available.

Not a good idea to use a RAID controller (or a controller in RAID mode) with FreeNAS.

 

[Miguel Nunes]

Not a good idea to use a RAID controller (or a controller in RAID mode) with FreeNAS.

Indeed you are right. Thank you for remembering. :)

Like I mentioned earlier we are trying to build an hybrid virtualization server using a 24 bays 4U rack. It was a shame when the PERC controller complained about the expander.

I am still trying to do some tests with the expander. We got a P410 with 1GB FBWC that is an HP product, that is compatible with a HP Proliant Microserver we have. I will test the expander and see if there is anything wrong with it.

I will post the results soon.

Again, thank you so much for your support. You have helped me reason correctly about the fault.

 

[Miguel Nunes]

The P410 saw the Drive in several ports.

It seems to be a problem with the PERC H310.

I have reset the configuration to factory default but it is still not working...