PowerEdge R730

[Andres Gonzalez]

Hello,

Any experience installing FreeNAS under a Dell PowerEdge R730 server with a RAID H730 PERC ?

I found that on other Dell servers (Power Edge R510) that also has a Perc disk controller FreeNAS cannot see the disk unless a Raid-0 is created (from the Raid controller bios) for every disk, thing that is not good as I understand ZFS does not access directly to the disk.

Thanks.

 

[Robert Trevellyan]

Any experience installing FreeNAS under a Dell PowerEdge R730 server with a RAID H730 PERC ?

I found that on other Dell servers (Power Edge R510) that also has a Perc disk controller FreeNAS cannot see the disk unless a Raid-0 is created (from the Raid controller bios) for every disk, thing that is not good as I understand ZFS does not access directly to the disk.

No experience, but it looks like a very capable system. However, unless you can set the H730 to JBOD mode, you'd be better off installing one or more of the recommended HBAs, e.g. the forum favorite IBM M1015 flashed to IT mode, or equivalent. As you've gathered, RAID0 is not a substitute for JBOD.

 

[Andres Gonzalez]

No experience, but it looks like a very capable system. However, unless you can set the H730 to JBOD mode, you'd be better off installing one or more of the recommended HBAs, e.g. the forum favorite IBM M1015 flashed to IT mode, or equivalent. As you've gathered, RAID0 is not a substitute for JBOD.

According to this: http://www.dell.com/learn/us/en/04/campaigns/dell-raid-controllers
it looks like the PERC H730 controller can be configured as JBOD, in such as case it can work, right ?

 

[Robert Trevellyan]

Looks likely. In fact, the H730 is on the FreeBSD HCL, so that's promising: https://www.freebsd.org/relnotes/CURRENT/hardware/support.html

 

[Tom85768]

Dell R730 was not a stable platform for providing NFS for vSphere 6.5 environments based on FreeNAS 11-Stable on bare metal.. Platform was 2 x Intel 10GbE NIC, 512Gb RAM, 12 x 1.6TB SSD 2 x write intensive 800Gb SSDs. Version 11-Stable of FreeNAS. NFS connectivity to RAID10 8Tb volume was fine until more than 4 of 5 hosts were connected. Then connections would drop of and RPC errors would occur. Attempts to enumerate connections with showmount -a showed rpc errors. Tried increasing NFS servers to 20. Server was 2 sockets, 14 cores per socket. No difference. Tried disabling TSO with ifconfig ix1 -tso (also made no difference). Platform unsuitable for NFS for vSphere 6.5 environments. Now attempting virtualised version.

 

[Ericloewe]

Dell R730 was not a stable platform for providing NFS for vSphere 6.5 environments based on FreeNAS 11-Stable on bare metal.. Platform was 2 x Intel 10GbE NIC, 512Gb RAM, 12 x 1.6TB SSD 2 x write intensive 800Gb SSDs. Version 11-Stable of FreeNAS. NFS connectivity to RAID10 8Tb volume was fine until more than 4 of 5 hosts were connected. Then connections would drop of and RPC errors would occur. Attempts to enumerate connections with showmount -a showed rpc errors. Tried increasing NFS servers to 20. Server was 2 sockets, 14 cores per socket. No difference. Tried disabling TSO with ifconfig ix1 -tso (also made no difference). Platform unsuitable for NFS for vSphere 6.5 environments. Now attempting virtualised version.

Are you talking about hardware RAID?

 

[Tom85768]

Absolutely not. The Dell Perc R730 was in passthrough mode with patrol mode disabled. I have since installed ESXi 6.5 Release A on the Dell R730 and virtualised the FreeNAS 11 system. I created 12 1.46Tb Datastores and then built a VM with 200Gb RAM, 4vCPU and 12 x 1.42Tb VMDKs. I installed the OS on another datastore which was the 800Gb Write intensive SSD. The system is performing beautifully. All 8 ESXi Hosts are connected to 2 NFS datastores each and I am running technet diskspd utility currently across 8 VM's all with different I/O patterns. The connection is currently showing 6Gbps throughput and RAM in use is the entire 200GB. I wanted to stay within a single NUMA boundary and hence I didn't use all the RAM available in the host. The system is performing perfectly now its' virtualised. I can only assume something on the Dell R730 doesn't like FreeBSD as the issues we had were very strange.

 

[Ericloewe]

The Dell Perc R730 was in passthrough mode

Using the mrsas stack? With disks presented as daX devices?

 

[Tom85768]

The disks were presented as daX devices if I remember correctly.