X10SRL-F w/ ESXi -- can't passthrough onboard SATA?

[averyfreeman]

Hey,

Does anyone else have this board? Supermicro X10SRL-F... or another X10 E5 2011-3 board, similar to it? (Has C612 chipset)

I just picked it up and put it together and I was about to move my disk array over to it when I noticed I don't see the onboard SATA controllers (it has two) in the list of PCI devices that can be VT-D passed through.

There's plenty of others, USB controllers, NVMe adapters, NICs, even the Aspeed IPMI VGA controller... but no SATA controllers?

My first FreeNAS build was an ESXi 5.5 passthrough VM using the onboard Cougar Point SATA controller from the C202 chipset of a Supermicro X9SCL-F. That was at least 3-4 years ago.

Why is it a newer board wouldn't be able to do the same thing? Especially with TWO onboard SATA controllers... Does it have something to do with being an E5 board?

Would love it if someone could shed light on this dismal situation...

Thanks!

 

[HoneyBadger]

Are you booting from an SSD or HDD attached to the SATA controller?

 

[averyfreeman]

Are you booting from an SSD or HDD attached to the SATA controller?

Hey HoneyBadger,

Nope, it's ESXi so I'm booting from a USB flash drive. There's a SATA drive connected to one of the two controllers, but it's not being used for anything OS-related.

Also, the NVMe adapter is connected to the datastore for ESXi, and that's available to passthrough (which, it seems obvious that if ESXi were checking for stuff that *shouldn't* be passed through, it wouldn't be there...)

Lastly, one would likely think that if there's one SATA controller with a disk attached and another with no disks, if it were concerned about an attached disk it would present the one without a disk for passthrough and not the one with the disk. It doesn't present either one (!)

WTF... I am running out of PCIe slots, I don't want to have to populate two of them with LSI controllers to be able to fill this 15-drive enclosure with my drives...

 

[Spearfoot]

Hey,

Does anyone else have this board? Supermicro X10SRL-F... or another X10 E5 2011-3 board, similar to it? (Has C612 chipset)

I just picked it up and put it together and I was about to move my disk array over to it when I noticed I don't see the onboard SATA controllers (it has two) in the list of PCI devices that can be VT-D passed through.

There's plenty of others, USB controllers, NVMe adapters, NICs, even the Aspeed IPMI VGA controller... but no SATA controllers?

My first FreeNAS build was an ESXi 5.5 passthrough VM using the onboard Cougar Point SATA controller from the C202 chipset of a Supermicro X9SCL-F. That was at least 3-4 years ago.

Why is it a newer board wouldn't be able to do the same thing? Especially with TWO onboard SATA controllers... Does it have something to do with being an E5 board?

Would love it if someone could shed light on this dismal situation...

Thanks!

Some finagling may let you pass your motherboard's SATA controllers through to an ESXi VM (see thread linked below). I no longer use this configuration; use at your own risk! :)

https://forums.freenas.org/index.ph...e-x10sl7-f-motherboard-sata-controller.51843/

 

[averyfreeman]

Some finagling may let you pass your motherboard's SATA controllers through to an ESXi VM (see thread linked below). I no longer use this configuration; use at your own risk! :)

https://forums.freenas.org/index.ph...e-x10sl7-f-motherboard-sata-controller.51843/

Thank you for pointing me in the right direction. Unfortunately I haven't been able to get it to work, but I'll keep trying, or maybe resort to RDM.

 

[Jessep]

I have this working fine with an X10SRI-F, I used the following link which it appears you already posted in.

https://forums.servethehome.com/index.php?threads/esxi-6-0-passthrough-onboard-sata.8902/

Specifically this post:

Code:

I'm not positive on the C610 chipset, but, as ehfortin mentioned in his thread, you'll need to add the Wellsburg controller to /etc/vmware/passthru.map.

You'll need the vendor and device id. Run lspci at the ESXi command line. Look for the line pertaining to your mass storage controller. It will likely be "0000:00:XX.X Mass Storage Controller: Intel Corporation Wellsburg AHCI Controller".

Make a note of the hex prefix (0000:00:XX.X) and then run 'lspci -n'.

Find the line corresponding to the prefix, it'll look something like '0000:00:1f.2 Class 0106: 8086:8d62 [vmhba2]'. That last part, "8086:8d62" in this case, are the vendor and device id.

Then add them to your passthru.map file like so:

# INTEL Wellsburg AHCI
8086 8d62 d3d0 false

 

[kdragon75]

Use physical mode RDMs!

 

[joeschmuck]

Use physical mode RDMs!

Agreed, this is an option and it should work fine. The other option is to add an HBA. Not too many options here.

 

[averyfreeman]

Use physical mode RDMs!

Oh, thanks - er, what's a physical mode RDM? I thought RDMs don't allow smartmontools reading, does physical mode solve that, or something?

First time I haven't been able to just pass through the controller, worked on all the E3 and Core boards I've had in the past... (grumble)

 

[kdragon75]

Oh, thanks - er, what's a physical mode RDM? I thought RDMs don't allow smartmontools reading, does physical mode solve that, or something?

First time I haven't been able to just pass through the controller, worked on all the E3 and Core boards I've had in the past... (grumble)

The most common use case is with tape drives as they use some odd scsi commands. The reason this works is because in physical compatibility mode, there is not attempt to translate the commands going to the disk. Using virtual mode, ESXi still translates communication to and from the disk.
https://kb.vmware.com/s/article/2009226

 

[averyfreeman]

Agreed, this is an option and it should work fine. The other option is to add an HBA. Not too many options here.

I wrestled a 9211-8i out of another server as a sacrificial lamb. I'll try physRDM later when I need to fill the last 7 drive bays.

Will probably start doubling m.2 SSDs here soon, too, on those neat dual m.2 x8 adapters Supermicro makes... Ahh, PCIe bifurcation...

Damn, after having nothing but MATX boards for years I didn't think when I got my first 7-slot 2011-3 board all the slots would be filled within the first few days (!).

 

[averyfreeman]

The most common use case is with tape drives as they use some odd scsi commands. The reason this works is because in physical compatibility mode, there is not attempt to translate the commands going to the disk. Using virtual mode, ESXi still translates communication to and from the disk.
https://kb.vmware.com/s/article/2009226

Very interesting. Sounds cool for like one or two drives but it sounds like a pain when installing by the handful... Thanks though.

 

[scurrier]

Did you check your BIOS settings to make sure there's nothing in there? I have this board, but do not virtualize with it.

There is a somewhat recent BIOS update for it, too (within last year?). Don't know if that would help.

 

[averyfreeman]

Did you check your BIOS settings to make sure there's nothing in there? I have this board, but do not virtualize with it.

There is a somewhat recent BIOS update for it, too (within last year?). Don't know if that would help.

Hi, yeah thanks for that, I updated the bios the day I got it to 3.1 and there's a little bit in the BIOS regarding the two SATA controllers and their modes, pretty standard - IDE, AHCI, "RAID" - it's preset to AHCI and that's what it should be. Nothing else really going on in those menus. Nothing SATA controller-related in PCIe devices menu.

I did have to read up pretty intensely about QPI, snooping, NUMA, ECC and RAM+Processor power settings to get ESXi to be more stable, even with optimized defaults it's been a bit of a trainwreck, but I think I have a good handle on it now. Sat down and just scoured the ESXi 6.7 best practices guide, then afterwards scoured Google for every single BIOS option I didn't know yet.

Anyway, I just passed through a 9211-8i. Hopefully it will be stable enough now I can move my pool over to it! But in regards to the OP, I have 100% given up on trying to passthrough the onboard AHCI.

 

[Virtualiser]

Hi. Bit late maybe. But I have exactly this board and was able to passthrough the onboard sata successfully in ESXi6.5
You need to do some fiddling on the commandline of ESX but all in all it was not to difficult.
Gonna try in Ubuntu 18.04 next and try to run Freenas in KVM :)

 

[Jessep]

UPDATE:
ESXi 6.7 Update 1
passthru.map now working again as it did in 6.5