My Dream System (I think)

[joeschmuck]

Well this board is a b**** to me. Right now I cannot use the failover meaning IPMI over the LAN 1 connection. Supermicro has me doing a few things, including reflashing the ethernet controller. I'll take a stab at it in the morning since the dedicated IPMI is working fine. My dream system is becoming a nightmare but I'll get through it. I was never in a hurry to replace my FreeNAS system in the first place so I can take my time.

 

[rogerh]

Just as a matter of interest, why do you need the failover if the dedicated IPMI port is working? Or is it just a matter of wanting things to work as they are supposed to? Idle, but genuine, question.

 

[joeschmuck]

Reflashing the BMC corrected the IPMI failover issue.

Just as a matter of interest, why do you need the failover if the dedicated IPMI port is working? Or is it just a matter of wanting things to work as they are supposed to? Idle, but genuine, question.

Because it should work as designed. If this wasn't a feature then it woudn't matter to me. If there is an issue with the MB then I'm still within my NewEgg return policy dates and I'd like to know before that happens. I'm sure there is something wrong with the firmware, the BMC shouldn't have stopped in the first place so hopefully this problem either never happens again or it happens all the time and a firmware upgrade is generated. I'm curious if I can recreate the problem but scared that I won't be able to recover as well and the BMC will be broken.

 

[Rand]

Its not a bug its a security feature;) Access to IPMI through regular network channels was considered a high risk scenario sometime last year iirc ;)

 

[joeschmuck]

I can see how it would be a security risk however in a home system, behind a firewall, I'm not worried at all. If someone can hack into my home network (I'm sure it's very possible for the true hackers out there), IPMI is the least of my worries.

 

[joeschmuck]

BTW, I do have a small version of FreeNAS running. It's backing up my main computer right now and once it's done, I have more testing to do such as safe shutdowns and automatic startups. I have discovered that when it comes to pass-thru SATA, you cannot do that with the onboard ports or a part of them, that sucks but I know this was a strong possibility from the beginning. I have a small add-on SATA board I'm using which supports 4 drives and I'm running just a mirrored pair for right now.

 

[Rand]

I have discovered that when it comes to pass-thru SATA, you cannot do that with the onboard ports or a part of them, that sucks but I know this was a strong possibility from the beginning. I have a small add-on SATA board I'm using which supports 4 drives and I'm running just a mirrored pair for right now.

You can't pass the onboard controller through even if there is no device attached? Or have you intstalled ESX on a drive connected to the sata controller?

 

[joeschmuck]

I might be able to pass through the onboard controller if I had an ESXi supported add-on SATA card, but the one I have will not let me create VMs. I even built a custom ESXi image with a driver which "should" have supported the add-on board but that failed. I'm not done researching this, I've only just begun but if I can make my add-on board support properly, then I'm done purchasing parts and I'll be a happy camper. But, as I've said many times, I'm taking my time about this, not rushing anything. I'm having difficulties with a proper shutdown right now, not sure if it's one of my add-on cards, IPMI, or just some setting I need to make. Once I'm done and have everything figured out, I plan to write up a nice and clear how-to for anyone who really wants to run FreeNAS on ESXi, but right now, it's not for the faint of heart.

 

[Rand]

Tried to run ESX today on my backup system but had to find out the hard way that Avoton does not have VT-d support :/

Re Sata - I assumed you followd this guide here or a similar one? http://www.v-front.de/2013/11/how-to-make-your-unsupported-sata-ahci.html

 

[Ericloewe]

That's funny, mine's been working fine with failover. In fact, it's sitting on my desk and the only cables are power, VGA, keyboard, mouse and a single Cat. 6 patch cable.

 

[Frallan]

I did a quick reboot to check mine and it's set to Failover with no possibility to change it. I even tried attaching a cable to the dedicated port but I was still unable to edit the Failover part.

 

[joeschmuck]

Tried to run ESX today on my backup system but had to find out the hard way that Avoton does not have VT-d support :/

Re Sata - I assumed you followd this guide here or a similar one? http://www.v-front.de/2013/11/how-to-make-your-unsupported-sata-ahci.html

Yes, that was the exact place I used. My add-on SATA controller is on the add-on list however it fails to work, specifically I can have the boot drive connected to the SATA add-on card, install ESXi and it boots, but I cannot create VMs because ESXi does not recognize the hard drive with which it booted off of as a datastore. It's odd and I will of course give it another shot using a different hard drive as the boot device just to see if I can make it work. My add-on card is an IOCrest SI-PEX40062 with a Marvel 88SE9235 chipset. As a pass-thru controller, it's running fine for FreeNAS, but it only has 4 SATA ports and I need 6 ports for my current pool. And I'm not recommending this card to anyone either, it's what I currently have from another computer, which I'll eventually put back into it so I'll need to get another add-on card. I'm not certain I want to go with an M1015 card just yet simply because I'm using ESXi, not FreeNAS directly. And I don't want to need to update IT firmware on the card periodically, but I need to learn more about this aspect to be honest. Hey, I know that I don't understand everything.

That's funny, mine's been working fine with failover. In fact, it's sitting on my desk and the only cables are power, VGA, keyboard, mouse and a single Cat. 6 patch cable.

IPMI is working just fine again, not sure what it failed in the first place. I used the IPMI Viewer to originally upgrade the IPMI firmware, maybe there was a flaw in it? I used the DOS method to reflash it and recover everything and it's been fine since.

I did a quick reboot to check mine and it's set to Failover with no possibility to change it. I even tried attaching a cable to the dedicated port but I was still unable to edit the Failover part.

I can change the Failover setting to dedicated using the IPMI interface, I prefer using Firefox and the IPMI IP address vice the IPMI Viewer, however the failover setting is working fine for me.

 

[Rand]

You could install on an USB stick instead? ESX can even handle USB3 as far as I have read... Stick size does not matter (1 GB is sufficient)

 

[jgreco]

You can install ESXi on a USB stick, yes. However, it cannot use that as a datastore, which seems to be the problem @joeschmuck is having, if I understand correctly.

 

[joeschmuck]

You could install on an USB stick instead? ESX can even handle USB3 as far as I have read... Stick size does not matter (1 GB is sufficient)

Interesting but my original idea was to place the motherboard SATA ports into Pass-thru and if I cannot get the add-on card to store my datastores, it's still a bust.

 

[Rand]

http://www.virten.net/2015/10/usb-devices-as-vmfs-datastore-in-vsphere-esxi-6-0/
Just for testing purposes o/c ;)

but I cannot create VMs because ESXi does not recognize the hard drive with which it booted off of as a datastore.

Have you tried a second drive?

 

[joeschmuck]

Another issue is a bare board X11SSM-F when running ESXi 6.0U1 as a clean install, I am unable to shut down the system from vSphere. It appears to halt but the video screen remains up and the system power of course remains on. Rebooting works. Supermicro states ESXi 6.0U1 should work on this board. Sure it runs but it will not shut down. This has me a bit concerned about using ESXi as my hypervisor.

http://www.virten.net/2015/10/usb-devices-as-vmfs-datastore-in-vsphere-esxi-6-0/
Just for testing purposes o/c ;)

Interesting read. I might take a look at this but as noted, not really supported and since I really want to put many VMs on my device, a USB device alone will not cut it.

 

[jgreco]

You'd think after twenty fricking years we could manage to make ACPI work reliably.

Possibly a more important question is if it hangs when you command a reboot. The power-off thing doesn't bother me quite so much as long as you're not using a cluster where the hosts are configured to power on and off.

 

[Rand]

USB3 attached SSD would be sufficiently fast. But indeed its not supported.

The question is what do you want to have in terms of hardware/vms in the end?
I understand you want to run FreeNas off the onboard SATA disks in pass through (to not have the hassle of updating mpt sas drivers)? That will at least need a secondary storage controller to have the datastore on -
you can take your old SATA card but that seems to be trouble. You can get any supported SAS card - ESX should run on it and no need to update to the latest bios all the time unless you have issues - no need to keep FreeNas in sync with it since its not using it.

You'd think after twenty fricking years we could manage to make ACPI work reliably.

Possibly a more important question is if it hangs when you command a reboot. The power-off thing doesn't bother me quite so much as long as you're not using a cluster where the hosts are configured to power on and off.

The question is whether its an ESX issue at all or a board issue. Try a live cd maybe and test shutdown from it?

 

[jgreco]

A lot of years of this seems to indicate that it's less a matter of X or Y and usually a particular combination. It may be that Linux works fine to shut it down but it's still a board issue.