Recipe for TN plus a X11SPM-TPF motherboard?

[winstontj]

I'm looking for a bit of guidance, thoughts, opinions, etc.
I have a x11spm-tpf machine with 6230n and 192gb (I'm in the market for 4x 32gb 2666 lrdimm + 2x 512gb pmem 100 dimm and my money/paypal is good).

I'm looking to build a tn-core vm that I'll use to backup (keep second local copy of data) our bare-metal tn-core and tn-scale machines.

What's the best way to approach this? I'm starting to think it would be much easier to buy a 9300 hba card, pass everything through to the VM and call it done. Should I give the VM 2x boot drives and have TN mirror virtual drives? Or just give tn one boot drive knowing that the datastore is on another tn machine with redundancy?

I've thought about possibly putting tn-scale on this machine and running esxi host as a vm inside of scale. But it was rumored that xi systems charges to use persistent memory. Can anyone confirm that? I am very actively seeking out 1tb optane pmem dimms for this server. It is my understanding from SM tech support that the x11spm-tpf mobo only operates in memory mode --so does that matter to the whole paid/unpaid xi systems thing? Wouldn't tn scale/core see "memory mode optane" as just memory dimms? Or are there flags and would I run into issues?

Has anyone ever lifted out a hba card and physically moved the hba card plus the drives to a different esxi host? Does it boot and just work or is it a disaster? I'm a little worried about running a vm on an esxi host that can't be moved/migrated.

Hope all that made sense. Thanks in advance for any feedback/thoughts.

 

[jhl]

If you connect a boot drive to the controller and pass the whole controller to Truenas, you don't even need a virtual drive. You can install and run everything on one (or mirrored) boot drives. The advantage of doing it this way is you can move to another hypervisor or even boot the system on bare metal and it should usually boot and not lose data (in my experience)... but there may be issues with network configuration etc if you boot the system as bare metal or boot inside a new VM that doesn't have the exact network configuration.

Redundant boot drives are a big convenience and help with uptime. Your datastore is separate from your system and you can also back up the system configuration somewhere, but even if you're just reinstalling and restoring from a saved config, it's a lot more hassle and downtime than losing one mirrored drive which doesn't even take down your system.

I've found it works best for me to not give Truenas a bare metal boot drive. I pass the HBA and datastore to Truenas and let the hypervisor handle redundancy and backup of the (virtual) boot drive. But I'm using Proxmox which supports backups and migration between physical hosts without a license. And I have a second system to back up my VMs to. Obviously if you have one node or your version of esxi doesn't support that type of backup it might make sense to use mirrored physical drives for your boot volume.