I have reached virtualisation nirvana :-)

[Patrick M. Hausen]

Now I have a machi^W^Wthree TrueNAS SCALE systems running on a single server. Ho ho ho!

I have this Supermicro E300-9A-4C system with a 4 core Atom 3558 CPU. By adding the AOC-SLG3-2M card I got space for 3 M.2 NVME drives in total.
Unfortunately while the E300 case features a PCIe x8 riser card, the mainboard (A2SDi-4C-HLN4F) has got a x4 slot, only.

So yesterday I swapped the mainboard for an X10SDV-4C-TLN4F. That's a Xeon-D1518 based board with a PCIe x16 slot and bifurcation support.
And 4 network interfaces just as the one I used before.

So now I run

• VMware ESXi 7 from a Transcend 370 series SATA SSD - boot and a small VMFS datastore
• 3 TrueNAS scale VMs each with
  • 2 cores
  • 8 GB RAM (locked, no overprovisioning)
  • a 16 GB boot drive on VMFS
  • a dedicated M.2 NVME SSD by passthrough
  • a dedicated network interface by passthrough

all in a system the footprint of a regular sheet of paper.





Sometimes technology is amazing ...

 

[neb50]

What is the purpose of running 3 TrueNAS VM's on a single host?
And same question on the passthru network cards for the VM's?

 

[Kris Moore]

Haha, love it :)

 

[Patrick M. Hausen]

What is the purpose of running 3 TrueNAS VM's on a single host?
And same question on the passthru network cards for the VM's?

Building a cluster without needing 3 hosts. Passthrough storage because that's how it's supposed to work. Passthrough network, so I won't have any issues with VMXnet and friends ...

 

[Newfoundland.Republic]

X10SDV-4C-TLN4F

That's ain't a cheap board!c But very nice. If I can put aside the money I may replace my X9DR3-LN4F+ with it!

Will a LSI 9210-8i work in the PCIe slot?

 

[Patrick M. Hausen]

Will a LSI 9210-8i work in the PCIe slot?

The board comes with 1 PCIe x16 so I don't see any reason why not. Not much use in that particular case, though. If you use any add-on card you are limited to a single 2.5" SATA drive.

 

[Newfoundland.Republic]

f you use any add-on card you are limited to a single 2.5" SATA drive

Are you sure about that? I was reading the manual and the block diagrams shows the PCIe slot as being seperate from the PCIe interface for the SATA drives. It does state that "The X10SDV M.2 is mux with the I-SATA0 port for legacy SATA SSD devices" which seems to me that the SATA0 is shared with the M.2 slot.

 

[Patrick M. Hausen]

I should have written chassis or enclosure instead of case - sorry. In that box there is room for only one drive if the slot in the riser card is occupied.

 

[Newfoundland.Republic]

@Patrick M. Hausen - Thanks! That makes sense. I was starting to think that I had grossly misread the manual (or was losing it ).

Thoughts about replacing the X9DR3-LN4F+ with the X10SDV-4C-TLN4F? It seems to me that it would use a lot less power.

--- Edit ---
And generate less heat!

 

[Patrick M. Hausen]

Thoughts about replacing the X9DR3-LN4F+ with the X10SDV-4C-TLN4F?

Isn't that a completely different beast? I mean, dual Xeon E5, 24 DIMM sockets - this a large server system. The 4 core Xeon D is not even in the same league ...

 

[Newfoundland.Republic]

Yes, but all I am using TrueNAS for is a file server...

 

[Patrick M. Hausen]

In that case - with a maximum of 128 GB of memory the Xeon D should do fine.

 

[Patrick M. Hausen]

I just added

3x Noctua NF-A4x20 PWM cooler
1x Supermicro SNK-C0057A4L active CPU cooler

Result: incredible small box running 4 hypervisor server systems with dedicated hardware each except for shared CPU and memory ...