2 ESXi compute nodes + 1 TrueNAS Scale storage

[razvan]

Hi everyone,

I need to configure 2 ESXi machines using iSCSI. For that I have this:

1. Two ESXi machines, each with 2x 10G NIC, but they are only compute nodes, no attached storage (the only storage is 2 drives in RAID1 that serves for the OS)
2. 1 storage server with 24 NVME drives (1.6TB each) and 4x 10G NICs, and I am thinking to use TrueNAS to create 2 POOLS, one for iscsi target and one for NFS share.
3. 1 Switch with 10G ports where all 3 servers are connected

I mention that unfortunately I have only one /24 subnet.

My goal is to have:

- 1 iSCSI target used for ESXi machines
- 1 NFS share that will be used by a K8S cluster that sits on top of those machines (will be using Storage Classes and Dynamic provisioning)

So having all these, how is better to configure both TrueNAS machine and ESXi machines on networking side so I can have the best performance/reliability result?

Thank you!

 

[jgreco]

Perhaps start by explaining why you would choose TrueNAS Scale for this...?

 

[razvan]

What do you suggest to use? I tried TrueNAS Core but it only see 12 of the 24 disks, idk why so I just tested TrueNas Scale and it worked. Just created for test a pool with all 24. I am open to other configurations.

 

[jgreco]

I tried TrueNAS Core but it only see 12 of the 24 disks,

Well, there has to be some reason for that. You haven't provided any description of your system. Most likely you have used an ill-advised way to attach the disks. Sometimes this will seem to work on Linux but not on FreeBSD, but this is usually only an illusion.

What do you suggest to use?

TrueNAS Core is the recommended solution for iSCSI and pretty much for every use where there is not something that you need that only Scale offers.

 

[razvan]

Well, there has to be some reason for that. You haven't provided any description of your system. Most likely you have used an ill-advised way to attach the disks. Sometimes this will seem to work on Linux but not on FreeBSD, but this is usually only an illusion.



TrueNAS Core is the recommended solution for iSCSI and pretty much for every use where there is not something that you need that only Scale offers.

Thank you, I will give another try with TrueNAS core.
How do you suggest to configure the network? I am thinking on using for every compute node a separate vmkernel nic (new vSwitch) used for iSCSI and on the storage part to use 2 NICs for NFS and 2 NICs for iSCSI. I don't know what is the best solution on networking part. Thank you!