free esxi vm / vmfs encryption

[phier]

hello,
maybe its out of topic here? but i cant see how to encrypt VM or vmfs store on ESXi 7.0 (free edition) Any idea?

thanks

 

[Patrick M. Hausen]

What gives you the idea ESXi could do that? You can put a datastore on TrueNAS via NFS or iSCSI and encrypt that in ZFS.

 

[phier]

documentation> https://communities.vmware.com/t5/ESXi-Discussions/vm-vmfs6-encyrption/m-p/2921615#M3715
but its not clear.

Not sure i get it ... put a what datastore on Truenas via nfs/iscsi?
thx

 

[Patrick M. Hausen]

Put a VMware datastore to store VMs on an NFS or iSCSI share in TrueNAS and encrypt that.

That's a solution to the problem of storing VMs encrypted. Or why would you want encrypted VMFS?
And things like this are one of the main reasons why people virtualise TrueNAS in ESXi and combine the two. Also snapshots, replication, ...

I still have not understood why you insist on running TN inside ESXi given the questions you are asking. I feel like you do not need either ESXi or TrueNAS.

 

[phier]

Put a VMware datastore to store VMs on an NFS or iSCSI share in TrueNAS and encrypt that.

yes but such a solution will have a performance issue (as we discussed earlier) its not a good idea to store VMs on TrueNAS pool. Correct me if i am wrong.

Or why would you want encrypted VMFS?

because i cant make there a physical security on a good level... so the only way is to store in ot VMFS, which i believed is supported by ESXi but apparently its not in free version.
I really dont want some unknown person pulls off drive; load it on his machine and boot my VMs ...

The only solution in my mind is to encrypt on OS level... linux lvm... windows truecrypt etc... if one doesnt want to pay 4k for esxi enterprise.

And things like this are one of the main reasons why people virtualise TrueNAS in ESXi and combine the two. Also snapshots, replication, ...

i mean u said receltny its not a good idea to store VMs on TrueNAS and if so not to use more then 50% capacity of the pool... cant recall.

I still have not understood why you insist on running TN inside ESXi given the questions you are asking. I feel like you do not need either ESXi or TrueNAS.

i wanted to make things elegant and separate stuff, i really dont want mess Truenas with a firewall or homeassistant or my personal VMs inside and doing different passthrough of different devices into these VMs.

thx

 

[phier]

hm.... any view on this here? pls

 

[Patrick M. Hausen]

What applications do you run that are so critical in terms of performance? How many dozen or hundreds of users do you have? See? Put VMs on NFS or iSCSI and be happy. It works. Splendidly. It is slower than putting the VM disk images on VMFS on an NVME disk, sure. You need to assess your needs, not try to wring the "optimal performance" out of your system as an academic exercise.

If encrypting VMs is essential to you, then put them on the TrueNAS store (mounted from ESXi - I understand you do not want to use bhyve) and encrypt that.

As for the 50% rule - yes. But what's the problem? Storage pools built from spinning disks are easily in the tens of terabytes even with consumer hardware, while VM disk images are regularly 20-100 gigabytes each at most. Create a 2 TB iSCSI zvol on your TrueNAS pool and use only 1 TB of it. 1 TB of VMFS is plenty! Create the pool from mirrored pairs and add an SLOG if you need to.

If you want to run Plex in a VM, put the media outside the Plex VM on TrueNAS storage. Dataset, NFS or SMB mount, done. Nobody stores terabytes of data inside monolithic VM images.

But please ... just go ahead and try things! Run your VMs. If they are fast enough for your needs you don't need an SLOG, period. Nobody can tell in advance what that "fast enough" is precisely, unless you run the d... thing and observe.

HTH, really,
Patrick

P.S. Nobody said it was a bad idea to store VMs on a TrueNAS pool. Only that there are certain constraints that need to be considered if you don't want to be disappointed. But that can easily be done. Large Enterprises have all their VMs on iSCSI in multi machine VMware clusters. Some of these iSCSI stores are TrueNAS.

 

[phier]

As for the 50% rule - yes. But what's the problem? Storage pools built from spinning disks are easily in the tens of terabytes even with consumer hardware, while VM disk images are regularly 20-100 gigabytes each at most. Create a 2 TB iSCSI zvol on your TrueNAS pool and use only 1 TB of it. 1 TB of VMFS is plenty! Create the pool from mirrored pairs and add an SLOG if you need to.

i am not sure but i can remember that you said time ago that its not a good idea to put VMs on Truenas (i am not saying now on main pool on magnetic drives) i mean on separate nvme drive... i got lost and i think there is lot of misunderstanding or so.

Yes, my view from the beginning was to use 2tb nvme passthrough to Truenas - 1x pool on top of it of size 2tb and then export it via iscsi or somehow else to esxi ...

 

[Patrick M. Hausen]

Then you have no redundancy. And that's the point of using TN combined with ESXi, isn't it? If you have just a single NVMe drive instead of wasting half your storage "because iSCSI and ZFS" I'd rather just use it as VMFS inside ESXi.

But then I don't see a need to encrypt my VMs. For me redundancy is the number one feature I would not go without. I even snapshot and replicate all my data to a second machine that is located at my company's office. I bought the machine, but of course I get power and Internet for free.

Priorities ... it all boils down to what you really want to do. There is no "perfect" setup, only one that satisfies your most important needs and possibly has got some drawbacks that are not quite so important.

Maybe we should turn this thread upside down and you tell us what you have set up so far and what you want to do next?

Fast, reliable, cheap - pick any two. This rule still holds. Of course your perfect setup exists: just build two boxes connected by 10 G Ethernet, one TrueNAS, one ESXi

Really, just kidding. I am definitely not intending to be condescending but we seem to have fundamentally different angles from which we view IT products. So if you like to continue, I will still try to help.

Patrick

 

[phier]

Then you have no redundancy. And that's the point of using TN combined with ESXi, isn't it? If you have just a single NVMe drive instead of wasting half your storage "because iSCSI and ZFS" I'd rather just use it as VMFS inside ESXi.

for a what reason u need a redundancy, what redundancy? You do a snapshoting (which is a backup/redundancy?).
Ok, so when using iSCSI over ZFS then you can use only 50% capacity of the drive because in case more space is used then there are performance issues.... still no idea why these performance issues are there ... and about what performance issues we are talking 1% speed reduction? 40% speed reduction etc...

 

[Patrick M. Hausen]

Redundancy in your storage system which is the main selling point of ZFS. I want all my storage to survive the failure of at least one component. So my VM storage is 2 NVMe disks in a mirror configuration. But then I do not export that to ESXi but run the VMs on top of that pool in TrueNAS.

But yes, that's my main point. Never ever have anything on just a single drive. Drives fail.

So in your case my line of thinking is:

• It's still just a single drive.
• Sharing over iSCSI wastes half of the capacity.
• At lower performance because of the iSCSI and network overhead (I don't have an idea how much precisely for this one).
• So just put the NVMe drive in ESXi and do regular snapshots and backups with GhettoVCB.
• Essentially: iSCSI in this case is a waste of ressources/money.

Now if you build a hundred terabyte pool for an enterprise VMware cluster, that's a different story.

There were some proper figures for the performance degradation when you fill a pool that serves as an iSCSI VM backend. And yes, the figures were dramatic somewhere between the 40% and the 60% mark. For a home setup that might still be ok above 60% ... noone can tell unless you measure.

Does anyone of the other regulars have the link to those benchmarks handy?

So my main motivation is "don't lose data" (obviously) but also "if possible keep availability as high as possible in a home setup". I run private yet productive services on these systems and I don't want my family Nextcloud nor my parent/teacher assoc wiki to be offline because of a single disk failure while I am on vacation.

So (ever had a look at my signature?)

• all my systems have IPMI
• I have VPN access to my home network from anywhere
• main NAS:
  • mirrored boot pool - two Transcend 32 G SATA SSDs
  • mirrored VM pool: two Samsung NVMe SSDs
  • storage pool: RAIDZ2 spinning disks
• snapshots and replication of the VMs from the SSD pool to the storage pool (RAIDZ2 > mirror in terms of redundancy)
• snapshots and replication of the VMs and everything else on the storage pool to a second system at my office

Now ... what are your main concerns and which are the applications you absolutely cannot do without?

 

[phier]

ah okay,
thank you for the explanation ... okay sounds logical to me ...

and i can see why u have redundancy, for me if the drive is off and system is down i can live with that... thats why i was thinking have a one drive and do snapshots...

well yes, other thing is ... money, once u start adding redundant components etc... $$$ goes rapidly up.

but i think you are talking more about High-Availability, because even with 1 or 2 drives it will be almost same Fault-tolerant solution.