samuel-emrys
Contributor
- Joined
- Dec 14, 2018
- Messages
- 136
Hi all,
I'm seeking some feedback on an idea to move my current setup (FreeNAS bare metal) to a VM managed under Proxmox. To provide a bit of context, my use case for a server has changed recently. In the past I've been content with FreeNAS as a file server, and use it to manage a range of largely passive services in jails. This has worked well, but I now have a need to run a variety of VMs for productivity reasons. This would include a Windows Server VM to expirment with AD, and a Windows VM for gaming with GPU passthrough, but also access to tools like the Adobe Creative Suite, as my daily driver is now a Linux system. I've attempted to virtualise things before using Bhyve, and my experience was that it was very unstable, and the tools for VMs available in FreeNAS currently are not everything I'd hope for. I realise Proxmox has ZFS capabilities, but I've no desire to re-create the wheel. Storage is something that FreeNAS does well, and it automates a lot of the functionality that I would have to manually implement in Proxmox. I could do this, but I have no interest in it.
Another obvious, and more supported alternative is ESXi. I'm a fan of the open source nature of Proxmox, and it seems like it has greater functionality at the free tier than what ESXi does. That said, I'm not completely opposed to considering this more closely if it truly does provide a tangible benefit, but I'd rather try Proxmox first before exploring that. To avoid questions about it later, I've read the following threads of note:
- https://www.truenas.com/community/t...-not-completely-losing-your-data.12714/page-6
- https://www.truenas.com/community/t...ative-for-those-seeking-virtualization.26095/
- https://www.truenas.com/community/threads/freenas-on-proxmox-whats-the-current-state-of-play.84287/
- https://www.truenas.com/community/threads/freenas-virtualized-on-proxmox.55849/
My server configuration is as follows:
- Supermicro X9-SRi-F
- Intel Xeon E5-2660 V2 (10c/20t)
- 6x WD Red 4TB (RAIDZ2) - named 'vault'
- 2x Samsung 860 Evo 500GB (Mirrored) - named 'jailhouse'
- 8x16GB (128GB) Samsung DDR3-1866 Registered ECC DIMMS
Currently, I've got two pools; one for the WD Reds which contains the bulk of my storage, and one for the SSDs, which is used for jails - this typically sees the most I/O.
So, with all of that said, I'm trying to think about what a migration to Proxmox would require. My initial thought is that I could migrate the SSD pool to Proxmox's control, and use it as the installation and VM storage pool (this may/will probably involve wiping it and re-creating the mirror; I don't mean migrate in the strict sense here). I could then either migrate my existing set of jails to 'vault', or transition the services provided in those jails to LXC containers. I would also need to make the 'vault' disks directly available to a FreeNAS vm, so I would need to pass through the on-board disk controller. Running
My read of this is that I have two disk controllers on my motherboard; one with 6 SATA ports, and one with 4 SATA/SAS ports. I could connect 'vault' to the 6 port disk controller, and my SSDs to the 4 port disk controller, then pass the 6 port controller directly through to the FreeNAS VM. My plan was to allocate 3x vCPUs to the FreeNAS VM and 16GB of RAM, leaving me 17x vCPUs and 112GB of RAM for other services and productivity VMs. I'll obviously play this by ear wrt performance. Any further drive/pool expansion would require the purchase of a HBA card.
One of the assumptions that I have here is that I will be able to create an NFS/SMB share from my FreeNAS VM to make the storage pool available to other VMs. I also have FreeNAS set up as a UPS master node, with my router configured as a slave node; will it be possible to maintain this with FreeNAS virtualised? My guess is that I would need to pass through the USB port being utilised for communication to the FreeNAS VM?
So, I have redundant VM and boot media, as well as have provided the FreeNAS VM with direct access to the disk controllers. Is there anything else I need to consider? Are there any holes that can be poked in this plan? Any suggestions or feedback are appreciated.
Cheers.
I'm seeking some feedback on an idea to move my current setup (FreeNAS bare metal) to a VM managed under Proxmox. To provide a bit of context, my use case for a server has changed recently. In the past I've been content with FreeNAS as a file server, and use it to manage a range of largely passive services in jails. This has worked well, but I now have a need to run a variety of VMs for productivity reasons. This would include a Windows Server VM to expirment with AD, and a Windows VM for gaming with GPU passthrough, but also access to tools like the Adobe Creative Suite, as my daily driver is now a Linux system. I've attempted to virtualise things before using Bhyve, and my experience was that it was very unstable, and the tools for VMs available in FreeNAS currently are not everything I'd hope for. I realise Proxmox has ZFS capabilities, but I've no desire to re-create the wheel. Storage is something that FreeNAS does well, and it automates a lot of the functionality that I would have to manually implement in Proxmox. I could do this, but I have no interest in it.
Another obvious, and more supported alternative is ESXi. I'm a fan of the open source nature of Proxmox, and it seems like it has greater functionality at the free tier than what ESXi does. That said, I'm not completely opposed to considering this more closely if it truly does provide a tangible benefit, but I'd rather try Proxmox first before exploring that. To avoid questions about it later, I've read the following threads of note:
- https://www.truenas.com/community/t...-not-completely-losing-your-data.12714/page-6
- https://www.truenas.com/community/t...ative-for-those-seeking-virtualization.26095/
- https://www.truenas.com/community/threads/freenas-on-proxmox-whats-the-current-state-of-play.84287/
- https://www.truenas.com/community/threads/freenas-virtualized-on-proxmox.55849/
My server configuration is as follows:
- Supermicro X9-SRi-F
- Intel Xeon E5-2660 V2 (10c/20t)
- 6x WD Red 4TB (RAIDZ2) - named 'vault'
- 2x Samsung 860 Evo 500GB (Mirrored) - named 'jailhouse'
- 8x16GB (128GB) Samsung DDR3-1866 Registered ECC DIMMS
Currently, I've got two pools; one for the WD Reds which contains the bulk of my storage, and one for the SSDs, which is used for jails - this typically sees the most I/O.
So, with all of that said, I'm trying to think about what a migration to Proxmox would require. My initial thought is that I could migrate the SSD pool to Proxmox's control, and use it as the installation and VM storage pool (this may/will probably involve wiping it and re-creating the mirror; I don't mean migrate in the strict sense here). I could then either migrate my existing set of jails to 'vault', or transition the services provided in those jails to LXC containers. I would also need to make the 'vault' disks directly available to a FreeNAS vm, so I would need to pass through the on-board disk controller. Running
pciconf -lv
on my existing FreeNAS host yields the following:Code:
user@helios:/mnt/vault/private/user# pciconf -lv ... ahci0@pci0:0:31:2: class=0x010601 card=0x062b15d9 chip=0x1d028086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = 'C600/X79 series chipset 6-Port SATA AHCI Controller' class = mass storage subclass = SATA ... isci0@pci0:6:0:0: class=0x010700 card=0x062b15d9 chip=0x1d6b8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = 'C602 chipset 4-Port SATA Storage Control Unit' class = mass storage subclass = SAS ...
My read of this is that I have two disk controllers on my motherboard; one with 6 SATA ports, and one with 4 SATA/SAS ports. I could connect 'vault' to the 6 port disk controller, and my SSDs to the 4 port disk controller, then pass the 6 port controller directly through to the FreeNAS VM. My plan was to allocate 3x vCPUs to the FreeNAS VM and 16GB of RAM, leaving me 17x vCPUs and 112GB of RAM for other services and productivity VMs. I'll obviously play this by ear wrt performance. Any further drive/pool expansion would require the purchase of a HBA card.
One of the assumptions that I have here is that I will be able to create an NFS/SMB share from my FreeNAS VM to make the storage pool available to other VMs. I also have FreeNAS set up as a UPS master node, with my router configured as a slave node; will it be possible to maintain this with FreeNAS virtualised? My guess is that I would need to pass through the USB port being utilised for communication to the FreeNAS VM?
So, I have redundant VM and boot media, as well as have provided the FreeNAS VM with direct access to the disk controllers. Is there anything else I need to consider? Are there any holes that can be poked in this plan? Any suggestions or feedback are appreciated.
Cheers.
Last edited: