From FreeNAS to ProxMox + FreeNAS + PFSense. Is it feasible?

[juspolo]

Hi folks,
I am thinking about virtualization recently to include a pfSense "machine" at home. However, I do not know if it is feasible or not.

While there are people that support virtualized FreeNAS (e.g. link1, link2, link3, and even two books ), there are also people who does not (e.g. link1, link2). Hence, I am a bit confused :
1. will my current configuration will run proxmox + FreeNAS + pfsense?
2. is the migration of my current freenas config to virtualized FreeNAS feasible?
3. is it better to keep my actual FreeNAS as FreeNAS and add a pfsense on a "wreck"?

My current configuration is a HP N54L with 16GB ECC, 3x2TB WD Red in RaidZ, and a 8GB USB stick. Additionally, I have an IBM M1015 (not flashed), a 60GB SSD, an intel NIC, and an old noisy m-itx atom mobo.
My idea is to: dump the FreeNAS configuration; remove the USB stick and HDDs; install intel NIC; install the SSD and proxmox (or ESXi) onto it; create the pfsense VM and provide access to WAN and LAN; create the FreeNAS VM and provide access to LAN; mount the HDDs and provide them direct access from FreeNAS VM; restoring FreeNAS configuration;
If anything goes wrong, then restore USB stick :)
Do you think it could work?

Additionally, while FreeNAS requires 8Gb of RAM, + 1Gb * 1Tb in ZFS (i.e. 14Gb), pfSense requries 1Gb of RAM, and 8Gb to ProxMox, my N54L has 16Gb. Should I move on another configuration (e.g. DELL T130 with 32Gb and a core I3)? Or should I keep my N54L and use the oldy noisy atom as pfSense machine?

All the best,
Juspolo

P.S. I have 2 users.

 

[Dice]

3. yes.

 

[Stux]

Your ram formula is not necessarily right btw. It's taking a general rule of thumb and making it too specific.

Virtualising FreeNAS on proxmox is off the beaten path. It might work... who knows?

And pfsense, which runs on any low end thing? Maybe you should go with 3) and fix its noisyness

 

[RodyMcAmp]

I have an all in one for my server, similar to yours but i do VMware esxi+freenas+pfsense + a bunch of other stuff. It works great. but make sure you do your homework. pretty good guide https://b3n.org/freenas-9-3-on-vmware-esxi-6-0-guide/

 

[brando56894]

You need a lot more if you're going virtualize it all on ESXi, I had ESXi running on my server (in my sig) and it was definitely slower than how it's running now off of FreeNAS 10. I'd suggest keeping them in physical machines.

 

[juspolo]

Thanks to everyone ;-)

Hence, the majority (@Dice, @Stux, and @brando56894 ) is voting to keep the FreeNAS machine as it is and put on the atom one with pfsense. I'll double check if pfsense supports ION giganet and if so, I'll buy an intel pro 1000 PCI to add the second NIC. Meanwhile, I'll change the USB stick from freenas with the SSD.

@RodyMcAmp great guide! What platform do you have?

 

[RodyMcAmp]

Xeon 1245v2 on an Asus workstation board, p9b or p9c, 32 gib ram, and an lsi 2008 ibm 1015, I have been happy with it.

Sent from my SAMSUNG-SM-N910A using Tapatalk

 

[juspolo]

Xeon 1245v2 on an Asus workstation board, p9b or p9c, 32 gib ram, and an lsi 2008 ibm 1015, I have been happy with it.

Sent from my SAMSUNG-SM-N910A using Tapatalk

Nice setup ;) thanks. It is more similar to @brando56894 than mine.
By yours setups, I believe that if I want to virtualize then, my N54L wouldn't have enough juice. While I do not believe that the CPU would be limiting for my use, I believe more that the limit is the 16GB of ram : \. Also, if @Stux highlights that the requirements are not stricht.

 

[brando56894]

Yea you definitely need more RAM, 32 GB is probably the lowest you could safely go, or 24 GB if you don't want to run in dual channel.

Your CPU could probably handle a VM or 2 but I wouldn't go much above that for fear of starving the host OS of resources.

 

[RodyMcAmp]

Honestly if your lan is not super busy you could dedicate 8 gb to freenas and 1-2gb pfsense and be ok. If you will be running jails bump that freenas ram up to 10-12. This would not be the best performance but it would work and a great learning experience to set up.

Sent from my SAMSUNG-SM-N910A using Tapatalk

 

[juspolo]

Honestly if your lan is not super busy you could dedicate 8 gb to freenas and 1-2gb pfsense and be ok. If you will be running jails bump that freenas ram up to 10-12. This would not be the best performance but it would work and a great learning experience to set up.

Sent from my SAMSUNG-SM-N910A using Tapatalk

I agree! Probably, as @brando56894 highlighted, I will need more RAM. However, I am planning to give it a try for the sake of "discovery". Hopefully, without losing any data :D
Additionally, as I do not run any jails so far and my nas is not so busy, I might succeed!

For the "experiment", I planned to give 1gb of ram to pfsense, at least 12gb to freenas, and migrate to ssd.

If the experiment seems stable and alright, then I'll move on that platform, but probably I'll flash and mount the IBM 1015 as HBA in order (i.e. avoid of using the ODD sata/eSata of N54l), and add another ssd to mirror proxmox with VMs on 2 SSDs.

I also ordered an intel pro gt NIC for the pfsense machine, as I found a cheap used one.

I'll keep you updated!

 

[juspolo]

Honestly if your lan is not super busy you could dedicate 8 gb to FreeNAS and 1-2gb pfsense and be ok. If you will be running jails bump that FreeNAS ram up to 10-12. This would not be the best performance but it would work and a great learning experience to set up.

Sent from my SAMSUNG-SM-N910A using Tapatalk

Finally, everything is up & running! ;)
Partially following this guide, I ended up by passing the hdd by id.

Proxmox is installed in a SSD with a pfSense VM and a FreeNAS VM (i.e. FN VM). Both VMs exploit host CPU as "host CPU", pfSense has 1gb of RAM, while FN has 8-12gb. LANs are used as VirtIO, which should maximize the performances (link) .
Both VMs load the same configuration of their physical machine. Thanks to that, FN VM recognized the drives and works like "nothing" changed.

The only issue that I am experiencing and that I cannot figure it out is that while with the physical machine I achieve 90/100 MB/s (i.e. download/upload), on the VM it is 105/60 MB/s. Any idea?

 

[RodyMcAmp]

I have not done extensive testing, between a physical machine and a virtual machine, but it is kind of amusing that it has flip floped around. I wonder if some of the upload slow down is tied to having pfsense on the same server. Should be easy to test i think, turn off the pfsense vm and use a physical router to test. This is partially why I am interested in using FreeNas itself for the hypervisor.

 

[juspolo]

I have not done extensive testing, between a physical machine and a virtual machine, but it is kind of amusing that it has flip floped around.

Definitely!

I wonder if some of the upload slow down is tied to having pfsense on the same server. Should be easy to test i think, turn off the pfsense vm and use a physical router to test. This is partially why I am interested in using FreeNAS itself for the hypervisor.

What do you mean that the upload is slowd down by pfsense?
Btw, I tested smb performance on a gbe port that was used only for freenas and proxmox web interface.
I'll try to move out the proxmox web gui from that port as well to see if anything changes. However, I still can not understand why the download improved