ESXi and FreeNAS - acceptable error?

[Jailer]

I've got new hardware coming in a couple weeks for my pfSense setup. Once that's up and running I'm going to use the current hardware for ESXi (not FreeNAS). I've been curious about learning it for some time and this will be the first time I'll have compatible hardware for it without having to purchase some.

 

[toyebox]

I've got new hardware coming in a couple weeks for my pfSense setup. Once that's up and running I'm going to use the current hardware for ESXi (not FreeNAS). I've been curious about learning it for some time and this will be the first time I'll have compatible hardware for it without having to purchase some.

This is my first time using it. I went with the new 6.5. It seems many long time ESXi goers dont like the new fully web based management. It works great for me, not knowing what it was like before. It really is a nice piece of software and I foresee a lasting relationship between me and it:D:D:p

 

[joeschmuck]

I'm not sure about those voltage readings, even if there is a lots decimal they still seem a bit odd. The first reading I would have expected to be something like 12.xx VDC. But they are all green. While it's not required, you might want to do a search on ESXi and your specific motherboard to see if the readings are as expected. Also, ensure you have the most current BIOS and BMC firmware.

I am very excited to push this thing to it's limits.

Well you have a great CPU and motherboard, you should be able to do a lot. The only thing I should caution you about is to ensure that if you use FreeNAS as a storage device for the VMs, ensure that you shutdown the VMs before shutting down FreeNAS. You can run into issues if you fail to do that. Yes, it's common sense until you forget and then you feel the pain.

 

[toyebox]

I'm not sure about those voltage readings, even if there is a lots decimal they still seem a bit odd. The first reading I would have expected to be something like 12.xx VDC. But they are all green. While it's not required, you might want to do a search on ESXi and your specific motherboard to see if the readings are as expected. Also, ensure you have the most current BIOS and BMC firmware.


Well you have a great CPU and motherboard, you should be able to do a lot. The only thing I should caution you about is to ensure that if you use FreeNAS as a storage device for the VMs, ensure that you shutdown the VMs before shutting down FreeNAS. You can run into issues if you fail to do that. Yes, it's common sense until you forget and then you feel the pain.

Yessir on both counts . as for using freenas as the storage device for the VMS.. I decided against it. I can just use a pair of 1tb drives to run testing VMS and such.

I will look into the voltages now.

 

[joeschmuck]

I've got new hardware coming in a couple weeks for my pfSense setup.

What would that be? I'd like to find some really low power setup that would run pfSense, but the key word is low power, preferably 10 watts or less (yes it's a pipe dream), or maybe 40 watts which would be more realistic. My current system pulls 80 watts and it's running ESXi, Sophos UTM and pfSense VMs. I'm sure I could shave off some of the watts by reducing the RAM and changing the hard drives for SSDs and under-clocking the motherboard, but would it be worth it? Probably not. Actually I just started using pfSense as of yesterday. My ISP provides two WAN IPs so I configured the second WAN IP to the pfSense VM. Now I'm trying to figure it out but I'm in no hurry. It's running for now using it's basic settings and I need to do a lot of reading to figure out more on what it can do and how to configure it. I'm using the second IP on my LAN using 192.168.2.0/24 and my primary LAN is 192.168.1.0/24. I have a few Windows VMs which access the 192.168.2.0/24 LAN so I'll be learning slowly.

 

[Jailer]

What would that be? I'd like to find some really low power setup that would run pfSense, but the key word is low power, preferably 10 watts or less (yes it's a pipe dream)

Not a pipe dream at all. It's a APU2C4 from pcengines. Pulls around 6 to 10 watts depending on load. Currently out of stock but expected back in about a week from now. My order is in just waiting on stock. It will handle around 500 Mb/s which is WAY faster than what my ISP provides so it will work great for me for quite some time.

Why did you give up on Sophos?

 

[toyebox]

What would that be? I'd like to find some really low power setup that would run pfSense, but the key word is low power, preferably 10 watts or less (yes it's a pipe dream), or maybe 40 watts which would be more realistic. My current system pulls 80 watts and it's running ESXi, Sophos UTM and pfSense VMs. I'm sure I could shave off some of the watts by reducing the RAM and changing the hard drives for SSDs and under-clocking the motherboard, but would it be worth it? Probably not. Actually I just started using pfSense as of yesterday. My ISP provides two WAN IPs so I configured the second WAN IP to the pfSense VM. Now I'm trying to figure it out but I'm in no hurry. It's running for now using it's basic settings and I need to do a lot of reading to figure out more on what it can do and how to configure it. I'm using the second IP on my LAN using 192.168.2.0/24 and my primary LAN is 192.168.1.0/24. I have a few Windows VMs which access the 192.168.2.0/24 LAN so I'll be learning slowly.

It's possible for sure.. But not if you ever plan on a 1gbE throughput while using any sort of packages. I use my 3770s board but it hardly pulls 30ish. I also run SNORT 24/7 along with a couple OpenVPN servers.

I verified that there are correct readings on the board, and that ESXi is just displaying them incorrectly. Here is directly from the IPMI sensors page.

 

[toyebox]

welp, there it is... the error is still there on the new system.

CAM status: SCSI Status Error
SCSI status: Busy
Retrying command

time to try a new drive i guess...

 

[joeschmuck]

Must be the PNY SSD.

 

[toyebox]

Must be the PNY SSD.

Already installing on a new disk (via ipmi I might add). Such an awesome feature. This is the first time dealing with ipmi compared to iAMT kvm. i am unsure which i like better, but both are fantastic!

 

[joeschmuck]

Not a pipe dream at all. It's a APU2C4 from pcengines. Pulls around 6 to 10 watts depending on load.

I checked a few places in the US and they are selling that board for $170. Ouch! For a few minutes I started getting excited. The case price was reasonable at least. I like the fact that people are installing ESXi on this device. I'm curious how powerful it is. Please toss me a message when you get it in and play with it, I'd like to know more. For myself, I would not use the WiFi, I'd just want the bare machine and later add a mSATA, but not right away, I may want to up the capacity to a larger size if I could do more with it. Yes, I'm drooling. Do you have any links for benchmarks or anything to state how fast this is?

 

[joeschmuck]

Why did you give up on Sophos?

I haven't yet, I'm just trying out pfSense. I like Sophos for the most part but it's not very easy to configure, it's actually a real pain in the butt. If pfSense works well and is easier to configure then odds are good I'll migrate over to it. If you're not a IT person who does this stuff for a living then you are the average home user who will hate Sophos because it's not intuitive.

 

[melloa]

If pfSense works well and is easier to configure

It is easy to configure. I'm running it with on two houses connected with IPSec, so if I can do it, it is easy.

 

[joeschmuck]

It is easy to configure. I'm running it with on two houses connected with IPSec, so if I can do it, it is easy.

Us old guys need to stick together.

 

[toyebox]

So i have a new box up and running.. again. i see a drastic improvement already. An example is, while having freenas running, i was trying to install windows in another VM.. This is where i started seeing the errors. Not only was i seeing the disconnection errors, but installing windows took ages. the "Expanding Windows Files" took about an hour which is insane. I now see it flying through those steps.

I really don't want to speak too soon.. I really hope it was just a crappy pny ssd that was the problem.

 

[joeschmuck]

What did you put in it's place? Another PNY SSD

 

[toyebox]

What did you put in it's place? Another PNY SSD

Ha, a funny guy i see!

Nope, I used a new in box 1TB seagate 7200rpm drive i have had in my closet for 2 years (i mean.. who really uses 1tb platters anymore?)
I will be putting a samsung 850 in there the next time i can get my hands on one.

As for that error while installing Freenas, where i had to disable trim.. I am not sure if that is gone or not. For troubleshooting purposes, i used the backup of the VM of freenas. didn't want to change too much configuration wise, that way i can point directly at the problem when i found it. Just for fun, i'll install freenas again in a fresh VM, seeing if i need to disable trim again.

 

[Jailer]

I checked a few places in the US and they are selling that board for $170.

Buy it direct from pcengines, there is a considerable markup elsewhere. I bought an entire working system for that price. $171.90 shipped to be exact. That's MB, 16GB mSATA drive, case, power adapter and serial cable for installation.

It's not a real powerful system but you don't need one for most connections unless you are trying to push line speed on a Gb connection. It has a passmark score of 1462 but it's more than enough for what I need. Most users report around 650 with iperf and around 480 to 550 NAT. I run snort but gain at my connection speed it won't be an issue for this to keep up.

And as I'm sure your aware, wireless support on FreeBSD is abysmal. If you want decent wireless get an AP. A Ubiquity AP AC Lite is next on my list of hardware purchases.

 

[toyebox]

Buy it direct from pcengines, there is a considerable markup elsewhere. I bought an entire working system for that price. $171.90 shipped to be exact. That's MB, 16GB mSATA drive, case, power adapter and serial cable for installation.

It's not a real powerful system but you don't need one for most connections unless you are trying to push line speed on a Gb connection. It has a passmark score of 1462 but it's more than enough for what I need. Most users report around 650 with iperf and around 480 to 550 NAT. I run snort but gain at my connection speed it won't be an issue for this to keep up.

And as I'm sure your aware, wireless support on FreeBSD is abysmal. If you want decent wireless get an AP. A Ubiquity AP AC Lite is next on my list of hardware purchases.

i have the Ubiquity AP pro in my apartment and the coverage and speed is great. have a 200/200 connection, and i get it all via wifi.

edit: as an update to this issue, still no more errors since drive change. Working on the windows VM now. just installed the drivers for the RX 480 in passthrough to do a bit of hardware encoding for my plex :D:D

 

[Jailer]

I'd love to see how long a Handbrake Blu Ray encode would take on that beast of yours. My desktop with a 6700k running at 4.5Ghz is pretty fast, I can't imagine how fast all those cores would be.