bhyve virtio boot disk in Windows?

[yottabit]

I have managed to get Windows 10 stable in bhyve by using an AHCI boot disk and virtio NIC (instead of the e1000 NIC which causes instability under load).

However, the disk performance using AHCI mode is abysmal. Because Windows 10 does a lot of stuff behind your back, and it's hard to disable everything, there are random periods where the disk I/O goes to 100% and the VM slows to a crawl. Using the virtio disk driver should dramatically speed up the disk I/O since bhyve won't have any slow emulation to handle.

I have had success loading the virtio driver pack and adding a *secondary* disk using virtio mode, but whatever I do, I cannot get the boot disk to use virtio.

If I change the existing boot disk to virtio, regardless of having a secondary disk using the virtio driver already loaded and working, Windows won't boot and the repair options don't help.

If I try a clean install with slipstreamed drivers into the ISO, it doesn't matter and Windows doesn't see the virtio disk.

If I try a clean install with the ISO and load the virtio drivers in the disk selection dialog, the VM crashes when the virtio disk driver is loaded (the other common ones load fine, like NIC).

Has anyone had any success getting this to work? I'm desperate!

I really, really, really don't want to go down the long and complicated path of installing Windows to an iSCSI target.

 

[yottabit]

I saw in another post, a link to a FreeBSD bug that seems to make the VirtIO disk driver crash the Windows guest: https://reviews.freebsd.org/D10581

I guess I'll start looking into making Windows install to an iSCSI target. Either that or install KVM in a Linux VM, and install Windows into KVM. Sigh.

 

[yottabit]

Making progress. The process to install Windows onto an iSCSI target was beyond daunting, so I opted for option B, to install KVM in a Linux VM, and then install Windows into KVM. :-/

Installing the latest net-inst image of Debian Linux was a breeze, using all virtio (NIC & disk). (Hint: select XFCE if you want a GUI for managing the VM; it's a very lightweight non-compositing window manager perfectly suitable for this purpose.)

Next I added a new zvol virtio disk to the Debian Linux VM, as well as attaching my Windows 10 install ISO as a CD-ROM, and the virtio driver pack ISO as a CD-ROM.

Back into Debian, edit the network config to create a bridge network interface. Restart networking.

Run virt-manager, create new VM, get through the initial wizard and then go back into the config details for tweaking:

• I set up 1 CPU w/ 16 cores so that Windows 10 would see 16 cores to use instead of only 2 CPU packages
• Attach both CD-ROMs for the Windows 10 ISO and the virtio driver package ISO (note that I attempted to slipstream the virtio drivers into the Windows 10 ISO and it didn't make a difference)
• Attach the disk as virtio raw, and use /dev/disk/by-id/<whatever> to select the secondary disk I added as a new zvol to hold Windows
• Attach the network as virtio
• Enable the boot menu

Now I just started the VM, selected the first CD-ROM as the boot drive, and Windows installation started.

The only part left is to load the virtio Windows drivers when you get to the dialog where it says no disk can be found for installation. Click the Load Driver link, navigate to your CD-ROM with the virtio Windows drivers, and load the following in order (this order is mentioned on many other posts in various forums):

• balloon
• kvmnet
• vioserial
• viostor

You may need to uncheck the "Hide hardware that isn't compatible" box to show the driver in one or more of the drivers above.

After the last one, you'll now see the disk. I've selected it and Windows is currently installing to it now.

I'll edit this post if I happen upon any other issues during the install, and then later I'll post a follow-up reply with disk benchmark comparisons. I'm hopeful the disk I/O is faster even with this ridiculous two-layered VM. :)

 

[yottabit]

Yeah, it didn't work out. The installation progresses quite a while, albeit very slowly, and eventually the Windows installer presents a typically useless error dialog without anything actionable, and stops.

Well. I guess I'm stuck just waiting for the FreeBSD and/or virtio fix.

 

[nanopete]

Looks like the bug is expected handled with 11.2, so we will have to wait a while.
https://redmine.ixsystems.com/issues/23457

Would be great if anyone had some workaround.

 

[GBillR]

After struggling all afternoon with trying to get the virtio drivers to load during the install process with Win 7 or Win 10, I happened to stumble upon this post.

I assume no one has managed to get the virtio drivers functioning in a Windows VM?

The performance of the AHCI drivers is horrible with windows. Even at idle, gstat shows significant and varying disk use up to 100% at times. I am wondering if anyone has seen anything different?

 

[yottabit]

After struggling all afternoon with trying to get the virtio drivers to load during the install process with Win 7 or Win 10, I happened to stumble upon this post.

I assume no one has managed to get the virtio drivers functioning in a Windows VM?

The performance of the AHCI drivers is horrible with windows. Even at idle, gstat shows significant and varying disk use up to 100% at times. I am wondering if anyone has seen anything different?

It's broken. It needs fixed upstream in FreeBSD first.

Sent from my Pixel XL using Tapatalk

 

[raidflex]

So basically Windows 10 support is broken if disk performance is poor. Pretty bad if this is the case since that bug you linked is 7 months old and still not fixed.

Sent from my Pixel 2 XL using Tapatalk

 

[GBillR]

Does anyone happen to know if there is an update on this issue? The bug looks open, but the underlying BSD issue is "abandoned"... not sure what that means for this problem.

Perhaps others have successfully found a work-around for this?

 

[alexandru.ulici]

I had the same situation on a FreeBSD 12.0 while trying to use a virtio-blk disk for a Win10 VM.
As in https://reviews.freebsd.org/D10581 , I modified into /usr/ports/usr.sbin/bhyve/block_if.h

the line
#define BLOCKIF_IOV_MAX 33
into
#define BLOCKIF_IOV_MAX 128

compiled & installed bhyve from ports.
Now it looks better, the Windows 10 VM did not crash while installing.

 

[yottabit]

Well, it's been a year now since I first reported this. The bug has now been open for 1.5 years.

I'm going to have to approach this in one of two ways:

1. Install ESXi in bhyve, and then Windows in ESXi. Hopefully this works, and performance isn't severely degraded.

2. Install ESXi on the bare metal, and then install FreeNAS and Windows both as VMs.

For now I'm running Windows in Google Compute Engine to suit my needs, but that's expensive and the latency in my application is a bit annoying.

 

[Robertr]

Any news?
Did the ESXi in bhyve and then Windows on top work out?
I'm looking to run Windows Server 2016 and it keeps crashing with the virtio driver.
I would REALLY like to avoid setting up another hardware box.

 

[yottabit]

Nope, bhyve doesn't support nested pages, required to virtualize a hypervisor.

The only solution for now is to run the awful AHCI emulation on the Windows host instead of the virtio driver, or to run ESXi on the bare metal and virtualize FreeNAS using I/O passthru for the storage.

Personally, I've given up on Windows in bhyve, and haven't bothered virtualizing FreeNAS, preferring to use preemptible VM instances in Google Cloud for now. It's cheap for my purposes.

 

[blanchet]

There is a patch for this issue. Unfortunately, it was merged only on May 2, 2019, that means it will reach FreeBSD only for the 12.1 release.

 

[diskdiddler]

Based on the contents of this thread, would I be right in thinking Windows 10 under FreeNAS is still not viable, even in 2020?

 

[blanchet]

iXsystems has back ported the bhyve patch to FreeNAS 11.3.
I have checked with

• FreeNAS-11.3-RC2
• virtio-win-0.1.164.iso
• Windows Server 2019 Evaluation
• virtio disk and virtio network

It works very well, and it is faster than before.
Therefore I think that Windows on FreeNAS bhyve is now a viable solution.

 

[artlessknave]

now if only I could figure out how to get windows to change from ahci to virtIO boot disk...

 

[diskdiddler]

I think I may have found some kind of bug, I can't be certain yet, but having a hell of a time getting this working with a brand brand new windows 10 iso.
I think it's 1909.

It worked fine with an older iso and older virtio driver, but seems to blue screen wth the new stuff.

 

[Dan Tudora]

I think I may have found some kind of bug, I can't be certain yet, but having a hell of a time getting this working with a brand brand new windows 10 iso.
I think it's 1909.

It worked fine with an older iso and older virtio driver, but seems to blue screen wth the new stuff.

hello
FN 11.3 RC2
I have the same problem with win10 1909
I install with 1809 and virtio stor and network and work(install with win10 iso on 1st CDROM and latest virtio iso in 2nd CDROM device)
is not so fast but working
regards

 

[WQian6]

How about using nvme as backend? bhyve on FreeBSD 12.1 can use nvme as controller.