SOLVED Cannot install TrueNAS Scale

[GChuck]

Why oh why cannot I not get TrueNAS Scale 22.12.0 installed on my old test system. Let me explain!

I have two systems running TrueNAS Core 12.0.U8.1. The first is my "production" server which is a true server (ASUS P11-C/M4L motherboard, Xeon E-2236 CPU, 64GB ECC memory, 4x4TB Seagate IronWolf drives) and the second is my backup/test server which is not a true server (Intel consumer motherboard, Core2-Duo cpu and 8GB memory, 3x3TB Seagate IronWolf drives). I mostly use this second server as a data backup of my primary server by booting it once a week, then synchronizing all the data files from the primary server to the secondary server. Once synchronization has complete, the system is then shutdown for another week.

However I also use it to test updates before updating the primary server.

Enough said about that. Now what I want to do is test TrueNAS Scale. So I download the ISO and using Rufus, copied it to a 16GB USB stick using an MBR partition scheme and a BIOS or UEFI Target System mode. I've tried writing using both ISO mode and DD mode. ISO mode boots where as DD mode does NOT boot!!

So next, I try booting the USB. As long as I have written the USB in ISO mode, the USB boots fine. I then go through the TrueNAS Scale questions (select device where to install, Admin password etc) and everything appears to go well. Finally after the install has been complete and I'm asked to remove the boot media and reboot the system, I do so, but the system never, ever boots. I'm always left with the message "No bootable device -- insert boot disk and press any key". Nothing I can do will get this computer to boot.

I've even gone so far as to create a bootable DVD of TrueNAS Scale, with exactly the same results as above. The DVD boots, I answer the questions, the installation appears to work correctly, but when it's time to boot the new server, it fails with the "No bootable device" message.

I have tried installing TrueNAS Scale as an upgrade to my TrueNAS Core system by first upgrading to version the Core system to 12.0-U3.1 and then when that completed (without issue), selecting the Scale 22.12.0 train, and then upgrading again. Again that appeared to work. This time at least the system was able to boot, but then the screen went blank (I left it for an hour) and never came back!

Granted, for what I use TrueNAS for, version 12.0-U8.1 certainly satisfies my needs (file server, media server and backup server). But my Xeon server sits idle for most of the time and I would like to use the virtualization features offered in Scale as they appear to be much more usable than the bhyv in Core. I have no doubt that my production server will upgrade without a problem, but considering all the problems I have had trying to upgrade my dev server, I don't dare even try upgrading my prod server for fear it will fail miserably.

I've had similar problems with Core installs after version 11.1 and was forced to use the bootable DVD method to get it installed. But at least I was able to get it working.

It seems that the TrueNAS developers don't really have a clue about how to create bootable media for their product. I got numerous Linux, FreeBSD and Windows ISOs that never have a problem booting. In fact I took the Debian 10.9 and 11.4 ISO distributions, created a bootable USB, then booted and installed onto the same system without issue. What gives??

If anybody can point me in the right direction to get TrueNAS Scale installed on my old Core2-Duo system I would appreciate it.

Thank you

Greg

 

[morganL]

Can you add your test system specs.... what is the boot device?

 

[GChuck]

My test system is a 16 year old desktop composed of an Intel DQ35JO motherboard with an Intel Core 2 Duo E8400 3.00GHz processor, 8 GB of DDR2 memory, 250 GB Western Digital SATA SSD drive for boot and either 3 x 3TB Seagate IronWolf or 4 x 4TB Seagate IronWolf spinners. The hard drive configuration changes depending upon what I'm working on. The drives are either in a raidz-1 or raidz-2 configuration. This whole system is then powered by a 850 Watt Corsair power supply.

When I first got this machine, it was to run Microsoft's Home Server and that it did for 10 years until Microsoft quit supporting the server software. I could have kept running it (without timely updates) but I decide to look for something to replace Home Server with. That lead me to FreeNAS back around 2017. All I need it for was is to act as a file server, media server and backup server for my home network. I originally had two of these boxes and they ran FreeNAS and eventually TrueNAS just fine. But everybody kept telling me that I need a 'real' server and ECC memory or I would eventually loose all my data so in 2020 I replaced one DQ35JO server with an ASUS P11C-M/4L mother board, Intel Xeon E-2236 CPU, 64GB of DDR4 2666 ECC memory, two 256GB Adata NVMe boot drives (mirrored) and 4 x 4TB Seagate IronWolf spinners (raidz-2). This machine runs 24x7 and has done so since early 2020.

Once a week on Saturday morning, I boot up the spare system and run a rsync job to synchronize the files between the two systems, then leave it running until Sunday morning. This allows the automatic scrub task to run. It runs every 5 weeks and usually takes 3.5 hours to run. Then shut down the backup system till next Saturday.

On occasion, I will use the backup system to test upgrades and/or updates before trying on the production system. That is what I'm trying to do now with absolutely no success.

As an aside, after FreeNAS changed from version 11.1 to 11.2 the ability to boot and install from a USB stick seemed to no longer work. I posted a note about this a few years back. However, I was still able to upgrade by using a bootable CD/DVD ROM.

Greg ...

 

[morganL]

Old motherboard.. does the BIOS support UEFI Boot?

Can you check BIOS settings.

Perhaps document the steps you precisely took to boot.. there are several optons: https://www.truenas.com/docs/scale/gettingstarted/install/installingscale/

SOLVED - Machine won’t boot after fresh install

Hello, Already using Core went for a try work scale but after install it, it won’t boot. It freezes randomly in different stages of the system boot. Already tried to only have the boot ssd on, but even though it will not boot. I am connected to a monitor using the nvidia quadro, should I try...

www.truenas.com

 

[GChuck]

This motherboard does NOT support UEFI boot.

The exact steps I took to boot this are simple. I created both a TrueNAS Scale bootable USB stick and a bootable DVD disc. Inserted either the disc or the USB stick and rebooted the computer. Computer restarted and eventually displayed the standard TrueNAS Scale installation display. I selected a fresh install, entered required Admin passwords and any other questions the installation script requested. Finally was asked if I wanted to install GRUB and said ok and then rebooted. Got the now infamous "No bootable device" message.

I'm not a Linux expert but can find my way around mostly. Anyway, I next created a bootable USB of Ubuntu and booted it on the same unbootable system. I wanted to use the Ubuntu Live mode to look at the SSD. When I looked at the SSD where Scale had installed the system I saw something strange. The first thing I noticed was that the partitioning scheme said it was GUID! Next, there were five partitions showing on the disk; first partition said 2.1mb and it was free space, the second partition was 1mb and was defined as BIOS boot (Legacy BIOS bootable), the third partition was 537MB and defined as EFI System, the fourth partition was 17GB and defined as Linux Swap and finally, the last partition 232GB and defined as ZFS.

To me those partitions were wrong and unbootable on my non UEFI system. On a vanilla install of Debian Linux 11.4 on the same hardware, I end up with a partitioning scheme which says it is Master Boot Record and I end up with one large partition for Linux and it is bootable with an additional 1GB swap partition.

So in my uneducated opinion, TrueNAS Scale does not have a clue about how to partition and boot a non-UEFI system even though the documentation indicates that it should run on a MBR computer. If TrueNAS Scale has been designed to run only on systems that support UEFI boot, then that is OK, but tell us in the documentation that older MBR computers will not be supported.

Greg

 

[morganL]

This motherboard does NOT support UEFI boot.

The exact steps I took to boot this are simple. I created both a TrueNAS Scale bootable USB stick and a bootable DVD disc. Inserted either the disc or the USB stick and rebooted the computer. Computer restarted and eventually displayed the standard TrueNAS Scale installation display. I selected a fresh install, entered required Admin passwords and any other questions the installation script requested. Finally was asked if I wanted to install GRUB and said ok and then rebooted. Got the now infamous "No bootable device" message.

I'm not a Linux expert but can find my way around mostly. Anyway, I next created a bootable USB of Ubuntu and booted it on the same unbootable system. I wanted to use the Ubuntu Live mode to look at the SSD. When I looked at the SSD where Scale had installed the system I saw something strange. The first thing I noticed was that the partitioning scheme said it was GUID! Next, there were five partitions showing on the disk; first partition said 2.1mb and it was free space, the second partition was 1mb and was defined as BIOS boot (Legacy BIOS bootable), the third partition was 537MB and defined as EFI System, the fourth partition was 17GB and defined as Linux Swap and finally, the last partition 232GB and defined as ZFS.

To me those partitions were wrong and unbootable on my non UEFI system. On a vanilla install of Debian Linux 11.4 on the same hardware, I end up with a partitioning scheme which says it is Master Boot Record and I end up with one large partition for Linux and it is bootable with an additional 1GB swap partition.

So in my uneducated opinion, TrueNAS Scale does not have a clue about how to partition and boot a non-UEFI system even though the documentation indicates that it should run on a MBR computer. If TrueNAS Scale has been designed to run only on systems that support UEFI boot, then that is OK, but tell us in the documentation that older MBR computers will not be supported.

Greg

Thanks for the write-up... there's clearly an incompatibility between your BIOS and the boot drive format. If no-one can resolve, lets report it as a bug. Most of the systems and testing are now UEFI, so this could be an escape. Could you test with Angelfish to see if its a new bug?

 

[GChuck]

I can't really see the point! As I mentioned above, this boot problem is not entirely new. It really started after the release of FreeNAS 11.2. FreeNAS 11.1-RELEASE worked without a problem, but when FreeNAS 11.2 was released, I could not get it to boot from a USB drive so couldn't install it. I eventually found a work-around by writing the ISO to a DVD disc and booting from that. So, as you can see, boot problems are not new with Free/TrueNAS. What's really funny is that I can install Windows 10 or Windows Server (pick your version) on this same machine without the slightest of problems.

I will try to install TrueNAS Scale 22.-2.1 (Angelfish??) tomorrow and see what happens.

Thanks for your help.

Greg

 

[GChuck]

Ok, I just tried to do an install using the TrueNAS 22.02.1 version with exactly the same results as the 22.12.0 version.

One of the things I noticed during the install of both versions was a message on the screen talking about loading GPT partitions into memory. Not sure what that means or even if it is significant.

Greg

 

[morganL]

Ok, I just tried to do an install using the TrueNAS 22.02.1 version with exactly the same results as the 22.12.0 version.

One of the things I noticed during the install of both versions was a message on the screen talking about loading GPT partitions into memory. Not sure what that means or even if it is significant.

Greg

Screenshot?

 

[morganL]

There's a few other threads on the issue. SCALE/Linux does use a different boot process (GRUB).
It is sensitive to AHCI and any formatting already on the drive.

TrueNAS Scale and Legacy BIOS

I have an HP Microserver Gen 8 that I want to try out TrueNAS SCALE on, however this only supports the legacy BIOS/MBR boot. I have gone through and got the installation process running. I have been able to install to the drive, but then can’t boot it. I know that there is an issue with HP...

www.truenas.com

Scale install issue grub

I tried upgrading from an existing truenas install to Truenas scale and was unsuccessful and got this error (see attached image) Please help, in advance; thanks!

www.truenas.com

The AHCI issue is described here: https://en.wikipedia.org/wiki/Advanced_Host_Controller_Interface

 

[GChuck]

Thanks again for your help. I've looked at all three of the links you have shown, and while they looked hopeful, none was able to get my problem fixed.

As for the screenshot. I'm running this on an old desktop and don't have anyway to capture the screen as it's progressing, short of me taking an actual photograph (which I did). So rather than uploading a photo, I have transcribed it exactly as shown below:

Warning: Partition table header claims that the size of partition table​

entries is 0 bytes, but this program supports only 128-byte entries.​

Adjusting accordingly, but partition table may be garbage.​

Warning: Partition table header claims that the size of partition table​

entries is 0 bytes, but this program supports only 128-byte entries.​

Adjusting accordingly, but partition table may be garbage.​

Creating new GPT entries in memory.​

The operation has completed successfully.​

The operation has completed successfully.​

The operation has completed successfully.​

The operation has completed successfully.​

umount: /mnt: not mounted.​

[0%] Creating dataset​

[0%] Extracting​

[7%] Extracting​

I again booted up an Ubuntu Live system on the computer and zeroed out the SSD I have been trying to install TrueNAS on (using dd) because some people have said that if the drive has remnants of old partition table, it might not get zeroed correctly. Also, while in the Ubuntu Live system, I created a new boot USB using the dd method, but as when creating the boot media with Rufus using dd mode, the boot USB would not boot on my system. I have also tried changing the disk mode from AHCI to IDE, but that also fixed nothing.

I have booted 8 different Linux derivatives (CentOS, Debian, Elementary, Fedora, Mint, OpenSUSE, Ubuntu and Zorin) on this system without issue.
I have booted FreeBSD 11, 12 and 13 on this system without issue.
I have booted Windows Vista, 7, 10 and even 11 on this system without issue.
I have booted Windows Server 2008, 2012, 2016 and 2019 on this system without issue.
I have booted Windows Home Server on this system without issue.
I have booted Proxmox on this system without issue.
I have booted XCP-ng on this system without issue.

So why can I not boot TrueNAS Scale? With the exception of most of the Linux's listed above, all of the ISOs can be written to a USB using stick using Rufus to support either a UEFI based system or a MBR based system. From what I have been able to glean, most Linux distros still do not support UEFI booting, but I may be wrong.

Just for some background, I'm a 75 year old ex Tech Support person who retired after 45 years in the computing biz in 2005. I started on IBM 1620s and Honeywell 200s in 1962, then IBM 360/40s before transitioning to Digital PDP 8s and 11s and finally the VAXen family. I primarily did operating system support. I'm telling you this just so that you might understand that I do a certain degree of knowledge regarding computers. It might not be totally current, but it an understanding!

That's why I keep blaming this problem I'm having on the developers of TrueNAS Scale. Theirs is the only system I've worked with, that I've had this much trouble with. If TrueNAS Scale is not designed to boot on MBR systems, then say so and I'll give it up and find an alternative. But from reading the documentation, it is supposed to be bootable on both UEFI and MBR systems. Ok, down off soapbox now!

Thanks again for all your help. I might try building from sources and see where that takes me. This is just a hobby for me and have the time to waste. If you need anything else from me, please don't hesitate to ask.

Greg

 

[morganL]

Thanks again for your help. I've looked at all three of the links you have shown, and while they looked hopeful, none was able to get my problem fixed.

As for the screenshot. I'm running this on an old desktop and don't have anyway to capture the screen as it's progressing, short of me taking an actual photograph (which I did). So rather than uploading a photo, I have transcribed it exactly as shown below:

Warning: Partition table header claims that the size of partition table​

entries is 0 bytes, but this program supports only 128-byte entries.​

Adjusting accordingly, but partition table may be garbage.​

Warning: Partition table header claims that the size of partition table​

entries is 0 bytes, but this program supports only 128-byte entries.​

Adjusting accordingly, but partition table may be garbage.​

Creating new GPT entries in memory.​

The operation has completed successfully.​

The operation has completed successfully.​

The operation has completed successfully.​

The operation has completed successfully.​

umount: /mnt: not mounted.​

[0%] Creating dataset​

[0%] Extracting​

[7%] Extracting​

I again booted up an Ubuntu Live system on the computer and zeroed out the SSD I have been trying to install TrueNAS on (using dd) because some people have said that if the drive has remnants of old partition table, it might not get zeroed correctly. Also, while in the Ubuntu Live system, I created a new boot USB using the dd method, but as when creating the boot media with Rufus using dd mode, the boot USB would not boot on my system. I have also tried changing the disk mode from AHCI to IDE, but that also fixed nothing.

I have booted 8 different Linux derivatives (CentOS, Debian, Elementary, Fedora, Mint, OpenSUSE, Ubuntu and Zorin) on this system without issue.
I have booted FreeBSD 11, 12 and 13 on this system without issue.
I have booted Windows Vista, 7, 10 and even 11 on this system without issue.
I have booted Windows Server 2008, 2012, 2016 and 2019 on this system without issue.
I have booted Windows Home Server on this system without issue.
I have booted Proxmox on this system without issue.
I have booted XCP-ng on this system without issue.

So why can I not boot TrueNAS Scale? With the exception of most of the Linux's listed above, all of the ISOs can be written to a USB using stick using Rufus to support either a UEFI based system or a MBR based system. From what I have been able to glean, most Linux distros still do not support UEFI booting, but I may be wrong.

Just for some background, I'm a 75 year old ex Tech Support person who retired after 45 years in the computing biz in 2005. I started on IBM 1620s and Honeywell 200s in 1962, then IBM 360/40s before transitioning to Digital PDP 8s and 11s and finally the VAXen family. I primarily did operating system support. I'm telling you this just so that you might understand that I do a certain degree of knowledge regarding computers. It might not be totally current, but it an understanding!

That's why I keep blaming this problem I'm having on the developers of TrueNAS Scale. Theirs is the only system I've worked with, that I've had this much trouble with. If TrueNAS Scale is not designed to boot on MBR systems, then say so and I'll give it up and find an alternative. But from reading the documentation, it is supposed to be bootable on both UEFI and MBR systems. Ok, down off soapbox now!

Thanks again for all your help. I might try building from sources and see where that takes me. This is just a hobby for me and have the time to waste. If you need anything else from me, please don't hesitate to ask.

Greg

Thanks for the history and explanation. My background is more networking and storage, so I'm engaging because I want to learn the issues, not because I'm the expert you might need.

There's clearly something going wrong, but we're both struggling to work it out. As the other links indicated, legacy BIOS boot is supported. GPT and MBR were made to be compatible. "classic MBR" is not the format of the boot device.

From your error message, it certainly seems like the formatting of the drive is the issue. The best article I could find on MBR/GPT indicated that there are BIOS versions that could not boot from GPT formatted drives.

IBM Developer

IBM Developer is your one-stop location for getting hands-on training and learning in-demand skills on relevant technologies such as generative AI, data science, AI, and open source.

developer.ibm.com

I haven't used it myself, but did you try the ventoy method rather than rufus... it solved problems for one user.

The other solution is to install TrueNAS CORE 13.0-U3.1 and then use the "manual update file method" to migrate from CORE to SCALE.

Migrating TrueNAS CORE to SCALE

Provides instructions on migrating from TrueNAS CORE to SCALE. Migration methods include using an ISO or manual update file.

www.truenas.com

 

[morganL]

Found this similar warning message here:

Reddit - Dive into anything

www.reddit.com

So, the 250GB WD SATA SSD boot drive.... how old is it? How is it connected?

 

[GChuck]

Morgan, as I mentioned previously, after FreeNAS went from 11.1-RELEASE to 11.2, the ISO would not boot from a USB stick (I tried everything I could think of at that time), so I installed a DVD drive in the box, wrote the ISO to a DVD disc and it then booted without problem. So whatever change the Free/TrueNAS team made to their ISO format happened sometime back in 2018.

I had used Ventoy before and it did solve that problem. Using Ventoy, I was able to boot 12.0-U8.1 (my currently used release), 13.0-U3.1 and Scale 22.12.0. The 13.0-U3.1 release installed correctly, while the Scale 22.12.0 release gave me the same problems as before. Using Ventoy, 22.12.0 offered me 2 ways to boot; Normal and Grub2. I tried both with the same non-bootable results.

Before posting my original message, I had also tried the upgrade from Core 13.0-U3.1 to Scale 22.12.0, again with the same non-bootable results even though the upgrade appeared to work correctly!

But good news! I just tried using Ventoy to install Core 13.0-U3.1 onto my system. Then going immediately to the update/upgrade screen told Core to change trains to Scale 22.12.0 and performed the upgrade. This time it eventually booted into the new Scale 22.12.0 system. Hooray! I now have a TrueNAS Scale system I can test on. Not sure how stable it is considering all that problems so far, but at least I can try it.

One thing I did a quick check on was the partition layout of the boot drive. After doing the upgrade method of installation, I ended up with 4 partitions on the drive. All the partitions were GUID partitions, BUT, the first partition was now defined as a 524KB BIOS boot partition, followed by a 17GB FreeBSD Swap partition, then a 233GB FreeBSD ZFS partiton and finally 11mb of free space.

I think that that 2.1MB free space partition before the BIOS partition might have been causing the boot problems. But not being a partitioning expert, I really don't know.

Anyway, I now appear to have a bootable system I can play with. Just one further question if you can answer it. The data pool I want to play with is a zraid-1 configuration and was created on my Core system. Can I import it into Scale without Scale changing it to the extent that it will no longer be usable in Core?

I really appreciate all your help. I will eventually replace this old box with something newer, but being on a pension, I have to watch all my pennies. Thanks again for all your help and have a safe, prosperous and happy 2023.

Greg

 

[Whattteva]

IBM Developer

IBM Developer is your one-stop location for getting hands-on training and learning in-demand skills on relevant technologies such as generative AI, data science, AI, and open source.

developer.ibm.com

I haven't used it myself, but did you try the ventoy method rather than rufus... it solved problems for one user.

The other solution is to install TrueNAS CORE 13.0-U3.1 and then use the "manual update file method" to migrate from CORE to SCALE.

Migrating TrueNAS CORE to SCALE

Provides instructions on migrating from TrueNAS CORE to SCALE. Migration methods include using an ISO or manual update file.

www.truenas.com

That's an interesting one. In my case, Rufus works, while Ventoy method boots but always hangs when it's unpacking the kernel.

 

[GChuck]

Rufus has always worked for me and that is why I still continue to use it. I had tried Ventoy before but never really felt "safe" with it. Anyway it did appear to work on this occasion where Rufus appears to have failed!

Greg

 

[morganL]

Rufus has always worked for me and that is why I still continue to use it. I had tried Ventoy before but never really felt "safe" with it. Anyway it did appear to work on this occasion where Rufus appears to have failed!

Greg

Its still a mystery, but good to have a datapoint... However, very glad your systems is working again.

My impression is that ventoy might be better at resolving media issues..... but I don't have great 1st hand knowledge.

 

[dAlexis]

Thanks 2 all, useful indeed. Was thinking, WTF is going on, because created boot stick by rufus and looking on endless ...hung_task_timeout... right now...

Downloaded ventoy, another attempt will be done. BTW, DVD is present in system.

Another questions
1. May I install on 2 USB sticks? 64G each. I marked them both during install, maybe wrongly.
2. CSM set in BIOS, but LSI is starting before, and as far, as I understood, UEFI is not working with LSI bios... May it cause problems?

BTW both usb sticks not visible after install attempt...

 

[Whattteva]

1. May I install on 2 USB sticks? 64G each. I marked them both during install, maybe wrongly.
2. CSM set in BIOS, but LSI is startiong before, and as far, as I understood, UEFI is not working with LSI bios...

1. I'd suggest using USB externals (can be SSD or HDD) instead. They're much more reliable than thumb drives.
2. My LSI card works fine with my UEFI server at least with P20 firmware.

 

[dAlexis]

1. I'd suggest using USB externals (can be SSD or HDD) instead. They're much more reliable than thumb drives.

Maybe stupid, but got used 2 this from FreeNas 9... Needed 2 rummage 4 ssd...

IMHO! The key is this

Adjusting accordingly, but partition table may be garbage.
Warning: Partition table header claims that the size of partition table
entries is 0 bytes, but this program supports only 128-byte entries.
Adjusting accordingly, but partition table may be garbage.
Creating new GPT entries in memory.

Something wrong in installer partition table initializarion. IMHO, needed 2 erase all and create new table from scratch, but installer is not doing this. BTW, Rufus install media boots OK, but the error above. I erased everything on sticks in win10 diskpart, but no success. Maybe, needed 2 init GPT partitions before installation? Notebook with ubuntu present...

And! May be important - rufus is too brainy, proposed 2 install a new grub wjile creating megia. May be the cause too