BUILD Install FreeNAS-11.1-U5 on QNAP TS-EC1680U-RP

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
Abstract:
QNAP TS-EC1680U-RP is a x86 NAS from QNAP that can easily run FreeNAS. The installation is done on a USB key so you can revert to the original QTS software at anytime. You need only 3 USB keys: one for the installer and two for the mirrored boot device.

Full Story:
I had an unused QNAP TS-EC1680U-RP in the computer room. The hardware is very decent, so I have tried to run FreeNAS on it.

Hardware:
  • QNAP TS-EC1680U-RP
  • 16 SATA bays for 3.5" or 2.5" hard disk
  • HBA: Marvell 88SE9215
  • Intel Xeon E3-1245v3 @ 3.40GHz
  • 32 GB DDR3 ECC
  • 4 x 1 GbE (Intel i210)
  • 2 x 10GbE SFP+ (Intel 82599ES)
  • 512MB boot device (ADATA IUM01-512MFHS)
  • Dual Power Supply
The only drawback of this NAS is the lack of IPMI for remote management.
The embedded boot device is too small (512MB) for FreeNAS. It is soldered on the motherboard, so you can no replace it.
Instead you need to use an external USB key to install FreeNAS. In my case, I use three KingstonDataTraveler 3.0 16GB.

Procedure to install FreeNAS
  • Create a FreeNAS USB installer key
  • Plug a USB keyboard and a HDMI monitor to the rear of the NAS
  • Plug the installer key in a USB slot at the rear of the NAS. You can use a USB2 or USB3 port, it does not matter.
  • Power on the NAS and press F2 to enter in the Insyde BIOS
Go to
Boot | Legacy

Change the boot order for the USB device to boot on the installer key instead of the ADATA IUM01-512MFHS that holds QTS. In means ADATA IUM01-512MFHS that must be at the bottom. With my Kingston USB key the screen looks like


KingstonDataTraveler 3.0
ADATA IUM01-512MFHS

Press F10 to save and exit.

The computer boots on the FreeNAS installer. At this time, insert two other USB keys in the USB3 ports. These USB keys will be used as mirrored boot device.

Select
1 Install/Upgrade

When FreeNAS installer asks for the boot device you want to use select your just-added USB keys (da2 and da3)
WARNING: never try to install on ADATA IUM01-512MFHS because it is too small (512MB) and it will kill QTS

At the end select
Boot via BIOS

At the installation end, the computer reboots.
Unplug the installer key, and press F2 to enter the BIOS again
Go to

Boot | Legacy

In my case I still use Kingston USB keys for the boot device, so the screen looks like.

KingstonDataTraveler 3.0
KingstonDataTraveler 3.0
ADATA IUM01-512MFHS

Press F10 to save and exit the BIOS

The computer will boot on the FreeNAS boot device and you can start to use it.
Everything works except the status LED for the hard disks. I do not know yet how to activate them in FreeNAS to locate a disk.

The BIOS settings is persistent. You do not have to setup the boot order after a power off.
So, if you wish to revert to QTS, just unplug the FreeNAS USB keys.

Conclusion
FreeNAS runs perfectly on the QNAP TS-EC1680U-RP.
I think that many other x86 NAS from QNAP (and Synology) can run FreeNAS, it they have enough memory and can boot from a USB key.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Hardware:
  • QNAP TS-EC1680U-RP
  • 16 SATA bays for 3.5" or 2.5" hard disk
  • HBA: Marvell 88SE9215
  • Intel Xeon E3-1245v3 @ 3.40GHz
  • 32 GB DDR3 ECC
  • 4 x 1 GbE (Intel i210)
  • 2 x 10GbE SFP+ (Intel 82599ES)
  • 512MB boot device (ADATA IUM01-512MFHS)
  • Dual Power Supply
If that is the model I think it is, I have four of them in my (at work) server room now. It is good to know. Thanks for the info.
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
Unfortunately, there is no option in the BIOS to force the fan speed (like on Supermicro motherboards), so the temperatures for the hard disks become too high (45C) in spite of the air cooling from the computer room. In such conditions, this server is not suitable for production.
I guess that QTS has its own software and interface to control the fan airflow, but is unsupported yet by FreeNAS.

(Note: I was wrong. Few days later, I have finally found a BIOS setting to force the fan speed. See instruction later in this thread)
 
Last edited:

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Unfortunately, there is no option in the BIOS to force the fan speed (like on Supermicro motherboards), so the temperatures for the hard disks become too high (45C) in spite of the air cooling from the computer room. In such conditions, this server is not suitable for production.
I guess that QTS has its own software and interface to control the fan airflow, but is unsupported yet by FreeNAS.
The ones I have are a slightly older model, they have the V2 processor, and the fans run full blast all the time.
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
Your remark was very helpful: it convinced me to browse again in the BIOS, and then I have finally found the setting to force the fans airflow.

On the QNAP TS-EC1680U-RP, the BIOS setting for the fans, is located in the menu Advanced | Hardware Monitor
For FreeNAS you need CPU Fan Smart Function: Disabled (the default for QTS is Enabled)

When Smart Fans are disabled, they run at their maximal speeds all the time, and then the hard disks temperatures go below 35C.
Indeed I realize that there is no CPU fan on this motherboard, there are only CHASSIS fans, that are connected as CPU fan. So in spite of its name this BIOS setting controls the CHASSIS fans.

Finally I may use this QNAP for production.
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
When running FreeNAS 11.1-u5 on the QNAP there is an issue with SMART and the USB devices.
So you have to disable SMART for each USB drives (da0, da1 and da2) in Storage | View Disks | device_name | Edit
After doing that, restart the SMART service. It should run normally.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
When running FreeNAS 11.1-u5 on the QNAP there is an issue with SMART and the USB devices.
So you have to disable SMART for each USB drives (da0, da1 and da2) in Storage | View Disks | device_name | Edit
After doing that, restart the SMART service. It should run normally.
That isn't just in the QNAP. I have always had problems with USB drives and the SMART service because USB drives usually don't work with SMART.
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
This a report after 5 months of running in production.

Operating conditions
  • The system runs 24/24H in a server room
  • There is a single vdev with 16x 4TB in raidz3.
  • The system is used as a ZFS replication target
  • It receives basically only sequential writings once per hour
Advantages
  • The system runs well: no hardware nor driver issue
  • the HBA (Marvell 88SE9215) runs without any issue
  • the chassis is very easy to service (the motherboard is on a tray)
  • FreeNAS is more interesting than with QTS
Drawbacks
  • no IPMI
  • the fan runs at full speed
  • The hard disk identification LEDs do not work (you have to label the tray with hard disk serial number instead)
Finally, it is decent platform to run FreeNAS.
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
I have tried to upgrade to FreeNAS-11.2u1 but it fails at the boot level with error "gptzfsboot error 1 lba xxxxxx"
I understand that FreeNAS-11.2 has a new boot loader (the FreeBSD boot loader) that does not support to boot on my
mirrored USB keys for this system.
So I have reinstalled FreeNAS-11.1u6.3 (that still uses Grub2 as boot loader) instead.
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
I have tried to upgrade to FreeNAS 11.1U7.

Whatever the installation method is (upgrade or fresh install) I have the same issue: the first boot works, but on the second boot, GRUB crashes and the computer reboot for ever.
It is not problem of the USB boot key it self because if I pull the key and plug it in my laptop, I can boot Freenas 11.1U7 normally.
I have also tryed to use a single USB boot key instead of a 2-mirror boot device but it does not change anything.

Therefore I think that there is something special with this QNAP computer and FreeNAS 11.1u7, so I have reinstalled FreeNAS-11.1u6.3 because it is the last version that works perfectly for me.
(on FreeNAS-11.1U6.3 I have also the message error: compression algorigthm inherit not supported but it does not prevent GRUB to start)
 

Attachments

  • freenas-11.1u7-grub-issue.gif
    freenas-11.1u7-grub-issue.gif
    430.5 KB · Views: 1,200
Last edited:

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
I have tried again with a single USB boot key in FreeNAS 11.1u7 and this time it works.
I have no problem anymore with GRUB. I have reboot 3 times, it always boot successfully.
 
Last edited:

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
Today I have tried to install FreeNAS on another QNAP model: TS-EC2480U-RP.
It is the same hardware than the QNAP TS-EC1680U-RP but with 24 disks instead of 16.

Like on the TS-EC1680U-RP, it is impossible to boot FreeNAS 11.2 with the FreeBSD boot loader on the QNAP TS-EC2480U-RP.
On the other hand, FreeNAS 11.1-U7 (which relies on GRUB to boot) works well

So I will use FreeNAS 11.1-U7 on my both QNAP
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
I have a feeling that there will be many older systems that are not going to work well with the new BSD boot loader.
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
The QNAP TS-EC1680U-RP has two internal mSATA slots. Therefore I have purchased two Samsung EVO 860 mSATA 250GB to use them as boot disks instead of USB keys.

First attempt
I have installed FreeNAS-11.2u3. At the installation ends, when the computer reboots, I receive "gptzfsboot error 1 lba xxxxxx" errors, but the boot continues to the BTX but it finally hangs for ever. It is very similar to the problem I had with USB boot devices. It means that there is an incompatibility between FreeBSD BTX and the BIOS of my QNAP server whatever the kind of the boot devices. So for the moment I am stuck with FreeNAS 11.1u7.

Second attempt
I have installed FreeNAS-11.1u7. The installation and reboot runs normally. To avoid configuring my server again, I have decided to import my FreeNAS configuration I had when booting with a USB key. After importing this configuration, the computer reboots but crashes, so it enters into an infinite loop reboot loop. It seems that importing a configuration file after changing the kind of boot device, is not supported. (with USB boot devices, the boot devices were da1 and da2, but when switching to mSATA, the boot devices become ada4 and ada5)

Third attempt
I have installed FreeNAS-11.1u7. But I have entered all the settings in the GUI again to avoid importing my previous FreeNAS configuration with the wrong boot devices. This time eveything works. I have rebooted 5 times without issues.

Conclusion
Despite my QNAP server still does not support FreeNAS 11.2u3, I am happy to have mSATA SSD boot devices instead of USB boot devices because:
  • The server boots faster
  • There is no pending USB key at the server rear
  • mSATA SSD should be more reliable than USB key
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
Last week, I have upgraded from FreeNAS-11.1u7 to TrueNAS Scale 20.02 (I have to reinstall from scratch)
Unlike TrueNAS Core that uses BTX, TrueNAS Scale boots with Grub, so I run well on my Qnap server.
 

kherr

Explorer
Joined
May 19, 2020
Messages
67
Thanks for the info ...... I've been wanting to use my 673 as a backup target, but to get rsync to work you basically need to be a code monkey (which I'm no where close) to get the SSH permission keys set. I currently have 16GB in it, although a lesser Rysen in it. I'll give it a try soon.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
How are you finding Scale (on Linux) vs the BSD based NAS?

At the first time, I was disappointed because the menu layout is not exactly the same than TrueNAS Core.
Except that minor issue, TrueNAS Scale is pretty good. All the features that I like, seem to be present (even the boot environments).

For the moment, TrueNAS Scale (Linux) is still in beta, and it is less mature than TrueNAS Core (BSD), so if your hardware supports both version it is safer to stay with TrueNAS Core.
But on the long term, I think that TrueNAS Scale (Linux) will close the gap and overtake TrueNAS Core (BSD).
It is a pity because I really liked FreeBSD, but I have to admit that Linux offers more technical possibilities than FreeBSD.
(KVM, docker, GlusterFS, hardware support.)

The only annoying issue I have found with TrueNAS Scale on my Qnap is:
I have to press Enter to boot TrueNAS Scale in Grub, otherwise it waits for ever. (it is a bug of the Qnap BIOS)
 

blanchet

Guru
Joined
Apr 17, 2018
Messages
516
With TrueNAS-Scale-22.02-RC1, the issue with GRUB seems to have disappeared.

The NAS starts normally: GRUB does not wait anymore that I hit ENTER to continue to boot :smile: .
 
Top