Server Upgrade - CORE or SCALE?

[unseen]

I decided that my current TrueNAS CORE server was getting a little long in the teeth (ASRock Rack E3C224D21, i3-4150 CPU, 16GB DDR3 ECC UDIMMS (2x8GB), 4xWD Red Pro 8T, boot and 'fastpool' on SATA SSDs) and decided to build something with a bit more punch.

The new machine is an ASRock Rack X470D4U (would have been the X570D4U if I could find one in Sweden), Ryzen 5 3600 CPU, 64GB DDR4 ECC UDIMMS (2x32GB), LSI 9211-8i HBA, boot to reuse 2 x 64GB Crucial SSDs, fastpool to use two new 256GB NVMe SSDs on add-in card, previous fastpool (on aData SATA SSDs to be retired) main pool on the 4x8TB WD Red Pro disks.

I've used the SilverStone CS831 chassis with a 700W SilverStone SFX power supply. I'm fully aware of the challenges with disk cooling that this case poses and have been carefully sealing up everywhere air can enter except for the drive bays as I've been putting it together. CPU cooling is an AIO liquid cooler with a 240mm radiator as there's no reasonable alternative which will fit in the available space. I've also added two 120mm Noctua Industrial fans which can spin at 3000 rpm if needed. All in all, I can pull lots of air in through the drive bays and will initially put two drives in each disk enclosure to spread out the disks. The HBA is now flashed to IT firmware and a 40mm fan has been screwed on top of the heatsink.

What I can't decide about is if I should stay on the FreeBSD version of TrueNAS or migrate to SCALE while I have the opportunity. (I'm assuming here that I can just export my configuration from CORE and reload it on SCALE so that my main pool is recognised and imported on the new server, just like if I had made the decision to use CORE. I've also exported the two jails that I run on the current server (urbackup and Jellyfin), but I guess I can't just import a FreeBSD jail into SCALE.)

I don't mind having to re-create my jails, but it will be a pain to lose my watched/unwatched status for the umpteenth time, on Jellyfin.

So I'd like to ask the community: If you were in my shoes, what would you do? Stay on CORE or migrate to SCALE?

 

[winnielinnie]

What I can't decide about is if I should stay on the FreeBSD version of TrueNAS or migrate to SCALE while I have the opportunity.

Is there a real need to migrate over to SCALE?

I don't mind having to re-create my jails, but it will be a pain to lose my watched/unwatched status for the umpteenth time, on Jellyfin.

You can copy the folder(s) where such data is saved.


Some examples, inside a jail:

qBittorrent stores such data under /var/db/qbittorrent <--- copy/backup this folder

Jellyfin stores such data under /var/db/jellyfin <--- copy/backup this folder

(Just pay attention to the ownership and permissions. It will have to match in your new jail. As long as you left the defaults, you should be fine. In my case, however, I have the qbittorent and jellyfin services running under a different user ("uid") than the upstream defaults.)

You can make a quick tarball backup of these directories, even while outside of the jail, and save them to a (temporary?) holding location on a proper dataset. (Note that there is a "space" neat the very end of the command, before "jellyfin".)

Code:

tar c -vaf /mnt/pool/somedataset/jellyfin-db-backup_$(date +%Y%m%d).tar.zst -C /mnt/pool/iocage/jails/jellyfin/root/var/db/ jellyfin

Now you will have the tarball archive jellyfin-db-backup_20236021.tar.zst in the directory /mnt/pool/somedataset/, which you can later use to extract back into the jail's root /var/db/ when needed.

Use "sudo" if you're not logged in as root. Change the paths accordingly to match your system and preferences.

 

[unseen]

Very true, I could make a backup of the Jellyfin database and that should preserve my watched/unwatched status nicely.

 

[unseen]

Is there a real need to migrate over to SCALE?

The only benefit I can see at the moment is better support for things like the CPU cooler. There are front-ends for the Corsair AIO coolers now which run on Linux after some excellent protocol reverse engineering. Things like USB devices don't have identical APIs between FreeBSD and Linux, so if nobody has already ported the Linux stuff over to FreeBSD, then there's some coding work to be done.

Seeing as programming is what I do to earn money, I've become less and less interested in doing more programming after my working day is over.

 

[winnielinnie]

Then after you create a new jail, you can reverse the order and re-extract the entire directory back to:

/mnt/pool/iocage/jails/jellyfin/root/var/db/

You should do this before installing the jellyfin package. (I'm assuming the jellyfin package "plays nice" upon first installation, in that it will detect and use an existing db folder; hopefully not wipe it out on first use. )

 

[unseen]

So far, I'm not hearing any real reason to migrate to SCALE. Even the AIO cooler seems to work quite well without even being connected to USB.

 

[2twisty]

I use SCALE because I am more familiar with Linux personally. If I were in an production environment, I'd probably still use CORE. BSD is more stable, but pickier about hardware. So if you have hardware that plays nice with CORE and you're comfortable with BSD, I'd stick with it.

 

[jgreco]

So far, I'm not hearing any real reason to migrate to SCALE

There's little reason, SCALE is generally an inferior platform unless you happen to need the Kube or scale-out features.

BSD is more stable, but pickier about hardware.

This isn't really true. It's said a lot by Linux advocates, but the equipment that you SHOULD be using on your NAS works perfectly fine on both CORE and SCALE. Oddly it's people on SCALE that seem to have more hardware issues, possibly because they buy into this line of reasoning.

 

[Whattteva]

There's little reason, SCALE is generally an inferior platform unless you happen to need the Kube or scale-out features.

Not even sure if that's worth using. I've seen way too many posts both here and on Reddit about the dreaded stuck on "Deploying" status forever usually (but not always) after an upgrade.

Frankly, I'd rather have to get my hands dirty with a little CLI jail management for predictable 24/7 running system than an easy 1-click deploy that may actually deploy only 50% of the time and have to pray and cross my fingers after every upgrade.

 

[unseen]

Thanks folks, I'll be staying with the tried and trusted FreeBSD version which has served me so well since FreeNAS release 9.1. That's never given me pain with an upgrade and has looked after my data at home for a long time now.

 

[unseen]

There's little reason, SCALE is generally an inferior platform unless you happen to need the Kube or scale-out features.

I have a proxmox cluster built from five 1 litre sized Lenovo ThinkCentre machines if I want to play with lots of virtual machines. The right horse for the right course!

 

[jgreco]

five 1 litre sized Lenovo ThinkCentre machines

So, what is that, about a mini-keg worth of mini PC's?

 

[unseen]

So, what is that, about a mini-keg worth of mini PC's?

Half a hogshead I'd say. :)

 

[victort]

If you want just a NAS, go with Core. There are a few scripts available to install popular services like Plex and Nextcloud. Other services will definitely require some cli work to install, but generally speaking they are foolproof if done right. Jails are also nice because they can have their own IP and MAC. The are basically like a VM, without the overhead. Did I mention they are very stable?

If you want a NAS with lots of Apps that can be installed, go with Scale. Plugins are going away so the only way to run services in Core will be manual jails. Someone could share the link about that article…

Scale is nice because of the Apps ecosystem, but has its downsides in not being able to have different IPs for the apps. Easy to workaround using ports and Traefik as a reverse proxy, but still…

Bottom line is that Core is stable as a NAS, and can run practically any service but needs to be installed manually in a jail.

Scale is for people who have lots of Apps and want simple straightforward installations. Although some things like editing config files are not as straightforward. At this point there have been a lot of breaking changes with Truecharts, but the official apps seem ok enough.

 

[2twisty]

I use SCALE because I am more familiar with Linux personally. If I were in an production environment, I'd probably still use CORE. BSD is more stable, but pickier about hardware. So if you have hardware that plays nice with CORE and you're comfortable with BSD, I'd stick with it.

Also, I may switch back to CORE at some point in the future. The biggest reason I am on SCALE is because I am using non-server motherboards, and the BSD support for the extra M.2 sockets isn't there. If I move the SSDs elsewhere or switch to a proper server chassis, I will go back.

I do not use apps or jails. My TN server is exclusively for serving data -- via SMB and NFS for my VMs on my separate VM host.

 

[unseen]

I like stable. I run Debian on my laptops.

I only have two jails: urbackup and Jellyfin. If there is something else I want to run which would benefit from direct access to the filesystem, it can be jailed on the NAS. I have no problems rolling my own jails from source - I even prefer it. Everything else goes in proper VMs or containers on the proxmox cluster, depending on trust level.

One thing which did come up while I am getting the new server ready is how to make a fresh and easy to restore backup of the current server. There's about 10TB of data on my NAS at the moment and it will only grow. The larger it gets, the harder it is to make backups.
I mistakenly thought that connecting a backup disk with USB 3.0 directly to the server would be the fastest way to store some zfs snapshots.

Boy was that slow! After checking how fast the process was going, I calculated about 68 days to back up the whole 10TB!

So I plugged the disk into my laptop and sent the zfs snapshot to it via netcat. Much better! It will take about 24 hours to send all datasets.

 

[unseen]

... My TN server is exclusively for serving data... ...I am using non-server motherboards...

I hope you have good backups of that data. If my data is important, it gets a server grade motherboard, ECC and proper NAS rated hard disks.

 

[winnielinnie]

There's about 10TB of data on my NAS at the moment and it will only grow. The larger it gets, the harder it is to make backups.

Don't worry. In a few years A.I. will be able to "regenerate" all of your data by providing it with a small sample.

* Make sure to include a varying sample of photos, videos, and documents. Linux ISOs are poor "seed material" for the A.I. to analyze.

 

[NickF]

I’m not sure why all the hate on SCALE at this point in the release cycle. I’be been using it in my homelab since Angelfish as a hypervisor and nas with great success…

Can’t say I’d recommend using apps for many workloads just yet though.

If you are hosting Jellyfin and Qbt in jails today, feel free to stay on core. I had always found jails to be more cumbersome than they were worth and ran apps in vms on a separate hypervisor until Scale.

 

[Whattteva]

I’m not sure why all the hate on SCALE at this point in the release cycle.

Hate is such a strong word. We simply don't recommend it due to issues like Linux memory management, apps stuck in Deploying status forever, higher idle CPU.

Can’t say I’d recommend using apps for many workloads just yet though.

Well, you just kinda' answered your own question. Apps are kinda' one of the big reasons why people use SCALE and if that's not really recommended... well you just took out a pretty significant selling point for it.