Boot drive size (32GB or 64GB?) and SATA Dom

[Patrick_3000]

I have SCALE installed on an ASRock Rack board and plan to switch my boot drive from NVME SSD (240GB) to SATA Dom in order to free up an NVME slot for use in my storage pool.

I am looking at either the 32GB or 64GB Supermicro SATA Dom. The 32GB is a bit cheaper, around $60 instead of $130, and I'm not sure if there is any advantage in the 64GB.

In particular, is 32GB enough, under all circumstances, for the boot drive in SCALE?

The minimum boot drive size recommended in the SCALE documentation is 16GB, but I've seen community posts where 32GB is recommended. But is even 32GB enough? The price difference between 32GB and 64GB is relatively minimal, so if there is any advantage at all in 64GB, I'll get it.

I don't currently have any apps or VMs installed and use SCALE for data storage only, but I'd like the flexibility to install apps or VMs in the future in case I ever want to do so (though I doubt that would affect the boot drive, as apps and VMs are installed in a storage pool).

 

[joeschmuck]

In particular, is 32GB enough, under all circumstances, for the boot drive in SCALE?

Yes, 32GB is more than enough for a boot drive. A boot drive is just that, it does not store much more than the operating system and a few configuration files. Any VM's you create will be on your storage pool.

 

[joeschmuck]

Out of curiosity, why not use a SATA SSD? Is it a power cable limitation? That option would be less expensive as well.

 

[Patrick_3000]

Yes, 32GB is more than enough for a boot drive. A boot drive is just that, it does not store much more than the operating system and a few configuration files. Any VM's you create will be on your storage pool.

Out of curiosity, why not use a SATA SSD? Is it a power cable limitation? That option would be less expensive as well.

I plan to remove the drive cage once I migrate from an HDD pool to an NVME SSD pool, because that will free up a lot of space inside the case and make maintenance easier. Unlike regular SATA SSDs, SATA Doms are small and attach directly to the motherboard, so I won't need to use the drive cage with them.

In addition, my eventual plans, which are not happening immediately but sometime down the road, involve switching to an ultra-quiet or even fanless cooler to have the flexibility of bringing the SCALE server to a quieter part of the house. These type of coolers are large and will not fit in the case with the drive cage installed.

Bottom line: I will be using SATA Dom, rather than regular SATA, so that I can take the drive cage out of the case and have a more spacious, easier to work with, system.

 

[Patrick_3000]

Thanks. It sounds like 32GB is enough.

 

[winnielinnie]

Wait, wait, wait! There is one catch. You need at minimum at 64GB boot-drive (boot-pool) capacity if you want the installer to create a 16GB swap file on the boot drive(s). This can be useful if you want to disable swap files on your storage pool drives.

(Unless this "64GB rule" has changed in recent versions of the installer?)

 

[Patrick_3000]

Wait, wait, wait! There is one catch. You need at minimum at 64GB boot-drive (boot-pool) capacity if you want the installer to create a 16GB swap file on the boot drive(s). This can be useful if you want to disable swap files on your storage pool drives.

(Unless this "64GB rule" has changed in recent versions of the installer?)

Thanks. This isn't something I'd considered.

I've never used a swap file, which as I understand it would only matter when RAM is low. I have 32GB RAM and eventually plan to upgrade to 64GB. In fact, an extra 32GB ECC RAM, which would upgrade to 64GB, is only $120, and I'd rather put my money into that than a larger boot drive.

On the other hand, if there is some reason I'd need a swap file, I'd be curious to hear it.

Incidentally, I just now SSH'd into the server and checked from the command line my current boot drive usage, and it's 3.3 GB.

 

[joeschmuck]

Wait, wait, wait! There is one catch. You need at minimum at 64GB boot-drive (boot-pool) capacity if you want the installer to create a 16GB swap file on the boot drive(s). This can be useful if you want to disable swap files on your storage pool drives.

(Unless this "64GB rule" has changed in recent versions of the installer?)

I'm curious how long the DOM would last. Maybe a SATA DOM would last, I know some didn't survive a long time but I suspect quality has gone up over the years.

A few more points to make might be, If you have enough RAM, your 2GB swap partitions on each drive are likely not to be used. The swap partitions on NVMe are going to be significantly faster than a SATA DOM.

To be honest, I didn't even consider the boot drive swap partition so that is on me for not knowing about it. I will do some reading on that so I'm informed and find out the benefits vs the cons, if there is any. I would suggest you do the same so you make an educated decision vice having a stranger tell you what is good or bad.

On the other hand, if there is some reason I'd need a swap file, I'd be curious to hear it.

A 2GB swap partition by default is created on every data drive. the user can manually change this value or eliminate it completely but it's only 2GB on each drive. A small price to pay for system stability.

Now why do you need a SWAP file in the first place? A Swap file is used when you run out of RAM and some RAM needs to be freed up to run an application, so some information will be moved out of RAM and on to the Swap file and then the application/files that need to be in RAM are able to move into RAM, until the data in the Swap file needs to be reloaded into RAM. That could be when some RAM is freed up or when the data that was put into the Swap file is required to be back in RAM. It is always smart to have some Swap partition for when it is ever needed, but you should design you system to never need to use the Swap partition. Using SWAP also slows TrueNAS down, but would you ever know it? Maybe not but if this is for ruining a VM, you very well might see slowdowns as data is being constantly swapped out because you didn't install enough RAM. In the TrueNAS GUI you can look at the SWAP partition to see the MIN/MAX size used. You want to see the MAX size stay under a few kbytes, zero is best of course. If you are using SWAP and you can increase your RAM, I'd recommend that.

Last thing, if your pool drives already have a Swap partitions established, you would need to backup all your data, destroy your pools, and recreate them with no Swap partition. It's easy to do but some folks would not want to go through that if they already have the system established. You should read up on SWAP Partitions.

So the decision is yours, and buying a 64GB DOM is perfectly fine and should work great, but out of the options you provided in the OP, the 32GB DOM would do the trick.

Thanks to @winnielinnie for posting that comment, it was a learning experience.

 

[winnielinnie]

On the other hand, if there is some reason I'd need a swap file, I'd be curious to hear it.

It's good to have some swap available, to avoid a situation where you run out of physical memory, and perhaps due to a memory leak or some other burden, the system slows down or freezes. To allow even just a couple GB of swap is like having an emergency bucket, even if you never end up using it. (If you're going to have swap, it might as well be on a fast SSD, like your boot drives, rather than slow HDDs.)

The problem with TrueNAS is that managing swap during a live system or installation is weirdly... inflexible.

The installer will not even offer to create a swap partition if your boot drive(s) is smaller than 64 GB. (Why?)

If your boot drive(s) is 64GB or greater, it will offer to create a swap partition at a set size 16 GB. (Why?)

Regardless if no swap partition exists on the boot drive(s), the default is to slice out a 2 GB partition from every storage drive to be used for swap.

I still don't know why the installer can't offer to create a swap partition on boot drive(s) that are 16GB or larger (since the OS doesn't take up much space). And when it does offer to create a swap partition, why it can't be a custom size? (2, 4, 6, 8, GB, etc.)

Why specifically "16 GB or nothing at all"?

Why only if the boot drive(s) is 64 GB or larger?

Surely this is a relic from the early days of FreeNAS?

EDIT: I see that @joeschmuck is using the same Google Chrome extension now.

 

[Patrick_3000]

Thanks for the information on swap partitions. I've been using some form of Truenas to store my data since 2016, first FreeNas, then Truenas Core, and now, since last year, Truenas Scale. In this time, I've never used swap partitions, and I've always had good transfer speeds, at least by the standard of hard drives, although of course much slower than SSD, which I'm in the process of upgrading to.

If I need more RAM, I'd rather buy it than rely on swap partitions, which are sure to be far slower than RAM. Low-end DDR4 ECC RAM, which has worked fine for me, is currently selling for something like $60 per 16 GB stick and $120 per 32 GB stick, which isn't bad. Moreover, swap partitions seem like they'd decrease the life of an SSD boot drive given the number of write operations I expect they'd entail. So, I doubt I'll use them.

Given that the official documentation for SCALE recommends 16GB minimum for the boot drive, and given that my boot drive currently contains only 3.3 GB of data based on a "df -H" command I ran today, I think I'm safe going with the 32 GB drive.

 

[Patrick M. Hausen]

I'm curious how long the DOM would last. Maybe a SATA DOM would last, I know some didn't survive a long time but I suspect quality has gone up over the years.

I have yet to see a single Supermicro SATA DOM fail. Oldest ones are running 6 years or longer, I don't quite have the dates, sorry. Expensive but very robust it seems.

 

[joeschmuck]

I have yet to see a single Supermicro SATA DOM fail. Oldest ones are running 6 years or longer, I don't quite have the dates, sorry. Expensive but very robust it seems.

So that sounds like the brand to buy. Thanks for the posting.