Overprovisioning NVMe (SSD) under SCALE for L2ARC

[MatthiasP]

Hi!
I would like to use only a part of a NVMe for a L2ARC. I found some bits for non Debian True/Freenas, but nothing seems to fit for me (or likely the problem is on the other side of the monitor).
As I would like to have a very basic NAS which is very easy to maintain, I would like to use the GUI for administoring, but I'm not afraid of using the terminal as I'm using different flavors of ubuntu/debian for more than 15 years.

I found that there are some ways to only use a part of a disk:
- make a partition and only use this part
I tried but the GUI in Scale only shows me the whole disk. I made a GPT partition an didin't format it, let the rest untouched.

I tried to disk_resize the disk, but every time the same error message showed up: Something like "a disk of this name doesn't exist or the drive is busy". I used fdisk -k and blkid to try to correctly identify the drives name, and also did this when I was booting the server and enter the console.
Here I guessed that disk_resize is a BSD tool and don't know if it is the right tool also for Truenas on Debian or not.

Also I found a way to use hdparm -Np to change the size the disk itself reports to the system, but this command is only supported upto a specific (sata?) standard. Therefore, I'm hesitated to use it as I don't know if there is the risk to damage the NVMe.

TLDR:
I would appreciate very much any kind help on howto use only a part of the NVMe. For me this seems nice as a bigger L2ARC also uses more RAM, and so I could first use a small portion and later try out if bigger is better.
I have some experience with ubuntu based systems, but don't get me wrong: I need a guiding hand for the correct steps to do: The appropriate commands to correctly identify the disks, for example, and more. This is because I don't know if for Truenas Scale different commands should be used as in ubuntu/debian or not. BSD/Truenas is better documented at this (RC) moment.

Thank you very much for your help!
Kind regards,
Matthias

 

[Hresna]

I’m you, 8 months later.

I had a finely tuned zfs topology in ubuntu with an l2arc on an nvme partition, but when I migrated to Scale, oddly enough the l2arc moved over with all its pre-warmed data still there. Not sure what it did with the partitioning though. I wonder if the l2arc max size parameter would achieve the same thing, perhaps, as overprovisioning (but would not allow use of nvme for installing apps / docker containers).

Did you ever figure it out? I would probably just SSH into the truenas and see what partitioning tools are included in the Debian install, and try to use the same methods as I did in ubuntu. Hasn’t worked for my zfs.conf file, but probably would for lower system-level stuff.

 

[MatthiasP]

Hi,
another 3 months later...
I didn't figure it out, because life needed the time.
Now I think I skip the idea of having an l2arc, and use it as drive for virtualization.
Did you figure it out?

 

[Hresna]

Hi,
another 3 months later...
I didn't figure it out, because life needed the time.
Now I think I skip the idea of having an l2arc, and use it as drive for virtualization.
Did you figure it out?

Haha, no. Gave the time back to life also.

Every time I try to use a basic Linux tool in TN I find it’s not installed or blocked off.. by design obviously. I’m just going to assume the overprovisioned partition I had done in Ubuntu is still set that way and move on with life. We’re I to do this again, I would probably try to parti on the boot disk to use as container space because I’m sacrificing a whole 256gb drive just for the is, but this is a common complaint.

 

[artlessknave]

all disks in TrueNAS are dedicated to one purpose, be it can be zfs data, boot, metadata, l2arc, etc. this is part of the appliances way of making configurations simple and consistent.
you would have to use a standard OS install to use ZFS in a non appliance way like this.

additionally, you haven't posted your hardware, which is a requirement of the forums.
L2ARC with less than 64GB of RAM is highly discouraged, due to being pointless; it wastes RAM that could otherwise be providing ARC, which is always better than L2ARC, and doesn't really help that much in most cases.

Not sure what it did with the partitioning though.

if you imported an existing pool everything would be the same as originally configured, however, this would be non standard to TrueNAS, and could lead to problems later.

parti on the boot disk

not possible in TrueNAS webUI. It *can* be hacked to work but pretty much everyone who has tried has ended up in the forums asking why weird things happen (one individual found that, to nobody's surprise, they were utterly unable to use the GUI to replace their boot disks, because the UI saw both partitions as having no serial number and, rightly, completely refused to touch anything)

 

[Hresna]

additionally, you haven't posted your hardware, which is a requirement of the forums.
L2ARC with less than 64GB of RAM is highly discouraged, due to being pointless; it wastes RAM that could otherwise be providing ARC, which is always better than L2ARC, and doesn't really help that much in most cases.
)

I’m not op…
I won’t speculate as to his use case for an l2arc. Mine is for video editing and it works extremely well. Arc with 32g ram has 99% hit ratio, 400G l2arc has 95%+ hit ratio after a few hours working in a project. I keep uncached data sets for stuff not part of source media.

I dig the appliance thing and know that a lot of this is “not supported”. I had it all running fine on Ubuntu, switched to TN just to lighten it further and power consumption also dropped. I know it might be a hassle to set up again if I have to reinstall for whatever reason. Most of what I needed is in startup scripts through the gui, so I’m not too worried.

I believe in Scale. It will get better and we won’t have to “hack” as many of the things as currently we must to access all of the ZFS server benefits.

 

[artlessknave]

Mine is for video editing and it works extremely well

there can be specialized cases where it does make sense, which are usually what the special disk types are for; special cases.
it's good if it's working for you, that's nice to hear. 95% ratios sounds like awesome numbers.