А couple of questions?

[andry]

hello everyone, can you please advise on a few things:
1) https://www.truenas.com/docs/references/zfsdeduplication/
I see that Deduplication can use ARC/L2ARC and it can use SLOG?
2) in a normal situation De-duplication tables be stored in Metadata? or as a separate table?
3) I don’t understand for what exactly, in what use cases, Metadata may be needed?
4) I found 2 ssd disks, I wanted to use them for cache, but they seemed to be a little different size - 238.47 GiB and
232.89 GiB , it says to me - "Mixing disks of different sizes in a vdev is not recommended." -
a) Could I force VDevs? (for example, a mirror) and what will happen then? it will only use ~232GiB ?
b) or should I somehow cut off partitions of the same size with my hands and make VDevs already on these partitions?
5) can I make Data VDevs on it, make 2 Zvol, and then use them as Cache VDev (L2ARC) and Log VDev (SLOG)? or do I need to cut them into 2 sections, and feed them as devices?

 

[Arwen]

Give this a read;

Resource - ZFS de-Duplication - Or why you shouldn't use de-dup

The TrueNAS forums occasionally have people who come across ZFS de-duplicate, and want to investigate its use. Or think it is a good idea, and want to implement it. Here are some suggested configuration details: Understand that you need CPU...

www.truenas.com

I don't have all the answers, but Metadata is for things like directory entries which are critical for recording which blocks a file uses. And any file attributes.

In regards to 2 SSD of different sizes for "cache". Note that L2ARC cache devices are generally not to be more than 5 times memory size, (total for all L2ARC cache devices). Some people go up to 10 times memory size... your mileage may vary.

It is NOT possible to Mirror L2ARC cache devices. It is possible you are mixing up terminology. Give these a read;

Terminology and Abbreviations Primer

We realize that new users have a lot to learn when they come to FreeNAS. There's a certain amount of confusion added to discussions when users pick random/approximate terms to describe things. I've spent a lot of time quietly trying to...

www.truenas.com

Introduction to ZFS

This is a short introduction to ZFS. It is really only intended to convey the bare minimum knowledge needed to start diving into ZFS and is in no way meant to cut Michael W. Lucas' and Allan Jude's book income. It is a bit of a spiritual...

www.truenas.com

It is also possible some of your comments about Metadata are about Metadata Special vDevs. Using precise language will help us understand you needs better.

TrueNAS does not support splitting up storage devices and sharing them in different vDevs or pools. If you do this manually, you also have to support it manually, like during disk failure. As well as any performance issues are complicated by this configuration.


TrueNAS is an appliance type software, Some people need / want specific things that are not generally supportable with TrueNAS, but ZFS can support. Not sure if that is the case here.

What is your use case?

 

[andry]

1) does deduplication use SLOG?

2)if I have "Vdevs Metadata" but don't have "Vdevs De-duplication tables" then De-duplication tables are stored in Metadata?

3) I think you answered my question

I will rephrase the following questions.

4)
I found 2 ssd drives, I wanted to use them in the vdev mirror, but they seem to be a slightly different size - 238.47 GiB and
232.89 GiB, tells me - "Mixing disks of different sizes in vdev is not recommended." Can I force create VDevs? will it only use 232.89 GiB from each drive? (the size of the smallest of the drives) or if it exceeds 232.89 GiB, it starts to write data to only 1 disk and then I can lose data?

5) how do i get my 2 ssds to be used as both SLOG and L2ARC ?

 

[Arwen]

1 - I don't know, and I doubt it. Please note a SLOG is not a write cache. It is a separate write log, (all pools have an internal write log, called ZIL, ZFS Intent Log). ZIL & SLOG are only useful for synchronous writes, like for NFS or iSCSI. Thus, my question about your use case.

2 - Yes, if you don't have a Special vDev for De-Dup tables, they will be in the normal pool vDev(s). But, note that De-Dup tables are loaded in to memory on pool import. (And written to stable storage on update.)

4 - Yes, you can mix device sizes in a vDev using the Force check box. The vDev will simply use the smaller of the 2 devices. The GUI has a set amount to check for size differences, and your 2 devices triggered it.

5 - You don't through TrueNAS GUI, that is what I meant about manual effort. This could have changed. Basically if you can't do it on your own, then you almost certainly can't support it on your own. But, in general L2ARC cache is not mirrored because the pool data vDevs contain 100% of the data. And depending on use, Mirroring of a SLOG would be recommended.

 

[ChrisRJ]

What is your use case?

That is a critical question for being able to give proper advice. Can you provide some background here?

 

[andry]

thanks for answers.,
I built a file server from hdd disks.
i am using Dataset as smb file storage.,
and now I want/try to use another Dataset with deduplication for games, - 2 computers connected to smb as disk mount.
if I use Dataset without deduplication - I steadily survive 1 gigabit of the network.
but when using deduplicated shares, I noticed that even with 1 computer, the speed can drop very much below 15 megabytes per second.
now I have added a ssd drive as l2arc connected via usb - but it only seems to get worse... - the speed dropped to 0-1 megabytes per second.
although it's strange - because when I checked the speed of ssd (connected via usb), the speed was about 500 megabytes per second.

I would also like to use 2 ssds for both l2arc and SLOG and possibly also for De-duplication tables.
(I believe that if I cut the partitions on the disk myself, then the partitions can be added to the pool , but since I have one "server" and it's the only one - I don't want to risk and experiment ..)

 

[ChrisRJ]

and now I want/try to use another Dataset with deduplication for games

Don't do that. Deduplication requires massive hardware (esp. RAM) to work properly. From what I remember the recommendation starts with 256 GB RAM as an absolute minimum.

With today's disk space prices, using deduplication does not make sense to save 5 or 10 TB.

 

[Arwen]

Using USB attached storage for ZFS data pool device(s) is very much not recommended;

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often in the last year or 2. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data...

www.truenas.com

How did you test that 500MBps speed?
On the same computer & TrueNAS version?

Using a storage device for 2 uses is not supported by the GUI. And the general recommendation is don't do it. And if you want / need to do it, you are on your own. Partly we don't want to see you loose data. Partly we would expect you to do all the work, install & support, your self. If you can't do the work your self, (which by your questions, you can't), then we are back to not recommending it be done.

TrueNAS is an appliance type software. Even though "root" access via SHELL is available, it does not mean you can do anything you want, and still have a stable, reliable and recoverable server. Lots of things done via SHELL won't be recorded into the TrueNAS configuration. Thus, they could be lost on reboot or recovery of a failed boot device.

Last, in the Resource on De-Dup I provided, it lists that fast CPU & fast memory are helpful to scan through the De-Dup table. If you have a slower CPU, even if it has more cores, that can slow down De-Dup.

Resource - ZFS de-Duplication - Or why you shouldn't use de-dup

The TrueNAS forums occasionally have people who come across ZFS de-duplicate, and want to investigate its use. Or think it is a good idea, and want to implement it. Here are some suggested configuration details: Understand that you need CPU...

www.truenas.com

But, over-clocking is not recommended due to heat and reliability concerns. So, if the normal behavior of your CPU & memory are not enough for fast De-Dup, then you either need to re-visit your CPU & memory choices. Or live with the speed for De-Dup.

And as stated in the De-Dup Resource, De-Dup is not some magic bullet to get more storage. There is a price.

 

[andry]

Don't do that. Deduplication requires massive hardware (esp. RAM) to work properly. From what I remember the recommendation starts with 256 GB RAM as an absolute minimum.

With today's disk space prices, using deduplication does not make sense to save 5 or 10 TB.

given that modern games weigh 60 GB or more, then saving 60 GB per game is not a little... (when installing 2 copies)
somewhere it was written that for deduplication you need 5 GB of RAM for 1 terabyte of disk space.

Using USB attached storage for ZFS data pool device(s) is very much not recommended;

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often in the last year or 2. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data...

www.truenas.com

How did you test that 500MBps speed?
On the same computer & TrueNAS version?

Using a storage device for 2 uses is not supported by the GUI. And the general recommendation is don't do it. And if you want / need to do it, you are on your own. Partly we don't want to see you loose data. Partly we would expect you to do all the work, install & support, your self. If you can't do the work your self, (which by your questions, you can't), then we are back to not recommending it be done.

TrueNAS is an appliance type software. Even though "root" access via SHELL is available, it does not mean you can do anything you want, and still have a stable, reliable and recoverable server. Lots of things done via SHELL won't be recorded into the TrueNAS configuration. Thus, they could be lost on reboot or recovery of a failed boot device.

Last, in the Resource on De-Dup I provided, it lists that fast CPU & fast memory are helpful to scan through the De-Dup table. If you have a slower CPU, even if it has more cores, that can slow down De-Dup.

Resource - ZFS de-Duplication - Or why you shouldn't use de-dup

The TrueNAS forums occasionally have people who come across ZFS de-duplicate, and want to investigate its use. Or think it is a good idea, and want to implement it. Here are some suggested configuration details: Understand that you need CPU...

www.truenas.com

But, over-clocking is not recommended due to heat and reliability concerns. So, if the normal behavior of your CPU & memory are not enough for fast De-Dup, then you either need to re-visit your CPU & memory choices. Or live with the speed for De-Dup.

And as stated in the De-Dup Resource, De-Dup is not some magic bullet to get more storage. There is a price.

Disconnected the ssd as l2arc - everything returned to normal.
How did you test that 500MBps speed? - Windows + CDM (crystaldiskmark)
Yes, I understand what you are talking about.
Thanks for the help!

 

[WI_Hedgehog]

De-duplication is rarely a good idea, ZFS built-in compression is almost always a better choice.

De-duplication is best used when data is purposely duplicated, such as in a corporate mail server. If the weekly newsletter is sent to 500 users and is graphics-intensive due to embedded (rather thank linked) graphics, there would be a space savings at the expense of the memory and processor time needed to run de-duplication. If users frequently send reports (like PDFs) to groups of people, the reduction in storage space coupled with having to read the data from only one place (vs. 10 mailboxes) could offset the overhead of the de-duplication system, and the total system overhead could see a net reduction. The backups of the e-mail system (which might be really slow and expensive if the media is tape) might greatly benefit from de-duplication.

The bottom line is don't use de-duplication unless there's a definite need to do so.

 

[Heracles]

Again :

No USB drive
No Deduplication

Also, seeing the way you did so far, I have doubts about the hardware you used for that server. Please, detail your hardware and config... Important is how your HDD are connected, configured and managed.

 

[WI_Hedgehog]

given that modern games weigh 60 GB or more, then saving 60 GB per game is not a little... (when installing 2 copies)
somewhere it was written that for deduplication you need 5 GB of RAM for 1 terabyte of disk space.


Disconnected the ssd as l2arc - everything returned to normal.
How did you test that 500MBps speed? - Windows + CDM (crystaldiskmark)
Yes, I understand what you are talking about.
Thanks for the help!

Oh, oh, it's another gaming setup. Here's the short of it:

• TrueNAS members aren't snobs; TrueNAS is Datacenter-level software, so members tend to focus on Datacenter-level data retention, hence all the requirements.
• TrueNAS is "expensive" in the regard gamers are trying to save money, not spend more money on non-game stuff. Gamers basically want a Lamborghini, and TrueNAS is a military personnel transport (comparitively big, slow, and expensive, but delivers a whole lotta eye-opener).
• Gamers tend to focus on speed, which is completely understandable. However, TrueNAS is not a great solution; you really, really want ext4 under Ubuntu or your favorite Linux flavor for a server, which is still out of scope. The ideal solution is RAID-0 on 3-4 SSDs in your gaming rig, then make a system image and occasional backups of saved-games (Synology, Acronis, other).

Members here really do want you to get to the end goal. The YouTube entertainment videos are just that, entertainment. A server sounds cool, but there are far better ways to game.

 

[ChrisRJ]

Gamers basically want a Lamborghini, and TrueNAS is a military personnel transport (big, slow, expensive).

Love that! ROTFLBTC

 

[HoneyBadger]

somewhere it was written that for deduplication you need 5 GB of RAM for 1 terabyte of disk space.

That's a very low estimate, based around a mixed-file storage use case. For deduplication of block storage resources, such as what I suspect you might be attempting to set up, you could easily require 20GB or more or RAM per TB of logical storage.

Please, detail your hardware and config... Important is how your HDD are connected, configured and managed.

As suggested above - the amount of RAM will also be crucial here.