Advice on Compression for large SMB share

[Julianh]

Can anyone point me in the right direction?

I've just installed TrueNas core, up and running, it's just running as a SMB share. It's reads I care about, write really doesn't matter at all. As it's video files, films, I read that I should set the compression to Gzip. Of the 5500 files, most will be untouched for months, some haven't been touched for years. But when acessed I want it to be fast. An initial wait for the start of decompression is okay, it's the sending/playing of the film that has to be quick. Did I select the right compression, and if not can I change it without copying the data over.
The other area I think I''ve messed up on it the pool usage, there's about 28TB of data, but the pool is 35.7TB I want to run the pool up to 95%, but I read past 80% "it" slows down, it that read or Write? Write I don't care about, it's just read.
I have a seperate server which has it's own storage and is used as a monthly backup, literealy just a rsync between them, then turned off for a month. I don't mind if I have to copy the data back from the backup to the truenas, all I care about is read speed for the SMB share, so it's a stripe. If I lose a disk and the data, I'll replace the disk and copy it back. I've accepted this will happen when I expand the pool anyway.
Thanks you for reading this and your advice

 

[sretalla]

Did I select the right compression

Maybe, depends on your hardware (which we don't know about).

can I change it without copying the data over.

No.

it's a stripe. If I lose a disk and the data, I'll replace the disk and copy it back. I've accepted this will happen when I expand the pool anyway.

Expanding the pool doesn't require that.

In any case:

it's the sending/playing of the film that has to be quick

Unlikely to require more bandwidth than a single disk can handle, so maybe all of your concerns are misplaced.

Video is also (usually) already compressed, so may not compress well/further in any case... questionable benefit from doing any further compression. (https://askubuntu.com/questions/455783/compressing-videos-create-even-larger-file)

 

[Julianh]

Here's a bit about the hardware, sorry I neglected it.
The server is a Dell t340, 32GB, it has had the h300 controller's firmware flashed as recomended. There are 5x 8TB sata drives for data and the OS runs off a 550gb SSD. It will have Chelsio 10GB cards between the Emby server and Trunas. Point to point, no switch. They're on order, so at the moment, it's running with the 1gb, just to get the data over while I'm waiting for the card to arrive.

 

[joeschmuck]

First of all, I personally would not recommend changing the compression from the default because video content like .mpg, .mpeg, etc. are already compressed. If you wanted to actually compress the video then you would have to re-encode it to a newer format, quite possibly a much lesser quality format. For example say you have a lot of 4k movies, 25GB each, your could re-encode them to 2k movies or 1k. Re-encoding them takes time unless you have a great video card to do the processing. And 4GB is better than 25GB when it comes to storage, and if you do the re-encoding well, the quality of the output can look very good. Of course if you are projecting everything in 4K on a 200" screen, the 4k content will likely look much better but on a 65" screen at a distance of 15 feet, yo might not even know the difference, but again, how you re-encode it makes a huge difference and there is a lot of trial and error unless you are just good at this kind of stuff.

I understand you are creating an unimportant server to host video content, that is fine as many people do that here. I do it too but only about 300GB of my data is video content out of about 5TB of data stored. I re-encoded much of my content years ago and I don't recall at all how I did it but I know it took me a long time to figure out what worked best for me, and days and days or re-encoding.

I want to run the pool up to 95%, but I read past 80% "it" slows down, it that read or Write? Write I don't care about, it's just read.

It is the write operations but that would be very bad practice to make a pool that full and no one here would tell you that it is proper to do that.

and if not can I change it without copying the data over.

If you desired to change the compression, it's a fairly easy matter but you do need to move the data over. What you would do is create a new dataset and choose the compression you desired. Then in the shell window (I prefer to use SSH), use the mv command to move the data from the original location to the new location. This would prevent you from having the copy all the information off first and then restoring it. But this is how I would do it if I wanted to change the compression for a dataset. Just be careful that you don't actually start using more space.