How to make Plex transcode in RAM?

[ryszardlski]

Hey Guys,

I'm new here and I registered solely for the purpose of creating this thread.
I have been searching this forum, Plex forum, YT and everything that google threw at me but I couldn't find the answer to my question.
"How to make Plex transcode in RAM on FreeNAS?"

Is there a way? Can I even do it, and if so - how?
I'm asking for an explanation for dummies, since I am completely new to this system.

Thanks in advance
Cheers

 

[nojohnny101]

"How to make Plex transcode in RAM on FreeNAS?"

I think that is the wrong question and likely asked because you don't understand how transcoding works. That is a CPU task.

Did you mean to ask about speeding up the storage where the files that plex works with lives? If that is what you meant, then likely storing your media on a separate SSD within plex would help although I can tell you from experience and from what I have heard from others, that CPU's are often the bottlenecks in regards to plex and transcoding, not spinning rust. This is obviously situation dependent and would matter if you have a lot of jails runnings that are resource intensive.

 

[ryszardlski]

Hi.

No. The question is correct. I want Plex to store transcoded files on RAM. I know transcoding itself is done by the CPU. However the transcoded files have to be somewhere. What I want is to have these files in RAM. On windows you can just write \transcode in the transcode directory setting in Plex and that’s it.

Cheers

 

[Redcoat]

Post #19 in this thread has some indications. Maybe that will help.

 

[ryszardlski]

Post #19 in this thread has some indications. Maybe that will help.

I have 1 problem with this though. I don't want to mount a specific Volume in RAM essentially creating a RAM disc. I know this is an option since I stumbled upon this thread before, during my search. I want to it to be a dynamic tmp folder not a set Volume.

You have the same option in Windows. You can create a RAM disc and point Plex to transcode in it, but you can also set the transcoding directory to \transcode which simply uses RAM as a temporary storage for the transcoded files. The second option leaves your RAM unused whenever you are not transcoding, while the RAM disc always eats a set amount of RAM from your system.

Hope I explained this understandably.

Cheers

 

[SweetAndLow]

Why can't you do the \transcode thing? Plex is the same on all platforms for the most part.

 

[peter2cfu]

Any chance of plex or Freenas people comment on this, there is just so much contradictory information that it's a bit farcical.

I have a system that working well except for transcoding, and no reason for this that I can see.

The system in question can be seen in my signature below.

How can I set up a transcode to use tmpfs FreeNAS ramdisk.

 

[sretalla]

Creating a RAM drive is covered here:

Testing the benefits of SLOG using a RAM disk!

Say you were considering getting a stupid fast SLOG... because your VMs are too slow... You could create a SLOG on a RAM disk. This is an incredibly stupid idea, but it does demonstrate the maximum performance gains to be had from using the absolutely fastest SLOG you could possibly get...

www.ixsystems.com

mdconfig -a -t swap -s 6g -u 1

There you would have a device of 6GB at /dev/md1 (vary the size to suit your needs)

If you then create a pool with it:
zpool create RAMpool md1

then Create the dataset to put in your plex jail
zfs create RAMpool/plextmp

Optionally export the pool so you can put it in the GUI (using Storage Pools Add, import existing pool after the export at CLI)

zpool export RAMpool

Then set the permissions you need for the Plex jail user in the GUI if you did that import, otherwise:
chown -R plex:plex /mnt/RAMpool/plextmp (substitute plex user UID if you don't have one matching on the host)
chmod -R 775 /mnt/RAMpool/plextmp

Then add the dataset in the jail in the GUI if you did the import, otherwise:
iocage exec plex mkdir /mnt/tmp
iocage fstab -a plex /mnt/RAMPool/plextmp /mnt/tmp nullfs rw 0 0


Set plex to use that location (/mnt/tmp in the example above) using the Plex Interface (Settings, Transcoder, Show Advanced, Transcoder temporary directory)

Once you tested that and are happy with it... you need to think about how you will do that every time your server restarts:
The mapping in the jail will work as long as the RAMpool is recreated with the same name and permissions each time, so no need to do that last part again.

You could think about setting up a pre-init task with the command:
mdconfig -a -t swap -s 6g -u 1 && zpool create RAMpool md1 && zfs create RAMpool/plextmp && chown -R plex:plex /mnt/RAMpool/plextmp && chmod -R 775 /mnt/RAMpool/plextmp

(thanks to DeMoB here for the idea on that one, even if I did the ramdisk a little differently using the article from @Stux)

EDIT: I noticed that my testing VM where I had tried some of this out (running TrueNAS Core Nightlies) crashed last night and after the reboot, it was expecting to see the RAMpool (because I had imported it in the GUI... so I expect that this is a bad idea... I have struck through the items above for that reason).
EDIT2: I saw that the iocage fstab was read-only (not much point in a temp location that you can't write to)... changed it to read-write.

 

[peter2cfu]

Creating a RAM drive is covered here:

Testing the benefits of SLOG using a RAM disk!

Say you were considering getting a stupid fast SLOG... because your VMs are too slow... You could create a SLOG on a RAM disk. This is an incredibly stupid idea, but it does demonstrate the maximum performance gains to be had from using the absolutely fastest SLOG you could possibly get...

www.ixsystems.com

mdconfig -a -t swap -s 6g -u 1

Thank you for a very complete and elegant solution/answer... :)

 

[peter2cfu]

As an update, I may have a possible issue see text in red, no idea if this was appropriate but allowed me to continue, below are the exact steps I used.

in FreeNAS web interface goto Shell (bottom of the left-hand main menu)

Code:

mdconfig -a -t swap -s 20g -u 1
zpool create RAMpool md1
zfs create RAMpool/plextmp
mkdir /mnt/RAMpool
mkdir /mnt/RAMpool/plextmp
chown -R plex:plex /mnt/RAMpool/plextmp
chmod -R 775 /mnt/RAMpool/plextmp
iocage exec plex mkdir /mnt/tmp
iocage fstab -a plex /mnt/RAMpool/plextmp /mnt/tmp nullfs rw 0 0

In the Plex web interface set plex transcoder option to use

Code:

/mnt/tmp

in the FreeNAS web interface under Tasks goto Init/Shutdown Scripts where I added these commands

Type: SHUTDOWN | Timeout:20

Code:

zpool destroy -f RAMpool && mdconfig -d -u 1

Type: POSTINIT | Timeout:60

Code:

mdconfig -a -t swap -s 20g -u 1 && zpool create RAMpool md1 && zfs create RAMpool/plextmp && mkdir /mnt/RAMpool && mkdir /mnt/RAMpool/plextmp && chown -R plex:plex /RAMpool/plextmp && chmod -R 775 /RAMpool/plextmp

Run the df command from FreeNAS Shell just to check, you should see something like this if all goes well!

Code:

RAMpool                                                      19807652         88    19807564     0%    /RAMpool
RAMpool/plextmp                                              19807652         88    19807564     0%    /RAMpool/plextmp
...

Run the zpool list command from FreeNAS Shell just to check, you should see something like this if all goes well!

Code:

root@media[~]# zpool list
NAME           SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
RAMpool       19.5G   644K  19.5G        -         -     0%     0%  1.00x  ONLINE  -
...

I skewed up a few times so others may find this useful, run the below commands from the FreeNAS Shell to remove the memory drive

Code:

zpool destroy -f RAMpool
mdconfig -d -u 1

And remember to remove the jails specific stuff as well, if you're not going to ramdisk anymore.

Other Notes
https://www.reddit.com/r/freenas/comments/72yvs8/how_can_i_create_a_ramdisk_for_plex_transcoding/ - will try this one out later, seems to be a simpler approach.

July:19:2020 Seems this no longer works with the latest update of FreenNAS 11.3-U3.2, for some reason the task section is no longer selectable, sub items show but nothing happens when you click them. A bug maybe so recreated the RAMdisk manually as above via the shell now all the jails work, but still no access to anything to do with tasks.

 

[peter2cfu]

as an addition, I'm not sure why but seems every time I reboot, I have to startup manually the ramdisk and can only do it by destroying then recreating as below.

Code:

zpool destroy -f RAMpool && mdconfig -d -u 1

then

Code:

mdconfig -a -t swap -s 20g -u 1 && zpool create RAMpool md1 && zfs create RAMpool/plextmp && mkdir /mnt/RAMpool && mkdir /mnt/RAMpool/plextmp && chown -R plex:plex /RAMpool/plextmp && chmod -R 775 /RAMpool/plextmp

and then manually startup all my jails, looks like I need to make a script to start this up, other than that all seems to work.

 

[Yorick]

This is the second time I've seen the idea of using RAM for transcode scratch. I'm puzzled - why? Write speed during transcode is far below what a HDD can deliver, ditto read. What's the advantage of moving this to RAM?

 

[peter2cfu]

This is the second time I've seen the idea of using RAM for transcode scratch. I'm puzzled - why? Write speed during transcode is far below what a HDD can deliver, ditto read. What's the advantage of moving this to RAM?

Because this seems to work, it did for me, I have HGST Ultrastar 4TB 3.5" SAS 12Gb/s and I was getting transcode anomalies now they seem to have gone away, also another very valid use case would be if you want to prolong your SSD life.

For me, I believe it was a latency related issue.

 

[Yorick]

I was getting transcode anomalies now they seem to have gone away

Interesting. What kind of anomalies?

 

[peter2cfu]

Freezing was almost guaranteed when I tried to scroll/jump video via on the web interface, almost all issues were exclusively web interface related, which makes sense as that's when transcoding is used from what I've read, which lead me to look at RAM for the transcoding.

Ohh and something else I noted, it seems the catalogue is now much faster at displaying! I no idea if it's related but it's noticeable.

 

[sretalla]

as an addition, I'm not sure why but seems every time I reboot, I have to startup manually the ramdisk and can only do it by destroying then recreating as below.

Actually that makes sense and I was thinking about it before you posted that...

Of course when ram is powered off, all data is lost, so our pool is toast, but the config still remembers it, so we would need to clean that (I guess you could put a shutdown command like "zpool destroy -f RAMpool && mdconfig -d -u 1"

Then the creation at each startup would be clean if you do it as a pre-init command.

Maybe that would be enough.

 

[peter2cfu]

Actually that makes sense and I was thinking about it before you posted that...

Stupid me, of course, I'm really new to this, and it right in front of my face :) thanks.

P.S I added what I have done to the above instructions, I guess we can call this almost a how-to, this works, it's not very elegant but it's functional, thank you for all your help, I would not have been able to get this far without your support, hope the documentation helps someone else and hopefully they can improve upon it.

 

[ornias]

This is the second time I've seen the idea of using RAM for transcode scratch. I'm puzzled - why? Write speed during transcode is far below what a HDD can deliver, ditto read. What's the advantage of moving this to RAM?

Guess I was the first? ;)

The advantage is mainly saving disk IO and bandwidth.
Be aware: Disks have a LOT worse read+write performance than read and write seperately (a factor 2 worse often)...


While it might not be a benefit in your usecase, some of us might need/want the IO and bandwidth for other uses... If you have just one vdev and a few users at the same time, it might come in handy.

 

[ornias]

as an addition, I'm not sure why but seems every time I reboot, I have to startup manually the ramdisk and can only do it by destroying then recreating as below.

Code:

zpool destroy -f RAMpool && mdconfig -d -u 1

then

Code:

mdconfig -a -t swap -s 20g -u 1 && zpool create RAMpool md1 && zfs create RAMpool/plextmp && mkdir /mnt/RAMpool && mkdir /mnt/RAMpool/plextmp && chown -R plex:plex /RAMpool/plextmp && chmod -R 775 /RAMpool/plextmp

and then manually startup all my jails, looks like I need to make a script to start this up, other than that all seems to work.

This is a VERY BAD way of doing it.
Why the hack do you create a zpool on a ramdisk?! You are currently caching your ramdisk in ram. Congratz.
With these kinds of setups i'm not amazed people run into weird issue tbh.

Just create a ramdisk the jail and add it to the jail fstab. No fuzz.

 

[peter2cfu]

Just create a ramdisk the jail and add it to the jail fstab. No fuzz.

I'm very new to this 1-2 weeks max I no idea how to interpret what you said into an actionable sequence that makes sense.

P.S what I have done exactly is clearly laid out above :)