<div class="bbWrapper"><i>Many of us run ZFS for myriad purposes. For those who use it in DCs or “work” purposes, I salute you, but this thread is for media enthusiasts who love ZFS for many of the same reasons, but have very different workloads. </i><br />
<br />
I imagine an ARC that is told to load files in anticipation as I and my users navigate Plex:<br />
- When a movie is selected, Plex “tells” ARC to go ahead and load it as soon as the movie is selected, so that if I hit play, the file is resident in RAM<br />
- When an episode of TV is selected, similarly, load it to ARC<br />
- When a TV series is selected, load the oldest unwatched into ARC<br />
- Etc<br />
<br />
Sure, there could be unnecessary reads, but those can be thwarted with a “lookie Lou” counter of titles-viewed-but-not-played-in-last-5-minutes, with a TTL. <br />
<br />
How can we bridge the gap between Plex and ZFS to make this possible? <br />
<br />
Thanks for your thoughts. TC</div>
<div class="bbWrapper"><blockquote data-attributes="member: 66480" data-quote="tc60045" data-source="post: 418094"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418094"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418094">tc60045 said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
How can we bridge the gap between Plex and ZFS to make this possible?
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote>By rewriting Plex to be ZFS-aware, at a bare minimum. Except even that wouldn't do it, because Plex runs in a jail, and the jail never sees ZFS.<br />
<br />
Think about this for a second--is it really a good idea for userspace software to be directly manipulating the ARC? I'm pretty sure the answer is "no."</div>
<div class="bbWrapper"><blockquote data-attributes="member: 3713" data-quote="danb35" data-source="post: 418096"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418096"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418096">danb35 said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
By rewriting Plex to be ZFS-aware, at a bare minimum. Except even that wouldn't do it, because Plex runs in a jail, and the jail never sees ZFS.<br />
<br />
Think about this for a second--is it really a good idea for userspace software to be directly manipulating the ARC? I'm pretty sure the answer is "no."
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote><br />
Doesn’t run in a jail on my box. And there is no reason why an ARC messaging bus couldn’t see suggestions. YMMV on a DC production box; this box is my own to configure.</div>
<div class="bbWrapper"><blockquote data-attributes="member: 66480" data-quote="tc60045" data-source="post: 418099"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418099"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418099">tc60045 said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
Doesn’t run in a jail on my box.
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote>Sorry, I figured that since you were posting on the FreeNAS forums, your suggestion had something, however remote, to do with FreeNAS.</div>
<div class="bbWrapper"><blockquote data-attributes="member: 66480" data-quote="tc60045" data-source="post: 418094"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418094"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418094">tc60045 said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
<i>Many of us run ZFS for myriad purposes. For those who use it in DCs or “work” purposes, I salute you, but this thread is for media enthusiasts who love ZFS for many of the same reasons, but have very different workloads. </i><br />
<br />
I imagine an ARC that is told to load files in anticipation as I and my users navigate Plex:<br />
- When a movie is selected, Plex “tells” ARC to go ahead and load it as soon as the movie is selected, so that if I hit play, the file is resident in RAM<br />
- When an episode of TV is selected, similarly, load it to ARC<br />
- When a TV series is selected, load the oldest unwatched into ARC<br />
- Etc<br />
<br />
Sure, there could be unnecessary reads, but those can be thwarted with a “lookie Lou” counter of titles-viewed-but-not-played-in-last-5-minutes, with a TTL.<br />
<br />
How can we bridge the gap between Plex and ZFS to make this possible?<br />
<br />
Thanks for your thoughts. TC
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote>Plex could just, you know, load the file into RAM (as much as possible). It's not rocket science, the ARC will give way to real memory usage.<br />
<br />
Now, a better question is, "does this make my server faster or better in any way?" I can see scenarios for both "yes" and "no". Given that media is not particularly difficult to stream from disk, I'd say it's probably best to keep it out of RAM as much as possible, beyond a smallish buffer.</div>
<div class="bbWrapper"><blockquote data-attributes="member: 3713" data-quote="danb35" data-source="post: 418096"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418096"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418096">danb35 said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
because Plex runs in a jail, and the jail never sees ZFS.
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote>It could, that's not a problem <i>per se</i>. Just another knob that has to be (carefully) exposed to the jail, similar to existing options for datasets assigned to jails.<br />
<br />
That's one of the many wonders of ZFS. It integrates with jails in a way that just makes so much sense that you go "How is this not on every OS on the planet yet!?".</div>
<div class="bbWrapper"><blockquote data-attributes="member: 37321" data-quote="Ericloewe" data-source="post: 418110"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418110"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418110">Ericloewe said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
Now, a better question is, "does this make my server faster or better in any way?" I can see scenarios for both "yes" and "no". Given that media is not particularly difficult to stream from disk, I'd say it's probably best to keep it out of RAM as much as possible, beyond a smallish buffer.
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote><br />
That is a better question. Try transcoding files out of RAM and then try transcoding off disk — world of difference. And with remote users, that difference matters. So if you have the RAM, why not just nudge ARC to make that possible?</div>
<div class="bbWrapper"><blockquote data-attributes="member: 66480" data-quote="tc60045" data-source="post: 418114"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418114"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418114">tc60045 said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
Try transcoding files out of RAM and then try transcoding off disk
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote>Well, you should never be transcoding straight from disk, hence the buffer. But we're getting into very specific details that are best left available for experienced users to tune.</div>
<div class="bbWrapper"><blockquote data-attributes="member: 37321" data-quote="Ericloewe" data-source="post: 418115"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418115"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418115">Ericloewe said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
Well, you should never be transcoding straight from disk, hence the buffer. But we're getting into very specific details that are best left available for experienced users to tune.
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote><br />
Exactly - so how might one nudge ARC to allow experienced users to tune. None of us is one who would stand in anesthetized awe of the immutable holy ARC. What say we tell ARC what we want it to cache? How?...</div>
<div class="bbWrapper"><blockquote data-attributes="member: 66480" data-quote="tc60045" data-source="post: 418121"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418121"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418121">tc60045 said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
Exactly - so how might one nudge ARC to allow experienced users to tune.
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote>Well, you wouldn't for situations like this, because the application itself is in a much better position to manage the buffer size and it can operate directly on the buffer, instead of making repeated syscalls to get this and get that.<br />
<br />
<blockquote data-attributes="member: 66480" data-quote="tc60045" data-source="post: 418121"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418121"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418121">tc60045 said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
None of us is one who would stand in anesthetized awe of the immutable holy ARC.
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote>Well, it's already tunable.</div>
<div class="bbWrapper"><blockquote data-attributes="member: 37321" data-quote="Ericloewe" data-source="post: 418122"
class="bbCodeBlock bbCodeBlock--expandable bbCodeBlock--quote js-expandWatch">
<div class="bbCodeBlock-title">
<a href="/community/goto/post?id=418122"
class="bbCodeBlock-sourceJump"
rel="nofollow"
data-xf-click="attribution"
data-content-selector="#post-418122">Ericloewe said:</a>
</div>
<div class="bbCodeBlock-content">
<div class="bbCodeBlock-expandContent js-expandContent ">
Well, you wouldn't for situations like this, because the application itself is in a much better position to manage the buffer size and it can operate directly on the buffer, instead of making repeated syscalls to get this and get that.
</div>
<div class="bbCodeBlock-expandLink js-expandLink"><a role="button" tabindex="0">Click to expand...</a></div>
</div>
</blockquote><br />
You trust your Nest; I take mine apart at the code level. You say to trust it; I smile at the improvements I’ve long enjoyed with my own machine learning code. <br />
<br />
As for tunable, nah; ARC needs an API like Nest has. Then we can call it tunable.</div>
