iXsystems, when are you going to stop pretending that CORE has (usable) plugins?

[danb35]

Plugins are broken. We all know it, and you have no intention of fixing them. Most notably, @Kris Moore has said not to use them, and that they're "more or less deprecated":

using plugins on CORE is a path to sadness. The plugin system is more or less deprecated and as you can tell, somewhat unstable.

The plugin system on CORE is for all intents and purposes deprecated and in pretty rough shape right now,

And users see it:

NextCloud plugin

I have installed TrueNAS-13.0-RELEASE but with great regret NextCloud is not installed: "pkg error: - nginx: Refusing to fetch artifact and run post_install.sh! Partial plugin destroyed" ... any ideas? thanks immensely.

www.truenas.com

Unable to install NextCloud plugin.

I an unable to install the Nextcloud plugin. "Error: NextCloud had a failure Exception: RuntimeError Message: pkg error: - nginx : Refusing to fetch artifact and run post_install.sh! Partial plugin destroyed". I recently upgraded to TrueNAS-13.0-RELEASE.

www.truenas.com

Error with Getting CA Private Key while installing Nextcloud Plugin

I have already tried to install Nextcloud several times. But I always get the following error message: nextcloud had a failure Exception: RuntimeError Message: Last 10 lines: Getting CA Private Key mysqladmin: [Warning] Using a password on the command line interface can be insecure. Warning...

www.truenas.com

Nextcloud plugin installation issue

Hi again, I am receiving this error "Error: Nextcloud had a failure Exception: RuntimeError Message: Last 10 lines: Partial plugin destroyed" Fresh TrueNAS installation, the only other plugin installed is plex. I tried multiple times, overright the same plugin, resetting my pool, new usb drive...

www.truenas.com

Plugins not installing on true nas

I am running true NAS in a hyper v VM to test its functions and to see if i can use it as my main server to do everything I need a server to do "Unifi server for work, pi hole for my internet, home bridge so i can have more things in home kit on my phone and a windows VM so I can remote in and...

www.truenas.com

Unable to Update Plex

Hi there I went to update the Plex Plugin and the following happen Error: Cmd('git') failed due to: exit code(128) cmdline: git clone -v https://github.com/freenas/iocage-plugin-plexmediaserver.git /mnt/Candace/iocage/jails/Plex/plugin stderr: 'Cloning into...

www.truenas.com

Who manages plugin updates? (Syncthing)

Version is 1.18.1 and that came out quite a long time ago. Is there a simple button to click and it deploys a new version somewhere, or is there more to it than that? I'd prefer to use my docker server but I'm trying to diagnose some strange behaviour.

www.truenas.com

But you're still pretending they're a real, usable thing:

Blog - Nextcloud and TrueNAS Collaborate to Help You Build Your Private Cloud

Today we are announcing a partnership with Nextcloud to provide an officially supported integration with TrueNAS. Nextcloud and TrueNAS are the #1 Open Source platforms for team collaboration and software-defined storage, respectively. The Nextcloud software suite will plug into TrueNAS and both...

www.truenas.com

You're being dishonest with your (prospective) users, and expecting us to pick up the pieces. It's getting more than a little frustrating. And while I'm focusing on Nextcloud above (partially because, less than a year ago, you made a big deal about your partnership with Nextcloud that would, you said, make the plugin not suck; and partially because I've spent a lot of time making Nextcloud work on CORE), the same is true of the rest of them (Asigra, another one you made a big deal about, seems to be abandonware from your perspective as well).

When are you going to be honest with your users and prospective users, and stop pretending that plugins are viable or usable for any purpose?

This isn't the first time you've been asked about this; see:

Just A Rant, Emby Plugin Update

Every... Single.... Time... I try to update any jail or plugin in any way, it doesn't work as expected. I have the Emby plugin installed and running fine in a 12.2-RELEASE-p13 jail on TrueNAS 13.0-RELEASE. TrueNAS tells me there is an update available, I stop the plugin and click the...

www.truenas.com

Just A Rant, Emby Plugin Update

Every... Single.... Time... I try to update any jail or plugin in any way, it doesn't work as expected. I have the Emby plugin installed and running fine in a 12.2-RELEASE-p13 jail on TrueNAS 13.0-RELEASE. TrueNAS tells me there is an update available, I stop the plugin and click the...

www.truenas.com

 

[Davvo]

This hurts your credibility so much.

 

[tprelog]

Shots fired, lol.

Seriously though, one thing more frustrating than trying to use plugins... was trying to maintain them. You can pick on the plugins I added a well, Hard as I tried, I never did get them right.

 

[danb35]

one thing more frustrating than trying to use plugins... was trying to maintain them.

I believe it. Plugins have been problematic ever since they were introduced (somewhere around 8.0, IIRC), and while I think part of the reason is that iX hasn't put enough resources into maintaining them, I don't doubt that a lot of the reason is that "enough resources" is actually a lot. I know I've put a lot more work into my Nextcloud script (which is the closest I've come to maintaining a plugin) than I'd expected to when I first started writing it.

 

[Jailer]

Just a shot in the dark here but I bet it's due to them eventually dropping core as soon as scale is mature enough to replace it. The world we live in is a Linux world and FreeBSD just doesn't have the industry support it needs to compete. How many in the business world rely on FreeBSD? Not very many. And once the remaining few that do finally give up on it FreeBSD will be nothing more than a "server" OS passion project. It saddens me because I love TrueNAS core and FreeBSD.

Docker.......

 

[danb35]

I bet it's due to them eventually dropping core as soon as scale is mature enough to replace it.

I've suggested that a few times and had my knuckles rapped by @morganL. Doesn't mean I don't agree with you that CORE doesn't seem to have much of a future, much as I'd like to be wrong.

But I don't think that's really (all of) what's going on; as I noted above, plugins have never worked all that well, even in the Warden/.PBI days. The difference now, I think, is that they now have a viable alternative in SCALE apps. So Kris can say "out loud" what many of us have been saying for years, with the addition of "switch to SCALE, however feature-incomplete it may be."

 

[Jailer]

It is unfortunate to see so many users have issues when you would expect things to just work and they don't. I know it's not windows but if you give users a big shiny button to 1 click install something you would at lest expect it to work. I don't think that's an unreasonable expectation.

 

[morganL]

I've suggested that a few times and had my knuckles rapped by @morganL. Doesn't mean I don't agree with you that CORE doesn't seem to have much of a future, much as I'd like to be wrong.

But I don't think that's really (all of) what's going on; as I noted above, plugins have never worked all that well, even in the Warden/.PBI days. The difference now, I think, is that they now have a viable alternative in SCALE apps. So Kris can say "out loud" what many of us have been saying for years, with the addition of "switch to SCALE, however feature-incomplete it may be."

No Knuckles were rapped... but let me try to clarify.

TrueNAS CORE and Enterprise have a future. TrueNAS 13.0 was just released. It the highest quality and best performing storage software in the family. It's just going GA which means we'll be making it the default software across the iXsystems product line. TrueNAS 13.0 is the fastest growing of the user bases and will overtake TrueNAS 12.0 next year. We are vigorously pursuing any ZFS/storage bugs found. CORE 13.0-U2 is probably the best software we have released.

However, we do think that SCALE will be the better hyperconverged and scaleout solution. That includes Apps and VMs. The range of Apps, the level of support for Linux and the number of users testing is far greater. We made the decision to not fight the Linux tidal wave, but give TrueNAS users the choice. SCALE is about 15% of the user base and growing fast. CORE is 85% and also growing, but more moderately (Since TrueNAS 12.0 has peaked).

We have made it pretty easy to migrate between CORE and SCALE. This is enormously complex, like changing from gas engines to electric, while keeping the API and UI similar and not dropping any data. Apps is the major area we can't automate. I also agree that Jails have some unique advantages that have not yet been replicated in SCALE. So there is no perfect single platform. Both SCALE and CORE benefits from the ongoing work in OpenZFS, Samba and Minio.

SCALE itself is making good progress from a maturity perspective. With SCALE 22.02.4 we are probably at a similar quality to 12.0-U4. There is still work to do, but it's increasingly solid. Bluefin will be better and is about to go to BETA.2. We expect that to get to 13.0 quality next year.

Getting back to Plugins and your criticisms... I would recommend a new user start with SCALE if they see converged applications being an important requirement. The infrastructure is better and there is a better development community. Perhaps we should de-emphasize Plugins in the marketing, as you point out.

For existing CORE users that need applications, it's a more complex calculation. There's all the investment in getting to where things are today. Some have migrated to SCALE already, most are still working through their issues on CORE. We continue to fix the bugs we find, but there is no magic wand. Make your decisions based on your needs.

For CORE users that are mostly using single node ISCSI, SMB, NFS, there's really no incentive to switch. Keep going until you see a reason to consider changing. We'll continue to ensure a high quality and still plan to introduce relevant features... for example the iX-Storj integration.

We realize this situation is more complex than previous years. We added a TrueNAS Software Status page to indicate where we are from a quality perspective.

 

[danb35]

Perhaps we should de-emphasize Plugins in the marketing, as you point out.

If your SVP of Engineering is saying in public what I've repeatedly quoted him as saying*, I'd say you should not only remove them (not merely "de-emphasize" them) entirely from the marketing, you should remove them from the product. In the alternative, he needs to make a public and very explicit correction. Because otherwise you're shipping a product with a significant feature that (at best) works poorly, with no apparent intent of making it work better.

the iX-Storj integration.

I hope this sees better reliability, documentation, and support than the Nextcloud and Asigra integrations--but based on those two, I'm not holding my breath. You have a long history of "overpromise and underdeliver" to overcome.

I don't enjoy being the downer here, but this problem has been going on, in one form or another, for nearly ten years, as long as then-FreeNAS has had plugins. They've never been a way to reliably install an application and have it work and stay up-to-date. Many of us have been advising against using plugins (and in some cases, putting a great deal of effort into alternative solutions), for these and other reasons, for several years. And now you (I'm understanding Kris as speaking for iX here) have agreed that they don't work, they aren't and won't be maintained, and they aren't going to get better--I can't think of any other way to interpret "for all intents and purposes deprecated and in pretty rough shape." So no, I don't think "de-emphasize" is enough.

*I think it's worth emphasizing that these are direct quotes, with links to the context. I'm not paraphrasing, nor am I taking him out of context.

 

[Patrick M. Hausen]

My gist with the situation is that you are putting FreeBSD and jails in a bad light with the current state of plugins. Both are proven and very robust technologies. So please change all advertising materials (at least) to e.g. replace "plugins and VMs" with "jails and VMs".
Fix the networking and point people to e.g. Klara Systems' documentation.

I still highly recommend CORE and jails to anyone who is admin savvy enough to roll their own applications with e.g. Ansible in an empty virtual environment.

 

[Etorix]

SCALE is about 15% of the user base and growing fast. CORE is 85% and also growing, but more moderately (Since TrueNAS 12.0 has peaked).

Excellent news… but I cant help noticing that means the user base is growing beyond 100%.

 

[morganL]

Excellent news… but I cant help noticing that means the user base is growing beyond 100%.

Yes, the whole user base is growing..... which has been the goal. More users are starting with Core than migrating to SCALE.

 

[winnielinnie]

I was looking for another chance to use this again!

 

[winnielinnie]

Just A Rant, Emby Plugin Update

Every... Single.... Time... I try to update any jail or plugin in any way, it doesn't work as expected. I have the Emby plugin installed and running fine in a 12.2-RELEASE-p13 jail on TrueNAS 13.0-RELEASE. TrueNAS tells me there is an update available, I stop the plugin and click the...

www.truenas.com

For the sake of consistent messaging, TrueNAS Core needs to drop all references to Plugins on its marketing and website, and replace it with: "Custom jails! Build your own FreeBSD jails to manually install and manage your own applications."

There's really no excuse for this. It's bad messaging. Harms the brand, and misleads new users.

Fine. Plugins are dead. Okay. But for the longest time this is what is being advertised in no subtle manner:

Those are all references to Core (not SCALE).

And to be clear, these have not just simply been on the website for the longest time... they're still there, even now.

 

[Kris Moore]

I just caught up to this thread, sorry I didn't catch this earlier! I'll try to provide some clarity on the engineering side of this discussion for everybody's sake.

Regarding the state of Plugins on TrueNAS CORE:

• They are built on top of Iocage (Deprecated) and on FreeBSD's Package System and Jails.
• Various Jail "Plugin" systems on FreeBSD have come and gone over the past decade. I personally wrote one of them. They just don't get the traction needed to make Jail-based systems attractive to the wider industry outside of niche use-case by existing BSD users and fans.
• We (the iX Side) are maintaining them to try and avoid breakages in 13.0 for our Enterprise Plugins, but not devoting much Engineering resource beyond that.
• Most plugins depend on upstream FreeBSD package systems for their boot-strapping and upgrading. This is very fragile and doesn't get much testing by upstream either, one major source of seemingly random breakages with no changes to the plugin itself.
• We are currently not planning to add any new Plugins to CORE unless absolutely necessary.
• Existing Plugins that become non-functional may be retired as necessary.

Regarding the state of VM's on TrueNAS CORE

• iX does not directly maintain the Bhyve VM backend or plan to add any new functionality beyond what exists today.
• What works today we will do our best to keep functional on 13 and avoid major regressions.
• Bhyve is still pretty immature compared to KVM and just doesn't have that level of polish we'd want to build longer-term VM support on top of, all future effort is shifting to KVM on SCALE.

Regarding TrueNAS SCALE Apps & VMs

• Linux Containers are the clear winner in the adoption war among developers, users and other software vendors. No contest, not even close.
• Most projects already provide Linux Containers natively which are well supported. FreeBSD packages often are non-existent, treated as an afterthought, or just don't have the user momentum to make them reliable and well tested.
• We are working heavily on the Apps framework currently. Current issues being worked on include fixing the OverlayFS support on ZFS (Which gets rid of the snapshot issues and performance penalty), as well as re-designing the Apps UI and Backends to handle Catalogs with thousands of Apps.
• We anticipate adding lots more Apps to the Official Catalog for SCALE in the near future.
• TrueNAS users who want a better supported 3rd party Apps on their system are encouraged to use SCALE, since that's where the momentum is going to be and they already provide a far better user experience today.
• KVM is already far more reliable on SCALE. Users expecting to do anything beyond the lightest of virtualization workloads will be better off doing this on SCALE now or in the future.

 

[danb35]

Regarding the state of Plugins on TrueNAS CORE:

Thanks for the background on why things are as they are, but I'm afraid you're missing my point. Lots of us have been saying for years that users should avoid plugins, they don't work reliably, they don't update reliably, they often include (at best) questionable configuration choices, etc. It's been more recently that you've made public statements in other threads (and now here) agreeing with this advice. So, well and good--we all agree that plugins are dead men walking, new users shouldn't use them, and existing users should come up with a path to move away from them. All of which feeds into my actual point, which (surprisingly enough) is the very title of the thread: when are you, iXSystems, going to align your marketing materials with this reality? Or are you going to continue to sell users on the plugin capability of CORE (as @winnielinnie has shown above in more detail than did I), and just let them be disappointed and badmouth your product when they find out that that feature, which you continue to advertise, doesn't work reliably and you don't intend to put any significant resources into making it work?

My gripe is not that plugins don't work. My gripe is that you know they don't work, you don't intend to fix them (and the reason really is irrelevant--I don't, and don't need to, attribute ill motives to you in this, but regardless of the reason, this is the case), and yet you still advertise them. And then leave us to pick up the pieces.

 

[Etorix]

@Kris Moore Thanks for your explanations from the engineering side. Do you have a fellow "SVP of Marketing" who could be directed to this thread?
@danb35 Maybe what's needed is to file a bug report against the web site…

 

[morganL]

Thanks for the background on why things are as they are, but I'm afraid you're missing my point. Lots of us have been saying for years that users should avoid plugins, they don't work reliably, they don't update reliably, they often include (at best) questionable configuration choices, etc. It's been more recently that you've made public statements in other threads (and now here) agreeing with this advice. So, well and good--we all agree that plugins are dead men walking, new users shouldn't use them, and existing users should come up with a path to move away from them. All of which feeds into my actual point, which (surprisingly enough) is the very title of the thread: when are you, iXSystems, going to align your marketing materials with this reality? Or are you going to continue to sell users on the plugin capability of CORE (as @winnielinnie has shown above in more detail than did I), and just let them be disappointed and badmouth your product when they find out that that feature, which you continue to advertise, doesn't work reliably and you don't intend to put any significant resources into making it work?

My gripe is not that plugins don't work. My gripe is that you know they don't work, you don't intend to fix them, and yet you still advertise them. And then leave us to pick up the pieces.

While I disagree with the generalization that plugins don't work, we do agree that they currently lack the reliability that people need for a sustainable solution. Both the iocage and the FreeBSD package system are just not nearly as good as the Docker/Kubernetes combination which have thousands (?) of times more users.

However, we agree you are making a valid point, but outside of Kris's responsibility. Kris and I have agreed on the summary above and it's now my action to change the marketing. There are lots of changes to be made. It will get done this quarter, but not overnight. We will move to focus on jails (which are good) and not plugins. The timing is right with Bluefin about to be released. We did not want to encourage a major transition of Plugin users to SCALE until it was more solid and had plenty of successful users.

In the short term, this thread can be referenced by anyone wanting to know the general strategy and for advice on what they should do/use.

 

[ChrisRJ]

While I disagree with the generalization that plugins don't work, we do agree that they currently lack the reliability that people need for a sustainable solution.

I have seen this differentiation many times; it is a typical inside-out view. Or in other words: It puts the company's perspective over that of customers.

From where I stand that is absolutely ok, as long as everybody is aware of it. It becomes problematic, however, when people are not aware and think of it as "absolute" truth. Then it is a sign of being disconnected from the customers. It often comes with being absorbed in corporate politics, structures, etc.

Perhaps a bit off-topic, but I wanted to share because it is such a widespread thing.

 

[kiriak]

I don't know how hard it is (or it was) the development and maintenance of the just 6 official plug-ins.
I don't know how hard it is (or it was) the development and maintenance of fa small number of scripts.

But these are last years' questions.

Sadly, he Core's development is already halted (beyond the minimum patches and upgrades until the Enterprise users move to Scale).
Just my personal opinion and possibly wrong.