HOWTO: Adding custom application repositories to SCALE 21.02 and beyond!

[Kris Moore]

SCALE Fans,

Just in time for the upcoming TrueNAS SCALE 21.02 release, support for multiple Catalog repositories has landed. This allows app developers to easily build, test and ship new 3rd party applications for SCALE. In order to help folks get started, I'll be writing down some of the steps you can take here, and eventually we'll turn this into a full-fledged article on the docs site.

To get started, you'll want to fork our official iX Catalog repository to some new location, as I did to my personal repo.

Next you'll want to create a chart which can be used to install your application. Checkout the catalog README for more information on how TrueNAS uses Helm charts + some additional files to function in a fully UI-driven environment such as TrueNAS SCALE.

In this example, I created a new chart for IPFS, mostly for my own amusement and to experiment with the helm chart creation process.

With the new chart pushed to my local repository, we can now proceed to adding this repository to TrueNAS. For this we'll go ahead and use the snazzy new CLI which just recently landed in SCALE nightly images as well.

In the example above, you can see that adding a new repository is just a one-liner in the CLI. Replace my URL and label with your own of course.

After the CLI command finished, I went ahead and brought up the Apps page of the UI, and sure enough, my new IPFS application was listed and ready for installation.

As you can see, apart from the time spent creating the chart, adding a new repository of applications was pretty quick and straightforward. Before anybody asks, yes we intend to add a GUI method to do this as well, which will land in the near future. As always we really do appreciate testers and their feedback in this process. Very much looking forward to seeing what applications others come up with.

 

[ornias]

Thanks @Kris Moore

To give some background on the story:
Before this was merged, testers needed to manually edit source files to show additional catalogs. This is not peristant across updates and might've caused issues in the future.

The big challenge with custom catalogs is quality: Creating good charts, Keeping them up-to-date and having solid CI.
This is also one of the huge issues with the TrueNAS CORE plugins and with a bit of luck the new SCALE Apps provide enough standardisation to finally get actually solid applications ready to rock on TrueNAS!

 

[bodly]

Very cool. I added @ornias catalog.

Code:

 app catalog create repository="https://github.com/truecharts/truecharts" label="TrueCharts Dev" branch="dev"

But... I can't install any of them. When I try I get an error about namespaces.

Installing
Error: [EFAULT] Unable to delete namespace: (404) Reason: Not Found HTTP response headers: < CIMultiDictProxy('Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '329ce537-b916-492b-aa1a-91d5134073ce', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd8f57d2a-971a-478b-8ac8-4e8981446003', 'Date': 'Fri, 05 Feb 2021 20:51:51 GMT', 'Content-Length': '194') > HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces \"ix-ombi\" not found","reason":"NotFound","details":{"name":"ix-ombi","kind":"namespaces"},"code":404}

It looks like it is trying to delete the ix-ombi namespace, which doesn't exist. I manually created it and things go further, but then I ran into this:

Error: [EFAULT] Unable to update namespace: (422) Reason: Unprocessable Entity HTTP response headers: < CIMultiDictProxy('Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Kubernetes-Pf-Flowschema-Uid': '329ce537-b916-492b-aa1a-91d5134073ce', 'X-Kubernetes-Pf-Prioritylevel-Uid': 'd8f57d2a-971a-478b-8ac8-4e8981446003', 'Date': 'Fri, 05 Feb 2021 20:59:21 GMT', 'Content-Length': '2022') > HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Namespace \"ix-ombi\" is invalid: metadata.labels: Invalid value: \"TRUECHARTS DEV\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')","reason":"Invalid","details":{"name":"ix-ombi","kind":"Namespace","causes":[{"reason":"FieldValueInvalid","message":"Invalid value: \"TRUECHARTS DEV\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')","field":"metadata.labels"},{"reason":"FieldValueInvalid","message":"Invalid value: \"TRUECHARTS DEV\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')","field":"metadata.labels"},{"reason":"FieldValueInvalid","message":"Invalid value: \"TRUECHARTS DEV\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')","field":"metadata.labels"},{"reason":"FieldValueInvalid","message":"Invalid value: \"TRUECHARTS DEV\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')","field":"metadata.labels"}]},"code":422}

So it looks like the CLI allows me to add a catalog with a space in it, but this is not allowed.

I can't figure out how to edit it or delete it or set it as inactive (which I think should be an option).

Trying to delete gives me this error:

app catalog delete id="TRUECHARTS DEV"
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 138, in call_method
result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self,
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1220, in _call
return await methodobj(*prepared_call.args)
TypeError: delete() missing 1 required positional argument: 'id'

Sorry for the long post.

It's super cool that community charts has gotten this far. Thanks for all your hard work!

 

[ornias]

Very cool. I added @ornias catalog.

Code:

 app catalog create repository="https://github.com/truecharts/truecharts" label="TrueCharts Dev" branch="dev"

View attachment 44918

But... I can't install any of them. When I try I get an error about namespaces.

*edit*
Whoops I read wrong.
No, you really should use truecharts-dev like in my example in the other thread.

*edit2/update*

Just to be clear:
Using the dev branch for TrueCharts is NOT required anymore.

First charts are released as of today:

TrueCharts

TrueCharts is a catalog of highly optimized TrueNAS SCALE Apps. All our apps are supposed to work together, be easy to set up using the TrueNAS UI and, above all, give the average user more than enough options to tune things to their liking...

www.truenas.com

 

[bodly]

No, you really should use truecharts-dev like in my example in the other thread.

Hopefully I will be able to remove or rename it soon.

 

[Kris Moore]

Hopefully I will be able to remove or rename it soon.

They are fixing the CLI issue shortly, you should be able to remove / rename probably early next week.

 

[nubian122]

They are fixing the CLI issue shortly, you should be able to remove / rename probably early next week.

This is going to be very cool. My issue is however, in my 3 installs of Truenas Scale - none of the Apps work. I select a pool, it creates the support file structure and then Igo into Apps and it shows the 3 original. I want to install Plex, but I get this:

Error: [EINVAL] chart_release_create.release_name: Does not match [a-z0-9]([-a-z0-9]*[a-z0-9])?

Any ideas?

Lastly I am having difficulty doing replication - will not work locally from pool to pool or to another server via the GUI. I am getting this:
[2021/02/06 00:14:10] ERROR [replication_task__task_8] [zettarepl.replication.run] For task 'task_8' unhandled replication error KeyError('sharenfs')
Traceback (most recent call last):

Thank you for anything you can do. I am using 2 Dell PowerEdge servers - 1 720 and 1 820. 3rd is a Asrock dual xeon system.

 

[Kieeps]

@nubian122 i also lile to name my containers with dashes, underscores and whatnot just to get some "imaginary" categorization... But as the error message states, keep it to 0 to 9 and a-z just name it plex and try again

 

[ornias]

@nubian122 i also lile to name my containers with dashes, underscores and whatnot just to get some "imaginary" categorization... But as the error message states, keep it to 0 to 9 and a-z just name it plex and try again

The thing with dashes is they are also used for internal k8s catagorisation with suffixes and prefixes. Thats why they are not supported.

 

[Kieeps]

That error message only states lowercase letters, is that also a limitation or just an oversight?

 

[stavros-k]

Hopefully I will be able to remove or rename it soon.

app catalog delete "TRUECHARTS DEV"

When you create a catalog it converts it to uppercase.
For some reason while on creation and all other functions you do label=something on delete you don't do id=something
If you want to see what catalogs you have installed do app catalog query label, repository, branch
If you want to delete something with spaces in it, put it in quotes! Works!

 

[ornias]

Just to be clear:
Using the dev branch for TrueCharts is NOT required anymore.

First charts are released as of today:

TrueCharts

TrueCharts is a catalog of highly optimized TrueNAS SCALE Apps. All our apps are supposed to work together, be easy to set up using the TrueNAS UI and, above all, give the average user more than enough options to tune things to their liking...

www.truenas.com

 

[shadofall]

Just to be clear:
Using the dev branch for TrueCharts is NOT required anymore.

First charts are released as of today:

TrueCharts

TrueCharts is a catalog of highly optimized TrueNAS SCALE Apps. All our apps are supposed to work together, be easy to set up using the TrueNAS UI and, above all, give the average user more than enough options to tune things to their liking...

www.truenas.com

. Woohoo. Now to figure out treafik :p never had a need for it before. But really don't wanna deal with nodeports lol

 

[ornias]

@shadofall Everything is abstracted away into TrueNAS SCALE, you don't actually deal with Traefik itself ;-)

It's actually a heavy customised package including cert-manager to automatically fetch any letsencrypt certificates.
Certificate generation might take up to 10 minutes though, be mindfull of that ;-)

 

[shadofall]

@ornias Kinda gathered that looking at the questions for the available containers that everything was more or less squared away with traefik support, just a matter of unsupported/self loaded containers for lack of a better way of wording it

although I'm really thinking after looking closer at it. traefik is just overkill for me. I don't access my home server from outside of my network. if I need to access it remotely I do it over a VPN. so unless i'm seriously overlooking something (always a possibility to miss something obvious) it just seems excessive, oh well something to keep researching and thinking over.

 

[tallnerd1985]

@ornias How do I refresh the Catalog after issuing the command to add your Truecharts because I feel like I am missing a key app in all the available namely called Sabnzbd that doesn't appear to show up.....

 

[stavros-k]

@ornias How do I refresh the Catalog after issuing the command to add your Truecharts because I feel like I am missing a key app in all the available namely called Sabnzbd that doesn't appear to show up.....

Code:

app catalog sync "TRUECHARTS"

should return <null>

 

[shadofall]

@ornias How do I refresh the Catalog after issuing the command to add your Truecharts because I feel like I am missing a key app in all the available namely called Sabnzbd that doesn't appear to show up.....

Your not missing it. Its not available at this time, ornias started with the containers he uses first. and is working down a list of of the requested apps.

 

[ornias]

Your not missing it. Its not available at this time, ornias started with the containers he uses first. and is working down a list of of the requested apps.

Actually most of whats currently merged isn't used by me personally ;-)

I wonder however where the idea comes from that sabnzbd was merged :P

 

[shadofall]

Actually most of whats currently merged isn't used by me personally ;-)

I wonder however where the idea comes from that sabnzbd was merged :P

I kinda figured that at this point but my statement should still be more or less valid, started with ones you use. then started down a list of requested based on some sort of priority (even if that priority is at your whim LOL), maybe with some non requested but popular ones in there.

as for the idea that it was merged. to be fair. you look at the list UI list and not github. you see all the major "arr" players, transmission and now nzbhydra, you would probably expect nzbget or sabnzbd to be there as well :P