SOLVED StorJ Configuration TLS Certificate Issues

[t0x]

Hi all - new to the forums so pardon if something is missing:


I am trying to install the Official StorJ app and I followed the directions found here:
Setting Up a Storj Node | 2 and the associated youtube video for good measure.

They seem a bit incomplete though. For example, it didn’t mention I needed to allow port 7777 outbound. But after a bit of working around, I was able to get most communication working.

In the instructions, it said to forward 20988, which didn’t result in a working node, so I changed the configuration both on TrueNAS and on the firewall to what is on the main Storj website here, “28967” and I was able to validate that connectivity can be established via Open Port Check Tool - Test Port Forwarding on Your Router.

What I am left with is the following error message in the logs:
RROR contact:service ping satellite failed {“Process”: “storagenode”, “Satellite ID”: “12rfG3sh9NCWiX3ivPjq2HtdLmbqCrvHVEzJubnzFzosMuawymB”, “attempts”: 11, “error”: “ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: tls peer certificate verification: not signed by any CA in the whitelist: CA cert”, “errorVerbose”: “ping satellite: failed to ping storage node, your node indicated error code: 0, rpc: tcp connector failed: rpc: tls peer certificate verification: not signed by any CA in the whitelist: CA cert\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatelliteOnce:149\n\tstorj.io/storj/storagenode/contact.(*Service).pingSatellite:102\n\tstorj.io/storj/storagenode/contact.(*Chore).updateCycles.func1:87\n\tstorj.io/common/sync2.(*Cycle).Run:160\n\tstorj.io/common/sync2.(*Cycle).Start.func1:77\n\tgolang.org/x/sync/errgroup.(*Group).Go.func1:75”}

And in the web interface, I am shown that the node is offline and that QUIC is misconfigured. Not sure how this can be since I am able to validate the port connectivity. I’ve seen a couple of posts about the identity being wrong, but I’m not sure what else I can do to correct it. I have DDNS configured through FreeDNS and all is working. I've even reverted the settings back to the original TrueNAS port 20988.

Any suggestions would be appreciated.

 

[t0x]

Figured it out after more tinkering. There should definitely be some updates to the documents:

1) Outbound 7777 is REQUIRED for communication with StorJ "Satellites" and needs to be open on the firewall.
2) You should follow https://docs.storj.io/node/dependencies/identity to create the identity OUTSIDE of TrueNAS and then use NFS/SMB/FTP/etc. to upload the generated credentials there. There appears to be a bug within the Official app that prevents proper secret/key creation when following the instruction mechanism.
3) outbound 8888 is REQUIRED for communication while creating & signing the identity. (if that is ever working again, you will need that port open).

 

[morganL]

I submitted this to docs... but its even better if Users make the request and can be contacted for clarification.

 

[danb35]

You should follow https://docs.storj.io/node/dependencies/identity to create the identity OUTSIDE of TrueNAS and then use NFS/SMB/FTP/etc. to upload the generated credentials there.

Can you clarify this a bit? Because what I'm understanding is to follow those instructions, then (e.g.) create a tarball containing ~/.local/share/storj/identity/*, copy it over to the NAS, and untar it into (if following iX' instructions) /mnt/(pool)/storj-node/identity/, such that the contents of that directory would be revocations.db and the storagenode directory. Right so far?

Because what I'm seeing when installing the app is that it seems to be generating a new identity, which I wouldn't expect to work.

 

[danb35]

There should definitely be some updates to the documents

There should be an update to the app itself:

• It defaults to port 20988 for the node, not 28967; the former won't work at all
• Even when set to use the correct port, it only opens it to TCP, not to UDP, and both are required. Otherwise, it gives this result:
  
  

You should follow https://docs.storj.io/node/dependencies/identity to create the identity OUTSIDE of TrueNAS

I didn't find this to be necessary; I gave the app an auth token and it created its own identity successfully. It took a while, but that seems to be normal.

Right so far?

No, it isn't; the app wants the cert/key files in storj-node/identity, and revocations.db in storj-node/config. But if it's generated any other config files there, I expect you'd want to erase those if you're providing new identity.

its even better if Users make the request

...and where would that be done? And for that matter, where would issues be raised against the app itself?

And why, in the name of all that is good and holy, are there separate forums here for TrueNAS -> Applications and Jails and TrueNAS -> TrueNAS SCALE -> Apps? Particularly when the former has a subforum for Storj Discussion?

 

[morganL]

...and where would that be done? And for that matter, where would issues be raised against the app itself?

In the case of storj .. there is a docs page on SCALE. It is weak on firewall rules, but that is where documentation should be improved.

/scale/scaletutorials/apps/communityapps/addstorjnode/

If the App itself is broken or needs significant improvement, we'd prefer that request on github (after there is agreement that the App is the issue).

Issues · truenas/charts

TrueNAS SCALE Apps Catalogs & Charts. Contribute to truenas/charts development by creating an account on GitHub.

github.com

 

[morganL]

And why, in the name of all that is good and holy, are there separate forums here for TrueNAS -> Applications and Jails and TrueNAS -> TrueNAS SCALE -> Apps? Particularly when the former has a subforum for Storj Discussion?

Its a good topic for discussion... in general, we didn't want to overwhelm plugin discussions with SCALE App configuration issues. But that was a decision from 18 months ago.

 

[danb35]

If the App itself is broken or needs significant improvement,

Well, the app defaults to using TCP port 20988 (which the user can change at install time or later, but it's still a bad default), and no UDP ports (and there's no way for the user to specify additional ports that I can see). Storj wants both TCP and UDP (as correctly noted on the docs page, but not implemented in the app) on port 28967 (not 20988 as stated on the docs page). This is specified in their own docs, linked up-thread, and in the app's web console:

Apparently it's working without QUIC--it's using bandwidth, and storing data on my pool--but they're considering it misconfigured.

The issue was reported seven months ago but closed as "completed" despite nothing having been done:

Storj Chart needs to expose TCP and UDP · Issue #762 · truenas/charts

With the storj app installed, I am unable to get Quic function because UDP is not being forwarded. Looking at the installed application indicates: Used Ports: 20909\TCP, 28967\TCP. It should be: 28...

github.com

 

[morganL]

Well, the app defaults to using TCP port 20988 (which the user can change at install time or later, but it's still a bad default), and no UDP ports (and there's no way for the user to specify additional ports that I can see). Storj wants both TCP and UDP (as correctly noted on the docs page, but not implemented in the app) on port 28967 (not 20988 as stated on the docs page). This is specified in their own docs, linked up-thread, and in the app's web console:
View attachment 66336
Apparently it's working without QUIC--it's using bandwidth, and storing data on my pool--but they're considering it misconfigured.

The issue was reported seven months ago but closed as "completed" despite nothing having been done:

Storj Chart needs to expose TCP and UDP · Issue #762 · truenas/charts

With the storj app installed, I am unable to get Quic function because UDP is not being forwarded. Looking at the installed application indicates: Used Ports: 20909\TCP, 28967\TCP. It should be: 28...

github.com

I created a new ticket - add a comment if you can be contacted by engineering team.

NAS-121758

 

[danb35]

Done, and attached a debug file.