SOLVED Fat fingered/bad tunable, lost web UI

ianwood

Dabbler
Joined
Sep 27, 2021
Messages
14
I added some tunables and may have fat fingered something because after reboot, some critical function seems to have gone sideways, most notably SSH, HTTP and SMB are not functioning. The server pings and I can get console access through the IPMI. What I see from the boot messages is that the web server, ssh, etc. are failing due to "no buffer space available" (probably because I put something in the tunables that I shouldn't have!).

vlcsnap-2022-07-06-17h53m42s719.png


These are the tunables I applied:

net.inet.tcp.recvspace= 4194304
net.inet.tcp.sendbuf_inc = 2097152
net.inet.tcp.sendbuf_max = 16777216
net.inet.tcp.sendspace = 4194304
net.isr.bindthreads = 1
net.isr.maxthreads = -1

Maybe I added too many buffers?

I also made some changes to the SMB conf on the shares but I don't think that would be blowing everything out of the water like this.

Question is how do I reverse the above tunables from the shell (without the web UI)?

Setup is as follows:
Truenas Core 12.0-U6
Asrock Rack ROMED8-2T, AMD EPYC 7251, 128GB DDR4-3200, LSI 9300-8i
2x Samsung 870 EVO (boot, mirrored)
2x Sabrent Rocket 4TB NVME M.2 (wip, mirrored)
6x Ultrastar 18TB (library, vdev 0, raidz2)
7x Ultrastar 14TB (library, vdev 1, raidz2)
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
Without the UI, you have to use the API. This is best done from PuTTY, with logging turned on.
  1. Run midclt call tunable.query | jq. This will create a long table of tunables, with index IDs.
  2. To delete a tunable, run midclt call tunable.delete <ID to delete>.
 

ianwood

Dabbler
Joined
Sep 27, 2021
Messages
14
Samuel Tai, you are a rockstar! We're back up. I will be more careful with my tunables from now on! Lesson learned.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
Some of your tunables will have reasonable values set for your hardware and memory size by enabling autotune under System->Advanced, and then rebooting.
 
Joined
Mar 5, 2022
Messages
224
Thanks @Samuel Tai, have autotune enabled already.
I disabled all of the RC tunables I had created and enabled them one at a time (rebooting between changes). Turns out the one that was causing the problem is "firewall_enable" -> "yes"
Any idea why this would break my login from a local host (not via open vpn)?
 

Attachments

  • tunables.jpg
    tunables.jpg
    90.9 KB · Views: 130
Last edited:

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
TrueNAS is not intended to have the firewall enabled. By default, the firewall blocks ALL traffic on ALL interfaces. You'd need to enable ALL the ports needed for TrueNAS on the interfaces you expect to use if you enable the firewall.
 
Joined
Mar 5, 2022
Messages
224
Top