SSH/SFTP - Can't log in: Access denied for user testuser by PAM account configuration

Joined
Apr 9, 2021
Messages
9
Hello,
Hope this post finds you well.

Following the docs, I've enabled the SSH service, created a home dir in a dedicated dataset (i.e. /mnt/mypool/client_sftp/testuser), created a user & group (testuser:testuser) pointing to that dir for home, and finally chown/chmod their home dir to testuser:testuser & 750 respectively. However, when connecting with Filezilla or SSH, it downloads the public key but immediately after throws, "FATAL ERROR: Remote side unexpectedly closed network connection" (SSH behaves similarly).

Then I tried following this guide which is nearly identical to what I did above, except uses a sub-dataset as user's home dir instead of manually creating a directory. This results in the same errors.

For kicks, I tried adding the user to wheel & sshd but predictably made no difference.

Later in this thread, you'll see I discovered that auth.log shows:

Jun 23 00:07:54 truenas 1 2023-06-23T00:07:54.515551-04:00 truenas.mydomain.com sshd 56541 - - fatal: Access denied for user testuser by PAM account configuration [preauth]

I am stuck. It's probably something incredibly obvious but I'm googling myself in circles at this point.

Thank you for reading & for any ideas!
 
Last edited:
Joined
Oct 22, 2019
Messages
3,641
What are the ownership and permissions of the following on your server:
Code:
ls -ld /mnt/mypool/client_sftp/testuser
ls -ld /mnt/mypool/client_sftp/testuser/.ssh
ls -ld /mnt/mypool/client_sftp/testuser/.ssh/authorized_keys
 
Last edited:
Joined
Apr 9, 2021
Messages
9
What are the ownership and permissions of the following on your server:
Code:
ls -ld /mnt/mypool/client_sftp/testuser
ls -ld /mnt/mypool/client_sftp/testuser/.ssh
ls -ld /mnt/mypool/client_sftp/testuser/.ssh/authorized_keys
Thanks for the quick reply!

/mnt/mypool/client_sftp/testuser/.ssh did not yet exist. I generated it from the TrueNAS shell thusly:

# su - testuser $ ssh anotheruser@anothersshserver

Once I logged in, testuser now has a .ssh directory.

After which, your commands yield the following:

$ stat /mnt/mypool/client_sftp/testuser 16626894957324076873 34 drwxr-x--- 3 testuser testuser 18446744073709551615 10 "Jun 22 21:51:17 2023" "Jun 22 23:26:36 2023" "Jun 22 23:26:36 2023" "Jun 22 21:51:17 2023" 4096 38 0x800 /mnt/mypool/client_sftp/testuser $ stat /mnt/mypool/client_ftp/testuser/.ssh 16626894957324076873 128 drwx------ 2 testuser testuser 18446744073709551615 3 "Jun 22 23:26:36 2023" "Jun 22 23:29:27 2023" "Jun 22 23:29:27 2023" "Jun 22 23:26:36 2023" 131072 1 0x800 /mnt/mypool/client_ftp/testuser/.ssh

The authorized_keys file does not exist, because I am not (and do not plan to) use key authentication - only interested in password authentication for now.

Otherwise, seems correct to me and also I've never had to do any of this on any other *nix OS after creating a new user - it always just gets sorted, though I have had situations where I inadvertently broke it by recursively changing permissions. This does not seem to be the case in this situation, though...I was hoping it would be that silly :(
 
Last edited:
Joined
Oct 22, 2019
Messages
3,641
Then you can get more hints from the client by invoking -vvv in your ssh command.

Code:
ssh -vvv testuser@server.address
 
Joined
Apr 9, 2021
Messages
9
$ ssh -vvv testuser@172.17.50.243 OpenSSH_8.9p1 Ubuntu-3ubuntu0.1, OpenSSL 3.0.2 15 Mar 2022 debug1: Reading configuration data /etc/ssh/ssh_config debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files debug1: /etc/ssh/ssh_config line 21: Applying options for * debug2: resolve_canonicalize: hostname 172.17.50.243 is address debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/statik/.ssh/known_hosts' debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/statik/.ssh/known_hosts2' debug3: ssh_connect_direct: entering debug1: Connecting to 172.17.50.243 [172.17.50.243] port 22. debug3: set_sock_tos: set socket 3 IP_TOS 0x10 debug1: Connection established. debug1: identity file /home/statik/.ssh/id_rsa type -1 debug1: identity file /home/statik/.ssh/id_rsa-cert type -1 debug1: identity file /home/statik/.ssh/id_ecdsa type -1 debug1: identity file /home/statik/.ssh/id_ecdsa-cert type -1 debug1: identity file /home/statik/.ssh/id_ecdsa_sk type -1 debug1: identity file /home/statik/.ssh/id_ecdsa_sk-cert type -1 debug1: identity file /home/statik/.ssh/id_ed25519 type -1 debug1: identity file /home/statik/.ssh/id_ed25519-cert type -1 debug1: identity file /home/statik/.ssh/id_ed25519_sk type -1 debug1: identity file /home/statik/.ssh/id_ed25519_sk-cert type -1 debug1: identity file /home/statik/.ssh/id_xmss type -1 debug1: identity file /home/statik/.ssh/id_xmss-cert type -1 debug1: identity file /home/statik/.ssh/id_dsa type -1 debug1: identity file /home/statik/.ssh/id_dsa-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.1 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.8-hpn14v15 debug1: compat_banner: match: OpenSSH_8.8-hpn14v15 pat OpenSSH* compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug1: Authenticating to 172.17.50.243:22 as 'testuser' debug3: record_hostkey: found key type ED25519 in file /home/statik/.ssh/known_hosts:1 debug3: load_hostkeys_file: loaded 1 keys from 172.17.50.243 debug1: load_hostkeys: fopen /home/statik/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug3: order_hostkeyalgs: have matching best-preference key type ssh-ed25519-cert-v01@openssh.com, using HostkeyAlgorithms verbatim debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent debug3: receive packet: type 20 debug1: SSH2_MSG_KEXINIT received debug2: local client KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,sntrup761x25519-sha512@openssh.com,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c debug2: host key algorithms: ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256 debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,zlib@openssh.com,zlib debug2: compression stoc: none,zlib@openssh.com,zlib debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug2: peer server KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519 debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,none debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,none debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none debug2: compression stoc: none debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ssh-ed25519 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none debug3: send packet: type 30 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug3: receive packet: type 31 debug1: SSH2_MSG_KEX_ECDH_REPLY received debug1: Server host key: ssh-ed25519 SHA256:J9fo8V+EkAcj9KvqFYIyqqrq8QL2M29LqJmDQLLzUzc debug3: record_hostkey: found key type ED25519 in file /home/statik/.ssh/known_hosts:1 debug3: load_hostkeys_file: loaded 1 keys from 172.17.50.243 debug1: load_hostkeys: fopen /home/statik/.ssh/known_hosts2: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory debug1: Host '172.17.50.243' is known and matches the ED25519 host key. debug1: Found key in /home/statik/.ssh/known_hosts:1 debug3: send packet: type 21 debug2: ssh_set_newkeys: mode 1 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug3: receive packet: type 21 debug1: SSH2_MSG_NEWKEYS received debug2: ssh_set_newkeys: mode 0 debug1: rekey in after 134217728 blocks debug1: get_agent_identities: bound agent to hostkey debug1: get_agent_identities: ssh_fetch_identitylist: agent contains no identities debug1: Will attempt key: /home/statik/.ssh/id_rsa debug1: Will attempt key: /home/statik/.ssh/id_ecdsa debug1: Will attempt key: /home/statik/.ssh/id_ecdsa_sk debug1: Will attempt key: /home/statik/.ssh/id_ed25519 debug1: Will attempt key: /home/statik/.ssh/id_ed25519_sk debug1: Will attempt key: /home/statik/.ssh/id_xmss debug1: Will attempt key: /home/statik/.ssh/id_dsa debug2: pubkey_prepare: done debug3: send packet: type 5 debug3: receive packet: type 7 debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com> debug3: receive packet: type 6 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug1: No credentials were supplied, or the credentials were unavailable or inaccessible No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000) debug1: No credentials were supplied, or the credentials were unavailable or inaccessible No Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000) debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: /home/statik/.ssh/id_rsa debug3: no such identity: /home/statik/.ssh/id_rsa: No such file or directory debug1: Trying private key: /home/statik/.ssh/id_ecdsa debug3: no such identity: /home/statik/.ssh/id_ecdsa: No such file or directory debug1: Trying private key: /home/statik/.ssh/id_ecdsa_sk debug3: no such identity: /home/statik/.ssh/id_ecdsa_sk: No such file or directory debug1: Trying private key: /home/statik/.ssh/id_ed25519 debug3: no such identity: /home/statik/.ssh/id_ed25519: No such file or directory debug1: Trying private key: /home/statik/.ssh/id_ed25519_sk debug3: no such identity: /home/statik/.ssh/id_ed25519_sk: No such file or directory debug1: Trying private key: /home/statik/.ssh/id_xmss debug3: no such identity: /home/statik/.ssh/id_xmss: No such file or directory debug1: Trying private key: /home/statik/.ssh/id_dsa debug3: no such identity: /home/statik/.ssh/id_dsa: No such file or directory debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password testuser@172.17.50.243's password: debug3: send packet: type 50 debug2: we sent a password packet, wait for reply Connection closed by 172.17.50.243 port 22
 
Joined
Apr 9, 2021
Messages
9
Appears to be rejecting my password.

From auth.log:

Jun 23 00:07:54 truenas 1 2023-06-23T00:07:54.515551-04:00 truenas.mydomain.com sshd 56541 - - fatal: Access denied for user testuser by PAM account configuration [preauth]

I've reset the password a few times just to be sure. I did it again after seeing this log. No dice.
 
Joined
Oct 22, 2019
Messages
3,641
Did you veer from any defaults in the configuration (via the GUI) of Services -> SSH ?

Is "Allow password authentication" enabled?

Are you using any auxiliary parameters?
 
Joined
Apr 9, 2021
Messages
9
Did you veer from any defaults in the configuration (via the GUI) of Services -> SSH ?

Yes, I also checked Kerberos Authentication because I have an Active Directory server, mainly for use with Samba shares but figured there would be no harm in allowing it for SSH and maybe would want to use that integration in the future.

Have NOT changed anything under Advanced Options.

To be sure, I disabled Kerberos and it still rejects password authentication.
 
Joined
Oct 22, 2019
Messages
3,641
Does this issue exist with a different client PC to test this on?

(Not sure if you've been trying from different computers.)
 
Top