Active Directory Services "missing"?

[EBelcourt]

Hi Team,

I'm building a new Freenas system on a SuperMicro 12 bay SAS2 HBA setup. After finally getting past some issues with memory it is now installed, seeing all disks and ready to start configuration. However, I have encountered a problem with Active Directory.

When I attempt to join the domain, the UI shows as "joined" by presenting me with a leave domain button but when you navigate out and back in, it disappears. A Kerberos realm appears for the domain i've "joined" it to as well as in the /etc/krb5.conf I get the following (domain obfuscated):

Code:

#
# krb5.conf(5) - configuration file for Kerberos 5
# $FreeBSD$
#

[appdefaults]
            pam = {
                   forwardable = true
                   ticket_lifetime = 86400
                   renew_lifetime = 86400
            }

[libdefaults]
            dns_lookup_realm = true
            dns_lookup_kdc = true
            ticket_lifetime = 24h
            clockskew = 300
            forwardable = true
            default_realm = DOMAIN.LOCAL

[domain_realm]
            domain.local = DOMAIN.LOCAL
            .domain.local = DOMAIN.LOCAL
            DOMAIN.LOCAL = DOMAIN.LOCAL
            .DOMAIN.LOCAL = DOMAIN.LOCAL

[realms]
            DOMAIN.LOCAL = {
                   default_domain = DOMAIN.LOCAL
            }

[logging]
            default = SYSLOG:INFO:LOCAL7

However, when I do a wbinfo -t I get:

Code:

root@freenas[/etc/directoryservice]# wbinfo -t
could not obtain winbind interface details: WBC_ERR_WINBIND_NOT_AVAILABLE
could not obtain winbind domain name!
checking the trust secret for domain (null) via RPC calls failed
failed to call wbcCheckTrustCredentials: WBC_ERR_WINBIND_NOT_AVAILABLE
Could not check secret

When I check the logs in /var/log/samba4/log.winbindd I get the following messages:

Code:

  winbindd version 4.10.12 started.
  Copyright Andrew Tridgell and the Samba Team 1992-2019
[2020/02/09 22:51:27.970959,  0] ../../source3/winbindd/winbindd_cache.c:3166(initialize_winbindd_cache)
  initialize_winbindd_cache: clearing cache and re-creating with version number 2
[2020/02/09 22:51:27.971978,  0] ../../source3/winbindd/winbindd_util.c:1257(init_domain_list)
  Could not fetch our SID - did we join?
[2020/02/09 22:51:27.972007,  0] ../../source3/winbindd/winbindd.c:1462(winbindd_register_handlers)

I thought this very odd, so I tried restarting/manipulating the service or trying to read the configuration file(s) from /etc/directoryservice/, I got a not found error so I went to the directory annndddd...there's nothing there?

Zip, zada, zilch.

As far as I can tell, it was a pretty nominal install of 11.2 and upgraded to 11.3, our directory appears healthy from all the checks I've done and there's nothing Fruitcake-e I can tell about it.

Wonder if anyone has seen this before?

(my Google/search fu did not bring up any relatable threads to my issue)

Many thanks in advance.

 

[anodos]

What is the output of "testparm -s"?

 

[EBelcourt]

Hello Anodos,

The results were (obfuscated in italics):

Code:

root@freenas[~]# testparm -s
Load smb config files from /usr/local/etc/smb4.conf
Loaded services file OK.
Server role: ROLE_DOMAIN_MEMBER

# Global parameters
[global]
        aio max threads = 2
        bind interfaces only = Yes
        disable spoolss = Yes
        dns proxy = No
        domain master = No
        enable web service discovery = Yes
        interfaces = 127.0.0.1 172.16.10.48
        kerberos method = secrets and keytab
        kernel change notify = No
        load printers = No
        local master = No
        logging = file
        max log size = 51200
        netbios name = STOR-01
        nsupdate command = /usr/local/bin/samba-nsupdate -g
        preferred master = No
        realm = DOMAIN.LOCAL
        restrict anonymous = 2
        security = ADS
        server min protocol = SMB2_02
        server role = member server
        server string = Storage Server
        template shell = /bin/sh
        unix extensions = No
        username map = /usr/local/etc/smbusername.map
        username map cache time = 60
        winbind cache time = 7200
        winbind enum groups = Yes
        winbind enum users = Yes
        winbind max domain connections = 10
        winbind nss info = sfu
        winbind use default domain = Yes
        workgroup = DOMAIN
        idmap config domain: range = 100000001-200000000
        idmap config domain: backend = rid
        idmap config *: range = 90000001-100000000
        idmap config * : backend = tdb
        allocation roundup size = 0
        directory name cache size = 0
        dos filemode = Yes
        include = /usr/local/etc/smb4_share.conf


[Backups]
        aio write size = 0
        ea support = No
        mangled names = illegal
        path = /mnt/Staging
        read only = No
        vfs objects = shadow_copy_zfs ixnas streams_xattr
        nfs4:acedup = merge
        nfs4:chown = true

I mean, it looks ok for the other linux systems i've seen, but I have an untrained eye.

Thanks

 

[anodos]

Can you PM me the contents of /var/log/middlewared.log

 

[EBelcourt]

PMed you with the Middleware logs.

Thank you!

 

[anodos]

I thought this very odd, so I tried restarting/manipulating the service or trying to read the configuration file(s) from /etc/directoryservice/, I got a not found error so I went to the directory annndddd...there's nothing there?

That's expected because we don't write anything there.

You can check state of directory services by issuing command 'midclt call directoryservices.get_state'.
You can stop the AD service by issuing command 'midclt call activedirectory.stop', and start it again via 'midclt call activedirectory.start'.
If wbinfo -t fails, you can run following commands
midclt call activedirectory.check_clockskew to check time difference with remote DC.
midclt call activedirectory.domain_info to verify your domain information.
midclt call activedirectory.validate_credentials will validate the bind credentials provided to us.

If you have properly joined AD, you should see a kerberos keytab in the 'ktutil list' output, and a kerberos ticket in 'klist' output. If these are present, you can also run command net -k ads status to verify that you have a computer object in AD. If this looks correct, and winbindd still returns error, check its status "service winbindd status". If winbindd is running and still responding with that error, then there's a trust issue with domain. net -k ads leave.
midclt call activedirectory.update '{"enable": false}' and then midclt call activedirectory.update '{"enable": true}'

 

[EBelcourt]

Hello Anodos,

That's expected because we don't write anything there.

Ahh, ok, must have been looking at old articles then. So I don't believe the server ever joined the domain because I did not have any objects in AD and it's confirmed by the lack of keytabs:

Code:

root@freenas[/var/log]# ktutil list
ktutil: krb5_kt_start_seq_get FILE:/etc/krb5.keytab: keytab /etc/krb5.keytab open failed: No such file or directory

Checking service status:

Code:

root@freenas[/var/log]# midclt call directoryservices.get_state
{"activedirectory": "FAULTED", "ldap": "DISABLED", "nis": "DISABLED"}

Clocks were within a tolerance of less than a second:

Code:

root@freenas[/var/log]# midclt call activedirectory.check_clockskew
{"pdc": "dc.domain.com", "timestamp": "2020-02-10 22:18:12.060151", "clockskew": "0:00:00.000334"}

Domain info was broken saying it couldn't find any logon servers after showing it just briefly beforehand (tried restarting service) but then I re-entered the details in the GUI and it's back to (obfuscated):

Code:

root@freenas[/var/log]# midclt call activedirectory.domain_info
{"LDAP server": "###.##.10.10", "LDAP server name": "dc.domain.com", "Realm": "DOMAIN.COM", "Bind Path": "dc=DOMAIN,dc=COM", "LDAP port": 389, "Server time": 1581373249, "KDC server": "###.##.10.10", "Server time offset": 0, "Last machine account password change": 0}

But when I do a credential validation, it fails. I know the password and username is correct, I even have temporarily changed the password a string of just letters and numbers in case special characters are at fault:

Code:

root@freenas[/var/log]# midclt call activedirectory.validate_credentials
[EFAULT] kinit for domain [DOMAIN.COM] with password failed: kinit: Password incorrect

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 130, in call_method
    io_thread=False)
  File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1091, in _call
    return await run_method(methodobj, *args)
  File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1015, in _run_in_conn_threadpool
    return await self.run_in_executor(self.__ws_threadpool, method, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/middlewared/main.py", line 1003, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/activedirectory.py", line 1076, in validate_credentials
    f"kinit for domain [{ad['domainname']}] with password failed: {kinit.stderr.decode()}"
middlewared.service_exception.CallError: [EFAULT] kinit for domain [DOMAIN.COM] with password failed: kinit: Password incorrect

There are no kerberos tickets either.

Could it be related to the cert errors in the middleware logs I sent over?

Many thanks!

Emile

 

[EBelcourt]

One thing to add, I cannot actually save the Directory config without a certificate in use, even though I have encryption and "validate certificates" disabled:

(I'm guessing this is where the middleware errors are coming from)

 

[naqu3]

Hello Emile,

Did you manage any headway on this? I encountered the same issues on one of our systems (11.3 U2), the first one to access AD.

Cheers,

Nich

 

[Oskar_815]

Hi,

I'm having similar problems. I just cannot join AD domain. It shows the button "Leave domain" just after I try to join, but if I reenter the page it doesn't show anything. Winbindd is not running, winbind -i says no sid is present (so it never joined the domain). Already checked usual stuff such as clock skew, permissions etc. (the AD user I use to join freenas to AD is a domain admin), but nothing seems to work.
I'm using freenas latest version and a win server 2019 domain controller.
If I don't sort this out I definitely won't use it at the company I work for (so far everything is in a home test environment, in virtual machines).

 

[garyg15]

I am having the exact same issue reported above. Anyone figure out a solution? Also my /etc/directoryservices/ActiveDirectory directory is missing. I am running 11.3 and just updated today.

 

[A71914]

I also have issues connecting to AD.
Running FreeNAS 11.3-U1
Windbindd is not running
I solved my problem by disabling Local Master in Services/SMB

 

[dave93cab]

Same issue here..

 

[anodos]

Same issue here..

What issue? The directory doesn't exist in 11.3 because directory services were rewritten.

 

[godhanuman]

Hello,
I have in production a freenas 11.2 I would like to update it in truenas.
i have a local active directory with all the configuration. I tried to upgrade to freenas 11.3 and the active directory stops working.
If the service has been rewritten how can I upgrade without losing the active directory? i have 100 users above. Thank you