Samba kinit OK but "net ads join" fails. Cached credentials issue?

[Ceri Hopkins]

Hi all

I am failing to join a HP-UX 11.31 host to an AD domain with an error not found elsewhere in my searches. Any pointers of where to look next would be appreciated.

The "net ads join" fails just before a Service Ticket would be requested. The "net" command seems to have trouble accessing cached credentials at this point - despite the prior debug suggesting all is right with the setup of Kerberos, smb.conf and initial interaction with the AD DC.

The debug ends as follows (the full output is attached) ...

...
[2009/08/06 16:20:47, 3] libads/sasl.c:ads_sasl_spnego_bind(291)
ads_sasl_spnego_bind: got OID=1 3 6 1 4 1 311 2 2 10
[2009/08/06 16:20:47, 3] libads/sasl.c:ads_sasl_spnego_bind(300)
ads_sasl_spnego_bind: got server principal name = tcs01addc$@TCSTEST.CO.UK
[2009/08/06 16:20:47, 3] libsmb/clikrb5.c:ads_krb5_mk_req(592)
ads_krb5_mk_req: krb5_cc_get_principal failed (No credentials cache found)
[2009/08/06 16:20:47, 10] libads/sasl.c:ads_sasl_spnego_bind(320)
ads_sasl_spnego_krb5_bind failed with: No credentials cache found, calling kinit
[2009/08/06 16:20:47, 10] libads/kerberos.c:kerberos_kinit_password_ext(89)
kerberos_kinit_password: using [MEMORY:net_ads] as ccache and config [/var/opt/samba/locks/smb_krb5/krb5.conf.TCSTEST]
[2009/08/06 16:20:47, 3] libsmb/clikrb5.c:ads_krb5_mk_req(592)
ads_krb5_mk_req: krb5_cc_get_principal failed (No credentials cache found)
[2009/08/06 16:20:47, 0] libads/sasl.c:ads_sasl_spnego_bind(328)
kinit succeeded but ads_sasl_spnego_krb5_bind failed: No credentials cache found
[2009/08/06 16:20:47, 1] utils/net_ads.c:net_ads_join(1470)
error on ads_startup: No credentials cache found
Failed to join domain: NT_STATUS_NO_SUCH_FILE
[2009/08/06 16:20:47, 2] utils/net.c:main(1082)
return code = -1

At the same point in the dialog on my test rig (running same CIFS server version but on 11.11 and the corresponding Kerberos and LdapUx packages) it carries on to get a service ticket and succeeds...

...
[2009/08/09 18:25:33, 3] libads/sasl.c:ads_sasl_spnego_bind(291)
ads_sasl_spnego_bind: got OID=1 3 6 1 4 1 311 2 2 10
[2009/08/09 18:25:33, 3] libads/sasl.c:ads_sasl_spnego_bind(300)
ads_sasl_spnego_bind: got server principal name = hoy$@SANDC.LOCAL
[2009/08/09 18:25:33, 3] libsmb/clikrb5.c:ads_krb5_mk_req(592)
ads_krb5_mk_req: krb5_cc_get_principal failed (No credentials cache found)
[2009/08/09 18:25:33, 10] libads/sasl.c:ads_sasl_spnego_bind(320)
ads_sasl_spnego_krb5_bind failed with: No credentials cache found, calling kin
it
[2009/08/09 18:25:33, 10] libads/kerberos.c:kerberos_kinit_password_ext(89)
kerberos_kinit_password: using [MEMORY:net_ads] as ccache and config [/var/opt
/samba/locks/smb_krb5/krb5.conf.SANDC]
[2009/08/09 18:25:33, 3] libsmb/clikrb5.c:ads_cleanup_expired_creds(526)
ads_cleanup_expired_creds: Ticket in ccache[MEMORY:net_ads] expiration Mon, 10
Aug 2009 04:25:33 BST
[2009/08/09 18:25:33, 10] libsmb/clikrb5.c:ads_krb5_mk_req(621)
ads_krb5_mk_req: Ticket (hoy$@SANDC.LOCAL) in ccache (MEMORY:net_ads) is valid
until: (Mon, 10 Aug 2009 04:25:33 BST - 1249874733)
[2009/08/09 18:25:33, 10] libsmb/clikrb5.c:get_krb5_smb_session_key(735)
Got KRB5 session key of length 16
[2009/08/09 18:25:33, 10] lib/gencache.c:gencache_del(173)
Deleting cache entry (key = NBT/SANDC.LOCAL#1C)
...
[ on to successful completion ]

Any ideas/guesses of where to go from here would be much appreciated. The AD server's event logs show the usual output up to the successful request for the Authentication Ticket (event id 672). After that I don't see the event id 673 - Service Ticket Request - that I get on the test rig, but nor do I see any other errors.

Regards
Ceri Hopkins

I have used a limited permission account as documented by HP - but please note that I get exactly the same problem when the Administrator account of the DC was used.

Software stack is:

CIFS-CFSM A.02.04 HP CIFS File System Module

CIFS-Client A.02.02.02 HP CIFS Client

CIFS-Development A.02.04 HP CIFS Server Source Code Files

CIFS-Server A.02.04 HP CIFS Server (Samba) File and Print Services

KRB5-Client B.11.31 Kerberos V5 Client Version 1.3.5.03
krb5client E.1.6.2.03 Kerberos V5 Client Version 1.6.2.03

LdapUxClient B.04.20 LDAP-UX Client Services

Kerberos configuration seems to be OK with kinit checking out.

# cat /etc/krb5.conf
[libdefaults]
default_realm = TCSTEST.CO.UK
default_tkt_enctypes = DES-CBC-MD5
default_tgs_enctypes = DES-CBC-MD5
ccache_type = 2

[realms]
TCSTEST.CO.UK = {
kdc = tcs01addc.tcstest.co.uk:88
}

[domain_realm]
.tcstest.co.uk = TCSTEST.CO.UK

[logging]
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmin.log
default = FILE:/var/log/krb5lib.log

# kinit CIFSAdmin@TCSTEST.CO.UK
# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: CIFSAdmin@TCSTEST.CO.UK

Valid starting Expires Service principal
08/06/09 14:16:41 08/07/09 00:16:41 krbtgt/TCSTEST.CO.UK@TCSTEST.CO.UK

[Steven E. Protter]

Shalom,

You may not like this but:

Try in smb.conf

security = domain

Have the system removed from the domain, there may be a record in the ADS domain controller.

Then try the join again, same syntax.

SEP

[Ceri Hopkins]

Thanks. I will try the suggestion (security = domain) on Wednesday of this week and report back. Is there any particular reason/suspicion to go this way - or is it just to test out an alternative?

[Steven E. Protter]

There are possible issues.

1) It may not work.
2) Your integration may not work as expected.

This may be due to Domain configuration, or your configuration.

Have a short test plan ready when you make the switch to make sure all functionality you need is present.

SEP

[UNIXGRUPPEN]

I'm experiencing the exact same issue!

HP-UX B.11.23 ia64
B8725AA A.02.04 HP CIFS Server

Domain works but ADS fails with the same message.

Did you solve it?

Regards,
Johan

[Ceri Hopkins]

Thanks for the suggestions. There was a bit of a delay getting back to the client site to action this - but here is my forum feedback for the interested.

My Plan B was "security=domain" as suggested, but I skipped straight to Plan C - using an older software stack - as there was a problem booting the 11.31 test server I had previously configured.

With the same AD test domain and users (no changes) and the previous configuration, all worked as expected with the following software from the Sept 2008 install media.

CIFS-Server A.02.03.03 HP CIFS Server (Samba) File and Print
KRB5-Client B.11.31 Kerberos V5 Client Version 1.3.5.03
LdapUxClient B.04.17 LDAP-UX Client Services

This works just fine.

I then decided to upgrade to A.02.04 to test out my previous failure and managed to replicate the output described in the original post. Unfortunately I didn't have enough time for a thorough investigation as I needed to back out and leave the client with a working configuration. Access to a test rig should help someone tease out the problems.

Suspecting the original problem to be Kerberos related I first installed A.02.04 without updating Kerberos to E.1.6.02. Ok, so the documentation says this is a requirement on 11.31 and no surprise when winbindd and smbd dies with errors such as:

/usr/lib/hpux32/dld.so: Unsatisfied code symbol 'krb5_get_init_creds_opt_alloc'

I installed KRB5CLIENT bundle for E.1.6.02 and all daemons seemed to start up OK -- hmm. Note that I have valid cached credentials from my previous work with A.02.03.03.

# klist -e
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: CIFSAdmin@TCSTEST.CO.UK

Valid starting Expires Service principal
09/02/09 15:07:24 09/03/09 01:07:24 krbtgt/TCSTEST.CO.UK@TCSTEST.CO.UK
Etype (skey, tkt): DES cbc mode with RSA-MD5, ArcFour with HMAC/md5
09/02/09 15:08:14 09/03/09 01:07:24 tcs01addc$@TCSTEST.CO.UK
Etype (skey, tkt): DES cbc mode with RSA-MD5, ArcFour with HMAC/md5


To replicate my previous experience I then decided to kdestroy my credentials and delete the computer definition in AD and I ended back at the the same point as described in the original posting - not being able to join the AD domain with:

Failed to join domain: NT_STATUS_NO_SUCH_FILE

Now, reverting to CIFS-Server A.02.03.03 and attempting to rejoin I got the following slightly different error.

[2009/09/02 15:19:12, 3] libsmb/clikrb5.c:ads_krb5_mk_req(478)
ads_krb5_mk_req: krb5_cc_get_principal failed (No credentials cache found)
[2009/09/02 15:19:12, 3] libsmb/clikrb5.c:ads_krb5_mk_req(478)
ads_krb5_mk_req: krb5_cc_get_principal failed (No credentials cache found)
[2009/09/02 15:19:12, 0] utils/net_ads.c:ads_startup(191)
ads_connect: No credentials cache found
[2009/09/02 15:19:12, 2] utils/net.c:main(890)
return code = -1

I ended up removing krb5client product for E.1.6.02 (leaving PHSS_39766 cumulative patch in place) and sticking with A.02.03.03 - albeit with an updated LDAPUX.

CIFS-Server A.02.03.03 HP CIFS Server (Samba) File and Print
KRB5-Client B.11.31 Kerberos V5 Client Version 1.3.5.03
PHSS_39766 1.0 KRB5-Client Version 1.3.5.03 Cumulative patch
1.6.2.03
LDAPUX LdapUxClient B.04.20 LDAP-UX Client Services

I hope this might help someone with access to a test rig tease out where a problem might lie setting up from scratch on 11.31 with Kerberos E.1.6.02 and A.02.04.

I would have liked to follow this through myself but no longer access to the relevant hardware.

I never needed to try Plan B, but am sure it would have worked!

[Court Campbell]

I am having the same issue, but I did want to note that the cifs-server docs state that you need to use krb5client version E.1.6.2. It says that version E.1.6.2.03 will not work. I am currently trying to get that version and see what happens.

[Court Campbell]

Got the depot. Worked like a charm.

KRB5CLIENT E.1.6.2 Kerberos V5 Client Version 1.6.2
PHSS_37666 1.0 KRB5-Client Version 1.3.5.03 Cumulative patch

[Ceri Hopkins]

Hi Court

Good work!

I've looked at Edition 12 of the Administrators Guide which say throughout that "E.1.6.2 or later" should work. I can find no reference to E.1.6.2.03 not working.

Can you please provide a reference to the exact document/version you found this is? Was it inside the depot?

I clearly missed this and can't be the only one who's main reference was the admin guide and release notes.

HP Part Number: B8725-90143
Published: May 2009
Edition: Edition 12

Thanks.

[eric roseme]

Sorry for the late notice on this - dunno how I missed it.

A bug was introduced into the krb5 client distribution that broke CIFS Server (Samba) interoperability with Windows domain joins (as you have found out).

The CIFS Server download site states this at: https://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=B8725AA

I also updated the CIFS Server Unified Login document for W2008, and put a warning in there on page 8: http://www.docs.hp.com/en/16212/CIFSUnifiedLoginV2.pdf

I need to update the CIFS Server Kerberos whitepaper too, and will add a similar warning.

Sorry for the delay on posting kbr5 client version 1.6.2.04 - hopefully in will be on software.hp.com in a couple of weeks. Outta my hands.

Eric Roseme

[Mike Dunphy_1]

Hello, I rolled back to KRB5CLIENT E.1.6.2 on HP-UX 11iv3 as the note suggested for
A.02.04

However looks like I have a more serious problem now that smbd will not even start

/usr/lib/dld.sl: Unresolved symbol: krb5_get_init_creds_opt_alloc (code) from /
opt/samba/bin/smbd

Any ideas ?

[Court Campbell]

https://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=B8725AA

Note: KRB5CLIENT D.1.6.2.01 on HP-UX 11iv2 and KRB5CLIENT E.1.6.2.03 on HP-UX 11iv3 are not compatible with HP CIFS Server. HP recommends that you use KRB5CLIENT D.1.6.2 on HP-UX 11iv2 and KRB5CLIENT E.1.6.2 on HP-UX 11iv3 with HP CIFS Server. To download these KRB5CLIENT versions, contact HP support center.