How It Works: Automatic Client Approval in Configuration Manager 2007

Purpose:
This post is intended to explain how client approval works when a Mixed Mode Configuration Manager 2007 site is configured to automatically approve clients in trusted domains and to offer insight into how to troubleshoot scenarios where this is not working as expected.  The configuration is shown below:




The short version:

1. The new client performs a CCM_POST to CCM_System_WindowsAuth on the MP.
2. The MP responds with a 401 as the request is anonymous and contains no security data.
3. The client requests a Kerberos ticket for http://MP_FQDN from Active Directory (e.g. http://SCCMMP.Contoso.com).
4. On obtaining the Kerberos ticket, the client performs another CCM_POST including the security data.
5. If the MP accepts the ticket then the client is authenticated and is considered to be trusted.
6. Whether the client is trusted or not, the MP executes the spUpdateClientRegistration stored procedure to update the database. If the client has authenticated properly, both the @ApprovalMethod and @IsIntegratedAuth parameters will be set to 1. If not, they are both set to 0.
   

Technical details:
The following is from data obtained in my lab during a successful test where a client in a child domain (Child.A2003.VM.local) was automatically approved by the MP in Child’s parent domain (A2003.VM.local). Relevant details:

• Client is WK02-020-51W.Child.A2003.VM.local with IP address 192.168.20.102
• The DC for the Child domain is DC03-020-52S with IP address 192.168.20.10
• The MP is SMS4-120-52S.A2003.VM.local with IP address 192.168.120.20
• The DC for the A2003 domain is DC02-110-61E with IP address 192.168.110.10

The IIS log shows:
2009-05-20 19:35:11 W3SVC1 192.168.120.20 CCM_POST /ccm_system_windowsauth/request - 80 - 192.168.20.102 ccmhttp 401 2 2148074254
2009-05-20 19:35:13 W3SVC1 192.168.120.20 CCM_POST /ccm_system_windowsauth/request - 80 CHILD\WK02-020-51W$ 192.168.20.102 ccmhttp 200 0 0
Code 2148074254 or 0x8009030E means SEC_E_NO_CREDENTIALS
MP_RegistrationManager.log shows:
MP Reg: DDR file written to E:\SMS\inboxes\auth\ddm.box\regreq\A8Z82CVO.RDR
Network Monitor shows:
Client issues the CCM_POST with no security information, starting in frame 4:

The MP responds with the HTTP 401 in frame 10:

The client requests a Kerberos ticket for the HTTP SPN of the MP from the Child domain DC in frame 13:

The Child DC refers the client to the A2003 domain DC in frame 14:

The client then makes the same Kerberos request to the A2003 DC in frame 15:

The A2003 DC responds with a Kerberos ticket in frame 16:

The client reissues the CCM_POST, with security information, in frame 17:

The MP responds with an HTTP 200 in frame 25:

SQL Profiler shows the MP executing spUpdateClientRegistration with values of 1 (True) for the relevant parameters so the client is set to Approved in the database (with parameters inlaid):
exec spUpdateClientRegistration (@SiteCode) "2P4", (@SMSID) "GUID:EEAF9390-94EB-43AE-A0DE-F374E3E7E03B", (@CSMSID) NULL, (@Identity) NULL, (@DeviceID) NULL, (@Certificate) 0x308201E03082014DA0030201020210C25D7C383E3CB6AE466BE10C22ACF3AE300906052B0E03021D05003025311530130603550403130C574B30322D3032302D353157310C300A06035504031303534D533020170D3039303531393139333434375A180F32313039303432363139333434375A3025311530130603550403130C574B30322D3032302D353157310C300A06035504031303534D5330819F300D06092A864886F70D010101050003818D0030818902818100BAEB59B8B0ECB451AB82FECF42791740BB3704BD00705126D3DA0FB7618B53955CFF142534B09ACDB97B6B8E42D10663453EA6C35A0BC30A3A83F3381B507A4D051C884E9789DC827223C1C92D0C4712458DAF96A8F15CABE2AB14ED9755A89034657CCB96A7086DFB92E928C35626F08F6A3FA451206850CEC0FC8EC671E09F0203010001A317301530130603551D25040C300A06082B06010401823765300906052B0E03021D050003818100A7A603A7E7AFB19E7518F62F5D4153FA3926AA8348D9EFCB9EC4C03AB991202CB2496B48489C6E6AB48E673ECA6DBED3A4EC7FBE1D3BB70682338FC5E4A86BCEBC4702EDBB41F060AD54117707AFCD0A2FC8AF17DF9E71F1206FBCE9295237322029956D2AA652B41F1258D617BBC4D07410F01CC40042D33BA97511F2B2DDCA, (@Thumbprint) 0xE824658E489FDBB6481ED7788E74877FB9DBCF0B, (@EncKey) 0x308201E13082014EA00302010202106866F6C06283B38644D10B448CFAE966300906052B0E03021D05003025311530130603550403130C574B30322D3032302D353157310C300A06035504031303534D533020170D3039303531393139333434385A180F32313039303432363139333434385A3025311530130603550403130C574B30322D3032302D353157310C300A06035504031303534D5330819F300D06092A864886F70D010101050003818D0030818902818100BC71E0923FBF50300A40C479CBCE5F49D20597C00873C5F6ED16538A41FF5FA54B402905DB424D95B1D67FFFD6245204E4B5FD6A751087BAFD6846FA1272E6F0405EEB647020BB8DFD2E78516C263ECE8831E17CD230C5637AA83AE222B900530977A57F390AD6542C344AB6DD6F8D7ED8CAED76A3F700ED0250ED64481C9D550203010001A318301630140603551D25040D300B06092B0601040182376502300906052B0E03021D0500038181008D63DEC508A14404372B3BEE2AD35D8B44C7C6E2AA8B81A9FA8280D416FFAB67DDDA1F37B981523DA6EEBF66CDE9F7C501412D1438E0FC490C022BC4CA854EFAA975D49182A3300C76577B4BE15B5FCB0014C36E0C418EE98F13AF6264E69C0DCFFFADD1D0800400A15630FA6C023B989D341F33E197885001601CFFC015AA0B,(@EncThumbprint) 0x6B186E7BC2B86B059FAE5E431C6C6CE40A943F3C,(@KeyType) 1, (@PublicKey) 0x06020000002400005253413100040000010001009FE071C68EFCC0CE50682051A43F6A8FF02656C328E992FB6D08A796CB7C653490A85597ED14ABE2AB5CF1A896AF8D4512470C2DC9C1237282DC89974E881C054D7A501B38F3833A0AC30B5AC3A63E456306D1428E6B7BB9CD9AB0342514FF5C95538B61B70FDAD326517000BD0437BB40177942CFFE82AB51B4ECB0B859EBBA, (@ValidFrom) "2009-05-19 19:34:47.000", (@ValidTo) "2109-04-26 19:34:47.000", (@AgentType) 0, (@SMBIOSID) NULL , (@MACAddress) NULL , (@HardwareID1) "2:D4D8AD1963DA464FC3EE60E5212310036AB9EDEC", (@ISVProxyID) NULL , (@AlwaysInternet) 0, (@InternetEnabled) 0, (@Force) 0, (@ApprovalMethod) 1, (@ResolutionMethod) 0, (@IsIntegratedAuth) 1, (@Version) "4.00.6221.1000", (@NetbiosName) "WK02-020-51W", (@FQDName) "WK02-020-51W.Child.A2003.VM.local", (@ManualConflictResolution) 0
Troubleshooting:
Standard troubleshooting of these issues should include:

1. Checking the duplicate GUID report to see if the problem client’s GUID is present. If so, run CCMDelCert on the problem client to force a new GUID to be generated and see if that resolves the issue.
2. If duplicate GUIDs do not apply then step through the following:
   1. Delete the problem client from the central site console and let the delete cascade to child sites, as appropriate.
   2. Set the SMS Agent Host service to manual on the client and reboot it.
   3. Start a network capture on the client (NetCap, NetMon 3.x, Ethereal, etc).
   4. Set the SMS Agent Host service back to Automatic and start it.
   5. Initiate a Discovery Data Collection Cycle via the Control Panel applet.
   6. If the client is re-inserted into the database as not approved then collect the network capture (and all relevant IP details) with the IIS log and MP_RegistrationManager.log from the MP to which the client reports directly for review.
3. If the network capture shows the client does get a Kerberos ticket then the IIS log should contain a Win32 error code indicating why the MP rejected it.
4. If the client does not get a ticket then the response from the DC in the capture should detail why. Kerberos logging on the client, per KB262177, may add some useful information to the System Log in Event Viewer as well. Since Directory Services supports Kerberos, they may be engaged for assistance in determining why no ticket was acquired.

Note: The MP_RegistrationManager.log does not contain much detail by default. Enabling Verbose and Debug CCM logging will add some extra entries which may be helpful.
Known reasons why this will fail:
Duplicate GUIDs:  Duplicate GUIDs can cause a myriad of client data integrity problems including client approval issues. If only a subset of the clients are impacted by the client approval failure then duplicate GUIDs should be investigated.
The HTTP SPN is registered under a user account:  Normally, there is no HTTP SPN for a server so the HOST SPN, which should always be on the computer object, is used in obtaining the Kerberos ticket. If web based services, such as SQL Reporting Services, are running under user context on the MP, and the HTTP SPN is linked to that user account, the Kerberos ticket obtained by the client will be for the same user. When presented to the MP, which runs as Local Service (System), it is rejected because it is linked to the wrong user.
The solution for this is to delete the HTTP SPN from the user object, move the web service running as the user to a different web site using a different port and create a new HTTP SPN to refer to the new port. For example, for a new web site using TCP port 81, the new HTTP SPN, created under the user object in AD, would be similar to:
HTTP://SCCMMP.Contoso.com:81
Reference http://msdn.microsoft.com/en-us/library/aa480609.aspx which states:

SPNs are only created for the HOST service and all built-in services use the HOST SPN. However, this implementation is transparent because built-in names act as an alias to the HOST service unless they have been specifically mapped to a Windows account.

And also:

When you use Windows Integrated Security, both Internet Explorer and IIS use the HTTP SPN to request service tickets and to process a request. As a result, when you use a domain user account in IIS 6.0 as the process identity, you must map the host-based HTTP SPN to the domain account that is used by the service.

Client and MP in different domains only share an external trust:
As is noted above, Kerberos is required for the client to authenticate to the MP. While Kerberos *may* work across an external domain trust, it is not supported. It is only supported across a forest trust between two Windows Server 2003 mode (or higher) forests.
Reference http://technet.microsoft.com/en-us/library/cc773178(WS.10).aspx which states:

When two Windows Server 2003 forests are connected by a forest trust, authentication requests made using the Kerberos V5 or NTLM protocols can be routed between forests to provide access to resources in both forests.

Anything else that causes Kerberos to fail:
Kerberos can fail for many reasons including time skews in the environment; DNS name resolution failures, etc. Generally, network capture data will show why it is failing though Kerberos logging per KB262177 can also be helpful.
More Information:
General information on Client Approval can be found at http://technet.microsoft.com/en-us/library/bb694193.aspx.