Red Hat Enterprise Linux Diagnostics and Troubleshooting
Resolve an issue related to LDAP user information and Kerberos authentication.
Outcomes
You should be able to diagnose and resolve issues with LDAP user information and Kerberos authentication.
As the student user on the workstation machine, use the lab command to prepare your system for this exercise.
This command installs and configures the IdM client on servera for use in this exercise.
[student@workstation ~]$ lab start security-ldap
Instructions
The servera machine is a client to the IdM server for LDAP and Kerberos KDC services for user authentication. At your organization, remote users log in to the servera client machine with SSH keys instead of passwords. User home directories automatically mount at login.
The operator1 user reported that when remotely logging in to the servera machine, the connection prompts for a password and either times out after entering a password or immediately disconnects.
The system administrator provided you with the necessary information to troubleshoot the issue. Use RedHat123^ as the password for the operator1 and admin user accounts. Client systems and accounts are in the lab.example.com domain that the IdM server supports.
Re-create the issue.
From the
workstationmachine, attempt to log in to theoperator1account on theserveramachine. The password prompt means that authentication is not working as expected and further investigation is necessary. Press Ctrl+C at any time to exit the authentication process.The expected failure should look like this example:
[student@workstation ~]$
ssh operator1@serveraoperator@servera's password:RedHat123^Connection to servera closed by remote host. Connection to servera closed. [student@workstation ~]$The failure might also appear as multiple password attempts until disconnection.
[student@workstation ~]$
ssh operator1@serveraPassword:RedHat123^Password:RedHat123^Password:RedHat123^operator1@servera's password:RedHat123^Permission denied, please try again. operator1@servera's password:RedHat123^Received disconnect from 172.25.250.10 port 22:2: Too many authentication failures Disconnected from 172.25.250.10 port 22 [student@workstation ~]$Log in to the
serveramachine and switch to therootuser.The
studentaccount succeeds, which indicates that the IdM server is functioning, and that the issue might be limited to theoperator1user.[student@workstation ~]$
ssh student@servera...output omitted... [student@servera ~]$sudo -i[sudo] password for student:student[root@servera ~]#
Start the troubleshooting process by verifying the IdM server's Kerberos functions.
Verify that the IdM service is providing Kerberos tickets.
Use the
kinit admincommand to authenticate and obtain Kerberos credentials. UseRedHat123^as the password.[root@servera ~]#
kinit adminPassword for admin@LAB.EXAMPLE.COM:RedHat123^[root@servera ~]#Verify that the Kerberos ticket is created.
The server's LDAP and Kerberos mechanisms appear to be functioning normally.
[root@servera ~]#
klistDefault principal: admin@LAB.EXAMPLE.COM Valid starting Expires Service principal 11/19/2021 00:12:27 11/19/2021 23:18:24 krbtgt/LAB.EXAMPLE.COM@LAB.EXAMPLE.COM
Next, diagnose SSSD operations.
Clear the SSSD log files on the
serveramachine and attempt to log in as theoperator1user to generate fresh logs. Search the log files for possible errors.Enable detailed SSSD debug logging.
[root@servera ~]#
sssctl debug-level 6Invalidate all objects in the SSSD cache. Invalidating cache objects ensures that you do not bypass the LDAP database and retrieve SSSD already cached information.
[root@servera ~]#
sss_cache -EClear previous SSSD logs to minimize the troubleshooting data set.
[root@servera ~]#
sssctl logs-removeTruncating log files... [root@servera ~]#Confirm that the
/var/log/sssd/sssd_lab.example.com.logfile is empty, or nearly empty. Run thesssctl logs-removecommand again if the file is larger than 1000 bytes.[root@servera ~]#
ls -l /var/log/sssd/total 4 ...output omitted... -rw-------. 1 root root 850 Nov 18 23:12 sssd_lab.example.com.log ...output omitted...Attempt to switch to the
operator1user account while gathering timestamps before and after the attempt. These timestamps narrow the scope of the data set.[root@servera ~]#
date; su operator1; dateThu Nov 18 23:13:19 EST 2021 su: user operator1 does not exist Thu Nov 18 23:13:19 EST 2021Review the SSSD logs for information about a failed request.
In the following log file, a request is made for the
operator1user with a data provider (DP) request type of Account. The search is based on a top-level domain component of .net, which is incorrect. The information from the system administrator indicates that the domain should belab.example.com.[root@servera ~]#
cat /var/log/sssd/sssd_lab.example.com.log...output omitted... (2021-11-18 23:13:19): [be[lab.example.com]] [dp_get_account_info_send] (0x0200): Got request for [0x1][BE_REQ_USER][name=operator1@lab.example.com] (2021-11-18 23:13:19): [be[lab.example.com]] [dp_attach_req] (0x0400):DP Request [Account #5]:New request. Flags [0x0001]. (2021-11-18 23:13:19): [be[lab.example.com]] [dp_attach_req] (0x0400): Number of active DP request: 1 (2021-11-18 23:13:19): [be[lab.example.com]] [sdap_search_user_next_base] (0x0400): Searching for users with base[cn=accounts,dc=lab,dc=example,dc=net](2021-11-18 23:13:19): [be[lab.example.com]] [sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with [(&(uid=operator1)(objectclass=posixAccount)(uid=*)(&(uidNumber=*)(!(uidNumber=0))))][cn=accounts,dc=lab,dc=example,dc=net]. (2021-11-18 23:13:19): [be[lab.example.com]] [sdap_get_generic_op_finished] (0x0400): Search result: No such object(32), no errmsg set (2021-11-18 23:13:19): [be[lab.example.com]] [sdap_search_user_process] (0x0400): Search for users, returned 0 results. (2021-11-18 23:13:19): [be[lab.example.com]] [sysdb_search_by_name] (0x0400): No such entry (2021-11-18 23:13:19): [be[lab.example.com]] [sysdb_delete_user] (0x0400): Error: 2 (No such file or directory) ...output omitted...
Locate and fix the incorrect configuration.
Review the SSD configuration.
Check the contents of the
/etc/sssd/sssd.conffile for a possible error in data provider entries. All domain entries should be in thelab.example.comdomain.[root@servera ~]#
cat /etc/sssd/sssd.conf[domain/lab.example.com] id_provider = ipa dns_discovery_domain = lab.example.net ipa_server = _srv_, utility.lab.example.com ipa_domain = lab.example.net ipa_hostname = servera.lab.example.com auth_provider = ipa chpass_provider = ipa access_provider = ipa cache_credentials = True ldap_tls_cacert = /etc/ipa/ca.crt krb5_store_password_if_offline = True [sssd] services = nss, pam, ssh, sudo domains = lab.example.com ...output omitted...Edit the incorrect values.
The
dns_discovery_domainandipa_domainkey values are incorrect. These key values should represent the.comtop-level domain component. Correct both entries and restart thesssdservice.The key values should match the following:
[root@servera ~]#
cat /etc/sssd/sssd.conf...output omitted... dns_discovery_domain =lab.example.com...output omitted... ipa_domain =lab.example.com...output omitted...Restart the
sssdservice to implement the new configuration.[root@servera ~]#
systemctl restart sssd
Verify the
operator1user by attempting to log in again.When functioning correctly, the user is not prompted for a password and has an automatically mounted home directory.
Return to
workstationas the student user.[root@servera ~]#
exit[student@servera ~]$exit[student@workstation ~]$Log in to the
serverasystem as theoperator1user.[student@workstation ~]$
ssh operator1@servera...output omitted... [operator1@servera ~]$Verify that the
operator1user has a home directory on theserveramachine.If the working directory returned is the system's root directory, then the automatic home directory mount failed. The home directory appears correct.
[operator1@servera ~]$
pwd/home/operator1 [operator1@servera ~]$Return to
workstationas thestudentuser.[operator1@servera ~]$
exit[student@workstation ~]$