Bookmark this page

Lab: Troubleshooting and Disaster Recovery Planning for IdM

Troubleshoot, diagnose, and fix issues with IdM configuration.

Outcomes

  • Find configuration errors and ensure that the IdM services run correctly.

As the student user on the workstation machine, use the lab command to prepare your environment for this exercise, and to ensure that all required resources are available:

[student@workstation ~]$ lab start maintenance-review

Instructions

  1. On the client machine, try to authenticate to IdM as the admin user.

    1. Log in to the client machine and try to authenticate to IdM:

      [student@workstation ~]$ ssh client
      [student@client ~]$ kinit admin
      kinit: Cannot contact any KDC for realm 'LAB.EXAMPLE.COM' while getting initial credentials
    2. Review the log messages for the sssd service:

      [student@client ~]$ journalctl -u sssd
      Jul 12 17:39:33 client.lab.example.com systemd[1]: System Security Services Daemon was skipped because all trigger condition checks failed.
      Jul 12 17:39:50 client.lab.example.com systemd[1]: Starting System Security Services Daemon...
      Jul 12 17:39:50 client.lab.example.com sssd[25934]: Starting up
      Jul 12 17:39:51 client.lab.example.com sssd_be[26115]: Starting up
      ...output omitted...
      Jul 12 17:50:12 client.lab.example.com sssd_pac[27603]: Starting up
      Jul 12 17:50:12 client.lab.example.com systemd[1]: Started System Security Services Daemon.
      Jul 12 17:50:14 client.lab.example.com sssd_be[27598]: Backend is offline

      The messages in the output do not show relevant information.

    3. Enable debug messages for the authentication process and review the output:

      [student@client ~]$ KRB5_TRACE=/dev/stdout kinit admin
      [27382] 1689128698.202540: Matching admin@LAB.EXAMPLE.COM in collection with result: -1765328243/Can't find client principal admin@LAB.EXAMPLE.COM in cache collection
      [27382] 1689128698.202541: Getting initial credentials for admin@LAB.EXAMPLE.COM
      [27382] 1689128698.202543: Sending unauthenticated request
      [27382] 1689128698.202544: Sending request (179 bytes) to LAB.EXAMPLE.COM
      [27382] 1689128698.202545: Resolving hostname idm.lab.example.com
      kinit: Cannot contact any KDC for realm 'LAB.EXAMPLE.COM' while getting initial credentials

      The messages in the output show that the command fails after trying to resolve the idm machine.

    4. Try to resolve a DNS record from the idm machine name server:

      Note

      The output of the dig command might be different in your environment.

      [student@client ~]$ dig idm.lab.example.com
      
      ; <<>> DiG 9.16.23-RH <<>> idm.lab.example.com
      ;; global options: +cmd
      ;; connection timed out; no servers could be reached
  2. Review the DNS configuration of the client machine. Try resolving the idm name server using different servers in the topology.

    1. Review the DNS configuration:

      [student@client ~]$ nmcli c show System\ eth0 | grep ipv4.dns
      ipv4.dns:                               172.25.250.10
      ipv4.dns-search:                        --
      ipv4.dns-options:                       --
      ipv4.dns-priority:                      0

      The output shows that the replica1 machine is the DNS server configured for the client.

    2. Try to resolve the idm name server by explicitly using the replica1 DNS server:

      [student@client ~]$ dig idm.lab.example.com @172.25.250.10
      ; <<>> DiG 9.16.23-RH <<>> idm.lab.example.com @172.25.250.10
      ;; global options: +cmd
      ;; connection timed out; no servers could be reached
    3. Try to resolve the idm name server by explicitly using the idm DNS server:

      [student@client ~]$ dig idm.lab.example.com @172.25.250.8
      ...output omitted...
      ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55022
      ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
      ...output omitted...
      
      ;; ANSWER SECTION:
      idm.lab.example.com.    1200    IN      A       172.25.250.8
      
      ;; Query time: 2 msec
      ;; SERVER: 172.25.250.8#53(172.25.250.8)
      ...output omitted...

      The idm DNS server is responding to requests but the replica1 DNS server is unreachable.

  3. Verify the status of the replica1 DNS service. Ensure that the replica1 machine has the correct server roles assigned.

    1. Open a new terminal tab and log in to the replica1 machine:

      [student@workstation ~]$ ssh replica1
      ##########################################################
      #                                                        #
      #         This IdM server is under maintenance.          #
      #  Do not start the ipa service or any related service.  #
      #                                                        #
      ##########################################################
      [student@replica1 ~]$

      The message of the day banner shows that the replica1 machine is under maintenance.

    2. Verify the status of the ipa service:

      [student@replica1 ~]$ sudo systemctl status ipa
      [sudo] password for student: student
      ○ ipa.service
           Loaded: masked (Reason: Unit ipa.service is masked.)
           Active: inactive (dead) since Wed 2023-07-12 17:50:01 EDT; 2h 52min ago
         Duration: 5min 6.415s
         Main PID: 31533 (code=exited, status=0/SUCCESS)
              CPU: 1.525s
      ...output omitted...
      Jul 12 17:50:01 replica1.lab.example.com systemd[1]: ipa.service: Deactivated successfully.
      Jul 12 17:50:01 replica1.lab.example.com systemd[1]: Stopped Identity, Policy, Audit.

      The ipa service is masked and, consequently, the DNS server is not running.

    3. On the idm machine, verify the status of the replica1 machine:

      [student@replica1 ~]$ exit
      logout
      Connection to replica1 closed.
      [student@workstation ~]$ ssh idm
      [student@idm ~]$ kinit admin
      Password for admin@LAB.EXAMPLE.COM: RedHat123^
      [student@idm ~]$ ipa server-show replica1.lab.example.com
        Server name: replica1.lab.example.com
        Managed suffixes: domain
        Min domain level: 1
        Max domain level: 1
        Enabled server roles: DNS server, IPA master
    4. Disable the server roles on replica1 to prevent clients from connecting to an unavailable server:

      [student@idm ~]$ ipa server-state replica1.lab.example.com --state=hidden
      ---------------------------------------------------
      Changed server state of "replica1.lab.example.com".
      ---------------------------------------------------
      [student@idm ~]$ ipa server-show replica1.lab.example.com
        Server name: replica1.lab.example.com
        Managed suffixes: domain
        Min domain level: 1
        Max domain level: 1
        Enabled server roles:
    5. Exit the idm machine:

      [student@idm ~]$ exit
      logout
      Connection to idm closed.
  4. On the client machine, review the Kerberos and SSSD configuration files. Ensure that the client machine has all IdM servers defined in the SSSD configuration. Update the DNS configuration to set idm and replica1 as servers.

    1. Review the LAB.EXAMPLE.COM realm definition in the Kerberos configuration file:

      [student@client ~]$ sudo cat /etc/krb5.conf
      [sudo] password for student: student
      ...output omitted...
      [libdefaults]
        default_realm = LAB.EXAMPLE.COM
      ...output omitted...
      [realms]
        LAB.EXAMPLE.COM = {
          kdc = idm.lab.example.com:88
          master_kdc = idm.lab.example.com:88
          admin_server = idm.lab.example.com:749
          kpasswd_server = idm.lab.example.com:464
          default_domain = lab.example.com
      ...output omitted...

      The LAB.EXAMPLE.COM realm definition is correct, and it is set as the default realm.

    2. Review the SSSD configuration file:

      [student@client ~]$ sudo cat /etc/sssd/sssd.conf
      [domain/lab.example.com]
      
      id_provider = ipa
      ipa_server = _srv_, replica1.lab.example.com
      ipa_domain = lab.example.com
      ipa_hostname = client.lab.example.com
      auth_provider = ipa
      ...output omitted...

      The SSSD configuration file has only the replica1 machine listed to authenticate to IdM. The client cannot reach IdM services because the replica1 machine is unavailable.

    3. Update the /etc/sssd/sssd.conf file to include the idm and replica2 machine in the ipa_server parameter:

      [domain/lab.example.com]
      
      id_provider = ipa
      ipa_server = srv, idm.lab.example.com, replica1.lab.example.com, replica2.lab.example.com
      ipa_domain = lab.example.com
      ipa_hostname = client.lab.example.com
      auth_provider = ipa
      ...output omitted...
    4. Restart the sssd service:

      [student@client ~]$ sudo systemctl restart sssd
    5. Update the DNS configuration to specify the idm and replica2 DNS servers. Restart the NetworkManager service:

      [student@client ~]$ sudo nmcli c mod "System eth0" ipv4.dns \
        172.25.250.8,172.25.250.13
      [student@client ~]$ sudo systemctl restart NetworkManager
  5. Verify that you can authenticate to IdM from the client machine, and then list the servers that have the IPA master role assigned.

    1. Authenticate to IdM as the admin user with RedHat123^ as the password:

      [student@client ~]$ kinit admin
      Password for admin@LAB.EXAMPLE.COM: RedHat123^
    2. List the servers that have the IPA master role assigned:

      [student@idm ~]$ ipa server-role-find --role 'IPA master'
      ----------------------
      3 server roles matched
      ----------------------
        Server name: idm.lab.example.com
        Role name: IPA master
        Role status: enabled
      
        Server name: replica1.lab.example.com
        Role name: IPA master
        Role status: hidden
      
        Server name: replica2.lab.example.com
        Role name: IPA master
        Role status: enabled
      ----------------------------
      Number of entries returned 3
      ----------------------------
    3. Exit the client machine:

      [student@client ~]$ exit
      logout
      Connection to client closed.

Evaluation

As the student user on the workstation machine, use the lab command to grade your work. Correct any reported failures and rerun the command until successful.

[student@workstation ~]$ lab grade maintenance-review

Finish

On the workstation machine, change to the student user home directory and use the lab command to complete this exercise. This step is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish maintenance-review

Revision: rh362-9.1-4c6fdb8