RHCSA Rapid Track

Objectives

After completing this section, you should be able to log in to a system and change the root password when the current root password has been lost.

Resetting the Root Password from the Boot Loader

One task that every system administrator should be able to accomplish is resetting a lost root password. If the administrator is still logged in, either as an unprivileged user but with full sudo access, or as root, this task is trivial. When the administrator is not logged in, this task becomes slightly more involved.

Several methods exist to set a new root password. A system administrator could, for example, boot the system using a Live CD, mount the root file system from there, and edit /etc/shadow. In this section, we explore a method that does not require the use of external media.

On Red Hat Enterprise Linux 8, it is possible to have the scripts that run from the initramfs pause at certain points, provide a root shell, and then continue when that shell exits. This is mostly meant for debugging, but you can also use this method to reset a lost root password.

To access that root shell, follow these steps:

At this point, the system presents a root shell, with the actual root file system on the disk mounted read-only on /sysroot. Because troubleshooting often requires modification to the root file system, you need to change the root file system to read/write. The following step shows how the remount,rw option to the mount command remounts the file system with the new option (rw) set.

To reset the root password from this point, use the following procedure:

At this point, the system continues booting, performs a full SELinux relabel, and then reboots again.

Inspecting Logs

Looking at the logs of previously failed boots can be useful. If the system journals are persistent across reboots, you can use the journalctl tool to inspect those logs.

Remember that by default, the system journals are kept in the /run/log/journal directory, which means the journals are cleared when the system reboots. To store journals in the /var/log/journal directory, which persists across reboots, set the Storage parameter to persistent in /etc/systemd/journald.conf.

[root@host ~]# vim /etc/systemd/journald.conf
...output omitted...
[Journal]
Storage=persistent
...output omitted...
[root@host ~]# systemctl restart systemd-journald.service

To inspect the logs of a previous boot, use the -b option of journalctl. Without any arguments, the -b option only displays messages since the last boot. With a negative number as an argument, it displays the logs of previous boots.

[root@host ~]# journalctl -b -1 -p err

This command shows all messages rated as an error or worse from the previous boot.

Repairing Systemd Boot Issues

To troubleshoot service startup issues at boot time, Red Hat Enterprise Linux 8 makes the following tools available.

Enabling the Early Debug Shell

By enabling the debug-shell service with systemctl enable debug-shell.service, the system spawns a root shell on TTY9 (Ctrl+Alt+F9) early during the boot sequence. This shell is automatically logged in as root, so that administrators can debug the system while the operating system is still booting.

Using the Emergency and Rescue Targets

By appending either systemd.unit=rescue.target or systemd.unit=emergency.target to the kernel command line from the boot loader, the system spawns into a rescue or emergency shell instead of starting normally. Both of these shells require the root password.

The emergency target keeps the root file system mounted read-only, while the rescue target waits for sysinit.target to complete, so that more of the system is initialized, such as the logging service or the file systems. The root user at this point can not make changes to /etc/fstab until the drive is remounted in a read write state mount -o remount,rw /

Administrators can use these shells to fix any issues that prevent the system from booting normally; for example, a dependency loop between services, or an incorrect entry in /etc/fstab. Exiting from these shells continues with the regular boot process.

Identifying Stuck Jobs

During startup, systemd spawns a number of jobs. If some of these jobs cannot complete, they block other jobs from running. To inspect the current job list, administrators can use the systemctl list-jobs command. Any jobs listed as running must complete before the jobs listed as waiting can continue.