Managing CentOS Migrations and RHEL Upgrades

Use the leapp preupgrade command to run a pre-upgrade analysis on the system. By default, the leapp preupgrade command determines the target version of the operating system based on the current version of RHEL that is used by the system and the supported upgrade path. The default target version can change as new versions of RHEL and the Leapp tool are released. With the leapp command version that is used in this course, for example, the command's default behavior is to upgrade a RHEL 8.9 system to RHEL 9.3.

In some scenarios, you might want to upgrade to a version that is different from the default target version. To specify the target version, you must use the --target option of the leapp preupgrade command.

In the following example, the system is currently running RHEL 7.9. When you run the leapp preupgrade command, you use the --target option to specify RHEL 8.6 as the target version. The leapp command must be run as the root user.

[root@host ~]# leapp preupgrade --target 8.6

The Leapp tool starts to perform checks and displays its progress on the command line. Depending on the properties of the system and its installed software, the analysis can take several minutes to complete.

The pre-upgrade analysis identifies potential problems with the upgrade, and assesses each issue's severity or risk factor.

The following list describes the severity levels of issues that the Leapp tool reports:

Before attempting to upgrade, you should at least resolve high and medium severity issues. Although you can still upgrade your system without resolving these issues, resolving them increases the chances of a successful upgrade.

The Leapp tool might mark some high severity issues as inhibitors. Inhibitors prevent you from upgrading the system until you have resolved them.

All issues are documented in a pre-upgrade report that the Leapp tool generates. After you run the leapp preupgrade command, the command prints a report overview and generates two report files with detailed information about the reported issues. The following example shows a report overview of a RHEL 7.9 system with RHEL 8.9 as the target version:

[root@host ~]# leapp preupgrade --target 8.9
...output omitted...
Debug output written to /var/log/leapp/leapp-preupgrade.log

============================================================
                      REPORT OVERVIEW
============================================================

Upgrade has been inhibited due to the following problems:
    1. Leapp detected loaded kernel drivers which have been removed in RHEL 8.
       Upgrade cannot proceed.
    2. Missing required answers in the answer file

HIGH and MEDIUM severity reports:
    1. Packages available in excluded repositories will not be installed
    2. Difference in Python versions and support in RHEL 8
    3. Leapp detected loaded kernel drivers which are no longer maintained
       in RHEL 8.
    4. GRUB2 core will be automatically updated during the upgrade

Reports summary:
    Errors:                      0
    Inhibitors:                  2
    HIGH severity reports:       0
    MEDIUM severity reports:     0
    LOW severity reports:        3
    INFO severity reports:       4

Before continuing consult the full report:
    A report has been generated at /var/log/leapp/leapp-report.json
    A report has been generated at /var/log/leapp/leapp-report.txt

============================================================
                   END OF REPORT OVERVIEW
============================================================

Answerfile has been generated at /var/log/leapp/answerfile

The report overview includes two categories of items that require your attention: inhibitors as well as high and medium severity reports. The Reports summary section provides a count of the reported items ranked by severity level.

The report overview also prints the location of the two full report files: a plain text file and a text file with JSON notation. These two files contain the same information in different formats. The full report contains detailed information about the reported issues, including suggestions for solving them.

The full report looks similar to the following example:

[root@host ~]# cat /var/log/leapp/leapp-report.txt
...output omitted...
Risk Factor: high
Title: Difference in Python versions and support in RHEL 8
Summary: In RHEL 8, there is no 'python' command. Python 3 (backward incompatible) is the primary Python version and Python 2 is available with limited support and limited set of packages. If you no longer require Python 2 packages following the upgrade, please remove them. Read more here: https://red.ht/rhel-8-python
Remediation: [hint] Please run "alternatives --set python /usr/bin/python3" after upgrade
...output omitted...
----------------------------------------
Risk Factor: medium
Title: chrony using default configuration
Summary: default chrony configuration in RHEL8 uses leapsectz directive, which cannot be used with leap smearing NTP servers, and uses a single pool directive instead of four server directives
...output omitted...

The leapp-report.txt report file contains details about each issue that is identified in the pre-upgrade analysis. Each item in the leapp-report.txt file is separated by a line of hyphens (-) and starts with the risk factor type. The report also includes suggestions to resolve issues that are based on best practices.

Finally, the leapp preupgrade command output also displays the location of the Leapp answer file, which you modify to input confirmation on upgrade choices.

To continue with the upgrade process, you must resolve all issues that are marked as inhibitors. Other issues are optional to resolve, although not resolving them might impact the system or the underlying application.

Most of the issues include a remediation suggestion to ease the resolution. Red Hat recommends addressing all issues from the report to improve the chances of a successful upgrade. To resolve reported issues, review the full report to understand the issue, determine possible root causes, and review any suggested remediations.

The actions you take to solve reported issues might vary depending on the issue, the standard practices in your organization, and the recommendations of the application provider. Make sure to review each reported issue and the suggested solution with your application provider.

In the following example, the pre-upgrade analysis detects some high severity issues that must be resolved. The first part of the report overview shows two inhibitor items: an issue with a kernel module and a missing answer in the Leapp answer file.

[root@host ~]# leapp preupgrade --target 8.9
...output omitted...
Upgrade has been inhibited due to the following problems:
    1. Leapp detected loaded kernel drivers which have been removed in RHEL 8.
       Upgrade cannot proceed.
    2. Missing required answers in the answer file
...output omitted...

Review the contents of the /var/log/leapp/leapp-report.txt file to learn more about these issues:

[root@host ~]# cat /var/log/leapp/leapp-report.txt
Risk Factor: high (inhibitor)
Title: Leapp detected loaded kernel drivers which have been removed in RHEL 8.
       Upgrade cannot proceed.
Summary: Support for the following RHEL 7 drivers has been removed in RHEL 8:
     - pata_acpi
...output omitted...
----------------------------------------
Risk Factor: high (inhibitor)
Title: Missing required answers in the answer file
Summary: One or more sections in answerfile are missing user choices:
         remove_pam_pkcs11_module_check.confirm
...output omitted...
----------------------------------------
Risk Factor: medium
Title: chrony using default configuration
Summary: default chrony configuration in RHEL8 uses leapsectz directive, which cannot be used with leap smearing NTP servers, and uses a single pool directive instead of four server directives
...output omitted...

The first inhibitor is an issue with incompatible kernel modules in the target version. You can resolve this issue by removing the kernel module. First, make sure that the module is not in use, and then use the rmmod command to unload it from memory.

[root@host ~]# lsmod
Module                  Size  Used by
...outout omitted...
pata_acpi              13053  0

The Used by column displays the number of processes that use the module, which is zero in this example.

[root@host ~]# rmmod pata_acpi

The second inhibitor item on the report is missing information in the Leapp answer file. This information is required for confirming suggested remediations.

For some reported issues, the Leapp tool recommends solutions. To apply the solution, you must confirm the suggested upgrade task. You must provide confirmation because executing the task might result in undesired states of the machine. You can provide confirmation in the Leapp answer file at /var/log/leapp/answerfile.

Review the Leapp answer file to learn more about the issue:

[root@host ~]# cat /var/log/leapp/answerfile
[remove_pam_pkcs11_module_check]
# Title:              None
# Reason:             Confirmation
# =================== remove_pam_pkcs11_module_check.confirm ==================
# Label:              Disable pam_pkcs11 module in PAM configuration? If no, the upgrade process will be interrupted.
# Description:        PAM module pam_pkcs11 is no longer available in RHEL-8 since it was replaced by SSSD.
# Reason:             Leaving this module in PAM configuration may lock out the system.
# Type:               bool
# Default:            None
# Available choices: True/False
# Unanswered question. Uncomment the following line with your answer
# confirm =

The pam_pkcs11 module is no longer available in RHEL 8 and its absence might cause issues for the user or for running applications. The Leapp tool requires that you acknowledge and confirm the removal of the pam_pkcs11 module. After you have taken the necessary steps to replace the module, you can confirm the action by setting the confirm variable to the True value.

[root@host ~]# cat /var/log/leapp/answerfile
[remove_pam_pkcs11_module_check]
...output omitted...
confirm = True

Before proceeding with the upgrade, confirm that you have resolved the issues and rerun the pre-upgrade analysis.

After resolving reported issues, run the pre-upgrade analysis again to confirm that there are no additional inhibitors or errors. The pre-upgrade analysis might stop at different points if the assessment fails a task. An initial report might not show errors, but if you change the system, then the Leapp tool might find errors or other issues in a future run. After you fix reported issues, run the pre-upgrade analysis until there are no reported inhibitors and errors.

Continuing with the preceding example, run the pre-upgrade analysis again after resolving the previously reported issues. This time, the report overview shows an error.

[root@host ~]# leapp preupgrade --target 8.9
...output omitted...
============================================================
                           ERRORS
============================================================

2024-03-05 20:48:31.319705 [ERROR] Actor: target_userspace_creator
Message: Cannot set the container mode for the subscription-manager.

...output omitted...

Following errors occurred and the upgrade cannot continue:
    1. Actor: target_userspace_creator
       Message: Cannot set the container mode for the subscription-manager.

...output omitted...

Reports summary:
    Errors:                      1
    Inhibitors:                  0
...output omitted...

The leapp-report.txt report file shows an issue with the subscription-manager tool.

[root@host ~]# cat /var/log/leapp/leapp-report.txt
Risk Factor: high (error)
Title: Cannot set the container mode for the subscription-manager.
Summary:
Key: 06583cd4c6f9e1629a9c1937643c3897618f33ef

This error can occur if the /etc/machine-id file does not exist or does not contain the expected information. Recreate the file with the correct content, and rerun the pre-upgrade analysis.

[root@host ~]# rm /etc/machine-id
rm: remove regular file '/etc/machine-id'? y
[root@host ~]# dbus-uuidgen --ensure=/etc/machine-id

After resolving the reported issues and fixing errors, rerun the pre-upgrade analysis to make sure that no new issues are found. Make sure to complete all the steps in your plan before starting the upgrade. Review the upgrade prerequisites, limitations, and known issues, and be sure to address all tasks or concerns.

If the report overview does not show issues and you have performed all the steps in your plan, then you can continue with the actual upgrade.