Bookmark this page

Chapter 11. Analyzing and Storing Logs

Abstract

Goal Locate and accurately interpret logs of system events for troubleshooting purposes.
Objectives
  • Describe the basic logging architecture used by Red Hat Enterprise Linux to record events.

  • Interpret events in relevant syslog files to troubleshoot problems or review system status.

  • Find and interpret entries in the system journal to troubleshoot problems or review system status.

  • Configure the system journal to preserve the record of events when a server is rebooted.

  • Maintain accurate time synchronization using NTP and configure the time zone to ensure correct time stamps for events recorded by the system journal and logs.

Sections
  • Describing System Log Architecture (and Quiz)

  • Reviewing Syslog Files (and Guided Exercise)

  • Reviewing System Journal Entries (and Guided Exercise)

  • Preserving the System Journal (and Guided Exercise)

  • Maintaining Accurate Time (and Guided Exercise)

Lab

Analyzing and Storing Logs

Describing System Log Architecture

Objectives

After completing this section, you should be able to describe the basic logging architecture used by Red Hat Enterprise Linux to record events.

System Logging

Processes and the operating system kernel record a log of events that happen. These logs are used to audit the system and troubleshoot problems.

Many systems record logs of events in text files which are kept in the /var/log directory. These logs can be inspected using normal text utilities such as less and tail.

A standard logging system based on the Syslog protocol is built into Red Hat Enterprise Linux. Many programs use this system to record events and organize them into log files. The systemd-journald and rsyslog services handle the syslog messages in Red Hat Enterprise Linux 8.

The systemd-journald service is at the heart of the operating system event logging architecture. It collects event messages from many sources including the kernel, output from the early stages of the boot process, standard output and standard error from daemons as they start up and run, and syslog events. It then restructures them into a standard format, and writes them into a structured, indexed system journal. By default, this journal is stored on a file system that does not persist across reboots.

However, the rsyslog service reads syslog messages received by systemd-journald from the journal as they arrive. It then processes the syslog events, recording them to its log files or forwarding them to other services according to its own configuration.

The rsyslog service sorts and writes syslog messages to the log files that do persist across reboots in /var/log. The rsyslog service sorts the log messages to specific log files based on the type of program that sent each message, or facility, and the priority of each syslog message.

In addition to syslog message files, the /var/log directory contains log files from other services on the system. The following table lists some useful files in the /var/log directory.

Table 11.1. Selected System Log Files

Log file Type of Messages Stored

/var/log/messages

Most syslog messages are logged here. Exceptions include messages related to authentication and email processing, scheduled job execution, and those which are purely debugging-related.

/var/log/secure

Syslog messages related to security and authentication events.

/var/log/maillog

Syslog messages related to the mail server.

/var/log/cron

Syslog messages related to scheduled job execution.

/var/log/boot.log

Non-syslog console messages related to system startup.


Note

Some applications do not use syslog to manage their log messages, although typically, they do place their log files in a subdirectory of /var/log. For example, the Apache Web Server saves log messages to files in a subddirectory of the /var/log directory.

References

systemd-journald.service(8), rsyslogd(8), and rsyslog.conf(5) man pages

For more information refer to the Troubleshooting problems using log files section in the Red Hat Enterprise Linux 8 Configuring basic system settings Guide at https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_basic_system_settings/index#troubleshooting-problems-using-log-files_getting-started-with-system-administration

Revision: rh124-8.2-df5a585