Bookmark this page

Review Syslog Files

Objectives

Interpret events in relevant syslog files to troubleshoot problems or review system status.

Log Events to the System

Many programs use the syslog protocol to log events to the system. Each log message is categorized by facility (the subsystem that produces the message) and priority (the message's severity).

The following table lists the standard syslog facilities:

Table 12.2. Overview of Syslog Facilities

CodeFacilityFacility description
0kernKernel messages
1userUser-level messages
2mailMail system messages
3daemonSystem daemon messages
4authAuthentication and security messages
5syslogInternal syslog messages
6lprPrinter messages
7newsNetwork news messages
8uucpUUCP protocol messages
9cronClock daemon messages
10authprivNon-system authorization messages
11ftpFTP protocol messages
16-23local0 to local7Custom local messages

The following table lists the standard syslog priorities in descending order:

Table 12.3. Overview of Syslog Priorities

CodePriorityPriority description
0emergSystem is unusable
1alertAction must be taken immediately
2critCritical condition
3errNon-critical error condition
4warningWarning condition
5noticeNormal but significant event
6infoInformational event
7debugDebugging-level message

The rsyslog service uses the facility and priority of log messages to determine how to handle them. Rules configure this facility and priority in the /etc/rsyslog.conf file and in any file in the /etc/rsyslog.d directory with the .conf extension. Software packages can add rules by installing an appropriate file in the /etc/rsyslog.d directory.

Each rule that controls how to sort syslog messages has a line in one of the configuration files. The left side of each line indicates the facility and priority of the syslog messages that the rule matches. The right side of each line indicates which file to save the log message in (or where else to deliver the message). An asterisk (*) is a wildcard that matches all values.

For example, the following line in the /etc/rsyslog.d file would record messages that are sent to the authpriv facility at any priority to the /var/log/secure file:

authpriv.*                  /var/log/secure

Sometimes, log messages match more than one rule in the rsyslog.conf file. In such cases, one message is stored in more than one log file. The none keyword in the priority field indicates that no messages for the indicated facility are stored in the given file, to limit stored messages.

Instead of being logged to a file, syslog messages can also be printed to the terminals of all logged-in users. The rsyslog.conf file has a setting to print all the syslog messages with the emerg priority to the terminals of all logged-in users.

Sample Rules of the rsyslog Service

#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
.emerg                                                 :omusrmsg:

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log

Note

The syslog subsystem has many more features beyond the scope of this course. To explore further, refer to the rsyslog.conf(5) man page and the extensive HTML documentation at /usr/share/doc/rsyslog/html/index.html that the rsyslog-doc package provides.

Log File Rotation

The logrotate command rotates log files to prevent them from taking too much space in the /var/log directory. When a log file is rotated, it is renamed with an extension that indicates the rotation date. For example, the previous /var/log/messages file is renamed to the /var/log/messages-20220320 file when it is rotated on 2022-03-20. After the previous log file rotates, it creates a log file and notifies the service that wrote the log file.

After rotations during typically four weeks, the earliest log file is discarded to free disk space. A scheduled job runs the logrotate command daily to see the rotation requirement of any log files. Most log files rotate weekly; the logrotate command rotates some log files faster, or more slowly, or when they reach a specific size.

Analyze a Syslog Entry

Log messages start with the earliest message at the start and the latest message at the end of the log file. The rsyslog service uses a standard format for recording entries in log files. The following example explains the anatomy of a log message in the /var/log/secure log file.

Mar 20 20:11:48 localhost sshd[1433]: Failed password for student from 172.25.0.10 port 59344 ssh2
  • Mar 20 20:11:48 : Records the time stamp of the log entry.

  • localhost : The host that sends the log message.

  • sshd[1433] : The program or process name and PID number that sent the log message.

  • Failed password for …​ : The message that was sent.

Monitor Log Events

Monitoring log files for events is helpful to reproduce issues. The tail -f /path/to/file command outputs the last ten lines of the specified file and continues to output newly written lines in the file.

For example, to monitor for failed login attempts, run the tail command in one terminal, and then run in another terminal the ssh command as the root user while a user tries to log in to the system.

In the first terminal, run the tail command:

[root@host ~]# tail -f /var/log/secure

In the second terminal, run the ssh command:

[root@host ~]# ssh root@hosta
root@hosta's password: redhat
...output omitted...
[root@hostA ~]#

The log messages are visible in the first terminal.

...output omitted...
Mar 20 09:01:13 host sshd[2712]: Accepted password for root from 172.25.254.254 port 56801 ssh2
Mar 20 09:01:13 host sshd[2712]: pam_unix(sshd:session): session opened for user root by (uid=0)

Send Syslog Messages Manually

The logger command sends messages to the rsyslog service. By default, the logger command sends the message to the user type with the notice priority (user.notice) unless specified otherwise with the -p option. It is helpful to test any change to the rsyslog service configuration.

To send a message to the rsyslog service to be recorded in the /var/log/boot.log log file, execute the following logger command:

[root@host ~]# logger -p local7.notice "Log entry created on host"

 

References

logger(1), tail(1), rsyslog.conf(5), and logrotate(8) man pages

rsyslog Manual

  • /usr/share/doc/rsyslog/html/index.html provided by the rsyslog-doc package

For further information, refer to Troubleshooting Problems Using Log Files at https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/configuring_basic_system_settings/assembly_troubleshooting-problems-using-log-files_configuring-basic-system-settings

Revision: rh199-9.0-4fecb06