RHCSA Rapid Track

Objectives

After completing this section, students should be able to interpret entries in relevant syslog files to troubleshoot problems or review system status.

Syslog files

Many programs use the syslog protocol to log events to the system. Each log message is categorized by a facility (the type of message) and a priority (the severity of the message). The facilities which are available are documented by the rsyslog.conf(5) man page.

The eight priorities are also standardized and ranked as follows:

The rsyslogd service uses the facility and priority of log messages to determine how to handle them. This is configured by the file /etc/rsyslog.conf and by *.conf files in /etc/rsyslog.d. Programs and administrators can change rsyslogd configuration in a way that will not be overwritten by updates to rsyslog by putting customized files with a .conf suffix in the /etc/rsyslog.d directory.

The #### RULES #### section of /etc/rsyslog.conf contains directives that define where log messages are saved. The left side of each line indicates the facility and severity of the log message the directive matches. The rsyslog.conf file can contain the character * as a wild card in the facility and severity field, where it either stands for all facilities or all severities. The right side of each line indicates what file to save the log message in. Log messages are normally saved in files in the /var/log directory.

A message handled by rsyslog can appear in multiple different log files. To prevent that, the severity field can be set to none, which means that none of the messages directed to this facility are added to the specified log file.

Instead of logging syslog messages to a file, they can be printed to the terminals of all logged-in users. In the default rsyslog.conf file, this is done for all messages that have "emerg" priority.

Sample rules section of rsyslog.conf

#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log

Log file rotation

Logs are "rotated" by the logrotate utility to keep them from filling up the file system containing /var/log/. When a log file is rotated, it is renamed with an extension indicating the date on which it was rotated: the old /var/log/messages file may become /var/log/messages-20141030 if it is rotated on October 30, 2014. Once the old log file is rotated, a new log file is created and the service that writes to it is notified.

After a certain number of rotations, typically after four weeks, the old log file is discarded to free disk space. A cron job runs the logrotate program daily to see if any logs need to be rotated. Most log files are rotated weekly, but logrotate rotates some faster, or slower, or when they reach a certain size.

Configuration of logrotate is not covered in this course. For more information, see the logrotate(8) man page.

Analyze a syslog entry

The system logs written by rsyslog start with the oldest message on top and the newest message at the end of the log file. All log entries in log files managed by rsyslog are recorded in a standard format. The following example will explain the anatomy of a log file message in the /var/log/secure log file:

Feb 11 20:11:48 localhost sshd[1433]: Failed password for student from 172.25.0.10 port 59344 ssh2

Monitor a log file with tail

It is especially helpful for reproducing problems and issues to monitor one or more log files for events. The tail -f /path/to/file command outputs the last 10 lines of the file specified and continues to output new lines as they get written to the monitored file.

To monitor for failed login attempts on one terminal, run ssh as user root while a user tries to log in to the serverX machine:

[root@serverX ~]$ tail -f /var/log/secure
...
Feb 10 09:01:13 localhost sshd[2712]: Accepted password for root from 172.25.254.254 port 56801 ssh2
Feb 10 09:01:13 localhost sshd[2712]: pam_unix(sshd:session): session opened for user root by (uid=0)

Send a syslog message with logger

The logger command can send messages to the rsyslog service. By default, it sends the message to the facility user with severity notice (user.notice) unless specified otherwise with the -p option. It is especially useful to test changes to the rsyslog configuration.

To send a message to rsyslogd that gets recorded in the /var/log/boot.log log file, execute:

[root;@serverX ~]$ logger -p local7.notice "Log entry created on serverX"