Bookmark this page

Chapter 6. Recording System Events with Audit

Abstract

Goal Record and inspect system events relevant to security by using the Linux kernel's Audit subsystem and supporting tools.
Objectives
  • Ensure Audit is installed and configured to record system events, and forward audit messages to a central audit server.

  • Search for events and generate reports from the audit log and interpret the results.

  • Write your own audit rules to configure the system to collect information about particular events.

  • Enable standard audit rule sets provided with Red Hat Enterprise Linux and identify potentially useful rule sets.

Sections
  • Configuring Audit to Record System Events (and Guided Exercise)

  • Inspecting Audit Logs (and Guided Exercise)

  • Writing Custom Audit Rules (and Guided Exercise)

  • Enabling Prepackaged Audit Rule Sets (and Guided Exercise)

Lab

Recording System Events with Audit

Configuring Audit to Record System Events

Objectives

After completing this section, students should be able to ensure that Audit is installed and configured to record system events, and that it forwards audit messages to a central audit server.

The Linux Audit System

The Linux Audit system is a mechanism in the kernel that provides a way to track security-related events and information on your systems. You define a set of rules that you load into the kernel to specify which events it should record in the audit log. The auditd system daemon writes the logged events to the local disk or forwards them to remote log servers. You can then use this information to discover and investigate the cause of violations or attempted violations of the system's security policy.

You can configure Audit to include or exclude events based on user identity, security contexts, or other labels. Some of the information that the Audit system can collect about events includes:

  • Date and time, type, and outcome of an event.

  • Sensitivity labels of subjects and objects.

  • Association of an event with the identity of the user who triggered the event.

  • All modifications to the Audit configuration and attempts to access Audit log files.

  • All uses of authentication mechanisms, such as SSH, Kerberos, and others.

  • Changes to any trusted database, such as /etc/passwd.

  • Attempts to import or export information into or out of the system.

You may need to use the Audit system in order to meet the compliance requirements of particular security certifications. Audit has been evaluated by National Information Assurance Partnership (NIAP) and Best Security Industries (BSI). It has been certified to LSPP/CAPP/RSBAC/EAL4+ on Red Hat Enterprise Linux 5 and OSPP/EAL4+ on Red Hat Enterprise Linux 6. Audit is designed to meet or exceed the requirements of the CAPP, LSPP, RSBAC, NISPOM, FISMA, PCI-DSS, and STIG certifications or compliance guides.

Auditing Your System with auditd

auditd is the user-space component of the Linux auditing subsystem. The systemd unit file that starts auditd normally loads any persistent audit rules when the daemon is started.

After audit rules have been loaded into the kernel, the kernel uses them to record events of interest. The kernel sends information about those events to auditd. The main role of auditd is to collect the audit event messages recorded by the kernel and save them to a log file.

When auditd is running, the log file configured for auditd (normally /var/log/audit/audit.log) collects the audit messages sent by the kernel. If auditd is not running for any reason, rsyslog receives the kernel audit messages.

Without any extra configuration, only a limited number of messages pass through the audit system, mainly authentication and authorization messages (users logging in, sudo being used, etc.) and SELinux messages. Administrators can add audit rules that control the audit system, watch files, or record information about any system call with the auditctl command. This is discussed later in this chapter. The remainder of this section focuses on how to configure auditd to process any logged events.

Configuring auditd

The audit RPM package includes the auditd service, basic configuration files, commands that you can use to configure the audit system and search the audit logs, and some samples of predefined audit rules that you can use a starting point for your own rules.

You can configure audit rules and the auditd service with the following files:

Filename Description
/etc/audit/auditd.conf

Main auditd configuration file.

/etc/audit/audit.rules

The audit rules loaded by auditd.

Do not edit this file. It is generated from the files in /etc/audit/rules.d when auditd starts.

/etc/audit/rules.d

Directory containing manually configured audit rules.

All files ending in .rules are combined into /etc/audit/audit.rules and loaded into the kernel when the auditd service starts.

/etc/audit/rules.d/audit.rules

Example name of a manually managed rule file.

In this section, we mostly discuss the /etc/audit/audit.conf file that configures the behavior of the auditd daemon itself. Rule files are discussed in an upcoming section.

You can determine if the auditd service is running and enabled with the systemctl status auditd and the systemctl is-enabled auditd commands.

Important

When auditd is started, its systemd unit file automatically rebuilds the audit rule set from rule files in /etc/audit/rules.d and attempts to load them into the kernel. If the service is reloaded it attempts to rebuild the rules file and load the new rules, using the augenrules command.

This may fail if a control rule has been loaded that makes the current rules immutable. In this case, you need to reboot the system to change the loaded rule set.

Adjusting auditd Settings to Manage Storage

Depending on what audit rules are loaded, it is possible for the audit log to become large very quickly. It is a good idea for the /var/log/audit directory to be a mount point with its own file system, so that there are no other processes competing with the audit log for storage space and for performance tuning reasons.

In addition, many security policies require that if the audit system cannot log an event to disk due to lack of storage space, the system must halt or drop to single-user mode immediately. This is to ensure that no event can occur without being logged. The auditd daemon has mechanisms to warn you if this is about to happen, and to take various actions when storage gets low.

To configure auditd behavior to comply with your security policy, you can adjust a number of variables in /etc/audit/auditd.conf:

Directive Description
log_file

The location of the log file, /var/log/audit/audit.log by default.

max_log_file

When the log file reaches this size in MB, trigger max_log_file_action.

max_log_file_action

By default, this is set to ROTATE, which rotates the log file, keeping the number of old files specified by the num_logs setting. If you set this to KEEP_FILES, the log file will be rotated but logs will never be deleted (ignoring num_logs).

KEEP_FILES will eventually fill up the storage unless you have an external method to back up and remove the old logs.

space_left

When the file system containing the log file has this much free space (in MB) remaining, trigger space_left_action.

This should be set high enough relative to the rate of incoming audit messages that the administrator has time to be notified by space_left_action and take action before the file system only has admin_space_left space free.

space_left_action

By default, this is set to SYSLOG, which logs a warning. If you set this to EMAIL, an email is sent to the address specified by the action_mail_acct directive if the machine has /usr/lib/sendmail configured (supplied by Postfix or Sendmail). Alternatively, you can set this to EXEC /path/to/script and the command /path/to/script will be run with no arguments.

admin_space_left

When the file system containing the log file has this much free space (in MB) remaining, trigger admin_space_left_action.

You never want the file system to reach this condition. This will usually suspend auditd or halt the entire system, leaving a small amount of free space left so that you can fix the issue.

admin_space_left_action

By default, this is set to SUSPEND, which causes auditd to stop writing audit records to the file system. This keeps the system running for sites that value system availability more highly than a perfect audit log.

Many security policies require that this be set to SINGLE (putting the system in single-user mode, allowing the admin to recover) or HALT (bringing down the entire system) because they do not permit you to operate the system if it is not maintaining a complete audit log.

disk_full_action

This action triggers if the file system containing the audit log is completely full.

Much like admin_space_left_action, this is set to SUSPEND by default and many sites may need to set this to SINGLE or HALT in order to comply with their security policy.

disk_error_action

If the file system is not full but reports some other write error, this event will trigger. It is set to SUSPEND by default, which will stop the logging of audit events. It's likely the system cannot log audit events anyway if this happens.

You can set this to SINGLE or HALT for the same reasons you might if the file system becomes full.

Adjusting auditd Settings to Tune Performance

You can improve the performance of the auditd service with the following settings in the /etc/audit/audit.conf configuration file:

  • Set the value for the flush parameter to INCREMENTAL_ASYNC to enable asynchronous flushing of records to storage after the number of writes specified by freq, which improves system performance. However, the tradeoff is that if there is a system crash, audit messages may be lost, which is why setting this to DATA or SYNC is sometimes required by compliance policies. Using INCREMENTAL_ASYNC with a relatively low freq setting ensures that records are quickly synchronized to the file system while maintaining good system performance.

  • Set the freq parameter to 50 to flush the Audit log after every fifty records (assuming that flush = INCREMENTAL_ASYNC is set). If flush is set to DATA or SYNC, this setting is ignored and every write to storage is synchronous.

Remote Logging with auditd

You can send audit messages to a remote system in two different ways. You can either send them to the local rsyslog service and configure rsyslog to forward them to a remote server, or you can configure your system to send them to a remote auditd service.

Configuring Clients

Before configuring your system to use either method to send messages to a remote server, you should make two changes to your system's /etc/audit/auditd.conf file:

  • Set log_format = ENRICHED to resolve UID, GID, system call number, architecture, and socket address information to names before transmitting each event. This ensures that the log information makes sense on a remote system that may have different values for these mappings.

  • Set name_format = HOSTNAME to include the machine's host name in each message. This makes it clear to which machine an audit event belongs.

If you are sending messages to rsyslog, you need to edit your client's /etc/audisp/plugins.d/syslog.conf file to include the line active = yes. You also need to configure your local rsyslog service to send log messages to a remote server in its /etc/rsyslog.conf file.

If you are sending messages to a remote auditd service, then you must install the audispd-plugins package on your client. You also need to edit your client's /etc/audisp/plugins.d/au-remote.conf file to include the line active = yes. You also need to edit your client's /etc/audisp/audisp-remote.conf file to include a remote_server directive set to the IP address or host name of the remote auditd server, and update the port directive if your remote server is not listening on the default 60/TCP port.

Configuring a Server

If you are configuring rsyslog to collect remote audit messages, you need to configure remote log reception in /etc/rsyslog.conf in the usual way. Make sure you open the firewall port for the service. (This is 514/UDP for standard syslog, but other ports for other protocols supported by rsyslog.)

If you are configuring auditd to listen for remote audit messages, you must edit the /etc/audit/auditd.conf file to uncomment the tcp_listen_port = 60 directive. This configures auditd to listen on port 60/TCP. You also need to make sure the server's firewall allows connections to that port. You need to restart auditd to put this change into effect, which requires a reboot.

Warning

Both methods of sending audit messages to a remote server use clear text protocols without encryption by default. This makes the content of the messages subject to interception, and messages may be tampered with.

If you are using rsyslog, you should configure it to use TLS authentication and encryption to protect your audit log traffic.

If you are using the native auditd transport, you should configure it to use Kerberos authentication and encryption in the /etc/audisp/audisp-remote.conf file on the client and /etc/audit/auditd.conf on the server. See the audisp-remote.conf(5) and auditd.conf(5) man pages for details.

References

auditd(8), auditd.conf(5), audispd.conf(5), audisp-remote.conf(5) and audit.rules(7) man pages

For more information, refer to the System Auditing chapter in the Red Hat Enterprise Linux 7 Security Guide at https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/security_guide/#chap-system_auditing

The upstream project documentation for Linux Audit is available at https://github.com/linux-audit/audit-documentation

Revision: rh415-7.5-813735c