Bookmark this page

Inspecting Audit Logs

Objectives

After completing this section, students should be able to search for events and generate reports from the audit log and interpret the results.

Interpreting Audit Messages

Audit events recorded in /var/log/audit/audit.log include a lot of information in a condensed format. A single event might actually log multiple audit records of different types to the log as separate messages. Each of these records might include several fields of information about the logged event.

This is an example of several audit records associated with a single audit event as it might appear in /var/log/audit/audit.log, recorded in unprocessed raw format by auditd.

type=SYSCALL1 msg=audit(1371716130.596:28708)2: arch=c000003e syscall=23 success=yes exit=4 a0=261b130 a1=90800 a2=e a3=19 items=1 ppid=2548 pid=26131 auid=5004 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="aureport" exe="/sbin/aureport" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="audit-access"5
type=CWD msg=audit(1371716130.596:28708):  cwd="/root"
type=PATH msg=audit(1371716130.596:28708): item=0 name="/var/log/audit" inode=17998 dev=fd:01 mode=040750 ouid=0 ogid=0 rdev=00:00 obj=system_u:object_r:auditd_log_t:s0

There are three audit records here, which are all part of audit event 28708.

1

Each audit record has a record type, sometimes called a message type, which is reflected by the type field starting each record. This is a SYSCALL record.

2

The msg field gives the time stamp for this record and the event ID. The part before the colon (in this case 1371716130.596) is the time stamp in number of seconds since the epoch, 00:00 UTC on January 1, 1970. You can convert epoch time to local time with the command date --date=@<epoch-time>. The part after the colon is the audit event number, 28708, which it shares with the other records for this event.

3

The type of the first record is SYSCALL; information about a system call made to the kernel. This syscall field indicates the number of the system call that was made (not its name). The mapping of system call numbers to names can vary between processor architectures, which is one reason why it can be challenging to interpret a log in raw format correctly without help. In a moment, you will use the ausearch command to provide this help.

4

The auid field is important. This field records the Audit UID of the user that triggered this event. This is the UID of the initial account used to log in to this machine by the user that triggered this event, even if they used sudo or su to become another user.

5

The key field is an identifier that you can use when searching for events. You can set keys in your custom auditing rules to make it easier to filter for certain types of events.

Searching for Events

The auditing system ships with the ausearch command, which is a powerful tool for searching audit logs. You can use ausearch to search for and filter various types of events. It can also translate numeric values into more readable values, such as user names or system call names, to interpret those events. The following table shows some of the common ausearch options. Other options exist to help you search for events based on user, terminal, or even virtual machine.

Option Description
-i

Interpret the log records, translate numeric values into names. This is very useful when you have raw log files.

--raw

Print raw log entries, do not even put separators between events. This is useful if you have other tools that can parse the raw log format. The short option -r is equivalent.

-a <EVENT-ID>

Show all records for the event that has <EVENT-ID> as its event ID.

-m <MESSAGE-TYPE>

Show all events that include a record with <MESSAGE-TYPE> as its message type. The long option --message is equivalent.

-f <FILENAME>

Search for all events related to a specific filename. The long option --file is equivalent.

-k <KEY>

Search for all events labeled with the <KEY> key.

--start [start-date] [start-time]

Only search for events after start-date and start-time. If you do not specify a starting time, the search assumes midnight. If you omit the starting date, the search assumes today.

The time format depends on your current locale. Other values that you can use include recent (past ten minutes), this-week, this-month, and this-year.

--end can be used to search for events that occurred before a specific date and time, and uses the same syntax.

For a complete list of options refer to the ausearch(8) manual page.

As a demonstration, the following output shows ausearch being used to look up an interpreted version of event 28708 from the previous example.

[root@demo ]# ausearch -i -a 28708
----
type=PATH msg=audit(07/31/2018 10:15:30.596:28708) : item=0 name=/var/log/audit inode=17998 dev=fd:01 mode=dir,750 ouid=root ogid=root rdev=00:00 obj=system_u:object_r:auditd_log_t:s0
type=CWD msg=audit(07/31/2018 10:15:30.596:28708) :  cwd=/root
type=SYSCALL msg=audit(07/31/2018 10:15:30.596:28708) : arch=x86_64 syscall=open success=yes exit=4 a0=261b130 a1=90800 a2=e a3=19 items=1 ppid=2548 pid=26131 auid=student uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts0 ses=1 comm=aureport exe=/sbin/aureport subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=audit-access

Each audit event is separated in the interpreted output by four dashes.

You can still see the three audit records for this event: PATH, CWD, and SYSCALL. The Red Hat Enterprise Linux 7 Security Guide provides useful information on how to interpret audit record types and fields in its Audit System Reference appendix.

The PATH record is a file involved in this event. It is named /var/log/audit (name=/var/log/audit), and is inode 17998 (inode=17998), on the file system on a device with major/minor numbers 253,1 (dev=fd:01, the device numbers are in hexadecimal format). Looking in /dev with ls -l, the /dev/dm-1 device turns out to have those numbers and is associated with a logical volume. The file is a directory with octal permissions 750 (mode=dir,750), owned by user and group root (ouid=root ogid=root), with the SELinux type auditd_log_t (obj=system_u:object_r:auditd_log_t:s0).

The CWD record is the current working directory associated with the process that triggered this event, which in this case is /root.

The SYSCALL record is the system call that triggered this event. The open() system call (syscall=open) was used to successfully (success=yes) open the file specified by the PATH record (the /var/log/audit directory). This was done by a process with PID 26131 (pid=26131). It was started by the /sbin/aureport executable (exe=/sbin/aureport) run with an effective UID of root (euid=root) and a SELinux domain of unconfined_t (subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023) by the root user (uid=root). The command was run on the pseudoterminal pts/0 (tty=pts0), probably a graphical terminal window or remote login session. The user originally logged in as student (auid=student) and has somehow become root since then. This record has the audit-access key set on it, to make its event easier to find with ausearch (key=audit-access).

Reporting on Audit Messages

Instead of reading individual audit messages, you can use the aureport command to get either a quick overview of audit messages or more detailed reports on specific types of events.

When run without any options, aureport shows an overview of how many different types of events are present in the logs. When you specify search options, mostly the same as those for ausearch, it shows a list of all events matching the search criteria. Common options include -i to interpret results, and --summary to condense the list into a summary. You can also create reports for specific types of events, such as a login report with the --login option, or an executable name report with the --executable option.

When the ausearch --raw command is used to search for specific events recorded in the Audit log files, the unformatted search results can be provided as input to the aureport command to generate formatted reports.

Note

Two specialized tools also exist: aulast and aulastlog. They replace last and lastlog respectively, but they parse the audit logs instead of /var/log/wtmp and /var/log/btmp.

Tracing a Program

To investigate the system calls performed by a process, you can run it with the autrace command, for example autrace /bin/date. Running the autrace command removes all custom auditing rules, replacing them with rules specifically for tracing the program you specified. When execution finishes, the autrace command clears those rules, and then provides an example ausearch command to investigate those events.

This is useful for troubleshooting or investigating programs of interest.

The following example traces the date command:

[root@demo ~]# autrace /bin/date
Waiting to execute: /bin/date
Thu Jul 31 11:38:46 CEST 2018
Cleaning up...
Trace complete. You can locate the records with 'ausearch -i -p 26472'
[root@demo ~]# ausearch --raw -p 26472 | aureport --file -i

File Report
===============================================
# date time file syscall success exe auid event
===============================================
1. 07/31/2018 11:38:46 /bin/date execve yes /bin/date student 29158
2. 07/31/2018 11:38:46 /etc/ld.so.preload access no /bin/date student 29161
3. 07/31/2018 11:38:46 /etc/ld.so.cache open yes /bin/date student 29162
4. 07/31/2018 11:38:46 /lib64/librt.so.1 open yes /bin/date student 29166
5. 07/31/2018 11:38:46 /lib64/libc.so.6 open yes /bin/date student 29173
6. 07/31/2018 11:38:46 /lib64/libpthread.so.0 open yes /bin/date student 29181
7. 07/31/2018 11:38:46 /usr/lib/locale/locale-archive open yes /bin/date student 29208
8. 07/31/2018 11:38:46 /etc/localtime open yes /bin/date student 29213

Warning

The autrace command removes any active audit rules or requires you to remove any active rules before you run it. This could cause you to miss events from other processes that those existing rules would record.

If the audit rules are locked in place, autrace will not be able to unload the existing rules and will not work.

References

ausearch(8), aureport(8), and autrace(8) man pages

For more information, refer to the System Auditing chapter in the Red Hat Enterprise Linux 7 Security Guide at https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/security_guide/#chap-system_auditing

For help decoding audit records and fields, refer to the Audit System Reference appendix in the Red Hat Enterprise Linux 7 Security Guide at https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/security_guide/#app-Audit_Reference

Revision: rh415-7.5-813735c