RH342 - ch09s02

Bookmark this page

Guided Exercise: Repairing SELinux Issues

Resolve a database service running issue.

Outcomes

You should be able to resolve SELinux issues.

As the student user on the workstation machine, use the lab command to prepare your system for this exercise.

[student@workstation ~]$ lab start security-selinux

This command confirms that the required hosts for this exercise are accessible and installs the database service.

Instructions

A coworker recently installed a custom database service for an application, which used a non-default configuration in the service. Your coworker failed to get the service running. Your manager assigned you to solve the issue and set redhat as the password for the root user on the database.

[student@workstation ~]$ ssh student@servera
[student@servera ~]$ sudo -i
[sudo] password for student: student
[root@servera ~]#

Diagnose the failed service.

View the status of the mariadb service.

[root@servera ~]# systemctl status mariadb.service
● mariadb.service - MariaDB 10.3 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Wed 2021-11-10 00:10:05 EST; 22s ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 8362 ExecStart=/usr/libexec/mysqld --basedir=/usr $MYSQLD_OPTS $_WSREP_NEW_CLUSTER (code=exited, status=1/FAILURE)
  Process: 8323 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mariadb.service (code=exited, status=0/SUCCESS)
  Process: 8298 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
 Main PID: 8362 (code=exited, status=1/FAILURE)
   Status: "MariaDB server is down"
...output omitted...

Restart the mariadb service to verify that the service fails.

[root@servera ~]# systemctl restart mariadb.service
Job for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xe" for details.

View the last log entry for the failed service.

The output reports several files with SELinux access failures, possibly because of customization of the service.

[root@servera ~]# journalctl -r
-- Logs begin at Tue 2021-11-09 23:22:48 EST, end at Wed 2021-11-10 00:15:55 EST. --
Nov 10 00:15:55 servera.lab.example.com setroubleshoot[8566]: AnalyzeThread.run(): Set alarm timeout to 10
Nov 10 00:15:55 servera.lab.example.com setroubleshoot[8566]: SELinux is preventing /usr/libexec/mysqld from write access on the file /opt/database/aria_log_control.

       *****  Plugin catchall_labels (83.8 confidence) suggests   *****

       If you want to allow mysqld to have write access on the aria_log_control file
       Then you need to change the label on /opt/database/aria_log_control
       Do
       # semanage fcontext -a -t FILE_TYPE '/opt/database/aria_log_control'
       where FILE_TYPE is one of the following: afs_cache_t, faillog_t, hugetlbfs_t, initrc_tmp_t, krb5_host_rcache_t, lastlog_t, mysqld_db_t, mysqld_log_t, mysqld_tmp_t, mysqld_var_run_t, puppet_tmp_t, security_t, user_cron_spool_t.
       Then execute:
       restorecon -v '/opt/database/aria_log_control'
...output omitted...

Troubleshoot and resolve the SELinux file context issue.

Search for all AVC denials in the audit.log file.

Look for a message that correlates to the previously viewed error in the journal.

[root@servera ~]# ausearch -m avc -ts recent
---
time->Wed Nov 10 00:29:39 2021
type=PROCTITLE msg=audit(1636522179.002:1164): proctitle=2F7573722F6C6962657865632F6D7973716C64002D2D626173656469723D2F757372
type=SYSCALL msg=audit(1636522179.002:1164): arch=c000003e syscall=257 success=no exit=-13 a0=ffffff9c a1=7ffe36a29310 a2=80042 a3=1b6 items=0 ppid=1 pid=8691 auid=4294967295 uid=27 gid=27 euid=27 suid=27 fsuid=27 egid=27 sgid=27 fsgid=27 tty=(none) ses=4294967295 comm="mysqld" exe="/usr/libexec/mysqld" subj=system_u:system_r:mysqld_t:s0 key=(null)
type=AVC msg=audit(1636522179.002:1164): avc:  denied  { write } for  pid=8691 comm="mysqld" name="database" dev="vda3" ino=25793706 scontext=system_u:system_r:mysqld_t:s0 tcontext=unconfined_u:object_r:usr_t:s0 tclass=dir permissive=0 
---
...output omitted...

The message indicates denied access to mysqld for writing to the database directory.

View the database configuration file to discover the service customization.

[root@servera ~]# less /etc/my.cnf.d/mariadb-server.cnf
...output omitted...
[mysqld]
datadir=/opt/database/
socket=/var/lib/mysql/mysql.sock
log-error=/var/log/database/mariadb.log
pid-file=/run/mariadb/mariadb.pid
...output omitted...

Verify the context of the directory that is configured as datadir.

[root@servera ~]# ls -dZ /opt/database
unconfined_u:object_r:usr_t:s0 /opt/database

Review the default datadir directory context.

[root@servera ~]# semanage fcontext --list | grep '/var/lib/mysql'
/var/lib/mysql(-files|-keyring)?(/.*)?  all files  system_u:object_r:mysqld_db_t:s0
/var/lib/mysql/mysql\.sock              socket     system_u:object_r:mysqld_var_run_t:s0

Set the persistent file context for the customized datadir directory.

[root@servera ~]# semanage fcontext -a -t mysqld_db_t "/opt/database(/.*)?"
[root@servera ~]# restorecon -Rv /opt/database
Relabeled /opt/database from unconfined_u:object_r:usr_t:s0 to unconfined_u:object_r:mysqld_db_t:s0
Relabeled /opt/database/ibdata1 from system_u:object_r:usr_t:s0 to system_u:object_r:mysqld_db_t:s0
Relabeled /opt/database/ib_logfile1 from system_u:object_r:usr_t:s0 to system_u:object_r:mysqld_db_t:s0
...output omitted...

Restart the mariadb service to verify the fix.

[root@servera ~]# systemctl restart mariadb.service
Job for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xe" for details.

Troubleshoot and resolve the SELinux port issue.

View the last log entry for the failed service.

[root@servera ~]# journalctl -r
-- Logs begin at Tue 2021-11-09 23:22:48 EST, end at Wed 2021-11-10 00:55:37 EST. --
Nov 10 00:55:37 servera.lab.example.com setroubleshoot[8870]: AnalyzeThread.run(): Set alarm timeout to 10
Nov 10 00:55:37 servera.lab.example.com setroubleshoot[8870]: SELinux is preventing mysqld from name_bind access on the tcp_socket port 3307.

       *****  Plugin bind_ports (92.2 confidence) suggests   *****

       If you want to allow mysqld to bind to network port 3307
       Then you need to modify the port type.
       Do
       # semanage port -a -t PORT_TYPE -p tcp 3307
           where PORT_TYPE is one of the following: mysqld_port_t, tram_port_t.
...output omitted...

View the database configuration file to discover the service port customization.
```
[root@servera ~]# grep port /etc/my.cnf.d/mariadb-server.cnf
port = 3307
```

Set the port context for the customized service port.

[root@servera ~]# semanage port --list | grep mysqld
mysqld_port_t   tcp   1186, 3306, 63132-63164
[root@servera ~]# semanage port -a -t mysqld_port_t -p tcp 3307

Restart the mariadb service to verify the fix.

[root@servera ~]# systemctl restart mariadb.service
[root@servera ~]# systemctl status mariadb.service
● mariadb.service - MariaDB 10.3 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2021-11-10 01:05:25 EST; 3s ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 9020 ExecStartPost=/usr/libexec/mysql-check-upgrade (code=exited, status=0/SUCCESS)
  Process: 8950 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mariadb.service (code=exited, status=0/SUCCESS)
  Process: 8926 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
 Main PID: 8989 (mysqld)
   Status: "Taking your SQL requests now..."
...output omitted...

Complete the database configuration and verify that the service is functional.

Stop the database service and set the password of the database root user.

[root@servera ~]# systemctl stop mariadb.service
[root@servera ~]# mysqld_safe --skip-grant-tables --skip-networking &
[1] 9082
211110 01:10:06 mysqld_safe Logging to '/var/log/database/mariadb.log'.
211110 01:10:06 mysqld_safe Starting mysqld daemon with databases from /opt/database/
[root@servera ~]# mysql -u root
...output omitted...
MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]> ALTER USER 'root'@'localhost' IDENTIFIED BY 'redhat';
Query OK, 0 rows affected (0.001 sec)

MariaDB [(none)]> exit
Bye
[root@servera ~]#

Restart the database service and confirm the password setting of the database root user.

[root@servera ~]# pkill mysql
[root@servera ~]# systemctl restart mariadb.service
[1]+  Done    mysqld_safe --skip-grant-tables --skip-networking
[root@servera ~]# mysql -u root -p
Enter password: redhat
...output omitted...
MariaDB [(none)]> exit
Bye
[root@servera ~]#

View the database service log to verify that event entries are being logged.

The log is not recording database event entries.

[root@servera ~]# tail /var/log/database/mariadb.log
...output omitted...
2021-11-10  1:17:02 0 [Note] InnoDB: Buffer pool(s) dump completed at 211110  1:17:02
2021-11-10  1:17:03 0 [Note] InnoDB: Shutdown completed; log sequence number 1625470; transaction id 21
2021-11-10  1:17:03 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2021-11-10  1:17:03 0 [Note] /usr/libexec/mysqld: Shutdown complete

Troubleshoot and resolve the SELinux context issue on the database log.

View the last log entry for the failed service.

[root@servera ~]# journalctl -r
...output omitted...
Nov 10 01:18:05 servera.lab.example.com setroubleshoot[9395]: SELinux is preventing /usr/libexec/mysqld from open access on the file /var/log/database/mariadb.log.

       *****  Plugin catchall (100. confidence) suggests   *****

       If you believe that mysqld should be allowed open access on the mariadb.log file by default.
       Then you should report this as a bug.
       You can generate a local policy module to allow this access.
       Do
       allow this access for now by executing:
       # ausearch -c 'mysqld' --raw | audit2allow -M my-mysqld
       # semodule -X 300 -i my-mysqld.pp
...output omitted...

Search for all AVC denials in the audit.log file.

Look for a message that correlates to the previously viewed error in the journal.

[root@servera ~]# ausearch -m avc -ts recent
...output omitted...
---
time->Wed Nov 10 01:18:03 2021
type=PROCTITLE msg=audit(1636525083.284:1234): proctitle=2F7573722F6C6962657865632F6D7973716C64002D2D626173656469723D2F757372
type=SYSCALL msg=audit(1636525083.284:1234): arch=c000003e syscall=257 success=no exit=-13 a0=ffffff9c a1=55a350340180 a2=441 a3=1b6 items=0 ppid=1 pid=9330 auid=4294967295 uid=27 gid=27 euid=27 suid=27 fsuid=27 egid=27 sgid=27 fsgid=27 tty=(none) ses=4294967295 comm="mysqld" exe="/usr/libexec/mysqld" subj=system_u:system_r:mysqld_t:s0 key=(null)
type=AVC msg=audit(1636525083.284:1234): avc:  denied  { open } for  pid=9330 comm="mysqld" path="/var/log/database/mariadb.log" dev="vda3" ino=435268 scontext=system_u:system_r:mysqld_t:s0 tcontext=unconfined_u:object_r:var_log_t:s0 tclass=file permissive=0

The message indicates denied access to mysqld for writing the /var/log/database/mariadb.log file.

Set the persistent file context for the customized log file.

[root@servera ~]# semanage fcontext --list | grep mysqld_log_t
/var/log/mariadb(/.*)?   all files      system_u:object_r:mysqld_log_t:s0
/var/log/mysql(/.*)?     all files      system_u:object_r:mysqld_log_t:s0
/var/log/mysql.*         regular file   system_u:object_r:mysqld_log_t:s0
[root@servera ~]# semanage fcontext -a -t mysqld_log_t "/var/log/database(/.*)?"
[root@servera ~]# restorecon -Rv /var/log/database
Relabeled /var/log/database from unconfined_u:object_r:var_log_t:s0 to unconfined_u:object_r:mysqld_log_t:s0
Relabeled /var/log/database/mariadb.log from unconfined_u:object_r:var_log_t:s0 to unconfined_u:object_r:mysqld_log_t:s0

Restart the mariadb service to verify the fix.

[root@servera ~]# systemctl restart mariadb.service
[root@servera ~]# tail /var/log/database/mariadb.log
...output omitted...
2021-11-10  1:44:04 0 [Note] Reading of all Master_info entries succeeded
2021-11-10  1:44:04 0 [Note] Added new Master_info '' to hash table
2021-11-10  1:44:04 0 [Note] /usr/libexec/mysqld: ready for connections.
Version: '10.3.27-MariaDB'  socket: '/var/lib/mysql/mysql.sock'  port: 3307  MariaDB Server

Return to workstation as the student user.

[root@servera ~]# exit
[student@servera ~]$ exit
[student@workstation ~]$

Finish

On the workstation machine, use the lab command to complete this exercise. This is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish security-selinux

Discuss Red Hat Enterprise Linux Diagnostics and Troubleshooting

Go to community

Welcome to the Red Hat Enterprise Linux Diagnostics and Troubleshooting (RH342) group!

cschunke

26 wrz 2023

We are excited to launch a space dedicated to the Red Hat Training course Red Hat Enterprise Linux Diagnostics and Troubleshooting! To gain the most value from this group - click the "Join Group" button in the upper right hand corner of the group home page.We encourage group members to collaborate in this group to discuss topics, ask questions, share best practices and tips, provide course feedback, and share their accomplishments as it relates to RH342.Read more about Red Hat Enterprise Linux Diagnostics and Troubleshooting here.

445

Revision: rh342-8.4-6dd89bd

Click Create to build all of the virtual machines needed for the classroom lab environment. This may take several minutes to complete. Once created the environment can then be stopped and restarted to pause your experience.
When a lab is created, click Start to run all of the virtual machines in the classroom.
Click Stop to stop all the virtual machines from running. This will not delete your lab.
If you Delete your lab, you will remove all of the virtual machines in your classroom and lose all of your progress.

Virtual machine actions

Click Start to power on the virtual machine.
Click Shutdown to gracefully shut down the virtual machine, preserving disk contents.
Click Power off to forcefully shut down the virtual machine, while still preserving disk contents.
Click Open console to connect to the system console of the virtual machine in a new browser.

Auto-stop timer

The Red Hat Learning Subscription entitles you to set allotment of lab time.
To help conserving your allotted time, the lab environment uses automatic timers to stop or destroy your lab environment when the timer expires.

Click the Auto-stop button [+] to extend the time you would like to spend with the labs.
Click the Auto-destroy button [+] to add day(s) to the auto-destroy timer.

Auto-stop has a maximum of 11 hours, and auto-destroy has a maximum of 14 days.
Be careful to keep the timers set while you are working, so that your environment doesn't shut down unexpectedly.
We also suggest not to set the auto-timers unnecessarily high, which could waste your lab time allotment