Red Hat Enterprise Linux Diagnostics and Troubleshooting

Bookmark this page
P1234567891011
PreviousNext

Guided Exercise: Collecting Information to Support Troubleshooting

Use log files to troubleshoot an issue with a web server.

Outcomes

You should be able to use log files to troubleshoot a web server issue.

As the student user on the workstation machine, use the lab command to prepare your system for this exercise.

[student@workstation ~]$ lab start strategy-collectinginfo

This command prepares workstation and servera for troubleshooting file access issues.

Instructions

Your servera machine is running a web server, serving the file http://servera.lab.example.com/test.html. A ticket came in from your testing manager that this file is not accessible from a web browser. No further information is given in the ticket.

Investigate this issue by using the log files on servera, and then fix the issue. For testing from the command line on workstation, as an alternative to opening a graphical browser, you can use the command curl http://servera.lab.example.com/test.html.

  1. Begin by trying to reproduce the problem.

    1. As student on workstation, attempt to access http://servera.lab.example.com/test.html. You can do this with a Firefox browser, or by executing the following command:

      [student@workstation ~]$ curl http://servera.lab.example.com/test.html
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access this resource.</p>
</body></html>

    2. Consider the possible causes for the HTTP 403 Forbidden error that you encountered. This error can have various reasons: file permissions, SELinux types, internal httpd configurations, and others.

      You know that the web server itself is running; you got an answer; and the firewall is open.

  2. Log in to servera and switch to the root user.

    [student@workstation ~]$ ssh student@servera
...output omitted...
[student@servera ~]$ sudo -i
[sudo] password for student: student
[root@servera ~]#

  3. Collect information from the web server logs on servera. The main logs for httpd are /var/log/httpd/access_log for all access attempts and /var/log/httpd/error_log for all errors.

    1. Check /var/log/httpd/access_log for any message about this failure.

      [root@servera ~]# grep test.html /var/log/httpd/access_log
...output omitted...
172.25.250.9 - - [02/Sep/2021:22:39:46 -0400] "GET /test.html HTTP/1.1" 403 199 "-" "curl/7.61.1"
...output omitted...

      The 403 in this output is the HTTP status code. Otherwise, you can see the requested URL, the date and time of the request, and the user agent that was used, but nothing that can help you further with this problem.

    2. Check /var/log/httpd/error_log for any message about this failure.

      [root@servera ~]# tail /var/log/httpd/error_log
...output omitted...
[Thu Sep 02 22:39:46.533187 2021] [core:error] [pid 6082:tid 140537780700928] (13)Permission denied: [client 172.25.250.9:51002] AH00035: access to /test.html denied (filesystem path '/var/www/html/test.html') because search permissions are missing on a component of the path
...output omitted...

      This message tells you that httpd is blocked by file permissions from reading the test.html file. This message rules out an internal configuration error for httpd, but leaves file permissions and SELinux as possible culprits.

  4. Inspect the file permissions on /var/www/html/test.html, and fix if necessary.

    1. Inspect the file permissions on /var/www/html/test.html.

      [root@servera ~]# ls -l /var/www/html/test.html
-rw-------. 1 root root 8 Sep  2 22:39 /var/www/html/test.html

    2. Those permissions do not look correct. Make the file world-readable.

      [root@servera ~]# chmod 644 /var/www/html/test.html

    3. Confirm that the permissions are now correct.

      [root@servera ~]# ls -l /var/www/html/test.html
-rw-r--r--. 1 root root 8 Sep  2 23:49 /var/www/html/test.html

    4. Return to workstation as the student user.

      [root@servera ~]# exit
[student@servera ~]$ exit
[student@workstation ~]$

  5. Test access to the file again, with either Firefox or curl.

    [student@workstation ~]$ curl http://servera.lab.example.com/test.html
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access this resource.</p>
</body></html>

    File permissions were an issue, but the problem is still not solved. Therefore one likely culprit remains: SELinux.

  6. Log in to servera and switch to the root user.

    [student@workstation ~]$ ssh student@servera
...output omitted...
[student@servera ~]$ sudo -i
[sudo] password for student: student
[root@servera ~]#

  7. Check the SELinux log for any denials that happened today, and fix any issues that you might spot.

    1. Check the SELinux log for any denials today.

      [root@servera ~]# ausearch -i -m avc -ts today
...output omitted...
type=AVC msg=audit(09/02/2021 22:39:46.532:4380) : avc:  denied  { getattr } for  pid=6082 comm=httpd path=/var/www/html/test.html dev="vda3" ino=1067530 scontext=system_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:samba_share_t:s0 tclass=file permissive=0
...output omitted...

      This message shows that the test.html file has an SELinux type of samba_share_t, which httpd is not allowed to open.

    2. Fix this issue by running a recursive restorecon on /var/www.

      [root@servera ~]# restorecon -Rv /var/www

    3. Return to workstation as the student user.

      [root@servera ~]# exit
[student@servera ~]$ exit
[student@workstation ~]$

  8. Test whether you can now access http://servera.lab.example.com/test.html from workstation.

    [student@workstation ~]$ curl http://servera.lab.example.com/test.html
ServerA

Finish

On the workstation machine, use the lab command to complete this exercise. This is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish strategy-collectinginfo

PreviousNext
Revision: rh342-8.4-6dd89bd