Bookmark this page

Chapter 6. Tuning System Performance

Abstract

Goal Evaluate and control processes, set tuning parameters and adjust process scheduling priorities on a Red Hat Enterprise Linux system.
Objectives
  • Control and terminate processes that are not associated with your shell, and forcibly end user sessions and processes.

  • Describe what load average is and determine processes responsible for high resource use on a server.

  • Optimize system performance by selecting a tuning profile managed by the tuned daemon.

  • Prioritize or de-prioritize specific processes, with the nice and renice commands.

Sections
  • Killing Processes (and Guided Exercise)

  • Monitoring Process Activity (and Guided Exercise)

  • Adjusting Tuning Profiles (and Guided Exercise)

  • Influencing Process Scheduling (and Guided Exercise)

Lab

Tuning System Performance

Killing Processes

Objectives

After completing this section, you should be able to:

  • Use commands to kill and communicate with processes.

  • Define the characteristics of a daemon process.

  • End user sessions and processes.

Process control using signals

A signal is a software interrupt delivered to a process. Signals report events to an executing program. Events that generate a signal can be an error, external event (an I/O request or an expired timer), or by explicit use of a signal-sending command or keyboard sequence.

The following table lists the fundamental signals used by system administrators for routine process management. Refer to signals by either their short (HUP) or proper (SIGHUP) name.

Table 6.1. Fundamental Process Management Signals

Signal numberShort nameDefinitionPurpose
1HUPHangup

Used to report termination of the controlling process of a terminal. Also used to request process reinitialization (configuration reload) without termination.

2INTKeyboard interrupt

Causes program termination. Can be blocked or handled. Sent by pressing INTR key sequence (Ctrl+c).

3QUITKeyboard quit

Similar to SIGINT; adds a process dump at termination. Sent by pressing QUIT key sequence (Ctrl+\).

9KILLKill, unblockable

Causes abrupt program termination. Cannot be blocked, ignored, or handled; always fatal.

15 default TERMTerminate

Causes program termination. Unlike SIGKILL, can be blocked, ignored, or handled. The polite way to ask a program to terminate; allows self-cleanup.

18CONTContinue

Sent to a process to resume, if stopped. Cannot be blocked. Even if handled, always resumes the process.

19STOPStop, unblockable

Suspends the process. Cannot be blocked or handled.

20TSTPKeyboard stop

Unlike SIGSTOP, can be blocked, ignored, or handled. Sent by pressing SUSP key sequence (Ctrl+z).


Note

Signal numbers vary on different Linux hardware platforms, but signal names and meanings are standardized. For command use, it is advised to use signal names instead of numbers. The numbers discussed in this section are for x86_64 systems.

Each signal has a default action, usually one of the following:

  • Term - Cause a program to terminate (exit) at once.

  • Core - Cause a program to save a memory image (core dump), then terminate.

  • Stop - Cause a program to stop executing (suspend) and wait to continue (resume).

Programs can be prepared to react to expected event signals by implementing handler routines to ignore, replace, or extend a signal's default action.

Commands for Sending Signals by Explicit Request

You signal the current foreground process by pressing a keyboard control sequence to suspend (Ctrl+z), kill (Ctrl+c), or core dump (Ctrl+\) the process. However, you will use signal-sending commands to send signals to a background process or to processes in a different session.

Signals can be specified as options either by name (for example, -HUP or -SIGHUP) or by number (the related -1). Users may kill their own processes, but root privilege is required to kill processes owned by others.

The kill command sends a signal to a process by PID number. Despite its name, the kill command can be used to send any signal, not just those for terminating programs. You can use the kill -l command to list the names and numbers of all available signals.

[user@host ~]$ kill -l
 1) SIGHUP      2) SIGINT      3) SIGQUIT     4) SIGILL      5) SIGTRAP
 6) SIGABRT     7) SIGBUS      8) SIGFPE      9) SIGKILL    10) SIGUSR1
11) SIGSEGV    12) SIGUSR2    13) SIGPIPE    14) SIGALRM    15) SIGTERM
16) SIGSTKFLT  17) SIGCHLD    18) SIGCONT    19) SIGSTOP    20) SIGTSTP
...output omitted...
[user@host ~]$ ps aux | grep job
5194  0.0  0.1 222448  2980 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job1
5199  0.0  0.1 222448  3132 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job2
5205  0.0  0.1 222448  3124 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job3
5430  0.0  0.0 221860  1096 pts/1    S+   16:41   0:00 grep --color=auto job
[user@host ~]$ kill 5194
[user@host ~]$ ps aux | grep job
user   5199  0.0  0.1 222448  3132 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job2
user   5205  0.0  0.1 222448  3124 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job3
user   5783  0.0  0.0 221860   964 pts/1    S+   16:43   0:00 grep --color=auto job
[1]   Terminated              control job1
[user@host ~]$ kill -9 5199
[user@host ~]$ ps aux | grep job
user   5205  0.0  0.1 222448  3124 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job3
user   5930  0.0  0.0 221860  1048 pts/1    S+   16:44   0:00 grep --color=auto job
[2]-  Killed                  control job2
[user@host ~]$ kill -SIGTERM 5205
user   5986  0.0  0.0 221860  1048 pts/1    S+   16:45   0:00 grep --color=auto job
[3]+  Terminated              control job3

The killall command can signal multiple processes, based on their command name.

[user@host ~]$ ps aux | grep job
5194  0.0  0.1 222448  2980 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job1
5199  0.0  0.1 222448  3132 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job2
5205  0.0  0.1 222448  3124 pts/1    S    16:39   0:00 /bin/bash /home/user/bin/control job3
5430  0.0  0.0 221860  1096 pts/1    S+   16:41   0:00 grep --color=auto job
[user@host ~]$ killall control
[1]   Terminated              control job1
[2]-  Terminated              control job2
[3]+  Terminated              control job3
[user@host ~]$ 

Use pkill to send a signal to one or more processes which match selection criteria. Selection criteria can be a command name, a process owned by a specific user, or all system-wide processes. The pkill command includes advanced selection criteria:

  • Command - Processes with a pattern-matched command name.

  • UID - Processes owned by a Linux user account, effective or real.

  • GID - Processes owned by a Linux group account, effective or real.

  • Parent - Child processes of a specific parent process.

  • Terminal - Processes running on a specific controlling terminal.

[user@host ~]$ ps aux | grep pkill
user   5992  0.0  0.1 222448  3040 pts/1    S    16:59   0:00 /bin/bash /home/user/bin/control pkill1
user   5996  0.0  0.1 222448  3048 pts/1    S    16:59   0:00 /bin/bash /home/user/bin/control pkill2
user   6004  0.0  0.1 222448  3048 pts/1    S    16:59   0:00 /bin/bash /home/user/bin/control pkill3
[user@host ~]$ pkill control
[1]   Terminated              control pkill1
[2]-  Terminated              control pkill2
[user@host ~]$ ps aux | grep pkill
user   6219  0.0  0.0 221860  1052 pts/1    S+   17:00   0:00 grep --color=auto pkill
[3]+  Terminated              control pkill3
[user@host ~]$ ps aux | grep test
user   6281  0.0  0.1 222448  3012 pts/1    S    17:04   0:00 /bin/bash /home/user/bin/control test1
user   6285  0.0  0.1 222448  3128 pts/1    S    17:04   0:00 /bin/bash /home/user/bin/control test2
user   6292  0.0  0.1 222448  3064 pts/1    S    17:04   0:00 /bin/bash /home/user/bin/control test3
user   6318  0.0  0.0 221860  1080 pts/1    S+   17:04   0:00 grep --color=auto test 
[user@host ~]$ pkill -U user
[user@host ~]$ ps aux | grep test
user   6870  0.0  0.0 221860  1048 pts/0    S+   17:07   0:00 grep --color=auto test
[user@host ~]$ 

Logging Users Out Administratively

You may need to log other users off for any of a variety of reasons. To name a few of the many possibilities: the user committed a security violation; the user may have overused resources; the user may have an unresponsive system; or the user has improper access to materials. In these cases, you may need to administratively terminate their session using signals.

To log off a user, first identify the login session to be terminated. Use the w command to list user logins and current running processes. Note the TTY and FROM columns to determine the sessions to close.

All user login sessions are associated with a terminal device (TTY). If the device name is of the form pts/N, it is a pseudo-terminal associated with a graphical terminal window or remote login session. If it is of the form ttyN, the user is on a system console, alternate console, or other directly connected terminal device.

[user@host ~]$ w
 12:43:06 up 27 min,  5 users,  load average: 0.03, 0.17, 0.66
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     tty2                      12:26   14:58   0.04s  0.04s -bash
bob      tty3                      12:28   14:42   0.02s  0.02s -bash
user     pts/1    desk.example.com 12:41    2.00s  0.03s  0.03s w
[user@host ~]$ 

Discover how long a user has been on the system by viewing the session login time. For each session, CPU resources consumed by current jobs, including background tasks and child processes, are in the JCPU column. Current foreground process CPU consumption is in the PCPU column.

Processes and sessions can be individually or collectively signaled. To terminate all processes for one user, use the pkill command. Because the initial process in a login session (session leader) is designed to handle session termination requests and ignore unintended keyboard signals, killing all of a user's processes and login shells requires using the SIGKILL signal.

Important

SIGKILL is commonly used too quickly by administrators.

Since the SIGKILL signal cannot be handled or ignored, it is always fatal. However, it forces termination without allowing the killed process to run self-cleanup routines. It is recommended to send SIGTERM first, then try SIGINT, and only if both fail retry with SIGKILL.

First identify the PID numbers to be killed using pgrep, which operates much like pkill, including using the same options, except that pgrep lists processes rather than killing them.

[root@host ~]# pgrep -l -u bob
6964 bash
6998 sleep
6999 sleep
7000 sleep
[root@host ~]# pkill -SIGKILL -u bob
[root@host ~]# pgrep -l -u bob
[root@host ~]# 

When processes requiring attention are in the same login session, it may not be necessary to kill all of a user's processes. Determine the controlling terminal for the session using the w command, then kill only processes referencing the same terminal ID. Unless SIGKILL is specified, the session leader (here, the Bash login shell) successfully handles and survives the termination request, but all other session processes are terminated.

[root@host ~]# pgrep -l -u bob
7391 bash
7426 sleep
7427 sleep
7428 sleep
[root@host ~]# w -h -u bob
bob      tty3      18:37    5:04   0.03s  0.03s -bash
[root@host ~]# pkill -t tty3
[root@host ~]# pgrep -l -u bob
7391 bash
[root@host ~]# pkill -SIGKILL -t tty3
[root@host ~]# pgrep -l -u bob
[root@host ~]# 

The same selective process termination can be applied using parent and child process relationships. Use the pstree command to view a process tree for the system or a single user. Use the parent process's PID to kill all children they have created. This time, the parent Bash login shell survives because the signal is directed only at its child processes.

[root@host ~]# pstree -p bob
bash(8391)─┬─sleep(8425)
           ├─sleep(8426)
           └─sleep(8427)
[root@host ~]# pkill -P 8391
[root@host ~]# pgrep -l -u bob
bash(8391)
[root@host ~]# pkill -SIGKILL -P 8391
[root@host ~]# pgrep -l -u bob
bash(8391)
[root@host ~]# 

References

info libc signal (GNU C Library Reference Manual)

  • Section 24: Signal Handling

info libc processes (GNU C Library Reference Manual)

  • Section 26: Processes

kill(1), killall(1), pgrep(1), pkill(1), pstree(1), signal(7), and w(1) man pages

Revision: rh199-8.2-3beeb12