Red Hat Enterprise Linux Diagnostics and Troubleshooting
Configure and test kernel crash dumps.
Outcomes
You should be able to configure the kdump service to send a crash dump to a remote server.
As the student user on the workstation machine, use the lab command to prepare your system for this exercise.
[student@workstation ~]# lab start kernel-crashdump
This command confirms that your systems are reachable.
Instructions
You are tasked to configure the servera system to send crash dumps to serverb. Use the scp command to send crash dumps. Store crash dumps in the /var/crash directory on serverb.
Log in to
serveraand switch to therootuser.[student@workstation ~]$
ssh student@servera...output omitted... [student@servera ~]$sudo -i[sudo] password for student:student[root@servera ~]#Enable and start the
kdumpservice.[root@servera ~]#
systemctl enable --now kdumpCreated symlink from /etc/systemd/system/multi-user.target.wants/kdump.service to /usr/lib/systemd/system/kdump.service. [root@servera ~]#systemctl status kdump● kdump.service - Crash recovery kernel arming Loaded: loaded (/usr/lib/systemd/system/kdump.service; disabled; vendor preset: enabled) Active: active (exited) since Wed 2021-11-10 16:26:46 EST; 3s ago Process: 1439 ExecStart=/usr/bin/kdumpctl start (code=exited, status=0/SUCCESS) Main PID: 1439 (code=exited, status=0/SUCCESS) Nov 10 16:26:45 servera.lab.example.com systemd[1]: Starting Crash recovery kernel arming... Nov 10 16:26:46 servera.lab.example.com kdumpctl[1439]: kdump: kexec: loaded kdump kernel Nov 10 16:26:46 servera.lab.example.com kdumpctl[1439]: kdump: Starting kdump: [OK] Nov 10 16:26:46 servera.lab.example.com systemd[1]: Started Crash recovery kernel arming.Modify the
/etc/kdump.conffile to send crash dumps to the/var/crashdirectory onserverb.Set
serverb.lab.example.comas the SSH target.ssh root@serverb.lab.example.com
Set
/root/.ssh/kdump_id_rsaas the SSH key.sshkey /root/.ssh/kdump_id_rsa
Modify the
core_collectorline to usescp. Use this line to replace any existingcore_collectorlines.core_collector scp
Set the
pathparameter to the/var/crashdirectory.path /var/crash
Generate and remotely install the SSH key on
serverb. If prompted, enterredhatas the password.[root@servera ~]#
kdumpctl propagatekdump: WARNING: '/root/.ssh/kdump_id_rsa' doesn't exist, using default value '/root/.ssh/kdump_id_rsa' kdump: Generating new ssh keys... kdump: done. /bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/kdump_id_rsa.pub" The authenticity of host 'serverb.lab.example.com (172.25.250.11)' can't be established. ECDSA key fingerprint is SHA256:NJAyJMx8B2AeIYHRnVLAuJ1XZwblomyOKowyfTwGrTY. Are you sure you want to continue connecting (yes/no/[fingerprint])?yes/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@serverb.lab.example.com's password:redhatNumber of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@serverb.lab.example.com'" and check to make sure that only the key(s) you wanted were added. kdump: /root/.ssh/kdump_id_rsa has been added to ~root/.ssh/authorized_keys on serverb.lab.example.comRestart the
kdumpservice to implement the configuration file changes.[root@servera ~]#
systemctl restart kdump[root@servera ~]#kdumpctl statuskdump: Kdump is operationalTrigger a system crash to test the
kdumpconfiguration.Enable SysRq functions by setting the value in the
/proc/sys/kernel/sysrqfile to1.[root@servera ~]#
echo 1 > /proc/sys/kernel/sysrqTrigger a system crash by setting the value in the
/proc/sysrq-triggerfile toc.[root@servera ~]#
echo c > /proc/sysrq-triggerNote
The SSH session hangs when the system crash is triggered. Press any key to return to the
workstationsystem.
Log in to
serverband switch to therootuser.[student@workstation ~]$
ssh student@serverb...output omitted... [student@serverb ~]$sudo -i[sudo] password for student:student[root@serverb ~]#Verify that the crash dump is present in the
/var/crashdirectory and view its contents.List the contents of the
/var/crashdirectory.[root@serverb ~]#
ll /var/crashtotal 0 drwxr-xr-x. 2 root root 67 Nov 10 16:44 172.25.250.10-2021-11-10-16:43:54View the contents of the crash dump.
[root@serverb ~]#
ll /var/crash/172.25.250.10-2021-11-10-16\:43\:54/total 1977448 -rw-r--r--. 1 root root 52465 Nov 10 16:44 kexec-dmesg.log -r--------. 1 root root 2024812544 Nov 10 16:44 vmcore -rw-r--r--. 1 root root 39176 Nov 10 16:44 vmcore-dmesg.txt
Return to
workstationas thestudentuser.[root@serverb ~]#
exit[student@serverb ~]$exit[student@workstation ~]$