Red Hat Enterprise Linux Diagnostics and Troubleshooting

Instructions

A request was submitted to allocate an additional 20 MiB of storage to the /mnt/lvm directory on servera. After the request was completed, a user reported that this directory is missing content. The user suggests that critical files were deleted during the requested allocation. The problem was escalated and you are assigned to identify the root cause of the problem. When root cause analysis is complete, restore the system to proper working order.

1. Log in to servera and switch to the root user.

   [student@workstation ~]$ ssh student@servera
   ...output omitted...
   [student@servera ~]$ sudo -i
   [sudo] password for student: student
   [root@servera ~]#

2. Attempt to list the content of the /mnt/lvm directory to confirm the issue that the user reported.

   [root@servera ~]# ll /mnt/lvm
   total 0

   The directory is empty, which means that the content is missing.

3. Check the contents of the /etc/fstab file to determine whether the directory contents mount on a separate file system.

   [root@servera ~]# grep /mnt/lvm /etc/fstab
   /dev/vg00/lvol0      /mnt/lvm     xfs     defaults     0 0

4. Verify that the file system on the logical volume is mounted at /mnt/lvm.

   [root@servera ~]# mount | grep /mnt/lvm
   [root@servera ~]#

5. Because the file system is not mounted, attempt to mount it manually.

   [root@servera ~]# mount /mnt/lvm
   mount: /mnt/lvm: can't read superblock on /dev/mapper/vg00-lvol0.

6. Consult the list of metadata backup and archive files for the volume group where the /dev/vg00/lvol0 logical volume resides to determine any changes to the logical volume configuration.

   [root@servera ~]# vgcfgrestore -l vg00
    ...output omitted...
   
     File:		/etc/lvm/archive/vg00_00002-1457658928.vg
     VG name:    	vg00
     Description:	Created before executing 'lvresize -L20M -f /dev/vg00/lvol0'
     Backup Time:	Mon Oct  4 23:55:42 2021
   
   
     File:		/etc/lvm/backup/vg00
     VG name:    	vg00
     Description:	Created after executing 'lvresize -L20M -f /dev/vg00/lvol0'
     Backup Time:	Mon Oct  4 23:55:42 2021
   
    ...output omitted...

7. Analyze the descriptions of each archive log and identify the archive that existed before the space allocation on the logical volume. Determine whether the recorded command in the archive log caused the issue that the user observed.

    ...output omitted...
   
     File:		/etc/lvm/archive/vg00_00002-1457658928.vg
     VG name:    	vg00
     Description:	Created before executing 'lvresize -L20M -f /dev/vg00/lvol0'
     Backup Time:	Mon Oct  4 23:55:42 2021
   
    ...output omitted...

   Rather than allocating another 20 MiB to the logical volume with the -L+20M option, the administrator erroneously reduced the logical volume to 20 MiB.

8. Revert the incorrect action that was executed during the attempt to allocate space to the logical volume.

   [root@servera ~]# vgcfgrestore -f /etc/lvm/archive/vg00_00002-1457658928.vg vg00
     Volume group vg00 has active volume: lvol0.
     WARNING: Found 1 active volume(s) in volume group "vg00".
     Restoring VG with active LVs, may cause mismatch with its metadata.
   Do you really want to proceed with restore of volume group "vg00", while 1 volume(s) are active? [y/n]: y
     Restored volume group vg00

9. Deactivate and reactivate the /dev/vg00/lvol0 logical volume to ensure that everything is updated.

   [root@servera ~]# lvchange -an /dev/vg00/lvol0
   [root@servera ~]# lvchange -ay /dev/vg00/lvol0

10. Mount the file system to determine whether the issue is resolved.

    [root@servera ~]# mount /mnt/lvm
    [root@servera ~]# ll /mnt/lvm
    total 12
    drwxr-xr-x. 94 root root     8192 Jan 19 22:26 etc

11. Correctly fulfill the user's request by allocating another 20 MiB to the /dev/vg00/lvol0 file system.

    1. Resize the logical volume:

       [root@servera ~]# lvresize -L+20M /dev/vg00/lvol0
         Size of logical volume vg00/lvol0 changed from 40.00 MiB (10 extents) to 60.00 MiB (15 extents).
         Logical volume lvol0 successfully resized.

    2. Increase the file system on the logical volume:

       [root@servera ~]# xfs_growfs /dev/vg00/lvol0
       meta-data=/dev/mapper/vg00-lvol0 isize=512    agcount=2, agsize=5120 blks
                =                       sectsz=512   attr=2, projid32bit=1
                =                       crc=1        finobt=1, sparse=1, rmapbt=0
                =                       reflink=1
       data     =                       bsize=4096   blocks=10240, imaxpct=25
                =                       sunit=0      swidth=0 blks
       naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
       log      =internal log           bsize=4096   blocks=1368, version=2
                =                       sectsz=512   sunit=0 blks, lazy-count=1
       realtime =none                   extsz=4096   blocks=0, rtextents=0
       data blocks changed from 10240 to 15360

12. Return to workstation as the student user.

    [root@servera ~]# exit
    [student@servera ~]$ exit
    [student@workstation ~]$