Red Hat Enterprise Linux Diagnostics and Troubleshooting
Verify a file system and repair file system corruption.
Outcomes
You should be able to check and repair an XFS file system.
As the student user on the workstation machine, use the lab command to prepare your system for this exercise.
[student@workstation ~]$ lab start storage-filesystem
This command creates the necessary file systems and files for this exercise.
Instructions
The /dev/vdb1 partition on servera contains an XFS file system, which holds the contents of the /etc directory from another system. Check the integrity of the XFS file system on /dev/vdb1. Repair any file system inconsistencies and then mount the file system at /mnt/etc_restore. If the file system repair relocates orphaned files to the file system's lost+found directory, then use the backup file, /root/etc.tgz, to determine the original file location and name. Restore the orphaned file to its proper location.
Log in to
serveraand switch to therootuser.[student@workstation ~]$
ssh student@servera...output omitted... [student@servera ~]$sudo -i[sudo] password for student:student[root@servera ~]#Check for any issues with the
/mnt/etc_restorefile system.Attempt to create the
/mnt/etc_restore/etc/testfile.[root@servera ~]#
echo hello > /mnt/etc_restore/etc/testYou can create the file without any issue.
View the contents of the
/mnt/etc_restore/etc/testfile.[root@servera ~]#
cat /mnt/etc_restore/etc/testhelloYou can view the content of the file without any issue.
List the files in the
/mnt/etc_restore/etc/security/console.appsdirectory.[root@servera ~]#
ls /mnt/etc_restore/etc/security/console.appsls: reading directory '/mnt/etc_restore/etc/security/console.apps/': Structure needs cleaningYou get an error message that the structure needs cleaning. Troubleshoot and resolve this issue.
Check the XFS file system on the
/dev/vdb1partition.Unmount the
/dev/vdb1file system, because a file system check cannot run on a mounted file system.[root@servera ~]#
umount /mnt/etc_restoreUse the
xfs_repaircommand with the-noption to check the file system with no corrective actions taken.[root@servera ~]#
xfs_repair -n /dev/vdb1Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan (but don't clear) agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 Invalid inode number 0x499602d2 xfs_dir_ino_validate: XFS_ERROR_REPORT Metadata corruption detected at 0x564c8914a8e8, xfs_dir3_block block 0xa7e0/0x1000 entry "subscription-manager" at block 0 offset 456 in directory inode 144973 references invalid inode 1234567890 would clear inode number in entry at offset 456... - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 entry "subscription-manager" at block 0 offset 456 in directory inode 144973 references invalid inode 1234567890 would clear inode number in entry at offset 456... No modify flag set, skipping phase 5 Phase 6 - check inode connectivity... - traversing filesystem ... Invalid inode number 0x499602d2 xfs_dir_ino_validate: XFS_ERROR_REPORT Metadata corruption detected at 0x564c8914a8e8, xfs_dir3_block block 0xa7e0/0x1000 entry "subscription-manager" in directory inode 144973 points to non-existent inode 1234567890, would junk entry bad hash table for directory inode 144973 (no data entry): would rebuild - traversal finished ... - moving disconnected inodes to lost+found ... disconnected inode 145282, would move to lost+found Phase 7 - verify link counts... No modify flag set, skipping filesystem flush and exiting.Note the reported errors with information that might be helpful for file system recovery.
Use the
xfs_repairutility to repair the file system inconsistencies.[root@servera ~]#
xfs_repair /dev/vdb1Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 Invalid inode number 0x499602d2 xfs_dir_ino_validate: XFS_ERROR_REPORT Metadata corruption detected at 0x56258b0938e8, xfs_dir3_block block 0xa7e0/0x1000 entry "subscription-manager" at block 0 offset 456 in directory inode 144973 references invalid inode 1234567890 clearing inode number in entry at offset 456... entry at block 0 offset 456 in directory inode 144973 has illegal name "/ubscription-manager": - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... bad hash table for directory inode 144973 (no data entry): rebuilding rebuilding directory inode 144973 - traversal finished ... - moving disconnected inodes to lost+found ... disconnected inode 145282, moving to lost+found Phase 7 - verify and correct link counts... doneMount the repaired file system at
/mnt/etc_restoreto analyze the file system repair results.[root@servera ~]#
mount /dev/vdb1 /mnt/etc_restoreDetermine whether the file system's
lost+founddirectory has orphaned files.[root@servera ~]#
ls -la /mnt/etc_restore/lost+foundtotal 4 drwxr-xr-x. 2 root root 19 Jan 19 23:07 . drwxr-xr-x. 4 root root 33 Jan 19 22:59 .. -rw-r--r--. 1 root root 62 Oct 13 11:00 145282Determine the correct name and location of the orphaned file in
lost+foundwith the output that was generated during the file system repair.Based on the generated output,
subscription-managershould be the name of the orphaned file....output omitted... Invalid inode number 0x499602d2 xfs_dir_ino_validate: XFS_ERROR_REPORT Metadata corruption detected at block 0x258e0/0x1000 entry "
subscription-manager" at block 0 offset 456 in directory inode 144973 references invalid inode 1234567890 clearing inode number in entry at offset 456... entry at block 0 offset 456 in directory inode 144973 has illegal name "/ubscription-manager": - process newly discovered i nodes... ...output omitted...Based on the generated output, determine the directory name for the
subscription-managerfile. Use thefindcommand to locate the directory with inode number144973....output omitted... Invalid inode number 0x499602d2 xfs_dir_ino_validate: XFS_ERROR_REPORT Metadata corruption detected at block 0x258e0/0x1000 entry "subscription-manager" at block 0 offset 456 in directory inode
144973references invalid inode 1234567890 clearing inode number in entry at offset 456... entry at block 0 offset 456 in directory inode 144973 has illegal name "/ubscription-manager": - process newly discovered i nodes... ...output omitted...[root@servera ~]#
find /mnt/etc_restore -inum 144973/mnt/etc_restore/etc/security/console.apps
Use the file system's content backup at
/root/etc.tgzto verify the orphaned file's name and location.Extract the backup file to view its contents for comparison.
[root@servera ~]#
tar -C /tmp -xzf /root/etc.tgzVerify the contents of the orphaned file against the backup copy of the identified file.
[root@servera ~]#
cd /tmp/etc/security/console.apps[root@servera console.apps]#diff -s subscription-manager /mnt/etc_restore/lost+found/145282Files subscription-manager and /mnt/etc_restore/lost+found/145282 are identical
When verified, move the orphaned file to its proper location on the
/dev/vdb1partition. Ensure that you move the file to the file system that is mounted on the/mnt/etc_restoredirectory.[root@servera console.apps]#
cd /mnt/etc_restore/etc/security/console.apps[root@servera console.apps]#mv /mnt/etc_restore/lost+found/145282 subscription-managermv: overwrite 'subscription-manager'?yVerify that you are able to view the contents of the
/mnt/etc_restore/etc/security/console.appsdirectory.[root@servera console.apps]#
ls /mnt/etc_restore/etc/security/console.appsconfig-util ovirt-hibernate ovirt-locksession ovirt-shutdown rhn_register setup subscription-managerReturn to
workstationas thestudentuser.[root@servera console.apps]#
exit[student@servera ~]$exit[student@workstation ~]$
