Bookmark this page

Match File Names with Shell Expansions

Objectives

Efficiently run commands that affect many files by using pattern matching features of the Bash shell.

Command-line Expansions

When you type a command at the Bash shell prompt, the shell processes that command line through multiple expansions before running it. You can use these shell expansions to perform complex tasks that would otherwise be difficult or impossible.

Following are the main expansions that Bash shell performs:

  • Brace expansion, which can generate multiple strings of characters

  • Tilde expansion, which expand to a path to a user home directory

  • Variable expansion, which replaces text with the value that is stored in a shell variable

  • Command substitution, which replaces text with the output of a command

  • Pathname expansion, which helps to select one or more files by pattern matching

Pathname expansion, historically called globbing, is one of the most useful features of Bash. With this feature, it is easier to manage many files. By using metacharacters that "expand" to match the file and path names that you are looking for, commands can act on a focused set of files at once.

Pathname Expansion and Pattern Matching

Pathname expansion expands a pattern of special characters that represent wild cards or classes of characters into a list of file names that match the pattern. Before the shell runs your command, it replaces the pattern with the list of file names that matched. If the pattern does not match anything, then the shell tries to use the pattern as a literal argument for the command that it runs. The following table lists common metacharacters and pattern classes that are used for pattern matching.

Table 3.2. Table of Metacharacters and Matches

PatternMatches
*Any string of zero or more characters
?Any single character
[abc…​]Any one character in the enclosed class (between the square brackets)
[!abc…​]Any one character not in the enclosed class
[^abc…​]Any one character not in the enclosed class
[[:alpha:]]Any alphabetic character
[[:lower:]]Any lowercase character
[[:upper:]]Any uppercase character
[[:alnum:]]Any alphabetic character or digit
[[:punct:]]Any printable character that is not a space or alphanumeric
[[:digit:]]Any single digit from 0 to 9
[[:space:]]Any single white space character, which might include tabs, newlines, carriage returns, form feeds, or spaces

For the next example, imagine that you ran the following commands to create some sample files:

[user@host ~]$ mkdir glob; cd glob
[user@host glob]$ touch alfa bravo charlie delta echo able baker cast dog easy
[user@host glob]$ ls
able  alfa  baker  bravo  cast  charlie  delta  dog  easy  echo
[user@host glob]$

In the next example, the first two commands use simple pattern matches with the asterisk (*) to match all the file names that start with "a" and all the file names that contain an "a", respectively. The third command uses the asterisk and square brackets to match all the file names that start with "a" or "c".

[user@host glob]$ ls a*
able  alfa
[user@host glob]$ ls *a*
able  alfa  baker  bravo  cast  charlie  delta  easy
[user@host glob]$ ls [ac]*
able  alfa  cast  charlie

The next example also uses question mark (?) characters to match some of those file names. The two commands match only file names with four and five characters in length, respectively.

[user@host glob]$ ls ????
able  cast  easy  echo
[user@host glob]$ ls ?????
baker  bravo  delta

Brace Expansion

Brace expansion is used to generate discretionary strings of characters. Braces contain a comma-separated list of strings, or a sequence expression. The result includes the text that precedes or follows the brace definition. Brace expansions might be nested, one inside another. You can also use double-dot syntax (..), which expands to a sequence. For example, the {m..p} double-dot syntax inside braces expands to m n o p.

[user@host glob]$ echo {Sunday,Monday,Tuesday,Wednesday}.log
Sunday.log Monday.log Tuesday.log Wednesday.log
[user@host glob]$ echo file{1..3}.txt
file1.txt file2.txt file3.txt
[user@host glob]$ echo file{a..c}.txt
filea.txt fileb.txt filec.txt
[user@host glob]$ echo file{a,b}{1,2}.txt
filea1.txt filea2.txt fileb1.txt fileb2.txt
[user@host glob]$ echo file{a{1,2},b,c}.txt
filea1.txt filea2.txt fileb.txt filec.txt

A practical use of brace expansion is to create multiple files or directories.

[user@host glob]$ mkdir ../RHEL{7,8,9}
[user@host glob]$ ls ../RHEL*
RHEL7 RHEL8 RHEL9

Tilde Expansion

The tilde character (~), matches the current user's home directory. If it starts with a string of characters other than a slash (/), then the shell interprets the string up to that slash as a username, if one matches, and replaces the string with the absolute path to that user's home directory. If no username matches, then the shell uses an actual tilde followed by the string of characters.

In the following example, the echo command is used to display the value of the tilde character. You can also use the echo command to display the values of brace and variable expansion characters, and others.

[user@host glob]$ echo ~root
/root
[user@host glob]$ echo ~user
/home/user
[user@host glob]$ echo ~/glob
/home/user/glob
[user@host glob]$ echo ~nonexistinguser
~nonexistinguser

Variable Expansion

A variable acts like a named container that stores a value in memory. Variables simplify accessing and modifying the stored data either from the command line or within a shell script.

You can assign data as a value to a variable with the following syntax:

[user@host ~]$ VARIABLENAME=value

You can use variable expansion to convert the variable name to its value on the command line. If a string starts with a dollar sign ($), then the shell tries to use the rest of that string as a variable name and to replace it with the variable value.

[user@host ~]$ USERNAME=operator
[user@host ~]$ echo $USERNAME
operator

To prevent mistakes due to other shell expansions, you can put the name of the variable in curly braces, for example ${VARIABLENAME}.

[user@host ~]$ USERNAME=operator
[user@host ~]$ echo ${USERNAME}
operator

Variable names can contain only letters (uppercase and lowercase), numbers, and underscores. Variable names are case-sensitive and cannot start with a number.

Command Substitution

Command substitution enables the output of a command to replace the command itself on the command line. Command substitution occurs when you enclose a command in parentheses and precede it by a dollar sign ($). The $(command) form can nest multiple command expansions inside each other.

[user@host glob]$ echo Today is $(date +%A).
Today is Wednesday.
[user@host glob]$ echo The time is $(date +%M) minutes past $(date +%l%p).
The time is 26 minutes past 11AM.

Note

An earlier form of command substitution uses backticks: `command`. Although the Bash shell still accepts this format, try to avoid it because it is easy to visually confuse backticks with single quotation marks, and backticks cannot be nested.

Protecting Arguments from Expansion

Many characters have a special meaning in the Bash shell. To prevent shell expansions on parts of your command line, you can quote and escape characters and strings.

The backslash (\) is an escape character in the Bash shell. It protects the following character from expansion.

[user@host glob]$ echo The value of $HOME is your home directory.
The value of /home/user is your home directory.
[user@host glob]$ echo The value of \$HOME is your home directory.
The value of $HOME is your home directory.

In the preceding example, with the dollar sign protected from expansion, Bash treats it as a regular character, without variable expansion on $HOME.

To protect longer character strings, you can use single quotation marks (') or double quotation marks (") to enclose strings. They have slightly different effects. Single quotation marks stop all shell expansion. Double quotation marks stop most shell expansion.

Double quotation marks suppress special characters other than the dollar sign ($), backslash (\), backtick (`), and exclamation point (!) from operating inside the quoted text. Double quotation marks block pathname expansion, but still allow command substitution and variable expansion to occur.

[user@host glob]$ myhost=$(hostname -s); echo $myhost
host
[user@host glob]$ echo "***** hostname is ${myhost} *****"
***** hostname is host *****

Use single quotation marks to interpret all text between the quotes literally.

[user@host glob]$ echo "Will variable $myhost evaluate to $(hostname -s)?"
Will variable host evaluate to host?
[user@host glob]$ echo 'Will variable $myhost evaluate to $(hostname -s)?'
Will variable $myhost evaluate to $(hostname -s)?

Important

It is easy to confuse the single quotation mark ( ' ) and the command substitution backtick ( ` ), on both the screen and the keyboard. The use of one when you mean to use the other leads to unexpected shell behavior.

 

References

bash(1), cd(1), glob(7), isalpha(3), ls(1), path_resolution(7), and pwd(1) man pages

Revision: rh124-9.0-398f302