Unix Command Line Crash Course
Learn about Unix shell, commands, and environment variables.
Hopefully, you have seen a command line interface, CLI, before. You can find them on nearly every system: Windows, Linux, macOS, and even my old Amiga had a CLI. Each of them work slightly different, but there is one standard which has come to dominate, which is based on the Unix operating system. Both Linux and macOS derive from the Unix operating system. When interacting with a CLI on Linux or Mac, you will use mostly the same Unix commands. Unix command line interfaces have also become popular on Windows the last few years through Windows Subsystem for Linux (WSL).
You interact with the Unix command line through a program, usually called terminal or console. Here is an example of using iTerm2 on my Mac. In this particular example, I am issuing commands to look the contents of the Godot directory, where I have stored some games I have developed.
If you use a graphical file manager such as Finder, this Godot directory would have looked as follows:
Why should you learn how to use a Unix command line interface? Isn’t it much easier to just click around in a file manager? Indeed, graphical user interfaces tend to be easier to explore and learn. But text-based interfaces have numerous advantages which are hard to replicate in graphical user interfaces:
- They often prove a lot more power and flexibility.
- Complex graphical interfaces are harder to learn than complex text-based interfaces.
- Text-based interfaces naturally lend themselves to automation, which can be a big time saver.
The second point requires some justification. It is true that it is easier to understand a graphical user interface, but only up to a point. Large and complex graphical applications can be hard to learn. What to do is not obvious. Finding the right buttons to click or dialogs to open may require a video or series of images with careful step-by-step instructions.
Text-based interfaces in contrast are straightforward to explain in articles such as this one. Complex operations which you easily forget how to do, so you can easily copy to a notebook and write an explanation. These notes can later easily be retrieved using search tools. Searching images of a graphical UI is much harder.
In this article, I will teach you how to do the following things:
- Navigate around the file system just as you would do with a graphical file manager.
- Look at what files are in a directory and filter out files or directories you are not interested in.
- Do operations on files.
- Explain the binary search path. It is used by the terminal to locate the command you write.
- What are environment variables, and what can you use them for?
Work with Files and Directories
When you start the Unix command line, you will get some kind of greeting. It will differ depending on the system and configuration you have. Mine looks like this:
Last login: Thu Jun 23 22:10:30 on ttys001
Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
~
❯
You will have a cursor next to a prompt, indicating that the CLI is waiting for you to type a command and press enter. The prompt at the start of the line is usually one of the symbols $, # or ❯. Mine is a ❯ because I am using the Starship prompt. It adds several cool features, but you don’t have to use it.
Let us look at the most basic commands ls, cd, pwd, man, cat, touch and file.
ls - List directory contents
With the ls command, you can see what files and directories are in a directory, just like you can with a file manager.
❯ ls /bin
[ expr pwd
bash hostname rm
cat kill rmdir
chmod ksh sh
cp launchctl sleep
csh link stty
dash ln sync
date ls tcsh
dd mkdir test
df mv unlink
echo pax wait4path
ed ps zsh
❯
This shows the contents of the bin directory on a Unix system. That is where Unix keeps binaries for the most common commands. We call a file containing a program you can run for binary.
The ls command is smarter than your file manager. You can do many neat tricks. Characters such as * and ? get interpreted in a special way. They form what we call glob patterns. You can use glob patterns to filter what files are show. The * matches any number of letters while ? matches a single letter. So a? would match files named ab, ab, ad, ae and so on, while *.txt would match files named foo.txt, bar.txt, baz,txt and so on. What is up with the foo and bar names? Those a just silly names we tend to use in programming and Unix circles. As soon as you see the words foo, bar, qux or baz you should know that they are placeholders. The specific word used isn’t important. It could be anything.
cd - Change working directory
To try out the ls command, we will use the cd and pwd commands. Just like with a graphical terminal, there is a current working directory. It is the directory your commands will apply changes to. You can change to a working directory with the cd command. If you have forgotten where you are, you can use the pwd command. pwd is sort for print working directory.
Let us jump a bit around and look at some directories on a macOS system (will be similar on Linux).
❯ cd /bin
❯ ls
[ df ln sh
bash echo ls sleep
cat ed mkdir stty
chmod expr mv sync
cp hostname pax tcsh
csh kill ps test
dash ksh pwd unlink
date launchctl rm wait4path
dd link rmdir zsh
❯ pwd
/bin
❯ cd /System
❯ ls
Applications DriverKit Volumes
Developer Library iOSSupport
❯ pwd
/System
touch and mkdir - Make files and directories
To be able to experiment a bit more with ls we will make our own directory and fill it with bogus files. We will go to our home directory. If you just write cd you will jump to the home directory for your user. Here we will make a directory called animals to experiment with ls.
From now on, I will show above the prompt what my current directory is and files will have a * as an ending, while directories have the ending /. You can get trailing slash on your system by writing ls -p instead of just ls. The -p is called a switch. Every Unix command comes with several switches affecting how they run.
Okay, let us make some dummy files with the touch command.
~
❯ mkdir animals
~
❯ cd animals
~/animals
❯ touch bird.png dog.jpg cat.jpg monkey.gif
~/animals
❯ ls
bird.png dog.jpg
cat.jpg monkey.gif
~/animals
❯ touch horse.png whale.png fox.gif
The mkdir command makes a directory with a given name, while touch can be used to make one or more empty files. With these files, we will look at how we can use glob pattern with ls to show only a subset of the files you have in the current directory.
~/animals
❯ ls
bird.png fox.gif whale.png
cat.jpg horse.png
dog.jpg monkey.gif
If you want to only see the .png files in the animals directory you can use the *.png glob pattern which means we don’t care what the name of the file is as long as it ends with .png
❯ ls *.png
bird.png horse.png whale.png
~/animals
❯ ls *.gif
fox.gif monkey.gif
You can find all three letter filenames with an o in the middle using the question mark ? which matches a single letter.
~/animals
❯ ls ?o?.*
dog.jpg fox.gif
man - Manual - Get help
Once you know the basics of Unix commands you want to be able to look at specific detailed information which does not make sense to cover in a blog post. We can look at the manual for each command using the man command. man ls will give us the manual page for the ls (list) command. It will drop you into a pager program. A lot of beginners immediately ask: How do I get out of this program?! Whether you are inside the vim program, less, or man you press q to quit. In the vim editor you may have to press ESC key and then colon : before you can hit q to exit.
Here is an overview of the most useful commands for navigating inside a man documentation page:
↑and↓arrow keys for going up and down one line.Ctrl-UandCtrl-Dfor going up and down a whole page at a time./foobarsearches for the textfoobarin the manual page. You just write a slash/and then type the text you are interested in finding. (foobaris a nonsense word).qquit the manual page viewing.
See if you can find the manual entry for the -l switch. It prints info about directory contents on long form. You can search for it by writing /-l. The entry should look like this:
-l (The lowercase letter “ell”.) List files in the long
format, as described in the The Long Format
subsection below.
We can try the -l switch to see how we get more info:
~/animals
❯ ls -l
total 0
-rw-r--r-- 1 erikengheim staff 0 Jun 24 14:11 bird.png
-rw-r--r-- 1 erikengheim staff 0 Jun 24 14:11 cat.jpg
-rw-r--r-- 1 erikengheim staff 0 Jun 24 14:11 dog.jpg
-rw-r--r-- 1 erikengheim staff 0 Jun 24 14:16 fox.gif
-rw-r--r-- 1 erikengheim staff 0 Jun 24 14:16 horse.png
-rw-r--r-- 1 erikengheim staff 0 Jun 24 14:11 monkey.gif
-rw-r--r-- 1 erikengheim staff 0 Jun 24 14:16 whale.png
This shows a whole bunch of information about each file. Each column is for the following:
- Whether you can read
r, writewor executexthe file. There is a triplet for the user, group file belongs to and everybody else. - Number of links to file. What appears as different files in different locations on your hard drive can all point to the same file.
- Owner of the file,
erikengheimin this example. - Group the file belongs to.
staffin this example. - Number of bytes in file. It is zero because we made empty files.
The last columns are for when the file was last modified and finally the name of the file.
cat - Concatenate - Join files and show contents
Our files are empty, so to be able to test the cat command used to show contents we have to actually edit the files and add some content. You can use any text editor storing text as UTF-8 encoding, or you can use the echo command to add text to files.
~/animals
❯ echo blue bird > bird.png
~/animals
❯ echo black cat > cat.jpg
~/animals
❯ echo red fox > fox.gif
You can use cat to look at the contents of individual files or concatenate the contents of multiple files and show it.
~/animals
❯ cat bird.png cat.jpg
blue bird
black cat
~/animals
❯ cat bird.png cat.jpg fox.gif
blue bird
black cat
red fox
echo - Display text
In many cases we need to display some text to the user. We can use echo for this purpose:
❯ echo Hello world
Hello world
This example many look at bit pointless but it makes sense when writing longer scripts doing multiple things where you want to give feedback to the user. You can also use it to show the contents of environment variables. We can use named variables to store text which can later be shown:
❯ NAME=Joe
❯ echo hello $NAME
hello Joe
Why do we need the dollar sign $? Because if you only wrote NAME the system would not be able to realize you want the contents of a variable named NAME. It would think you wanted to write the text NAME:
❯ echo hello NAME
hello NAME
So while you don’t need the dollars sign $ when setting an environment variable, you will need it when using it.
How Commands are Located with Binary Search Path
Most commands such as ls, cat, cp and rm are actual small programs which are loaded and executed by the operating system. When you type a command the operating system will look for that command in different locations. You can find out where the command is found by the operating system by using the which command:
❯ which ls
/bin/ls
❯ which cat
/bin/cat
❯ which vim
/usr/bin/vim
How does the operating system know to look in the /bin directory for ls and cat while the vim editor is in the /usr/bin directory? That is done using the binary search path. It is stored in the environment variable called PATH. We can look at the contents of binary search path by using the echo command.
❯ echo $PATH
/usr/local/bin /usr/bin /bin /usr/sbin /sbin
But maybe we have programs in other locations. How do we make the operating system find them? Say we have binaries in the /opt/bin directory which we want to use. Each search path is separate with a colon : and we can simply insert the existing path when redefining what the PATH variable should be:
❯ PATH=/opt/bin:$PATH
There is however one problem with what we just wrote. As soon as you close the terminal window the path will be forgotten. That is why you need to add this line to the .profile or .zshrc file in your home directory. Which file you add to depends on whether you use the bash or zsh shell. You also have to add the export keyword otherwise the environment variable will not be known outside the shell configuration file .profile or .zshrc. So you write:
export PATH=/opt/bin:$PATHBecause a path may contain spaces, we tend to use quotation marks " around text strings to show that it is one contiguous block of text. Hence we usually write:
export PATH="/opt/bin:$PATH"In this example we put /opt/bin first, but what if we wrote it like this instead?
export PATH="$PATH:/opt/bin"That is perfectly valid and means that when looking for a Unix command the system will first look in one of the paths already defined in PATH before looking in /opt/bin. That may matter in case you have commands with the same name in multiple locations. For instance your operating system may come installed with the python language but then you install your own newer version to /opt/bin. Since you want your newer version to be used by default you put the /opt/bindirectory first, so it gets priority.
Environment Variables
Environment variables can have many uses. We just saw the PATH environment variable used to specify the binary search path. Various Unix tools require a text editor. For instance the Git version control system will sometimes open an editor to let to edit commit messages. How is you operating system supposed to know what editor to open? You can specify that with the EDITOR environment variable. Although sometimes you want different editors for different programs. For instance I don’t want to use my editor in the same way for Git as with the Julia programming language. Fortunately Julia allows me to specify an editor in the JULIA_EDITOR environment variable which takes precedence over what the EDITOR environment variable says. Let me show you:
export EDITOR="mate -w"
export JULIA_EDITOR=mateFor Git I used the -w switch because it will cause the calling program (Git) to halt until I close the editor. That is not desired behavior for me when using Julia.
The text editor I use, TextMate allows you to set environment variables in a preference panel. You can see variables such as BOARD, SERIALDEV and TM_JULIA. This is useful because any program started by TextMate will have access to these environment variables.
Exported environment variables are inherited by child processes. What do you mean? A running program is called a process. Processes have parent-child relationships. The terminal window is a process and when run a command such as ls, man or touch you are creating a child process of this window, or more specifically the shell process running inside the window.
Any running program could launch a child process, and define environment variables which should be inherited by the child process. TextMate allows you to write shell scripts, Python, Ruby or Julia scripts which could read these environment variables.
Thus the use of environment variables is not limited to text-mode applications. You can use environment variables to pass data from and to any program.