The UNIX system

The UNIX system is a major breakthrough that inspired many other operating systems. It’s particularly visible in Linux and macOS.

You may even read the term “Unix-like systems”.

The UNIX core philosophy has highlighted critical principles such as:

  • multi-tasking
  • multi-user
  • modularity
  • single responsibility
  • composition
  • portability
  • extensibility

Back in the 60’s/70’s, UNIX introduced the concept of “Making each program do one thing well”, and this is still massively applied in programming.

N.B.: There are many other principles you might want to explore

Permalink to heading Core components Core components

Permalink to heading The Kernel The Kernel

The heart of UNIX is the kernel. It interacts with critical elements such as the hardware, the memory, or the filesystem.

It works with the highest level of permissions.

Permalink to heading The shell The shell

The shell intreprets the various commands and utilities to call the programs and execute the tasks.

Permalink to heading The filesystem The filesystem

The filesystem is composed of directories and files that contain data. It’s a hierarchical tree.

Permalink to heading A quick overview of the Unix Filesystem A quick overview of the Unix Filesystem

Any dev is probably familiar with the UNIX file system. At least, you may have already seen something similar to the following:

/
├── bin/
├── dev/
├── etc/
│    └── passwd
├── home/
├── lib/
├── mnt/
├── root/
├── proc/
├── mnt/
├── sbin/
├── tmp/
└── usr/
    └── bin/

Those directories contain other directories and files that are essentials for the server or the operating system.

While Linux distributions and other operating systems may have their specificities (e.g., special files and directories), the global structure will likely look like that.

It’s called the directory tree and it’s a typical multi-level UNIX hierarchy.

Permalink to heading /home/ sweet home /home/ sweet home

Pretty straightforward, as it’s the home folder for each user with, for example, special configurations.

Permalink to heading / vs. /root/ / vs. /root/

The top / is the system root directory. All folders and files are its descendants, including /home/. It’s like C:\ in Windows.

/root/ is the home directory for the root user.

Permalink to heading /usr/bin vs. /bin/ vs. /sbin/ /usr/bin vs. /bin/ vs. /sbin/

The /usr/bin, for example, usually contains executable files that are not needed for booting or repairing the system but still used by the user. /usr/ is a read-only directory that contains applications like your favorite browser.

The /bin/ folder contains essential binaries such as cat or ls, which are also basic command lines you an use in the terminal. /sbin/ is for “system binaries” you cannot use without the necessary permissions, like maintenance utilities.

Permalink to heading The privileged /etc/ The privileged /etc/

/etc/ stores essentia configuration files and databases for the system. For example, the /etc/passwd file is one of the most critical ones. It contains very sensitive information about the users on the system (e.g., usernames, passwords, group ids, user ids, login shell, home directory, and more).

Only the root user is supposed to edit such file.

Permalink to heading /dev/ vs. /proc/ /dev/ vs. /proc/

/dev/ contains non-standard files for devices and virtual devices (also known as pseudo-devices), and /proc/ contains kernel processes.

Disks will be represented as file, for example, /dev/sda and its partitions could be /dev/sda1, /dev/sda2, /dev/sda3, and so on.

Permalink to heading What is /dev/null? What is /dev/null?

Administrators and dev may partially or completely redirect the output to /dev/null to clear it, so they can skip unnecessary lines like “Permission denied”, for example, in global scans with grep.

Permalink to heading Mounting /mnt/ Mounting /mnt/

/mnt/ is for mount points. You typically use it to mount temporary file system like external storages or USB drives.

Roughly speaking, mounting means attaching another filesystem to the current filesystem.

Permalink to heading /var/ /var/

/var/ is for writeable contents such as logs.

Permalink to heading /tmp/ /tmp/

/tmp/ is for temporary files. Such files can be deleted at any moment, for example, when the system restarts.

To preserve files between reboots, you may use the /var/tmp/ directory instead.

Permalink to heading What is vi? What is vi?

Vi is the standard editor for UNIX systems. You can type vi NAME_OF_YOUR_FILE in the terminal to create/edit a file within a UNIX system and ESC followed by :wq to save your work.

Permalink to heading What are processes? What are processes?

Processes are created when you execute a program. Even a simple ls -lah creates a process. Processes are also known as “instances” of programs.

UNIX systems involve normal processes and background processes. To start a background process, you can use &:

./myexecutable &

To list processes, you can use ps:

ps -e | less

You’ll get info such as process ids and many more. To kill a specific process, you can use kill -9 PROCESS_ID.

Permalink to heading The concept of pipes, stdin, and stdout The concept of pipes, stdin, and stdout

Pipes allow passing or holding data between processes. In Unix-like systems, it will be represented as a temporary file.

Pipes can also be considered as redirections. The result (output or stdout) of a command is used as input (stdin) for another command for further processing.

The syntax used for piping several commands is |. This is a one way flow and a left-to-right direction:

cat data.txt | grep "bootleg"

It’s a powerful built-in mechanism to chain commands and run complex procedures.

Permalink to heading What are UNIX sockets? What are UNIX sockets?

UNIX domain sockets (or IPC sockets) are connections between processes on the same host operating system (~ the same machine, no TCP overhead involved).

It uses the local filesystem to create an IPC channel (~ independent process not affected by other processes) between processes.

Unlike pipes, it’s a two-way communication.

Permalink to heading Why people say “everything is a file” in UNIX Why people say “everything is a file” in UNIX

The assertion “everything is a file” is convenient to describe advanced abstract concepts. Many internal features such as sockets will appear in the filesystem (e.g., mysql.sock) even if they don’t contain any data.

It is how the system represents streams of bytes exchanged between programs.

However, in practice, not everything is actually file, for example, system processes, but maybe saying that “everything behaves as a file” would be more accurate.

Permalink to heading Misknown commands or options you might want to use Misknown commands or options you might want to use

CommandDescription
df –kh DIR/disk usage in the given directory in human readable format
head -7 myfile.txtdisplay first 7 lines of a file
diff -w file1.txt file2.txtdisplay diff between files but ignore whitespaces
cd -go quickly to the previous directory
free -hmemory usage in human readable format
uname -rdisplay kernel info
!111execute again the command with the number 111 in the history list

Note that some Unix-like systems may have different commands for the same purpose.