Featured image

Table of Contents Link to heading

Text Files on Linux Link to heading

Configuration files, logs, scripts, and most system data on Linux are plain text — no proprietary format, no binary encoding. This is a foundational Unix design principle: store data as text so that any tool can read, process, and transform it.

The consequence for sysadmins is that a small set of text processing tools — cat, grep, sort, cut, wc, awk, sed — can handle the vast majority of log analysis, configuration inspection, and data extraction tasks without specialised software.

Viewing File Contents Link to heading

cat: Concatenate and Display Link to heading

cat reads one or more files and writes their content to standard output. Its name reflects its original purpose — concatenating multiple files — but it is used most often simply to display file content.

cat /etc/hostname
cat /etc/hosts
cat file1.txt file2.txt > combined.txt    # concatenate two files into one
cat -n file.txt                           # display with line numbers
cat -A file.txt                           # show non-printing characters (useful for debugging whitespace)

cat is appropriate for small files. For files with thousands of lines, use a pager.

less and more: Paged Viewing Link to heading

For large files, use a pager that displays content one screen at a time.

less is the standard choice — it does not load the entire file into memory, starts immediately regardless of file size, and supports bidirectional navigation:

less /var/log/syslog
less +G /var/log/syslog     # open at the end of the file

more is simpler and more universally available, but only scrolls forward. less is preferred whenever available.

Navigation in less: arrow keys scroll, Space pages forward, b pages back, /pattern searches, q quits.

head and tail: Viewing File Extremes Link to heading

head displays the first N lines of a file (default: 10):

head /etc/passwd                  # first 10 lines
head -n 20 /var/log/syslog        # first 20 lines
head -n -5 file.txt               # all but the last 5 lines

tail displays the last N lines (default: 10):

tail /var/log/auth.log            # last 10 lines
tail -n 50 /var/log/nginx/error.log    # last 50 lines
tail -n +100 file.txt             # all lines from line 100 onward

tail -f: Following Live Log Files Link to heading

tail -f (follow) reads new lines as they are appended to a file — the primary tool for monitoring live logs during troubleshooting or deployment:

tail -f /var/log/syslog
tail -f /var/log/nginx/access.log
tail -f --retry /var/log/app.log          # keep retrying if file is inaccessible
tail -n 100 -f /var/log/syslog            # start with last 100 lines, then follow
tail -f --sleep-interval=0.5 /var/log/app.log   # check every 0.5 seconds

Press Ctrl+C to stop following.

I/O Redirection Link to heading

Every process has three default data streams: STDIN (0), STDOUT (1), and STDERR (2). Redirection operators reroute these streams to files or other commands.

Standard Output (STDOUT) Link to heading 

ls /usr/bin > filelist.txt        # overwrite file with output
ls /usr/bin >> filelist.txt       # append output to file
echo "started" > /tmp/status.txt  # write a string to a file

> overwrites; >> appends. Using > on an existing file discards its previous contents immediately.

Standard Error (STDERR) Link to heading 

ls /nonexistent 2> errors.txt         # redirect errors to file
ls /nonexistent 2>> errors.txt        # append errors to file
ls /nonexistent 2> /dev/null          # discard errors silently

/dev/null is a special device that discards all data written to it — useful for suppressing unwanted output.

Redirecting Multiple Streams Link to heading 

# Redirect both STDOUT and STDERR to the same file
ls /usr/bin /nonexistent &> all_output.txt

# Redirect STDOUT and STDERR to separate files
ls /usr/bin /nonexistent > output.txt 2> errors.txt

# Redirect STDERR to STDOUT (then pipe or redirect both together)
command 2>&1 | grep "error"

Standard Input (STDIN) Link to heading 

# Use a file as input to a command that reads from STDIN
tr 'a-z' 'A-Z' < input.txt > uppercase.txt

# tr reads from input.txt instead of the keyboard
# uppercase.txt receives the capitalised output

2>&1 means “redirect file descriptor 2 (STDERR) to wherever file descriptor 1 (STDOUT) is currently going.”

Sorting: sort Link to heading

sort rearranges the lines of a file or STDIN:

sort file.txt                         # alphabetical ascending
sort -r file.txt                      # reverse (descending)
sort -i file.txt                      # case-insensitive
sort -n file.txt                      # numeric sort (not lexicographic)
sort -h file.txt                      # human-readable numeric sort (1K, 2M, 3G)
sort -u file.txt                      # unique — remove duplicate lines
sort -k3n /etc/passwd                 # sort by field 3 numerically
sort -t: -k3n /etc/passwd             # sort /etc/passwd by UID (field 3, colon-delimited)
sort -o sorted.txt file.txt           # write sorted output to file
Tip
sort -t: -k3n /etc/passwd is a useful one-liner for inspecting UID assignments on a system. The -t: sets the field delimiter to colon and -k3n sorts by the third field (UID) numerically.

Counting: wc Link to heading

wc (word count) counts lines, words, and bytes in a file or STDIN:

wc file.txt               # lines, words, bytes
wc -l file.txt            # line count only
wc -w file.txt            # word count only
wc -c file.txt            # byte count only
wc -m file.txt            # character count (multi-byte aware)
wc -L file.txt            # length of the longest line

# Count files in a directory
ls /etc | wc -l

# Count log entries matching a pattern
grep "ERROR" /var/log/app.log | wc -l

wc

Filtering: cut and grep Link to heading

cut: Extract Columns Link to heading

cut extracts specific columns or character ranges from each line of text. It is particularly useful for processing structured, delimiter-separated files like /etc/passwd.

# Extract characters 1-10 from each line
cut -c1-10 file.txt

# Extract field 1 from a colon-delimited file
cut -d: -f1 /etc/passwd           # usernames only

# Extract multiple fields
cut -d: -f1,3 /etc/passwd         # usernames and UIDs

# Extract from a range of fields
cut -d: -f1-4 /etc/passwd         # fields 1 through 4

# Extract from field 3 onward
cut -d: -f3- /etc/passwd

grep: Pattern Matching Link to heading

grep filters lines that match a pattern, printing only matching lines to STDOUT. It is one of the most frequently used text processing tools in systems administration.

# Basic pattern search
grep "error" /var/log/syslog
grep "sshd" /var/log/auth.log

# Case-insensitive
grep -i "Error" /var/log/syslog

# Fixed string (disables regex, faster for literal text)
grep -F "192.168.1.1" /var/log/nginx/access.log

# Recursive search in directory
grep -r "PasswordAuthentication" /etc/ssh/

# Show line numbers
grep -n "FAILED" /var/log/auth.log

# Show N lines of context around matches
grep -C3 "segfault" /var/log/kern.log     # 3 lines before and after
grep -A5 "ERROR" /var/log/app.log         # 5 lines after each match
grep -B2 "ERROR" /var/log/app.log         # 2 lines before each match

# Invert match (lines that do NOT match)
grep -v "^#" /etc/ssh/sshd_config        # exclude comment lines

# Extended regex (see below)
grep -E "^(root|admin)" /etc/passwd

# Print only the matching portion (not the whole line)
grep -o "[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+" /var/log/nginx/access.log

# Count matching lines
grep -c "GET /api" /var/log/nginx/access.log

# Print filename for each match (when searching multiple files)
grep -l "error" /var/log/*.log

# Binary file handling
grep --binary-files=without-match "pattern" /path/to/dir

Regular Expressions Link to heading

Regular expressions (regex) define patterns for matching text. Linux commands support two standards: Basic (BRE) and Extended (ERE).

Basic Regular Expressions (BRE) Link to heading

Available in grep, sed, less, and others by default:

Character Matches
. Any single character except newline
[ ] Any single character from the set; [^...] negates
* Zero or more of the preceding character
^ Start of line (when at the beginning of the pattern)
$ End of line (when at the end of the pattern)
\ Escape the next character (treat literally) 
grep "r..f" /usr/share/dict/words     # r + any 2 chars + f (e.g., roof, ruff)
grep "[0-9]" /etc/passwd              # lines containing a digit
grep "[a-d]" file.txt                 # lines containing a, b, c, or d
grep "re*d" file.txt                  # red, rd, reed, rood (zero or more e)
grep "^root" /etc/passwd              # lines starting with "root"
grep "bash$" /etc/passwd              # lines ending with "bash"
grep "re\*" file.md                   # literal asterisk (escaped)

Extended Regular Expressions (ERE) Link to heading

ERE adds quantifiers and alternation. Use grep -E or egrep:

Character Matches
? Zero or one of the preceding item
+ One or more of the preceding item
| Alternation: either the left or right expression
() Grouping: limit scope of alternation or apply quantifiers
{n,m} Between n and m repetitions of the preceding item 
grep -E "colou?r" file.txt            # color or colour
grep -E "colou+r" file.txt            # colour, colouur, etc. (at least one u)
grep -E "cat|dog" file.txt            # lines containing cat or dog
grep -E "learn(t|ed)" file.txt        # learnt or learned
grep -E "[0-9]{3}-[0-9]{4}" file.txt  # phone number pattern (e.g., 555-1234)
grep -E "^(root|admin|sudo)" /etc/passwd   # lines starting with these users

Pipes: Composing Commands Link to heading

The pipe operator | passes the STDOUT of one command as the STDIN of the next. This is the mechanism for building multi-step text processing pipelines from simple single-purpose tools.

# Count unique IP addresses in an nginx access log
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

# Find users with shells (not /sbin/nologin or /bin/false)
grep -v "nologin\|false" /etc/passwd | cut -d: -f1

# Find the 10 largest files in /var/log
find /var/log -type f | xargs ls -lS 2>/dev/null | head -10

# Search man page for a keyword
man bash | grep -i "history"

# List all listening TCP ports with process names
ss -tlnp | grep LISTEN

Each command in a pipeline processes only the output of the previous command — no intermediate files, no context switching to disk. Pipelines are one of the most expressive features of the Unix/Linux CLI and the foundation of shell scripting efficiency.