More bash snippets

Here are some more bash useful bash snippets:

Arithmetic, like incrementing a counter:

echo $j

Looping over things:

# Run a command several times with specific values
for i in 1 2 3; do echo $i; done

# Loop over a specifc range of values using 'seq'
for i in $(seq 1 3); do echo $i; done

# Run a command on all txt files in current directory
for f in `ls *.txt`; do head $j; done

Redirecting output into files:

# Redirect standard output
date 1> out.log

# Redirect error output
blah 2> error.log

# Redirect both into different files
for cmd in date blah; do $cmd; done 1> out.log 2> error.log

# Redirect any kind of output into same file
for cmd in date blah; do $cmd; done &> all.log

Here’s a snippet I use for running a long running command, for example over a range of objects (accessible via some kind of ID)

# Open a screen session
screen -S BigJob

# Launch the job loop
for id in 1 2 3 4 5; do echo $id; date; do_something_with $id; done &> bigJob.log

# Detach again by holding CTRL-A and pressing D

The jobs are running safely in a screen environment, i.e. you don’t have to worry that you accidentaly kill the process by getting disconnected from the SSH session. And you can check the progress live via

tail -f bigJob.log

As the snippets prints the $id you can see which object/id is currently processed; and using date in the snippet allows you to work out how long each job took.