May 22, 2019 - More bash snippets


More bash snippets

Here are some more bash useful bash snippets:

Arithmetic, like incrementing a counter:

echo $j

Looping over things:

# Run a command several times with specific values
for i in 1 2 3; do echo $i; done

# Loop over a specifc range of values using 'seq'
for i in $(seq 1 3); do echo $i; done

# Run a command on all txt files in current directory
for f in `ls *.txt`; do head $j; done

Redirecting output into files:

# Redirect standard output
date 1> out.log

# Redirect error output
blah 2> error.log

# Redirect both into different files
for cmd in date blah; do $cmd; done 1> out.log 2> error.log

# Redirect any kind of output into same file
for cmd in date blah; do $cmd; done &> all.log

Here’s a snippet I use for running a long running command, for example over a range of objects (accessible via some kind of ID)

# Open a screen session
screen -S BigJob

# Launch the job loop
for id in 1 2 3 4 5; do echo $id; date; do_something_with $id; done &> bigJob.log

# Detach again by holding CTRL-A and pressing D

The jobs are running safely in a screen environment, i.e. you don’t have to worry that you accidentaly kill the process by getting disconnected from the SSH session. And you can check the progress live via

tail -f bigJob.log

As the snippets prints the $id you can see which object/id is currently processed; and using date in the snippet allows you to work out how long each job took.

May 17, 2019 - Back up a VirtualBox image


Back up a VirtualBox image

What are we trying to achive: A backup of a VirtualBox image with a size as small as possible.

‘Zero’ out the harddisk

As you keep using your virtual machine and writing and deleting things to the virtual harddisk, you’re filling it up with 0’s and 1’s. As you probably know, deleting a file doesn’t delete its binary data, just the file system reference to the data on the disk. Furthermore data chunks with random 0’s and 1’s can’t be compressed very well. So in order to achive a good compression rate, we need to fill up the ‘free’ space with uniform data, e. g. 0’s:

Boot into the virtual machine and run as root:

dd if=/dev/zero of=/tmp/zeros bs=10M
rm /tmp/zeros

This command will write zeros into the file /tmp/zeros until there is not disk space left. Then we delete this file again.

Shutdown the virtual machine.

Compress the virtual harddisk

On the host machine:

Find out where the virtual harddisk (usually [VM_NAME].vdi) file is stored.


# On Linux:
vboxmanage modifymedium disk /path/to/disk.vdi --compact

# on Windows:
VBoxManage.exe modifymedium disk "C:\path\to\disk.vdi" --compact

Export the virtual machine

Finally in the VirtualBox GUI: ‘Export Appliance’

May 16, 2019 - Search for apt packages


Search for apt packages

Quite often you want to compile or run something and all you get is something like “Error: not found”.

In these cases I usually used

apt-cache search XYZ

in order to find out which package might be missing on my Debian system.

If the package name really contains XYZ then I’d find it. But if libXYZ is actually in package ABC I would not.

A much better tool for this is ‘apt-file’. You’ll probably have to install it. The package like the command is simply called ‘apt-file’.

Then a

apt-file search

will most certainly find the package you have to install in order to get the !

Often apt-file returns a lot of not so useful stuff. If you’re looking for a specific command, try searching for ‘/usr/bin/command’, e.g.

apt-file search /usr/bin/nmap

if you want to know which package contains the nmap tool.

Caveat: apt-file uses your local apt cache. Make sure it’s up-to-date with

apt-get update