Jul 25, 2023 - More Bash

More Bash

Here are some more, hopefully useful hacks for bash scripts.

Default options

It is often recommended to set a few options in bash scripts as default:

set -euo pipefail

What that means:

  • -e : Immediately exit the bash script if a command fails.
  • -u : Immediately exit the bash script if an undefined variable is called.
  • -o pipefail : In case of chained (“piped”) command (e. g. grep something /var/log/something.log | sort) fails, return the error code of the failed command for the chain/pipe.

This is sometimes also referred to as unofficial “Bash strict mode”. But it comes with caveats!

Redirect output

# Disable stderr
exec 2>/dev/null

# Disable stdout
exec 1>/dev/null

# Disable both
exec 2>&1 1>/dev/null

# Redirect all output to some kind of log file
exec 1> /tmp/output.log
exec 2>&1

The internal field separator (IFS)

This variable defines how the bash splits strings. Useful when used in combination with read, for example:

IFS='|' read key value <<< "someKey|someValue"; echo "key: $key; value: $value"

Default values

# If nothing is provided as first argument, use someFile.txt as filename
filename=${1:-someFile.txt}

Multiline text snippets

cat << EOF
This is some
input over
multiple lines
EOF
mail -s "Alarm!" admin@example.com << EOF
Someone tried to hack your server!
Kind Regards,
The friendly hacker
EOF

Note: EOF is just some marker, you can call it whatever you want. EOF (end of file) or EOT (end of transmission) is often used.

Nov 3, 2022 - Linux time machine

Linux time machine

A few days ago my nextcloud server broke apart, completely. It had to be re-installed from scratch. I hadn’t realised that my backup wasn’t working, so I didn’t have a mysql dump. Luckily I still could mount the harddisk and rescue the /var/lib/mysql directory, with all the *.frm, *.ibd, etc. database files.

I re-installed the server and tried to simply copy the recovered database files over. Of course that didn’t work, because the database files were created with an older mariadb version. At least the error message said exactly what version they were created with. So I had to get the same version again, then I could spin up the database and do a mysqldump, which I then can re-import into the up-to-date mariadb again.

On a Debian system that’s basically impossible. Unless you’re lucky and a Debian release has exactly the version you need. You can get these older versions from previous releases or a newer version by enabling testing repositories. But even if you could get the version you need, if you’ve ever tried that, you know what a pain this is! Dependency hell!

But there’s Arch to the rescue! So I span up an Arch VM (see Vagrant file at the bottom). Went to the Arch archive: Arch Archive . Found the mariadb package of the specific version and noted the date when it was added. SSH’d into the Arch VM and adjusted the /etc/pacman.d/mirrorlist with the specific date URL e.g. Server=https://archive.archlinux.org/repos/2014/03/30/$repo/os/$arch. Then got the past keyring with pacman -S archlinux-keyring ca-certificates followed by a complete reset of the system to the past with pacman -Syyuu. And I had a linux system with the exact version of mariadb and all the necessary dependencies I wanted. Truely awesome! I don’t know of any other linux distribution which can do this!

Detailed instructions here: How to restore all packages to a specific date

Here’s a Vagrant file if you quickly wanna spin up an Arch VM:

Vagrant.configure("2") do |config|
  config.vm.box = "archlinux/archlinux"
  config.vm.network "private_network", ip: "192.168.56.100", :name => 'vboxnet0', :adapter => 2

  config.vm.provider "virtualbox" do |vb|
    vb.name = "archy"
    vb.memory = "4096"
    vb.cpus = "2"
  end

  config.vm.provision "shell", inline: <<-SHELL
    sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/g' /etc/ssh/sshd_config
    echo 'MaxAuthTries 100' >> /etc/ssh/sshd_config
    systemctl restart sshd.service
  SHELL
end

You can then simply ssh into vagrant@192.168.56.100 (this is using Virtualbox with a ‘vboxnet0’ host network setup); I don’t like the vagrant ssh workaround.

Nov 2, 2022 - Post installation script

Post installation script

In several earlier blog posts I mentioned a few things to do after you set up a new server: Hardening sshd config, enable firewall, etc.

Here’s a compact form of all this as a script, which you can customise and simply run after an installation. It will create a specific user, and only this user is allowed to ssh with public-key authentication (password authentication will be disabled). UFW will be installed and enabled as firewall. Fail2ban installed and enabled as basic brute-force protection. And a classic LAMP stack will be installed. On a debian system you should only have to edit the ‘Options’ section. For other distros you obviously have to adjust the script accordingly.

#!/bin/bash

###### Options

user="SOME_USER"

key="SOME_USERS_SSH_KEY"

aptpackages="vim screen lynx mariadb-server mariadb-client apache2 libapache2-mod-php php" # classic LAMP server

snappackages="certbot" # Have to run 'certbot --apache' manually later

######


# redirect all output to log file
exec >> /root/post_install.log
exec 2>&1


# update base installation
apt-get -y -q update
apt-get -y -q upgrade


# add the main user
adduser --disabled-password --gecos "" $user
mkdir /home/$user/.ssh
chmod 700 /home/$user/.ssh
echo "$key" >> /home/$user/.ssh/authorized_keys
chmod 644 /home/$user/.ssh/authorized_keys
ssh-keygen -t rsa -N '' -f /home/$user/.ssh/id_rsa
chown -R $user:$user /home/$user/.ssh


# harden ssh config
sed -i "s/.*PubkeyAuthentication.*/PubkeyAuthentication yes/g" /etc/ssh/sshd_config
sed -i "s/.*PasswordAuthentication.*/PasswordAuthentication no/g" /etc/ssh/sshd_config
sed -i "s/.*PermitRootLogin.*/PermitRootLogin no/g" /etc/ssh/sshd_config
echo "AllowUsers $user" >> /etc/ssh/sshd_config
echo "$user      ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers


# setup firewall
apt-get -y -q install ufw
ufw default deny incoming
ufw allow ssh
ufw allow http
ufw allow https
ufw --force enable


# setup fail2ban
apt-get -y -q install fail2ban
cat <<EOT > /etc/fail2ban/jail.local
[DEFAULT]
bantime = 1h

[sshd]
enabled = true

[apache-auth]
enabled  = true

[apache-badbots]
enabled  = true

[apache-noscript]
enabled  = true

[apache-overflows]
enabled  = true
EOT
systemctl reload fail2ban


# Install additional packages
if [ -n "$aptpackages" ]
then
  apt-get -y -q install $aptpackages
fi


# Install snap packages
if [ -n "$snappackages" ]
then
  apt-get -y -q install snapd
  snap install core
  snap refresh core
  snap install --classic $snappackages
fi