Rust, again

I think I got the fundamentals now. But I still lack proper practical Rust experience on a larger scale. So I was looking at some courses, like the Rust course from Let’s Get Rusty. The guy has some really good videos on youtube. Now he also offers a course, which is basically the videos glued together with an online platform where you can interact with other Rust learners and ask questions. The course is 500 USD and you’ll get some kind of certificate if you finish the exams/projects. My worries there are, first it’s a lot of money, second how ‘accredited’ is this certificate when it comes to looking for a Rusty job. Nevertheless, if these kind of courses are your preferred way to learn, do it, it’s good, the guy knows what he’s talking about.

But I got some more books: “Practical Rust Projects” and “Rust in Action”. Both are very practical. But again, the projects aren’t that interesting. I think in order to persevere, I need some little projects which I personally find very interesting. One is cryptography! Don’t get me wrong, that’s gonna be on a very amateurish level. So I started a little command line program ‘secrets’, written in Rust. It will simply allow the user to en/decrypt messages using various ciphers. At the moment I’ve only implemented a transposition cipher. Next will be the ‘Shamir’s secret sharing’ cipher. I got the idea from Christof Paar’s lectures about modular arithmetic (generally highly recommended!). And I will use the above mentioned books more “on the case” whenever I have to look up a ‘best practice’ to get something specific done in Rust.

The most important thing next to the languange itself is to know some main libraries. So far I got familiar with:

• clap for writing CLI applications.
• anyhow for dealing with errors in an application.
• thiserror for defining your own error types when you write a library.
• serde pretty cool de/serialisation library.
• num-bigint for dealing with large integers.
• regex for regular expressions.
• imap to connect to mail servers via IMAP.

Also important, how to actually structure your project. Coming from an object oriented view, I like to bundle functionality in small, overseeable files. This is possible with Rust too, although most easy beginner examples always show max two files, the lib.rs and the main.rs. But if I have different objects (Structs) I tend to put them in their own .rs file. Not just Structs, also other “bundles” of common functionality. In Rust that’s called ‘Modules’.

For example for the ‘secrets’ application, I added all functionality for the transposition cipher into the file transpose.rs:

pub fn encode(message: &str, password: &str) -> Result<String> {
    ...
}
pub fn decode(message: &str, password: &str) -> Result<String> {
    ...
}
...

(I should actually also create a Trait (equivalent to Interface in other languages) for these methods)

In order to use the code in transpose.rs, I have to add it as module to the lib.rs:

pub mod transpose;

Then I can use it in main.rs:

use secrets::transpose;

If you don’t use it in main.rs but only in other modules on the “same level” then you don’t need the pub keyword. For larger projects you can also split up modules into different directories. But that’s one of these things I’m not really sure yet what the best practice is.

The bigger things I still have to get a grasp on are:

• Async - Asychronous programming. Another area which is a huge source of bugs in other programming languages. But in Rust, once it compiles, you’re pretty safe.
• Livetimes - The general advice seems to be: Avoid dealing with them, if it’s not strictly necessary. But sometimes it is. For example in asynchronous environments.
• Webassembly - Just curious, interesting area. I’m not a web developer. Maybe that’s something which will get me a bit more interested in that field?

So much for now… :-)

More Bash

Here are some more, hopefully useful hacks for bash scripts.

Default options

It is often recommended to set a few options in bash scripts as default:

set -euo pipefail

What that means:

• -e : Immediately exit the bash script if a command fails.
• -u : Immediately exit the bash script if an undefined variable is called.
• -o pipefail : In case of chained (“piped”) command (e. g. grep something /var/log/something.log | sort) fails, return the error code of the failed command for the chain/pipe.

This is sometimes also referred to as unofficial “Bash strict mode”. But it comes with caveats!

Redirect output

# Disable stderr
exec 2>/dev/null

# Disable stdout
exec 1>/dev/null

# Disable both
exec 2>&1 1>/dev/null

# Redirect all output to some kind of log file
exec 1> /tmp/output.log
exec 2>&1

The internal field separator (IFS)

This variable defines how the bash splits strings. Useful when used in combination with read, for example:

IFS='|' read key value <<< "someKey|someValue"; echo "key: $key; value: $value"

Default values

# If nothing is provided as first argument, use someFile.txt as filename
filename=${1:-someFile.txt}

Multiline text snippets

cat << EOF
This is some
input over
multiple lines
EOF

mail -s "Alarm!" admin@example.com << EOF
Someone tried to hack your server!
Kind Regards,
The friendly hacker
EOF

Note: EOF is just some marker, you can call it whatever you want. EOF (end of file) or EOT (end of transmission) is often used.

Linux time machine

A few days ago my nextcloud server broke apart, completely. It had to be re-installed from scratch. I hadn’t realised that my backup wasn’t working, so I didn’t have a mysql dump. Luckily I still could mount the harddisk and rescue the /var/lib/mysql directory, with all the *.frm, *.ibd, etc. database files.

I re-installed the server and tried to simply copy the recovered database files over. Of course that didn’t work, because the database files were created with an older mariadb version. At least the error message said exactly what version they were created with. So I had to get the same version again, then I could spin up the database and do a mysqldump, which I then can re-import into the up-to-date mariadb again.

On a Debian system that’s basically impossible. Unless you’re lucky and a Debian release has exactly the version you need. You can get these older versions from previous releases or a newer version by enabling testing repositories. But even if you could get the version you need, if you’ve ever tried that, you know what a pain this is! Dependency hell!

But there’s Arch to the rescue! So I span up an Arch VM (see Vagrant file at the bottom). Went to the Arch archive: Arch Archive . Found the mariadb package of the specific version and noted the date when it was added. SSH’d into the Arch VM and adjusted the /etc/pacman.d/mirrorlist with the specific date URL e.g. Server=https://archive.archlinux.org/repos/2014/03/30/$repo/os/$arch. Then got the past keyring with pacman -S archlinux-keyring ca-certificates followed by a complete reset of the system to the past with pacman -Syyuu. And I had a linux system with the exact version of mariadb and all the necessary dependencies I wanted. Truely awesome! I don’t know of any other linux distribution which can do this!

Detailed instructions here: How to restore all packages to a specific date

Here’s a Vagrant file if you quickly wanna spin up an Arch VM:

Vagrant.configure("2") do |config|
  config.vm.box = "archlinux/archlinux"
  config.vm.network "private_network", ip: "192.168.56.100", :name => 'vboxnet0', :adapter => 2

  config.vm.provider "virtualbox" do |vb|
    vb.name = "archy"
    vb.memory = "4096"
    vb.cpus = "2"
  end

  config.vm.provision "shell", inline: <<-SHELL
    sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/g' /etc/ssh/sshd_config
    echo 'MaxAuthTries 100' >> /etc/ssh/sshd_config
    systemctl restart sshd.service
  SHELL
end

You can then simply ssh into vagrant@192.168.56.100 (this is using Virtualbox with a ‘vboxnet0’ host network setup); I don’t like the vagrant ssh workaround.