Jul 16, 2024 - Internal services on public domain

Internal services on public domain

In the post before I told you a bit about my experience with Home Assistant. Now I want to make my Home Assistant available to the outside world via a (sub)domain, so that I can connect the HA app on my phone for example. There were two problems to solve:

  1. The modem of my internet provider seems to block all incoming connections. It does not allow to specify a host/ip as DMZ. Initially I thought that this is pretty annoying. On the other hand it probably adds a nice security layer. Solution: I need to setup an ssh tunnel from the HA machine to an outside machine with a public IP.

  2. I don’t want to register a domain and/or IP for each service I want to expose (might also do the same for NextCloud later, etc.). So on the machine with the public IP I have to setup subdomains and proxies which redirect from the subdomain https://ha.mydomain.com to the port of the tunnel http://localhost:1234 . There’s lots of docs on this for Nginx, but not so much for Apache, so this was pretty fiddly, but I got it working in the end.

1 - The tunnel

#!/bin/bash
user="john"
public_ip="1.2.3.4"
public_port="1234"
service_ip="192.168.1.123"
service_port="4321"

while :
do
	date
	echo "open ssh connection"
	ssh -o ExitOnForwardFailure=yes -o ServerAliveInterval=60 -o ServerAliveCountMax=5 -R 0.0.0.0:$public_port:$service_ip:$service_port -N $user@$public_ip
	echo "ssh connection closed... waiting 10min until restart..."
	sleep 600
done

This is a little bash script that maintains an SSH tunnel between a machine which runs a service (service_ip) on a certain port (service_port) in the LAN and a machine with a public ip address (public_ip). Some SSH options explained: ExitOnForwardFailure makes sure the ssh command fails if the port is already in use (or other ‘secondary’ issues). ServerAliveInterval and ServerAliveCountMax ensure that the ssh server doesn’t close the connection due to inactivity (client will send null packets in the given interval). -R establish remote port forwarding. -N specifies that you only want a tunnel, not a remote shell. If for whatever reason the SSH connection is terminated, the script will wait 10min and then try again. Note: This script should obviously run on a machine within the same LAN network as the ‘service’ machine; and you have to setup public key authentication for $user on the $public_ip machine (e.g. using ssh-copy-id).

2 - The proxy

On the Apache server add a named virtual host for the subdomain, which only purpose is to redirect to https:

<VirtualHost *:80>
    ServerName ha.mydomain.com
    RewriteEngine on
    RewriteCond %{SERVER_NAME} =ha.mydomain.com
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

Then add the subdomain to the ssl config with the specific redirect to the local port (which you’ve tunnel with previous script):

<IfModule mod_ssl.c>
    <VirtualHost *:443>
        ServerName ha.mydomain.com
        RewriteEngine on

        ProxyPass "/" "http://localhost:1234/" upgrade=websocket
        ProxyPassReverse "/" "http://localhost:1234/"

        Include /etc/letsencrypt/options-ssl-apache.conf
        SSLCertificateFile /etc/letsencrypt/live/mydomain.com-0001/fullchain.pem
        SSLCertificateKeyFile /etc/letsencrypt/live/mydomain.com-0001/privkey.pem
    </VirtualHost>
</IfModule>

I used cerbot/letsencrypt to set SSL for the server. It might look a bit different for you. Note: You might have to run certbot --apache --expand in order to get a new certificate which includes the subdomain!

See the upgrade=websocket option. This is very important and took me ages to find out! You have to enable websocket proxy for HA, and that’s the line you need.

If you use Nginx instead of Apache, just google, there are plenty of examples.

May 27, 2024 - Weather station with Firebeetle/ESPHome

Weather station with Firebeetle/ESPHome

A while ago I bought a box full of Firebeetles, an offer I just couldn’t resist. But I didn’t really have time to do something with them. But then I came across Home Assistant - you can run it on a Raspberry Pi - and the addon ESPHome. At some point I have to write dedicated posts about the Firebeetle and Home Assistant…

Anyway, with ESPHome and HA it’s really easy to quickly spin up a little sensor/control/etc project. Something I had in mind for a long time, was a kind of “weather station” to monitor outside temperature, humidity and pressure. A quick Google suggested the BME280 chip. And I also needed a battery for the Firebeetle. You can directly plug in a lithium battery via a PH 2.0 JST connector.

In ESPHome you create a new device, paste in the config, connect your Firebeetle to the PC and flash it, done. Then just add the visualisation of the sensor readings in HA. That easy!

Well, working out the details of the config took me a while. One big problem of the Firebeetle is the high power consumption. It comes with lots of computing power, built-in Bluetooth, built-in Wifi, etc. But that has a price, power usage. So you want the Firebeetle to sleep most time, just wake up quickly, read the sensors, send them to HA, sleep again. And suddenly your config becomes quite a bit more complex (explanations below):

substitutions:
  my_ip: 192.168.168.160
  my_gw: 192.168.168.1
  my_sn: 255.255.255.0
  mqtt_host: 192.168.168.100

esphome:
  name: weather
  friendly_name: Weather
  on_boot:
    - priority: -300
      then:
        - script.execute: read_sensors

esp32:
  board: firebeetle32
  framework:
    type: arduino

# Enable logging
#logger:

# Enable Home Assistant API
#api:
#  encryption:
#    key: "xxx"

#ota:
#  password: "xxx"

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  fast_connect: true
  manual_ip:
    static_ip: $my_ip
    gateway: $my_gw
    subnet: $my_sn

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Weather Fallback Hotspot"
    password: "xxx"

mqtt:
  broker: $mqtt_host
  username: !secret mqtt_user
  password: !secret mqtt_password
  birth_message:
  will_message:

captive_portal:

deep_sleep:
  id: sleepy

i2c:
  sda: 21
  scl: 22
  scan: true
  id: bus_a

sensor:
  - platform: bme280_i2c
    id: bme_id
    i2c_id: bus_a
    address: 0x76
    temperature:
      name: "W Temperature"
      on_value: 
        - then:
          - lambda: |-
              id(updates)++;
    pressure:
      name: "W Pressure"
      on_value: 
        - then:
          - lambda: |-
              id(updates)++;
    humidity:
      name: "W Humidity"
      on_value: 
        - then:
          - lambda: |-
              id(updates)++;
    update_interval: 60s

  - platform: adc
    id: battvcc_id
    name: "W Battery Vcc"
    pin: 
      number: GPIO34 # A2
      allow_other_uses: true
    accuracy_decimals: 2
    attenuation: 11dB
    filters:
      - multiply: 2.0
    update_interval: never
    on_value: 
      - then:
        - lambda: |-
            id(updates)++;

  - platform: adc
    id: batt_id
    name: "W Battery"
    pin: 
      number: GPIO34 # A2
      allow_other_uses: true
    accuracy_decimals: 0
    attenuation: 11dB
    filters:
      - multiply: 2.0
      - calibrate_linear:
        - 4.20 -> 100
        - 4.06 -> 90
        - 3.98 -> 80
        - 3.92 -> 70
        - 3.87 -> 60
        - 3.82 -> 50
        - 3.79 -> 40
        - 3.77 -> 30
        - 3.74 -> 20
        - 3.68 -> 10
        - 3.45 -> 5
        - 3.00 -> 0
      - clamp:
          min_value: 0
          max_value: 100
    unit_of_measurement: '%'
    update_interval: never
    on_value: 
      - then:
        - lambda: |-
            id(updates)++;

globals:
  - id: updates
    type: int
    restore_value: no
    initial_value: '0'
script:
  - id: read_sensors
    then:
      - lambda: |-
          id(updates) = 0; 
      - component.update: battvcc_id
      - component.update: batt_id
      - component.update: bme_id
      - wait_until:
          lambda: |-
            return (id(updates) >= 5);
      - deep_sleep.enter:
          id: sleepy
          sleep_duration: 60min

The main points:

  • Use fixed IP addresses, not DHCP (I just fished out an old Wifi router and set up a dedicated LAN for HA).
  • Disable logging.
  • Disable the HA API. That means you cannot update your Firebeetle ‘over-the-air’, and HA can’t read your sensor values. But we won’t need that.
  • The sensors are read by a ‘script’ - in above example that happens once every 60min - then sleep.
  • The consequence of this is, that your Firebeetle will show up as ‘not available’ and it’s sensor readings as ‘None’ (except the 2 or 3 sec out of the 3600 sec when it actually is active).
  • For that reason, use MQTT. Get the HA MQTT integration and see the mqtt bit in the config.

Some notes about the script:

It has an internal counter, which increments for each sensor value read (these are triggered byt component.update). After all 5 values are read (temperature, humidity and pressure from BME280 chip, and as little bonus the battery voltage and charge via adc (internally from pin A2)) it will send the Firebeetle into deep sleep for another 60min.

One thing which really tripped me, was this: I set up a few more Firebeetles with DHT22 sensors in the house to get temperature and humidity of different rooms. So I had a few sensors with name “Temperature” on different Firebeetles. Oh, the MQTT integration doesn’t like that! The sensor name needs to be unique! I don’t know why. On the MQTT level this is not a problem at all, but the HA MQTT integration can’t handle it. Hence note the ‘W’ in:

    temperature:
      name: "W Temperature"

Nov 1, 2023 - Rust, again

Rust, again

I think I got the fundamentals now. But I still lack proper practical Rust experience on a larger scale. So I was looking at some courses, like the Rust course from Let’s Get Rusty. The guy has some really good videos on youtube. Now he also offers a course, which is basically the videos glued together with an online platform where you can interact with other Rust learners and ask questions. The course is 500 USD and you’ll get some kind of certificate if you finish the exams/projects. My worries there are, first it’s a lot of money, second how ‘accredited’ is this certificate when it comes to looking for a Rusty job. Nevertheless, if these kind of courses are your preferred way to learn, do it, it’s good, the guy knows what he’s talking about.

But I got some more books: “Practical Rust Projects” and “Rust in Action”. Both are very practical. But again, the projects aren’t that interesting. I think in order to persevere, I need some little projects which I personally find very interesting. One is cryptography! Don’t get me wrong, that’s gonna be on a very amateurish level. So I started a little command line program ‘secrets’, written in Rust. It will simply allow the user to en/decrypt messages using various ciphers. At the moment I’ve only implemented a transposition cipher. Next will be the ‘Shamir’s secret sharing’ cipher. I got the idea from Christof Paar’s lectures about modular arithmetic (generally highly recommended!). And I will use the above mentioned books more “on the case” whenever I have to look up a ‘best practice’ to get something specific done in Rust.

The most important thing next to the languange itself is to know some main libraries. So far I got familiar with:

  • clap for writing CLI applications.
  • anyhow for dealing with errors in an application.
  • thiserror for defining your own error types when you write a library.
  • serde pretty cool de/serialisation library.
  • num-bigint for dealing with large integers.
  • regex for regular expressions.
  • imap to connect to mail servers via IMAP.

Also important, how to actually structure your project. Coming from an object oriented view, I like to bundle functionality in small, overseeable files. This is possible with Rust too, although most easy beginner examples always show max two files, the lib.rs and the main.rs. But if I have different objects (Structs) I tend to put them in their own .rs file. Not just Structs, also other “bundles” of common functionality. In Rust that’s called ‘Modules’.

For example for the ‘secrets’ application, I added all functionality for the transposition cipher into the file transpose.rs:

pub fn encode(message: &str, password: &str) -> Result<String> {
    ...
}
pub fn decode(message: &str, password: &str) -> Result<String> {
    ...
}
...

(I should actually also create a Trait (equivalent to Interface in other languages) for these methods)

In order to use the code in transpose.rs, I have to add it as module to the lib.rs:

pub mod transpose;

Then I can use it in main.rs:

use secrets::transpose;

If you don’t use it in main.rs but only in other modules on the “same level” then you don’t need the pub keyword. For larger projects you can also split up modules into different directories. But that’s one of these things I’m not really sure yet what the best practice is.

The bigger things I still have to get a grasp on are:

  • Async - Asychronous programming. Another area which is a huge source of bugs in other programming languages. But in Rust, once it compiles, you’re pretty safe.
  • Livetimes - The general advice seems to be: Avoid dealing with them, if it’s not strictly necessary. But sometimes it is. For example in asynchronous environments.
  • Webassembly - Just curious, interesting area. I’m not a web developer. Maybe that’s something which will get me a bit more interested in that field?

So much for now… :-)