Jun 26, 2019 - Useful git commands

Useful git commands

Just a collection of git commands I use on a daily basis:

# Clone a repository (using https)
git clone https://github.com/[some_user]/[some_repo].git

# Add your fork as remote (using ssh)
git remote add my_repo git@github.com:/[username]/[some_repo].git

# If working with submodules don't forgot to update
git submodules update --init

# Update information about remote branches
git fetch --all

# List remote branches and tags
git ls-remote [repo]

# Checkout (-t : follow upstream branch)
git checkout -t [repo]/[branch]

# Pull the latest changes from the remote upstream branch
git pull

# List local branches
git branch

# Checkout local branch
git checkout [branch]

# Create a new branch
git checkout -b [branch]

# Push current branch
git push [repo]

# Delete branch
git branch -d [branch]

# Rename branch ([old] optional, if omitted = current branch)
git branch -m [old] [new]

# Show commits
git log

# Show the changes of a specific commit
git show [sha]

# Check which files have been changed
git status

# Check the changes ([file] optional)
git diff [file]

# Add changes (add everything: [file] = . )
git add [file]

# commit
git commit

# Edit last commit
git commit --amend

# Reset branch to a specific commit
git reset --hard [sha]

# If that was a mistake and you have to get back to a commit
# later than the resetted commit, find the sha with
git reflog # then followed but git reset

# Cherry pick a specific commit (e.g. from another branch)
git cherry-pick [sha]

# Rebase, e.g. you branched off master, then master changed:
git checkout master
git pull
git checkout [previous branch]
git rebase master

# Merge other branch into current branch
git merge [branch]

# Tag and sign a version
git tag -s -a v1.2.3 -m "Tag version 1.2.3"
git push origin v1.2.3

# Delete tag
git push --delete origin v1.2.3
git tag -d v1.2.3

Jun 3, 2019 - Vagrant and Ansible

Vagrant and Ansible

Showing some basic concepts of Ansible using a VM provisioned with Vagrant. For details see the Ansible and Vagrant documentation.

Objectives

As little demo project I want to create an Ansible role ‘torproxy’ which installs and runs tor, listening for proxy clients on an IP which is accessible on the LAN.

A bit of background (caution: very simplistic): Tor is a network of nodes, which is used to disguise the origin of requests. You can use tor as proxy in the webbrowser. If you then access a website, the request is passed through several nodes in a practically untraceable way. Only the last node (the ‘exit node’) will access the website, requests the information and passes it back to you through the tor network again. You can also use tor to provide a service within the tor network itself. This service is run completely anonymously without revealing any information about the physical host itself. I. e. you can basically set up tor for four different purposes:

  • As internal node only to support the network anonymously.
  • As exit node to support the network. But your IP address will be visible to the outside, and recognizable as part of the tor network. Caution: This can lead to problems.
  • Running an own service within the tor network.
  • Providing a proxy to the tor network, e.g. to use with your browser.

I want to do the latter, set up a machine which provides a proxy to the tor network usable within the LAN.

Note: Alternatively you can use torbrowser which bundles a ‘security hardened’ Firefox with a tor proxy running in the back. Although more secure, I don’t really like it. I prefer to use my own browser with a separate tor proxy.

Stuff you need

Set up a VM with Vagrant

First create a directory for our project called ‘torproxy’.

Then create a Vagrant config which spins up a Debian/Stretch virtual machine using VirtualBox, with 1GB RAM and 1 CPU. Assign IP 192.168.99.99 to it. Copy over the local public SSH ID so that we (and Ansible) can simply login via vagrant@192.168.99.99. If you don’t have one, create it now with ssh-keygen!

torproxy/Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "debian/stretch64"
  
  config.vm.network "private_network", ip: "192.168.99.99"

  config.vm.provider "virtualbox" do |vb|
    vb.name = "torproxy_vm"
    vb.memory = 1024
    vb.cpus = 1
  end

  config.vm.provision "shell" do |s|
    ssh_pub_key = File.readlines("#{Dir.home}/.ssh/id_rsa.pub").first.strip
    s.inline = <<-SHELL
      echo #{ssh_pub_key} >> /home/vagrant/.ssh/authorized_keys
    SHELL
  end
end

Create Ansible role ‘torproxy’

We could put everything in a playbook, but it’s cleaner and reusable if we create an Ansible role for our task.

Create the directory structure roles/torproxy/tasks. Then put everything we need to install tor into main.yml:

torproxy/roles/torproxy/tasks/main.yml

---
- name: Install some required packages, apt-transport-https
  become: true
  apt:
    name: apt-transport-https
    update_cache: yes

- name: Install some required packages, policykit-1
  become: true
  apt:
    name: policykit-1

- name: Add tor apt key
  become: true
  apt_key:
    id: A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89
    url: https://deb.torproject.org/torproject.org/A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89.asc

- name: Add tor apt repository
  become: true
  apt_repository:
    repo: deb https://deb.torproject.org/torproject.org stretch main

- name: Install tor
  become: true
  apt:
    name: tor
    update_cache: yes

- name: Enable SOCKS proxy
  become: true
  lineinfile:
    path: /etc/tor/torrc
    regexp: '^#SocksPort 192'
    insertafter: '^#SocksPort 192'
    line: SocksPort :

- name: Allow only local network access
  become: true
  lineinfile:
     path: /etc/tor/torrc
     regexp: '^#SocksPolicy rej'
     insertafter: '^#SocksPolicy rej'
     line: SocksPolicy accept /24
  notify:
    - restart tor

The last line in the task’s main.yml triggers a ‘restart tor’ handler. We still have to create that one. Create the roles/torproxy/handlers directory and add the following to the main.yml:

torproxy/roles/torproxy/handlers/main.yml

---
- name: restart tor
  become: true
  service:
    name: tor
    state: restarted

Create Ansible playbook

torproxy/playbook.yml

---
- hosts: torproxy_vm
  roles:
    - role: torproxy
      vars:
        proxy_port: 8888

Add the VM to the Ansible hosts

/etc/ansible/host

[torproxy_vm]
192.168.99.99

[all:vars]
ansible_user=vagrant

Run it

In the torproxy project directory

vagrant up # Spin up the VM
ansible-playbook playbook.yml # Run the tor proxy installation

Then set the proxy settings of your browser to Socks5 proxy 192.168.99.99 port 8888 and enjoy the anonymity.

May 22, 2019 - More bash snippets

More bash snippets

Here are some more bash useful bash snippets:

Arithmetic, like incrementing a counter:

i=1
j=$((i+1))
echo $j

Looping over things:

# Run a command several times with specific values
for i in 1 2 3; do echo $i; done

# Loop over a specifc range of values using 'seq'
for i in $(seq 1 3); do echo $i; done

# Run a command on all txt files in current directory
for f in `ls *.txt`; do head $j; done

Redirecting output into files:

# Redirect standard output
date 1> out.log

# Redirect error output
blah 2> error.log

# Redirect both into different files
for cmd in date blah; do $cmd; done 1> out.log 2> error.log

# Redirect any kind of output into same file
for cmd in date blah; do $cmd; done &> all.log

Here’s a snippet I use for running a long running command, for example over a range of objects (accessible via some kind of ID)

# Open a screen session
screen -S BigJob

# Launch the job loop
for id in 1 2 3 4 5; do echo $id; date; do_something_with $id; done &> bigJob.log

# Detach again by holding CTRL-A and pressing D

The jobs are running safely in a screen environment, i.e. you don’t have to worry that you accidentaly kill the process by getting disconnected from the SSH session. And you can check the progress live via

tail -f bigJob.log

As the snippets prints the $id you can see which object/id is currently processed; and using date in the snippet allows you to work out how long each job took.

©2023 Floki.cc.