OpenVPN with 2 factor authentication on the Raspberry Pi

This guide, describes how to set up a VPN (Virtual Private Network) on your raspberry pi to allow you to remotely connect to your home network.

A VPN allows a remote computer to act as though its part of your home newtork - vpn clients can see computers sitting on your home lan.  I've found it really useful when I've been working away staying in hotels with relatively slow internet connections.  The VPN in itself does not take up much band width, so once you are connected, and remote desktopped to one of your home PC's, you can operate at your home networks speed.  As the VPN is configured to use UDP, you get a much more stable connection (believe it or not!).

Prerequisites

• Vanilla install of Raspbian (You should be able to login using pi/raspberry)

Approach

We're going to use OpenVPN and Ansible to orchestrate the install, and set up the configuration.  

We'll use google authenticator to provide 2 factor authentication, configured as a PAM module.

To test, we'll use a mobile phone, connected to the internet via 3g.

Setup Ansible

We're going to install ansible directly onto the Pi.  Log onto the raspberry pi, using pi/raspberry:

ssh-keygen    (go with defaults and no password)
sudo apt-get update
sudo apt-get install python-dev python-pip sshpass git
sudo pip install ansible
git clone https://github.com/coder36/raspi-ansible.git
cd raspi-ansible

Register the raspberry pi with anisble:

ssh 127.0.0.1  (selecting yes when asked about key fingerprint, password: raspberry)

cd ~/raspi-ansible
ansible-playbook -i live bootstrap.yml -k     (password: raspberry)

Edit inventory

The live file is called the inventory.  This lists the network config, which openvpn needs.  Edit this to match your own network configuration:

[pi]
127.0.0.1 isp_ip="123.1.12.22" local_lan="192.168.101.0" local_lan_mask="255.255.255.0" vpn_lan="10.8.0.0" vpn_lan_mask="255.255.255.0"

• isp_ip - set this to your ip address.  You can find your ip address here
• local_lan - set this to your network subnet.  *** important!
• local_lan_mask - usually 255.255.255.0
• vpn_lan - go with the default.
• vpn_lan_mask - go with the default.

Run the playbook

cd ~/raspi-ansible
ansible-playbook -i live vpn.yml
sudo /etc/network/if-up.d/openvpn_firewall

This will configure and start the openvpn server.

It also will create a client file: /etc/openvpn/client.ovpn which can be used to configure the openvpn client.

Home router configuration

• Forward port 1194 (UDP) to your raspberry pi.

Testing

We'll test the vpn connection using a mobile phone, simply because its the quickest way to prove that its working:

• Install openvpn on your phone.
• Install google authenticator on your phone.
• Somehow get the client.ovpn onto your phone. (You could email it to yourself then download it!)

Create VPN user 

For each user of your vpn, you will need to generate a user on the raspberry pi, and a run google-authenticator.  Log onto the raspberry pi using pi/raspberry:

sudo adduser fred (provide password as 'monday', and go with defaults for the remaining of the prompts)
su - fred   (password: monday)
google-authenticator  (Answer y to the questions)

This will generate a QR code. Scan this into your phone using the google authenticator app. There is also a URL, which you could email to the user:

On your phone

• Disable wifi, to ensure that you are not connecting via your home network. Use mobile data ie. 3g.
• Open openvpn and import client.ovpn

You will be prompted for a username and password:

username: fred password: monday13520

The password, is fred's password concatenated with the google authenticator verification code provided by the app.

If everything is happy, you should now be connected and able  to navigate to your routers home page ie.: http://192.168.101.254

SSL Certs

You should generate your own certificates and private keys and copy them into /home/vagrant/playbooks/roles/vpn-server/templates/etc/openvpn. Until you do this, a potential hacker would in theory be able to intercept your and inspect your VPN traffic.  On ubuntu I use tinyca2 and on windows XCA to generate my certificates.

Route all traffic through VPN

Edit  /home/vagrant/playbooks/roles/vpn-server/templates/etc/openvpn. Uncomment the line:

push "redirect-gateway def1"

Debugging

 If things are not working, try connecting using the openvpn client whilst running:

sudo tail -f /var/log/syslog 
sudo tail -f /var/log/auth.log

Look in  syslog for openvpn issues, and auth.log for google authenticator issues.

Good luck!

Links

Raspberry Pi

Latest Raspbian image

Ansible VPN playbook

Windows SSL Certificate Manager

Update (Feb 2016)

1) VPN SSL certificate updated to never expire
2) server.conf updated to point to correct pam module: openvpn-plugin-auth-pam.so

3) Added config to optionally route all traffic through vpn

Question framework for govuk sites

Online forms

I've spent most of my career working on government projects, creating websites which ask a set of questions and collect data.  On every project I work on, we seemingly reinvent the same thing,   This is where GDS have really added value, by creating a common set of libraries and tools to standardize how gov.uk sites look and feel.

What's missing, is a common approach to writing questionnaire type applications.  That's why I developed doopgovuk.

Doopgovuk

Doop is a Ruby on Rails question framework for govuk sites, inspired by the work GDS have done to standardize the cross government internet presence.  It allows you to easily create a questionnaire using a simple DSL.  The results are amazing.  It looks even better on a mobile device.

Try out the Child Benefit Demo developed with doop.

Self Assessment Demo which uses radio / button combo's to answer questions:

Getting started

Assuming ruby, rails and nodejs is installed:

sudo apt-get install qt4-dev-tools libqt4-dev libqt4-core libqt4-gui xvfb

These are needed to support the headless capybara test suite.

$ rails new govsite
$ cd govsite
$ echo "gem 'doop'" >> Gemfile
$ bundle
$ rails generate doopgovuk demo
$ rails s

Navigate to http://localhost:3000/demo/index

The DSL

When developing a doop application, to get a feel for what's involved, I've outlined some of the steps:

1) Start by defining a YAML structure which lists all the questions to be asked (see here):

page: {
  preamble: {
   _page: "preamble",
   _nav_name: "Preamble",
   income_more_than_50000: {
   _question: "Do you have an income of more than £50,000 a year ?"
  },
  do_you_still_want_to_apply: {
    _question: "Do you still want to apply for child benefit?"
  },
  proof_of_id: {
    _question: "We need proof of your identity",
    _answer: {}
  }
},

2) You then define a set of handlers which are called whenever a question is answered:

on_answer "/page/preamble/enrolled_before"  do |question,path, params, answer|
  answer_with( question, { "_summary" => answer } )
end

on_answer "/page/preamble/enrolled_before"  do |question,path, params, answer|
  answer_with( question, { "_summary" => answer } )
  enable( "/page/preamble/year_last_applied", answer == "Yes" )
end

3) And finally write the .erb to display the questions (see here):

%=question_form doop, res do %>
  <%=question "/page/preamble" do |root, answer| %>

    <%=question "/page/preamble/income_more_than_50000" do |root,answer| %>
      <button name="b_answer" value="Yes">Yes</button><br/>
      <button name="b_answer" value="No"></button>
    <% end %>

    <%=question "/page/preamble/do_you_still_want_to_apply" do |root,answer| %>
      <button name="b_answer" value="Yes">Yes, I still want to apply for child benefit</button
    <% end %>

    <% when_answered "/page/preamble" do %>
      <button>Continue and Save</button>
    <% end %>

  <% end %>
<% end %>

Give it a go...

Links

Doop github project

Child Benefit Demo

Self Assessment Demo

GDS

GOV UK Elements

Why fake RPM's ?

In IT companies across the globe, infrastructure and servers are provisioned at the click of a button - this is the world the Devops engineer is creating.  The deployment tool we used at this particular organisation is called ansible.  Ansible allows the programmatic install of RPM's and config files on target servers - our whole estate was built off the back of ansible scripts.

We wanted to develop the ansible scripts using TDD (Test Driven Development), so that we could put the whole regression test and deployment into a pipeline.

Importantly for TDD to work you get yourself into a feedback loop:

1) Write specs
2) Update ansible scripts
3) Run specs  (this will start with an empty base server build then apply the ansible scripts)
4) Repeat 2 and 3 until specs pass

The problem we had was that running the specs could take up an hour.  The feedback loop was way too long!  It turned out that some of the RPM's which the ansible scripts install took the best part of 30 mins to complete.

So we made a compromise - we would install fake rpms for the longest running offenders (luckily these were "bloated 3rd party monitoring tools" so not critical to the servers functionality)

FPM

FPM is a ruby gem to create .rpm's (or .debs if running on ubuntu).  To install you will need ruby:

curl -sSL https://get.rvm.io | bash -s stable

rvm install 1.9.3

rvm use 1.9.3 --default

gem install fpm

Fake it

The idea, is to add the fake rpm to the yum repository (alongside the original), but give the fake a higher version number, so it gets picked up by the yum install command.

What we found was that in some of the in house rpm's, the epoch had been set to the unix epoch time and in fact whenever the rpm was regenerated the epoch would increase.  In this scenario, the highest epoch wins, irrespective of the version number.

So the strategy for creating the winning the version, is very much dependent on how the original RPM was versioned:

mkdir -p /tmp/fake_package
fpm -s dir -t rpm -n sysmontool -v 999999999 --epoch 2000000000 -f -a noarch -C /tmp/fake_package ./
fpm -s dir -t rpm -n patchmontool -v 0.0.1 --epoch 2000000000 --iteration 999 -f -C /tmp/fake_package ./

Strangely, you can not just set the epoch to 9999999999,   We actually set the epoch to `date +%s`

Conclusion

The fpm command will create a .rpm.  Copy this into your yum repository and do a yum update.  You make need to do a yum clean all if its not getting picked up.

From experience, for server builds which are automated, it might be a good idea to set the yum,conf metadata_expire field to a low value.  This avoids the confusing scenario of thinking that something has installed/updated when it hasn't.

How to pair using TMUX

Why?

Tmux provides a way to remotely pair using a shared terminal.  This is great as it means you can pair when working from another office or from home.  It's ideal if you are comfortable using a command line, and enjoy memorizing keyboard shortcuts.

Issues

Re-sizing windows - when one side of the pair has a different resolution or differently sized terminal window, if you're not careful, you will see screen sheering.  This was completely solved by using putty or cygwin and ensuring that both clients use the same font settings.  Once its working, it works really well:

Weird control characters - If you see weird characters on the screen then set the window translation -> character set translation to UTF-8.  This seems to be only a problem in putty.

Git bash - Tmux does not work nicely with git bash

Server Setup

Assuming you are using vagrant on your local PC, create a vagrant server on 192.168.33.10, using the following vagrant snippet when setting up the network:

config.vm.define "a" do |server|
  ## usual config
  ## ...
  server.vm.network "private_network", ip: "192.168.33.10"
  server.vm.network "forwarded_port", guest: 2222, host: 22
end

What's important here is that you forward 2222 to the ssh port 22.

ssh onto your vagrant instance and install and run tmux:
     
yum install -y tmux
tmux

Now use the ip of your physical box to ssh in:

Client Setup

Install cygwin or putty.  From cygwin remotely attach to the tmux session:

ssh vagrant@<ip>  
tmux attach  (this will attach to the session started above)

Remote Connectivity

• Use a wired connection.  Once I plugged in directly to the home router a lot of the connectivity issues went away (despite having a 5 bar wireless connection).
• If using vmware or virtual box to host vagrant instance, ensure you use Bridged Mode.  If you use NAT mode, we found the connection would occasionally drop.  
• If you have both vmware and virtual box running, then it may not be possible to enable bridged mode.  Uninstall one of them!

Viber

We tried several voip clients, including skype and googlr hangouts... The most stable turned out to be Viber.  I installed viber as a client on the remote PC.  Combined with a wired connection, it made the voip experience rock solid.

• Use viber with a hands free headset.  I used a cheap bluetooth headset from amazon.
• Be warned that voip eats bandwidth - try and get a wifi connection.

Conclusion

Tmux takes a bit of getting used to - you may find the default bind key is a bit of a stretch (ctrl-b).  I changed it to (ctrl-a).  Use this Tmux cheat sheet to quickly get up and running.  I've put together a set of dotfiles to make the whole tmux experience a lot nicer: https://github.com/coder36/dotfiles

Back button bindings

Sometimes you need to change what happens when the browser back button is pressed - for example you might want to disable the back button, or as I did, bind it to a hidden button.

The simplest way is as follows (assuming you are using jquery):

Javascript:

$( function() {
  history.pushState("back", null, null);
  if (typeof history.pushState === "function") {
    history.pushState("back", null, null);
    window.onpopstate = function (evt)
    {
      history.pushState('back', null, null);
      alert( "Back button pressed!" )
     };
   } 
})


Coffeescript:

$( () ->
  history.pushState("back", null, null);
  if typeof history.pushState == "function"
    history.pushState("back", null, null);
    window.onpopstate = (evt) ->
      history.pushState('back', null, null);
      $( "#back_a_page" ).click()
)

The coffeescript example will result in the button with id="back_a_page" getting clicked, whenever the back button is pressed.  

In chrome, to quickly test, this, go to  github  (which uses jquery) and hit the F12 button, then select console.  Paste in the javascript code and hit enter.  Now hit the back button :) 

Ubuntu - restarting your network

Don't forget that when you do this over an ssh/putty session then your connection will be dropped:

sudo ifdown -a
sudo ifup -a

sudo /etc/init.d/networking restart

Wordchain puzzle

Word Chains

Taken from http://codekata.pragprog.com/2007/01/kata_nineteen_w.html

Write a program that solves word-chain puzzles:

Find a chain of words start starting with one particular word and ending with another. Successive entries in the chain must all be real words, and each can differ from the previous word by just one letter. For example, you can get from "cat" to "dog" using the following chain.
 
    cat
    cot
    cog
    dog
 
The objective of this kata is to write a program that accepts start and end words and, using words from the dictionary, builds a word chain between them. Find the shortest word chain that solves each puzzle

You can turn lead into gold with the following chain:
     
    lead
    load
    goad
    gold
 
Try to capture the developer mindset and record how you to got to you solution.

Development process

How did I get to my solution ?  It's all about covergence.  I start out and see where it goes, see what ideas pop up as I go along, see what sticks etc.  My generate approach is as follows:

1) Write Tests - this gets me thinking about how should I represent the data, and helps drive the interface.
2) Start writing code
3) Get to a working solution (it's not always pretty at this point)
4) Refactor Refactor Refactor until I hit a universal truth ie. the code becomes so simple it must be true!

To solve a problem I generally have a series of insghts - by this I mean ideas which seems obviously the right thing to do and possibly are a deep truth of the solution.

Definitions:

distance - Given a "from" word and a "target" the distance is defined to be the number letters which would need to be changed in the "from" word to mutate it to the "target" word.  ie.  cat -> bot has a distance of 2

Here we go:

Write the unit tests - this brings the first decision - should I use List<String>'s or arrays [] to represent the dictionary? Choose [] as it makes the unit tests slightly more readable.

Thought - its going to be difficult testing with a large dictionary having to search through 1000's of words, multiple roots through the wordchain "space".

• Insight 1: Allow the tests to define the dictionary.

Idea - find a word in the dictionary which has a distance of 1 from the current word, but also contains at least 1 letter contained in the "to" word.  ie. for the wordchain cat -> dog then cot is 1 from cat and 2 from dog.  The only downside is that this strategy will only work for some word chains...

   "cat", "cot", "cog", "dog"

coding.....

The code is getting fairly complictated - theres lots of edgecases.
Idea - I'll need to keep track of words I've tried which result in dead ends.  Can I exclude these ?  from future searched or can  I?

coding.....

What happens if I need to go to a word which has no letters in common with the target word?  This is starting to get too complicated.  What I have learned is that I some how need to keep track of what wordchains I've visited which have resulted in deadends.

Idea - I'm thinking that the most reliable solution would be to brute force this, ie. work out every possible word chain.  This will produce some long chains.  Some of the chains would be dead ends and these can be discounted. Visually a brute force approach would look like a word chain tree with branched coming of branches etc.

• Insight 2: Think of the word chains as a tree to which we can prune dead branches.
• Insight 3: Use self recursion
• Insight 4: Use a stack to represent the branches

I'd need to model this tree as a Map<String, Stack<String>> with the key to the map being the root of the branch.

coding ...

This is looking better.  Can I get rid of the Map ?  I think I'm on the right track as the code is getting simpler.  The self recursion is looking good.

Amazing! - I've got something that works, my unit tests are passing.  The sef recursion involves passing a lot of things around which is not ideal, but this a break through.  I can now run refactor run my tests, refactor, run my tests etc. I can now drive hard at the solution and hopefully glean some more insights.

Time to bring in the full size dictionary.  As expected some very long chains are created 50+.  How can I reduce this ?  What about converging from either direction ?  ie dog -> cat and cat -> dog.  Interestingly coming from a different direction reduces the chain length.  The size seems to be dependent on the ordering of the dictionary.

Idea -> Generate a word chain which may contain 30 words.  Randomly shuffle the words then use this list to form the dictionary for another run and so on.  How many times would I need try.  This would by an interesting experiment - I could converge on a word chain by pure luck :)

• Insight 5:  Backup the chain when the number of words in the chain exceeds some limit

In this way the algorithm will do the pruning for me.  1 line of code !!!
 

if ( stack.size() > maxDepth ) return false;  // prune checkout

Refactor refactor refactor...

I'm going to split the code up so that the algorithm is concentrated in once class to demonstrate the simplicity of the solution.  The recursive algorithm itself is defined in 10 lines of code.

enough.... :)

Here's my solution

https://github.com/coder36/wordchains

public class WordchainImpl implements Wordchain {
    private String [] dict;
    private int depth;

    public WordchainImpl(String[] wordList, int depth) {
        this.dict = wordList;
        this.depth = depth;
    }

    public WordchainImpl(String[] wordList) {
        this(wordList, 10);
    }


    public String [] getChain( String from, String to ) {
        // check that from and to exists
        if ( ! ( dictHas(from) && dictHas(to) ) ) return new String[0];

        for ( int i=0; i < depth; i++ ) {
            Stack stack = new Stack<>();
            recurse( from, to, stack, i );
            if ( stack.size() != 1 ) return stack.toArray( new String[stack.size()] );
        }
        return new String[0];
    }

    private boolean recurse( String from, String to, Stack stack, int maxDepth ) {
        stack.push( from );
        if( stack.size() > maxDepth ) return false;  // prune
        if ( from.equals( to) ) return true;

        for ( String w: dict) {
            if ( w.length() == from.length() && dif( from, w ) == 1 && !stack.contains(w) ) {
                if ( recurse( w, to, stack, maxDepth ) ) return true;
                stack.pop();
            }
        }
        return false;
    }

    private int dif( String a, String b ) {
        int dif = 0;
        for ( int i=0; i< a.length(); i++ ) {
            if ( a.charAt(i) != b.charAt(i) ) dif++;
        }
        return dif;
    }

    private boolean dictHas(String word) {
        for ( String s: dict) {
            if ( s.equals( word ) ) return true;
        }
        return false;
    }
}

Build Instructions

git checkout https://github.com/coder36/wordchains.git
cd wordchains
mvn clean install
java -cp target/wordchain-1.0-SNAPSHOT.jar org.coder36.wordchain.FindMyWordchain

gold
goad
load
lead