How to use the serial interface to communicate with an FRDM-KL25Z in Linux

This short guide will show how to communicate with the FRDM-KL25Z using the Serial and USBSerial classes from the mbed library in Linux.

Serial and USBSerial are two different methods for creating serial connections using mbed. USBSerial creates an emulated serial port over a normal USB port while Serial uses the built in serial port (if there is one). In the case of the FRDM-KL25Z  this can be done through the virtual serial port provided by OpenSDA.

This small example will create two serial connections; one using the serial port built into OpenSDA, and one over the normal USB port. It will pass everything given to either interface on to the other interface.

#include <mbed.h>
#include <USBSerial.h>

Serial debug(USBTX, USBRX);
USBSerial usb;

int main()
{
  while(1) {
    if(usb.readable()) {
      debug.putc(usb.getc());
    }
    if(debug.readable()) {
      usb.putc(debug.getc());
    }
  }
}

Compile the code and load it onto your FRDM-KL25Z, then  connect it to your computer using both USB ports and run the following command.

ls -l /dev/ttyACM*

Your output should look something like this.

crw-rw---- 1 root dialout 166, 0 Aug 19 13:56 /dev/ttyACM0
crw-rw---- 1 root dialout 166, 1 Aug 19 14:19 /dev/ttyACM1

Each of these device files are connected to one of the serial interfaces on the FRDM-KL25Z. To use them you will first have to add your user to the dialout group.

sudo usermod -a -G dialout USERNAME

You can use any terminal emulator of your choice to connect to these serial ports. I will be using minicom for this example.

Open two terminals. On the first run:

minicom -b 9600 -D /dev/ttyACM0

And in the second:

minicom -b 9600 -D /dev/ttyACM1

‘-b 9600’ sets the baud rate. The default value used by mbed is 9600.

If everything is working correctly whatever you write in one terminal should now appear on the other, and vice versa. If it doesn’t create new lines when you press ‘enter’ and it instead jumps to the beginning of the current line, tell minicom to add linefeeds by pressing ‘Ctrl-a a’.

Advertisements

A step-by-step guide to using the FRDM-KL25Z in Linux with GCC and the mbed library

Freescale FRDM-KL25Z
By Viswesr (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)%5D, via Wikimedia Commons
I recently bought a Freescale FRDM-KL25Z to experiment with ARM and embedded systems programming, and I thought I should share my experiences with using it in a completely open source Linux environment so that others may learn from my (many) mistakes. The FRDM-KL25Z is an inexpensive development platform built around an ARM Cortex M0+ based microcontroller.  It has support from several commercial development environments, but there is a lack of documentation for how to make it work on Linux using nothing but open source software. After some research, trial and error, and almost bricking the thing, I finally got everything to work.

This post will show you how to set up a development environment around the FRDM-KL25Z using the GCC toolchain and mbed library. To make it more general I will not use any IDE. This will all be by hand. Once you have it working it should not be a problem

I had to buy a freedom board for a school class, and I was frustrated at first to find that there was no support for the linux platform. I'm personally a fan of the teensy ARM microprocessors which are supported by GCC or the Arduino IDE, now I can add the new micro to my growing collection. Thanks again for the great article!

to set up an IDE around it as well, but that is left as an exercise to the reader.

Getting everything to work will require a few steps.

  1. Install the mbed firmware
  2. Install the GCC ARM toolchain
  3. Set up debugging tools
  4. Download and build the mbed library
  5. Create a simple program

Installing the mbed firmware

There are several firmware variants you can install on your FRDM-KL25Z. You can use whichever version you like as long as it supports the CMSIS-DAP debugging interface (i.e. any firmware other than what came pre-installed). The one I have chosen to use is the mbed firmware. It is easy to use and performs a sanity check on any binaries you give it to make sure that you don’t accidentally brick your device (which you very well might do if you are not careful).

To install the mbed firmware first download the newest version. At the moment  (2014-08-13) that would be this one.

Start the board in bootloader mode by holding down the reset button while plugging in the board using the SDA USB port.https://mbed.org/handbook/CMSIS-DAP

A mass-storage device named BOOTLOADER will appear. If you were using Windows it would be enough to copy the firmware file to this device and restart the board, but since you are using Linux it isn’t quite that simple. You first have to mount it using the ‘msdos’ file system type.

sudo mount -t msdos /dev/sdd /mnt

In my case the board shows up as /dev/sdd. This may, of course, be different in your case. Copy the firmware file to /mnt and restart the board. You should now see a mass-storage device named MBED.

To test that everything is working you can try one of mbed’s example programs. Sign in or create an account and click “Import Program”. This will take you to their online IDE where you can export the program to a ‘.bin’. Copy this file to the MBED  device and press the reset button. If everything is working then the led will probably start blinking (depending on the example you chose).

You could stop here and just use mbed’s online IDE. The rest of this guide will show you how to compile your own programs offline with GCC and the mbed library, and how to get the USB debugging interface to work.

Install the GCC ARM toolchain

The next step is to install the tools needed to compile your own programs. For this you want the gcc-arm-none-eabi from launchpad. Download the one ending with linux.tar.bz2, unless you want to compile it from source.

Just extract it somewhere, add the bin directory to you path variable, and you are done.

Set up debugging tools

With any new firmware the FRDM-KL25Z will support USB debugging using the CMSIS-DAP debugging interface. To use it we need to install OpenOCD and hidapi.

First you must download and install hidapi.

git clone http://github.com/signal11/hidapi.git
cd hidapi
./bootstrap
./configure
make
sudo make install

Next, do the same for OpenOCD.

git clone http://openocd.zylin.com/openocd
cd openocd
./bootstrap
./configure --enable-maintainer-mode --enable-cmsis-dap --enable-hidapi-libusb

Open ‘tcl/target/kl25.cfg’ and add the following text to the end.

adapter_khz 50
$_TARGETNAME configure -event gdb-attach {
    halt
}
make
sudo make install

Permissions

By default only the root user has access to the device files used for debugging. If you want to give a normal user permission to use the debugging tools you will have to set up some udev rules.

Create the file ‘/etc/udev/rules.d/45-mbed_debugger.rules’ and add the following to it.

SUBSYSTEM=="usb", ATTR{idVendor}=="0d28", ATTR{idProduct}=="0204", MODE="0660", GROUP="plugdev"

Similarly, create the file ‘/etc/udev/rules.d/99-hidraw-permissions.rules’ and add the following to it.

KERNEL=="hidraw*", SUBSYSTEM=="hidraw", MODE="0664", GROUP="plugdev"

This will give all users in the ‘plugdev’ group read and write access to all relevant device files. You could use any group you want, or just a specific user.

To test that it is working run the following command as a non-privileged user with the FRDM-KL25Z plugged in.

openocd -c "interface cmsis-dap" -f /usr/local/share/openocd/scripts/target/kl25.cfg

If it complains about not finding the libhidapi-hidraw.so.0 library check that ‘/usr/local/lib’ is in ‘/etc/ld.so.conf’, run ‘ldconfig’, and try again.

If you now open a second terminal you should be able to connect to the device with GDB.

arm-none-eabi-gdb --eval-command "target remote localhost:3333"

Download and build the mbed library

The mbed development platform consists of an online browser based IDE and compiler (as you saw earlier), and an open source library that simplifies the development of embedded systems code by hiding low level details from the programmer. This is part we want. To use the library offline without the IDE you will have to  download and compile the source code.

git clone https://github.com/mbedmicro/mbed.git
cd mbed/workspace_tools

In addition to the main mbed library, the repository contains a number of other useful libraries that can be installed.

–rtos Real Time Operating System
–usb USBDevice
–dsp Digital Signal Processing
–fat SDFileSystem

The following command will build the mbed library and all the optional libraries that are compatible with the FRDM-KL25Z.

python build.py -m KL25Z -t GCC_ARM --rtos --usb --dsp --fat

When it is done you can find the compiled libraries in ‘build’. Move them to where you want the mbed libraries to be located.

Create a simple program

Finally everything that you need to build your own programs is in place, but before you can start programming we will have to pull all of the pieces together and show how to compile a program from source code. For this example we will, of course, create a simple program that turns the three LEDs on and off.

Most of the work needed to build a program for the FRDM-KL35Z has already been taken care of for you by the mbed library. All you must do is link everything together and compile it. You could set this up by hand, but I recommend you start out by using this makefile. It will set up all the paths and include directories for you. What is left is to set the path to the mbed libraries in MBED_PATH, set the program name in TARGET,  add any local source and header folders to SRC_DIRS and INC_DIRS, and to tell it which mbed libraries to use.

Next, create the file ‘src/main.cpp’ and add the following to it.

#include <mbed.h>

Ticker tick;

DigitalOut led1(LED_RED);
DigitalOut led2(LED_GREEN);
DigitalOut led3(LED_BLUE);

void flip() {
    led3 = !led3;
}

int main()
{
    led3 = true;

    //Flip the blue LED every 5 seconds
    tick.attach(&flip, 5.0);

    //Flip the red and green LED once every second
    while (true) {
        led1 = true;
        led2 = false;
        wait (1.0);
        led1 = false;
        led2 = true;
        wait (1.0);
    }
}

Don’t forget to add the ‘src’ folder to SRC_DIRS in the makefile, then compile with ‘make’.

If everything is working as it should it should now create a ‘build’ directory with the file ‘mbed.bin’ in it (assuming you did not change TARGET). Copy this file to the MBED device, wait for it to load it, and then press the reset button. The LED should now start blinking in different colors.

That’s all. You are now ready to start programming your own programs. Build something interesting.

Some helpful tips on how not to lock your FRDM-KL25Z and save yourself much pain

Unless you are careful you will sooner or later manage to “lock” your FRDM-KL25Z. This happens when you accidentally enable the security bits. If you have not set up the mbed library (in particular the *.ld files) correctly then this could happen very easily. The mbed firmware will protect you from this mistake by refusing to flash binaries with the security bits enabled, but there are ways to bypass it using the debugger and OpenOCD.

To check if you have set the security bits open the binary in a hex editor and look at the contents of address 0x0000040C. The last two bits should be “10”. If it isn’t, something is wrong and you should NOT attempt to flash this binary to the board. Seriously, don’t do it!

If you ignored my advice and flashed the board with a binary that had the security bits enabled then you will have to unsecure it. This can be a little complicated, but if you read these instructions and still didn’t follow my advice then it’s your own fault. Supposedly there are tools you can use to unsecure the device, but I have not found any that are free and works in Linux. Fortunately, some variants of the FRDM-KL25Z firmware will unset the security bits when they are installed. You can then reinstall the firmware you want to use. This can require some trial and error, but I have had success with the MSD-DEBUG-FRDM-KL25Z_Pemicro_v114.SDA firmware.

Save yourself a lot of trouble and make sure that you don’t do this in the first place. Unless you know what you are doing only program the device by copying binaries to the MBED mass-storage device.

Sources

http://mbed.org/handbook/Firmware-FRDM-KL25Z
http://karibe.co.ke/2013/08/setting-up-linux-opensource-build-and-debug-tools-for-freescale-freedom-board-frdm-kl25z/
http://embeddedworldweb.blogspot.se/2013/08/mbed-gcc-with-eclipse-kl25z-part-1.html
http://mcuoneclipse.com/2012/11/04/how-not-to-secure-my-microcontroller/

JaspXML uploaded to Github

The first small step in cleaning up my simulator is complete. The result is the JaspXML parser. A simple XML parser/iterator for Java, designed for reading data from large XML files without the memory cost of a DOM parser, or the complexity of a SAX/StAX parser.

It is based on code I wrote for a project during a university course several years back. I needed a way of reading data from XML files, and since I did not know how large these data files would be I used a StAX parser. But StAX parsers are a pain to work with so I hid it behind a more DOM like interface to make it easier to use. It was ugly, but it worked. I later reused and improved the parser during my master’s thesis, when I once again needed a simple way of parsing very large XML files.

JaspXML is based on that final version. I completely rewrote, reorganized, documented, and tested it. What was left is a small and simple XML iterator that can step through an XML document using three commands; next(), down() and up().

next()

Steps forward one step and returns the next element at the current depth of the XML document.

down()

Steps down to the children of the current element.

up()

Steps back up to the parent of the current element.

An example application is provided that uses JaspXML to parse an XML document and prints out the contents of each element.

How to use setxkbmap to rebind Caps Lock

Caps Lock must be the second most useless key on my keyboard (what does Scroll Lock even do?) and one of the first things I usually do after installing a new Linux system is to rebind it to something useful, such as backspace. This is very easy to do (in X11 at least). Just call this script on login.

#!/bin/sh

#Set the keyboard layout to "Swedish" 
setxkbmap -layout se

#Replace Caps Lock with Backspace
setxkbmap -option caps:backspace

#Required to allow key repeat
xmodmap -e "clear Lock"

You can replace Caps Lock with other keys by changing ‘caps:backspace’ to whatever you want.

Replace Caps lock with escape:

setxkbmap -option caps:escape

Replace Caps Lock with Ctrl:

setxkbmap -option ctrl:nocaps

Reset all options and bindings:

setxkbmap -option

How to paste text when copy/paste is disabled

It is sometimes the case that an application will not allow you to paste in text, forcing you to write it out by hand (either because it is poorly written or because of perceived security risks). These instructions are the results of having to manually enter long and complex passwords for virtual Windows machines one time too many, and will show you how to bypass such restrictions using a simple script.

Dependencies

Klipper

xdotool

typeclipboard

#!/bin/sh
 
#This script will type out the content of the Klipper clipboard as if the user had typed it themself.
 
#Copy the content of the KDE clipboard
to_write=$(qdbus org.kde.klipper /klipper getClipboardContents)

#Clear the clipboard. Uncomment this line if you will be copying anything sensitive.
#qdbus org.kde.klipper /klipper clearClipboardHistory

#Wait for one second to give the user time to release any keys that may have been pressed.
sleep 1
 
#Type out the content of the clipboard
xdotool type --clearmodifiers "$to_write"

Common problems

If the output does not match the content of the clipboard it is possible that xdotool is using the wrong keyboard layout. This can be fixed by adding ‘setxkbmap -layout LAYOUT’ to your shell profile, where LAYOUT is the keyboard layout you want (e.g. ‘se’ for Swedish).

Try not to press down any modifier keys (Ctrl, Shift, Alt, etc.) when running the script. With ‘–clearmodifiers’  xdotool will attempt to unset all modifiers, but this will not always work. The script will type out anything you give it as if you had typed it out yourself. This includes things like keyboard shortcuts. Strange and dangerous things can happen if you are not careful. The script will wait for one second before doing anything to give you time to take your hands of the keyboard.

Create a shortcut

To create a keyboard shortcut for the script in KDE4, go to ‘System Settings -> Shortcuts and Gestures -> Custom Shortcuts’, then ‘Edit -> New -> Command/URL’. Under ‘Trigger’, select the keyboard shortcut you want, and under ‘Action’, give it the path to the script.

Pasting passwords to Windows machines running in VirtualBox

This was the reason why I wrote the script in the first place. To paste a password, simply copy it and click on the Windows password field, then press the host key (right Ctrl) so that VirtualBox does not capture the keyboard, followed by the keyboard shortcut you chose. The script will then type out the password as if you had typed it out yourself, bypassing the disabled copy/paste functionality.

How to set up a simple backup system using Duply/Duplicity

This guide will show how to set up a simple backup system using Duply, that creates encrypted incremental backups of client data and stores them both on a central backup server and on a secondary offsite server. All backups are encrypted by the clients using GPG so that no data could be compromised if someone gain access to the backup servers. The guide assume that Debian is used,  but the instructions should work on any Linux distribution with only minor changes.

Duply backup archtecture
Duply backup archtecture

The guide provide step-by-step instructions for setting up a backup server, a secondary server, and a single client. The following names are used to identify them.

  • Client: client.example.com
  • Backup server: server.example.com
  • Offsite server: offsite.example.com

Duply/Duplicity

Duply is a front end to the Duplicity backup application that give you a more user friendly interface and simplifies common tasks. Duplicity isn’t the most powerful backup application available, but it is a very good choice when you don’t need the functionality of, for example, Amanda or Bacula. The strength of Duplicity is its simplicity and ease of use, particularly when combined with Duply. You can, with a few commands and two simple configuration files, create a complete backup schedule with client side encryption.

Another advantage of using Duplicity is that everything is handled by the client. All you need from the server is the ability to store files on it. Other, more advanced, backup solutions typically use some form of backup and scheduling daemon on the server. With Duplicity it is enough if you have remote SSH access to an account on the server; everything else is handled client side. This is particularly useful when backing up desktops and laptops that are not always online.

There are of course disadvantages to using Duplicity. Since Duplicity doesn’t use a central server for managing backups it will not scale well. Each client is responsible for what to back up and when to back it up, and must be configured and managed separately. This makes Duplicity an excellent choice for small networks where there are only a handful of machines to back up, but for larger networks you may want to use something else.

Step 1: Install Duply (client.example.com)

First install Duply and Duplicity on the client.

# apt-get install duply

It is possible to do everything in this guide with only Duplicity. Duply simply provides an interface for Duplicity that is easier to work with. If, for some reason, you don’t want to use Duply you could instead use Duplicity directly.

Step 2: Add a backup folder on the server (server.example.com)

You need an account on the server to store backups on. You could use a single account for backing up several client machines, but I would recommend that you create separate accounts for each client and only give them a restricted shell. This will limit the potential damage that could be done by compromised or misbehaving clients.

Step 2.1: Install RSSH

If you have not done so already, first install RSSH. RSSH provides a restricted shell that only allow a user to perform actions needed for SFTP, SCP and rsync.

# apt-get install rssh

By default RSSH will not allow rsync to be used. We will need this later, so edit ‘/etc/rssh’ and enable rsync access.

# set the log facility.  "LOG_USER" and "user" are equivalent.
logfacility = LOG_USER
 
allowscp
allowsftp
allowrsync
 
# set the default umask
umask = 027

Step 2.2: Create a client user account

Next you need to create a user account and backup folder for the client. How you do this is up to you and your particular needs (and paranoia). I will give two examples of how you could set this up. By simply storing backups in the home directory of the user, and a slightly more complex method that should work better if you are backing up several clients.

Simple solution

A simple solution is to give each client a user account on the server and let each client store their backups in their home directories. This is easy to set up, but if you wish to copy all backups to a secondary offsite server your will have to either do this as root, or give the offsite server access to each client account separately.

When creating client user accounts on the server I recommend disabling password based login so that only key based SSH authentication is allowed, and to only give clients restricted shell access to the server.

# adduser backup-client --disabled-password --shell /usr/bin/rssh

For this example I’m using the ‘~/backup’ folder for storing backups.

# mkdir -p /home/backup-client/backup/
# chown backup-client:backup-client /home/backup-client/backup/

A warning should be made here. If you have more than one user on the server and you are using the default umask, then other users will (probably) have read access to your backups. This should not be a problem if the backups are encrypted, otherwise you may want to at the very least change the umask, or use the advanced solution.

Advanced solution

A somewhat more advanced solution is to give each client a separate user account on the server, but store all backups in a central backup folder and use ACLs to give a single user (in this case backup-duply) read access to all backups. This requires a few more steps, but makes it possible to copy all backups to a secondary server using only the one backup account.

First create the backup account.

# adduser backup-duply --disabled-password --shell /usr/bin/rssh

Create a backup folder.

# mkdir /var/backup

Use ACLs to give backup-duply read access to the backup folder and all subfolders. This will be inherited by new folders and backups when they are created.

# setfacl -R -d -m user:backup-duply:r-X /var/backup
# setfacl -R -m user:backup-duply:r-X /var/backups

Optionally, use ACLs to override the default umask and set more restrictive read permissions.

# setfacl  -R -d -m other::--- /var/backup
# setfacl  -R -m other::--- /var/backup/*

Create a user account for the client.

# adduser backup-client --disabled-password --shell /usr/bin/rssh

Then create a folder in the backup directory where the client can store backups.

# mkdir -p /var/backup/backup-client/
# chown backup-client:backup-client /var/backup/backup-client/

Step 3: Set up SSH access

You will need to give the client SSH access to the new user account on the server.

If the client does not already have an SSH key, you must first create one.

$ ssh-keygen

Save it in the default location (~/.ssh/id_rsa). You could also create a separate set of keys for Duply, but that will not be covered by this guide.  Since the key will be used non-interactively, you should give it an empty passphrase. This will create two keys. The private key id_rsa, and the public key id_rsa.pub. You must always keep the private key secret!

The next step is to add the public key to the list of authorized keys on the client’s user account on the server.

On the server, create the file ‘/home/backup-client/.ssh/authorized_keys’ if it does not already exist.

# mkdir /home/backup-client/.ssh/
# touch /home/backup-client/.ssh/authorized_keys
# chown -R backup-client:backup-client /home/backup-client/.ssh/

Next, copy the content of the client’s public key (id_rsa.pub) to the authorized_keys file on the server.

Finally, you must add the server’s public key (/etc/ssh/ssh_host_rsa_key.pub) to the client’s list of known hosts (~/.ssh/known_hosts). You can do this manually or, alternatively, if you try to connect to the server it will offer to do this for you. Adding the key manually is more secure and is the recommended way to do this if you do not trust the network in-between the client and the server. Until the keys are in place and the server and client can authenticate each other it is possible to perform a man-in-the-middle attack.

To test that everything is working, try to connect to the server.

$ ssh backup-client@backup.example.com

It should give you the following message.

This account is restricted by rssh.
Allowed commands: scp sftp rsync

Step 4: Create GPG keys (client.example.com)

If you want to encrypt or sign your backups you will need a GPG key. If you do not already have a key the next step will be to create one.

To create a GPG key use the command:

$ gpg --gen-key

Then choose:

(1) RSA and RSA (default)

You can use the defaults for the rest if you wish, but choose a strong passphrase. I recommend you use some form of password generator to create it. Make a copy of the passphrase and store it somewhere safe. You should also create a backup of ‘~/.gnupg/’. You will not be able to decrypt your backups if you lose them.

In the end you should get something like:

$ gpg --list-keys
/root/.gnupg/pubring.gpg
------------------------
pub   2048R/C8B4OI9S 2014-05-16
uid                  Your Name <Yourname@example.com>
sub   2048R/JF5PWSHP 2014-05-16

Here, C8B4OI9S is the name of the main signing key, and we have one subkey JF5PWSHP for encryption. These keys are enough to create encrypted and signed backups.

Step 5: Create a backup profile (client.example.com)

It is now time to set up a Duply backup profile. The following command will create a new profile named “backup_all”.

$ duply backup_all create

This will create a folder ‘~/.duply/backup_all/’, where all relevant configuration files are stored. The backup schedule and other options are listed in ‘~/.duply/backup_all/conf’. The following is a sample configuration file using the GPG keys created earlier.

#GPG encryption key
GPG_KEYS_ENC='JF5PWSHP'
 
#GPG signing key
GPG_KEY_SIGN='C8B4OI9S'
 
#GPG passprase
GPG_PW='YOUR_PASSPHRASE'
GPG_PW_SIGN='YOUR_PASSPHRASE'
 
#Compress backups using bzip
GPG_OPTS="--compress-algo=bzip2 --bzip2-compress-level=9"
 
#Send backups to server.example.com using sftp over SSH. Uncomment the one matching your setup.
 
#Simple solution. Store backups in ~/backup
#TARGET='sftp://backup-client@server.example.com/backup/'
 
#Advanced solution. Store backups in /var/backup/
#TARGET='sftp://backup-client@server.example.com//var/backup/backup-client/'
 
# base directory to back up
SOURCE='/'
 
# Time frame for old backups to keep, Used for the "purge" command.
# see duplicity man page, chapter TIME_FORMATS)
# defaults to 1M, if not set
MAX_AGE=6M
 
# Number of full backups to keep. Used for the "purge-full" command.
# See duplicity man page, action "remove-all-but-n-full".
# defaults to 1, if not set
MAX_FULL_BACKUPS=3
 
# activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3)
# forces a full backup if last full backup reaches a specified age, for the
# format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
# Uncomment the following two lines to enable this setting.
MAX_FULLBKP_AGE=1M
DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
 
# verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
# default is 4, if not set
VERBOSITY=5
 
# temporary file space. at least the size of the biggest file in backup
# for a successful restoration process. (default is '/tmp', if not set)
TEMP_DIR=/tmp
 
# more duplicity command line options can be added in the following way
# don't forget to leave a separating space char at the end
DUPL_PARAMS="$DUPL_PARAMS --extra-clean --num-retries 1"

Next, you will have to create an exclude file ‘~/.duply/backup_all/exclude’. This file tells Duplicity which files to back up, and which to ignore.

The following is a sample exclude file that will back up all files in ‘/etc’, ‘/home’ and ‘/var/www’, except for temporary files.

- **/*~
+ /etc
+ /home
+ /var/www
- **

The format is very simple. Duplicity will check each file against the rules in this file, starting from the top, until it finds a rule that matches. If the rule is preceded by a ‘+’ the file will be backed up, and if it is preceded by a ‘-‘ it will be ignored. For example, the file ‘/etc/resolv.conf’ will match the rule ‘+ /etc’ and be backed up, while ‘/etc/resolv.conf~’ will match the rule ‘- **/*~’ and be ignored. The last rule ‘- **’ tells duplicity to ignore all files that didn’t match an earlier rule.

To test that everything is working, run:

$ duply backup_all backup

If this finishes without errors then all that is left to do is to create a cron job. You should also make a backup of ‘~/.duply/’. You will need these configurations files when restoring backups.

Run ‘crontab -e’ and add the line:

@daily /usr/bin/duply backup_all backup_cleanup_purge --force > /dev/null

Step 6: Copy backups to an offsite server (offsite.example.com)

You should always have at least one additional copy of all backups stored offsite. A simple way to this is to use rsync to copy all backup data to a secondary server. This can be done in several ways depending on which solution you chose in step 2, and on your own requirements. I will give two examples where the offsite server connects to and pulls the backup data from the primary server. You can also do the opposite and let the primary server connect to the offsite server and push data to it.

Simple solution

If you followed the earlier steps for the simple solution you need to give the offsite server SSH access to each client account on the primary server. Assuming that the previous steps have been taken to give the client SSH access to the server, all you need do is copy the contents of ‘~/.ssh/id_rsa.pub’ from the offsite server to the authorized_keys files in each client account on the primary server.

Next, add the following cron job to the offsite server for each client account.

@daily /usr/bin/rsync -a --delete backup-client@server.example.com:/home/backup-client/backup/ /your/backup/folder/backup-client/

Before adding this line to crontab you should try the command with ‘–dry-run -v’ enabled to see that it does what you intended. You may also want to leave out ‘–delete’ if you don’t want old backups to be removed from the offsite server once they have been deleted from the primary server.

This solution is quite simple to implement when you only have one or two machines backed up to separate user accounts on the server, but it quickly becomes impractical since you will have to remember to update crontab each time you add or remove a client. This could also create a potential security risk since the offsite server will have write permissions for the backups on the primary server. If the offsite server were to be compromised it could potentially delete backups from the primary server.

Advanced solution

If you followed the earlier steps for the advanced solution you need to give the offsite server SSH access to the backup-duply account. This is done in the same way as in the simple solution described above.

Next, add the following cron job to the offsite server.

@daily /usr/bin/rsync -a --delete backup-duply@server.example.com:/var/backup/ /your/backup/folder/

Before adding this line to crontab you should try the command with ‘–dry-run -v’ enabled to see that it does what you intended. You may also want to leave out ‘–delete’ if you don’t want old backups to be removed from the offsite server once they have been deleted from the primary server.

This solution will create a copy of all backups for all clients with a single command. It also only gives the offsite server read permission for the backups on the primary server. Even if the offsite server were to be compromised the backups on the primary server would be safe.

The first post

With this post I suppose that my new website/blog has been officially started. I have not yet decided if I’m actually going to use it as a blog or not. This thing is running WordPress after all, and blogging is what it was made for, but for now I will use it as a convenient place to put any interesting code or projects I make.

I have already uploaded the latest version of the P2P/reputation simulator I wrote for my master’s thesis. I may upload other things as well as soon as I take the time to go through my old projects and clean them up a bit.