Ubuntu Diskless Workstation and X-Terminal Howto – Part 3 X-Terminal Setup

At this point in the series we should have a working diskless workstation. Basically we have a full desktop running from an NFS share. This article will explore how to use the server to deliver a desktop, instead of our old laptop. In my case my server is much more powerful in terms of CPU and RAM than an old laptop, so the best linux experience will come from running applications on the server with the output showing on the laptop, which is the definition of an X-Terminal.

There are two steps to doing this – preparing the server and preparing the client.

There is also some tuning options that we can do with the terminal to make sure we get the best performance from the diskless that we can.

So, lets dive into the server preparation. Ubuntu 9.10 and up have a newer GDM that doesn’t have the gdmsetup configuration program available any more. So, we need to edit a file to turn on something called XDMCP. Our server is running Ubuntu 10.10, and officially XDMCP is not supported on this version, however it can be made to work. XDMCP is a method to allow an X server (in our case the diskless) to pull up a logon window from the server.

Basically a single file needs to be edited to allow xdmcp support. That file is /etc/gdm/custom.conf. A section called xdmcp is added to this file to enable the feature:


Once that section is added you can restart gdm with the following command:

sudo /etc/init.d/gdm restart

You can verify that the server is listening on UDP 177 with the following command:

netstat -uan

and look for the following entry in the command output:

udp 0 0*

One thing to note, this may not work properly on Ubuntu 10.10 if you have IPv6 enabled. You can disable IPv6 by adding ipv6.disable=1 to the kernel command line. This can be done by editing /etc/default/grub and modifying the line beginning with GRUB_CMDLINE_LINUX_DEFAULT=", adding the parameter to look something like this:

GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1 quiet splash"

Then run the following command on the server:

sudo update-grub2

Then reboot the server to disable IPv6. Unfortunately this will remove IPv6 support from the system, so if your relying on IPv6 – beware.

Ok – now lets move on the to the diskless. Lets make sure its booted up and showing the logon screen for Ubuntu. When the screen is up, drop to a command line by hitting Ctrl-Alt-F1. Logon to the system from the command line (hopefully you created a userid on your diskless when you built it).

Now lets bring down the gdm service so we can start X manually:

sudo /etc/init.d/gdm stop

Once gdm is stopped, we can try to use XDMCP to pull up the logon window from the server:

X -query "server IP address or hostname"

This should bring up a logon window from the server. If it does, then we can move on to the next step – if it doesn’t verify the server is configured properly as mentioned above. You can check the kernel command line on the server by running:

cat /proc/cmdline

You can try to login to the server if the window comes up. To continue, you will probably want to drop to the console again by hitting ctrl-alt-f2 and killing X with killall X. This should kill the X server.

In my normal setup I set Ubuntu to boot into console mode without GDM. This allows me to log on, and run a script that starts the X server and pulls the logon window from the server.

To set the diskless to boot to a text console, we need to modify the boot file used to PXE boot the diskless. In a previous article we showed you how to set up a PXE boot file for the diskless. Let’s pull that file up again and make the change we need. Basically the simple addition of a the kernel parameter “text” is needed. For example:

APPEND root=/dev/nfs rw initrd=initrd.img-2.6.35-24-generic.nfs nfsroot= ip=x.x.x.x:y.y.y.y:g.g.g.g: text acpi=force

The parameter is the second to the last one (before acpi=force). Adding this to your PXE boot file (or to a grub command line) will make ubuntu default to non-graphical boot. The next thing we need is a script to start X and use xdmcp to pull up the logon window from the server. Here is an example:

X -dpi 100 -ac -query

There are a couple of extra parameters in this script than what we used before. The first one forces the DPI to be 100, which will keep the desktop font size looking normal, and the -ac, which disables access control. Disabling access control still allows local X client programs such as a video player to run on the same desktop that is being pulled from the server. Running multimedia or OpenGL based software over the X protocol across the network is a less than optimal experience, so those should be run locally. More on that later. Lets save the command above in a script on the diskless called startterm. Make sure you make it executable and save it somewhere in your path.

The process will be to boot the diskless, logon as your user id, and run the startterm script, and you will be able to logon to your server.

Improving the Experience

We should have a diskless workstation set up as an X-Terminal. You should be able to use anything on the server you want – minus multimedia/sound apps. They will work but the video in mplayer, for example, will be less than adequate and there won’t be any sound unless you are near the server – because sound will still come from the server’s sound system!

Network sound is actually pretty easy to setup, but it requires some more tweaking on the diskless workstation – Enter Pulse audio system mode. Pulseaudio is the default sound server used on Ubuntu. Its actually pretty powerful, once you understand how to use it. Normally Pulseaudio is configured in per user mode, and is started when a user logs into gnome. Since we aren’t starting gnome on the diskless workstation (its running on the server) we need to reconfigure Pulseaudio to work in system mode, so its always running. Pulseaudio system mode is actually frowned upon in most scenarios, but this is one use case where it makes sense.

First we need to edit some configuration files to enable Pulseaudio system mode. They are located in the /etc/pulse directory. The first one is daemon.conf – edit it and look for the section that looks like this (its near the beginning):

; daemonize = no
; fail = yes
; allow-module-loading = yes
; allow-exit = yes
; use-pid-file = yes
; system-instance = no
; enable-shm = yes
; shm-size-bytes = 0 # setting this 0 will use the system-default, usually 64 MiB

And modify it to look like this:

daemonize = yes
; fail = yes
allow-module-loading = no
allow-exit = no
; use-pid-file = yes
system-instance = yes
enable-shm = no
; shm-size-bytes = 0 # setting this 0 will use the system-default, usually 64 MiB

The last file is /etc/pulse/system.pa – edit the file and add this to the bottom of the file:

### Enable TCP access to local sound devices
load-module module-native-protocol-tcp auth-anonymous=1
load-module module-zeroconf-publish

After these changes, you should reboot the diskless workstation to enable Pulseaudio in system mode, or you can run the following command to restart it:

sudo /etc/init.d/pulseaudio restart

If you restart the diskless, log on, and use the startterm script to log back into the server, you should here the ubuntu logon sound coming from your diskless speakers. If you don’t here are some things to check:

1. Log on locally with sudo /etc/init.d/gdm start instead of running startterm and make sure your audio system works.
2. Install padevchooser on the server, with sudo apt-get install padevchooser and run it from the diskless while logged into the server. It will place an icon in the menu bar. Right clicking on it should allow you select the sink that reflects your diskless – it will look like something pulse@

In our configuration we have enabled zeroconf so if the diskless is on the same subnet, it should just work out of the box, so to speak.

One more thing – you may need to add your diskless user that you logon to the system with to the pulse and pulse-access groups. This is important when running local applications that use Pulseaudio (see next section for details). Our user is term1 and this is how to add term1 to the groups needed:

usermod -a -G pulse,pulse-access term1

Local Applications

As I mentioned above, local applications can be handy for things that don’t run well over the X protocol across the network. Video is one of those applications. Luckily, its possible to create icons on the servers gui interface (maybe as part of the gnome ui) that actually start and run applications on the diskless workstation itself. The process is pretty simple. Essentially its an ssh session from the server to the diskless that starts an application with the display redirected to the diskless. Remember the -ac parameter in the startterm script? This is why its needed. -ac disables access control with allows this type of functionality to work properly.

Here is an example of a script that starts mythtv on the diskless:

/usr/bin/ssh term1@diskless1 /home/term1/bin/mythlocal

This script resides on the server – it starts and ssh to the diskless and then calls another script on the diskless call mythlocal. Here is what mythlocal looks like:

DISPLAY=localhost:0.0 mythfrontend -l /home/term1/mythfrontend.log -v important,general

Basically all this does is set the DISPLAY environment variable to the local X console on the diskless, and then start mythfrontend with options to log myth events to a log file. After a few seconds myth will start on the diskless and use the diskless video hardware to display Mythtv.

Since we are using Pulseaudio on the diskless it’s important to enable Pulseaudio output on Mythtv as well.

There is a caveat to making this work – the diskless must be configured to use key based ssh authentication, instead of password based. If you don’t do this, you won’t see the password prompt generated by the ssh session, and therefore Myth will never get started on the diskless.

Setting up ssh key based authenication is beyond the scope of this article – click here for a good place to start with this. I will attempt to get a future article together on this at a later date.

Tuning Performance

NFS isn’t nearly as efficient as local disk, so the less we write to the nfs based root file system, the better. One thing we can do is sent system log traffic to another form of storage. In our case, we will use tmpfs, which is temporary storage in the diskless computer’s memory. This means that system log files will not survive a reboot (unless you customize the system to dump them to the nfs root fs during reboot).

Essentially we are just going to redirect the /var/log /tmp and /var/tmp directories to tmpfs. This is done by adding the following to the /etc/fstab on the diskless:

tmpfs /var/log tmpfs defaults 0 0
tmpfs /tmp tmpfs defaults 0 0
tmpfs /var/tmp tmpfs defaults 0 0

After rebooting the diskless, all the system logger traffic will go to tmpfs and it should allow the diskless to be a little more responsive.

Well, I think that should about cover it. Comments and other suggestions are welcome as always.

Next up – Ubuntu vs Gentoo vs Arch Linux – I started out with RedHat 4 back in 1996, and moved to Gentoo in 2004, then to Ubuntu in 2008 – I have recently been playing around with Gentoo (again) and Arch Linux, and I may have found my next “favorite” linux distro – Arch Linux.

I also have tested debootstrap, which is a way of building a diskless filesystem on the server without installing ubuntu on the diskless and copying it across. I will see if I can get a quick article out on that process as well. Stay tuned!

Ubuntu Diskless Workstation and X-Terminal Howto – Part 2 – Preparing the diskless image

Well, after a short break, we are back to our diskless X-Terminal Howto. Today we will cover building the actual diskless image that will be used to boot the X-Terminal. Using ubuntu, the easiest way to do this is to simply install ubuntu on the machine that will be used for the diskless terminal. You can also use a virtual machine to build the image, however this will probably have some interesting issues regarding udev and device detection.

So, basically we be installing ubuntu on the machine itself, and use that install to make an nfs bootable image. So, go ahead and download an ubuntu install cd and have at it. A base install is usually sufficient with working X-Server, network devices etc. You don’t need to install a bunch of applications since those will be served by the server the X-Terminal will attach to. You should make sure you have all the latest updates applied to your install. I also recommend installing the nfs-kernel-server package as well with the command:

apt-get install nfs-kernel-server

Once you have ubuntu installed on your target machine, there are a couple of things we need to do to prepare for making the image NFS bootable.

The first thing we are going to do is modify the initial ram filesystem image to be better suited for network booting. To do this, open a terminal and change to the following directory :

cd /etc/initramfs-tools

We will edit the initramfs.conf file in this directory:

sudo pico initramfs.conf

There are 3 lines in this file that we will want to change. They are shown below:


Change these lines to read:


Diskless workstations typically require standard wired ethernet connections to work properly. There are some alternatives for wireless, but those are for another article.

After making the changes to this file we will need to regenerate the initial ram disk image for the current kernel. This image will be loaded instead of the standard ubuntu initial ramdisk to allow network booting. Here is the command for creating a new image:

sudo mkinitramfs -o /boot/initrd.img-2.6.35-22-generic.nfs 2.6.35-22-generic

Note the .nfs on the end of the filename (initrd.img-2.6.35-22-generic.nfs) make sure this filename is based on the current kernel version, that can be determined with the command uname -a. The last part of the command is the kernel version to make the initial ramdisk image for.

Here is a sample from a box running ubuntu 10.10:

default@ubuntu-1010:~$ uname -a
Linux ubuntu-1010 2.6.35-22-generic #35-Ubuntu SMP Sat Oct 16 20:36:48 UTC 2010 i686 GNU/Linux
default@ubuntu-1010:~$ sudo mkinitramfs -o /boot/initrd.img-2.6.35-22-generic.nfs 2.6.35-22-generic

The next couple of modifications will have to wait until we have copied the image to the server. So, lets do that next. The best way to copy the image to the server is to mount the servers shared directory that we created in the last article.

The directory was /srv/export/netboot/diskless1. So, lets mount that directory on your workstation. First, lets create a mount point:

sudo mkdir -p /mnt/server

Then we can mount the directory:

sudo mount -t nfs :/srv/netboot/diskless1 /mnt/server

After mounting the directory change to the root of the workstation’s root filesystem and copy the filesystem to the server:

cd /
sudo cp -Rax * /mnt/server

The cp command switches mean copy recursively – ie include all subdirectories (-R) and preserve permissions (a) and limit the copy to one filesystem (x).

After the root file system image is copied from the workstation to the server, we will have to configure the pxelinux command/boot file for the diskless image, as well as move the initrd and kernel image files to the server’s startup_files directory we created in the last article.

First, we will configure the pxelinux command/boot file for the diskless image. This file is a small text file that tells PXE the root file system path, and kernel parameters used for booting the diskless workstation. This file also contains IP autoconfiguration parameters that tell the linux kernel how to configure its network interfaces. Generally, this is handled through dhcp. In ubuntu 10.04 and 10.10 it seemd that in order to boot the diskless reliably, PXE will boot and get the dhcp reservation we created in the first article. Afterwards, the kernel will use hard coded IP address parameters in the kernel command line that match the dhcp reservation the was setup for the workstation. This may not make sense now, but it will soon.

This command/boot file must be saved in a particular location and with a particular name in order for PXE to find it. Essentially, the file needs to be saved in the following directory (if you have been following my server setup examples):


The name of the file is the hexadecimal equivalent of the workstations IP address. So, if you used as the IP address reservation on the dhcp server, then the file name will be called:


This is equivalent to 192(C0).168(A8).0(00).127(7F). You can use the Ubuntu calculator to determine the hex equivalent of your diskless workstations ip.

For our example, the workstation IP will be, the IP of the NFS server will be, and the path to the diskless root file system will be: /srv/netboot/diskless1.

So, lets look at a sample boot file for the diskless:

DEFAULT /vmlinuz-2.6.35-22-generic
APPEND root=/dev/nfs rw initrd=initrd.img-2.6.35-22-generic.nfs nfsroot= ip= acpi=force

The first line specifies the kernel image to use. This is tftp’d from the server’s startup_files directory by the diskless workstation at boot. The second line specifies the kernel command line. The root= parameter specifies that the root filesystem will be NFS mounted. The rw means the root filesystem will be read/write. The initrd= parameter specifies the initial ramdisk image we created earlier in this article. The nfsroot parameter specifies the nfs servers path to the root filesystem. The last two parameters, ip= specifies the IP address to use on the LAN interface when the kernel starts. This in the past used to be ip=dhcp. For some reason, in my environment this seems to have stopped working with the latest ubuntu 10.04 kernel and 10.10 kernels. The ip= parameter specifies the IP address of the workstation, the ip of the nfs server, the default gateway, and the network mask, respectively. The :::none are for other unused parameters. Lastly, the acpi=force is for systems where acpi is detected properly. Many older older machines need this parameter to function properly.

Make sure your file includes the right parameter for the ip=, kernel image name, and initrd name. For reference, the kernel image is located in /boot on the workstations image, and the initial ram disk image is what we created earlier in this article.

I will update the article if I can figure out what caused ip=dhcp to stop working properly.

Anyway, we have two more changes to make on the disk image to get a successful boot. The first one is to tell the diskless workstation not to run the normal network configuration. Since our network interface will already be up as part of the PXE process, if the Ubuntu NetworkManager attempts to configure the network stack, we will loose connection to the root filesystem on NFS, causing the kernel init process to fail. So, basically we will tell NetworkManager not to touch the LAN interface by modifying the /etc/network/interfaces file.

If we are using eth0 as our network connection on the diskless terminal, then use the following interfaces file:

auto lo
iface lo inet loopback

iface eth0 inet manual

The last thing we will have to modify is the /etc/fstab file. This file is responsible for telling the Linux kernel which devices to use for the needed file systems on the system. In our case we will simply modify the entry for the root file system.

Normally the entry for the root file system looks something like this:

# / was on /dev/sda1 during installation
UUID=50e370fd-eaf8-486a-adf0-ddb5fe1fb9be /               ext4    errors=remount-ro 0       1

We will change this entry to look like this:

/dev/nfs        /               nfs     defaults    	1       1

This basically tells the kernel that the root file system is nfs mounted.

NOTE: If you intend to leave a hard drive in the diskless workstation, the install of Ubuntu will have set up a swap partition. If you have a machine with low memory, its recommend that you leave the drive in and let Ubuntu detect the swap partition as it boots from the network. If you have 1GB of RAM or more, you will probably be safe removing the hard disk altogether.

The next step is to attempt to boot the diskless workstation from the network. Essentially this usually requires telling the PC’s BIOS to boot from a network device as the first device to attempt.

Usually this is in the BIOS setup. It will vary by PC. Usually this is referred to as Network boot or setting the NIC as the first startup device in your PC’s boot order.

So if you are ready, set your PC to boot from the network, make sure its connected and reboot. If all goes well you will see a screen that looks like this:

If something doesn’t work right, check over your configuration or send me a comment and I try to help. Otherwise your PC should come up into an Ubuntu Desktop logon screen as if it was running from an internal disk drive.

Next article is using the workstation as an X-Terminal for a server (rather than a standalone desktop) and some tuning for performance.

Until next time…

Ubuntu Diskless Workstation and X-Terminal Howto – Part 1 – Preparing the Server

Why a diskless workstation? In today’s day and age, this is a good question. My take on it is that I have a multiuser operating system (Linux) running on a large machine (Dual Xeon 5160 with 12 GB RAM) that could be used from several locations in the house by deploying X-Terminals. Rather than spending money on X-Terminals, why not use an old laptop or desktop machine to access the servers gui, applications and resources? That is my reasoning, and I also add some local application support for things not suitable for the X protocol over the network, such as Video, so while the machine is diskless, it isn’t just a dumb display device.

Preparing the Server

Lets get started with preparing the server side of the diskless environment.  For our purposes we will be using a traditional NFS based environment for running the diskless workstation.  This process should work on Ubuntu Lucid or Karmic Server or Desktop Installations.

Several things need to be installed on the server:

  • tftp server
  • dhcp server
  • nfs server
  • pxelinux.0 and startup files

Install the tftp sever, dhcp server and nfs server with the following command:

sudo apt-get install atftpd && sudo apt-get install dhcp3-server && sudo apt-get install nfs-kernel-server

You will also need to download the pxelinux.0 file.  You can get that file here:


For our example, we will need to create a directory to house our diskless file systems, and pxelinux startup files.  We will use /srv/netboot.  Create this directory on your server with the following command:

sudo mkdir /srv/netboot

In this directory, lets create the following subdirectories:

sudo mkdir /srv/netboot/diskless1

“diskless1” is where the root filesystem of our diskless machine will live.

sudo mkdir /srv/netboot/startup_files

“startup_files” is where kernel images, pxe and initrd images will reside for booting diskless workstations – this is similar to the /boot directory on a linux system. The workstation will access these files via tftp.

Lets share the needed directories via NFS.  To do this, we need to edit the exports file on the server and add the following line:

/srv/netboot/diskless1    *(rw,sync,no_root_squash,no_subtree_check)

Then, tell the nfs server to refresh its export list with exportfs command:

sudo exportfs -avr

Next we will configure the tftp server, atftpd. The configuration file for atftpd is /etc/default/atftpd. We will customize ours to look like this:

#OPTIONS="--tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr --mcast-ttl 1 --maxthread 100 --verbose=5 /var/lib/tftpboot"
OPTIONS="--daemon --port 69 --tftpd-timeout 300 --retry-timeout 5     --mcast-port 1758 --mcast-addr --mcast-ttl 1 --maxthread 100 --verbose=5  /srv/netboot/startup_files"

The first two lines of the file are the default entries – we are going to comment them out and add the second two lines as shown above. After editing the file, restart the tftpd service with the command:

/etc/init.d/atftp restart

Next, we need to configure the DHCP server. Some custom options are required for the DHCP server to tell the diskless workstation how to retrieve its boot files. In our example, we will use a network, so the DHCP server configuration will look something like this (this also assumes that this is the only DHCP server for the network having more than one is bad news unless configured properly) assuming your DNS servers are and and your default router (route to the internet) is

# The ddns-updates-style parameter controls whether or not the server will
# attempt to do a DNS update when a lease is confirmed. We default to the
# behavior of the version 2 packages ('none', since DHCP v2 didn't
# have support for DDNS.)
ddns-update-style none;

# option definitions common to all supported networks...
option domain-name "rtwsecurenet.com";
option domain-name-servers,;
option ntp-servers;

default-lease-time 172800;
max-lease-time 259200;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.

# This is a very basic subnet declaration.
subnet netmask {
  option routers;

host diskless1  {
      hardware ethernet     00:04:76:3F:A6:36;
      next-server ;
      option root-path      "/srv/netboot/diskless1";
      filename              "/pxelinux.0";

In the example above I created a reservation for the diskless workstation. DHCP reservations are based on mac addresses. The hardware ethernet parameter above should be editted to reflect the diskless workstation’s ethernet card mac address. The fixed-address specifies the IP address that will be assigned to the diskless workstation. The last 3 parameters, called DHCP options, give booting information to the diskless workstation. These should be updated to reflect the appropriate server IP (next-server) and path (option root-path). Substitute your server’s IP address and path for your installation. The DHCP server that ships with Ubuntu linux is the only one that I am aware of that supports per reservation DHCP options such as these. Be sure to also change the DNS server settings the in the dhcp.conf file to the servers for your site/network.

The last part of preparing the server is the configuration of pxelinux and the diskless boot files.

Configuring PXE

The first thing we need to do is take the pxelinux.0 file you downloaded above and copy it to the /srv/netboot/startup_files directory we created above.  Then, in the /srv/netboot/startup_files directory, make the following directory (as root or using sudo):

mkdir pxelinux.cfg

The pxelinux.cfg directory will contain the pxelinux boot configuration file for each diskless workstation.  We will configure these files after we build our diskless workstation image.

This should about wrap it up for the server configuration piece of the diskless workstation setup.  The next part will cover the building of a diskless workstation image from a standard ubuntu-desktop installation.

Stay tuned!

Ubuntu Apache LAMP Server Quick Howto – Part 2 – SSL Websites

Hi – Welcome back to the Ubuntu Apache LAMP Server Quick Howto.  This second part of the series will cover configuring SSL for use with Apache for securing website content.  SSL stands for Secure Sockets Layer, and is a method of providing transport layer encryption or TLS (transport layer security).  SSL typically uses X.509 digital certificates to perform authentication and encryption functions.   X.509 is based on public key infrastructure or PKI.  PKI is beyond the scope of this article (sounds like a good topic for a future article), but we will at least create a self-signed certificate for instructional purposes here.

SSL adds a few caveats to Apache configuration.  One of those is name based virtual hosting.  SSL does not allow the use of name based virtual hosts.  From the apache SSL/TLS Strong Encryption Guide:

“Why is it not possible to use Name-Based Virtual Hosting to identify different SSL virtual hosts?

Name-Based Virtual Hosting is a very popular method of identifying different virtual hosts. It allows you to use the same IP address and the same port number for many different sites. When people move on to SSL, it seems natural to assume that the same method can be used to have lots of different SSL virtual hosts on the same server.

It comes as rather a shock to learn that it is impossible.

The reason is that the SSL protocol is a separate layer which encapsulates the HTTP protocol. So the SSL session is a separate transaction, that takes place before the HTTP session has begun. The server receives an SSL request on IP address X and port Y (usually 443). Since the SSL request does not contain any Host: field, the server has no way to decide which SSL virtual host to use. Usually, it will just use the first one it finds, which matches the port and IP address specified”

My experience is that Apache will reject the configuration if you attempt to use named based virtual hosts with SSL.

So, with this in mind, if I have multiple “sites” to serve with SSL, I either use different IP addresses for each site, or I serve one site with multiple directories of content that can be referenced by http://<site fqdn>/directory.  For example, if I had a site called www.stuff.com and it used SSL, I would add content for different functions/site at https://www.stuff.com/systeminfo or http://www.stuff.com/shopping.

Anyway, so, how do we set up SSL?  First thing is to make sure its enabled on your LAMP server.  To turn on SSL use the following command:

sudo a2enmod ssl

followed by

sudo /etc/init.d/apache2 restart

Also – we should create a specific document root directory for the SSL Apache to use.  This way, if you have both SSL and non-SSL sites on the same server, not all the non-ssl sites will be published automatically by the SSL definition.

sudo mkdir /var/www-ssl

Creating a self signed certificate:

In order to use SSL with Apache you must have a certificate.  Here is how to make a self signed certificate.  Note:  Self Signed certs are fine for testing and lab servers, but are not recommended for production environments or internet facing servers.

First, you will need to create a private key file:

sudo openssl genrsa 1024 > host.key
sudo chmod 400 host.key

Next, you will need to create the certificate:

sudo openssl req -new -x509 -nodes -sha1 -days 365 -key host.key > host.cert

After hitting enter, you will be prompted for several pieces of information that will be encoded into the certificate.  Here is an example output:

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Kansas
Locality Name (eg, city) []:Topeka
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Test.com
Organizational Unit Name (eg, section) []:IT
Common Name (eg, YOUR name) []:www.test.com
Email Address []:admin@test.com

Note, the Common Name should be the fully qualified domain name of your web server.  Your cert and key will be in the current directory, and will be called host.cert (cert) and host.key (key).  Both of these files will be needed for the next step.  As root or using sudo, copy the key file to the /etc/ssl/private directory on your LAMP server, and copy the cert file (host.cert) to the /etc/ssl/certs directory.

Next go to the /etc/apache2/sites-available and edit the default-ssl file.  This file controls the default SSL virtual host configuration.  We will be modifying the following some the lines in the file.  As root, find the beginning stanza in the file (with your favorite editor) and it will look something like this:

<IfModule mod_ssl.c>
<VirtualHost _default_:443>
            ServerAdmin webmaster@localhost

            DocumentRoot /var/www
            <Directory />
                         Options FollowSymLinks
                         AllowOverride None
            <Directory /var/www>
                        Options Indexes FollowSymLinks MultiViews
                        AllowOverride None
                        Order allow,deny
                        allow from all

Change the webmaster@localhost to an email address of the person responsible for the server (this isn’t required, but is probably good practice).

Also, you should probably add a ServerName directive right after the ServerAdmin directive.  The ServerName directive should reflect the fully qualified domain name as the server.

The DocumentRoot directive should be changed to /var/www-ssl – the directory we made earlier.  Lastly, the Directory /var/www directive should be changed to Directory /var/www-ssl.

Here is the same code snip, modified for a server called www.test.com:

<IfModule mod_ssl.c>
<VirtualHost _default_:443>
           ServerAdmin admin@test.com
           ServerName www.test.com
           DocumentRoot /var/www-ssl
           <Directory />
                  Options FollowSymLinks
                  AllowOverride None
           <Directory /var/www-ssl>
                  Options Indexes FollowSymLinks MultiViews
                  AllowOverride None
                  Order allow,deny
                  allow from all

Next, lets find the following couple of lines:

SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

These lines tell Apache where to find the SSL certificate to use for the site.  We are going to replace these with the certificate and private key we created earlier.  We will change them to read:

SSLCertificateFile    /etc/ssl/certs/host.cert
SSLCertificateKeyFile /etc/ssl/private/host.key

Ok – make sure you save your default-ssl file with the changes we have detailed above.  After the file is saved, run the following command:

a2ensite default-ssl

followed by:

/etc/init.d/apache2 restart

Now, its time to test to see if you can connect to the server with SSL. You might want to create an index.html file at the /var/www-ssl directory first though. To do this simply create a file with “It works, SSL!” in it as root and save it to the /var/www-ssl directory with the name of index.html. Afterwards, open Firefox, and go to https://<server name or IP address>. You probably get a certificate error, since we created a self-signed cert for our tutorial. You should tell Firefox to trust the certificate. If all goes well, you should see: It works, SSL! at the top of the page.

If all went well, we should have a working Apache server with SSL. At this point, you can place content directly under /var/www-ssl or in folders under that directory. If you have some folders under the non-ssl Document root (normally /var/www) simply sim-linking those directories to /var/www-ssl will publish them via SSL as well.

So, what if it doesn’t work? Generally, this is the result of either SSL not being enabled, or an errror in your default-ssl file. Double check that you have enabled SSL with the a2enmod command, and make sure the default-ssl file is correct, and that it has been enabled with the a2ensite command. Also note, that when enabling SSL for the first time, make sure you restart Apache with the command:

/etc/init.d/apache2 restart

To make sure SSL support is enabled. You can check that Apache is listening for SSL connection by doing a

netstat -tan

and looking for an entry like this:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address     State
tcp        0      0   *           LISTEN


If that line is not present, then double check your configuration.

Next time we will focus on the Mysql portion of a LAMP server – look for it soon.

Ubuntu Apache LAMP Server Quick Howto – Part 1 – Apache Basics

Linux web application servers typically use the Linux/Apache/Mysql/PHP stack. Linux being the OS, Apache the web server layer, Mysql provides the database, and PHP the dynamic HTML/Scripting language. Their is an amazing amount of LAMP based applications out there, so getting to know how to administer a LAMP server is a key skill set for running Linux Application servers.

This first article will focus on installing the LAMP stack on an Ubuntu machine, and administering the Apache web server.

There are a few of concepts I would like to cover first though. One of them seems to escape a lot of people in this space. This is a the concept of name based virtual hosts. Name based virtual hosting is the ability for a web server to serve content based on the URL of the incoming request. This is a method of allowing a single server to serve multiple websites content without needing multiple IP addresses. Essentially the server processes the URL request by matching it against a know set of virtual host definitions. When it matches a URL to a virtual host, it serves content from the directory structure assigned to that virtual host.

The other concept is fully qualified domain names. A fully qualified domain name or FQDN is a hostname containing the host and the domain, including the top level domain suffix. Common top level domains (TLDs) are .gov, .com, .org etc. So an example of a FQDN would be: www.linuxpoweruser.com. The www is the host, and linuxpoweruser.com is the domain, so www.linuxpoweruser.com indicates the host called www in the domain linuxpoweruser.com.

The last concept is how what part of a URL is handled by the domain name system (DNS). DNS is responsible for resolving the part of a URL that is between the http:// and the next / in a URL. If there are no following /’s in a URL than DNS processes all of the URL after the http://. I am a DNS administrator for a large retailer and I am constantly asked to add an entry to add a redirect in DNS to allow, for instance, a URL called http://www.test.com to redirect to http://www.retest.com/test/test.html. Since DNS only handles the “www.test.com” or “www.retest.com” everything after the / in the destination, ie /test/test.html is not handled by DNS, and therefore DNS cannot do this sort of redirection. In this case, I can modify DNS to do a redirect for www.test.com to go to the name www.retest.com (or its corresponding server IP address) but its up to the virtual hosting definition or, a bit of redirect html code to handle the rest. Ok, enough said about that.

First, in order to install the LAMP stack on an Ubuntu system, we need to make sure we install the associated packages:

sudo apt-get install apache2 php5-mysql libapache2-mod-php5 mysql-server php5-gd phpmyadmin

There are some commands that are very useful to control Apache features and served content. The first set of commands will control the availability of Apache features. For instance, you might want to enable home directory public_html serving, which is the ability to serve content from a users public_html folder in their home directory. Content is then accessed at the web browser by going to http://server ip/~username. To do this, use the a2enmod command:

sudo a2enmod userdir && sudo /etc/init.d/apache2 restart

Other apache modules can be enabled with this a2enmod command. Here is an example that covers server side includes (we will cover server side includes in a later article):

sudo a2mod include && sudo /etc/init.d/apache2 restart

The default document root is /var/www (you need superuser privileges to write in this directory), any files in this directory that are world readable will be accessible by entering the following in a web browser:

http://server ip or name/filename

Anything in a subdirectory below /var/www will appear by appending the directory name to the url. For example, if there is a directory with content at /var/www/mywebsite, it would be accessible with the following url:

http://server/mywebsite/filename where server is the name or IP of your apache server.

New sites can be added by creating site definition files in the /etc/apache2/sites-available directory. Files in this directory are essentially apache configuration files that can be read in or included when apache starts up. This is useful for adding new websites via virtual host definitions, or enabling SSL for your sites.
Edit a new configuration file for the new site :

gksudo gedit /etc/apache2/sites-available/<filename>

Here is an example of a site definition file for adding a directory phpsysinfo as a virtual host on the server, accessible via the sysinfo.rtwsecurenet.com url.

<Directory “/var/www/phpsysinfo”>

# Possible values for the Options directive are “None”, “All”,
# or any combination of:
#   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
# Note that “MultiViews” must be named *explicitly* — “Options All”
# doesn’t give it to you.
# The Options directive is both complicated and important.  Please see
# http://httpd.apache.org/docs/2.2/mod/core.html#options
# for more information.

#-Indexes disables directory browsing +Includes turns on SSI

Options -Indexes FollowSymLinks +Includes

# AllowOverride controls what directives may be placed in .htaccess files.
# It can be “All”, “None”, or any combination of the keywords:
#   Options FileInfo AuthConfig Limit
AllowOverride All

# Controls who can get stuff from this server.
Order allow,deny
Allow from all

#add this to allow for different default page names

DirectoryIndex index.html index.shtml index.php


<VirtualHost *:80>

ServerName sysinfo.rtwsecurenet.com
DocumentRoot /var/www/phpsysinfo


Note :80 in VirtualHost statement – needed if running combination of ssl and non-ssl sites. We will break down the components of this file later.

To enable the new site use the a2ensite command:

sudo a2ensite <sitename> && sudo /etc/init.d/apache2 restart

Later, if you would like to disable that site, you can use the a2dissite command to remove the site:

sudo a2dissite <sitename> && sudo /etc/init.d/apache2 restart

Note, that if the site content is at /var/www or a directory below it, you do not have to create a site file.  The site file is only used to include other directories other than what is below /var/www or to create virtual hosts for making content available at a specific host name.

So what is happening with this a2ensite/a2dissite commands?  Essentially, a2ensite makes symbolic links in the /etc/apache2/sites-enabled/ directory for the appropriate file/site being enabled (from the /etc/apache2/sites-available directory).  The a2dissite command simply deletes the symbolic links.

Beware of some web applications that install themselves outside of the normal /var/www and /etc/apache2/sites-available directory structures. An example comes to mind: phpmyadmin. Phpmyadmin is a php based tool for administering Mysql servers. Phpmyadmin on ubuntu does not install in the /var/www folder. Instead it installs in /usr/share/phpmyadmin. It also stores configuration in /etc/phpmyadmin. In the /etc/phpmyadmin folder there is an apache.conf file. This file is executed when Apache is started, and it includes the /usr/share/phpmyadmin directory in the web servers directory structure, so that it appears at http://servername/phpmyadmin, as if it was a directory under /var/www. How does it do this? Another directory, /etc/apache2/conf.d includes symlinks to files to be included in the apache configuration. Its actually very similar to sites-enabled/available. In /etc/apache2/conf.d/ there is a phpmyadmin.conf that is symlinked to /etc/phpmyadmin/apache.conf. These symlinks are created by apt during package installation.

So, in short we have covered installing the LAMP stack, and controlling Apache’s configuration for serving content. In the next article, we will cover creating Secure Sockets Layer (SSL) websites using apache.

Client IPSEC VPNs with Linux and Juniper Netscreen

Today, as promised I am going to show everyone how to set up a client IPSEC VPN to a Juniper Netscreen FW/VPN appliance from a Linux machine.  Juniper is a market leader in the Firewall and VPN space, and provides appliances from the Small office Home Office footprint all the way up to the largest enterprise data center gateways.  The small office version, currently an SSG 5, is based on Juniper’s ScreenOS.  This tutorial only covers ScreenOS configuration.  Juniper is now marketing a new platform called the SRX series. These units run JUNOS, which is Junipers Router OS outfitted with VPN and Firewall functionality pulled from ScreenOS.

The Linux system examples I show should work on Ubuntu (and Gentoo, provided your kernel has support for IPSEC configured).

So why VPN?  As I have mentioned in previous posts a VPN is a secure method for using a public network for private communications.  Most VPNs fall in to type types:  IPSEC and SSL.  We will be covering IPSEC VPNs in this posting.  I typically use a VPN for access to my employer’s network while on the road so I can be productive while traveling.  With the right technology, you can gain access to your home network the same way.

I have a small home network that is protected by a Juniper Netscreen SSG5.  This SSG 5 is configured to allow one or more “client” vpns to connect to it.  Essentially a client VPN is a single PC talking to a LAN via a VPN server or gateway, as opposed to a site to site VPN, which generally connects two networks or LANs together over the internet.

Here is a little background on IPSEC.

IPSEC VPNs generally consist of two phases.   Phase 1 is an identification phase, where two IPSEC gateways identify each other.  If the identification is successful (the two gateways trust each other) then Phase 2 which is the ‘tunnel’ phase, can occur.  During this phase, the two gateways negotiate the IP subnet traffic that will be allowed to traverse the tunnel and how to encrypt that traffic.  Generally this traffic is protected by using 3DES or AES encryption, which dynamic key rotation.  IPSEC, if set up properly is very hard to compromise.

Phase 1 typically uses UDP port 500 to perform IKE/ISAKMP negotiations.  Phase 2, in our case, will use IPSEC ESP (Encapsulated Security Payload) which is a transport layer protocol (like TCP and UDP, runs at layer 4 of of the OSI model). If a NAT firewall is detected in between the two gateways, then ESP can be encapsulated in UDP port 4500 or UDP port 500 depending on the implementation used.  This encapsulation is called IPSEC NAT traversal, or NAT-T.

We will start with the following example network layout:

Diagram of Example VPN NetworkFirst we will start with the Netscreen configuration, which is best performed from the netscreen command line.  Keep in mind that this will work on several Netscreen models sold over the last few years.  This includes the Netscreen 5XT, and 5GT.  These units can be found on Ebay for less than $100, and are highly recommended for use as a home or small office firewall.

Note:  It is best to have a static IP for the VPN gateway side of the connection.  Dynamic will work, however it will become difficult to track the ip as the ISP reassigns addressing to the Juniper via DHCP or other means.  Newer Juniper Netscreens can support dynamic dns registration with DDNS providers such as dyndns.org, which would make tracking the ip easier.

So, lets get started.  From an ssh or telnet session, login to your netscreen.  The first thing we need to do is define VPN users and a group – here is an example of creating a user “rwalters”.  This user id is used for IKE negotiation.

Phase 1: (IKE gateway negotiation)

User definition:

set user "rwalters" uid 1
set user "rwalters" ike-id u-fqdn "rettw@rtwnetwork.com" share-limit 1
set user "rwalters" type ike
set user "rwalters" "enable"

After creating a user, we should add that user to a group.  While not required, if you want more than one client VPN to be active at one time, you should add your user to a group, as the group will be used in the IKE gateway definition, and any of the users in the group will be allowed to authenticate in Phase 1.

Here we create a group called dialupusers:

set user-group "dialupusers" id 1
set user-group "dialupusers" user "rwalters"

Next we will define the IKE Phase 1 gateway definition.  In most cases, we use pre-shared key authentication, which is basically a password.  Other forms of credentials can be used as well such as RADIUS or X-AUTH but those are beyond the scope of this tutorial.  Here is an example of the ScreenOS command for IKE gateway definition:

set ike gateway "Publicdialupvpn" dialup "dialupusers" Aggr outgoing-interface "ethernet0/0" preshare "<password>" proposal "pre-g2-3des-sha"
set ike gateway "Publicdialupvpn" nat-traversal udp-checksum
set ike gateway "Publicdialupvpn" nat-traversal keepalive-frequency 5

The first command creates an IKE gateway called “Publicdialupvpn” and associates the “dialupusers” group with it.  It also defines the outgoing interface, ethernet0/0 and the preshared key or password.  You should use a complex password in place of the <password> shown in the command.    The Aggr means aggressive mode, which is used when the IP address of one of the gateways is dynamic – this case the laptop will almost always have a dynamic IP address.  The remaining commands set nat-traversal capabilities.  Without NAT-T, Phase 2 will not come up if a NAT-Router or firewall is present in the middle of the network.  The last piece is the encryption used for IKE traffic – in this case 3DES, with SHA-1 hashing algorithm.

Next, we will define the Phase 2 tunnel.  An example is below:

set vpn "Publicdialupvpn" gateway "Publicdialupvpn" no-replay tunnel idletime 0 proposal "g2-esp-3des-sha"

This command is pretty simple.  It defines a VPN (or tunnel) called Publicdialupvpn, using the IKE gateway definition of the same name. It also turns on no-replay, which prevents replaying of traffic (a common method of hacking VPNs) and sets the transport to ESP, and encryption to 3DES, with SHA-1 hashing.

The last piece of Netscreen configuration is the Firewall policy to allow the encrypted traffic to the internal network.  This policy is actually used as part of phase 2 as well, since phase 2 requires the exchange and agreement on the ip addresses that will be allowed to traverse the tunnel (known as a policy based VPN in Juniper terminology).

In this example we will create an address object for the local LAN:

set address "Trust" "Local LAN"

Then we will create a policy to allow it to be tunnelled to from the outside.  Note that Dial-Up VPN is a default address book entry that ships with ScreenOS.  Note that you can limit the ports/protocols allowed through the tunnel by changing the “ANY” to a specific service, such as http, for instance.

set policy id 8 from "Untrust" to "Trust" "Dial-Up VPN" "Local LAN" "ANY" tunnel vpn "Publicdialupvpn" id 0x3 log
set policy id 8
set log session-init

So, that should take care of the Netscreen side of the equation.  Next we will tackle the Linux side which is the fun part.  The beauty of Linux IPSEC is that its one of those built in features that you would have to pay for if using one of those operating systems made in Redmond, WA.

Linux IPSEC has a few requirements.  If using Ubuntu, all the kernel requirements are already fulfilled.  If you’re using Gentoo or a custom kernel, make sure the following is set in your kernel config:

Under networking:


Under Cypto/Block:


Under Crypto/Ciphers:


Linux IPSEC is handed by the kernel in conjunction with two different packages:  ipsec-tools, and racoon.  Both of these must be installed and configured for our VPN to work properly.  On Ubuntu you can install these with:

apt-get install ipsec-tools

apt-get install racoon

On gentoo, just a simple:

emerge ipsec-tools

Should install the necessary software.

Three files files need to be customized for Linux IPSEC VPNs to work with a Juniper Netscreen.  The first file, ipsec-tools.conf, usually resides in /etc, and the second, psk.txt in /etc/racoon. The third file, is racoon.conf, which is also in the /etc/racoon directory.

The ipsec-tools.conf file handled Phase 2 security association, and the racoon.conf/psk.txt file provide IKE (Phase 1) and dynamic re-keying of encryption between two VPN endpoints.  Here is an example of an ipsec-tools.conf file that would work for our sample VPN diagram and Juniper configuration above.

#!/usr/sbin/setkey -f


spdadd any
-P out ipsec esp/tunnel/;

spdadd any
-P in ipsec esp/tunnel/;

The information in this file is pretty straight forward.  Essentially its a tunnelling policy.  It basically states that all traffic to, which is our remote LAN behind the Juniper from (the Internet IP of our laptop) be tunnelled through the esp tunnel between (the internet IP of our Juniper) and (our laptop) and vice versa.  Note that is file must be updated with the existing IP address on the laptop or remote PC every time a VPN is started. For instance if your laptop gets an ip address of from the ISP your using to connect to the internet, all of the ip addresses in the ipsec-tools.conf file will have to be changed to

The other two files /etc/racoon/psk.txt, and /etc/racoon/racoon.conf are relatively static.   The first file, psk.txt, is essentially a list of IP addresses of remote VPN gateways and the pre-shared key to use for a password when doing IKE with that gateway.

The password in this file should match the password used in the netscreen Phase 1 IKE configuration shown above.  Here is a sample of that file:

# IPv4/v6 addresses    <your pre-shared key password>    mekmitasdigoat    0x12345678    whatcertificatereally
3ffe:501:410:ffff:200:86ff:fe05:80fa    mekmitasdigoat
3ffe:501:410:ffff:210:4bff:fea2:8baa    mekmitasdigoat
foo@kame.net    mekmitasdigoat
foo.kame.net    hoge

The last file, racoon.conf controls Phase 1 IKE negotiation, Phase 2 VPN SA setup and VPN re-key.  This file rarely changes for a particular vpn definition.

Here is a sample racoon.conf:

path pre_shared_key "/etc/racoon/psk.txt";

# Remote host
exchange_mode aggressive;

# Change this to your local ID
my_identifier user_fqdn "rettw@rtwnetwork.com";
lifetime time 28800 sec;
proposal {
encryption_algorithm 3des;
hash_algorithm sha1;
authentication_method pre_shared_key;
dh_group modp1024;

# A sample sainfo section
# Create one for each subnet you want to access, etc.
#sainfo address any address any
sainfo anonymous
pfs_group modp1024;
lifetime time 3600 sec;
encryption_algorithm 3des;
authentication_algorithm hmac_sha1;
compression_algorithm deflate;

Generally there are two sections to racoon.conf.  The first section controls IKE parameters, and Phase 1 negotiations.  The second section controls Phase 2 negotiations.  There are a couple of things I would like to point out about.  First, notice the local id or user fqdn.  This should be the same as one of the users created on the Juniper side.   The second item is the lifetime and encryption types.  The first lifetime value in the IKE/Phase 1 section dictates the length of time the two gateways will trust each other without re-identification.  The Phase 2 section has its own lifetime parameter as well.  This controls the re-key time on the tunnel, in this case every 3600 seconds or one hour, the tunnel encryption keys will be renegotiated.

Note also the “aggressive mode” statement – this should match the Phase 1 definition on the Juniper as well.

So, after the files have been configured correctly, you need to start the VPN.  The first step is to run the ipsec-tools.conf file, so it should be executable.  The last step is to start racoon.  Here are the example commands.


/etc/init.d/racoon start

After the racoon daemon is started, try to ping something on the remote LAN – for example,  The ping will start the IPSEC negotiations.  Watching your /var/log/daemon.log or /var/log/messages will show you what is happening.

So – what if it doesn’t work?  Double check your configuration.  Juniper ScreenOS event log (command: get event) is very helpful in determing what is happening.  The linux /var/log/daemon.log and /var/log/messages will also be helpful.

Ok – so this seems like a lot of work, modifying files and starting services – I agree its not quite optimal, at least on the Linux side.  So, I wrote a couple of scripts – one to determine your PC/Laptop’s IP address, create the ipsec-tools.conf file, run it, and then run racoon.  As a bonus, it also replaces your resolv.conf with another resolve.conf – in case there is a dns server for your remote network you would like to use to be able to resolve machine names on your protected LAN.  The second script clears the ipsec-tools.conf policies, and stops racoon, and replaces the resolv.conf with the original one.

With the scripts below, starting a VPN to your office or home lan is as simple as connecting to the internet, and running a the script.  Afterwards, the to undo the changes, you run another script.  This way, there are only one time changes needed to /etc/racoon/psk.txt and /etc/racoon/racoon.conf

Here is the script for starting the vpn:


#script to find outgoing internet interface (by opening a socket to google.com)
#and build a vpn policy file for ... and turn up the VPN tunnel

import socket
import os

def OutputSpace2file():
filehandle.write ( ' ' )
s = socket.socket()
# Connect to google.com to find out going IP address
ipport = s.getsockname()
ipaddr = ipport[0]
destip = ''
destsubnet = ''
print "Setting VPN tunnel up from Source:",ipaddr,"To IP address:",destip
print "For destination subnet",destsubnet
print "Generating ipsec-tools.conf file in /etc"
filehandle = open ('/etc/ipsec-tools.conf','w')
filehandle.write ( '#!/usr/sbin/setkey -fn' )
filehandle.write ( 'n')
filehandle.write ( 'flush;n' )
filehandle.write ( 'spdflush;nn' )
filehandle.write ( '#outboundn' )
filehandle.write ( 'spdadd ' )
filehandle.write ( ipaddr )
filehandle.write ( destsubnet )
filehandle.write ( 'anyn' )
filehandle.write ( '    -P out ipsec esp/tunnel/' )
filehandle.write ( ipaddr )
filehandle.write ( '-' )
filehandle.write ( destip )
filehandle.write ( '/require;nn' )
filehandle.write ( '#inboundn' )
filehandle.write ( 'spdadd ' )
filehandle.write ( destsubnet )
filehandle.write ( ipaddr )
filehandle.write ( 'anyn' )
filehandle.write ( '    -P in ipsec esp/tunnel/' )
filehandle.write ( destip )
filehandle.write ( '-' )
filehandle.write ( ipaddr )
filehandle.write ( '/require;n' )
# set permissions on new policy file
rc = os.system( 'chmod a+x /etc/ipsec-tools.conf' )
# Check return code for permissions command
if rc != 0:
print "Error Setting permissions!"
# Continue by runninng ipsec-tools.conf script
print "Running ipsec-tools.conf script."
rc = os.system( '/etc/ipsec-tools.conf' )
# Check return code for command
if rc != 0:
print "Error running ipsec-tools.conf script!"
# Continue by reloading racoon
print "Reloading racoon IKE server."
rc = os.system( '/etc/init.d/racoon reload' )
# Check return code for command
if rc != 0:
print "Error running /etc/init.d/racoon reload!"

# backup existing /etc/resolv.conf and build new one for remote lan

print "Backing up /etc/resolv.conf and setting resolv.conf to rtwsecurenet.com DNS"
rc = os.system( 'mv /etc/resolv.conf /etc/resolv.conf.bak' )
if rc != 0:
print "Error backing up /etc/resolv.conf"
filehandle = open ('/etc/resolv.conf','w')
filehandle.write ( '# File created by LPU vpn-start.py scriptn' )
filehandle.write ( 'search homelan.comn' )
filehandle.write ( 'nameserver' )

And for stopping the VPN:


#script to stop vpn started by vpn-start.py and restore resolv.conf

import socket
import os

# stop vpn by flushing ipsec policies
print "Flushing IPSEC policies"
rc = os.system( '/usr/sbin/setkey -FP' )
# Check return code for permissions command
if rc != 0:
print "Error flushing policies!"
# Continue by runninng ipsec-tools.conf script
print "Restoring /etc/resolv.conf to previous state"
rc = os.system( 'cp /etc/resolv.conf.bak /etc/resolv.conf' )
# Check return code for command
if rc != 0:
print "Error restoring /etc/resolv.conf - name resolution will not work properly!"
# Continue by reloading racoon
print "Stopping racoon IKE server."
rc = os.system( '/etc/init.d/racoon stop' )
# Check return code for command
if rc != 0:
print "Error running /etc/init.d/racoon stop!"

So, there you have it.  I use this type of vpn set up regularly to access my home network when on the road.  So far, I haven’t found anything that keeps it from working, unless the hotel ISP network assigns my laptop a 192.168.0 IP.  This of course would render the vpn useless, since you can’t tunnel from to as it will confuse the ip stack and routing processes.  Feel free to use these scripts in your own environment.

While some purists may say, why not use linux for both ends of the tunnel. While this can be done, its hard to beat the price and usability of Juniper’s Netscreen line. It is an excellent firewall, and used units can be found on Ebay for less than $100 now – look for Netscreen 5GT or 5XT.

I have tried to cover some key concepts of IPSEC VPNs, but this is by no means a complete overview, but more of a implementation for a specific application. There are several IPSEC references available on the internet via this Wikipedia article.

Network Emulation with Linux Netem

Back in the day, I used an open source program called NistNET to emulate a WAN for my company’s network test lab on a linux machine.  I was able to solve a multitude of issues and test our applications in a WAN environment with this product.  Unfortunately, NistNET is no longer maintained, and until recently I had no open source tool for emulating a network in my arsenal.  The other day, while playing Call of Duty 2 with some friends on my dedicated linux server, I decided that I was tired of having an unfair advantage since my latency to the server was 1 ms, and they all had 50-70 ms or more, so I went on a search for something I could use to add delay to my connection to the server (one of my buddies says I am too honest).  After some searching, I came upon netem, which to my surprise is and has been part of the linux kernel for some time.  I know, some of you linux guys and gals out there are saying “tell me something I don’t know”  but, ashamedly, I didn’t know about this one perhaps because I don’t do much of that kind of work any more.

Anyway, using Netem, I could do exactly what I wanted.  I can add enough delay to my client to game server traffic (actually, its server to client, which I will explain later) to make it seem like I have a lot more network between the server and me than I actually do.  Whether or not this decreases my advantage in the game is yet to be seen.

Anyway, several “effects” are present on most wide area networks today.  A common effect, latency, can have a drastic effect on the way network communications protocols behave.  Latency is also one of the key issues with playing on line games, especially those that require fast reaction to on screen events.  High latency creates what gamers refer to as lag.

The Netem function of linux provides the capability to modify the parameters of egress traffic (ie, traffic exiting the machine and destined for another point on on the network).  With netem, its possible to create artificial delay, thus creating latency.  Other possibilities are rate limiting (controlling the amount of bandwidth traffic can use), causing packet loss, and jitter.  Packet loss can result in very poor performance with TCP applications.  Jitter, also known as variable delay, is bad for real time streaming applications such as voice over IP.

Anyway, you could probably see why this kind of stuff would be important to a network engineer, especially in a lab environment.

So – on to how to use netem.  Netem is controlled by the tc command, which is part of the iproute2 package and is included in with most linux distributions.

Using the tc command, we can easily tell a linux host to delay all packets exiting a network interface using this command:

tc qdisc add dev eth0 root netem delay 80ms

This will add 80ms of delay to all packets leaving the eth0 interface.  To test the result of this command, just do a ping from you machine before issuing the command, and then after:

ping -n

PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=0.103 ms
64 bytes from icmp_seq=2 ttl=64 time=0.394 ms
64 bytes from icmp_seq=3 ttl=64 time=0.070 ms
64 bytes from icmp_seq=4 ttl=64 time=0.101 ms
--- ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2998ms
rtt min/avg/max/mdev = 0.070/0.167/0.394/0.131 ms

Enter the tc command for adding delay to eth0:

tc qdisc add dev eth0 root netem delay 80ms

Then ping again:

ping -n
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=80.0 ms
64 bytes from icmp_seq=2 ttl=64 time=80.0 ms
64 bytes from icmp_seq=3 ttl=64 time=80.0 ms
64 bytes from icmp_seq=4 ttl=64 time=80.4 ms
--- ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 80.073/80.164/80.414/0.246 ms

Notice the difference in delay (~80ms).

We can also add variable delay (jitter) as most wide area networks (such as the internet) have some jitter associated with them.  The following command will add +/- 10ms of jitter to the 80ms delay shown in the last example:

tc qdisc add dev eth0 root netem delay 80ms 10ms

Now lets do the ping again:

PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=72.6 ms
64 bytes from icmp_seq=2 ttl=64 time=84.6 ms
64 bytes from icmp_seq=3 ttl=64 time=86.7 ms
64 bytes from icmp_seq=4 ttl=64 time=84.0 ms
--- ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 72.648/82.023/86.752/5.510 ms

Looks even more like a real internet connection now.

To see what qdisc (short for queuing discipline) parameters have been applied to an interface (in this case eth0) use the following command:

tc qdisc show dev eth0

Sample output follows:

qdisc netem 8003: root limit 1000 delay 80.0ms  10.0ms

The last part of the output shows that a delay of 80ms +/- 10ms is applied.

Now, for the important part – how do you turn this off? It took a while to find this in the netem documentation:

tc qdisc del dev eth0 root

This will remove all queuing discipline parameters from the eth0 interface on your system.

So this is great, but not necessarily what I am looking for.  Adding delay wholesale to the server would also increase my fellow gamers latency as well as mine, and the idea is to level the playing field.

That is ok, since netem/tc has a way to only place qdisc’s on specific traffic.  In my test network, I have two machines.  One running Windows 7 (this case the Call of Duty Client) and one running Ubuntu 9.10 (the COD2 server).  The Windows machine has an IP of, and the server

On the linux server, I run the following commands as root:

tc qdisc add dev eth0 root handle 1: prio
tc qdisc add dev eth0 parent 1:3 handle 30: netem delay 140ms 10ms distribution normal
tc filter add dev eth0 protocol ip parent 1:0 prio 3 u32 match ip dst flowid 1:3

This set of commands creates a simple priority queuing discipline, attaches a basic latency netem at hook 3, and then tells all traffic to to be priority 3, thus subject to the netem delay of 140ms +/- 10ms (with a normal statistical distribution of jitter).

These commands do exactly what I was wanting – making my delay to the game server about equal to my friends.

So far, it seems to work, however it’s not optimal.  The reason its not optimal is because only the packets coming from the server to my Windows client machine are being delayed.  A true internet connection would have delay in both directions.  Since netem only affects the egress of traffic from a network interface, technically you would have to delay the traffic as it leaves the client PC, and delay the traffic as it leaves the server back towards the client.   Since Windows doesn’t have a netem facility (at least not without some expensive commercial software such as that from Shunra) the best way to do this would be to run Call of Duty 2 on Linux using wine (which is another article for a another time).  That way I could induce delay on both machines, and get a “more perfect”  simulation of the internet.

To show existing filters such as those set by the last set of commands you can use the following commands:

tc filter show dev eth0


tc qdisc show dev eth0

Here is an example output:

tc filter show dev eth0

filter parent 1: protocol ip pref 3 u32
filter parent 1: protocol ip pref 3 u32 fh 800: ht divisor 1
filter parent 1: protocol ip pref 3 u32 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:3
match c0a8000f/ffffffff at 16

tc qdisc show dev eth0

qdisc prio 1: root bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
qdisc netem 30: parent 1:3 limit 1000 delay 140.0ms  10.0ms

In summary, netem is a perfect example of what I love most about Linux – flexibility and utility beyond what most commercial Operating Systems (especially those from Redmond) offer out of the box for free. While the way I have implemented it is not quite perfect, netem can provide what I am looking for in terms of simulating network conditions. As I mentioned above, perfection could be achieved by using linux as the client as well as the server, or by making a bridge between the two machines with a dual interface linux machine doing netem delay on both interfaces.

Netem has several capabilities that I didn’t cover, which can be found at the following links:

The Linux Foundation has an overview page here.
Here is pdf file showing more details on netem and tc usage.

Next up – Client IPSEC VPNs from a linux laptop to a Juniper Netscreen VPN/FIrewall device. Coming Soon!

SSH Tunnelling (aka Poor Man’s VPN)

Tunnelling of TCP traffic can be performed from the ssh command on Linux or with Putty on Windows, and can be thought of as a poor man’s VPN.  A VPN is a virtual private network, or a method of using a public network such as the internet to  securely transmit data via an encrypted “tunnel”.

VNC is a method of gaining access to a remote GUI on Linux and Windows machines.  VNC is typically considered insecure and not recommended for use on the open internet.  With an SSH tunnel, this doesn’t have to be an issue, as ssh provides security to an otherwise insecure protocol.

Here is an example of how to use VNC over an SSH tunnel:

Start VNC server on a Linux host ssh server to only listen to Loopback interface:

vncserver :1 -localhost

On client machine, start ssh with the following command line:

ssh -L 5901:localhost:5901 <server ip> [-l <login>]

The to access VNC via the SSH tunnel, use the following command on the client machine:

vncviewer localhost:1

What happens?

The ssh process on the client sets up a TCP port redirection on the loopback interface of port 5901, to the loopback interface on the server machine on TCP port 5901.  The vncviewer command connects to the the redirected port on the local loopback interface, which then gets directed over the tunnel to the server machine’s loopback on port 5901, where the vncserver is listening.

This will allow vnc protocol to be securely tunnelled across the SSH connection.

The previous examples showed connecting to services running on the ssh server itself.  Its also possible to use the SSH server to redirect traffic to other machines on the network behind it.

Sometimes we might have to access a Windows Machine behind an a linux SSH server that is connected to the internet (such as linux system performing firewalling for a home network). We can use SSH tunnelling to connect to Windows Remote Desktop as well. (I know this is a linux blog, but most of us out there still have to deal with Windows from time to time)

The following example assumes the following network layout:

Client PC –> Internet–>SSH Server on Firewall–>Private Network–> Windows XP
(                                                                                                        (
(ssh server can be behind firewall as long as its accessible from Internet)

1. Make sure Windows XP host is running RDP

2. On client PC, start SSH with tunnelling as follows: 4000:

On Putty this tunnel definition looks like this (click add after completing the boxes):

Which is exactly like the ssh command on Linux:

ssh -L 4000: <server> -l <userid>

To connect to the RDP service on the internal Windows XP system, from a client Windows system connected to the internet via the ssh connection:

Use Remote Desktop Connection application that comes with XP, but use this as the address to connect to:

Like the previous example, this causes the program to connect to port # 4000 on the local loopback interface, which then is redirected to the machine on port number 3389 at the other end of the ssh tunnel.  Port # 4000 is used to avoid conflicting with port #3389 on the client as it could have its own RDP server running.

Any TCP based communications can be tunnelled this way over ssh, creating a secure connection for any unsecure protocol. This is also a mechanism for bypassing firewall rules.  As long as the SSH server traffic is allowed (TCP port 22).  It can be used to gain access to other ports that might not be allowed by a local firewall, simply by using a remote ssh server as a proxy for other traffic.