Enabling MySQL Logging on Archlinux

This morning I was doing some troubleshooting on a PHP program that couldn’t seem to access the locally running MySQL database server. Upon searching for some logging to figure out what was going on, I came across the fact the MySQL as installed from the Arch repositories, doesn’t enable much along theses lines.

MySQL supports error logging to /var/lib/mysql directory by default. The file is usually in the form .err. This file shows mysqld startup errors and other basic troubleshooting information. I was looking for something a little more in depth, such as client connections, and SQL statement logging. Apparently this type of logging is referred to as general logging. After some experimentation, it appears that enabling this log requires a few simple system changes.

Enabling general logging for MySQL

The first step is to figure out where you want MySQL to store its general log. In our example, we will put it in the /var/log/mysql directory, in file called general.log. This directory and file must be created, and permissions set so that mysql can write to it. MySQL will not create this file on its own. So lets create the directory and file with the following commands:

sudo mkdir /var/log/mysql
sudo touch /var/log/mysql/general.log
sudo chown mysql:mysql -R /var/log/mysql

Then, in order tell MySQL to enable general logging, a couple of lines so be added to the /etc/mysql/my.cnf file in the [mysqld] section – see last two lines in the example my.cnf file below:

# The MySQL server
port = 3306
socket = /var/run/mysqld/mysqld.sock
datadir = /var/lib/mysql
key_buffer_size = 16M
max_allowed_packet = 1M
table_open_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
general_log = 1
general_log_file = /var/log/mysql/general.log

That’s all it takes! Save the my.cnf file, and restart mysqld with:

/etc/rc.d/mysqld restart

You should now the general.log file in /var/log/mysql. An example might look like this (showing some client connections and select statements):

/usr/bin/mysqld, Version: 5.5.21-log (Source distribution). started with:
Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock
Time Id Command Argument
120318 10:10:26 1 Connect UNKNOWN_USER@localhost as on
1 Quit
120318 10:10:39 2 Connect root@localhost on
2 Init DB webcalendar
2 Init DB webcalendar
2 Query SELECT COUNT( cal_value ) FROM webcal_config
2 Query SELECT cal_permissions FROM webcal_access_function
WHERE cal_login = 'admin'
2 Query SELECT cal_permissions FROM webcal_access_function
WHERE cal_login = '__default__'
2 Init DB webcalendar
2 Query SELECT cal_login, cal_lastname, cal_firstname, cal_is_admin, cal_email, cal_passwd FROM webcal_user ORDER BY cal_lastname, cal_firstname, cal_login

The above snip shows MySQL starting and an application hitting a database.

Configuring MySQL to log errors to the system log

That last thing I want to cover is getting MySQL to log to the system log, instead of its own log in /var/lib/mysql. I mentioned above that MySQL logs errors to /var/lib/mysql/.err. We will make a simple change to log those same messages to the system log, named to the daemon.log.

This one is pretty simple – add the following section to the /etc/mysql/my.cnf file:


And then restart MySQL with:

/etc/rc.d/mysqld restart

You should see messages in the daemon.log file that look something like this:

Mar 18 10:14:54 localhost mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
Mar 18 10:14:54 localhost mysqld: 120318 10:14:54 InnoDB: The InnoDB memory heap is disabled
Mar 18 10:14:54 localhost mysqld: 120318 10:14:54 InnoDB: Mutexes and rw_locks use GCC atomic builtins
Mar 18 10:14:54 localhost mysqld: 120318 10:14:54 InnoDB: Compressed tables use zlib 1.2.6
Mar 18 10:14:54 localhost mysqld: 120318 10:14:54 InnoDB: Initializing buffer pool, size = 128.0M
Mar 18 10:14:54 localhost mysqld: 120318 10:14:54 InnoDB: Completed initialization of buffer pool
Mar 18 10:14:54 localhost mysqld: 120318 10:14:54 InnoDB: highest supported file format is Barracuda.
Mar 18 10:14:54 localhost mysqld: 120318 10:14:54 InnoDB: Waiting for the background threads to start
Mar 18 10:14:55 localhost mysqld: 120318 10:14:55 InnoDB: 1.1.8 started; log sequence number 2218816
Mar 18 10:14:55 localhost mysqld: 120318 10:14:55 [ERROR] /usr/bin/mysqld: Can't find file: './performance_schema/events_waits_current.frm' (errno: 13)

In summary, the above changes will allow you to “see” what MySQL is doing both in terms of the daemon itself, and the actual activity being processed by the database server. These changes could also work on other distros, but in particular ArchLinux by default enables only error logging to the MySQL directory only.

Up next – configuring syslog-ng as syslog server for network devices.

Ubuntu vs Gentoo vs Arch – A comparison

Trials with Ubuntu and Gentoo Linux

Arch Linux is now my default distribution – supplanting Ubuntu Linux. Why? Well, its a long story, but I will try to keep it to the point. I am a tinkerer at heart – I love Linux because it allows you to tinker – essentially its a wide open book. Ubuntu has taken Linux down a different path. Ubuntu’s specialty is ease of use, and it excels at that. Ubuntu’s ease of use comes at a price though – flexibility and customization is somewhat diminished due to Ubuntu’s focus on the user experience.

I first moved to Ubuntu in 2008, when Hardy Heron, 8.04LTS was released. I came over as a frustrated Gentoo User. After using Gentoo Linux since 2004 (I used Redhat exclusively before that) I became a bit disenfranchised with Linux as a whole. It seemed that in order to get complete customization and control, I was going to have to suffer through hours of patching and compiling. Ever compile OpenOffice.org on a Pentium 4? It takes hours. I loved Gentoo for its ultimate in customization, but loathed it for the update process.

Ubuntu was a breath of fresh air in many ways – Install it and it just worked. It renewed my faith in Linux. Over time though I discovered several shortcomings. Canonical’s use of Gnome themes that were difficult to change colors on, their move to the non-standard Unity desktop, the fact a total reinstall (or upgrade) was needed to get to a new version of the OS, sticking with specific kernel versions for 6 months at a time, and the fact that by default there isn’t a way on Debian based distributions to get a consistent and reliable dump of daemon status (ie rc-status on Gentoo or rc.d list on Arch) – this is ground level Unix functionality and Ubuntu just plain doesn’t provide a workable solution! These shortcomings led me to look elsewhere after a few years.

One major advantage in to Gentoo, for my needs, is a that it’s a rolling release distribution. This means that kernel, system libraries, the tool chain and applications are updated in place over time, without requiring a wholesale reinstall or upgrade process like typical binary release distros. When you have 10 machines, this becomes a major benefit in keeping them in sync in terms of OS and software versions.

I actually went back to Gentoo for a bit on one system, only to find the same old issues.

Enter Arch Linux

On a whim one night I was perusing Distrowatch, and stumbled upon Arch Linux. Arch is one of the few rolling release distributions that is binary based (minimal compiling for upgrades) and isn’t based on Gentoo Linux. It’s billed as a simple, lightweight Linux, and it definately lives up to its name. Arch has an active user community, has a large and regularly updated core and community maintained package base, and offers the ability, similar to Gentoo, to compile thousands of other packages from the Arch User repository. There is that word again – compiling. But honestly, I have had to do very little of it.

Don’t let the simple, lightweight, moniker steer you aware, Arch is capable of running the most advanced server functions and the latest desktop environments. The lightweight and simple is in maintaining the system. Most system configuration is handled in one file – /etc/rc.conf. The daemon startup order is specified by one line in rc.conf – the order of them named is the order they will start! Imagine that – no messing with symlinks in /etc/init.d anymore.

Package management is simple, kernel updates and initrd images are handled automatically, and support for diskless machines and thin client build processes is second to none.

What’s not to love?

I am sure that their are individuals who would find Arch too much work, and frankly Ubuntu is probably best for them. Linux wouldn’t be Linux with a choice of distros. Its one of its key selling points for me.

Coming up next – more system administration hints for Arch Linux!

Got another distro that you like – leave a comment and let me know!

Ubuntu Diskless Workstation and X-Terminal Howto – Part 3 X-Terminal Setup

At this point in the series we should have a working diskless workstation. Basically we have a full desktop running from an NFS share. This article will explore how to use the server to deliver a desktop, instead of our old laptop. In my case my server is much more powerful in terms of CPU and RAM than an old laptop, so the best linux experience will come from running applications on the server with the output showing on the laptop, which is the definition of an X-Terminal.

There are two steps to doing this – preparing the server and preparing the client.

There is also some tuning options that we can do with the terminal to make sure we get the best performance from the diskless that we can.

So, lets dive into the server preparation. Ubuntu 9.10 and up have a newer GDM that doesn’t have the gdmsetup configuration program available any more. So, we need to edit a file to turn on something called XDMCP. Our server is running Ubuntu 10.10, and officially XDMCP is not supported on this version, however it can be made to work. XDMCP is a method to allow an X server (in our case the diskless) to pull up a logon window from the server.

Basically a single file needs to be edited to allow xdmcp support. That file is /etc/gdm/custom.conf. A section called xdmcp is added to this file to enable the feature:


Once that section is added you can restart gdm with the following command:

sudo /etc/init.d/gdm restart

You can verify that the server is listening on UDP 177 with the following command:

netstat -uan

and look for the following entry in the command output:

udp 0 0*

One thing to note, this may not work properly on Ubuntu 10.10 if you have IPv6 enabled. You can disable IPv6 by adding ipv6.disable=1 to the kernel command line. This can be done by editing /etc/default/grub and modifying the line beginning with GRUB_CMDLINE_LINUX_DEFAULT=", adding the parameter to look something like this:

GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1 quiet splash"

Then run the following command on the server:

sudo update-grub2

Then reboot the server to disable IPv6. Unfortunately this will remove IPv6 support from the system, so if your relying on IPv6 – beware.

Ok – now lets move on the to the diskless. Lets make sure its booted up and showing the logon screen for Ubuntu. When the screen is up, drop to a command line by hitting Ctrl-Alt-F1. Logon to the system from the command line (hopefully you created a userid on your diskless when you built it).

Now lets bring down the gdm service so we can start X manually:

sudo /etc/init.d/gdm stop

Once gdm is stopped, we can try to use XDMCP to pull up the logon window from the server:

X -query "server IP address or hostname"

This should bring up a logon window from the server. If it does, then we can move on to the next step – if it doesn’t verify the server is configured properly as mentioned above. You can check the kernel command line on the server by running:

cat /proc/cmdline

You can try to login to the server if the window comes up. To continue, you will probably want to drop to the console again by hitting ctrl-alt-f2 and killing X with killall X. This should kill the X server.

In my normal setup I set Ubuntu to boot into console mode without GDM. This allows me to log on, and run a script that starts the X server and pulls the logon window from the server.

To set the diskless to boot to a text console, we need to modify the boot file used to PXE boot the diskless. In a previous article we showed you how to set up a PXE boot file for the diskless. Let’s pull that file up again and make the change we need. Basically the simple addition of a the kernel parameter “text” is needed. For example:

APPEND root=/dev/nfs rw initrd=initrd.img-2.6.35-24-generic.nfs nfsroot= ip=x.x.x.x:y.y.y.y:g.g.g.g: text acpi=force

The parameter is the second to the last one (before acpi=force). Adding this to your PXE boot file (or to a grub command line) will make ubuntu default to non-graphical boot. The next thing we need is a script to start X and use xdmcp to pull up the logon window from the server. Here is an example:

X -dpi 100 -ac -query

There are a couple of extra parameters in this script than what we used before. The first one forces the DPI to be 100, which will keep the desktop font size looking normal, and the -ac, which disables access control. Disabling access control still allows local X client programs such as a video player to run on the same desktop that is being pulled from the server. Running multimedia or OpenGL based software over the X protocol across the network is a less than optimal experience, so those should be run locally. More on that later. Lets save the command above in a script on the diskless called startterm. Make sure you make it executable and save it somewhere in your path.

The process will be to boot the diskless, logon as your user id, and run the startterm script, and you will be able to logon to your server.

Improving the Experience

We should have a diskless workstation set up as an X-Terminal. You should be able to use anything on the server you want – minus multimedia/sound apps. They will work but the video in mplayer, for example, will be less than adequate and there won’t be any sound unless you are near the server – because sound will still come from the server’s sound system!

Network sound is actually pretty easy to setup, but it requires some more tweaking on the diskless workstation – Enter Pulse audio system mode. Pulseaudio is the default sound server used on Ubuntu. Its actually pretty powerful, once you understand how to use it. Normally Pulseaudio is configured in per user mode, and is started when a user logs into gnome. Since we aren’t starting gnome on the diskless workstation (its running on the server) we need to reconfigure Pulseaudio to work in system mode, so its always running. Pulseaudio system mode is actually frowned upon in most scenarios, but this is one use case where it makes sense.

First we need to edit some configuration files to enable Pulseaudio system mode. They are located in the /etc/pulse directory. The first one is daemon.conf – edit it and look for the section that looks like this (its near the beginning):

; daemonize = no
; fail = yes
; allow-module-loading = yes
; allow-exit = yes
; use-pid-file = yes
; system-instance = no
; enable-shm = yes
; shm-size-bytes = 0 # setting this 0 will use the system-default, usually 64 MiB

And modify it to look like this:

daemonize = yes
; fail = yes
allow-module-loading = no
allow-exit = no
; use-pid-file = yes
system-instance = yes
enable-shm = no
; shm-size-bytes = 0 # setting this 0 will use the system-default, usually 64 MiB

The last file is /etc/pulse/system.pa – edit the file and add this to the bottom of the file:

### Enable TCP access to local sound devices
load-module module-native-protocol-tcp auth-anonymous=1
load-module module-zeroconf-publish

After these changes, you should reboot the diskless workstation to enable Pulseaudio in system mode, or you can run the following command to restart it:

sudo /etc/init.d/pulseaudio restart

If you restart the diskless, log on, and use the startterm script to log back into the server, you should here the ubuntu logon sound coming from your diskless speakers. If you don’t here are some things to check:

1. Log on locally with sudo /etc/init.d/gdm start instead of running startterm and make sure your audio system works.
2. Install padevchooser on the server, with sudo apt-get install padevchooser and run it from the diskless while logged into the server. It will place an icon in the menu bar. Right clicking on it should allow you select the sink that reflects your diskless – it will look like something pulse@

In our configuration we have enabled zeroconf so if the diskless is on the same subnet, it should just work out of the box, so to speak.

One more thing – you may need to add your diskless user that you logon to the system with to the pulse and pulse-access groups. This is important when running local applications that use Pulseaudio (see next section for details). Our user is term1 and this is how to add term1 to the groups needed:

usermod -a -G pulse,pulse-access term1

Local Applications

As I mentioned above, local applications can be handy for things that don’t run well over the X protocol across the network. Video is one of those applications. Luckily, its possible to create icons on the servers gui interface (maybe as part of the gnome ui) that actually start and run applications on the diskless workstation itself. The process is pretty simple. Essentially its an ssh session from the server to the diskless that starts an application with the display redirected to the diskless. Remember the -ac parameter in the startterm script? This is why its needed. -ac disables access control with allows this type of functionality to work properly.

Here is an example of a script that starts mythtv on the diskless:

/usr/bin/ssh term1@diskless1 /home/term1/bin/mythlocal

This script resides on the server – it starts and ssh to the diskless and then calls another script on the diskless call mythlocal. Here is what mythlocal looks like:

DISPLAY=localhost:0.0 mythfrontend -l /home/term1/mythfrontend.log -v important,general

Basically all this does is set the DISPLAY environment variable to the local X console on the diskless, and then start mythfrontend with options to log myth events to a log file. After a few seconds myth will start on the diskless and use the diskless video hardware to display Mythtv.

Since we are using Pulseaudio on the diskless it’s important to enable Pulseaudio output on Mythtv as well.

There is a caveat to making this work – the diskless must be configured to use key based ssh authentication, instead of password based. If you don’t do this, you won’t see the password prompt generated by the ssh session, and therefore Myth will never get started on the diskless.

Setting up ssh key based authenication is beyond the scope of this article – click here for a good place to start with this. I will attempt to get a future article together on this at a later date.

Tuning Performance

NFS isn’t nearly as efficient as local disk, so the less we write to the nfs based root file system, the better. One thing we can do is sent system log traffic to another form of storage. In our case, we will use tmpfs, which is temporary storage in the diskless computer’s memory. This means that system log files will not survive a reboot (unless you customize the system to dump them to the nfs root fs during reboot).

Essentially we are just going to redirect the /var/log /tmp and /var/tmp directories to tmpfs. This is done by adding the following to the /etc/fstab on the diskless:

tmpfs /var/log tmpfs defaults 0 0
tmpfs /tmp tmpfs defaults 0 0
tmpfs /var/tmp tmpfs defaults 0 0

After rebooting the diskless, all the system logger traffic will go to tmpfs and it should allow the diskless to be a little more responsive.

Well, I think that should about cover it. Comments and other suggestions are welcome as always.

Next up – Ubuntu vs Gentoo vs Arch Linux – I started out with RedHat 4 back in 1996, and moved to Gentoo in 2004, then to Ubuntu in 2008 – I have recently been playing around with Gentoo (again) and Arch Linux, and I may have found my next “favorite” linux distro – Arch Linux.

I also have tested debootstrap, which is a way of building a diskless filesystem on the server without installing ubuntu on the diskless and copying it across. I will see if I can get a quick article out on that process as well. Stay tuned!

Ubuntu Diskless Workstation and X-Terminal Howto – Part 2 – Preparing the diskless image

Well, after a short break, we are back to our diskless X-Terminal Howto. Today we will cover building the actual diskless image that will be used to boot the X-Terminal. Using ubuntu, the easiest way to do this is to simply install ubuntu on the machine that will be used for the diskless terminal. You can also use a virtual machine to build the image, however this will probably have some interesting issues regarding udev and device detection.

So, basically we be installing ubuntu on the machine itself, and use that install to make an nfs bootable image. So, go ahead and download an ubuntu install cd and have at it. A base install is usually sufficient with working X-Server, network devices etc. You don’t need to install a bunch of applications since those will be served by the server the X-Terminal will attach to. You should make sure you have all the latest updates applied to your install. I also recommend installing the nfs-kernel-server package as well with the command:

apt-get install nfs-kernel-server

Once you have ubuntu installed on your target machine, there are a couple of things we need to do to prepare for making the image NFS bootable.

The first thing we are going to do is modify the initial ram filesystem image to be better suited for network booting. To do this, open a terminal and change to the following directory :

cd /etc/initramfs-tools

We will edit the initramfs.conf file in this directory:

sudo pico initramfs.conf

There are 3 lines in this file that we will want to change. They are shown below:


Change these lines to read:


Diskless workstations typically require standard wired ethernet connections to work properly. There are some alternatives for wireless, but those are for another article.

After making the changes to this file we will need to regenerate the initial ram disk image for the current kernel. This image will be loaded instead of the standard ubuntu initial ramdisk to allow network booting. Here is the command for creating a new image:

sudo mkinitramfs -o /boot/initrd.img-2.6.35-22-generic.nfs 2.6.35-22-generic

Note the .nfs on the end of the filename (initrd.img-2.6.35-22-generic.nfs) make sure this filename is based on the current kernel version, that can be determined with the command uname -a. The last part of the command is the kernel version to make the initial ramdisk image for.

Here is a sample from a box running ubuntu 10.10:

default@ubuntu-1010:~$ uname -a
Linux ubuntu-1010 2.6.35-22-generic #35-Ubuntu SMP Sat Oct 16 20:36:48 UTC 2010 i686 GNU/Linux
default@ubuntu-1010:~$ sudo mkinitramfs -o /boot/initrd.img-2.6.35-22-generic.nfs 2.6.35-22-generic

The next couple of modifications will have to wait until we have copied the image to the server. So, lets do that next. The best way to copy the image to the server is to mount the servers shared directory that we created in the last article.

The directory was /srv/export/netboot/diskless1. So, lets mount that directory on your workstation. First, lets create a mount point:

sudo mkdir -p /mnt/server

Then we can mount the directory:

sudo mount -t nfs :/srv/netboot/diskless1 /mnt/server

After mounting the directory change to the root of the workstation’s root filesystem and copy the filesystem to the server:

cd /
sudo cp -Rax * /mnt/server

The cp command switches mean copy recursively – ie include all subdirectories (-R) and preserve permissions (a) and limit the copy to one filesystem (x).

After the root file system image is copied from the workstation to the server, we will have to configure the pxelinux command/boot file for the diskless image, as well as move the initrd and kernel image files to the server’s startup_files directory we created in the last article.

First, we will configure the pxelinux command/boot file for the diskless image. This file is a small text file that tells PXE the root file system path, and kernel parameters used for booting the diskless workstation. This file also contains IP autoconfiguration parameters that tell the linux kernel how to configure its network interfaces. Generally, this is handled through dhcp. In ubuntu 10.04 and 10.10 it seemd that in order to boot the diskless reliably, PXE will boot and get the dhcp reservation we created in the first article. Afterwards, the kernel will use hard coded IP address parameters in the kernel command line that match the dhcp reservation the was setup for the workstation. This may not make sense now, but it will soon.

This command/boot file must be saved in a particular location and with a particular name in order for PXE to find it. Essentially, the file needs to be saved in the following directory (if you have been following my server setup examples):


The name of the file is the hexadecimal equivalent of the workstations IP address. So, if you used as the IP address reservation on the dhcp server, then the file name will be called:


This is equivalent to 192(C0).168(A8).0(00).127(7F). You can use the Ubuntu calculator to determine the hex equivalent of your diskless workstations ip.

For our example, the workstation IP will be, the IP of the NFS server will be, and the path to the diskless root file system will be: /srv/netboot/diskless1.

So, lets look at a sample boot file for the diskless:

DEFAULT /vmlinuz-2.6.35-22-generic
APPEND root=/dev/nfs rw initrd=initrd.img-2.6.35-22-generic.nfs nfsroot= ip= acpi=force

The first line specifies the kernel image to use. This is tftp’d from the server’s startup_files directory by the diskless workstation at boot. The second line specifies the kernel command line. The root= parameter specifies that the root filesystem will be NFS mounted. The rw means the root filesystem will be read/write. The initrd= parameter specifies the initial ramdisk image we created earlier in this article. The nfsroot parameter specifies the nfs servers path to the root filesystem. The last two parameters, ip= specifies the IP address to use on the LAN interface when the kernel starts. This in the past used to be ip=dhcp. For some reason, in my environment this seems to have stopped working with the latest ubuntu 10.04 kernel and 10.10 kernels. The ip= parameter specifies the IP address of the workstation, the ip of the nfs server, the default gateway, and the network mask, respectively. The :::none are for other unused parameters. Lastly, the acpi=force is for systems where acpi is detected properly. Many older older machines need this parameter to function properly.

Make sure your file includes the right parameter for the ip=, kernel image name, and initrd name. For reference, the kernel image is located in /boot on the workstations image, and the initial ram disk image is what we created earlier in this article.

I will update the article if I can figure out what caused ip=dhcp to stop working properly.

Anyway, we have two more changes to make on the disk image to get a successful boot. The first one is to tell the diskless workstation not to run the normal network configuration. Since our network interface will already be up as part of the PXE process, if the Ubuntu NetworkManager attempts to configure the network stack, we will loose connection to the root filesystem on NFS, causing the kernel init process to fail. So, basically we will tell NetworkManager not to touch the LAN interface by modifying the /etc/network/interfaces file.

If we are using eth0 as our network connection on the diskless terminal, then use the following interfaces file:

auto lo
iface lo inet loopback

iface eth0 inet manual

The last thing we will have to modify is the /etc/fstab file. This file is responsible for telling the Linux kernel which devices to use for the needed file systems on the system. In our case we will simply modify the entry for the root file system.

Normally the entry for the root file system looks something like this:

# / was on /dev/sda1 during installation
UUID=50e370fd-eaf8-486a-adf0-ddb5fe1fb9be /               ext4    errors=remount-ro 0       1

We will change this entry to look like this:

/dev/nfs        /               nfs     defaults    	1       1

This basically tells the kernel that the root file system is nfs mounted.

NOTE: If you intend to leave a hard drive in the diskless workstation, the install of Ubuntu will have set up a swap partition. If you have a machine with low memory, its recommend that you leave the drive in and let Ubuntu detect the swap partition as it boots from the network. If you have 1GB of RAM or more, you will probably be safe removing the hard disk altogether.

The next step is to attempt to boot the diskless workstation from the network. Essentially this usually requires telling the PC’s BIOS to boot from a network device as the first device to attempt.

Usually this is in the BIOS setup. It will vary by PC. Usually this is referred to as Network boot or setting the NIC as the first startup device in your PC’s boot order.

So if you are ready, set your PC to boot from the network, make sure its connected and reboot. If all goes well you will see a screen that looks like this:

If something doesn’t work right, check over your configuration or send me a comment and I try to help. Otherwise your PC should come up into an Ubuntu Desktop logon screen as if it was running from an internal disk drive.

Next article is using the workstation as an X-Terminal for a server (rather than a standalone desktop) and some tuning for performance.

Until next time…

Ubuntu Diskless Workstation and X-Terminal Howto – Part 1 – Preparing the Server

Why a diskless workstation? In today’s day and age, this is a good question. My take on it is that I have a multiuser operating system (Linux) running on a large machine (Dual Xeon 5160 with 12 GB RAM) that could be used from several locations in the house by deploying X-Terminals. Rather than spending money on X-Terminals, why not use an old laptop or desktop machine to access the servers gui, applications and resources? That is my reasoning, and I also add some local application support for things not suitable for the X protocol over the network, such as Video, so while the machine is diskless, it isn’t just a dumb display device.

Preparing the Server

Lets get started with preparing the server side of the diskless environment.  For our purposes we will be using a traditional NFS based environment for running the diskless workstation.  This process should work on Ubuntu Lucid or Karmic Server or Desktop Installations.

Several things need to be installed on the server:

  • tftp server
  • dhcp server
  • nfs server
  • pxelinux.0 and startup files

Install the tftp sever, dhcp server and nfs server with the following command:

sudo apt-get install atftpd && sudo apt-get install dhcp3-server && sudo apt-get install nfs-kernel-server

You will also need to download the pxelinux.0 file.  You can get that file here:


For our example, we will need to create a directory to house our diskless file systems, and pxelinux startup files.  We will use /srv/netboot.  Create this directory on your server with the following command:

sudo mkdir /srv/netboot

In this directory, lets create the following subdirectories:

sudo mkdir /srv/netboot/diskless1

“diskless1” is where the root filesystem of our diskless machine will live.

sudo mkdir /srv/netboot/startup_files

“startup_files” is where kernel images, pxe and initrd images will reside for booting diskless workstations – this is similar to the /boot directory on a linux system. The workstation will access these files via tftp.

Lets share the needed directories via NFS.  To do this, we need to edit the exports file on the server and add the following line:

/srv/netboot/diskless1    *(rw,sync,no_root_squash,no_subtree_check)

Then, tell the nfs server to refresh its export list with exportfs command:

sudo exportfs -avr

Next we will configure the tftp server, atftpd. The configuration file for atftpd is /etc/default/atftpd. We will customize ours to look like this:

#OPTIONS="--tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr --mcast-ttl 1 --maxthread 100 --verbose=5 /var/lib/tftpboot"
OPTIONS="--daemon --port 69 --tftpd-timeout 300 --retry-timeout 5     --mcast-port 1758 --mcast-addr --mcast-ttl 1 --maxthread 100 --verbose=5  /srv/netboot/startup_files"

The first two lines of the file are the default entries – we are going to comment them out and add the second two lines as shown above. After editing the file, restart the tftpd service with the command:

/etc/init.d/atftp restart

Next, we need to configure the DHCP server. Some custom options are required for the DHCP server to tell the diskless workstation how to retrieve its boot files. In our example, we will use a network, so the DHCP server configuration will look something like this (this also assumes that this is the only DHCP server for the network having more than one is bad news unless configured properly) assuming your DNS servers are and and your default router (route to the internet) is

# The ddns-updates-style parameter controls whether or not the server will
# attempt to do a DNS update when a lease is confirmed. We default to the
# behavior of the version 2 packages ('none', since DHCP v2 didn't
# have support for DDNS.)
ddns-update-style none;

# option definitions common to all supported networks...
option domain-name "rtwsecurenet.com";
option domain-name-servers,;
option ntp-servers;

default-lease-time 172800;
max-lease-time 259200;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.

# This is a very basic subnet declaration.
subnet netmask {
  option routers;

host diskless1  {
      hardware ethernet     00:04:76:3F:A6:36;
      next-server ;
      option root-path      "/srv/netboot/diskless1";
      filename              "/pxelinux.0";

In the example above I created a reservation for the diskless workstation. DHCP reservations are based on mac addresses. The hardware ethernet parameter above should be editted to reflect the diskless workstation’s ethernet card mac address. The fixed-address specifies the IP address that will be assigned to the diskless workstation. The last 3 parameters, called DHCP options, give booting information to the diskless workstation. These should be updated to reflect the appropriate server IP (next-server) and path (option root-path). Substitute your server’s IP address and path for your installation. The DHCP server that ships with Ubuntu linux is the only one that I am aware of that supports per reservation DHCP options such as these. Be sure to also change the DNS server settings the in the dhcp.conf file to the servers for your site/network.

The last part of preparing the server is the configuration of pxelinux and the diskless boot files.

Configuring PXE

The first thing we need to do is take the pxelinux.0 file you downloaded above and copy it to the /srv/netboot/startup_files directory we created above.  Then, in the /srv/netboot/startup_files directory, make the following directory (as root or using sudo):

mkdir pxelinux.cfg

The pxelinux.cfg directory will contain the pxelinux boot configuration file for each diskless workstation.  We will configure these files after we build our diskless workstation image.

This should about wrap it up for the server configuration piece of the diskless workstation setup.  The next part will cover the building of a diskless workstation image from a standard ubuntu-desktop installation.

Stay tuned!

NVIDIA 256.53 Drivers and older Games

Its time for a quick howto – for some time I have been sticking with older NVIDIA driver releases on my Ubuntu machines do to a buffer overrun error on some older games when using driver sets newer than 185.18.36. The interesting thing is that equivalent Windows drivers exhibited the same problem, except that eventually the issue was fixed on Windows. After doing some searching I came across the answer to fix this issue – thanks to Conky on nvnews.net.

The problem stems from the newer drivers reporting a GL Extension Version String that is too long for some older programs to process, thus resulting in the buffer overrun. To fix this, NVIDIA included an environment variable that can be set to report older (and thus shorter) version strings.

The game I was having the most trouble with was Call of Duty 1. I know, why would I want to play such an old version? I am a COD addict and still play COD1 all the way up to the current COD:MW2. Under linux, COD1 must be run using Wine. Normally cd’ing to the Call of Duty Directory and running the following command will start the game:

wine CoDSP.exe

However, using NVIDIA 256.53 (or anything beyond the 185.18.36 driver version) will result in the buffer overrun error. To fix this issue, change your startup command to the following:

__GL_ExtensionStringVersion=17700 wine CoDSP.exe

The __GL_ExtensionStringVersion is an environment variable that in this case is set to 17700. What this is doing it telling the NVIDIA driver to report the GL_ExtensionString as if the driver was from the 177.00 series. This effectively eliminates the overrun error and allows the game to start.

This issue, is mentioned in the NVIDIA driver readme file (here is the snippet), as was kindly pointed out by Conky over at nvnews.net:

Some applications, such as Quake 3, crash after querying the OpenGL extension string

Some applications have bugs that are triggered when the extension string is longer than a certain size. As more features are added to the driver, the length of this string increases and can trigger these sorts of bugs.

You can limit the extensions listed in the OpenGL extension string to the ones that appeared in a particular version of the driver by setting the __GL_ExtensionStringVersion environment variable to a particular version number. For example,

__GL_ExtensionStringVersion=17700 quake3

will run Quake 3 with the extension string that appeared in the 177.* driver series. Limiting the size of the extension string can work around this sort of application bug.
—End Quote—

So, there you have it. I learned that you’re never “too good” read the readme files.

Dealing with multiple USB sound devices

Ever been plagued by your linux system detecting your webcam as the first sound device instead of your USB headset?  This is a common problem with Linux, but can be easily remedied with editing your systems module configuration.  On Ubuntu, the important file is /etc/modprobe.d/alsa-base.conf.  This file controls the module settings and parameters for the alsa sound system.  First, we need to understand what to put in this file.

With both of your USB devices connected to the system, run the following command:


Here is a sample output:

Bus 001 Device 008: ID 046d:09a2 Logitech, Inc. QuickCam Communicate Deluxe/S7500
Bus 001 Device 005: ID 03f0:2f24 Hewlett-Packard
Bus 001 Device 007: ID 058f:6362 Alcor Micro Corp. Hi-Speed 21-in-1 Flash Card Reader/Writer (Internal/External)
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 005: ID 051d:0002 American Power Conversion Uninterruptible Power Supply
Bus 002 Device 004: ID 10f5:0200 Turtle Beach Audio Advantage Roadie
Bus 002 Device 003: ID f617:0905
Bus 002 Device 002: ID 046d:c521 Logitech, Inc. MX620 Laser Cordless Mouse
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

This output shows all the USB devices connected to your system.  In our example, the two devices we are looking for are:

Bus 001 Device 008: ID 046d:09a2 Logitech, Inc. QuickCam Communicate Deluxe/S7500

Bus 002 Device 004: ID 10f5:0200 Turtle Beach Audio Advantage Roadie

Frequently, on this system, the webcam steals the default input device because of its internal mic.  I would like the default input/output to be assigned to the turtle beach roadie USB headset.  The key here is something that all alsa modules have support for which is an index parameter.  All also modules allow indexes to be assigned to a particular device.  This works pretty well when your cards use different alsa modules.  Simply specifiying module options setting the index # for your particular devices module resolves the issue.  So what happens with USB audio devices, since they use the same snd-usb-audio module?  It seems that linux simply assigns indexes in the order the devices are discovered.

The lsusb output above shows two important pieces of information for each of the sound devices in question.  The USB VID (Vendor ID) and PID (Product ID) can be used to control index assignment.  In our case these values are:

046d:09a2 for the Logitech, and 10f5:0200 for the Turtle Beach headset.  The set of hexadecimal numbers is the Vendor ID and Product ID for the device, in the format VID:PID.

The VID:PID combination can be used in conjuction with the Index value to control the way linux sees the devices.  The format for the module option is:

options snd-usb-audio index=<x>,<y> vid=0x<vid for index x>,0x<vid for index y> pid=0x<pid for index x>,0x<pid for index y>

So, with this in mind, all we have to add to the /etc/modprobe.d/alsa-default.conf file is this:

options snd-usb-audio index=0,1 vid=0x10f5,0x046d pid=0x0200,0x09a2

After rebooting the system, we should never see our Webcam grab the default input again (making us have to go to the Pulse Audio settings and reconfigure the sound system).

Ubuntu Apache LAMP Server Quick Howto – Part 2 – SSL Websites

Hi – Welcome back to the Ubuntu Apache LAMP Server Quick Howto.  This second part of the series will cover configuring SSL for use with Apache for securing website content.  SSL stands for Secure Sockets Layer, and is a method of providing transport layer encryption or TLS (transport layer security).  SSL typically uses X.509 digital certificates to perform authentication and encryption functions.   X.509 is based on public key infrastructure or PKI.  PKI is beyond the scope of this article (sounds like a good topic for a future article), but we will at least create a self-signed certificate for instructional purposes here.

SSL adds a few caveats to Apache configuration.  One of those is name based virtual hosting.  SSL does not allow the use of name based virtual hosts.  From the apache SSL/TLS Strong Encryption Guide:

“Why is it not possible to use Name-Based Virtual Hosting to identify different SSL virtual hosts?

Name-Based Virtual Hosting is a very popular method of identifying different virtual hosts. It allows you to use the same IP address and the same port number for many different sites. When people move on to SSL, it seems natural to assume that the same method can be used to have lots of different SSL virtual hosts on the same server.

It comes as rather a shock to learn that it is impossible.

The reason is that the SSL protocol is a separate layer which encapsulates the HTTP protocol. So the SSL session is a separate transaction, that takes place before the HTTP session has begun. The server receives an SSL request on IP address X and port Y (usually 443). Since the SSL request does not contain any Host: field, the server has no way to decide which SSL virtual host to use. Usually, it will just use the first one it finds, which matches the port and IP address specified”

My experience is that Apache will reject the configuration if you attempt to use named based virtual hosts with SSL.

So, with this in mind, if I have multiple “sites” to serve with SSL, I either use different IP addresses for each site, or I serve one site with multiple directories of content that can be referenced by http://<site fqdn>/directory.  For example, if I had a site called www.stuff.com and it used SSL, I would add content for different functions/site at https://www.stuff.com/systeminfo or http://www.stuff.com/shopping.

Anyway, so, how do we set up SSL?  First thing is to make sure its enabled on your LAMP server.  To turn on SSL use the following command:

sudo a2enmod ssl

followed by

sudo /etc/init.d/apache2 restart

Also – we should create a specific document root directory for the SSL Apache to use.  This way, if you have both SSL and non-SSL sites on the same server, not all the non-ssl sites will be published automatically by the SSL definition.

sudo mkdir /var/www-ssl

Creating a self signed certificate:

In order to use SSL with Apache you must have a certificate.  Here is how to make a self signed certificate.  Note:  Self Signed certs are fine for testing and lab servers, but are not recommended for production environments or internet facing servers.

First, you will need to create a private key file:

sudo openssl genrsa 1024 > host.key
sudo chmod 400 host.key

Next, you will need to create the certificate:

sudo openssl req -new -x509 -nodes -sha1 -days 365 -key host.key > host.cert

After hitting enter, you will be prompted for several pieces of information that will be encoded into the certificate.  Here is an example output:

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Kansas
Locality Name (eg, city) []:Topeka
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Test.com
Organizational Unit Name (eg, section) []:IT
Common Name (eg, YOUR name) []:www.test.com
Email Address []:admin@test.com

Note, the Common Name should be the fully qualified domain name of your web server.  Your cert and key will be in the current directory, and will be called host.cert (cert) and host.key (key).  Both of these files will be needed for the next step.  As root or using sudo, copy the key file to the /etc/ssl/private directory on your LAMP server, and copy the cert file (host.cert) to the /etc/ssl/certs directory.

Next go to the /etc/apache2/sites-available and edit the default-ssl file.  This file controls the default SSL virtual host configuration.  We will be modifying the following some the lines in the file.  As root, find the beginning stanza in the file (with your favorite editor) and it will look something like this:

<IfModule mod_ssl.c>
<VirtualHost _default_:443>
            ServerAdmin webmaster@localhost

            DocumentRoot /var/www
            <Directory />
                         Options FollowSymLinks
                         AllowOverride None
            <Directory /var/www>
                        Options Indexes FollowSymLinks MultiViews
                        AllowOverride None
                        Order allow,deny
                        allow from all

Change the webmaster@localhost to an email address of the person responsible for the server (this isn’t required, but is probably good practice).

Also, you should probably add a ServerName directive right after the ServerAdmin directive.  The ServerName directive should reflect the fully qualified domain name as the server.

The DocumentRoot directive should be changed to /var/www-ssl – the directory we made earlier.  Lastly, the Directory /var/www directive should be changed to Directory /var/www-ssl.

Here is the same code snip, modified for a server called www.test.com:

<IfModule mod_ssl.c>
<VirtualHost _default_:443>
           ServerAdmin admin@test.com
           ServerName www.test.com
           DocumentRoot /var/www-ssl
           <Directory />
                  Options FollowSymLinks
                  AllowOverride None
           <Directory /var/www-ssl>
                  Options Indexes FollowSymLinks MultiViews
                  AllowOverride None
                  Order allow,deny
                  allow from all

Next, lets find the following couple of lines:

SSLCertificateFile    /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key

These lines tell Apache where to find the SSL certificate to use for the site.  We are going to replace these with the certificate and private key we created earlier.  We will change them to read:

SSLCertificateFile    /etc/ssl/certs/host.cert
SSLCertificateKeyFile /etc/ssl/private/host.key

Ok – make sure you save your default-ssl file with the changes we have detailed above.  After the file is saved, run the following command:

a2ensite default-ssl

followed by:

/etc/init.d/apache2 restart

Now, its time to test to see if you can connect to the server with SSL. You might want to create an index.html file at the /var/www-ssl directory first though. To do this simply create a file with “It works, SSL!” in it as root and save it to the /var/www-ssl directory with the name of index.html. Afterwards, open Firefox, and go to https://<server name or IP address>. You probably get a certificate error, since we created a self-signed cert for our tutorial. You should tell Firefox to trust the certificate. If all goes well, you should see: It works, SSL! at the top of the page.

If all went well, we should have a working Apache server with SSL. At this point, you can place content directly under /var/www-ssl or in folders under that directory. If you have some folders under the non-ssl Document root (normally /var/www) simply sim-linking those directories to /var/www-ssl will publish them via SSL as well.

So, what if it doesn’t work? Generally, this is the result of either SSL not being enabled, or an errror in your default-ssl file. Double check that you have enabled SSL with the a2enmod command, and make sure the default-ssl file is correct, and that it has been enabled with the a2ensite command. Also note, that when enabling SSL for the first time, make sure you restart Apache with the command:

/etc/init.d/apache2 restart

To make sure SSL support is enabled. You can check that Apache is listening for SSL connection by doing a

netstat -tan

and looking for an entry like this:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address     State
tcp        0      0   *           LISTEN


If that line is not present, then double check your configuration.

Next time we will focus on the Mysql portion of a LAMP server – look for it soon.