Sunday, December 9, 2007

OpenBSD on Soekris -- A Cheater's Guide

I've been using Soekris devices for quite some time. Basically any time I need to get some routing, firewalling, skullduggery, etc, that doesn't require serious CPU, I toss a Soekris box at the problem. They are great little devices -- low power, dead quiet and rock solid.

The obvious downside of a system like the Soekris is the wimpy CPU. This really is only an issue during installation and during the initial system configuration. After that, the box is a real work horse.

Below are the steps I recently used to get my NET4801 running OpenBSD 4.2 -current. The difference here is that I use qemu to make use of the considerably faster CPU on my desktop to breeze through the install and initial configuration.

  1. Download install42.iso from your local mirror
  2. Plug in your CF card that you'll use in your Soekris. Take note of what device it gets assigned
  3. Start qemu, replacing /dev/sdb with whatever device your CF is:
    qemu -hda /dev/sdb -cdrom install42.iso -boot d
  4. Install as usual. Configure your interface to use DHCP, as anything else won't work inside qemu. Set default console to com0 and set the speed to match your Soekris (9600)
  5. Finish installation. Halt. Stop qemu. Restart without the iso:
    qemu -hda /dev/sdb
  6. Once booted, edit /etc/fstab so that / is mounted with noatime, read-only. My /etc/fstab looks like this:
    /dev/wd0a / ffs ro,noatime 1 1
  7. Now put the volatile stuff into MFS so you won't wear out your CF too fast. Create an MFS directory for /var:
    mkdir /mfs
    cp -rp /var /mfs/var
  8. Similarly for /dev:
    mkdir /mfs/dev
    cp /dev/MAKEDEV /mfs/dev
    cd /mfs/dev
    ./MAKEDEV all
  9. Add the appropriate lines to /etc/fstab to ensure that /dev and /var get mounted as MFS at boot. Change values for -s and -i as you feel necessary. This works for me a on 1G CF:
    swap /var mfs rw,-P=/mfs/var,-s=32768,noexec,nosuid 0 0
    swap /dev mfs rw,-P=/mfs/dev,-s=8192,-i=128,noexec,nosuid 0 0
  10. Now symlink /tmp to /var/tmp so that temporary files can be written to:
    rm -Rf /tmp
    ln -s /var/tmp /tmp
  11. Install rsync to handle synchronizing /var. This assumes you've set $PKG_PATH to your favorite local mirror:
    pkg_add rsync
  12. Add a cronjob to periodically sync any changes to /var. I prefer a weekly job. Add something like the following to root's crontab:
    1  0  */7  *  *  /usr/local/bin/rsync -az --delete /var/ /mfs/var/
  13. Finally, edit the shutdown script to sync any unsynchronized changes at shutdown time. Add the following to the end of /etc/rc.shutdown:
    /usr/local/bin/rsync -vaz --delete /var/ /mfs/var/

Thats it. Halt your OpenBSD installation, stop qemu and install the CF in your Soekris. Any further configuration can be done by way of sshd or the serial console, but don't forget the / is mounted read-only, so don't forget to mount it read-write if you need to change something.



laurent said...


Nice guide. One question for you though - as I understand it, /var is being synced back to /mfs/var, however /mfs resides in / which has been mounted read-only. How is it that you sync back to a read-only filesystem?


Jon Hart said...


I read your comment and said to myself "It just works". Then I went and took a look at my Soekris and, sure enough, / is mounted rw, however that is probably because I've needed to update my pf.conf a few times since last boot.

I am not sure how this post has gone this long without me noticing this slight discrepancy.

The catch is this point:

Most volatile stuff lives on /var and /dev, which is mounted rw in memory. The rsync in rc.shutdown and cron is designed to keep this in sync with what is in /mfs so that it can be repopulate at boot.

However the problem is that the cronjob and rc.shutdown should temporarily remount / rw to succeed, because with / mounted ro they will certainly fail:

# mount -o ro /
# mount
/dev/wd0a on / type ffs (local, noatime, read-only)
mfs:2518 on /var type mfs (asynchronous, local, noexec, nosuid, size=32768 512-blocks)
mfs:25922 on /dev type mfs (asynchronous, local, noexec, nosuid, size=8192 512-blocks)
# touch /var/blah
# touch /mfs/var/blah
touch: /mfs/var/blah: Read-only file system

Thanks for pointing this out. If I am correct in my assumptions, I'll update this post accordingly.