Installing Scientific Linux 5
on a _4 Watt_ ALIX 2D3 Single Board Computer
The PC Engines ALIX is a single board computer based on the AMD Geode LX800 X86-compatible CPU. The 2D3 version of the board that I am using has 3 ethernet ports, a serial console port, two USB connections, and internal sockets for a compact flash card, a PATA/IDE laptop drive, and a miniPCI card. The board is low power ( less than 4 watts ) and cheap. The board plus a metal case plus a high efficiency AC Adapter costs less than $200 from Netgate.
Most people using these boards are running OpenWRT because of its router/wireless origins and small memory footprint. However, the ALIX has 256MB of RAM, and large Compact Flash cards are cheap ( I bought some 8GB Kingston cards for $20 ). There are only two real problems with the ALIX for installing a standard distro - the CF cards write /very/ slowly, and the ALIX does not have a display. I spent a couple of frustrating weeks attempting to get !OpenWRT running on the ALIX; many of the packages are poorly documented and inadequately tested. I was finally stopped because the udhcpc program would not play nice with my direct ethernet link to the Optical Network Terminal connected to Verizon FIOS.
I am using the ALIX board to replace a slow and power-hungry ancient laptop that is acting as my firewall/router, and running Scientific Linux 5. The Geode processor is not only faster, it has a built-in AES encryption engine; with the right kernel patches I expect it will run !OpenVPN links 3x faster using 10% of the power.
Most of the following instructions are pretty generic, and and can be adapted to any CD/DVD installed distro. However, please do not change this page for those distros - add a link and build a similar page for that distro.
Installing Scientific Linux 5.0 on a Compact Flash Card
I did not try to install from the SL5 install DVD directly to the ALIX. While that might be possible with some kind of !MiniPCI or USB connected display card, there are easier ways. I chose to pull the hard drive from a desktop PC with USB2 ports (USB1 is /too slow/), connect a compact flash card through a generic CF to USB adapter (do *not* use the one from Radio Shack - they melt) and followed a somewhat customized install procedure, outlined below.
Compact flash is a strange drive media - it writes slowly and it wears out after too many writes. So many of the things that are often done with hard drives - swap partitions, log files, /tmp - are turned off, sent to other machines, or redirected to RAM. Perhaps 3 partitions can be used as a redundant software RAID system to store more volatile information on the CF card. However, that would make the already slow write process 3X slower. USB storage - flash drives, external hard drives, etc - may be a better way to store larger amounts of volatile data.
Why install on a diskless desktop?
Ignorance and caution. The automated install assumes that the drive configuration during installation matches the drive configuration during boot, and sets up grub with the CF in whatever drive order it was assigned with the other disk there. Quite often, this messes up the installed disk, or puts the wrong information on the drive being set up. Also, if you select the wrong drive during the install, you can wipe it. Better to start out safe. All my desktop drives are in slide-out swap trays, so for me it is a matter of a few seconds to make a machine diskless.
Partitioning without LVM
LVM makes moving a drive from machine to machine a little complicated. Rather than use the standard partitioning, I deleted all the standard LVM partitions and did a custom setup from scratch: Create custom layout.
I set up the Compact Flash "drive" with 4 primary partitions:
- a 200MB boot partition
- a 1.7GB root partition
- one cylinder as swap
- and the rest as a /spare partition.
I will not actually use the swap, but it makes the setup complain less. I will let the installer mkfs the spare, but I will remove it from /etc/fstab later.
Here is the partition table I ended up with on my "8GB" (7692 MB) card:
/dev/sda1 /boot ext3 200MB /dev/sda2 / ext3 1702MB /dev/sda3 /spare ext3 5786MB /dev/sda4 swap 5MB
Keep in mind that the bigger the area you actually use, the longer it takes to copy. I can read an 8GB CF card to disk in 20 minutes. I needed 6 hours to write the whole CF. A stripped version of SL5 fits nicely in 1.2 GB, so I am just wasting time if I use too much of the card for the portion I want to copy. I will be ignoring sda3 and sda4, so I don't have to copy those parts of the CF card to another CF card if I don't want to.
Selecting Packages
I unchecked all the boxes. I can use yum to add the specific server packages I need, later. However, you can also click "customize now" and laboriously select the packages you need (bind, openssh, dhcp, etc). Leave out everything that depends on X or a desktop (office apps, etc.). Install the distro. The bare bones install completed in 35 minutes - don't believe the "Remaining Time" calculation, because CF is a lot slower.
After the install, I wasted an hour removing the packages I didn't want. This is really not necessary, you can only save a few percent of space this way. My motivation was to reduce updates and dependencies, mostly, but it is surprising how interdependent these packages are. Being unfashionable, I also removed the ipv6 stuff. I can add that back in if Verizon FIOS ever supports it.
Rebooting after install
We have more configuration to do before we can boot this CF card on the ALIX.
My desktop would not boot from the CF card. So, I restarted the computer with the SL5 install DVD and booted with linux rescue. I enabled the network - that did not give me name service, but at least I could ssh to and scp from numeric IP addresses. The diskless desktop will boot up with the root partition of the cfcard in /mnt/sysimage . You can chroot /mnt/sysimage and then follow the instructions below.
You can also mount the CF card images on a running computer. CF cards will hotswap in an external CF-to-USB adapter. That may have advantages, as you have access to a more complete working environment, and can copy files locally.
Whichever modification environment you use, it is probably better to get the card limping along on the ALIX at the earliest stage possible. If you need to redo the steps because the card won't boot, it is better to have less investment in it.
Before installing in the ALIX: Editing /etc/fstab
The CF card in the USB adapter will have partitions sda1 through sda4. In the ALIX, the same partitions will be hda1 through hda4 . Fortunately, the partitions are labeled, and are referred to in /etc/fstab with the labels. You will still be editing /etc/fstab for flash memory friendliness, though.
First, remove the line for /spare , unless you really need it. Even if you are willing to put up with the extra hours copying to another CF card (and you will want a prepared spare CF card, it is cheap insurance), it also takes a while to do an fsck on.
Next, remove the line for /swap . You will typically be using less than 50MB of the 256MB on the ALIX, and flash memory is too slow and wears out too fast to use it for swap.
Third, add a line for a ramdisk /tmp . Here's mine:
tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0
Fourth, change your root and boot partitions to "noatime". This keeps the kernel from rewriting the "atime" attribute of a file when it is accessed. This is slow and wears out the flash. This may hypothetically break some programs, but I have not encountered a problem.
LABEL=/ / ext3 defaults 1 1 LABEL=/boot /boot ext3 defaults 1 2
becomes
LABEL=/ / ext3 defaults,noatime 1 1 LABEL=/boot /boot ext3 defaults,noatime 1 2
You are now more flash-friendly. Be aware that everything you store in tmp is volatile and will disappear with the next reboot.
Do not base your security on that, though - see this paper.
While the !RAMs are soldered down on the ALIX, the whole system, case and all, can be dunked in LN2 while it is running, then unplugged from power and taken to Dr. Horrible's secret lair, the CF card replaced with a dump program, and all your secrets extracted from the frozen memory. Bwa-Ha-Ha!
Before Installing in the ALIX: Preparing the Serial Console
The ALIX does not have a screen, but it does have a serial console port. This comes from the factory set for 38400 baud. You can increase the rate to 115200 baud (hit s during the memory test during the ALIX boot), but hey, 38400 is good enough, and brings back youthful memories of serial connections over modems. We will assume 38400 baud, 8N1 . To connect with the ALIX, I use a usb-to-serial adapter, a 9-pin-to-9-pin null modem adapter, and a 9-pin to 9-pin female-to-female adapter. Many computers and laptops have serial ports, and you can use a cable and a null-modem adapter to achieve the same effect. The typical serial port on a linux system is ttyS0, and with the usb-to-serial adapter, it is typically ttyUSB0 . You can use *minicom*, appropriately configured, to talk through a serial port. I set up both ends for a VT102 terminal, 24 lines of 80 characters.
You will eventually be using *ssh* to get to the ALIX, but it is easier to set up keys and such on the CF when it is running in the ALIX.
And now, to work! I adapted Allen Sanabria's instructions:
First, edit /etc/inittab, and add the line !co:2345:respawn:/sbin/agetty ttyS0 38400 vt102 in front of the mingetty lines:
# Run gettys in standard runlevels co:2345:respawn:/sbin/agetty ttyS0 38400 vt102 1:2345:respawn:/sbin/mingetty tty1 2:2345:respawn:/sbin/mingetty tty2
Second, add a console boot stanza for /etc/boot/grub.conf. *Turn off* the splash image, and perhaps the hidden menu:
before, as installed (your setup may look a little different):
...
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Scientific Linux (2.6.18-8.1.3.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-8.1.3.el5 ro root=LABEL=/1
initrd /initrd-2.6.18-8.1.3.el5.imgafter:
...
default=0
timeout=5
# splashimage=(hd0,0)/grub/splash.xpm.gz
# hiddenmenu
serial --unit=0 --speed=38400
terminal --timeout=5 serial console
title Scientific Linux (2.6.18-8.1.3.el5) console
root (hd0,0)
kernel /vmlinuz-2.6.18-8.1.3.el5 ro root=LABEL=/1 console=ttyS0,38400n8
initrd /initrd-2.6.18-8.1.3.el5.img
title Scientific Linux (2.6.18-8.1.3.el5)
root (hd0,0)
kernel /vmlinuz-2.6.18-8.1.3.el5 ro root=LABEL=/1
initrd /initrd-2.6.18-8.1.3.el5.imgYou may want to delete the second stanza; leaving it in doesn't hurt anything. I'm not sure how a kernel update will be configured, though, so you may have to re-edit this file after that happens, before your next reboot.
Before shutting down and moving the CF card to the ALIX
Most of the changes you just made are buffered in RAM on the installer desktop machine. You should run *sync* to write them out to the CF. Syncronization is also performed by the umount operation, which is run during shutdown. However, you might forget and pull the CF card. I would rather make sure the system saved my work.
Rebooting on ALIX - Further Configuration
The ALIX does not permit the CF card to be hotswapped, so only insert or remove the CF card when the ALIX is disconnected from the power source (either the wall wart, or low voltage Power Over Ethernet on the eth0 port).
Plug in the serial console, and connect your inner network (or whatever serves DHCP ) to the right-most ethernet socket on the ALIX. Start up minicom and configure it. I suggest 38400 8N1 VT102, all the modem dial stuff cleaned out, in an xterm sized like a VT102 with 80 columns and 24 ones.
Then plug the power into the ALIX, and watch it boot. It will take about one minute, about as long as it takes on a typical machine. If it does not boot, look at this page for SL5 and ALIX boot errors. The ALIX comes up in runlevel N 3 - networking but no X display.
The ALIX will do some kudzu craziness during startup, will try (and fail) to get DHCP info over ports eth1 and eth2, won't have a hostname, etc. After logging in (as root, of course), you can run ifconfig eth0 to find out what IP address got assigned to your ALIX by your network. You can use that address to log in with ssh (the standard SL5 load comes up with sshd already running.
If you don't have DHCP on your network, or for some other reason you don't already have an IP address, you can assign one in the console window with something resembling:
ifconfig eth0 192.168.1.66 netmask 255.255.255.0 gw 192.168.1.1 up
Change the numeric assignments above to match your network configuration.You can make the assignment permanent by editing /etc/sysconfig/network-scripts/ifcfg-eth0, similar to the way ifcfg-eth1 is edited below.
Once you are logged in (either with the console or with ssh), you fix all these problems using vi running on the ALIX (or whatever text editor your religion permits, if you remembered to install it).
The configuration chores
1) Edit the appropriate information into the /etc/sysconfig/network file
2) If you plan on spoofing the MAC address on the WAN port, edit /etc/sysconfig/network-scripts/ifcfg-eth0 to remove the HWADDR line
3) You will probably assign a fixed address to eth1 (the center ethernet socket), and use it as the gateway and DHCP server for your LAN network. Edit /etc/sysconfig/network-scripts/ifcfg-eth1 to look something like:
# VIA Technologies, Inc. VT6105M [Rhine-III] DEVICE=eth1 ONBOOT=yes BOOTPROTO=static PEERDNS=no IPADDR=192.168.53.1 NETMASK=255.255.255.0
4) Do the same to eth2 (the leftmost ethernet socket), and use that as the gateway and DHCP server for your DMZ network. I will be using that for public wifi, and will be controlling it with nocat.
MORE LATER
Little ALIX Hacks
Here are a few tricks I've found useful with the ALIX.
Spoofing a MAC address
If you are using Comcast or Verizon FIOS, your service is probably locked to a specific MAC address on a specific piece of hardware. If the CAT5 cable to your Verizon ONT or older cable modem runs to an ethernet port, the 48 bit MAC address of that port will not match the built-in MAC address of the ALIX WAN port (eth0 on the right side?). While the MAC address is set by the ALIX BIOS at boot time, it can be reset before the ALIX attempts to connect when it starts up eth0 . I added a small startup script, /etc/rc.d/init.d/macaddr, and used chkconfig to add it to the startup at S07macaddr before S10network brings up the ethernet ports and makes dhcp requests. Edit the file /etc/macaddr/eth0 (and eth1 and eth2) for your own desired MAC address. For example,
00:D0:03:19:12:3F
in /etc/macaddr/eth1 will assign that MAC hardware address to port eth1, regardless of the original address assigned by PC Engines to the ALIX.
If there is nothing in those files, the MAC addresses will be left alone.
Perhaps there is some automagical way to include the desired mac address in one of the standard files in SL5, so this is all unnecessary ... send me a clue!
A Perl script to simplify ifconfig
The ALIX has three ports, plus localhost. That is a lot to display with ifconfig in a 24 line console window. If you have a lot of VPN ports, or you added additional USB ethernet ports, or a wireless card, the output of ifconfig can run on for dozens of 10 line entries. I wrote a little Perl script, ifbr which presents a briefer version of ifconfig output. Download that from my website into another machine, and scp it into /usr/local/bin/ifbr on your ALIX machine,
then chmod 755 usr/local/bin/ifbr so it will execute.
After you have the CF card set up properly
CF cards are cheap - much cheaper than the time you invested doing all that setup. Make a spare CF card! You can use *dd* to copy the image off the CF card to a disk file on one of your other computers, and later you can use *dd* to write it onto another card. Again, if you fit everything important into partitions within the initial 2GB of the CF card, you can save time and space by only copying the first 2GB. With the original card in your trusty CF-to-USB adapter, type:
dd if=/dev/sda of=[ImageFileName] bs=1M count=2000
And you will only copy the contents of the first two partitions, and a little more. When you build the new card, you can use the /spare partition on /dev/sda3 (or /dev/hda if it is mounted) after you do a mkfs to set itup.
The ALIX board in its cute little red case. A higher resolution version of the picture is available here.
The backup battery was added after purchase. The C19 electrolytic capacitor should have been placed about 1.5mm to the left when the ALIX board was designed, I had to cut away some of the plastic on the battery holder to make it fit. Perhaps with a different holder, the battery can go in upside down (and the holder turned 180 degrees).
From the PC Engines site
[Questions? SL5AlixQuestions]
.
.
Under Construction ... More soon!
Last update 2011 January 22
Copyright 2011 Keith Lofstrom
Publication Rights Reserved - I hope to publish this in a Linux magazine.