tomclegg.net


Diary
Examples
  >256-router<
    adzap
    apache-double-reverse
    cacti-adodb-php4
    debian-quota
    diskonmodule
    dollarsperbyte
    dynip
    ezmlm-linux
    fbsdhabits
    freebsdclone
    macbook-quantal-sound
    maildirpop3d-awfulhak
    mandy
    md
    mrtg
    net-snmp
    nodefaultroute
    oracle9i
    oracle9i-bsd5
    oracle9i-client
    oracle9i-nat
    php-cgi
    php-commandline
    php-image
    php-kics
    php-mini_httpd
    pinouts
    pizzaperdollar
    plesk-symlink-php
    pxe
    qmail-linux
    qmail-qfilter
    racoon-sonicwall
    redundant-vpn
    rewriterule
    seahorse-workaround
    setting-locale-failed
    smalldog
    snmpv3-cacti
    spamassassin
    squid-tproxy
    supfile
    suse73
    svc-nmbd
    svc-smbd
    svc-smtpd
    toyotastereo
    vm
    vn-file
    wmp-invalid
    xcode-remote-install
    xen-eth0-renamed
    xen-monowall
    xen3-ubuntu-dapper
    zz-update-grub-fail
Hire Tom
Mostly Mozart
Patches
School
Scrapbook
Software
Telephones




colocation
comments
davidireland
edsgranola
faq
funsites
goodlooking
goodmovies
google-earth-saucy-amd64
houserules
liberating
resume
resume2
scratch
shopping
snacks
todo
university
warisbogus

Building a FreeBSD router on a 256M disk
Posted June 3, 2003

Assemble the computer. Put the solid state hard disk on the primary ATA controller, and a bootable CD-ROM drive on the second controller.

BIOS setup:

date/time = current UTC date/time
floppy A = none
hard disk C = auto detect
halt on = no errors
power up state after AC power loss = previous state
boot device = cdrom

Boot with FreeBSD [4.7] CD #1 in the CD-ROM drive.

FreeBSD installer:

  • You're certain you don't need to configure the kernel.
  • Start a standard installation.
  • Press A to use entire disk, then Q.
  • Install standard boot block (no boot manager).
  • Create one 256M partition: C 247M Enter Enter / Enter.
  • Create a swap partition to make the installer happy: C Enter S Enter.
  • Quit: Q.
  • Choose "Minimal install."
  • Choose "Install from a FreeBSD CD/DVD."
  • Last chance! Are you SURE...? Yes. (Why is the default "yes"?)
  • "WARNING! Unable to swap to /dev/ad0s1b: Device not configured." Press Enter. (Perhaps 1M swap is too small?)
  • (Some time passes)
  • "Congratulations! You now have FreeBSD installed on your system." Enter.
  • "SLIP/PPP network devices?" N.
  • "Function as a network gateway?" Y.
  • "Configure inetd?" N.
  • "Anonymous FTP?" N.
  • "NFS server?" N.
  • "NFS client?" N.
  • "Default security profile?" N.
  • "PLEASE NOTE" about security. Enter.
  • "Customize console?" N. (I like font=swiss and keyrepeat=fast but who cares...)
  • "Time zone?" Y.
  • "CMOS clock set to UTC?" Y.
  • Choose your time zone.
  • "Does `PST' look reasonable?" Y.
  • "Enable Linux binary compatibility?" N.
  • "Non-USB mouse?" N.
  • "Browse the package collection?" N.
  • "Add any initial user accounts?" Y.
  • Add a "tom" user: U Enter tom Enter Enter Enter s3cret Enter Enter wheel Enter Enter Enter Enter.
  • Finished adding users: X Enter.
  • "You must set the root password." Enter s3cret Enter s3cret Enter.
  • "Chance to set any last options?" N. (You go back to the main menu anyway.)
  • Finished installing: X.
  • "Are you sure? The system will reboot." Y.

Press Delete or whatever to re-enter BIOS setup. Change boot device to "drive C" or "primary ATA" or whatever the disk is. Save & exit.

FreeBSD will boot with no network configuration ("Amnesiac").

Log in as root.

Set hostname and override some inappropriate defaults:

cat <<EOF >>/etc/rc.conf
hostname=testrouter.example.com
sendmail_enable=NONE
usbd_enable=NO
fsck_y_enable=YES
syslogd_enable=NO
inetd_enable=NO
firewall_enable=YES
firewall_type=/etc/firewall
EOF
cat <<EOF >/etc/firewall
add allow ip from any to any
EOF
echo autoboot_delay=2 >>/boot/loader.conf

Add network configuration:

echo nameserver 172.16.0.1 >>/etc/resolv.conf
cat <<EOF >>/etc/rc.conf
ifconfig_fxp0="inet 172.16.12.8 netmask 255.255.255.128"
ifconfig_fxp1="inet 10.10.10.1 netmask 255.255.0.0"
defaultrouter=172.16.12.1
EOF
sync

Yank the power cord. Wait. Restore power. The machine should power up, and FreeBSD should boot automatically after fixing the filesystem.

Log in as root.

Type "df -m" to check your disk space. I have 111MB available at this point.

Install some packages (if desired):

mount /cdrom
cd /cdrom/packages/All
pkg_add bash-2.05b.004.tgz && chsh -s /usr/local/bin/bash
pkg_add psmisc-17.tgz
pkg_add rsync-2.5.5_1.tgz

Type "df -m" again. I have 109MB available.

Install daemontools and ucspi-tcp:

cd /tmp
fetch http://cr.yp.to/ucspi-tcp/ucspi-tcp-0.88.tar.gz
tar xzf ucspi-tcp-0.88.tar.gz
(cd ucspi-tcp-0.88; make setup check)
rm -r /tmp/ucspi-tcp-0.88*

cd /tmp
fetch http://cr.yp.to/daemontools/daemontools-0.76.tar.gz
mkdir -p /package
cd /package
tar xzf /tmp/daemontools-0.76.tar.gz
(cd admin/daemontools-0.76 && package/install && rm -r compile)
rm /tmp/daemontools-0.76.tar.gz

Do a quick reboot: type "init 1" at a root prompt, switch to ttyv0 with alt-F1, press Enter for /bin/sh, then press control-D to return to multiuser mode. Log in and type "pstree" -- svscan should now be running.

Type "halt" to shut down. Disconnect the power. Remove the CD-ROM drive. Put the cover back on the computer.

Power up again, and log in as root.

Install djbdns:

cd /tmp
fetch http://cr.yp.to/djbdns/djbdns-1.05.tar.gz
tar xzf djbdns-1.05.tar.gz
(cd djbdns-1.05 && make setup check)
rm -r /tmp/djbdns-*

Set up a DNS cache.

  • Replace "10.10.10" with your network number. "10.9.8" means any host whose IP address starts with 10.9.8. is allowed to use the DNS cache.
  • The echo commands disable logging. There are better ways, but this is OK.
cd
pw useradd Gdnscache -s /usr/bin/true -d /nonexistent -u 901
pw useradd Gdnslog -s /usr/bin/true -d /nonexistent -u 902
mkdir -p /etc/service
dnscache-conf Gdnscache Gdnslog /etc/service/dnscache 10.10.10.1
touch /etc/service/dnscache/root/ip/10.10.10
cp -p -i /etc/service/dnscache/log/run{,.orig}
echo '#!/bin/sh' >/etc/service/dnscache/log/run
echo 'exec cat > /dev/null' >>/etc/service/dnscache/log/run
ln -s /etc/service/dnscache /service/

Install qmail:

pw groupadd qmail -g 82
pw groupadd nofiles -g 81
pw useradd alias -g nofiles -u 81 -s /nonexistent -d /var/qmail
pw useradd qmaill -g nofiles -u 83 -s /nonexistent -d /var/qmail
pw useradd qmaild -g nofiles -u 82 -s /nonexistent -d /var/qmail
pw useradd qmailp -g nofiles -u 84 -s /nonexistent -d /var/qmail
pw useradd qmailq -g qmail -u 85 -s /nonexistent -d /var/qmail
pw useradd qmailr -g qmail -u 86 -s /nonexistent -d /var/qmail
pw useradd qmails -g qmail -u 87 -s /nonexistent -d /var/qmail
cd /tmp
fetch http://cr.yp.to/software/qmail-1.03.tar.gz
tar xzf qmail-1.03.tar.gz 
(cd qmail-1.03 && make setup check)
rm -r /tmp/qmail-1.03*

Set up QMQP service on some other machine(s), so that connections are accepted from this machine. (Details not included here.)

Configure qmail to deliver mail via QMQP instead of trying to queue it on this machine.

  • Replace 10.10.10.99 and 172.16.0.99 with the IP address of your QMQP servers.
  • Replace example.com with your domain name.
cd /var/qmail/control
echo 10.10.10.99 >qmqpservers
echo 172.16.0.99 >>qmqpservers
hostname >idhost
echo example.com >me

cd /var/qmail/bin
ln -sf qmail-qmqpc qmail-queue

Configure a new kernel. (Do this on a different machine with more disk space to spare!) Here are the significant changes I made to GENERIC to come up with SAUNA:

# cd /usr/src/sys/i386/conf
# diff GENERIC SAUNA
19a20,26
> options IPFIREWALL
> options IPFIREWALL_FORWARD
> options IPFIREWALL_DEFAULT_TO_ACCEPT
> options IPDIVERT
> options DUMMYNET
> options RANDOM_IP_ID
> device pcm
> 
21,22d27
< cpu             I386_CPU
< cpu             I486_CPU
25c30
< ident           GENERIC
---
> ident           SAUNA

Build the new kernel. Skip the buildworld step if you haven't updated anything in /usr/src since your last buildworld.

cd /usr/src
make -j4 buildworld
make buildkernel KERNCONF=SAUNA
mkdir -p /tmp/SAUNA/modules
mkdir -p /tmp/SAUNA/usr/{bin,sbin}
mkdir -p /tmp/SAUNA/usr/share/man/{man4,man8}
make installkernel KERNCONF=SAUNA DESTDIR=/tmp/SAUNA
tar -C /tmp/SAUNA -cjf /tmp/SAUNA.tbz kernel modules usr

Copy SAUNA.tbz to your new machine, and install the new kernel. You can download my kernel if you don't want to build your own. Make sure the MD5 checksum is correct.

cd /tmp
fetch http://tomclegg.net/software/SAUNA.tbz
md5 SAUNA.tbz
MD5 (SAUNA.tbz) = 24c1a1ffb392094765ee73caba95acd5
tar -C / -xjf SAUNA.tbz
chflags schg /kernel

Set up ramdisks for supervised services, /var/run, /var/spool/lock, and /dev:

mkdir /mfs
mkdir /dev.boot
cp /dev/MAKEDEV /dev.boot/
cd /dev.boot && sh MAKEDEV all
cat <<EOF >>/etc/rc.early
/sbin/fsck -y /

/sbin/umount -f /mfs
/sbin/mount_mfs -s 1600 -T qp120at dummy /mfs
/bin/mkdir -p /mfs/run /mfs/spool/lock /mfs/service

/usr/bin/tar -C /etc/service -cpf - . | /usr/bin/tar -C /mfs/service -xf -

/sbin/umount -f /dev
/sbin/mount_mfs -s 1600 -i 1024 -T qp120at dummy /dev
/usr/bin/tar -C /dev.boot -cpf - . | /usr/bin/tar -C /dev -xf -
EOF

rm -r /var/run
ln -s /mfs/run /var/
rm -r /var/spool/lock
ln -s /mfs/spool/lock /var/spool/
ln -sf /mfs/service/dnscache /service/
echo /mfs/service/dnscache/root >/etc/service/dnscache/env/ROOT
reboot

"df" and "mount" should demonstrate that /mfs is now a 734K ramdisk:

df
Filesystem  1K-blocks   Used  Avail Capacity  Mounted on
/dev/ad0s1a    248910 125614 103384    55%    /
mfs:15            734    146    530    22%    /mfs
mfs:20            638     64    524    11%    /dev
procfs              4      4      0   100%    /proc
mount
/dev/ad0s1a on / (ufs, local)
mfs:15 on /mfs (mfs, asynchronous, local)
procfs on /proc (procfs, local)

Now, change "rw" to "ro" on the root filesystem entry in /etc/fstab. In the future, when you do need to write to the drive, use "mount -u -o rw /" to make it writeable. When you're finished, use "mount -u -o ro /" to change it back to read-only.

Change /etc/mail/mailer.conf so it uses qmail instead of sendmail:

mount -u -o rw /
cat <<EOF >/etc/mail/mailer.conf
sendmail        /var/qmail/bin/sendmail
send-mail       /var/qmail/bin/sendmail
mailq           /usr/bin/true
newaliases      /var/qmail/bin/qmail-newu
hoststat        /usr/bin/true
purgestat       /usr/bin/true
EOF

Set up a supervised natd process, and reboot to synchronize /mfs/service with /etc/service:

mount -u -o rw /
cd /etc/service
mkdir natd
cd natd
cat <<EOF >run
#!/bin/sh
killall -9 natd
exec fghack natd -f natd.conf
EOF
cat <<EOF >natd.conf
interface fxp0
dynamic yes
EOF
chmod +x run
ln -s /mfs/service/natd /service/
reboot

Set up a name server on 127.0.0.1 for local dns data:

pw useradd Gtinydns -u 903 -d /nonexistent -s /usr/bin/true
tinydns-conf Gtinydns Gdnslog /mfs/service/tinydns 127.0.0.1
cd /mfs/service/tinydns/log
cp -i -p run run.orig
(echo '#!/bin/sh'; echo 'exec cat >/dev/null') >run
cp -r -p /mfs/service/tinydns /etc/service/tinydns
ln -s /mfs/service/tinydns /service/

Remove some of the larger unnecessary programs:

rm /usr/libexec/sendmail/sendmail
rm /usr/libexec/named-xfer
rm /usr/sbin/named
rm /usr/sbin/ntpd

Things left to do:

  • Supervise qmail-smtpd (if you want this machine to accept SMTP connections and queue messages using QMQP).
  • Set up firewall rules in /etc/firewall.
  • Set up static_routes in /etc/rc.conf.
  • Update sshd or restrict it to known client addresses.
  • Install clockspeed.
  • Set clock at boot time using taiclock.