Monthly Archives: March 2012

The port 0 trick

The port 0 trick came in handy when writing eximunit, and it’s something surprisingly few developers know about, so I thought it worth recounting here:

The problem: you want to set up a web/mail/whatever server programmatically (e.g. as part of some tests). This server wants to bind to port 80/25/whatever. Your first problem is that it can’t bind to these because you’re not running your tests as root (or as an administrator on Windows).

The lazy approach at this point is to hard-code a port number over 1024, which you don’t have to be privileged to bind to. But this all falls to bits if you want to run the same test simultaneously on the same machine, or you need lots of different ports during the course of one test.

At this point, you can reach for the port 0 trick: on both Windows and Linux, if you bind a socket to port 0, the kernel will assign it a free port number somewhere above 1024. Truly well-written software (e.g. Jetty) will not only let you configure it to bind to port 0, but will make it easy to parse its logs to obtain the actual port number it got assigned. Less helpful software (Tomcat) will let you configure it to bind to port 0, but print 0 in all its logs, never the actual number. And the majority of software just won’t let you put 0 as a port number in its configuration.

At this point (subject to a slight race condition), you can grab some port numbers yourself and feed them to whatever you’re trying to configure:

def findFreePorts(howMany=1):
    """Return a list of n free port numbers on localhost"""
    results = []
    sockets = []
    for x in range(howMany):
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.bind(('localhost', 0))
        # work out what the actual port number it's bound to is
        addr, port = s.getsockname()

    for s in sockets:

    return results

The above is written in Python, but it translates trivially to any programming language which knows what a socket is.

You’ll note that the correct way to get, say, five free port numbers is to call the above method once with 5 as its argument. If you wrote a simple method which just returned one number, there would be nothing to stop it returning the same number each time you called it (the Linux kernel is rather more helpful, and will usually hand out a different number to each port 0 request, but I wouldn’t  rely on this).


I’ve decided to give up on maintaining my own SVN server – Mercurial is my version control of choice these days, and for the scale of the open source stuff I do, there’s no good reason not to use Bitbucket.

I’ve done my best to fix all the historical links pointing to so they go to the files now on my Bitbucket account, but do get in touch if I’ve missed one.

Introducing eximunit

For a few years now, I’ve run a hosting co-operative with a few friends. Although the cost savings versus all renting VMs individually are probably marginal at best these days, one of the nice things about it is the chance to run things like our incoming MX on one machine only, instead of all having to run our own anti-spam and other measures. The incoming mail is handled by Exim, and each user of our system can add domains for which mail is processed. They get to toggle SMTP-time rejection of spam and viruses, and specify the final destination machine for incoming mail to their domain.

This has all been working well for over  two years, but occasionally something has to change: a few months ago, we got rid of sender verify callouts, now widely considered abusive by SMTP server admins, and more recently we added support for tagging messages with headers to say if they passed or failed DKIM verification. And every time I make such a change, I worry that I might have inadvertently broken something. This server handles mail for 30 domains and 8 people, some of who rely on it to run businesses! Panic!

I usually end up reassuring myself by doing some ad-hoc testing by hand after reconfiguring the server. At the most basic level, whatever your SMTP server is, you can use netcat to have a conversation with it on port 25:

d@s:~$ nc localhost 25
220 ESMTP Exim 4.71 Sat, 17 Mar 2012 09:51:20 +0000
HELO localhost
250 Hello localhost []
250 OK
550-Callout verification failed:
550 550 Unrouteable address
221 closing connection

And there, I’ve just convinced myself that one of our features is still working: the mailserver should call forward to the final destination for mail to addresses to check the local part (‘someaddress’ in this case) is valid, and reject the message up-front if it’s not.

Exim also has a load of other toys you can take advantage of: say I want to check how mail to is routed:

d@s:~$ exim4 -bt
R: hubbed_hosts for
 router = hubbed_hosts, transport = remote_smtp
 host []

(IP addresses changed for example purposes, obviously)

And finally, there’s debug mode: you can run

exim4 -bhc <ip address>

to run a complete ‘fake’ SMTP session as though you were connecting from the given IP address. You can send messages, but they won’t actually go through, and exim prints a lot of debug output to give you a clue as to its inner workings as it decides how to route the message.

This is all very well, but a quick brainstorming session gives a list of over 30 things I might want to check about my mailserver:

  • Basic check that mail is accepted to our domains
  • Only existent addresses on our domains should have mail accepted
  • Domains with SMTP-time spam rejection on should have spam rejected
  • Same for viruses
  • Same for greylisting

Testing all these by hand isn’t going to fly, so what tools can we find for automating it? A bit of Googling turns up swaks, which looks quite handy, but suffers from two drawbacks for me: first, it’s a bit low-level, and a collection of scripts calling it will be a bit difficult to read and maintain for testing all 30 of my assertions. Second, it really sends the e-mails in the success case, and I don’t want my users to get test messages or have to set up aliases for receiving and discarding them. swaks will definitely become my tool of choice for ad-hoc testing in future, but meanwhile…

The other promising Google result is Test::MTA::Exim4, which is a Perl wrapper for testing an exim config file. However, a few problems: (1) it’s Perl, and I Don’t Do Perl. (2), it’s limited to testing the routing of addresses, so it’s not going to cut it for checking spam rejection etc.

Having at least pretended not to be suffering from NIH syndrome, let’s spec out a fantasy system for doing what I want: I would like to be able to write some nice high-level tests in my favourite language, Python, which look a bit like this

class HubbedDomainTests(EximTestCase):
    Tests for domains our server acts as the 'proxy MX' for, doing
    scanning etc before forwarding the mail to the destination machine

    def testProxiedMailAccepted(self):
        """Proxied mail should be accepted"""
        session = self.newSession()

    def testLocalPartsVerifiedWithDestinationMachine(self):
        """Local parts should be verified with the destination machine"""
        session = self.newSession()

I could then run these in the usual manner for Python unit tests, and lastly, I want them backed by an exim4 -bhc session so that they’re as realistic as possible without actually sending messages.

This post is long enough already, so I’ll cut to the chase and say that I’ve made a start on writing it, and you can find out more at Bitbucket. In a follow-up post, I’ll talk about how it was done.