Eric Allmans keynote

| No Comments | No TrackBacks
Eric was the creator of sendmail. He started off with a review of the history of sendmail. Sendmail is an old program, that has survived well, in a very changing well, with no initial commercial support. It got kicked of in 1980 at U.C. Berkeley. Eric was meant to be working on RDBMS. Back when it started CPUS were 8 or16 bit running at less than 1 MIPS. Disks were much less than 1 GByte. Network was less than 56 Kbps.

At Berkley they got an arpanet connection. As they had so few ports, there was congestion on usage (people were getting into fights over who could use it). Eric worked out that the killer app was email, the professors weren't really trying to use all of arpanet, but were trying to use mail. So he wrote a store and forward mail system.

He showed some diagrams of the different components, and a code sample. He started off with some design principles. He wasn't going to change the local mail store, he wasn't going to redesign user agents. He had to make delivermail adapt to the world, and one programmer is finite.

His quick hack (delivermail) had some issues. He showed a table of email addresses in different systems, and where it was going. Berkeley got awarded DARPA contract to write 4.2BSD, and Eric got approaced by Bill (don't know who he is) to write the mail thing. So Eric agreed and sendmail was first released in 1982.

In the years 1981 to 1990 the unix was were waged, and most unix grabbed sendmail and lots of different versions of sendmail sprung up. Eric came back to Berkley and (for various reasons) decided to do a sendmail re-write, which pulled in a bunch of these versions. So he made sendmail 8, and integrated new SMTP protocols, new Protocols like DSNs and LDAP. The sendmail 'Bat' book on sendmail got written.

Then in 1998 he started sendmail inc, so he could get back doing coding and other people could do support. But he found out that starting a company means more about money, marketing and sales than coding. This was one of the first commercial/open source hybrid companies.

So Sendmail inc implemented encryptiong and authentication, and milter, virtual hosting, multiple logical queues, LDAP and lots more checking. They were driven by commercial needs, though opensource needs those things as well.

Eric then moved onto the second part of his talk, what lessons has he learned from the history of sendmail. Firstly he said that requirements always change. It started with reliability (mail had to get through, or see an error response). Then there was functionality and performance needed. Then it became about protection (against spam and viruses). Then legal and regulatory compliance, and keeping costs down. He highlighted that the Waterfall model of development doesn't work for all of this.

Eric has been critizised for some of his design decisions. Though some of his decisions were probably the right one at the time. Send mail is 'overly general'. But the mail problem is in flux, so sendmail is a tool for the mail problem, not a solution.

He said using tabs as active characters was the worst decision in his life. But use of rewriting rules was the right decision, even though it could have been implemented a bit better in hindsight. He says that message munging was absolutely essential for interoperability at the time. It's still heavily used, but not always necessary.

He agree's that the syntax of the configuration file is pretty ugly, but not fundamentally flawed. Today he would choose something different.

The queue keeps two files per message, one for envelope and header and one for body. In retrospect these days he'd use enevelopes in a database. But it was the right decision for the time (remember embedded databases didn't exist back then).

He agree'd that the syntax of m4 was painful. But he needed a macro language. There really wasn't anything else he could use, and he wasn't going to write his own. He said the main problem is 'dnl' statements, which aren't even necessary.

He found that with masquerading he made the wrong decision for the extending versus changing features debate. He did masquerading the wrong time the first time. Then he added new features around it to try and fix it, but he should have just changed it. He didn't want to break existing sites, but this created a larger problem for the future.

Some people believe that sendmail has allowed broken software to persist, following Postel's Law (liberal in what you accept and conservative in what you generate). At the time every thing was broken, so it was the right decision. But the long term effect is that broken software has no incentive to get fixed.

If he was doing it again today he would have not used the v7 mailbox syntax (those mboxes). he's used a bunch of modern tools that didn't exist back then. He'd use more privilege separation. He'd create a string abstraction. He'd use separate mailbox names from unix user ids, and a cleaner configuration file.

But he'd still use C for the implementation. He'd still use syslog and rewriting rules. He thinks that OO languages are a mistake. He says OO languages do too much under the covers. He'd continue to do things in small(ish) chunks. He still would have written syslog (he wrote syslog as a side project). He wouldn't rely too heavily on outside tools, as tools have their own cost.

He wrapped up with some takeaways. KISS principle actually works. Even though sendmail isn't simple today, each step was a simple progression. If you don't know what you're doing, advance designs don't help much. That was certainly the case when sendmail was being started, because the mail systems weren't designed yet. Flexibility beats performance when the world changes. He suggests you fix things early, because the installed base only gets bigger. Documentation is key to broad acceptance. The design space is always changing.

No TrackBacks

TrackBack URL:

Leave a comment

About this Entry

This page contains a single entry by Geoff Crompton published on January 27, 2011 9:47 AM.

latest and coolest html5 media was the previous entry in this blog.

bat phone is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.



Powered by Movable Type 4.23-en