Greenspun’s Tenth Rule and variations

For those who have not heard Greenspun’s Tenth Rule, it states that:

Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

By the way, Greenspun‘s rules 1 to 9 do not exist.

Seven months ago, during a discussion about Prolog, I asked Ozan S. Yigit to reformulate Greenspun’s tenth rule for Prolog. Oz replied:

Any sufficiently complicated modern program contains a buggy, informal implementation of prolog that casual observers confuse with lisp.

Just hours earlier I was basically a listener in a discussion that involved NoSQL. While clearly I am not a NoSQL advocate, I am no hater either, but what I heard lead me to the following reformulation of Greenspun’s rule, this time involving the relational model:

Those who blindly adopt #NoSQL will discover a variation of Greenspun’s tenth rule

I am sure that many other variations exist. In fact the Wikipedia page on Greenspun’s Tenth Rule contains a Prolog variation similar to Ozan’s and an Erlang version. So if you know of (or can make up) any other, please post it here (or somewhere).

An alternative to FEATURE(mailertable)

Using FEATURE(mailertable) one can instruct sendmail to route email for certain destination via a specific relay. A mailertable is essentially a static map that instructs sendmail where to route email for certain destinations ignoring DNS MX RRs (or other information). Example:

yahoo.com   smtp:[server.example.com]
yahoo.com.hk   smtp:[server.example.com]
yahoo.com.mx   smtp:[server.example.com]
yahoo.com.br   smtp:[server.example.com]
yahoo.com.cn   smtp:[server.example.com]
yahoo.com.sg   smtp:[server.example.com]

Why would one want to do that? Your customers may have been hit by a botnet and as a result your outgoing mail server may have sent enormous amount of spam. Since most high-profile mail hubs use some kind of reputation scheme on the IP addresses that contact them, it is quite probable that your outgoing mail server is experiencing delays, or worse denied delivery despite the fact that in the meantime you have done your best to stop the botnet and clear your queues. I know for it has happened to me.

A mailertable is a quick solution to route email through another mail server just for recipient domains that implement such policies. But it is far from perfect for the Postmaster has no way to know all the domains that Yahoo! Mail in the above example hosts in order to construct a mailer table. Luckily, when high-profile mail hubs (like Gmail, Yahoo! Mail and Hotmail) implement good patterns on their DNS MX RRs, a programmatic (instead of a static) solution can be deployed:

LOCAL_CONFIG
Kbestmx bestmx -T.TMP

LOCAL_RULE_0
R$+ < @ $+ > $*         $: $(bestmx $2 $: NOTFOUND $) $| $1 < @ $2 > $3
R$+.hotmail.com. $| $+ < @ $+ > $*      $#esmtp $@ [server.example.com] $: $2 < @ $3 > $4
R$+ $| $+ < @ $+ > $*   $: $2 < @ $3 > $4

In the above snippet, any email that is directed to a domain that is served by Hotmail’s servers is routed via server.example.com. For the record, our outgoing webmail server achieved a senderscore of 50, and although a filter stopped the plaque, Hotmail silently discarded email originating from it. Using the above solution restored communications for our users.

New eBooks on Graph Theory

My twitter stream and my INBOX brought to my attention two new books on Graph Theory:

  • Graph Theory and Complex Networks: An Introduction” by Maarten van Steen. It is very interesting to note that this book is also available electronically as a personalised PDF. As the author notes: “When you write a book containing mathematical symbols, thinking big and acting commercially doesn’t seem the right combination. I merely hope to see the material to be used by many students and instructors everywhere and to receive a lot of constructive feedback that will lead to improvements. Acting commercially has never been one of my strong points anyway”.
  • The other book is the fourth edition of Reinhard Diestel’s “Graph Theory“. This book is also available electronically in different formats. I bought the student edition for €12.50 (offer expires in Aug 15, 2010).

PS: On a side-note I decided to buy a BeBook Mini

Using bestmx for discarding outgoing email

The following ruleset discards email that originates from domains for which we are not best MX. It is meant to be applied on outgoing email servers:

LOCAL_CONFIG
Kbestmx bestmx -T.TMP

LOCAL_RULESETS
SLocal_check_mail
R$*                               $: $>canonify $1
# You may (or may not) want to comment the following line
R < @ >                           $#OK
R$* < @ $+. > $*          $1 < @ $2 > $3
R$* < @ $+ > $*                   $: $2
# Short circuit certain domains (and host names)
Rexample.com                           $#OK
R$* . example.com                      $#OK
R$*                               $: $(bestmx $1 $: NO $)
# If a temporary error occurs, do not block
R$*.TMP                           $#OK
Rserver.example.com.          $#OK
R$*                               $#discard $: $1

This works for as long as spammers do not use domains for which they do not control the DNS zones. If they do control the DNS zones they can easily add your relays as MX to them. In such cases the above ruleset must be modified to lookup the name servers for domains that server.example.com is best MX and then decide to discard. However the above trick erased thousands of outgoing spams yesterday.

PS: Like I posted on twitter: I rewrote the above filter in ~35 lines of Perl (subroutine filter_sender for MIMEDefang’s mimedefang-filter). The sendmail version is both more compact and readable (at least to me).

MVP

Definitely Suarez. It seems that the probability of scoring by a penalty kick is ~75% [in pdf], so by stopping the ball with his hand (and being sent off) at the very end of the match he introduced a well known chess trick into football: Sacrificing the Queen in order to win. So far in my lifetime I have seen the hand of God twice, only now it was within the rules.

It is quite probable that the rules of the game will change: In such cases not only the player will be sent off, but the goal will count too.

URU vs GHA 4 – 2 (1 – 1)

Panda-IMAP

Since Mark Crispin left the UW, development on the UW-IMAP toolkit paused. Mark however continued developing the toolkit under the name Panda-IMAP.

Panda-IMAP is not publicly available. Mark Crispin allows access to it to people (and organizations) that donate to the development of the project. Since I am a dedicated user of the UW-IMAP toolkit and time had passed since the last version of the “old” UW-IMAP toolkit (back in 2008) on April 23, 2010 I personally donated $100 to the project. Replacing UW-IMAP with Panda-IMAP was a piece of cake and given that we are planning to move to mix format mailboxes, I am extremely happy with the result of using Panda-IMAP so far.

ΥΓ: Ερώτημα για όσους υπερασπίζονται το άστοχο tweet της κυρίας Άννας Διαμαντοπούλου: Τα παραπάνω σημαίνουν πως το κόστος του mail server είναι $100, ή όποιο άλλο ποσό αποφάσιζα να δώσω από την τσέπη μου;

A simple use of Google’s Public DNS servers

Google Public DNS s a free, global Domain Name System (DNS) resolution service, that you can use as an alternative to your current DNS provider. The service’s DNS servers IP addresses are easily memorable even by end users (who the service aims to help most) and they are:

  • 8.8.8.8 and
  • 8.8.4.4

There are other uses for the service. Many system administrators use it for troubleshooting DNS problem in their infrastructure as an objective third party with a DNS view from “outside” their network (plus you can say to your manager that hey this is Google’s DNS view of this zone setup when nothing else helps).

Those of us who host domains, web sites and mail infrastructures have at times faced the problem that domains come and go somewhere else. However, domain owners / administrators / subcontractors / etc often neglect to inform the previous infrastructure that the domain has a new home. Then appears the phenomenon that most of the Internet knows how to access the web site (and where to route email) at the new home, with the exception of the previous ISP or hosting provider. Most of the times the previous hosting provider will find out when the contract runs out, which at times may take as long as 6 months (and I’ve seen longer times too).

In a few cases the previous hosting provider will find about the move because of complaints by its current customers who cannot reach the domain, the old customer complaining that there exist people who do not see the new site (but hey did you ever ask us to put it down?) or simply by pure chance.

In such cases as above the objectiveness of the Google Public DNS system can be of use to the DNS master who wants to maintain a clean setup. One can feed the following script with a file that contains one domain per line (the domains that you host) and ask Google who does it see as their designated DNS servers. In the old days one would ask a fellow admin at another ISP for shell access (I use SDF for similar purposes) or for query access. There is no need for that now :)

#!/usr/bin/perl
# This hack assumes that your nameservers are under the example.com domain

$ns = "8.8.8.8";
## $ns = "8.8.4.4";

while(<>) {
	next if (m/^#/);
	chop;
	$domain = $_;
	open DIG, "dig \@$ns $domain ns +short|" or die;
	while(<DIG>) {
		chop;
		next if (m/\.example\.com\.$/);
		print "$domain $_\n";
	}
	close DIG;
}

As is shown by the small script above the idea is pretty simple and can easily be customized to suite any local setup.

Polymath projects for other disciplines too?

While I was revisiting Gowers‘ “Mathematics: A Very Short Introduction” my mind wandered to the first Polymath project (essentially a massively collaborative effort to solve certain mathematics problems where participation seemed to follow the 90-9-1 principle). Anyone who wants to learn more about Polymath can start from “A gentle introduction to the Polymath project

Anyway, as I was reading the paragraph I was looking for, it struck me: Do other disciplines have similar efforts? Wouldn’t it be nice if they did? If not, why? One minute later a second strike came:

– Wait a minute! We were there before Polymath! We have Hackathons!

Although more free spirited (in a Hackathon anyone can tackle what they want) the outcome is to the benefit of the society concerned with the event.

However hackathons seem disconnected from academic enviroments and it is a pitty. Big conferences occur yearly and people have fun discussing their work at the hallway tracks exchanging ideas and strategies. It seems a bit of waste that so many bright minds together do not sit around a blackboard, or even collaboratively over the Net, and discuss about attacking a problem, any problem, that has endured the test of time. Bright Math people did it, why not the rest?

With HDMS approaching, maybe this is something to consider for the last session. So if anyone from those going to Cyprus is reading, keep this at the back of your head. I could be wrong and such an effort may not be feasible in another discipline, but I would like to know why.

on picking an MTA

Sometimes I get asked on what is my MTA (Mail Transfer Agent) of choice. Almost always I am asking “What do you want to do with it?”. Personally, in most places I install sendmail. There are cases (cases where one would use FEATURE(nullclient) or similar) where I install nullmailer, for I find it unnecessary to run sendmail.

People sometimes ask me why do I choose sendmail and not Postfix (or Qmail in the old days) or even Exim since we are running a mostly Debian shop. Leaving the monolithic argument aside (which is kind of funny when most people that use it are using a monolithic kernel OS anyway) I am using sendmail because of its expressive power. I can find a way to express what I am thinking (filtering, routing, etc) in its modem noise of a programming language or milters like MIMEDefang (IIRC, there’s a wonderful PDF presentation by Ricudis on the Turing completeness of the sendmail.cf language but I have no link to it).

It is not that I have not used other MTAs. Hell, I was even running Postfix alpha versions right after it was renamed from VMailer. And occasionally I am running MeTA1 instances. But I always return to sendmail. If it does not suit you, it is OK. Pick the one MTA that can help you build the setup that you have in mind, be it Exim, Postfix, netqmail, commercial software like Exchange or CommuniGate, whatever. If it works for you and your team, then it is the right choice. Endless debates are for people who have too much free time.

However, if there is one recommendation that I can share, this is it: If you are serious about email (routing) invest some time reading the bat book. He who can understand a complex piece of software like sendmail, can guide himself through any email system.

(triggered by a brief conversation I had with a friend this afternoon)