tldcheck – a script to check domain availability on all TLDs and ccTLDs

@dstergiou asked:

I am looking for a tool to see under which TLDs a domain is registered. E.g. check all TLDs for domains matching “test123”

One way to check is to visit registrars that cover TLDs and ccTLDs (like EuroDNS or Gandi) and submit your query to their forms. However since (at least TTBOMK) none of them offers any query API to check programmatically, one can script his way to an almost complete solution. ICANN provides a list of all available TLDs and we can iterate over it:


#!/usr/bin/perl

## To grab the TLDs: wget -c http://data.iana.org/TLD/tlds-alpha-by-domain.txt

use Net::DNS;

$check = shift or die;

open F, "< tlds-alpha-by-domain.txt" or die;
while (<F>) {
        next if (m/^#/);
        chop;
        push @tlds, $_;
}
close F;

foreach $i (@tlds) {
        my $domain = $check . "." . $i . ".";
        my $res = Net::DNS::Resolver->new;
        my $query = $res->query($domain, "NS");

        if ($query) {
                ## foreach $rr (grep { $_->type eq 'NS' } $query->answer) { print $rr->nsdname, "\n"; }
                print $domain, "\n";
        }
}

The above hack comes with three problems:

  • It uses Net::DNS which makes it a little slow. There is room for improvement.
  • It finds out about registered domain names that have domain name servers that answer queries about them. This is not always the case. For example in .GR one can register a domain without having it served.
  • It also does not take into account strict domain hierarchies, like .co.uk, .net.uk, .org.uk but minimal changes are needed to test that too.

Somewhat incomplete as a solution, but adequate as a 5 minute hack for most purposes.

P.S. @stsimb offers a similar solution using dig:

for sfx in TLDs; do dig +short ns test123.$sfx; done

Moacyr Barbosa

“Only three people have, with just one motion, silenced the Maracanã: Frank Sinatra, Pope John Paul II and me.” —Ghiggia about his goal.

Moacyr Barbosa lived the rest of his life as an outcast. In his own words “The maximum punishment in Brazil is 30 years imprisonment, but I have been paying, for something I am not even responsible for, by now for 50 years”

Robert Green will be fried by the English press, but an outcast he will not become.

(Inspired by today’s goal, scored by Dempsey on Green)

on team formation

In a meeting today a friend (quietly) observed that opening a process to a wider audience very fast may compromise the very process that interests the intended audience. I replied back with the thesis that:

“Whenever data increases, quality drops (for any quality metric)”

I first heard that thesis 15+ years back in a meeting about data warehouse quality. Usually when few people get together for a certain task it goes like this:

Small team, with people working towards similar goals

Increase the number of participants and you get something like this:

More people join the party, and things get interesting

Add a political twist and some power-play (personal or between organizations) and you get this:

Politics and power-plays set the project's final course (do nothing)

This is to be expected. David Alan Grier in “The Dictator and the Web Design” (IEEE/Computer, May 2009) notes:

“Traditional management theories identify such fights as the second part of a four-stage development process for small groups, the forming-storming-norming-performing steps that psychologist Bruce Tuckman identified in the 1960s. “Group members become hostile toward one another as a means of expressing their individuality and resisting the formation of group structure,” Tuckman claimed.

In Tuckman”s model, committee members must go through a period in which they express their objections to the collaboration in emotional terms (the storming stage) before they can learn to work together (norming) and actually accomplish their goals (performing).”

So there, as long as performance does not go “our way”, quality drops. By the way, this also explains why Panathinaikos B.C. prevails over Olympiacos B.C. in Greek A1. They both have excellent players, but Panathinaikos make sure that all are focused to the same direction. They are a team performing, while the others are still forming.

Χέστηκα για το Πρωτάθλημα

Περιμένω την ημέρα που κάποιος παράγοντας θα πάρει την ομάδα και θα φύγει από το γήπεδο (όπως κάποτε ο Βουλινός) και ας πάρει ο αντίπαλος το πρωτάθλημα. Είναι τουλάχιστον ηλίθιο να έχεις βάλει €35M σε μια ομάδα και να παρακαλάς να αδειάσει το γήπεδο.

Θυμάμαι πως ο πρώτος αγώνας που με πήγε ο πατέρας μου ήταν Ολυμπιακός – Ηρακλής (φιλικό 4-3 με Νόιμαν, Κουσουλάκη, Ορφανό, Παπαμιχαήλ) στη Θύρα 14 (με την 7 κλειστή για έργα, όλοι ήταν στη 14). Σκέφτομαι πως εγώ δεν έχω πια τη δυνατότητα να πάρω τα παιδιά μου στο γήπεδο, όχι στις “σκληρές” θύρες, αλλά ούτε και στις ήρεμες. Και αν σκεφτεί κανείς τι σημαίνει να πάει μια πενταμελής οικογένεια στο γήπεδο -και πόσα θα ακουμπήσει σε εισητήρια, φαγητό, branded merchandise κ.λπ.- θέλω να δω in the long run ποιος θα χάσει. Οι οικογένειες που δεν θα πάνε στο γήπεδο ή οι ομάδες που θα παίζουν σε άδεια (όχι από τιμωρία) γήπεδα;

(Το παιχνίδι δεν έχει αρχίσει ακόμα)

a bit of history on the relatonal model

Thanks to Software Memories we learn about David Childs and his work on Extended Set Theory. I quote from the blog post:

“Way back in 1968, Childs wrote a paper outlining how set theory, relations, and tuples could be applied to data management.

And that’s where I did a double-take, because 1968 < 1970. Sure enough, Footnote #1 in Codd’s seminal paper is to Childs’ 1968 work. Indeed, Childs’ paper is the only predecessor Codd acknowledges as having significant portions of his idea.”

It seems that there was life before God Codd after all.

sendmail load configuration

This post is about a neat trick that I have not seen many times discussed. According to the configuration README the default values for controlling load averages are:

  • confQUEUE_LA (QueueLA) Load average at which queue-only function kicks in. Default values is (8 * numproc) where numproc is the number of processors online (if that can be determined).
  • confREFUSE_LA (RefuseLA) Load average at which incoming SMTP connections are refused. Default values is (12 * numproc) where numproc is the number of processors online (if that can be determined).

However in “Sendmail Theory and Practice” (I am a proud owner of both editions) Paul Vixie and Fred Avolio propose a different approach:

“Astute readers will note that the value shown for Ox (QueueLA) is larger than the value shown for OX (RefuseLA), and that this is opposite from the configuration files you may have seen elsewhere. Setting them as shown here gives Sendmail a range of load average in which it is capable of delivering messages from its queue but incapable of receiving new messages. This is intentional. If you set Ox to be less than OX, Sendemail has instead a range of load average in which it can receive new mail (thus adding to the queue) but cannot deliver any queued mail. We believe that mail queues should become smaller or stay the same size when the load average is high. After watching our large mail gateway computers melt down many times over the years, we have learned that it is better to let other hosts’ mail stay where it was -on other hosts- when our load average is high, than to accept it even though we don’t plan to do anything with it until load average becomes low again.”

In other words although the defaults suggest otherwise, it may be wiser to have QueueLA > RefuseLA. This piece of advice is on both the 1995 (1st) and 2002 (2nd) editions of the book. A pearl that comes from 1995 that is still relevant.

AthCon begins

“A beginning is the time for taking the most delicate care that the balances are correct” –Frank Herbert’s Dune.

AthCon begins today. Since it is the first AthCon it really begins today. It is a non-product, non-vendor-biased conference aiming to present the best research and cutting edge exploitation techniques from the field’s leading experts. I feel extremely privileged that I was invited to participate in the (first) PC of such an effort. However due to the 24-hour strike of the public transportation workers and the law of unexpeted consequences that always finds opportunity to emerge I will not be able to attend the event. I was really looking forward to watch:

  • “OWASP Top 10 – 2010: Towards a secure Software Development Lifecycle” by Konstantinos Papapanagiotou
  • “Context-Keyed Payload Encoding: Fighting the Next Generation of IDS” by Dimitrios Glynos and
  • “BNF (Backus-Naur Form) based blackbox fuzzing” by Chariton Karamitas

Maybe these kind souls will email me their presentations.

Good luck AthCon and be a nice journey. See you next year and every year!