Author: adamo

I am a Greek DevOps Engineer ranting about everything, from email to sports; to living in Athens, Greece.

OODA Loop and DevOps

I spotted on twitter the other day a Donkey Kong analogy for DevOps:

The Donkey Kong DevOps analogy
The Donkey Kong DevOps analogy

Regardless of how fun (and close to heart) it is, the analogy is flawed because it describes a sequence. We do not wait to compete all the steps in the ladder in order to get to the next level. Nor, do we restart only when a barrel hits us.

Fast transients is what we do. Fast transients is a term conceived by John Boyd and he first used it for air combat: “the ability to change altitude, airspeed and direction in any combination”. This is after all the essense of the Release Early; Release Often mantra. Push your system out in the wild so as to get a grasp of where the audience wants to direct it. Or plan for organized abandonment. According to Boyd, what matters most is the tempo of change: “fast transients suggeststhat -in order to win or gain superiority- we should operate at a faster tempo than our adversaries, or inside our adversaries’ time scales”1,2

So it is no wonder that I belive that although not so funny, the OODA loop describes how we work:

OODA Loop from CTOVision's "I've got the OODA Blues"
OODA Loop from CTOVision’s “I’ve got the OODA Blues”

Because as Boyd wrote:

“Orientation isn’t just a state you are in; it’s a process. You’re always orienting […] A nice tight little world where there’s no change – dinosaurs; they’re going to die. The name of the game is not to become a dinosaur […] If you are in an equilibrium position, you’re dead”

Now think of that in terms of what you do just to keep current with the tools of the trade and what you do in order to monitor, manage and evolve your infrastructure.


[1] – A vision so noble, Daniel Ford
[2] – Which reminds me of the Nyquist sampling theorem

An online test for Benford’s Law

I’ve been fascinated with Benford’s Law ever since I read Mark Nigrini‘s “Digital analysis using Benford’s Law“. As part of the fun I’ve used it to check whether email subject length follows it. So I figured, why not make a check available online? And here it is:

– http://first-digit.appspot.com

You copy-paste your number series and the application displays a column chart of the occurences of the first digit and the theoretical occurencies expected.

The first 50 Fibonacci numbers and Benford's Law
The first 50 Fibonacci numbers and Benford’s Law

Use it and tell me what you think. Many thanks to Vaggelis Tripolitakis for finding the first bug. Be gentle with it as it runs as a free Google App Engine service. Depending on my free time and feedback I get, I may add the rest of the tests that Nigrini describes (second digit, last digit and first two digit distributions).

If you like it very much, you can buy me the kindle edition of the second edition of Nigrini’s book.

Record your observations

“Physics: there was the key. Record your observations. Apply physical principles.Speculate, but only trust proven conclusions. If I were to make any progress, I’d have to treat the task as a freshman physics problem. Time to update my notebook.” —Cliff Stoll, The Cuckoo’s Egg.

Recording observations. Updating notebooks. Something we computer people frequently forget regardless of the big data hype and logging infrastructures that we build.

Startups as Deviance

Yesterday evening I was attending these gentlemen (four young unemployed guys trying to find their way by their description) at a Ruby meeting held at CoLab. Their presentation was split in two parts. The first part was about going from zero to a demo application in Rails in just over four months (and some of the development decisions they made).

The second part was about their idea. This part attracted most of the questions, so we got to learn about the original idea and a bit about how it evolved along with the hazards they had to deal with on Azzure and Amazon virtual machines working with literally a zero budget. But although untold, the story of the way that the team formed and operates was also presented. And while thinking about it, it struck me that it followed the Best and Luckenbill model from “Organising Deviance”:

Form of Organization Mutual Association Mutual Participation Division of Labor Extended Organization
Loners no no no no
Colleagues yes no no no
Peers yes yes no no
Mobs yes yes yes no
Formal Organizations yes yes yes yes

I would call these 4 young men Peers by the model above, because although some division of labor exists, it is not to the point of separation of duties yet. But while I am writing this, I am thinking that the parallels with the Best and Luckenbill model are to be expected. For what are startups if not deviant organisations aiming to disrupt the status quo in their own way?

deviance, n.:
a state or condition markedly different from the norm

I wonder how fast these guys are going to transform to a Formal Organization once they receive funding.

When will it be done?

– You say estimates; I hear deadlines

We all know the system administration rule of thumb about when a task will be done:

Estimate the time it will take you and double it. Then double it again and add some more.

This is not a baseless rule. You know how much it will take you to finish the task. What you cannot predict in this interrupt driven line of work that we do, is how much noise you will have to deal with simultaneously while dealing with the task at hand. Or what unexpected circumstances will unearth because of misinfromation, poor documentation or simply bad luck. So you need breathing space in order to complete the task. For when you give an estimate, users take this as written in stone. When you are of your estimate they view this as a broken promise, regardless of what caused the extra delay. They just do not care. All they care about is that you “promised” it will be ready at some time and it is not. And because they do not care, you always need more time than you think.

Sometimes this also has the added advantage that when you are lucky enough to work uninterrupted and finish within the time frame promised, polite users will thank you for your efforts to complete as fast as you could. They see this as a proof that you care about their pain and do your best. Oh, yes the rest again do not care.

Why was I reminded of this? Well because our DBA hang on his wall the following formula:

T_c = \frac{b + 4m +w}{6}

Where Tc is time to complete, b is the best time, w is the worst time and m is the most likely time. You can read more about the formula and its history here [pdf].

A junior DBA discovers consulting

After reading this (in Greek) a friend, who works as a junior DBA at a bank, sent me these dialog exchanges between him and a highly costing consultant:

  • Friend: I think that rebuilding the indices once a week just because this works is not the proper method to deal with the problem. Shouldn’t we be looking at other stuff like table fragmentation for example?
  • Consultant: Look, rebuilding and reorganizing the indices once a week is good practice because you know, it works!

Two years pass and:

  • Consultant (the same): In order to be sure when to rebuild the index, you should look at the table fragmentation level.

By the way, if someone is interested in a junior DBA who can put in the hours needed to solve problems and is not afraid of studying in depth in order to do so, drop me a note so that I can put you in touch.

So long and thanks for the fish

Mark Reed Crispin, inventor of IMAP, passed away on Friday, December 28, 2012 at Martha and Mary Healthcare Services in Poulsbo Washington. He was born on July 19, 1956 in Camden New Jersey and was 56 years of age.

Very few people have (almost single handledly) designed protocols and implemented (free) software that has facilitated communication among millions of people. Mark was one of them and thanks to his monumental achievement of IMAP (and the fact that the Net cannot “forget” his posts on comp.mail.imap) he will always be remembered and always there to teach those who want to hear him.

As someone who has had the privilege to exchange personal emails with him, I am deeply saddened by his departure. I think he is the first of my email heroes that leaves.