Author: adamo

I am a Greek DevOps Engineer ranting about everything, from email to sports; to living in Athens, Greece.

“SYLK: File format is not valid”

Recently I was involved in a project that required producing a CSV file as output. To convert the output in Microsoft Excel format, I used Google Docs where I uploaded the CSV file and exported it as a .xls. When trying to open the file from Excel, one would get the following (not very helpful) message:

SYLK: File format is not valid

SYLK are files typically used to exchange data between applications. Fortunately, Microsoft has documented the problem:

A SYLK file is a text file that begins with “ID” or “ID_xxxx”, where xxxx is a text string. The first record of a SYLK file is the ID_Number record. When Excel identifies this text at the beginning of a text file, it interprets the file as a SYLK file. Excel tries to convert the file from the SYLK format, but cannot do so because there are no valid SYLK codes after the “ID” characters. Because Excel cannot convert the file, you receive the error message.

The quick workaround to the above is to not use “ID” as the first two characters in the CSV file to be imported. And like the article says, this problem does not occur if the first two letters are lowercase “i” and “d”.

Ah, the joys of report generating. Full of slight annoyances…

TCPMUX – a mostly overlooked TCP service

TCPMUX is described in RFC-1078 (written some 20 years ago). A reference implementation by Network Wizards can be found at ftp://ftp.nw.com/nw/software/tcpmux.c . It is also implemented in DragonFlyBSD’s inetd, NetBSD’s inetd and FreeBSD’s inetd. OpenBSD does not support for it.

The Protocol

A TCP client connects to a foreign host on TCP port 1. It sends the service name followed by a carriage-return line-feed . The service name is never case sensitive. The server replies with a single character indicating positive (“+”) or negative (“-“) acknowledgment, immediately followed by an optional message of explanation, terminated with a . If the reply was positive, the selected protocol begins; otherwise the connection is closed.

The 15+ years I have been a sysadmin I have never seen anyone making a use of it, which is a pity: Most of the time I see fellow sysadmins who want to write a custom daemon, either write it as a standalone server (usually starting with passivesock.c or passiveTCP.c from Comer’s Internetworking with TCP/IP vol.3), or writing is as a simple stdin/stdout application that is started via inetd. The most trivial problem is sometimes more than trivial:

– What port will this application run on?

It seems that 65535 ports is a lot of freedom to choose from and most people want to use “interesting” port numbers (for any definition of interesting). Add firewall policies and router access lists in the picture, you can have a non-technical deadlock in no time!

TCPMUX might be a choice to help simplify / avoid such situations. Any service that supports TCPMUX listens on port 1/tcp and can be forked by inetd(8) (either internally or externally with the help of a tiny server). After all, it can be considered as an “inetd inside inetd” (the classic inetd responding to requests on a port, TCPMUX responding to requests based on the name of the service) and even if you do not want to use TCPMUX, a similar (homegrown) solution might be the answer to keeping your packet filters lean and less complex. It does not have to be less complex than it has to be though. The Wikipedia article on tcpmux clearly identifies risks that come with deploying it. Personally, I view tcpmux as an old and simple TCP RPC mechanism.

Appendix: tcpmux.c

Since the Network Wizards site seems to be down / taken over by some other entity, here is the original tcpmux daemon code (also at github https://github.com/a-yiorgos/tcpmux ):

Continue reading “TCPMUX – a mostly overlooked TCP service” →

4’33”

Lefteris explained 4’33” to me late one night while taking a break in the lab. I think it was one of those nights when lab members would work through the night, take a break downtown and then come back again and work till morning. Lately I find myself doing late night coding again and thinking of 4’33” mostly because of this:

In 1951, Cage visited the anechoic chamber at Harvard University. An anechoic chamber is a room designed in such a way that the walls, ceiling and floor absorb all sounds made in the room, rather than reflecting them as echoes. Such a chamber is also externally sound-proofed. Cage entered the chamber expecting to hear silence, but he wrote later, “I heard two sounds, one high and one low. When I described them to the engineer in charge, he informed me that the high one was my nervous system in operation, the low one my blood in circulation.” Cage had gone to a place where he expected total silence, and yet heard sound. “Until I die there will be sounds. And they will continue following my death. One need not fear about the future of music.” The realisation as he saw it of the impossibility of silence led to the composition of 4′33″.

So that’s the sound of silence I am listening to. Wow!

c-client callbacks

* This is mostly for personal copy-paste reasons

Those who take the time to develop applications using UW-IMAP (or Panda IMAP) know that there are a number of callbacks that need to be defined. What follows is the simplest (do nothing) version of them.

#include "c-client.h"

void
mm_flags(MAILSTREAM *stream,unsigned long number) {
}

void
mm_status(MAILSTREAM *stream,char *mailbox,MAILSTATUS *status) {
}

void
mm_searched(MAILSTREAM *stream,unsigned long number) {
}

void
mm_exists(MAILSTREAM *stream,unsigned long number) {
}

void
mm_expunged(MAILSTREAM *stream,unsigned long number) {
}

void
mm_list(MAILSTREAM *stream,int delimiter,char *name,long attributes) {
}

void
mm_lsub(MAILSTREAM *stream,int delimiter,char *name,long attributes) {
}

void
mm_notify(MAILSTREAM *stream,char *string,long errflg) {
}

void
mm_log(char *string,long errflg) {
}


void
mm_dlog(char *string) {
}

void
mm_login(NETMBX *mb,char *user,char *pwd,long trial) {
}

void
mm_critical(MAILSTREAM *stream) {
}

void
mm_nocritical(MAILSTREAM *stream) {
}

long
mm_diskerror(MAILSTREAM *stream,long errcode,long serious) {
}

void
mm_fatal(char *string) {
}

The Unwritten Laws of Engineering

MEMagazine is running a three part series entitled “The Unwritten Laws of Engineering” by W. J. King and James G. Skakoon. W. J. King first published the three series articles in Mechanical Engineering magazine in 1944. Briefly the laws are:

  1. However menial and trivial your early assignments may appear, give them your best efforts.
  2. Demonstrate the ability to get things done.
  3. Develop a “Let’s go see!” attitude.
  4. Don’t be timid—speak up—express yourself and promote your ideas.
  5. Strive for conciseness and clarity in oral or written reports; be extremely careful of the accuracy of your statements.
  6. One of the first things you owe your supervisor is to keep him or her informed of all significant developments.
  7. Do not overlook the steadfast truth that your direct supervisor is your “boss.”
  8. Be as particular as you can in the selection of your supervisor.
  9. Whenever you are asked by your manager to do something, you are expected to do exactly that.
  10. Cultivate the habit of seeking other peoples’ opinions and recommendations.
  11. Promises, schedules, and estimates are necessary and important instruments in a well‑ordered business.
  12. In dealing with customers and outsiders, remember that you represent the company, ostensibly with full responsibility and authority.
  13. Do not try to do it all yourself.
  14. Every manager must know what goes on in his or her domain.
  15. Cultivate the habit of “boiling matters down” to their simplest terms.
  16. Cultivate the habit of making brisk, clean‑cut decisions.
  17. Learn project management skills and techniques, then apply them to the activities that you manage.
  18. Make sure that everyone, managers and subordinates, has been assigned definite positions and responsibilities within the organization.
  19. Make sure that all activities and all individuals are supervised by someone competent in the subject matter involved.
  20. Never misrepresent a subordinate’s performance during performance appraisals.
  21. Make it unquestionably clear what is expected of employees.
  22. You owe it to your subordinates to keep them properly informed.
  23. Never miss a chance to commend or reward subordinates for a job well done.
  24. Always accept full responsibility for your group and the individuals in it.
  25. One of the most valuable personal traits is the ability to get along with all kinds of people.
  26. Never underestimate the extent of your professional responsibility and personal liability.
  27. Let ethical behavior govern your actions and those of your company.
  28. Be aware of the effect that your personal appearance and behavior have on others and, in turn, on you.
  29. Beware of what you commit to writing and of who will read it.
  30. Analyze yourself and your subordinates.
  31. Maintain your employability as well as that of your subordinates.

ASME has published the expanded version of these laws as a book. From the introduction of the book we learn that these laws are the result of direct observation for 17 years in four engineering departments. Also “many of these laws are generalizations to which exceptions occur in special circumstances. There is no thought of urging a servile adherence to rules and red-tape, for there is no substitute for judgement; vigorous individual initiative is needed to cut through formalities in emergencies. But in many respects these laws are like the basic laws of society; they cannot be violated too often with impunity, notwithstanding striking exceptions in individual cases”.

[via]

The Tom Clancy Theorem

Laura McLay reminds us that in The Hunt for Red October Vice Admiral James Greer says:

“The likelihood of a secret’s being blown is proportional to the square of the number of people who are in on it”.

Anything that resembles Metcalfe’s Law, I dig. My first reaction was to wonder whether Tom Clancy had heard of Metcalfe’s Law. But thinking it through, I saw that it did not matter. Given that N people know something that is considered as a secret, representing them as a network, that would make them a clique (or an almost clique). In such a network there exist N(N-1) directed channels of communication, hence the N2 heuristic.

In her post, Laura McLay points to J. Michael Steele’s “Models for Managing Secrets” which was published in 1989. In this paper the author
builds on top of the clique a simple communication model and reaches to the conclusion that:

“The expected window of secrecy decays quadratically with the number of people who are in on the secret.”

The more the people, the faster it will get out in the open. The paper then examines one more (complicated) model, some counter-measures of disinformation and points to Game Theory for further study of such models.

So, if the numbers are correct, cablegate was waiting to inevitably happen.

Quick note on Lattices, Relations and stuff

Thanks to @mosabou I got to read about Formal Concept Analysis. What a cool concept! Of the first things that I read was “A First Course in Formal Concept Analysis” (an introduction to the subject without the mathematics). While going through the examples, I kept thinking Where have I seen that before? Relational Databases, that’s where! And it seems that my intuition was correct: see the excellent “Gentle introduction to Relational Lattice” by Vadim Tropashko and the links from there.

[ I vaguely remember Yannis introducing the lattice concept in a database context in a course lecture a few years back, but have to admit of not looking much into it back then. ]

This is mostly a note for people who insist on thinking that theory is disconnected from practice. Especially the ones who write SQL code and insist on not realizing that they deal with sets (and set theory). Stop holding an umbrella and invest some time in your math.

If you cannot kill the content, kill the path that leads to it

One can stop content distribution by DDoS-ing the networks hosting it. This is a direct attack from one opponent to another. There are also some indirect attacks that people rarely think about (or notice). For the content to be reached, two things must be available: routing and DNS. And these are services that are not necessarily under the administrative control of any of the two parties in conflict. And they can even be easier targets, since they can be put in the position to choose between one customer and the rest of their 500K customers.

With Wikileaks now moving to wikileaks.ch, are we to expect a DDoS on the .ch DNS servers?

When are we going to see Wikileaks blackhole routing? Or routing to its DNS servers being blackholed? Or even to its parent ccTLD, making whole countries invisible to DNS? I wonder whether has anybody collected any data on that…