The Policy

I have witnessed the following scenario many times in my life – sometimes it happened to me, sometimes to some colleague. It always goes like this:

1. A company wants to protect itself against loss of something very important (e. g. company reputation, money, life); some prominent people sit together and devise a strict policy for everyone to follow.

2. The policy is cumbersome, so over time, people either ignore it (at least every now and then) or clever people find loopholes and work around it.

3. Years go by; by sheer luck nothing really bad happens.

4. Someday, someone (maybe a new colleague) discovers the truth and in a rush of creativity comes up with a smart and simple solution that achieves 99% of the desired effect while at the same time keeping those who have to use it happy.

5. The idea is rejected by management on the grounds that a) the proposed solution is not perfect either and b) the original policy can’t be so bad as nothing really bad has happened so far!

6. Our once-proud hero gets frustrated.

Why is this obviously good idea rejected? Answer: for political reasons!

Even though our inventor had the best intentions, by suggesting a new approach he caused a lot of problems to many people. He indirectly claimed that:

1. There has been a big threat over years.

2. There are people in the company who haven’t followed management’s policies for years.

3. The designers of the original policy did a poor job.

4. The managers failed to see the obvious: the policy is cumbersome and hard to follow.

5. The managers failed to ensure that the policy is actually followed.

6. He is much smarter than others (i. e. his managers).

Had the new approach been accepted, management would have admitted all these “accusations”. Now it should be clear why the idea was dismissed.

The Three Most Important Tools of a Software Developer

What are the most important tools of a software developer? In my view, the keyboard, a programmer’s editor and a scripting language are the top candidates. Compared to fancy design tools they might look mundane – but only at first sight: combined and mastered, they have the potential for making a developer an order of magnitude more efficient.

KEYBOARD

Let’s face it: as a software developer, you spend most of your time writing. Not just code but also test scripts, documentation and email. Hence, the keyboard is your most important basic tool, just like the saw is the most important basic tool of a carpenter.

It is a sad fact that many programmers don’t master touch typing and instead use the “Hunt and Peck” approach: they awkwardly enter text through slow and error-prone two-fingered typing.

The main problem with two-fingered typing is that since it is so cumbersome and slow, developers tend to avoid writing: they won’t write the Perl prototype of an application that would be so useful for the next trade fair and they certainly won’t write comprehensive documentation. Today, a large part of the communication in software projects happens through email, especially if the team is distributed over the globe – what a pity if you cannot type efficiently!

If you cannot touch-type as a programmer, you simply have to learn it. Now.

Fortunately, it’s rather easy. Within a couple of weeks you can achieve marvelous results by just practicing 20 minutes a day. There are countless programs for self-study out there, many of which are free, like KDE’s KTouch. Just do a search on your favorite search engine for “touch typing”. There are even games, like typespeed that you can use to improve your typing skills. Try it out, it’s great fun! However, it is important that you finish a systematic course first; that is, you know the positions of the basic keys blindfolded.

Once you are sufficiently proficient with the basic keys, you should tackle the remaining special keys as well, since they are quite important for software developers: punctuation marks, the ‘at’ sign, braces and brackets. Don’t stop until you know your keyboard inside out. If you catch yourself peeking at the keyboard too often, consider buying a keyboard with blank keys, like DAS Keyboard: a piano player doesn’t need letters on her keyboard and neither do you.

PROGRAMMER’S EDITOR

For years I had used various editors and IDEs for my daily work: for Java development, I used Eclipse, for C++ development Visual Studio, for batch file and test development as well as log file viewing and editing UltraEdit and TextPad; I used Outlook and Lotus Notes for composing emails and usually Microsoft Word for any other kind of documentation. Even though I read the Pragmatic Programmer’s advise “Use a single editor well”, I didn’t know how to live by it – until a colleague gave me a nice tour of the VIM editor about three years ago.

Ever since that day, my developer life has changed profoundly. Now, I use VIM for at least 90% of my writings. I even use it for composing emails, provided I expect my email to be longer than three lines of text. It’s just sooo cool! Now more do I use IDEs for building my projects; instead, I build from withing VIM by invoking a command-line build script; VIM’s “quickfix” feature lets me easily navigate between compiler warnings and errors. For me, the only legitimate use of an IDE is when doing source-level debugging; for changing the code and compiling I switch back to VIM.

I’m not trying to convince you to to learn VIM, since it is a large investment. It took me a couple of month to get some proficiency and even after three years, I’m still learning. But what a worthwhile investment it is for me!

Whatever editor you chose as your single editor, make sure you know it inside-out. Always ask “is there a smarter, more efficient way to accomplish what I want?” and check with the documentation. For instance, you should know how to program it, know how to create abbreviations and shortcuts, know how to record macros. Find out how to run external tools (like executing makefiles or checking out code from version control). Most importantly, learn all the hotkeys and key combinations for tasks that occur frequently, like deleting the word under the cursor (in VIM type ‘daw’) or indent the current paragraph (in VIM type ‘>ip’).

A programmer’s editor corresponds to a carpenter’s workbench: It serves as an integration platform for all your work.

SCRIPTING LANGUAGE

Even a craftsman sometimes needs power-tools to save time and get mundane and repetitive jobs done; so does the programmer and that’s where scripting languages excel.

By scripting languages I mean dynamic languages like Perl, Python or Ruby that don’t use static type checking like traditional languages such as C and Java do. Scripting languages are highly expressive and have constructs that let you achieve miracles in just a few lines of code. As an example, this Perl code

executes “my_prog.exe”, captures its output and counts the total number of error messages. If you are like me and think that this is still too long, here is the one-line version:

Try this in a traditional programming language of your choice and compare the total number of characters typed – expect that it will be 10 to 20 times as as many.

Most of the time, you will use scripting languages for automation and text processing; however, you can also use it to sketch and smear in order to develop prototypes that you might later recode in a traditional programming language.

A mastered scripting language is a great time-saver: It enables you to write lots of little utilities that make your and your colleagues life easier.

CONCLUSION

The combined power of touch-typing, a programmer’s editor and a scripting language makes a software developer much more productive in two respects: first, it ensures that jobs get done in little time and second, it encourages programmers to try out ideas. The latter is the basis for long-term developer motivation and outstanding performance.

How to Statically Initialize Arrays with Arbitrary Values

[Warning: Low-level C stuff ahead!]

Imagine a situation where you want to statically initialize an array with values different to 0:

This approach works, at least until someday you want to increase the array size to, say, 200. In this case, you have to add 192 times “42, ” to the initializer list. What a dread!

Everything would be easy, if you wanted to zero-initialize the array:

With zero-initialization, all you have to do is specify the value of the first element – all of the remaining elements will automatically be set to zero.

But sometimes you need a value different to 0 and you don’t want an additional call to “memset()” at run-time. Or you cannot use “memset()” because your array is stored in a read-only ROM segment and you cannot change the array’s values dynamically.

Basically, what you want is this:

Then, you would only have to make a single change to alter the size of the array (or the initialization value).

Alas, it turns out that it is impossible to define a macro that does the job we expect from “STATIC_INIT”. Think about it for a while. How would you solve this problem?

Sometimes, it is possible to replace a call to an impossible macro with the inclusion of a header file; I call this technique the “Replace macro call with file inclusion” trick:

The two defines represent the macro parameters and the inclusion of the header file represents the actual macro call.

You probably wonder what the contents of “static_init.h” are, but it’s instructive to spend some time on this problem yourself. Afterwards, you can have a look at my solution.

Note that this approach is not limited to single values – you can also use it for more complex initializations. For instance, if you need an alternating sequence of ’42’ and ’13’ you would do this:

I’ve also used the “Replace macro call with file inclusion” trick to encapsulate #pragmas and other compiler-specific features. Consider the case where you are working on a multi-platform project that uses different compilers. Consider further that you have a piece of code that generates compiler warnings and you want to locally turn compiler warnings off:

Now the problem with this approach is that #pragmas are compiler-dependent, which means that you will end up with something like this:

Not only does this litter the code – it is also a maintenance nightmare.

Usually, the solution is to encapsulate compiler specific features in #defines; alas this obvious strategy doesn’t work for #pragmas:

So it is time to roll out our trick once again:

Where, for instance, “warnings_off” looks like this:

You probably won’t need this trick very often, but when you do, it is good to know that it’s there.

The Awful Java Programming Language

In 1880, Mark Twain wrote a satirical essay entitled “The Awful German Language” where he explains his difficulty in getting the hang of the complicated German grammar. Today, I felt like it was time for me to nag a little bit about the Java programming language.

Don’t get wrong – I’m actually a Java fan. Years ago, I took (and successfully passed) the “Java Certified Developer” exam; I’ve written a considerable amount of Java code in my life. I’ve always loved Java for its run-time environment: the platform-independence you get from the virtual machine and the huge set of powerful libraries. I’m not complaining about Java per se – I’m only complaining about the Java programming language.

Obviously, it’s easy to find weak spots in any programming language – that’s why there are so many of them. Today, I only want to focus on one thing – Java’s wordiness, or rather, Java’s lack of expressiveness.

When developing software, I often start out with a Perl prototype. Yes, Perl is a quirky language, but it is also very powerful and most of its power comes from its very expressive grammar. It is easy to get your job done with Perl.

The other day, I was working on a seemingly simple problem, along these lines: a server produces a text file, containing user vs. usage-time records:

Another process – my program – takes this list as input, sums up all the usage-times for a user and prints an alphabetically sorted list:

That’s it. Sounds like a five minute job, which it is, if you happen to use Perl:

But since I needed this functionality as part of a larger Java product, I bravely recoded it in Java:

Wow! I used the regular expression library and the HashMap container class; I even used Java 5’s autoboxing feature as well as generic containers (both of which save you a lot of casting); nevertheless, the Java version is more than four times longer than the Perl script. Here are the stats:

Is this additional typing really such a big problem? It definitely is! Studies have shown that the number of lines written per developer per time-unit is by and large independent of the programming language used, which means that you are three to four times more productive in Perl than you are in Java. But there is another angle to this problem: if your programming language is highly expressive (like Perl, Python, and Ruby) you are many times more likely to actually write programs. You cannot imagine how many useful programs I have written in Perl that I would have never written in C/C++/Java – just because of all that typing!

I’ve never liked Sun’s “it-is-so-simple-even-a-brain-dead-monkey-can-now-program” paradigm. As a professional software developer – just like a craftsman – I need powerful tools: tools that get the job done in as little time as possible. These tools might be dangerous – very much like a power drill or a circular saw – but in the hands of an expert they can achieve miracles.

But Perl is not perfect, either. In my view, it is good for short programs, less than 1000 lines of code. For larger projects (possibly involving more than one developer) statically typed languages — like Java — are probably a better choice.

Spec Coders

If you’ve been working as a professional software developer for some time, you must have come across a “Spec Coder”.

Spec Coders are usually well-educated, have good grades and know every detail about dozens of specs. Frequently, they come with social skills well above the level of the average (nerdy) coder. They are – therefore – a human resource manager’s dream.

Alas, what they can’t do is write decent code, I mean, readable, reusable, efficient, elegant code. Sure, they’ve read “Kernighan & Ritchie” and they might have taken a three-day Java course, but between their excellent domain knowledge and their (basic) knowledge of a programming language is a big gap in every Spec Coder: a total lack of design capabilities due to a total lack of passion for software development.

This lack of passion clearly shows in their “if-this-do-that-else-do-another-thing” style of programming. You won’t find abstractions, base classes, function pointers, recursion. Just a plain translation of boring spec words into code that sucks.

Here’s an example.

Let’s assume our Spec Coder is assigned the task to write some code for a file system and that THE SPEC says:

3.2 File selection behavior
…
After selecting a file, the file’s record marker shall be undefined, except for ‘blue’ files, in which case the record marker shall point to the first file record.
…

Our Spec Coder will immediately turn this into code like this (don’t worry about the use of global variables here – it’s just an example):

So far so good, at least until someday THE SPEC gets extended (obviously by a non-programmer):

3.2 File selection behavior
…
After selecting a file, the file’s record marker shall be undefined, except for ‘blue’ files, in which case the record marker shall point to the first file record.
If the file system is in state ‘raw’, the record marker shall always be undefined.
…

Again, the Spec Coder will directly translate this to code:

How disgusting! Our Spec Coder didn’t take the time to refactor the code logic according to the new requirement. In fact, a typical Spec Coder wouldn’t even think that refactoring this code was a sensible thing to do. Why? Because there wouldn’t be a 1:1 mapping between the code and THE SPEC anymore. Arrrgh!

Any true software developer would have jumped at the chance. A true software developer would have mentally rephrased THE SPEC to read “A file’s record marker is always undefined, except for ‘blue’ files when the system state is not ‘raw'”:

This code is definitely more efficient and readable. If you don’t like the ternary operator, just use an if-else statement (are you a Spec Coder, by any chance?):

Are Spec Coder’s really dangerous? I think so! Because of their attitude, with every change they make, the code gets slightly more complicated – little by little – until it’s such a mess nobody understands anymore. Once the critical mass is reached, even otherwise decent developers tend to implement their changes by adding little “ifs” here and there. But there is another problem with Spec Coder’s: It is usually difficult to get rid of them, because their usually good spec knowledge makes them indispensable for a company.

Weird, isn’t it?

Douglas Adam’s Cookies

I recently read a funny story by Douglas Adams called “Cookies“.

Here is a summary.

Douglas arrives early for a train, and buys a cup of coffee, a newspaper and a packet of cookies. He takes a seat at a table, opposite of a businessman. After a while, the guy leans forward grabs the packet of cookies, tears it open and eats one. In his story, Douglas explains how he – as an Englishman – is not at all able to cope with this situation, so he just takes a cookie himself. A minute later, the guy takes another cookie and so does Adams. This procedure repeats until the packet is empty. Finally, the guy leaves – they both exchange “meaningful looks”. When the train arrives, Adams gets up, takes his newspaper and discovers underneath it – his packet of cookies! Douglas ends the story by saying that “The thing I like particularly about this story is the sensation that somewhere in England there has been wandering around for the last quarter-century a perfectly ordinary guy who’s had the same exact story, only he doesn’t have the punch line.”

A very funny story, indeed…

After my giggling had settled, I couldn’t stop pondering about it. Doesn’t this seemingly improbable story happen all the time, only slightly differently? How often do we think somebody is plain wrong or is unfair to us, when in fact it is exactly the other way around? How often do engineers, salespeople and managers talk at cross purposes? Isn’t this a terrible good example of a paradigm shift, a total change in one’s perspective?

Douglas picked up the newspaper, which triggered the paradigm shift. If he hadn’t picked up the newspaper, he would have walked around for the rest of his life believing that the other guy was pretty weird. So what is the lesson to be learned? Whenever we think we are treated badly or others are plain wrong, we should try hard to find the newspaper — and pick it up.