« Posts under Code

Technical Debt is Not Bad!

“There once was a Master Programmer who wrote unstructured programs. A novice programmer, seeking to imitate him, also began to write unstructured programs. When the novice asked the Master to evaluate his progress, the Master criticized him for writing unstructured programs, saying, “What is appropriate for the Master is not appropriate for the novice. You must understand Tao before transcending structure.”
The Tao of Programming, Book 3, Verse 2.

I consider Ward Cunningham’s “technical debt” analogy as one of the most important metaphors in software development. According to him, he invented it to explain to his manager why his team needed to do refactorings: like financial debt, technical debt tends to grow over time and the burden gets bigger and bigger due to compound interest.

During the last decade, quality-minded developers have done a great job of educating their managers about technical debt but it seems to me that there is now the notion that all technical debt is bad and must be avoided. But Ward Cunningham never said such a thing — and rightly so.

In business, not all debt is considered bad. It’s a huge difference if someone takes out a 200k loan to buy a flashy sports car or if the owner of a construction company takes out the same 200k loan to buy a bulldozer. The sports car is a pure liability that makes the owner poorer whereas the bulldozer is an investment that will generate money in the future.

The same holds for technical debt. If you initially implement a limited version of a feature (e. g. slow, bloated, lacking proper error-handling) just to enable other teammates to carry on with their own work, that’s good technical debt.

So when does technical debt qualify as good technical debt? To me, good technical debt is

a) taken on consciously
b) managed
c) repaid timely

By contrast, bad technical debt arises out of laziness or incompetence, it lurks in the code base and won’t be repaid. Rather, like credit card debt in real life, more debt is accumulated over time and new loans are taken out just to repay the interest rates of other loans. This is the kind of technical debt that is to be avoided in the first place.

You can use a knife to kill somebody, but you can also use a knife to whittle and prepare food. Surgeons even use knives to save people’s lives. Thus, a knife is neither good nor bad per se and so is technical debt.

A Small Matter Of Interface Design

“Bad design shouts at you.
Good design is the silent seller.”
— Anon

Retired C++ hero Scott Meyers once wrote an article for the renowned IEEE Software magazine titled “The Most Important Design Guideline?“. According to him, the most important design rule is this:

“Make interfaces easy to use correctly and hard to use incorrectly”

I couldn’t agree more. While this rule is applicable to any system, today I want to present it in the light of software interface (API) design. As an illustrative example, I’m going to borrow a method from a former article of mine:

From what you’ve been given, it’s pretty easy to conclude what this routine does and how it’s supposed to be used: ‘getTemperature’ returns a temperature value (let’s say in degrees centigrade) and a status as out parameter. Even without further documentation, it’s not hard to guess that the temperature returned can only be trusted if the status is ‘good’. Most likely, this method reads the temperature value from a hardware sensor that sometimes is busy or dysfunctional.

If you ask me, this design easily passes another important design rule, the so-called “Principle of Least Astonishment“. However, it fails miserably when we apply Scott Meyer’s interface design rule. Why? Because it seduces programmers into using it incorrectly:

As you can see, it’s possible for programmers to get what they want (the temperature) without checking the status value.

The compiler usually doesn’t warn you if you forget to inspect the status variable, because it doesn’t know what ‘getTemperature’ is doing: it might pass it on to another function or store the pointer in a static/global variable.

How can we change the interface such that this mistake is impossible to happen?

Often in such or similar situations, experienced programmers shout “exceptions to the rescue!”. But will exceptions really save us? Let’s see:

Now, ‘getTemperature’ doesn’t provide a status any longer, it rather throws an exception if the temperature value is unreliable or not available. But again, the compiler cannot enforce that programmers provide the try/catch block. Unlike Java, C++ doesn’t have checked exceptions (where checked means that the compiler enforces that exceptions are either caught or declared) and for good reason. So an inconsiderate programmers might naively write:

Even worse, the fact that there is no status parameter any more doesn’t remind them or reviewers at a later point in time that a necessary check has been omitted. Depending on the reliability of the sensor, such code can run for years without problems until it suddenly crashes due to an uncaught exception.

If exceptions don’t help, what does? How about swapping the return value with the status code?

While some would argue that this version is a bit harder to use incorrectly, it’s still possible to do so:

Again, the programmer doesn’t check the status, this time by ignoring the return value. No compiler I’m aware of warns you if you forget to check a function’s return value. Static analysis tools (like PC-lint) do, but who uses them, anyway? And of those good people who do, who enables this particular warning? There are many functions whose return values you almost never care about, like ‘printf’ for example, which returns the number of characters printed, or ‘memcpy’ which returns a pointer to the destination. Therefore, such warnings are usually turned off, even if they are supported.

If you are so lucky and use a C++17 compiler, you can mark a return value as ‘nodiscard’:

Should you now dare to ignore the returned status value, your compiler will report an error. Or will it? Unfortunately not! I don’t know why the people who devised the C++17 standard didn’t go for a compile-time error. Instead, if the return value is ignored by programmers, “implementations should issue a warning in such cases”, they wrote. Arrgh! They didn’t even have the decency to demand a warning. As usual, “should” doesn’t guarantee anything.

Still, I’ve never come across a C++17 conforming compiler that didn’t issue a warning, irrespective of the warning level (unless you force the compiler to suppress all warnings). Therefore, I think that this last version of ‘getTemperature’ is the first that is likely to receive Scott’s blessings.

In a future installment, we’ll explore other alternative versions of ‘getTemperature’ and determine if they comply with the most important interface design rule, which is so important that it bears repeating: “Make interfaces easy to use correctly and hard to use incorrectly”. Stay tuned!

Taiming Unwieldy Initialization Routines

“Please forgive the long letter; I didn’t have time to write a short one.”
― Blaise Pascal

Sometimes, you need to write a component initialization method that does many things: initialize hardware (e. g. various sensors), multiple libraries, and so on. Usually, some initialization steps are dependent on the successful execution of previous steps, so if one step fails, you want to skip the rest of the initialization procedure to avoid crashes.

Such initialization methods can easily become 100+ lines long, which is far off from the 5 to 10 lines that Uncle Bob and followers of the Clean Code school of thought recommend. That’s why it’s often a good idea to put the initialization steps into methods of their own, each consisting of only 5 to 10 lines of code:

While this achieves the primary goal of reducing the length of the initialization method and works if only three steps are involved, a cascade of ten nested if-statements is not at all pleasant to look at. Clean Coders would certainly frown upon it.

Of course, C++ coders (clean or unclean) have a well-known solution for such cases: exceptions. Put a try/catch block around your code and throw exceptions if an initialization step fails:

But maybe you are not allowed to use exceptions in your project. How come? Some embedded folks abhor the idea of exceptions (for many, sometimes unjustified reasons) and even disable support for exceptions via compiler switches. Or, your language doesn’t support exceptions in the first place, like good ol’ C. So what do you do if you want to have cleaner code, anyway?

How about this technique:

This code is definitely shorter and more readable than the original code as well as the exception-based variant. But how does it work?

You simply pass a reference to a boolean flag (or pointer, if you prefer) to every initialization step routine. If the flag is already false, you skip executing the initialization code; otherwise you execute it and only set the flag to false in case of an error:

This approach flattens the nested-if cascade by moving the check for the successful execution of a previous step inside the method implementing the next step.

Usually, I try to avoid detailed error codes as much as possible. I prefer binary (boolean) results which only tell if something executed without failure. Should an error occur, I send detailed error information to a logger. If that’s not sufficient to you, you can use an enum instead of a boolean flag:

Instead of passing a reference to a boolean flag to you methods, you would pass a reference to an instance of enum Error. The rest of the code should stay mostly the same.

This Is Why We Mock, Part III

The refactorings I made in the previous episode, laid the groundwork for unit-testing. Now has come the time to talk about mocks. What are mocks, anyway?

From a component developer’s point of view, there are usually “foreign” components as well; that is, components that you are dependent on but you don’t want to unit-test. You don’t even want to #include or link-in such components when executing your unit tests. Often, such foreign components are developed by another person, department, or organization. In other cases, these components run only on a particular hardware. Sometimes, these components are not (fully) implemented yet when you start developing your component. Foreign components are outside the scope of the UUT (unit under test).

Mocks act as substitutes for foreign components. They adhere to the same interface as the foreign components, otherwise they wouldn’t be compatible. The cool thing about mocks is, that you, as a unit test developer, can specify how the mock should behave. So when you have a mocked Person class with a method “greet”, and the real implementation would normally respond with “Hello”, you could instruct your Person mock to return “hello”, “howdy”, “”, a nullptr, basically anything you want. This allows you to write unit tests that check that your UUT code behaves correctly, no matter what is returned by the greet method. Setting-up such behavior in the real foreign component would be extremely difficult, if not impossible. A mock is basically a puppet on a string — you pull the strings and it behaves accordingly.

Let’s do a concrete example using our temperature control app. We defined an “ISensor” interface and rewrote the whole app to only work against this interface. From this interface, we’ll derive a mock (I’ll use googlemock, one of the most popular, powerful, and reliable mocking frameworks for C++).

I suppose you can easily figure out what MOCK_METHOD does: It declares a method “getTemperature” that returns an “int” and takes “status” as a parameter, which is exactly the signature defined in “ISensor”. Now let’s toy around with this mock a little bit:

The interesting part is what happens in EXPECT_CALL; it basically means this:

  1. We expect, that during the execution of our test case, there will be exactly one call to “getTemperature” (“WillOnce”).
  2. We demand that the call to “getTemperature” yields a value of 200.

When we call the “getTemperature” method on the next line, “sensor” (our mock) will comply with our demand and return 200. That’s why the EXPECT_EQ statement will succeed. Let’s call “getTemperature” twice:

In this case, we’ll get

If we wanted to set our expectations such that “getTemperature” is called multiple times, we could use “.WillRepeatedly” instead of “.WillOnce” and our test would thereafter pass.

What about the “status” parameter? Is there a way to demand that output parameters are set to a certain value as well? You bet!

Here we have multiple demands on our mock (“DoAll”). The mock will set the value of the dereferenced first pointer argument to “bad” and the return value to 200. The same goes (with different values) for the second test case. Nice, isn’t it?

Since we now have a basic understanding of how mocking works, we can resume our task of unit-testing our temperature control app. Instead of using concrete sensor and heater objects, we’ll dependency-inject their mocked counterparts and set expectations and demands on our mocks. We then execute our UUT (the controller) and check if it behaves correctly (expect that the heater mock is turned on or off). For brevity’s sake, I’ll only show the “happy path” where the sensor status is never “bad”:

The remaining test cases can be found in this GitHub repository.

This concludes my mini series on mocking in C++. My aim was not to teach you all about mocking (or googlemock in particular), but rather wet your appetite. Another aim was to raise your level of awareness regarding “design for testability”, especially programming against interfaces instead of concrete classes. Your foreign components should always be derived from pure interfaces. Otherwise, you simply can’t dependency-inject mocks during unit testing — at least not in C++.

To be able to fully unit test our code — this is why we mock!

Previous episode

This Is Why We Mock, Part II

In the previous episode, I claimed that our temperature control application is utterly untestable. That’s quite a broad statement, so before going any further, I’d like to clarify what I really meant.

You can always load our code on an embedded device (like a Raspberry Pi) and attach an appropriate physical temperature sensor as well as a real heater. With the help of an additional external thermometer and an additional external heater you can create various conditions and check whether our system behaves as expected. This is called system testing. Even though it’s ultimately necessary, system testing is tedious, time-consuming, error-prone, expensive, depends on hardware and what-have-you. If you discover a bug at this stage, you have to repeat the whole manual procedure again (and again and again).

When I, as a developer, talk about testing, I generally mean unit testing. Unit testing can be done at your desk, on your PC, without external hardware, within seconds. Even better, it can be fully automated and parallelized. So to be more precise: our temperature control app is utterly impossible to unit-test.

What actually is the unit that we want to test? Our focus is on the control logic, so the class “Controller” is our unit-under-test (UUT). We want to verify whether all logic paths are covered and whether they produce the expected output (heater turned on or off).

In the remainder of this post, I’m going to rewrite the code such that unit testing becomes possible. Let’s go!

One major nuisance with the existing code is that it contains an endless loop. It’s impossible to look inside the loop from a unit test-case. So in a first step, I’ll factor out the meat of the loop, the code where the action takes place.

As you can see, all the loop in “run” now does is invoke a method called “step”. “step” contains the control logic and we have now a much better grip on it. By repeatedly calling “step” from within our test cases we can single step through the code logic and check after every step if we get the desired effects. (Therefore, I like the method name “step”, what about you?)

The fact that we aggregate the sensor and the heater as members of our controller is a gross violation of modern software design principles, as we make our code (the controller) depend on concrete implementations (classes). If we instead depend on interfaces, we can substitute one implementation (a hardware temperature sensor) with a fake/mock temperature sensor (more on mocks later), as long as they both adhere to the interface. So let’s get rid of concrete classes and introduce interfaces:

Our familiar hardware sensor implements this interface, just assume that the actual implementation hasn’t changed from the first episode:

We proceed in a similar fashion with the heater:

Finally, we can rewrite the controller code such that it works against abstract interfaces instead of concrete implementations:

All dependencies (sensor, heater) are passed via so-called dependency injection as interfaces to our controller class.

Obviously, in order to work as before, our application still must depend on concrete classes. But only in a single place — the main function:

Our code hasn’t changed a bit in functionality, it behaves just as before. But contrary to the initial design, it’s now utterly testable, er, unit-testable, I mean.

In the next episode, we’ll talk about mocks and finally implement the unit tests. Stay tuned!

Previous episode
Next episode

This Is Why We Mock, Part I

Everytime I watch one of those “This is why we ride” videos, I immediately want to jump on my bike and head for some twisty mountain roads. What’s funny is that after watching these videos, even people who don’t have a driver’s license feel a strong urge to jump on a bike.

I wrote this post in a similar spirit, hoping to convince C++ developers that unit testing with mocks is both, necessary and fun. My main focus is presenting a case for using mocks, since I regularly meet even experienced coders who are not familiar with mocks. If you’re not familiar with mocks, it’s a clear sign that you have never really unit-tested your code. It’s that simple! You might have unit-tested some free-standing library or algorithm that you wrote, but when an application consists of multiple interacting components implemented by different people, there’s no way for you to test your component in isolation without linking-in the components you depend on. If you do so, you’re not doing unit testing. You’re rather doing integration testing. And what do you do if the components you depend on are still under development? Do you postpone your work and wait until they are fully done? What if some components are hardware-dependent and only available on the target platform? Do you leave the comfort of your host platform and execute your tests only on the target platform? No! What you do is sprinkle some “design for testability” over your code and test against mocks. Suck it up!

Let’s do an example, a very basic temperature control application. It consists of a heat sensor, a heater, and the controller. The controller is a simplistic on-off controller which reads the current temperature from the sensor and compares it against a target temperature. If the temperature is below the target temperature, the controller turns the heater on, if the temperature is above the target temperature, it turns it off. To make it a bit more interesting, I added another requirement.

To avoid frequent on/off toggling of the heater around the target temperature point, the controller supports so-called hysteresis, a temperature range below the target temperature in which the controller doesn’t turn on the heater even though there is a deviation from the target temperature. Here’s an example: if the target temperature is set to 23 degrees and there’s a hysteresis of 2 degrees, the controller will not turn on the heat if the current temperature is 22 degrees. Not even if the current temperature is 21 degrees. It will, however, turn on the heater at 20 degrees (or less) as this is outside the range of the hysteresis.

But wait, there’s another point to mention. The temperature sensor provides a status, which tells if the temperature reading can be relied upon. If the sensor status is “bad”, the measurement must be ignored. If the status is “bad” for two times in a row, it means that the sensor is broken and the controller application shall (for the sake of safety) turn the heater off and exit.

If you like, you can pause here and think about how to implement and test this application.

Done? Here’s my naive implementation:

Does this code implement the requirements correctly? I don’t know for sure, but I assume it does. Do you agree?

Correct or buggy, it’s hard to agree. But we can easily agree on one thing: this code is utterly untestable. In the next part, I’ll refactor it such that it becomes one hundred percent testable, which allows us to move from correctness assumptions to knowledge. Stay tuned!

Next episode