Pointers in C, Part IV: Pointer Conversion Rules

Pointers in C

“Failure is the key to success; each mistake teaches us something.
— Morihei Ueshiba

Sometimes, someone walks up to you and claims that there is a bug in your well-crafted code. Then, after having successfully proved that individual wrong, it occurs to you that there is indeed a bug—albeit a different one! Those are quite humbling experiences, but experiences that we should be most grateful for.


This episode was triggered by feedback that I received from a reader regarding a “Dangerously Confusing Interfaces” post. In said post, I advise that instead of accepting a pointer to “uncopied” memory like this:

‘WriteAsync’ should rather take a pointer to an opaque data structure named ‘uncopied_memory’:

“uncopied” memory means that for the sake of efficiency, the called function doesn’t copy the provided data but instead expects you to keep it alive and unchanged while the called function is executed asynchronously. Since the suggested interface change requires an explicit cast to an ‘uncopied_memory’ pointer, it’s a lot less likely that a temporary buffer allocated from the stack is passed accidentally. The idea of the proposed approach is that every call to ‘WriteAsync’ requires an explicit cast that acts as a reminder to the programmer that the buffer’s contents must be preserved.

For instance, if you wanted to pass a structure that I used in the previous installment of this series to ‘WriteAsync’, you would do it like this:

But back to the question. What the reader was worried about is that since ‘measurements_t’ and ‘uncopied_memory’ are by no means compatible, wouldn’t a cast to an ‘uncopied_memory’ pointer constitute a violation of the “strict aliasing rule“?

Actually, when it comes to the “strict aliasing rule,” the fact that these structs have incompatible members doesn’t really matter—even if you accessed the stored value through a pointer to a struct with an identical set of members you would be in trouble; if the tag names of the structs are different, it already counts as a violation of the “strict aliasing rule.”

The key word here is access. If you just create a pointer to incompatible types, everything is fine. Within ‘WriteAsync’ you just cast the received ‘uncopied_memory’ pointer into a ‘uint8_t’ pointer and access the provided data byte-wise, which is always safe, as you know (if you didn’t know, go back and read my previous post).

So far, so good. We don’t access stored memory through incompatible pointers; we only do pointer conversion, which is always safe, isn’t it? I replied to my reader that everything was fine, there was no violation of the “strict aliasing rule.”

Nevertheless, I couldn’t rid myself of this nagging feeling about whether the conversion/cast is really always safe.


The venerable book “The C Programming Language” by Brian Kernighan and Dennies Ritchie has this to say on pointer conversions:

A pointer to one type may be converted to a pointer to another type. The resulting pointer may cause addressing exceptions if the subject pointer does not refer to an object suitably aligned in storage. It is guaranteed that a pointer to an object may be converted to a pointer to an object whose type requires less or equally strict storage alignment and back again without change; the notion of “alignment” is implementation-dependent, but objects of the char types have least strict alignment requirements. As described in Par.A.6.8, a pointer may also be converted to type void * and back again without change.

Let me paraphrase: pointer conversion is safe unless the alignment requirements of the target type are less or equal to the alignment requirements of the source type. The converted pointer can be converted back to the original pointer without problems.

Though, the statement “The resulting pointer may cause addressing exceptions” is not clear to me. What does it mean? If the target type has stricter alignment requirements, do you get “addressing exceptions” when you create the pointer or when you access memory through it? Let’s assume that we are on a typical platform where objects of type ‘double’ are aligned on an 8-byte boundary and ‘chars’ have no alignment requirements (‘chars’ are aligned on a 1-byte boundary, so to speak.):

The conversion (1) is 100% safe and so is the corresponding read-access (2): the alignment requirements of type ‘char’ are less than the alignment requirements of type ‘double’. (4) is 100% unsafe, but what about (3)? Aren’t we just creating a pointer? To find out, I had to dig deep into my copy of the C99 language standard. Eventually, I found what I call the “pointer conversion rule”: A pointer to an object or incomplete type may be converted to a pointer to a different object or incomplete type. If the resulting pointer is not correctly aligned for the pointed-to type, the behavior is undefined. Otherwise, when converted back again, the result shall compare equal to the original pointer. When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of the object.

There you have it and much more precise than the paragraph from “The C Programming Language.” Believe it or not—statement (3), the sheer pointer conversion already gets you into the realm of undefined behavior. Who knew?

So what does this mean regarding the conversion/cast from a ‘measurements_t’ pointer to an ‘uncopied_memory’ pointer? As we know from the standard, it would be safe if the alignment requirements for ‘uncopied_memory’ were less or equal to the alignment requirements of ‘measurements_t’.

In the previous example, we had to deal with primitive types (‘char’, ‘double’) whose alignment requirements can easily be determined. In order to find out about the alignment requirements for structs, we need to dive once more into the C99 standard document: A pointer to a structure object, suitably converted, points to its initial member (or if that member is a bit-field, then to the unit in which it resides), and vice versa. There may be unnamed padding within a structure object, but not at its beginning.

Meditate on this for a while. Paraphrased, this all means that the alignment requirements of a struct are the same as the alignment requirements of a struct’s first member. So the question boils down to this: Are the alignment requirements of a ‘void’ pointer (‘uncopied_memory’s first member) less or equal to the alignment requirements of a ‘char’ (‘measurements_t’s first member)?

Of course, they’re not! A pointer type (like void*) is more or less just an integer type in disguise that is capable of holding all the addresses of your system and as such, pointer types have the same alignment requirements as regular integer types. On a 32-bit platform, pointers typically comprise 4 bytes. Thus, on typical 32-bit platforms, they will need to be aligned on 4-byte boundaries.

By contrast, a character (like the first element of measurements_t) comprises exactly one byte and thus has no alignment requirement—it can be stored at any address in memory.

Since the alignment requirements of the first element of ‘uncopied_memory’ are stronger than the alignment requirements of ‘measurements_t’, we can conclude that my advice to cast to ‘uncopied_memory’ may yield undefined behavior. Not because of the “strict aliasing rule,” but because of a violation of the “pointer conversion rules.”

To solve the problem, the type of the ‘dummy’ member of ‘uncopied_memory’ needs to be changed to ‘char’, a type that has the weakest alignment requirements. I have updated the “Dangerously Confusing Interfaces” post accordingly.

Yet Another Software Metaphor: Pitoning

“Gravity wins over all other known forces”
— Andrea M. Ghez

As software developers, we have to deal with conflicting goals on a daily basis. One example is finding an answer to the question whether to hack out new features or clean up existing code. Or, in slightly more professional words: shall we make progress in terms of functionality or in terms of quality?


Non-technical/business people usually don’t understand the importance of software quality and tend to err on the side of rapid feature development—they assume that by focusing on features, they can sell their product sooner. Software developers by and large do understand the importance of software quality but usually also err on the side of feature development. Why? Because it’s much more fun to work on something new than doing mundane quality improvements.

As a result, many projects focus almost exclusively on churning out features. What these teams often fail to realize is that not only are features progressing rapidly—the overall risk is progressing rapidly as well.

In projects that opt for such a “code-and-fix” model, software entropy (some prefer the term technical debt) will constantly increase and even if these projects get finished one day (which is really not that certain) the messy code base might not be strong enough to support the features required for the next major release. Even worse, nobody is able to understand and maintain the messy code apart from the original authors, who have probably left the company in the meantime, anyway.


Software quality is yet another term for investment protection. It ensures the money that went into a project is properly insured. For all but the small, short-lived projects such insurance is crucial for the long-term success of any company.

Metaphors and analogies like “investment protection” and “technical debt” have quite a history in software development. For decades we have talked about “building software,” “software architecture,” “bugs,” and rightly so: by comparing something we are not familiar with to something we understand well, we can gain insights that help us understand the unfamiliar topic.


Recently, I had an idea for yet another software-related metaphor, which I call “Pitoning.” Pitoning compares software development to rock climbing, and it helps in finding trade-offs between feature progress and quality.

With the exception of free solo climbers, rock climbers don’t rush to the summit without breaks. Doing so would be extremely fast, but also highly risky—even a minor slip could be fatal. Instead, after they have made some progress, sane rock climbers will drive a piton into the rock and hook into it before climbing on. This “pitoning step” costs some time, of course, but it increases the probability that climbers will make it safely to the top significantly.

When we set out to develop a feature, we typically don’t know what’s the best design, so we experiment and try out different approaches. During this exploratory phase, we don’t want to waste time making the code as beautiful as possible, since it might be replaced by something else, eventually. However, once we have a working solution, we do the pitoning; that is, we add missing unit tests, we refactor, we make the code robust and in general industrial-strength, which may sometimes even include drawing UML diagrams and writing documentation.


The key concept behind pitoning is that a) experimentation and learning are not just good but essential and b) cleanup is done immediately after a working solution has been implemented—not at “some” point in the future. In certain cases, the pitoning phase will be longer than the exploratory phase. But skipping pitoning is not an option—gravity will surely get us. Many products and even whole companies have gone bust because of product recalls and lost reputation.

Oops, They Did It Again!

“There are no secrets that time does not reveal.”
— Jean Racine

In an act of unprecedented arrogance, Apple once again dictated to their users what they believe is good for them — this is yet another example of how they take away their users’ freedom.

Yesterday, they finally admitted what measurements by various users had suggested earlier: one year ago Apple introduced a feature that deliberately slows down the performance of older iPhone models in order to “deliver the best experience to their customers”.

This is, of course, total BS. It’s obvious that the real motive was to boost sales of their new products. Why? Because they kept it secret. Period.

If they really had the best of their customers in mind, they would have announced such a drastic step in good time. Further, they would have given their users a choice (something totally alien to them) by presenting a simple opt-in dialog:

Dear loyal customer, we’ve discovered that the capacity of your phone’s battery has become low. We strongly suggest that you either a) buy one of our latest models, b) replace your battery (which will cost you many $$$, because you can’t do it yourself), or reduce the CPU performance which will DELIVER THE BEST EXPERIENCE to you. What do you want us to deliver to you?

1. Our latest iPhone model
2. A replacement battery
3. The best experience

Some time ago, I wrote a post which details why I don’t buy iThings and this recent incident just adds to the list. However, I’m enough of a realist to doubt that it will reduce the total number of iThings under this year’s Christmas trees.

Pointers in C, Part III: The Strict Aliasing Rule

“Know the rules well, so you can break them effectively.”
— Dalai Lama XIV

One of the lesser-known secrets of the C programming language is the so-called “strict aliasing rule”. This is a shame, because failing to adhere to it takes you (along with your code) straight into the realm of undefined behavior. As no one in their right mind wants to go there, let’s shed some light on it!


First of all, we have to clarify what “aliasing” really means, or rather aliasing of pointers. Take a look at this example:

Here, ‘p1’ and ‘p2’ are aliased to the same object ‘value’; that is, they point to the same object. If you update ‘value’ through ‘p1’:

a read through ‘p2’ will reflect this change:

Because of the possibility of aliasing, a C compiler is prevented from applying certain optimizations. Consider:

You might think that any decent compiler would generate simplified code equivalent to this:

It’s not a matter of decency — the compiler just can’t do this optimization! Here’s the assembly output that clearly shows that the return value is loaded from memory:

The optimization is not possible because the caller could call ‘silly’ like so:

In this case, ‘x’ and ‘y’ are aliased to the same ‘value’, which means ‘silly’ must return 1 not 0. Consequently, ‘*x’ must be read from memory, every time. Period.


If you think about it, even though it may happen, pointer aliasing won’t happen very often in practice. Why waste so much potential for optimization for the uncommon case? Most likely, the folks from the C standards committee had the same line of thinking. They introduced rules that state when pointer aliasing must not happen. Enter the strict aliasing rule.

To facilitate compiler optimization, the strict aliasing rule demands that (in simple words) pointers to incompatible types never alias. Pointers to compatible types (like the two ‘int’ pointers ‘x’ and ‘y’ in ‘silly’) are assumed to (potentially) alias. Let’s make the pointer types incompatible (‘short*’ vs. ‘int*’):

As you can see, this time no load from memory is performed — 0 is returned instead. The optimization is possible because the compiler assumes that aliasing is not allowed in this case.


But what happens if pointers to incompatible types nevertheless alias? After all, this can happen quite easily. Maybe not in the ‘silly’ example, but in real-world production code:

In an attempt to convert data stored in a buffer (maybe read over a network connection) into a high-level structure, a pointer to ‘struct measurements_t’ is aliased with a pointer to a ‘uint_8’. Since both types are incompatible (pointer to struct vs. pointer to ‘uint8_t’) this code is a violation of the strict aliasing rule. Experienced C developers most likely recognized immediately that this code yields undefined behavior, but they would have probably attributed it to struct padding and alignment issues. The real reason, as we know by now, is a violation of the strict aliasing rule.


So what exactly is the strict aliasing rule and what does “type compatibility” mean? Here’s an excerpt from the ISO C99, standard, chapter 6.5:

An object shall have its stored value accessed only by an lvalue expression that has one of the following types:

  • a type compatible with the effective type of the object,
  • a qualified version of a type compatible with the effective type of the object,
  • a type that is the signed or unsigned type corresponding to the effective type of the object,
  • a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object,
  • an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or
  • a character type.

Such Standardeese is often hard to digest, so let me try to clarify it a bit. Aliased pointer access is fine if:

1. The pointed-at types are identical. Note that typedefs are just type aliases and don’t introduce new types:

2. The pointed-at types are identical apart from the “signed-ness” (e. g. ‘int’ vs. ‘unsigned int’).
3. The pointed-at types are identical apart from qualification (e. g. ‘const int’ vs. ‘int’).
4. The rule “an aggregate or union type that includes one of the aforementioned types among its members” is highly confusing and probably doesn’t mean much. Check this out for details.
5. The pointed-at types are different, but the pointed-at type through which the access is made is a pointer to character:

Conversely, aliased pointer access is not defined if the pointed-at types are fundamentally different. Note that this includes pointers to structs that are identically defined but have different tag names:


The strict aliasing rule was introduced to give the compiler vendors some leeway regarding optimizations. By default, the compiler assumes that pointers to (loosely speaking) incompatible types never alias. As a consequence, you, the programmer, have to make sure that this rule is obeyed.

Here’s some disquieting news: a lot of existing code isn’t conforming to the strict aliasing rule, but the code works (or appears to work) fine anyway. As an example, the ‘convert’ function above, which aliases a struct to an array of bytes might work fine on an Intel x86-based platform, which supports unaligned memory access. However, if you use ‘convert’ on an ARM-based platform, you will get a “bus error” exception that will crash your system. In other cases, nonconforming code just works by coincident, with a particular compiler, or a particular compiler version at a particular optimization level.

To me, knowing about the strict aliasing rule is as important for every systems developer as knowing about the other systems programming “secrets” like alignment, struct padding, and endianness.


I’m indebted to the authors of the following two blog post. From the first post, I shamelessly stole the idea for the ‘silly’ function, which the author originally called ‘foo’, which I found silly :-) The second post gives a much more detailed coverage of the strict aliasing rule and also discusses the interesting technique of “casting through a union”.

A GCC Compiler Mistake

“Most of the evil in this world is done by people with good intentions.”
— T.S. Eliot

Errors, defects, bugs, blunders — when we talk about software-related errors, we often use terms loosely and synonymously — but there are differences. For instance, in his book “The Design of Everyday Things“, Donald A. Norman makes a clear distinction between “mistakes” and “slips”:

“Errors come in several forms. Two fundamental categories are slips and mistakes. Slips result from automatic behavior, when subconscious actions that are intended to satisfy our goals get waylaid en route. Mistakes result from conscious deliberations.”

In short: mistakes are the result of faulty ideas whereas slips are errors made when implementing an idea. Usually, slips are not just easy to make, but also easy to fix. Fixing mistakes is typically much harder.

One of the easiest slips to make in C/C++ is to inadvertently do a boolean test on an assignment expression:

which is equivalent to:

While in some rare cases this is exactly what the developer had in mind, in 99% of all cases it’s not. Hence, boolean-testing assignments is explicitly banned by many C/C++ coding standards and frowned-upon by most developers.

But what’s all the fuzz about, you might ask. If an unlucky developer forgets to type the second ‘=’, any decent 21st century compiler surely generates a warning, doesn’t it? Well, the answer is, as we shall see, both, yes and no.

If you compile the example above with GCC (I’ve tried version 5.4.0) using options ‘-W -Wall’, you do get a warning:

warning: suggest parentheses around assignment used as truth value

GCC’s reasoning is this: if developers really wanted to truth test the assignment (there are still people out there who do, as strange as this may sound), they need to put an extra pair of parentheses around the assignment, to show their intend:

Requiring an extra set of parentheses seems to be a neat idea, but it’s the devil in disguise. For one thing, it reminds me of Sledge Hammer saying “Trust me, I know what I’m doing” (which was usually entailed by disaster), for another, it doesn’t work reliably. In order to explain, I first need to put the same slip in a slightly more complicated expression:

In this case, you not just get a warning, your compiler will refuse to compile this code. Why? According to C’s precedence rules, the assignment operator has lower priority than the ‘&&’ operator, which means that the code is equivalent to

The C language standard says that the result of an ‘&&’ expression is a so-called “rvalue” and an rvalue is more or less read-only. Thus, assigning ‘d’ to it is just not possible and GCC is right when it barks:

error: lvalue required as left operand of assignment

A slip that doesn’t compile is a kind slip, you might think, but read on. We only got lucky by accident, so to speak.

Many coding standards, like MISRA, for instance, require that you put parentheses around subexpressions to clearly show what precedence you have in mind, instead of relying on obscure operator precedence rules. Hence, instead of

you have to write

MISRA exists to make coding errors unlikely, but if a MISRA-abiding developer forgets the second ‘=’, he’s out of luck, at least if he’s using GCC:

Now the devil reveals himself: since the parentheses are properly placed, there is no attempt to assign to an rvalue, so there won’t be a compile-time error and because of GCC’s “parentheses feature” mentioned above, GCC doesn’t issue a warning, either.

Early in my career as a software developer, I read the aforementioned book “The Design of Everyday Things” and I believe it left a mark on me. One of the book’s unforgettable key lesson is this:

“When you have trouble with things — whether it’s figuring out whether to push or pull a door or the arbitrary vagaries of the modern computer and electronics industry — it’s not your fault. Don’t blame yourself: blame the designer.”

GCC’s “extra parentheses” feature is far from neat design — it’s rather bad design that doesn’t work in all contexts and gives developers a false sense of security. It was deliberately put in and correctly implemented but the idea was wrong from the outset. Thus, it’s not a slip, but obviously a mistake.

Dangerously Confusing Interfaces IV: The Perils of C’s “safe” String Functions

“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.”
–Mark Twain

Buffer overflows are among the most frequent causes of security flaws in software. They typically arise in situations such as when a programmer is 100% certain that the buffer to hold a user’s name is big enough — until a guy from India logs in. Thus, well-behaved developers always use the bounded-length versions of string functions. Alas, they come with differing, dangerously confusing interfaces.


Let’s start with ‘fgets‘:

No matter what users type into their terminals, ‘fgets’ will ensure that ‘user_name’ is a well-formed, zero-terminated string of at most 29 characters (one character is needed for the ‘\0’ terminator). The same goes for the ‘snprintf‘ function. After executing the following code

‘buffer’ will contain the string “The”, again, properly zero-terminated.

Both functions follow the same, easy-to-grasp pattern: you pass a pointer to a target buffer as well as the buffer’s total size and get back a terminated string that doesn’t overflow the buffer. Awesome!


In order to copy strings safely, developers often reach for ‘strncpy‘ to guard themselves against dreaded buffer overruns:

Unfortunately, this is not how ‘strncpy’ works! We assumed that it followed the pattern established by ‘fgets’ and ‘snprintf’ but that’s not the case. Even if ‘strncpy’ promises that it never overflows the target buffer, it doesn’t necessarily zero-terminate it. What it does is copy up to ‘sizeof(buffer)’ bytes from ‘user_name’ to ‘buffer’ but if the last byte that is copied is not ‘\0’ (i. e. ‘user_name’ comprises more than ‘sizeof(buffer)’ characters), ‘strncpy’ leaves you with an untermiated string! A traditional approach to solve this shortcoming is to enforce zero-termination by putting a ‘\0’ character as the last element of the target buffer after the call to ‘strncpy’:

Using ‘strncpy’ without such explicit string termination is almost always an error — a rather insidious one, as your code will work most of the time but not when the buffer is completely filled (i. e. your Indian colleague “Villupuram Chinnaih Pillai Ganesan” logs on).

Boy, oh boy is this inconsistent! ‘fgets’ and ‘snprintf’ give you guaranteed zero-termination but ‘strncpy’ doesn’t. A clear violation of the principle of least surprise. Apparently, ‘strncpy’ fixes one safety problem and at the same time lays the foundation for another one.


Can it get worse? You bet! How do you think ‘strncat‘, the bounded-length string concatenation function, behaves? Ponder this code:

But this is wrong, of course: the third argument to ‘strncat’ (let’s call this argument ‘n’) is not the size of the target buffer. It is the maximum number of characters to copy from the source string (‘string2’) to the destination buffer (‘buffer’). If the length of the source string is greater or equal to ‘n’, ‘strncat’ copies ‘n’ characters plus a ‘\0’ to terminate the target string. Confused? Don’t worry, here’s how you would use it to avoid concatenation buffer overruns:

Yuck! What’s the likelihood that people remember this correctly?


Even if the different interfaces and behaviors of the bounded-length string functions in the C API make sense for certain use cases (or made sense at some point in time), the upshot is that they confuse programmers and potentially lead to new security holes when in fact they were supposed to plug them. What’s a poor C coder supposed to do?

As always, you can roll your own versions of bounded/safe string functions or use my safe version of ‘strcpy’. If you rather prefer something from the standard library, I’d suggest that you use ‘snprintf’ as a replacement for both, ‘strncpy’ and ‘strncat’:

Looks like ‘snprintf’ is the swiss army knife of safe string processing, doesn’t it? The moral is this: use whatever you’re comfortable with, but refrain from using ‘strncat’ or ‘strncat’ directly.

More dangerously confusing interfaces…