Characterization Testing Explained

Characterization Testing Explained

This is a technique I’ve been using for years but it wasn’t until recently that I realized that it actually has a specific name. Using Characterization Testing we lock the current behavior in one or several tests before we go refactor code (usually untested legacy) to be comfortable that our improvement didn’t break any existing behavior.

“A characterization test is a means to describe (characterize) the actual behavior of an existing piece of software, and therefore protect existing behavior of legacy code against unintended changes via automated testing.”

When you find a piece of code (usually coupled legacy code) that is completely untested it can be a bit intimidating to start refactoring without knowing what this change might break or if it will introduce unexpected behavior somewhere else in our application (or consuming applications).

Sure, you might confirm that the code still compiles after you’ve refactored it but that’s not a 100% indicator that it still works depending on all various conditions, states and inputs. Unless this piece of code or feature is well documented (which is usually not the case) "changing untested code is very much like shooting in the dark".

What I like to do in these situations is to lock the current behavior inside several tests (either Unit- or Integration test or both) so that I know how it works before I start improving it. 

These kinds of characterization tests don't have to be pretty and they usually aren't, since you are not “allowed” to make any improvements in terms of testability to the legacy code to create a test worth running. 

Note that these kinds of tests “do not infer correctness of the results. It merely helps detect unwanted effects of software changes.”

Once we have this test setup we can start improving the legacy code and be comfortable that our improvements did not break the previous behavior based on the conditions we covered in our tests. 

Eventually we’ve made enough improvements to the legacy code so that our previous (not so pretty) characterization test can also be improved and refactored, "as long as we dont change the expected output from the previous conditions, states and inputs" (unless you announce a breaking change to consumers).

Now I would lie to you if I said that I always use this technique whenever I’m faced with untested code. Sometimes the code in which I’m refactoring is simply impossible to lock in a test in its current state without first breaking it from the ground up, but I at least try to create at least one characterization test as soon as I’ve done enough refactoring to have a functioning test setup.

These kind of tests bring a log of comfort especially when working with legacy code and it's a technique I highly recommend.

Cheers friends! ❤️