This article covers Chapter 9, Practical: Building a Unit Test Framework.
To build a minimal testing library, I need nothing more than tests and results. To keep reporting as simple as possible, I will start with console output. The
report-result function tests a
result, and prints
FAIL, plus a
form with supporting detail:
Now any function can be a test. The detail message can often be the same form that caused the error, so I will pass the same form twice: once for evaluation, and again (quoted!) for use in the detail message:
The console output for
test-+ looks like this:
The fact that I want to pass the same form twice, but with different evaluation semantics, just screams macro. Sure enough, I can clean up the code with a macro:
The macro expands the form twice, once for evaluation and once quoted for the detail message. Now I can replace calls to
report-result with simpler calls to
Hmm. The calls to
check are cleaner than the calls to
report-result in the earlier example, but the
check itself still looks repetitive. Solution: a better
check macro that can handle multiple forms:
The quoting and unquoting is a little more complex–play around with
macroexpand-1 to see how it works.
With the better
check in place, test functions are quite simple:
So far I have tests and console output. Next, I need some way to aggregate a set of checks into a single, top-level "checks passed" or "checks failed".
I would like to simply
and together all the individual checks, but that does not quite work. As in many languages, Clojure's
and short-circuits and stops evaluating when it encounters a logical
false. That's no good here: Even if one test fails, I still want all the tests to run.
Since it is a question of optional evaluation, a macro is appropriate. The
combine-results macro works like
and, but it always evaluates all the forms:
check can use
combine-results instead of
All existing functionality still works, and now I can see a useful return value from a test.
Tests ought to have names. In fact, tests ought to support multiple names. You can imagine a test detail report saying:
Check math->addition->associative passed: ...
associative is the name of a check,
addition is the name of a function, and
math is the name of another function that called
First, I need a variable to store a sequence of names:
Printing the variable as part of a result is easy:
Now for the hard part: populating the collection of names. For this, I will introduce a
The macro expansion perfomed by
deftest is nothing new:
deftest turns around and
defns a new function named
name. The interesting part is the call to
binding, which rebinds
*test-name* to a new collection built from the old
*test-name* plus the name of the current test.
The new binding of
*test-name* is visible anywhere inside the dynamic scope of the
binding form. The dynamic scope includes any function calls made inside the binding, and their function calls, and so on ad infinitum … or until another
binding performs the same trick again. This gives exactly the semantics we want:
test-namean an argument all over the place. Nested functions "remember" a stack of their caller's names through
binding. Code after the
bindingwill never see the values
*test-name*takes during the
deftest in place, I can defined a hierarchy of nested tests:
test-all-of-nature will demonstrate multiple levels of nested name in a test report:
From here, better formatting of the console message is just mopping up.
When I first read Practical Common Lisp, this was my favorite chapter. The testing library evolves quickly and naturally to a substantial feature set. (In case you didn't keep count, the entire "framework" is less than twenty lines of code.)
Try implementing the unit-testing example in your language of choice. Don't just implement the finished design. Work through each of the iterations described above:
I would love to hear about your results, and I will link to them here.