Documentation Beyond Tests

Recently, I encountered an interesting, if twisted, interpretation of the "Tests as Documentation" principle. Reviewing some more-complicated-than-usual code, I asked the responsible colleague, if there was some documentation and, if not, that it would a be good idea to add some. He quickly responded that, of course, he'd provided good test coverage for the code at hand and since we worked under the "tests as documentation" principle it was all there. Indeed, he did write neat, compact tests, who will do their job, and still, in my opinion, he missed the point.

Coming from agile development methodologies, the idea behind this phrase is to minimize unnecessary and inflexible documentation in favor of tests. The latter are maintained anyways and ideally describe the code's behavior precisely. From my point of view, a more specific meaning depends on the kind of test we are talking about and what is documented by it.

Acceptance / customer tests belong to a class of checks, that verify on an abstract level that the system meets its business requirements. Here, the detailed implementation is irrelevant: The question is /if/ and not /how/ the business value is achieved. At this level, the documentation basically is a list of features that are hopefully of importance to someone. In case of larger systems, where there are tons of otherwise forgotton features, this is valuable knowledge.

In the area of unit tests the focus lies on documenting (and fixing) the behavior of a piece of code. By just looking at the tests, it should immediately follow what the code requires as input and what it will produce as output. This is even more so in the case of white-box tests that make the interface of the component precise. As a consequence, the test should be /readable/, in the sense that it ought to come as close to natural language as is feasible. This kind of low-level description is what my collegue meant, and he was right in the sense, that his tests did deliver this.

So, these two kinds of tests fix, define and therefore document behavior (and structures) of a system. Unfortunately, none of them considers /why/ something has been done in a particular way (or at all). Of course, this is by design - the tests do their jobs nicely. My point is, that there is knowledge, that - while absolutely essential - is not and cannot be provided by these tests alone.

Interestingly, on the business side this idea is well understood. If a company does something, there needs to be a good reason to do it and to do it this way. Each of the features that the acceptance tests verify, will be either part of a customer contract or a strategic plan from management. Requirements engineering has found nice solutions for this, e.g. user stories. Because, in the end, cost-effectiveness and accountability are of the essence.

At the level of code, this notion has not yet surfaced equally. If I see some complicated piece of code, I don't only want to know what it does, and if it does it. I want to know why we need it, why the developer chose this particular implementation, and why it must be so complicated.

I believe that the tests above are not capable nor intended to answer these important questions and plead in favor of good ol', natural language, classical documentation.

Handy Tools for Java Development

Some links to basic, but nevertheless interesting tools for java development, especially in larger projects. All of them are available as eclipse plugins.

Code quality

FindBugs: Very useful to detect (potential) bugs, deficiencies and general ugliness.

UCDetector finds code that is not referenced and therefore (possibly) not used anymore. (Even points out that something is referenced by testing code, only.) Works on classes as well as on more fine-grained levels. Also proposes to reduce visibility of structures, e.g. to make things private.

EclEmma is a java code coverage tool. This becomes a powerful utility in combination with JUnit-Tests. Running the unit-test using the coverage starter lets EclEmma analyze the tested code. After the tests complete, the code covered by your tests is visualized in the editor window. (Green for covered, red for not.) This lets you easily identify code fragments that no test ever "visited".

Testability Explorer: Reports on the testability of your classes in terms of a number. Gives a quick overview of which classes definitely need some work.

Debugging

VisualVM: Stand-alone program that allows you to connect to a running java-prog and dump its heap. The latter can then be opened for analysis, e.g. by the Memory Analyzer (see below). An interesting feature is to let VisualVM automatically dump the heap in case the program under scrutiny receives an OutOfMemoryError. This way you don't need to sit next to the screen all the time, until the memory leak occurs.

Note that you need to use the *Java 6 JDK*. Both VisualVM and the inspected program need to use it. If your default is another Java installation, then you need to point to the Java 6 JDK using the command-line argument "--jdkhome".

Memory Analyzer (MAT) does just that: It loads a heap file that might have been created by VisualVM for instance. Then, it tries to detect classes responsible for a memory leak and more. See the home page for details.

Management

Projectset: Manages dependencies between multiple projects. Takes versions / branches of VCSs into account. The adaptor to CVS is included, the ones for SVN are available from the project's homepage.

PDDL Examples

Going to collect some examples for PDDL, problem collections from competitions, especially interesting configurations, etc.
For PDDL being the primary language of planning, there are way to little (centralized) resources on the topic, so this is my lil' contribution to help people find something and get started.