Documentation Beyond Tests

Recently, I encountered an interesting, if twisted, interpretation of the "Tests as Documentation" principle. Reviewing some more-complicated-than-usual code, I asked the responsible colleague, if there was some documentation and, if not, that it would a be good idea to add some. He quickly responded that, of course, he'd provided good test coverage for the code at hand and since we worked under the "tests as documentation" principle it was all there. Indeed, he did write neat, compact tests, who will do their job, and still, in my opinion, he missed the point.

Coming from agile development methodologies, the idea behind this phrase is to minimize unnecessary and inflexible documentation in favor of tests. The latter are maintained anyways and ideally describe the code's behavior precisely. From my point of view, a more specific meaning depends on the kind of test we are talking about and what is documented by it.

Acceptance / customer tests belong to a class of checks, that verify on an abstract level that the system meets its business requirements. Here, the detailed implementation is irrelevant: The question is /if/ and not /how/ the business value is achieved. At this level, the documentation basically is a list of features that are hopefully of importance to someone. In case of larger systems, where there are tons of otherwise forgotton features, this is valuable knowledge.

In the area of unit tests the focus lies on documenting (and fixing) the behavior of a piece of code. By just looking at the tests, it should immediately follow what the code requires as input and what it will produce as output. This is even more so in the case of white-box tests that make the interface of the component precise. As a consequence, the test should be /readable/, in the sense that it ought to come as close to natural language as is feasible. This kind of low-level description is what my collegue meant, and he was right in the sense, that his tests did deliver this.

So, these two kinds of tests fix, define and therefore document behavior (and structures) of a system. Unfortunately, none of them considers /why/ something has been done in a particular way (or at all). Of course, this is by design - the tests do their jobs nicely. My point is, that there is knowledge, that - while absolutely essential - is not and cannot be provided by these tests alone.

Interestingly, on the business side this idea is well understood. If a company does something, there needs to be a good reason to do it and to do it this way. Each of the features that the acceptance tests verify, will be either part of a customer contract or a strategic plan from management. Requirements engineering has found nice solutions for this, e.g. user stories. Because, in the end, cost-effectiveness and accountability are of the essence.

At the level of code, this notion has not yet surfaced equally. If I see some complicated piece of code, I don't only want to know what it does, and if it does it. I want to know why we need it, why the developer chose this particular implementation, and why it must be so complicated.

I believe that the tests above are not capable nor intended to answer these important questions and plead in favor of good ol', natural language, classical documentation.

Handy Tools for Java Development

Some links to basic, but nevertheless interesting tools for java development, especially in larger projects. All of them are available as eclipse plugins.

Code quality

FindBugs: Very useful to detect (potential) bugs, deficiencies and general ugliness.

UCDetector finds code that is not referenced and therefore (possibly) not used anymore. (Even points out that something is referenced by testing code, only.) Works on classes as well as on more fine-grained levels. Also proposes to reduce visibility of structures, e.g. to make things private.

EclEmma is a java code coverage tool. This becomes a powerful utility in combination with JUnit-Tests. Running the unit-test using the coverage starter lets EclEmma analyze the tested code. After the tests complete, the code covered by your tests is visualized in the editor window. (Green for covered, red for not.) This lets you easily identify code fragments that no test ever "visited".

Testability Explorer: Reports on the testability of your classes in terms of a number. Gives a quick overview of which classes definitely need some work.

Debugging

VisualVM: Stand-alone program that allows you to connect to a running java-prog and dump its heap. The latter can then be opened for analysis, e.g. by the Memory Analyzer (see below). An interesting feature is to let VisualVM automatically dump the heap in case the program under scrutiny receives an OutOfMemoryError. This way you don't need to sit next to the screen all the time, until the memory leak occurs.

Note that you need to use the *Java 6 JDK*. Both VisualVM and the inspected program need to use it. If your default is another Java installation, then you need to point to the Java 6 JDK using the command-line argument "--jdkhome".

Memory Analyzer (MAT) does just that: It loads a heap file that might have been created by VisualVM for instance. Then, it tries to detect classes responsible for a memory leak and more. See the home page for details.

Management

Projectset: Manages dependencies between multiple projects. Takes versions / branches of VCSs into account. The adaptor to CVS is included, the ones for SVN are available from the project's homepage.

Rapid Prototyping and Software Engineering?

Today, somehow, I stumbled upon 3d printing, a form of Rapid Prototyping. This reminded me of a talk, delivered by Bre Pettis at 25C3 last year. Again, I began to wonder where this might not only be cool - which it undoubtfully is - but where it also could be useful. (This is not so obvious.) So, as expected, I thought about where to use rapid prototyping at work, i.e. in software engineering. And I think I've identified a spot where it might come in handy. (There are likely to be many more, and I'll post about them, if they come to mind.)

Some time ago, I had an idea to better understand where bad code eats up manpower in terms of support. I figured, given an architecture in adequate detail, it might be useful to print it on a sheet of paper and put that on a pin board. Then, using for example colored pins, we could get an impression of where problems lie. For example, one might want to remember performance bottle necks, occurrences of bugs or black holes in the code, regions that no one understands nor wants to touch.

I've been too busy studying lately to actually put this forward, but still think its a good idea. Easy to use, trivial to evaluate. The only drawback is that it really doesn't look all that nice. (There is worse, but still ...) And here comes 3d printing and saves the day.

Take a look at the following FMC diagram, depicting an architecture of a (still) fictitious browsergame.

Now imagine this as a 3d model, as architects either show them on screen or build them in miniature. The blocks (agents) would rise, the nested ones even further above, the bubbles (storages) would actually be round. Well, the channels might just pose a problem, but anyways! Using materials like polystyrene or plaster, we would be provided with a model of our system that we could make use of as discussed before. With pins, we will be able to mark hot spots and so on. Only this time, it will look cool.

Unfortunately, the 2d solution would work just as fine. So, we're back at square one, having found ourselves a neat solution (to a problem, actually) - but is it actually useful?

If I find out, I'll do another post.