Automatic tests and system library: a short riddle

Today, let me offer a slightly different kind of post. For once, I will not give any advice but rather propose a riddle for you to solve.

As I have already mentioned in a previous post, regularly running automatic test suites (à la JUnit) is a guarantee against program decay. With a continuous integration server such as Jenkins, reports periodically built from test results present an up-to-date status of the code condition. Test suites are a barrier against regression. Thus enabling, sometimes extremely aggressive, code refactoring. Some teams even use automatic tests as a non perishable form of documentation, or alternatively as executable specifications. The cost of the manual verification phase can also be reduced, by progressively converting the most repetitive testing scenarios into automatic tests. In short, automatic tests let you attain a surprisingly high level of software quality.

So, it is considered good practice, to at least add one automatic test, for every bug found and fixed. To adopt test driven development may even be more rewarding, but that is another story. However, making a manual verification scenario totally automatic may sometimes prove particularly tricky. This is, in particular, the case for any piece of software which depends on some low level library upon which the programmer has no control whatsoever. Herein lies the crux of today's riddle: how to write automatic tests for code relying on non modifiable external libraries.

Let us consider a concrete Java example:

public class Program {
    public static void main(String[] arguments) throws Exception {
        if (arguments.length < 1) return;
        Program program = new Program();
        program.process(arguments[0]);
    }

    public void process(String fileName) throws Exception {
        ...
        FileInputStream file = new FileInputStream(fileName);
        ...
        int value = file.read();
        ...
        file.close();
    }
}

During its execution (method process), this program opens the file (new FileInputStream(fileName)) whose name is passed as argument to the entry point (method main). It then reads some content from this file (int value = file.read()).

Let us suppose the validation team found the following problem: during one of its run, the program brutally stopped. The call to method read unexpectedly threw a java.io.IOException. This may happen, for instance, when accessing a distant file system which suddenly becomes unavailable because of a severed cable. This is clearly an execution context which is difficult to systematically reproduce at each run of an automatic test.

How would you write an automatic non-regression test which replays this scenario?

Oedipus and the Sphinx

by Ingres, 1808

by Gustave Moreau, 1864

by Salvador Dali, 1960