Saturday, May 30, 2015

Economy of Means: On the Elimination of Inheritance (3/6)

From inheritance to decorator

This is one of the advanced questions I sometimes ask in interviews:
Suppose you have three classes A, B and C. Class A has a method f which performs some task. Classes B and C inherit A. Both classes override method f to add some extra behaviour to the base version. Class B outputs some log:
class B extends A {
    public override void f() {
        super.f(); 
        log("f was called");
    }
}
, while class C increments some counter:
class C extends A {
    private int counter;

    public C() {
        this.counter = 0;
    }

    override public override void f() {
        super.f();
        this.counter++;
    }
}
Now, you would like to instantiate a class which mixes all behaviours together: the base behaviour, the log and the counter increment. How would you do it? I am expecting a clean design, which can easily evolve in the future if needed.

Most candidates will struggle, yet the core idea is again to use a combination of interface and composition in replacement of inheritance. Through a gradual refactoring, we flatten the class hierarchy and introduce the decorator pattern.

First let us define interface I, which declares method f:
interface I {
    void f();
}
Then, we can have all classes implement interface I. Next, we can modify class B so that, instead of inheriting class A, it expects an object with interface I as argument of its constructor. In its implementation of method f, class B calls the base version of method f from this interface rather than from its super-class:
class B implements I {
    private I i;

    B(I i) {
        this.i = i;
    }

    void f() {
        this.i.f(); 
        log("f was called");
    }
}
We do the same with class C:
class C implements I {
    private I i;
    private int counter;

    public C(I i) {
        this.i = i;
        this.counter = 0;
    }

    public void f() {
        this.i.f();
        this.counter++;
    }
}
 
Classes B and C are now both decorators: if we view classes as API transformers, then we can see they both expect an interface I as input, and they both build instances which provide the same interface I, yet adding a layer of behaviour in the middle.

Peeling the onion

Thanks to the decorator pattern, getting any combination we could wish for is extremely easy:
  • new A() provides only f,
  • new B(new A()) provides f and logging,
  • new C(new A()) provides f and statistics,
  • new B(new C(new A())) or new C(new B(new A())) provide f, logging and statistics. 
This design is extensible: one can easily code an additional decorator. It will assemble seamlessly with the other decorators during instantiation.
At last, as was already shown in the previous recipe example, we could, if necessary, implement a factory that offers shortcuts to rapidly instantiate a variety of classes, all implementing interface I.


Avoiding override

As previously mentioned here, overridden methods are sometimes a form of dead code. Indeed, writing some code in the base class only to scratch it in the children, is not very elegant. Hopefully, keyword override can be replaced by the use of interfaces. Here is how.
Let us start again from our canonical hierarchy: class A father of classes B and C. Class A defines a method f, which is overridden in C (but not in B). All other things being equals, it is preferable to create an interface I which defines method f and have class A accept this interface as one of its constructor argument.
Two additional classes which both implement interface I are created out of the definitions of f present in A and C. By splitting our implementation into smaller components, we gained flexibility:
  • We can assemble a class B with the version of method f which was originally present in C, mixed in.
  • Interface I (possibly after some renaming) can now be reused outside of its original context.

Sunday, May 17, 2015

Economy of Means: On the Elimination of Inheritance (2/6)

The path away from inheritance

This section shows more complex designs that are based on inheritance. In each case, I explain how it is possible to avoid inheritance.

Default behaviour

Writing some generic engine that drives plugins is a powerful way to factor code and speed up development. Consider an hypothetical case where plugins would have to comply with the following interface:
interface IEngineEventsListener {
    Statistics processStartup(Car);
    void processAcceleration(Car, int);
    void processDecelaration(Car, int);
    void processStop(Car);
    void processCrash(Car, Crash);
    // etc...
    void processEvent20(Driver driver);
}
As you can see, the interface is quite rich. And it can be pretty tedious to provide a full plugin implementation. Now suppose all actions are optional. So it has been decided to add a class which implements standard default behaviours for every method:
class DefaultEngineEventsListener
                    implements IEngineEventsListener {
    public Statistics processStartup(CarData) {
        /* return some default value */
    }
    public void processAcceleration(CarData, int) {
        /* do nothing */
    }
    // etc...
}
Plugin developers can now inherit from this class and override only the methods for which a specific behaviour is required.
To me, this pattern is acceptable only as long as the default behaviour is empty and there are no method with a return value. Otherwise, overridden methods and default values are a form of dead code. This goes against one of the minimalistic rules:
Never ship code which will not be executed in the production environment.
In practice, actual designs generally tend to belong to one of these two categories:
  • most plugins implement only one method, in which case methods can be registered independently,
  • most plugins implement most methods, in which case cost of implementing the whole interface every time is acceptable.
Additionally, the variety of methods can be reduced by having their signatures converge. Taking this idea to the extreme, one could substitute all methods by only one by merging all data under a common interface (the union type as described previously in section A matter of choice). But then, of course, the burden of type dispatch would shift onto the plugin implementer.
As a last note, less strongly typed programming languages such as python or javascript simply allow partial implementations. Also, the syntax to register individual methods (or closures) tends to be lighter.
 

 

Antagonism between class hierarchy and methods call chain

Let us do some cooking. A WitchSoup is a Recipe. Base class Recipe is abstract since, during initialization, it calls abstract method getAdditionalTasks:
abstract class Recipe {

    private List<Task> _tasks;

    public Recipe() {
        this._tasks = new ArrayList<Task>();
        this._tasks.add(new PrepareUstensils());
        this._tasks.addAll(getAdditionalTasks());
        this._tasks.add(new CleanKitchen());
    }

    abstract List<Task> getAdditionalTasks();

    public execute() {
        for (Task task in this._tasks) {
            task.executes();
        }
    }
}

class WitchSoup extends Recipe {

    public WitchSoup() {
        super();
    }

    List<Task> getAdditionalTasks() {
        List<Task> additionalTasks = new ArrayList<Task>();
        additionalTasks.add(new MeltButter());
        additionalTasks.add(new MashBeans());        
        additionalTasks.add(new Boil());
        return additionalTasks;
    }
}
For some of you, this code may seem awkward. Indeed, it is wrong in more than one way. First, every new recipe will result in the creation of one more class. This leads to extreme verbosity. Second, class Recipe is abstract so it can not be instantiated: it has zero value on its own.
Last and worse, a loop has now been introduced into the classes dependencies. Simple inheritance creates a dependence from a class to its parent. An abstract class also depends on its descendants. Hence, method getAdditionalTasks can be viewed as a callback in the interface which separates WitchSoup from Recipe. As you may have experienced when writing unit tests with mocks, interfaces with callbacks are always harder to reason with. Classes WitchSoup and Recipe are actually very tightly coupled and cannot any longer be thought of as separate entities.

Unfortunately, I sometimes find this pattern in real (admittedly poor) production code (often buried down under a few more obfuscation layers).
Here is how this mess could be cleaned up:
class Recipe {

    private List<Task> _tasks;

    public Recipe(List<Task> additionalTasks) {
        this._tasks = new ArrayList<Task>();
        this._tasks.add(new PrepareUstensils());
        this._tasks.addAll(additionalTasks);
        this._tasks.add(new CleanKitchen());
    }

    public execute() {
        for (Task task in this._tasks) {
            task.executes();
        }
    }
}

class RecipeBook {

    Recipe retrieveWitchSoupRecipe() {
        List<Task> additionalTasks = new ArrayList<Task>();
        additionalTasks.add(new MeltButter());
        additionalTasks.add(new MashBeans()); 
        additionalTasks.add(new Boil());
        return new Recipe(additionalTasks);
    }
}
Abstract method getAdditionalTasks was simply replaced by a standard parameter of the Recipe's constructor. As class Recipe is now concrete, the need for WitchSoup or any other specific Recipe descendant disappears. Each recipe is replaced by a different creation method belonging to the factory class RecipeBook. The class hierarchy has flattened dramatically.

Flavor network: graph of ingredients as paired in recipe


From template method to strategy

Let us deepen our understanding of keyword abstract by playing with the template method pattern. In the following code, method accumulate combines all the integers present in its list parameter. It calls the abstract method combine to process integers pairwise. Any combinator can be chosen by implementing various subclasses. For instance, class MaximumAccumulator combines the integers by choosing the maximum, while SumAccumulator performs the addition.
abstract class Accumulator {
    public int accumulate(List<Integer> input) {
        int result = 0;
        for (Integer element in input) {
            combine(result, input);
        }
        return result;
    }

    public abstract int combine(int element1, int element2);
}

class MaximumAccumulator extends Accumulator {
    public int combine(int element1, int element2) {
        if (element1 < element2) return element2;
        return element1;
    }
}

class SumAccumulator extends Accumulator {
    public int combine(int element1, int element2) {
        return element1 + element2;
    }
}
Using pattern strategy we can replace inheritance by both an interface and composition. First we introduce interface Combinator:
interface Combinator {
    int combine(int element1, int element2);
}
Then, the Accumulator class can accept a Combinator as a constructor argument instead of expecting subclasses. The class loses its abstract status:
class Accumulator {

    private Combinator combinator;

    public Accumulator(Combinator combinator) {
        this.combinator = combinator;
    }

    public int accumulate(List<Integer> input) {
        int result = 0;
        for (Integer element in input) {
            this.combinator.combine(result, input);
        }
        return result;
    }
}
At last, classes MaximumAccumator and SumAccumulator can now implement interface Combinator. We also rename them to express their roles more precisely:
class MaximumCombinator implements Combinator {
    public int combine(int element1, int element2) {
        if (element1 < element2) return element2;
        return element1;
    }
}

class SumCombinator implements Combinator {
    public int combine(int element1, int element2) {
        return element1 + element2;
    }
}
As an option, we can also provide the following factory, so that code consumers have an easy way to instantiate different kinds of accumulators:
class AccumulatorFactory() {
    Accumulator createMaximumAccumulator() {
        return new Accumulator(new MaximumCombinator());
    }

    Accumulator createSumAccumulator() {
        return new Accumulator(new SumAccumulator());
    }
}
Note how, by removing class inheritance, the code has improved in several ways:
  • Class Accumulator and implementations of Combinator are not coupled anymore. Now the combinators can be reused outside of this particular context.
  • The responsibilities are more focused. So it is easier to have class names which faithfully express their implementation's intent.
  • Unit testing will be simpler. First, since Accumulator is not abstract anymore, it can now be tested in isolation. Tests of SumAccumulator and MaximumAccumulator would have been unnecessarily complex to write and redundant since they need to exercise the common inherited code twice. Now we can easily test the combinator's code.

By allowing the father's code to call some method defined in its children, the abstract keyword couples classes even more tightly than simple inheritance does. As a first conclusion of this example and the previous one, we can say the abstract keyword represents a strong code smell. As a coding rule, it seems safer to avoid it altogether.
Naval strategy (Yi Sun-sin turtle ship)


A side note on unit testing

The previous example briefly mentioned the topic of writing unit tests. So I am taking this opportunity to quickly discuss the impact of inheritance on tests.

I am a mockist. Writing unit tests the mockist style consists in replacing all dependencies by mock objects. Each mock object behaviour is described on a test by test basis by succinct propositions. The goal is to write tests as unitarily as possible, so that:
  • only few tests fail whenever a code change which breaks past behaviour occurs,
  • after each code change, it is easy to know which test suite should be run to detect potential regressions,
  • tests run faster,
  • and most importantly, the cost of writing and maintaining tests does not escalate with the program's size (or the classes' dependence depth).
So, from the mockist standpoint, inheritance is a problem. Inheritance allows to split code among separate units, but prevents from testing these units in isolation. Tests of the descendants in a class hierarchy are necessarily redundant since they exercise the code of the ancestor's classes multiple times.

Circus by Marc Chagall

Sunday, May 10, 2015

Economy of Means: On the Elimination of Inheritance (1/6)

Introduction

Today I bring an end to a question I have been pondering for more than a decade. It started as soon as my first real-life software projects, at the time in C#:
Should this piece really be coded with inheritance? Wouldn't composition work just fine? On which grounds can the choice be made?

High level languages, such as C#, are supposed to make your life easier. They provide succinct syntax for powerful yet simple abstract concepts which are automatically translated, by the compiler or interpreter, into long lists of low level machine instructions. At least, that is the theory. In practice, high-level languages are shipped with a variety of shiny new constructs, which reflect the tastes of their inventor, or the fashion of the time. Junky programmers will just happily jump on the band-wagon, using any gadget to quickly produce what I see as no more than unmaintainable disposable paper software.
On the contrary, thorough programmers, which need to produce larger software (many functionalities, several team members, and at least a few years of maintenance), must painfully learn the correct way to use their tools. The school of thought I adhere to is minimalism. Hence, I personally believe that only a few well-chosen orthogonal constructs in any programming language are always sufficient. Having experienced various flavours (imperative, pure, strongly typed, static, dynamic, modular, object oriented, functional, event-based, declarative, synchronous, relational...), I am not afraid to reduce (mutilate according to language zealots) any programming language syntax in order to unveil its core paradigm. For every construct, I like to decide whether it is possible to do without or at least delineate the precise conditions that make it applicable. The benefits of restricting syntax to its bare minimum are greater code homogeneity, increased readability, and shorter learning curve for team newcomers or language beginners. Isn't reduced risk of astonishment a worthwhile goal in itself?
Here is a list of language options which generate dilemma (at least for me):
  • possibility to omit braces for one-line blocks,
  • initialization next to the field declaration (rather than in the constructor),
  • for loop (rather than while),
  • goto statement (rather than structured programming),
  • global variables (see A case against static),
  • static methods,
  • Long (rather than long),
  • struct (rather than classes in C#),
  • casts (rather than generics/templates),
  • HashMap (rather than Hashtable in Java),
  • objects (rather than modules in OCaml),
  • decorators as annotations (in Python),
  • new (rather than factory constructor in Javascript),
  • ...
Inheritance contributes to the existing plethora of paradoxes of choice.

If class B inherits class A, then it receives all methods and properties of A. Class A is called the base class (or father); class B is the child. From a software design perspective, a class hierarchy with only two elements does not make much value. So let us add another offspring of A named class C. Following UML conventions, this simple canonical hierarchy is represented by the following schema:

In various situations, this triptych, is often, unfortunately, chosen as the design by default. In the following, I will describe alternative and often preferable designs that avoid inheritance. Through these examples we will gradually grasp the true nature of inheritance.

Side note: Although, all code is written with the Java syntax, the same concepts are present in most mainstream object oriented programming languages.

A bit of reductionism

Deaf to YAGNI

In any code base, you sometimes (more than you would ever have wished for) encounter a degenerate class hierarchy. That is a hierarchy with only two members: the father and its son. In the best case, it is an intermediate state after a previous unfinished refactoring session. Just after a dead classes has been removed. But most often, it is a consequence of programmers not living in the present: in anticipation of a future evolution of the software, they just made it unnecessarily complex.
The most recent encounter with this pattern that I remember is this:
  • a class UserManager with most methods abstract,
  • and a single implementation RemoteUserManager.
The cure is easy: merge the two classes and get rid of all unnecessary syntax. In the process, we picked a more telling name for the resulting class by calling it RemoteAuthenticationService.

The future is not easily predicted, even more in software development. Today's preparation for a potential evolution often turns out to be a hindrance in meeting tomorrow's concrete requirements. To make a physical analogy, it is like walking further in a direction because you think that is where you will be asked to go next, but then having to walk back an opposite path when the destination changes.

Euclid said it best:
"In any triangle two sides taken together in any manner are greater than the remaining one."


Simply put, stay focused on implementing just enough code to meet the current required behaviour. It is a safer strategy, by a large margin. At least that is what Yagni says.

DRYing the wrong way

Frequently, small helper functions are found in the parent of a class hierarchy. For instance, a function that converts hours into milliseconds, could belong to a hierarchy of Tasks. Or a function which formats speeds either as kilometers or miles per hour, could be part of a hierarchy of Vehicules.
Coders did that probably because they were in a hurry. It is so easy to move up common code in a parent class. At least, didn't they resort to copy-pasting.

First, as a general rule of thumb, methods which do not access the object state (through keyword "this"), most probably do not belong to the correct class. Helper methods couple the whole hierarchy with unrelated code. Hence, helper methods are clearly not a recommended usage of inheritance and can advantageously be replaced by composition. Composition will let you write less monolithic code by distributing the responsibilities into distinct smaller and reusable class.
However, pulling up helper methods may be a useful intermediate step when refactoring unmaintainable code.

A matter of choice

Sometimes, we want to store different kinds of data in a single collection. For instance, we could have various types of shapes (Triangle, Square, Circle...). Then, we can define a common interface Shape and have all objects implement it. Interface inheritance is the OO way of providing union types.

At other times, various objects must be processed to compute results of similar nature. Building on the previous example, we could need to compute the shapes' respective surfaces. Then, we simply define method computeSurface on the Shape interface and have each shape implement it its own way.
Often, the treatment can not be directly coded inside the objects, but lies somewhere else. Consider, for instance, a compiler which outputs a syntax tree, or a database which accepts commands. In order to be able to code the treatment, some kind of dispatch mechanism is needed. The naive implementation consists in branching according to the object type. Since its recommendation by the GOF, the visitor pattern has become a popular design. However, I find a map from object types to the appropriate treatment proves to be a much more flexible and less intrusive implementation. But that's material for another post.

Interface inheritance also allows the construction of complex recursive data-structures. To give a simple example, consider arithmetic expressions which are comprised of:
  • literals, such as constants integers,
  • compound expressions such as the binary addition of two other expressions.
With these heterogeneous pieces of data unified under a common type, we could easily perform a variety of operations: evaluation into an integer, conversion into a string representation, height count, size (total number of nodes) count, translation to yet some other formalism, symbolic manipulations (such as factorisation)...
And of course, we could also store several expressions into a unique worklist.

The two opposite sides of inheritance


As a summary for this first set of simple examples, I would like to point out that inheritances mainly serves two orthogonal purposes:
  • reuse (or factor) common code or data,
  • union distinct objects under the same type.
More complex inheritance usages are always a subtle combination of these two fundamental aspects in various proportions.
As such, inheritance is a language feature which is not atomic. It can be expressed as a combination of:
  • code composition for the factoring aspect,
  • interface inheritance (or simply duck-typing in more dynamic OO objects) for the union aspect,
  • and some glue code, mainly to forward calls.
As we will see in the following, more complex examples, it proves almost always preferable to break down inheritance into its constituents. The design becomes more focused and readier for change.



Note: In order to make it more easily digestible, this topic will be split into 6 different posts.

 Next

Monday, August 25, 2014

Facing annihilation: a powerful koan

There has been no new post on this blog for the past two years, and today I would like to explain why. Let me first issue a disclaimer:
  • this post will not be about programming, even though it is still about something you will most probably not learn at school,
  • ignorance is bliss.
For these two reasons, you may prefer avoiding to read the information that follows. 

If you are still with me, let us stride forward: two years ago, I stumbled across an enlightening and yet depressing conference by Guy McPherson. (There are now more up to date presentations of his, see the link section at the end of this post). With the risk of misrepresenting the quality of this 30 minutes talk, densely packed with data, let me try to summarize its main message:
  • global average world temperature increased by 0.85 degree Celsius against pre-industrial level. 
link: http://www.eea.europa.eu/data-and-maps/figures/global-annual-average-temperature-deviations-1850-2007-relative-to-the-1850-1899-average-in-oc-the-lines-refer-to-10-year-moving-average-the-bars-to-the-annual-land-and-ocean-global-average-9
A 0.85 degree Celsius increase
  • In consequence, tremendous impacts are already being felt across the globe, in particular in the northern region: the arctic.
  • So much so, that once frozen organic matter on land (permafrost) is coming back to life, releasing methane in the the process; simultaneously, methane molecules which were once trapped into ice cages (methane hydrates) at the bottom of the ocean are now escaping, moving up and entering the atmosphere.
  • Methane, although shorter lived than carbon dioxide in the atmosphere, is an extremely potent greenhouse gas, thirty times as much.
Methane hydrate or burning ice!
  • We hence find ourselves in the presence of a vicious cycle: global warming triggers methane releases, a strong driver for more warming and so on.
  • Such cycles are called self-reinforcing (or positive) feedback loops in system dynamics. At the time of the video, Guy cites 4 feedback loops, in more recent talks he presents more than 20 of them (admittedly not all of the same importance).
  • Widely accepted, conventional models of global warming (Intergovernmental Panel on Climate Change, United Nations Environment Programme, Hadley Center for Meteorological Research, Nasa Goddard Institute for Space Studies...) predict a range of temperature increase going from 1 to 10 degrees Celsius by 2100. 
Slowly melting permafrost
  • As dire as some of these projections may seem, none of them take any of the feedback loops into account. Unfortunately, feedback loops are no small matter: Malcolm Light of the Arctic Methane Emergency Group, declares, probably a bit hyperbolically, in a 2010 paper that methane releases going exponential would lead to the demise of all life on earth by mid-century. He even pinpoints year 2031 for the northern hemisphere.

Incredulous?

Global average temperature increases are, for most of us, completely abstract notions. It may thus seem inconceivable that a difference of a couple (4 to 6) degrees could trigger the full scale extinction of our species. Well, 6 more degrees on top of a scorching summer could certainly translate into even more mega-fires (as in Russia), droughts (as in Australia), crop failures (as in California), nuclear reactor shutdowns (as in France), and cattle death, than with today's mere 0.85°C increase... It could also mean winter mild enough for human pathogens to strive (for instance malaria or parasites).

More importantly, these figures are global average which misrepresent local heterogeneity. As indicated in the image from the climate reanalyzer, March 20 of this year, global world temperature was 0.65 degrees above the 1979-2000 period (which is roughly equivalent to 0.85 degrees Celsius against pre-industrial level, only the baseline differs). Yet on that same day a large part of Russia was witness to a full 20 degrees increase. So the average hides really large swings for some places of the globe.

Even more crucially, we really rather need to consider the impacts at the ecosystem level. With the non-linearity that such complex systems often display, nobody can accurately predict what living conditions a 6 degrees increase will produce for anyone of us.
Guy asserts that it leads to the end of the human experience. To support that claim, he offers several explanation which all boil down to the destruction of our natural habitat:
  • insufficient food supplies: crop protein denaturation could be a consequence of severe heat shocks,
  • impossible thermoregulation: higher temperatures will strain thermoregulation abilities to the limit for mammals as large as humans,
  • no oxygen: if deserts replace forests on lands, and phytoplankton in the ocean is completely wiped out, then our two only sources of oxygen vanish, (for the record, phytoplankton is already 40% depleted in comparison to 1950 levels)
  • global nuclear meltdown: a scenario where several meltdowns triggered by, unplanned for, extreme weather events would raise radiations to unlivable levels for terrestrial organisms.

Dire planetary status

Frankly speaking, I don't know how things will really play out. Again nobody can accurately predict the future, but we can easily observe the trends. So before denial kicks in, I would like to review a few facts (not predictions) which illustrate the gravity of our current predicament:
  • at 400ppm, CO2 levels in the atmosphere are now reaching a level unseen in the last 15 million years. (To put that statement in perspective consider that civilization emerged only 10,000 years ago, Homo Sapiens are only 200,000 years old, while Homo Habilis evolved around 2.3 million years ago.)
  • In fact, since the industrial revolution, we drastically changed the composition of the atmosphere, dumping methane (now topping 1700ppb), nitrous oxide, ozone and even synthetic gases such as chorofluorocarbons (CFC), hydrochlorofuorocarbons (HCFC) halons...
  • Unfortunately, this is an ongoing and worsening trend: carbon dioxide emissions from burning fossil fuels set a record in 2013.
Dumping always more carbon in the atmosphere.
The arctic Death spiral.
  • Forests too are declining. Worst of all, primary forests are cut down. The hunger for fresh agricultural land is unsatiable. Don't you think carbon neutral schemes smell dubious? In any case, climate models most probably over-estimate the extent of future forests and thus their real ability to sequester carbon. Even more devastating: atmospheric pollution, especially ozone, seems to be killing trees worldwide.
  • We are now living a mass extinction of life on earth as reported in this documentary. The population numbers speak for themselves: land animals are down 28% since 1970, marine birds are down 30% since 1995, big ocean fish are down 90% since 1950, freshwater fish are down 50% since 1987, marine animals are down 28% since 1970 and songbirds are down 50% since 1950. In the same vein, Paul MacCready, in this excellent TED presentation shows the evolution of the mass of all land based vertebrates. The share of wild animals went from being 99.9% to a tiny 3% in the 10 thousand years since the start of civilization.

Homo Sapiens (?) is winning the war...

This is all without talking about the miserable state of the ocean, soil erosion, natural resources depletion, garbage (such as plastic, radioactivity...) accumulation. And the list just goes on, endlessly... Blinded by desire, we produce what destroys us in far greater quantity than the rest of the creation can instantaneously balance.


Retreating glaciers

Reversing global trends?

There really is no room for optimism. Before that, most of the current global trends fueled by human behavior should first be reversed (in no particular order):
  • Burning of coal and other fossil fuels should be stopped.
  • Our numbers should be decreasing (ideally via access to education, contraception, financial incentives, and equitable one child policy). Population is the multiplier of everything else.
  • Forest cover should be increasing: trees should be planted, forest let to grow.
  • Human population should retreat from cities to be given the opportunity to learn living hard but more sustainable and decent agricultural, pastoral or forager lives.
  • In short, our current economy should come to a halt, thus reducing the count of all our energy-consuming machines: cars, planes, phones, computers...

In addition to these policies directly related to climate change, the following would also most probably be wise to implement:
  • The production of several synthetic substances should be totally banned. In particular products that can not be naturally decomposed in realistic time-frames should never see the light: plastic, composites, nuclear wastes... Similarly, substances that potentially pose irreversible risks to the biosphere, ecosystems, animal or human health should not be put in circulation. Think: CFC, pesticides, most genetically modified organisms, nanoparticles. The burden of proof should be carried by the producers rather than the consumers and any new product considered toxic before being demonstrated as safe.
  • Nuclear power-plants should be shut down before the next radical accident.

Note that there is a 40 years time-lag between carbon emissions and their full impact on the climate. So even if some drastic adjustments were made today, we would have to wait several years before feeling the first improvements.

Predictions

Now is the time for a fun but totally useless game: predictions. Predictions let us exercise and showcase our brain power. They are fun.
But they are really useless too; just another diversion. All the data we have in our hands about the current state of our habitat should be more than enough to convince us of the urgency to change course. Predictions are simply another skillful way of maintaining the status-quo. Build an abstraction, and start endless discussions. Blind ourselves to the present by looking far into a potential future (this works with the past too).

Let me first warn you: my predictions are most probably wrong. Other (Thomas Robert Malthus, Paul Ehrlich, Richard Duncan...), have been somewhat off before. Wrong, but still as valid as any of the so called projections issued by respectable institutes: a world population of 9.6 billion by 2050 according to the United Nations 2013 report? Continued growth in world liquid fuels production up to 2040 according to the US Energy Information Administration? Continued GDP growth for both China, the US and India up to 2030 according to the IMF? I don't think so!

Here is what I rather believe today:
  • Real economic growth has already or will stop before 2020. Resources availability will decline dramatically.
  • Major industrial systems (nuclear reactors, skyscrapers, roads, sewers, water supply systems, electrical grids, computer networks, cities) will fail because of lack of maintenance, nature impacts, or human induced chaos. Large non-tangible systems (health care, government, retirement plans) will break down too. All these symbols of our hubris have continued growth as a first requirement.
  • Human population trends will then quickly reverse. Death rates will increase. Maybe through large scale epidemic or starvation. Great migrations will be unstoppable.
  • Geoengineering will be tried and fail. Technical Progress Raises More and Greater Problems than It Solves. With hindsight, doesn't this axiom stated by Jacques Ellul in his Technological Bluff sound painfully true?
  • By 2040, I expect our numbers to be no more than 1 billion. War for the last habitable places (the poles? underground?) will have started.
  • By the end of this century our species will have gone extinct.
  • In the longer run, will planet Earth then go Venus or still be habitable (by complex life-forms)? I don't know. Does it really matter?

Simple models

These predictions are not based on fancy, complicated mathematical models, rather on simple observations.

First, let us admit our species has a dynamic of its own. Our collective course certainly seems not to be controlled by rational decisions at the individual level. On one hand, various religious beliefs, propaganda, advertisement (should I say automatisms put in place as control mechanisms?) most often trump objective analysis. On the other hand, the most powerful members of our species either have no real control, are extremely cynical, or simply plain dumb. Who knows?
Human behavior displays a lot of inertia. Do smokers stop as soon as they are told the consequences? Why was it necessary for the world fisheries to go into decline before any fishing quota to be put in place? Garret Hardin introduced a simple model called tragedy of the commons to explain this group dynamic. For Nate Hagens, ancient brain characteristics drive our current consumptive behavior. Jay Hanson makes a similar claim. Our fate derives from our very nature.
It thus seems reasonable to assume the human species will try to keep its current dynamic as long as possible: trying to monopolize always more!

Total catch of southern bluefin tune. A quota system was introduced only in 1980.

Our global dynamic collides violently with the limits we are now reaching on every front. We most probably are at our last doubling time.

At this point, forced decline is the only direction. Could the St Matthew reindeer's experiment be a precedent of the speed at which things will unravel for us? Indeed, Ugo Bardi developed a simple model, the Seneca cliff, which shows that staggering maintenance costs and pollution make descent faster than ascension. 

Arx tarpeia Capitoli proxima!
In any case the future will not mirror the last 100 years. Linear extrapolation does not make sense anymore. We are in uncharted exponential, fractured territory. The only elements of comparisons should be other mass extinction events.

What now?

When confronted to this information, people react in a variety of ways. Mostly denial. Changes of this magnitude in such a short time-frame seem unbelievable. Recently, recurrent global catastrophes have started to slowly instill doubt in the dominant narrative which roughly states that in this best of all possible worlds the baron's castle was the most magnificent of all castles. However, extinction is not part of most people's plan yet. It is too hard to accept. It takes time to mentally shift gear. It is hard to reverse lifelong indoctrinations.
I believe denial and hope will progressively shift into despair, shame and anger. The primary reason I am writing today is to go past these feelings. I wanted to state the inescapable facts as clearly as I understand them and then use them as a new inner foundation for the time ahead. Now I am simply left with a challenging koan:
Living with the certainty of near annihilation.







Bibliography

Some hand-plucked links, organized by topics.

Climate change


A slowed down, meandering jet stream has been causing all sorts of bizarre weather patterns in the recent years all over the northern hemisphere. A bit of theory here.

War against the Environment: scale of the Human impact


Understanding the models, predicting the future


Numerous others have made equally dire predictions:
  • For Stephen Hawking, we need to leave the planet before 200 years.
  • James Lovelock expects 80% of humans to perish by 2100.
  • Paleontologist Peter Ward believes global warming will drive us extinct, similarly to previous mass extinction episodes.
  • James Hansen warned against the burning of coal.
  • Some models of IPCC predict a 10 degree rise by 2100, thus making the planet unlivable.

Individual responses


Friday, December 28, 2012

Code metamorphosis

In this post, I would like to show through an example, how we can always progressively improve even the most badly written piece of code.
I will start from a somewhat unintelligible program and, step by step, rewrite it until it really pleases the eyes. Doing so, I will preserve its external behavior, which is roughly specified as follows:
The program inputs 3 possibly null strings. It shall interpret them as 2 integers and an operator among +, -, * and /, to then evaluate the expression and display the result.
Any invalid input value is signaled by an appropriate error message.
I am assuming the existence of two library methods:
  • Integer readInteger(String) transforms a string into the integer it represents. This method expects a non-null argument, and returns null on any string which does not correctly parse as an integer,
  • void println(String) prints its string argument on the standard display and terminates with a new line.
Here is our starting point, a program which fulfills the specification:
void calculate(String number1, String operator, String number2) {
  Integer value1;
  Integer value2;
  if (null != number1) {
    value1 = readInteger(number1);
    if (null != value1) {
      value2 = readInteger(number2);
      if (null != value2) {
        if ("+".equals(operator)) {
          println(value1 + value2);
        } else if ("-".equals(operator)) {
          println(value1 - value2);
        } else if ("*".equals(operator)) {
          println(value1 * value2);
        } else if ("/".equals(operator)) {
          if (!new Integer(0).equals(value2)) {
            println(value1 / value2);
          } else {
            println("Division by zero.");
          }
        } else {
          println("Unknown operator: " + operator);
        }
      } else {
        println("Invalid argument: " + number2);
      }
    } else {
      println("First argument is not a valid number.");
    }
  } else {
    println("Missing first argument.");
  }
}
Even though it does the job, this program does not conform my personal aesthetics. Here are the reasons why:
  1. Lack of structure. There is a complete absence of separation of concerns in this monolithic, heavily indented piece. This piece does not express its intent at all, which is really to perform 3 distinct steps: validate and parse its input, then perform the computation, and at last present the result (or an error message) to the user.
  2. Difficult code extensions. Nowadays, programs often need to change their behavior (sometimes drastically and very quickly). So they must be built in a way that allows for future evolutions. For instance, in our case, what if the result needs to be output in a message box rather than the standard output? Or what if, error messages must be dumped separately on the standard error? Multiple lines of code would need to be rewritten. What about handling more complex expressions (with an arbitrary number of operators)? Or replacing all integers with floating points? Similarly, adding error reporting for integer overflows in an uniform way is not a trivial matter. This code is not cut for change, mainly because of its high level of coupling. Adding more functionality will also add to the current mess and lead to code decay.
  3. Hard to read. For such a small task, this code is obviously too hard to understand. It has too many lines of code (length) and a too high level of imbrication (width) for one method. There is no quick way of breaking it down. If kept as it is, maintenance will be costly.
  4. Hidden bug. Ignoring the fact that integer overflows are not avoided, there is another bug in this small piece. Did you spot it? Don't worry if you did not: the process of cleaning it up will, at some point, make this bug surface.
  5. Low testability. I am having a hard time believing that this piece of code can be thoroughly covered with automatic unit tests. In particular, since it is immediately dumped on the console, I can not think of any straightforward way of sensing the output value of the program. The presentation layer is buried too deeply inside the method body to be readily replaced by a fake implementation suitable for testing purposes. This code needs a serious testability improvement.
Believe it or not, despite its chaotic appearance, this program has its own logic. Let me venture an explanation as to why you may encounter this type of code more frequently than desirable:
  1. Single return. Of the epic fight against gotos, remains one bad habit: namely to have only one return point per method. The modern motivation for this rule is to prepare a single break-point for every method when inspecting the program execution using a debugger. Being of the school of thought that running the debugger is already a sign you lost control over your code, I completely disregard this constraint.
  2. Error cases come second. Did you notice that else branches always contained the error cases? This is so that the correct execution can be read first.
  3. Lazy programming. The coder was driven only by the result to compute. He did not care about readability, maintainability, evolutivity or testability at all. In consequence, inputs are parsed when needed, results are output as soon as they become available in total disregard of any program structure.
  4. Low level defensive programming techniques. Were you surprised by the peculiar shape of conditionals? There are two distinct tricks here. First, all calls to equals are made on a constant object, rather than on a variable. (e.g."*".equals(operator)). The rationale being to avoid potential exception to be thrown if the variable happens to be null. On the surface, this may seem reasonable. But I do not like this coding style. Again I believe it shows a lack of control: at every point, the programmer should know whether any variable may be null, or not. If it may then a check should be performed first to handle this case once and for all. If not, then there is no need to worry. It should not be necessary to sacrifice conditional expressions' readability. In any case, later in this post, I will present a programming technique to drastically minimize any null value flowing through the program. Second, in order to ensure a value is not null, this programmer believes that writing (null != x) is preferable to the more straightforward (x != null). This may be true in C, since a small typographical error and the comparison transforms into the assignment x = null. But not so in Java which has a distinct type of boolean. The type-checker protects you.
  5. Grouped variable declarations. All variables used in this method are declared together at the beginning of the method. This is a C habit, which is unnecessary in more modern languages such as Java or C#. It is preferable to let variables have the tightest possible scope since it reduces the burden of understanding the meaning of many variables too early. It also decreases risks of misuses. Some would oppose that variables declared inside a loop are inefficient, since they will be allocated and freed multiple times. I honestly don't know if that is true: optimizing compilers most probably know better. In any case, I highly doubt that any performance bottleneck ever arises from such practice.
We are now almost ready to start refactoring this piece of code. Before we do so, I am going to assume there is a non-regression test suite for this method. It is going to be run at the end of each rewriting step in order to make sure the program external behavior is preserved.

 

Step I: Early return and program intent

First, let us insert some early returns. This will enable us to clearly separate the 3 phases of the program: input validation, computation and presentation. For various reasons, some of you may not like early returns. I do. Anyways, as you will see, in this case, they are only a temporary artifact to disappear in a later refactoring phase.
void calculate(String number1, String operator, String number2) {
  // Phase 1/3: validate input 
  if (number1 == null) {
    println("Missing first argument.");
    return;
  }
  Integer value1 = readInteger(number1);
  if (value1 == null) {
    println("First number is not a valid integer.");
    return; 
  }
  Integer value2 = readInteger(number2);
  if (value2 == null) {
    println("Invalid integer: " + number2);
    return;
  }
  if (operator == null) {
    println("Missing operator.");
    return;
  }
  // Phase 2/3: computation
  Integer result;
  switch (operator) {
    case "+":
      result = value1 + value2;
      break;
    case "-":
      result = value1 - value2;
      break;
    case "*":
      result = value1 * value2;
      break;
    case "/":
      if (value2.equals(0)) {
        println("Division by zero."); 
        return;
      }
      result = value1 / value2;
      break;
    default:
      println("Unknown operator: " + operator);
      return;
  }
  // Phase 3/3: presentation
  println(result);
}
Here is a list of things we did in this step :
  • Early returns were introduced in order to eliminate error cases as soon as possible. They allowed us to separate the input validation from the rest of the method.
  • An intermediary variable was introduced to store the computation result and separate the presentation layer from the rest of the method.
  • Variable declarations were delayed until needed. Variable initialization is now extremely close from, if not in the same statement as, its declaration.
  • Conditionals were inverted to feel more natural, and the sequence of if/else-if was replaced by a switch statement to make the alternatives more visual.
Admittedly, the program is now longer, but a lot easier to understand. However, in doing so, we also slightly changed the behavior of the program: in the case where the operator is null, the error message became "Missing operator." instead of the previous "Unknown operator: null".

 

Step II: Extract methods and expressions

Let us stop for a moment, and take some time to listen to our code. As I am gaining experience, I am finding this is a worthy practice. Often, the code knows better what it wants to look like. We just have to take notice, handle it softly and follow its lead. Here, the two first large blocks, each starting with a line of comment are eager to become full-fledged methods. In order to increase readability, we will thus extract two methods:
  • Expression parseInput(String number1, String operation, String number2)
  • Integer evaluate(Expression expression)
To do so, we first need to introduce the Expression: an object which encapsulates the output of the parsing phase and which is then evaluated. The expression consists in two integers combined with an operation:
enum EOperation { Plus, Minus, Mult, Div }

class Expression {
  private Integer value1;
  private EOperation operation;
  private Integer value2;
  Expression(Integer value1, EOperation operation, Integer value2) {
    this.value1 = value1;
    this.operation = operation;
    this.value2 = value2;
  }
  public Integer getValue1() { return this.value1; }
  public EOperation getOperation() { return this.operation; }
  public Integer getValue2() { return this.value2; }
}
The code can now easily be rewritten into :
void calculate(String number1, String operator, String number2) {
  Expression expression = parseInput(number1, operator, number2);
  if (expression == null) return;
  Integer result = evaluate(expression);
  if (result == null) return;
  println(result); 
}

Expression parseInput(String number1, String operator, 
                      String number2) {
  if (number1 == null) {
    println("Missing first argument.");
    return null;
  }
  Integer value1 = readInteger(number1);
  if (value1 == null) {
    println("First number is not a valid integer.");
    return null;
  }
  Integer value2 = readInteger(number2);
  if (value2 == null) {
    println("Invalid integer: " + number2);
    return null;
  }
  if (operator == null) {
    println("Missing operator.");
    return null;
  }
  EOperation operation;
  switch (operator) {
    case "+":
      operation = EOperation.Plus;
      break;
    case "-":
      operation = EOperation.Minus;
      break;
    case "*":
      operation = EOperation.Mult;
      break;
    case "/":
      operation = EOperation.Div;
      break;
    default:
      println("Unknown operator: " + operator);
      return null;
  }
  return new Expression(value1, operation, value2);
}

Integer evaluate(Expression expression) {
  Integer result;
  switch (expression.getOperation()) {
    case EOperation.Plus:
      return expression.getValue1() + expression.getValue2();
    case EOperation.Minus:
      return expression.getValue1() - expression.getValue2();
    case EOperation.Mult:
      return expression.getValue1() * expression.getValue2();
    case EOperation.Mult:
      if (expression.getValue2().equals(0)) {
        println("Division by zero.");
        return null;
      }
      return expression.getValue1() / expression.getValue2();
  }
  throw new IllegalArgumentException();
}
There are admittedly now more methods and the code looks even more verbose. But the gain in structure and modularity should be well worth it.
You may be wondering the need to throw an IllegalArgumentException at the end of method evaluate. That is because the compiler does not know this line of code is unreachable (because all enum cases are covered in the switch). It would therefore otherwise emit a compilation error.
Note also, that we did not include any default case in the preceding switch statement. This is done on purpose, so that, if we are to augment the enum with another operation, we will be warned at compile time of all the cases which are not handled.

 

Step III: Exceptions and clean error report

There are two strong code smells left in the previous piece of code:
  • methods parseInput and evaluate both propagate errors up by encoding them into the null return value,
  • error messages are scattered all around.
There is therefore still a high level of coupling between the code which performs the computation task, error handling and the presentation layer. We are going to remedy this situation with the introduction of two exceptions which will respectively signal parse errors and computation errors :
class ParseError extends Exception { }
class DivisionByZero extends RuntimeException { }

void calculate(String number1, String operator, String number2) {
  try {
    Expression expression = parseInput(number1, operator, number2);
    Integer result = evaluate(expression);
    println(result);
  } catch (ParseError e) {
    println("An error occurred while parsing: " + e);
  } catch (DivisionByZero e) {
    println("Division by zero");
  }
}

Expression parseInput(String number1, String operator, 
                      String number2) throws ParseError {
  if (number1 == null) {
    throw new ParseError("Missing first argument.");
  }
  Integer value1 = readInteger(number1);
  if (value1 == null) {
    throw new ParseError("First number is not a valid integer.");
  }
  Integer value2 = readInteger(number2);
  if (value2 == null) {
    throw new ParseError("Invalid integer: " + number2);
  }
  if (operator == null) {
    throw new ParseError("Missing operator.");
  }
  EOperation operation;
  switch (operator) {
    case "+": 
      operation = EOperation.Plus;
      break;
    case "-":
      operation = EOperation.Minus;
      break;
    case "*":
      operation = EOperation.Mult;
      break;
    case "/":
      operation = EOperation.Div;
      break;
    default:
      throw new ParseError("Unknown operator: " + operator);
  }
  return new Expression(value1, operation, value2);
}

Integer evaluate(Expression expression) {
  Integer result;
  switch (expression.getOperation()) {
    case EOperation.Plus:
      return expression.getValue1() + expression.getValue2();
    case EOperation.Minus:
      return expression.getValue1() - expression.getValue2();
    case EOperation.Mult:
      return expression.getValue1() * expression.getValue2();
    case EOperation.Mult: 
      if (expression.getValue2().equals(0)) {
        throw new DivisionByZero();
      }
      return expression.getValue1() / expression.getValue2();
  }
  throw new IllegalArgumentException();
}
I am now fairly satisfied with the current state of the code. We still can and will go a bit further. However, before doing so, we are first going to eliminate a bug and improve test coverage.


Step IV: Fixing a bug and DRY investment

By now, it should be obvious! The parsing phase has a flaw in that it does not check whether the second number may be null. Let us first write a non-regression test which fails:
public void 
test_parseInput_ShouldThrowParseErrorWhenSecondNumberIsNull() {
  try {
    Program program = new Program();
    program.parseInput("0", "+", null);
    Assert.fail();
  } catch (ParseError e) {
  }
}
Please take note how straightforward that test was to write. Writing a similar test harness with the first, monolithic, version of the code, would have been a lot less fluid. It would have required us to first redirect the standard output into a buffer, in order to then assert the correct error message is printed out. Learning how to write such complicated tests is a good exercise. Especially so if you are working with legacy code. But in the long run, it is not worth it. It is largely preferable to rewrite your code in order to make it easily testable.
Fixing the bug would be equally easy. But we are going to make a bit more work than necessary and take this opportunity to further clean up the code. Faithful to the DRY (Don't Repeat Yourself) motto, we will eliminate any repetition by introducing three more methods :
  • ensureIsNotNull(String input) throws a ParseError on a null input,
  • parseInteger(String input) converts the string representation of an integer into its value, and throws a ParseError to signal failure,
  • parseOperation(String input) converts the string representation of an operation into a EOperation.
Here are the methods whose code is modified:
Expression parseInput(String number1, String operator, 
                      String number2) throws ParseError {
  Integer value1 = parseInteger(number1);
  EOperation operation = parseOperation(operator);
  Integer value2 = parseInteger(number2);
  return new Expression(value1, operation, value2);
}

void ensureIsNotNull(String input) throws ParseError {
  if (input == null) throw new ParseError("Missing input.");
}

Integer parseInteger(String input) throws ParseError {
  ensureIsNotNull(input);
  Integer result = readInteger(input);
  if (result == null) throw new ParseError("Integer expected.");
  return result;
}

EOperation parseOperation(String input) throws ParseError {
  ensureIsNotNull(operator);
  switch (input) {
    case "+": return EOperation.Plus;
    case "-": return EOperation.Minus;
    case "*": return EOperation.Mult;
    case "/": return EOperation.Div;
    default:
      throw new ParseError("Unknown operator: " + operator);
  }
}
I would like to draw your attention to the fact that it is extremely easy (in comparison to a large if else block) to move an unitary instruction such as ensureIsNotNull(number1); around the code. We could for instance, push it up, in an effort to eradicate null values as early as possible. Removing all null values at system boundaries, allows us to safely assume no value can ever be null in the code body. Reducing of that much the risk of null pointer exceptions.

Even more importantly, at the same time we fixed the bug, we limited even more the probability of a similar bug to reappear later. Here, the safeguard consists in having method ensureIsNotNull being called inside method parseInteger. So that, as long as we use this method to parse any other integer, then the same mistake can simply not, by construction, be made.


Step V: Improving testability and seam

Unit tests that assert the correctness of error messages would also be nice to have. These messages may, for instance, be contractual.
One way to do so, would be to redirect the standard output into a buffer (using method System.setOut in Java). Another, simpler way, would be to plug a fake output stream inside the program constructor and write the test as follows:
public void test_calculate_SignalsDivisionByZero() {
  FakeOutpuStream output = new FakeOutputStream();
  Program program = new Program(output);
  program.calculate("10", "/", "0");
  Assert.assertEquals("Division by zero", 
                      output.getLastPrintedLine());
}
There are two necessary ingredients to make such a test possible. First, a FakeOutputStream, that is a custom OutputStream which really does nothing except store the last printed string. Method getLastPrintedLine can then be called to retrieve this string in order to build the assertion. Second and more importantly, it should be possible to choose the output stream used for all outputs of the program. The best way is to have the program constructor accept this dependency as a parameter. After the previous refactoring steps, methods println is now called in very few different places and it is thus very easy to modify the program to this end:
class Program {
  private OutputStream output;

  Program(OuputStream output) {
    this.output = output;
  }

  void calculate(String number1, String operator, String number2) {
    try {
      Expression expression = parseInput(number1, operator, number2);
      Integer result = evaluate(expression);
      this.output.println(result);
    } catch (ParseError e) {
      this.output.println("An error occured while parsing: " + e);
    } catch (DivisionByZero e) {
      this.output.println("Division by zero");
    }
  }

  // ...
}
As often, by making the program more testable, we also improved its design. The fact that this piece of code is printing to a stream can now be deduced looking at the constructor signature only. It also becomes easy to plug different kinds of presentation layer, as long as they extend the OutputStream interface.
To inject the output stream program dependency may have seem fairly easy to do here. Sometimes, it may require more work or may not appear so evidently. In any case, it is a very powerful and sane programming technique. This technique supports the stance that we are not building monolithic programs but rather program components. A major coding attitude shift.
As a final note for this section, let me admit that error messages should ideally not be present as constant strings in the program but rather all read from an external resource. But that is for another story.


Step VI: Evaluation the OO way

Let us now wander into the realm of (hard-core) object orientation, and for some of you maybe superfluous bonuses. This next rewrite step originates from a simple observation:
Method evaluate does not need the instance of Program at all, but it rather extensively accesses fields of its unique argument, the Expression.
Consequently the program would surely gain in encapsulation and modularity, if we were to move this method into class Expression. Going a step further, it would make even more sense to dispatch the treatment for each operator into the operations themselves which would relieve the need for the switch statement. It is actually common knowledge that switch statements rigidify object oriented programs. The conventional remedy is a small amount of object inheritance. Here is how it can be done in practice:
interface IOperation {
  Integer evaluate(Integer value1, Integer value2); 
}

class PlusOperation implements IOperation {
  public Integer evaluate(Integer value1, Integer value2) {
    return value1 + value2;
  }
}

class MinusOperation implements IOperation {
  public Integer evaluate(Integer value1, Integer value2) {
    return value1 - value2;
  }
}

class MultOperation implements IOperation {
  public Integer evaluate(Integer value1, Integer value2) {
    return value1 * value2;
  }
}

class DivOperation implements IOperation {
  public Integer evaluate(Integer value1, Integer value2) {
    if (value2.equals(0)) throw new DivisionByZero();
    return value1 / value2;
  }
}

class EOperation {
  public static final PLUS = new PlusOperation();
  public static final MINUS = new MinusOperation();
  public static final MULT = new MultOperation();
  public static final DIV = new DivOperation();
}

class Expression {
  private Integer value1;
  private IOperation operation;
  private Integer value2;

  Expression(Integer value1, IOperation operation, Integer value2) {
    this.value1 = value1;
    this.operation = operation;
    this.value2 = value2;
  }

  Integer evaluate() {
    return this.operation.evaluate(value1, value2); 
  }
}

class Program {
  // ...
  void calculate(String number1, String operator, String number2) {
    try {
      Expression expression = parseInput(number1, operator, number2);
      Integer result = expression.evaluate();
      this.output.println(result);
    } catch (ParseError e) {
      this.output.println("An error occurred while parsing: " + e);
    } catch (DivisionByZero e) {
      this.output.println("Division by zero");
    }
  }
  // ...
} 
Each operation has now its own class. These classes all inherit from a common interface which declares an evaluate(String, String) method. To ensure some sort of backward compatibility, we kept an EOperation class, which can be used as an enum by providing several static, final, non-modifiable singleton operations. To be honest, I would never use this trick in an industrial setting. (And don't worry, I will get rid of it in the final refactoring step). For all the reasons exposed in my previous post (see A case against static), this is borderline and unnecessary. But I wanted to make the point that the parsing part of the code need not be changed at all in this refactoring phase (plus, for once, I wanted to please all the memory performance junkies out there: doing so avoids a re-allocation of an IOperation at every call to method calculate).

Without any doubt, this rewrite steps improved the code structure. Note how none of the fields in class Expression escapes the private scope anymore. A sure sign of better encapsulation. Furthermore, modification of the code to handle another operation would be largely more localized than in the initial piece.


Step VII: Ultimate polishing step

Before feeling satisfied, we are going to eliminate the last code smells: the presence of a switch statement and a (fake) enum. To do so I will use yet another trick: that of changing control flow code into data. We will simply put all operations into a hash-map indexed by the string representation of the operation. Then parsing any operator simply amounts to getting the corresponding operation into the hash-map. If no operation is found, then it is a parsing error. As a bonus, since the hash-map will be initialized once in the program constructor, there is really no need for the fake EOperation enum anymore. Here are the two methods which change during this rewrite step:
class Program {
  private OutputStream output;
  private HashMap<IOperation> operations;

  Program(OuputStream output) {
    this.output = output;
    this.operations = new HashMap<IOperation>();
    this.operations.put("+", new PlusOperation());
    this.operations.put("-", new MinusOperation());
    this.operations.put("*", new MultOperation());
    this.operations.put("/", new DivOperation());
  }
  // ...
  IOperation parseOperation(String input) throws ParseError {
    ensureIsNotNull(operator);
    IOperation result = this.operations.get(operator);
    if (result == null) 
      throw new ParseError("Unknown operator: " + operator);
    return result;
  }
  // ...
}

 

A word on the final code version

Let us have a look at the final version of the code:
interface IOperation { 
  Integer evaluate(Integer value1, Integer value2); 
}

class PlusOperation implements IOperation {
  public Integer evaluate(Integer value1, Integer value2) {
    return value1 + value2;
  }
}

class MinusOperation implements IOperation {
  public Integer evaluate(Integer value1, Integer value2) {
    return value1 - value2;
  }
}

class MultOperation implements IOperation {
  public Integer evaluate(Integer value1, Integer value2) {
    return value1 * value2;
  }
}

class DivOperation implements IOperation {
  public Integer evaluate(Integer value1, Integer value2) {
    if (value2.equals(0)) throw new DivisionByZero();
    return value1 / value2;
  }
}

class Expression {
  private Integer value1;
  private IOperation operation;
  private Integer value2;

  Expression(Integer value1, IOperation operation, Integer value2) {
    this.value1 = value1;
    this.operation = operation;
    this.value2 = value2;
  }

  Integer evaluate() {
    return this.operation.evaluate(value1, value2);
  }
}

class Program {
  private OutputStream output;
  private HashMap<IOperation> operations;

  Program(OuputStream output) {
    this.output = output;
    this.operations = new HashMap<IOperation>();
    this.operations.put("+", new PlusOperation());
    this.operations.put("-", new MinusOperation());
    this.operations.put("*", new MultOperation());
    this.operations.put("/", new DivOperation());
  }

  void calculate(String number1, String operator, String number2) {
    try {
      Expression expression = parseInput(number1, operator, number2);
      Integer result = expression.evaluate();
      this.output.println(result);
    } catch (ParseError e) {
      this.output.println("An error occured while parsing: " + e);
    } catch (DivisionByZero e) {
      this.output.println("Division by zero");
    }
  }

  Expression parseInput(String number1, String operator, 
                        String number2) throws ParseError {
    Integer value1 = parseInteger(number1);
    IOperation operation = parseOperation(operator);
    Integer value2 = parseInteger(number2);
    return new Expression(value1, operation, value2);
  }

  void ensureIsNotNull(String input) throws ParseError {
    if (input == null) throw new ParseError("Missing input.");
  }

  Integer parseInteger(String input) throws ParseError {
    ensureIsNotNull(input);
    Integer result = readInteger(input);
    if (result == null) throw new ParseError("Integer expected.");
    return result;
  }

  IOperation parseOperation(String input) throws ParseError {
    ensureIsNotNull(operator);
    IOperation result = this.operations.get(operator);
    if (result == null) {
      throw new ParseError("Unknown operator: " + operator);
    }
    return result;
  }
}

First, if you examine conditionals, you will see they are all of the most simple form: if some condition, then throw some exception. Two of them perform input validation, the other two filter results returned by calls to library functions (parseInteger and get from the HashMap can both return null). To sum up, conditionals are all placed at the system boundaries to eliminate unwanted values. It is no coincidence, but rather a choice to perform defense at the perimeter and to fail brutally the earliest possible.
Collecting a few metrics, we can produce the following tables:

Structure Classes Methods Interfaces Hierarchy depth
First program 1 1 0 0
Last program 6 12 1 1

Complexity Instructions Max. instructions/function Max. imbrication level
First program 26 26 5
Last program 44 7 1

Control flow if else switch throw catch
First program 8 8 0 0 0
Last program 4 0 0 3 2

Features occurrences of null enum generic
First program 3 0 0
Last program 3 0 1

Even though this program has more code than the initial version, methods are all very short. The longest one (method evaluate) has only 7 instructions. And the maximum imbrication depth is only one. Taken in isolation, each method does not amount to much, but it is the combination which proves valuable. Testing multiple small methods is easy. So much so, that some developers will not understand the value of such simple tests: seeing it only as an unnecessary repetition of what is already obvious in the code itself. Adding new behavior will be faster and more localized than in the initial monolithic version.
In brief, that is how I like software: minimalist.


Conclusion

In conclusion, I would like programmers to not satisfy themselves with code written in the style of the first sample of this post. Aesthetic matters.
At another level, I would like managers to understand that code quality is desirable, that change is possible. Not only that, but semantics preserving micro-changes are possible too. We performed a drastic code rewrite a step at a time. Each step was fairly small, allowing small commits and minimizing risks. Success is a journey, not a starting point.

Refactoring code is a multi-facet activity. Let me recapitulate the various techniques that were introduced in this post:
  • separating concerns (step I),
  • inverting conditionals and using early returns to make the code more malleable (step I),
  • slightly altering the program behavior (sometimes even the specification) to open up more factorization opportunities (step I),
  • detecting code smells (step II),
  • replacing comments by well-named methods (step II), 
  • extracting methods to improve structure (step II),
  • using exceptions to reduce cognitive load and fail early (step III),
  • cleaning up code to let bugs surface (step IV),
  • adding test before fixing bug (step IV),
  • DRY (step IV),
  • taking advantage of fixing bugs to clean up code (step IV),
  • making code testable (step IV and V),
  • injecting dependencies as constructor parameters to enable tests (step V),
  • replacing switch statements with inheritance (step VI),
  • increasing object encapsulation by moving methods around (step VI),
  • converting code into data: switch into hash-maps (step VII),
  • eliminating null from the system, filtering it at the boundaries, having defense at the perimeter (final word).