Feed the Daemon: Faster Builds with Maven Reactor Modules #SnailToRocket

Many Apache Maven reactor projects, especially those with multiple reactor modules, will greatly benefit from using the Apache Maven Daemon (mvnd). This is due to the fact that often reactor modules of the Apache Maven build can be compiled in parallel. However, depending on the structure of the inter-module dependencies, the build could still end up mostly serial. This will result in slow build time (compilation, test execution, packaging).

This is especially true in single-module projects where no modules can be parallelized. Using the Apache Maven Daemon will not yield many benefits, except the already-running daemon (the Java VM executing Apache Maven).

So, how can we ease the pain of slow builds, resulting from a serial reactor module execution order?

If you have never worked with Apache Maven reactor modules before, here is a small primer: Instead of having just one pom.xml file producing a single .jar or .war file as its main result, a reactor has multiple pom.xml files, arranged in a hierarchical directory structure. Each directory containing a subdirectory with a pom.xml file will not create a .jar artifact. Instead, it will point to one or multiple subfolders containing pom.xml projects which will produce the .jar artifacts.

However, most of those reactor modules will usually have a dependency on other modules of this so-called reactor build. Apache Maven will figure out in which order those modules can be built without creating a dependency cycle, then proceed.

Each module contains its own package, classes and tests. Those will be shielded from other classes unless a direct compile- or provided-scope dependency was declared.

Potentially, multiple modules of this reactor build can be built at the same time — in parallel. And this is where we are heading.

In the example above, we only have two modules. They need to be built sequentially, as the "cli app module" depends on "module_1".

To really get some gain, we need more modules. Maybe your application can be torn down into multiple "libraries".

If multiple modules do not have inter-project dependencies on each other, they can be built in parallel. The number of parallelism can be set using the -T (threads) flag on mvn and defaults to 1. The Apache Maven Daemon (mvnd) defaults to the number of cores minus one (nproc -1), with a minimum of 1.

For each of those independent reactor modules, compilation and test execution will be done at the same time. This will lead to better utilization of resources, like CPU and memory (RAM).

All demos in the next two sections are uploaded to this repository: bmarwell/snail2rocket-feed-the-daemon-demo. Please check it out and see if you get similar results!

This example will slow test execution by defining methods in src/main/java with a Thread#sleep() method with a default sleep period of 100ms. You can experiment with the sleep period using mvnd verify -Dmethod.timeout={timeout_ms}.

Now you may think: »Great! Let’s chop down all our projects into many, many modules!«

But, alas, there is always a catch (as always). Things to consider:

Now, there is more:

In this article, we learned how to significantly speed up your build by modifying the architecture of your software. In some cases, you can cut down total build time not only by half, but even by a factor of five or more. Plus you get enforced architectural discipline at no extra cost.

So, should you use this technique? As always: it depends. Do experiment in your projects and see if you seed a benefit in either architecture (clearer responsibilities, less complex testing) or build time - or both. If so, show this to your team and speak about adapting a more complex reactor build!

This article is part of the #SnailToRocket series.