In the past few years new frameworks have been introduced. As UNL Solutions is currently working on a larger microservice-based application using Java, we want to check which Java framework is the best fit for such an architecture.
As of 2020, Java remains one of the most popular programming languages to build web applications — although the competition from newer languages like Go, Python, and TypeScript has increased significantly.
Inside the Java programming, the Spring Framework is a standard for microservice development. Spring framework is easy to use and allows for an efficient and painless development.
However, in the past few years new frameworks have been introduced. As UNL Solutions is currently working on a larger microservice-based application using Java, we want to check which Java framework is the best fit for such an architecture.
Our main focus, therefore, will be the ease of development + the resource management of the generated microservices.
Resource management in Spring (and most of the Java Platform, actually) never has had the best reputation, especially when it comes to the overhead required by a single process. With the raise of microservice architectures, and their huge number of small instances, this becomes more and more of an issue.
Java Microservice Frameworks: candidates Spring
Spring came into being in 2003 as a response to the complexity of early Java Enterprise. In its core, Spring started as a dependency injection (DI) and an aspect-oriented programming (AOP) framework and evolved into an easy-to-use web application framework. Through its extensive documentation, widespread usage, and countless libraries, Spring allows developers to create and maintain applications efficiently and provides a flat learning curve.
Spring performs DI at runtime using reflection. Thus, when a spring application is started, the classpath is scanned for annotated classes. Based on this, the concrete objects are instantiated and linked.
While this is very flexible and developer friendly, it can make the startup slow and it’s very memory-consuming. Also, it’s quite difficult to migrate this mechanism to GraalVM as it doesn’t support reflection.
Micronaut
Micronaut is a modern full stack–microservices framework, introduced in 2018 by the creators of the Grails Framework.
It provides all the tools necessary to build full-featured microservice applications. At the same time, it aims to provide a fast startup and a reduced memory footprint. This goal is achieved by using Java annotation processors to perform DI, create aspect-oriented proxies, and configure the application at compile time instead of runtime.
Many of the APIs within Micronaut are inspired by Spring and Grails. This is by design and helps bring in new developers quickly. Micronaut, therefore, provides modules like Micronaut HTTP, data, security, and connectors to various other technologies. However, the maturity of those libraries is still behind its Spring counterparts.
Quarkus
Quarkus is a Kubernetes-native Java framework introduced by Red Hat in 2019. It’s built on top of standards such as MicroProfile, Vert.x, Netty, and Hibernate.
The goal of Quarkus is to make Java a leading platform in Kubernetes by allowing faster startup, low memory consumption, and near-instant scale up in container-orchestration platforms. Quarkus reaches this by using custom Maven plugins to perform as much work possible during compile time instead of build time (in Quarkus, this is also called compile time boot).
Quarkus is using mostly existing standard technologies, but it’s open for extension. However, since the project has only been started a year ago, the maturity and compatibility of those extensions isn’t always clear. This is likely to change in the future as the platform will grow.
Helidon MicroProfile
The MicroProfile project was started 2016, when it was unclear if and how Oracle would continue working on Java Enterprise.
Like its predecessor, JEE, MicroProfile is a specification that can be implemented by various vendors.
Multiple such implementations have since been presented, most notably Payara Micro and Helidon MP. Payara is a Jakarte EE server derived from GlassFish, and Payara Micro is its MicroProfile implementation. Helidon is a runtime started by Oracle in 2018, offering its own implementation of the MicroProfile specification.
As they’re derived from JEE, the MicroProfile specifications are mature and well documented. However, there is a lack of connectors for modern technologies or replacements for libraries like Spring Data and Spring Security.
Also, the future of MicroProfile is unclear as, in the meantime, the development of Jakarta EE (also within the Eclipse Foundation) has started. It, therefore, seems likely the two projects will be merged — or at least closely coordinated — in the future.
Time to compare the mentioned frameworks
To compare the mentioned frameworks, we’ve implemented a simple application using each of them. The example application consists of a REST interface to create, read, update, and delete objects and a relational database connector that stores these objects into a table.
If a framework supports different ways to access databases, we tried to implement sample projects for the different variants. Then we compared the performance of those applications.
We ran this application using an OpenJDK Docker image. If a framework supports the generation of native GraalVM images, we also compared the performance of those.
We have compared the performance of those applications on three key phases:
How easy was it to implement the example application? To implement the frameworks, we had to check the documentation as well as search for information on platforms like Stack Overflow.
How long does it take to compile the application? We’ve measured the time it takes to perform a clean build, including the generation of the Docker image. For GraalVM, this includes the time to generate the native image.
How long does it take to start an application? Here I have measured the time it takes between running docker up until the application correctly answers the first HTTP request. Also, we compared the measured memory footprint of the idle applications just after boot.
Load: How many requests can the application handle at peek? We used JMeter to perform load testing and tested the application with 25% of the requests performing a database write and 75% of them performing only a database read. Then we measured the memory footprint of the application on its peak performance again.
We executed all tests on a Google Cloud Platform virtual machine with four Intel Haswell CPUs and 15 GB of memory running Ubuntu 19.01. All measurements have been repeated multiple times to avoid disturbing factors.
Now it’s time to see results.
Ease of development
As we only had previous knowledge using Spring Boot, this is a bit of an unfair comparison. However, when checking the documentation and the available information and examples, Spring is by far the easiest framework to start with.
Micronaut’s documentation is well done, and it has a similar API to Spring and Grail. Therefore, it’s easy for a Spring developer to start with it.
Quarkus has a bit of a steeper learning curve, in our opinion, as the libraries and APIs as less mature compared to Spring and Micronaut.We were especially missing easy database access.
But Helidon was, in our opinion, clearly in last as we struggled quite a bit to get the application to run.
Compilation
The compile times when using OpenJDK were quite similar for all the frameworks and were between 6.98 seconds (Spring using JDBC) and 10.7 seconds (Quarkus).
However, the generation of native GraalVM images turned out to be quite time intensive and took between 231.2 seconds (Micronaut using JDBC) and 351.7 seconds (Micronaut using JPA). This makes native images basically useless for development, as waiting four minutes for compilation of a simple application is just too much.
Startup
The Spring Boot application using Spring Data took an average of 8.16 seconds to boot. Removing JPA and Spring Data reduced this to only 5.8 seconds.
Here, Micronaut (5.08 seconds using JPA and 3.8 seconds using JDBC) and Quarkus (5.7 seconds) kept their promise of lower startup times.
Only Helidon MP was even slower than Spring — with 8.27 seconds on average.
However, the real winner here is GraalVM. The bootup times for the native images were between 1.39 seconds (Quarkus) and 1.46 seconds (Micronaut using JDBC), significantly faster than the OpenJDK implementations.
The memory usage directly after boot presented itself quite similarly. Spring allocated 420 MB of memory (using Spring Data) and 261 MB (using JDBC).
Micronaut was at 262 MB using JPA and 178 MB using JDBC.
And Quarkus, at 197 MB, performed way better. Helidon MP, with 414 MB, was similar to Spring Boot.
Also here, the native GraalVM images outperformed the OpenJDK implementation substantially, using only between 7 MB (Quarkus) and 27 MB (Micronaut using JPA) of memory.
Peak performance
Under load, Spring Boot performed quite well, being able to serve 342 (using Spring Data) and 216 (JDBC) requests per second (r/s) and using 581 MB (Spring Data) and 484 MB (JDBC) of memory. Helidon was clearly last, only able to serve 175 r/s while allocating over 1 GB of memory.
The other frameworks were able to serve between 400 r/s (Quarkus running as a native image) and 197 r/s (Quarkus on OpenJDK). The various Micronaut implementations were somewhere in-between, with a slight advantage for JDBC over JPA and native images over OpenJDK.
In terms of memory usage, Quarkus on OpenJDK performed surprisingly well, consuming only 255 MB of memory. This is even less that the same application running as a native image — where it took 368 MB of memory on average.
Micronaut, however, turned out to be quite wasteful. The JPA implementation running in OpenJDK used 880 MB on average, which is more than 50% above the memory usage of Spring. However, using JDBC and native images helped Micronaut to reduce its memory footprint down to 367.8 MB of memory.
Our conclusion:
The new Java frameworks Micronaut and Quarkus promised faster startup times and lower memory footprints compared to existing frameworks like Spring and MicroProfile.
They do follow up on this promise — but only when idle or under a small load. Here they outperform Spring, especially when combining them with native GraalVM images. However, under load, they don’t offer much of an edge, not even when running as native images.
And as Spring still offers, by far, the best developer experience, it’s still the best suited Java framework for a microservice application, in my opinion — even considering its poor performance at startup.
Surprising for us was the huge costs of using Hibernate/JPA/Spring Data. Even for this very simple application, the overhead in terms of memory (but also the r/s) were huge. Here, we especially liked the solution of Micronaut Data, which can auto generate repository code without the need of the JPA. This is really something that could be added to Spring Data as well.
The native GraalVM images turned out to be incredibly fast and memory efficient at startup, but under load, they don’t offer a significant advantage. As the generation of a native GraalVM provides some additional difficulties and the compilation time increases drastically, this technique is currently only useful if a fast startup is needed — e.g., in a serverless architecture or to scale up very fast. In all other cases, the costs are still way too high compared to similar performances under load.
