Glowroot is a lightweighted but feature-rich java VM agent, which will allow you to collect and evaluate metrics from almost any VM. For larger setups, you might want to install a central collection instance. This guide covers how to set up Glowroot Central on OpenLiberty, a lightweigt and extremely fast Java Application Server (aka Servlet Container).
Downloading and installing OpenLiberty
An overview of the Liberty family
The first step is to download and install WebSphere Liberty Profile or OpenLiberty. The distributions "OpenLiberty" and "IBM WebSphere Liberty Profile" differ in what they provide: Liberty Profile is available with enterprise support and advances featuers. For this showcase and most applications, OpenLiberty is more than suficient.
Both distributions are available in multiple flavours / editions:
Just contains the internal OSGi Framework. Best if you want to create your own, tightly feature-restricted container.
Contains all the microProfile features plus some common web features, like jaxrs, CDI, concurrency, monitoring, JWT (Json Web Tokens), etc.
This distribution is enough to run Glowroot Central on Openliberty.
It is just 51 MiB for v18.104.22.168!
The most common distribution, if you came from the Java EE (Enterprise Edition) standards. Contains only some of the more advanced features, light ejbLight, JPA, webSockets, and others.
Still just 86 MiB!
- Jakarta EE 8
As the name suggests, contains all features of the Enterprise Edition which are needed to get a JavaEE certification. Contains the EJB features.
Downloading and installing OpenLiberty MicroProfile Edition
Now, this step couldn't be easier, especially for a classic installation. Just head over to the runtime release page, download the latest MicroProfile archive and extract it to a directoy of your choice. Note that I like to rename the resulting "wlp" directory to something like "openliberty-22.214.171.124mp" to keep track of multiple installations.
Configuring and installing glowroot-central on Open Liberty
The actual app installation of glowroot-central is pretty easy, as well. It is done in four steps:
- Creating a liberty service instance
- Copying the glowroot-central.war artifact to the apps folder
- Configuring the OpenLiberty server instance
- Configuring Glowroot
Creating a liberty server instance
The first step will be to create a self-contained instance for glowroot. Unlike you might be used from tomcat, you start a JVM (process) for every (web) application from their own directory with their own directory without duplicating the Liberty installation. Go into the liberty installation directory and type:
user@aeon:~/.local/apps/openliberty-126.96.36.199mp$ ./bin/server create glowroot-central Server glowroot-central created.
That's it! The basic configuration was created in the subdirectory
./usr/servers/glowroot-central. This directory is called a server directory, in this case this is the "glowroot-central" server.
Pro tip: If you want to use the same server instance with multiple installations of OpenLiberty, set the environment variable
WLP_USER_DIR to something like
$HOME/.local/share/wlp-usr prior to running the create command. This way you do not need to copy your server over to the new installation and maintain multiple instances.
Copying the artefact to its place
Go to the glowroot release page and download the latest
glowroot-central-*-dist.war. If the latest version is still 0.13.6, you should rather use 0.13.5 because of an infinispan upgrade bug which is not yet released.
Then copy the artefact into the
apps/ folder of your liberty server directory.
Configuring the OpenLiberty server instance
Now open the file
server.xml of your glowroot-central server. In the features section, you will see the "microProfile-3.3" feature activated. This is a "meta-feature", because it pulls in other features (such as mpConfig, mpHealth, etc.). We can reduce the number of features by specifying them one by one like so:
<server> <!-- Enable features --> <featureManager> <!-- glowroot needs at least servlet-3.1, but only 4.0 is included in recent mp and web editions. --> <feature>servlet-4.0</feature> </featureManager> <!-- other tags omitted for brevity --> <server>
You will also need tell Liberty where to find the app. We could have placed the war file into the dropins/ folder instead for automatic discovery, but by specifying the war application manually we have more control (like specifying a custom context root i.e. root path).
<server> <!-- … features … --> <application id="glowroot" location="glowroot-central-0.13.5-dist.war" name="glowroot" type="war" context-root="/" /> </server>
While we could configure many more things (more on that later), this is sufficient for now.
Pro tip: If you want to specify a specific JDK/JRE which OpenLiberty should use for running the server, open the file server.env and set the
JAVA_HOME variable. This will be specific for this very server and will not affect anything else on your PC.
Configuring Glowroot Central
Glowroot will pick up it's configuration from a system variable called
glowroot.central.dir. Let’s create this directory and point glowroot to it.
user@aeon:~/.local/share/wlp-usr/servers/glowroot-central$ mkdir glowroot-central-properties user@aeon:~/.local/share/wlp-usr/servers/glowroot-central$ touch glowroot-central-properties/glowroot-central.properties user@aeon:~/.local/share/wlp-usr/servers/glowroot-central$ echo "-Dglowroot.central.dir=$PWD/glowroot-central-properties" >> jvm.options
As you can see, you can configure Liberty’s jvm options (including system properties) using a newline-delimited file called
jvm.options. For the time being, we will not bother about any other JVM options and create the glowroot config.
# default is cassandra.contactPoints=127.0.0.1 cassandra.contactPoints=127.0.0.1 # default is to connect without credentials cassandra.username= # default is to connect without credentials cassandra.password= # default is cassandra.keyspace=glowroot cassandra.keyspace= # default is cassandra.consistencyLevel=QUORUM cassandra.consistencyLevel= # default is grpc.bindAddress=0.0.0.0 grpc.bindAddress=0.0.0.0 # default is grpc.httpPort=8181 # set this to "none" in order to not serve gRPC over HTTP grpc.httpPort=8181 # default is grpc.httpsPort=none # set this to a port number in order to serve gRPC over HTTPS on that port # the certificate and private key to be used must be placed in the same directory as this properties # file, with filenames "grpc-cert.pem" and "grpc-key.pem", where grpc-cert.pem is a PEM encoded # X.509 certificate chain, and grpc-key.pem is a PEM encoded PKCS#8 format private key without a # passphrase (for example, a self signed certificate can be generated at the command line meeting # the above requirements using OpenSSL 1.0.0 or later: # "openssl req -new -x509 -nodes -days 365 -out grpc-cert.pem -keyout grpc-key.pem") grpc.httpsPort=none jgroups.configurationFile=jgroups-tcp.xml jgroups.localAddress=aeon.fritz.box jgroups.localPort=7800 jgroups.initialNodes=localhost:7800 jgroups.password=my_glowroot_top_secret_password
The above config will go to the
glowroot-central.properties file we touched earlier. We are assuming a cassandra installation on the same machine, and we have two more ports, which will give us three ports in total:
- TCP Port 9080 where liberty will serve Glowroot’s web interface
- TCP Port 7800 for jgroup (glowroot node discovery)
- TCP Port 8181 where the glowroot agents will connect to.
Running Glowroot Central on OpenLiberty
Now that you have everything configured (leaving out cassandra for yourself), you can fire up OpenLiberty with the glowroot-central server. Therefore, head back to the OpenLiberty directory and run
user@aeon:~/.local/apps/openliberty-188.8.131.52web$ ./bin/server run glowroot-central Launching glowroot-central (Open Liberty 184.108.40.206/wlp-1.0.43.cl200820200721-1900) on Eclipse OpenJ9 VM, version 11.0.6+10 (en_GB) [AUDIT ] CWWKE0001I: The server glowroot-central has been launched. [AUDIT ] CWWKZ0058I: Monitoring dropins for applications. [AUDIT ] CWWKT0016I: Web application available (default_host): http://localhost:9080/ [AUDIT ] CWWKZ0001I: Application glowroot started in 4.355 seconds. [AUDIT ] CWWKF0012I: The server installed the following features: [servlet-4.0]. [AUDIT ] CWWKF0011I: The glowroot-central server is ready to run a smarter planet. The glowroot-central server started in 6.083 seconds. 2020-08-18 09:47:49.390 WARN org.glowroot.common.util.Version - could not find Build-Time attribute in META-INF/MANIFEST.MF file 2020-08-18 09:47:49.392 INFO org.glowroot - Glowroot version: 0.13.5 2020-08-18 09:47:49.394 INFO org.glowroot - Java version: 11.0.6 2020-08-18 09:47:49.395 INFO org.glowroot - Glowroot home: $USER/.local/share/wlp-usr/servers/glowroot-central/glowroot-central-properties 2020-08-18 09:47:50.531 INFO o.i.f.GlobalComponentRegistry - ISPN000128: Infinispan version: Infinispan 'Infinity Minus ONE +2' 9.4.5.Final 2020-08-18 09:47:51.009 INFO o.i.r.t.jgroups.JGroupsTransport - ISPN000078: Starting JGroups channel ISPN 2020-08-18 09:47:53.202 INFO org.infinispan.CLUSTER - ISPN000094: Received new cluster view for channel ISPN: [aeon-58615|0] (1) [aeon-58615] 2020-08-18 09:47:53.298 INFO o.i.r.t.jgroups.JGroupsTransport - ISPN000079: Channel ISPN local address is aeon-58615, physical addresses are [127.0.0.1:7800] 2020-08-18 09:47:54.199 INFO c.datastax.driver.core.ClockFactory - Using native clock to generate timestamps. 2020-08-18 09:47:54.214 INFO com.datastax.driver.core - DataStax Java driver 3.6.0 for Apache Cassandra 2020-08-18 09:47:54.218 INFO c.d.driver.core.GuavaCompatibility - Detected Guava >= 19 in the classpath, using modern compatibility layer 2020-08-18 09:47:54.366 INFO com.datastax.driver.core.NettyUtil - Did not find Netty's native epoll transport in the classpath, defaulting to NIO. 2020-08-18 09:47:54.642 WARN org.glowroot.central.CentralModule - waiting for Cassandra (127.0.0.1)...
Well, that’s it! The
run command will actually start the server as a foreground process, suitable for a quick test or running in containers. You can also use the
start command instead for forking the serer into the background. You will find the logs in the server's
Browse to http://localhost:9080/ to see Glowroot’s web interface once cassandra is set up.
Pro tip: Boot up a quick cassandra instance from docker by running this command beforehand:
docker run \ -p 127.0.0.1:9042:9042 \ -p 127.0.0.1:9160:9160 \ --name cassandra \ cassandra:3
Additional configuration options for OpenLiberty
Disabling dropin scanning and early servlet loading
To disable update triggers when changing the configuration or active file system scanning, you can add an applicationMonitor tag with the specific attributes.
You might also want to add two webContainer options which will allow liberty to load and initialize the servlets early and skip some (usually not needed) processing of resources in the META-INF folder of your web applications:
<server> <!-- previous options omitted for brevity --> <!-- do not defer servlet loading and skip resource processing. --> <webContainer deferServletLoad="false" skipMetaInfResourcesProcessing="true" /> <!-- do not scan the dropin folder and do not reload configuration. --> <applicationMonitor updateTrigger="disabled" dropinsEnabled="false"/> </server>
Activating more logging
Furthermore, you can activate some additional logging options. For example, http access logging (similar to Apache httpd’s access_log) and change the messages.log-timestamps to the ISO format (yyyy-mm-dd).
<server> <!-- previous options omitted for brevity --> <!-- use the ISO date/time format and do not rotate logs early. --> <logging isoDateFormat="true" maxFileSize="0" /> <httpAccessLogging enabled="true" /> </server>
Set JVM options (like max heap) for OpenLiberty
Another popular configuration is controlling how much heap the web application can consume by modifying the
jvm.options file. For instance, you might want to add the following entries to your configuration file.
# set min and max heap -Xms512m -Xmx2048m # set the region, language and timezone of this server -Duser.language=en -Duser.region=GB -Duser.timezone=Europe/London # force servlet parameter and body encoding to UTF-8 -Dfile.encoding=UTF-8 -Dclient.encoding.override=UTF-8
Use a different java runtime for Glowroot Central Collector
OpenLiberty was developed with the Eclipse OpenJ9 JVM in mind. The OpenJ9 JVM is another OpenJDK variant which was derived from the former proprietary IBM J9 VM and is an alternative to hotspot. It also includes a different heap layout and different garbage collectors, as well as more JVM debug options.
Download Java 11 OpenJ9 from adoptopenjdk.net or using
jabba install firstname.lastname@example.org and set the path via your
jabba install email@example.com echo "JAVA_HOME=$HOMEfirstname.lastname@example.org" >> server.env
In conclusion, you now know how to set up Glowroot Central Collector on OpenLiberty and have learned about the most important configuration options for both the actual application and OpenLiberty.
Do you have more questions? Put them into the comments!
- Similar article: Running Guacamole on IBM Liberty Profile
- OpenLiberty project
- OpenLiberty on GitHub
- OpenLiberty configuration reference
- Glowroot github page