My Adventures in Coding

November 9, 2018

Jetty – Profiling with JMX and VisualVM

Filed under: Jetty,JMX — Brian @ 3:36 pm
Tags: , ,

JMX (Java Management Extensions) combined with VisualVM are very useful tools to have when profiling a Java web app running on a Jetty web server, especially when you want to pinpoint what part of your application is using the most CPU or Memory.

This is a quick guide to help you enable JMX on your Jetty web server and start profiling your application using VisualVM.

Setup JMX on Jetty

Configure JMX

First, we will need to add the following JMX configuration section to your Jetty web server’s start.ini file.

IP Address: You will need to update the text “YOUR_IP_ADDRESS_HERE” in the configuration to be the IP Address of the server this config is being added to, to be able to connect to JMX remotely. If you are only using VisualVM from the same machine then localhost will work fine. So replace YOUR_IP_ADDRESS_HERE with either “localhost” if you will be running VisualVM on the same machine, or the IP Address of your Jetty server if you will use VisualVM remotely.

Port: The default port is 1099, which is why those lines are commented out, but you can change the port if you would rather use a different one.

# ---------------------------------------
# Module: jmx-remote
# Enables remote RMI access to JMX
# ---------------------------------------
--module=jmx-remote

## The host/address to bind the RMI server to.
jetty.jmxremote.rmiserverhost=YOUR_IP_ADDRESS_HERE

## The port the RMI server listens to (0 means a random port is chosen).
# jetty.jmxremote.rmiserverport=1099

## The host/address to bind the RMI registry to.
jetty.jmxremote.rmiregistryhost=YOUR_IP_ADDRESS_HERE

## The port the RMI registry listens to.
# jetty.jmxremote.rmiregistryport=1099

Restart Jetty

Now that the config has been changed, you will need to cycle Jetty for the JMX changes to take effect.

You can verify JMX is running on port 1099 using netstat:

netstat -tunlep | grep LISTEN | awk '{print $4}'

That is it! You now have JMX enabled on your Jetty server and are ready to start profiling!

Profile with VisualVM

Open/Install VisualVM

If you have JDK 8 installed, you should have VisualVM in your bin folder with the name “jvisualvm.exe”, for example:
C:\Program Files\Java\jdk1.8.0_191\bin\jvisualvm.exe

If you have a newer version of the JDK, VisualVM no longer comes bundled so you must download the standalone version.
https://visualvm.github.io/download.html

Connect to a Remote Jetty Server

Open VisualVM

Right click on the “Remote” option and select “Add Remote Host”:
1-Add_Remote_Host

Enter the IP Address of the Jetty server and click “OK”:
2-Enter-IP-Dialog

Now find the server address in the “Remote” section, right click, and select “Add JMX Connection”:
3-Add_JMX_Connection

In the dialog box that opens, enter the JMX port, which in this case is 1099 and click OK:
4-Add_JMX_Connection_Dialog

The JMX connection will show up in the “Remote” section, right click on it and select “Open”:
5-Open_JMX_Connection

You will now see the overview page:
6-Overview

Monitoring in VisualVM

Monitoring

Go to the “Monitoring” tab to see an overview of the application’s CPU, Memory, and Threads:
VisualVM Overview

Threads

Go to the “Threads” tab to see stats on all threads:
7-Threads-Tab

Sampling CPU & Memory in VisualVM

One of the many helpful features of VisualVM is the ability to sample CPU or Memory for a short period of time. This can be very useful when a performance problem is happening on a server and you want to find out what is using the most CPU or Memory at that moment in time.

Sample CPU

To sample CPU, go to the “Sampler” tab and click the “CPU” button:
8-Sampler-CPU

The sampler will run until you click the “Stop” button:
8-Sampler-CPU-Stop

After stopping, you can expand an item to get more detail:
9-Sampler-CPU-Details

Hot Spots

The Sampler also has a useful feature called “Hot spots” which shows you areas of the code that are the most active. I have used this feature before to help pinpoint methods in our code that were using the majority of our server’s CPU:
10-Sampler-CPU-Hot-Spots

Sampling by Memory

Exactly the same as sampling by CPU, just use the “Memory” option instead.

Sampling by Package

The Sampler also allows you to filter by package name. So for example if you suspect a problem is within a certain package, you can restrict the Sampler to just return results for that package.

In the upper right hand corner of the “Sampler” tab you will see a checkbox called “Settings”. When you check that box the profiled packages option will become visible:
11-Sampler-Package-Filtering

Note, that by default the option “Do not profile packages” is selected, so if you want to profile a specific package be sure to switch to “Profile only packages”:

Here I am setting it to “Profile only packages” and adding the name of a package I want to filter on:
12-Sampler-Package-Filtering-Edit

Note: In the profile packages window:
com.google.gson.* (All classes in package)
com.google.gson.** (All classes in package and subpackages)

So now I am only sampling classes in the package I have specified:
13-Sampler-Package-Filtering-Run

And of course you can still open the “Hot spots” window as well.

Fequently Asked Questions

My stats stopped updating

This usually means that the app was restarted so the PID changed. When VisualVM gets disconnected, it does not give you any feedback. Instead your charts will just stop updating.

First, you must close the existing tab:
14-Close-The-Tab

Second, you must also close the existing JMX connection as well since it is tied to the PID:
14-Remove-JMX-Connection

Third, you can now open a new connection following the steps explained in the previous section.

I hope that helps!

Blog at WordPress.com.