Enable JMX for Scala/Sbt Project

Java Management Extensions (JMX) is a Java technology that supplies tools for managing and monitoring applications, system objects, devices (such as printers) and service-oriented networks. Those resources are represented by objects called MBeans (for Managed Bean).

In this blog, I will show the steps to enable it for Scala/Sbt project and also show how to connect VisualVM to a remote server

Prerequisite

Follow the link below to install VisualVM: https://visualvm.github.io/.

If you don’t want to use VisualVM, Jconsole is an option too.

Setup sbt project with JMX

Enable jmx settings in build.sbt

javaOptions in Universal ++= Seq(
  "-Djavax.management.builder.initial=",
  "-Djava.rmi.server.hostname=127.0.0.1",
  "-Dcom.sun.management.jmxremote=true",
  "-Dcom.sun.management.jmxremote.port=9186",
  "-Dcom.sun.management.jmxremote.rmi.port=9186",
  "-Dcom.sun.management.jmxremote.ssl=false",
  "-Dcom.sun.management.jmxremote.authenticate=false"
)

Note: if you are adding JVM settings here, you need to prefix with -J, eg,

"-J--Xmx3000M"

With this configured and if you run your program locally, jconsole and VisualVM can auto detect running processes already.

Setup for remote access

If you want to to debug service running on a remote server. e.g staging environment. you will need to expose the jmx port at the same time.

Below instructions are based on a helm chart context, since all our services are deployed to EKS through helm chart.

Add ports to Service

apiVersion: v1
kind: Service
metadata:
  ...
spec:
  type: {{ .Values.service.type }}
  ports:
    - port: {{ .Values.service.jmx.port }}
      targetPort: jmx
      protocol: TCP
      name: jmx
  selector:
    app: {{ include "application.name" . }}

Add ports to deployment.yaml

Under, Deployment.spec.template.spec.containers

ports:
  - name: jmx
    containerPort: {{ .Values.container.jmx.port }}

Finally, you can provide the port number for different envs in Values.yaml

container:
  jmx:
    port: 9186

service:
  type: ClusterIP
  jmx:
    port: 9186

Start port forwarding after service deployed to EKS

kubectl port-forward -n [namespace_name] service/[app_name] 9186:9186

Visualize with jconsole

Start jconsole in command line easily with

jconsole

Add Connection

View Metrics

Visualize with VisualVM

Add JMX Connection

Input JMX Connection

Start exploring

Click ok button, wait for a second, then you should see:

Start to exploring more data

Tuning JVM parameters

If you want to test different JVM settings, this can be done in helm chart Values.yaml:

- key: "JAVA_OPTS"    
  value: "-XX:+UnlockExperimentalVMOptions -Xms3000M -Xmx3000M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=4 -XX:+CMSClassUnloadingEnabled -XX:MaxRAMFraction=2 -XX:NewSize=3000M -XX:MaxNewSize=3000M -XX:+CMSParallelRemarkEnabled -XX:MaxMetaspaceSize=1000M -XX:+UseCGroupMemoryLimitForHeap"

This can be added in build.sbt as well, but it loses the flexibility to tune them for different environments e.g staging, performance, production.