...using jolokia to monitor/manage SwitchYard

Install jolokia

Get the latest jolokia war file from their website, rename it to jolokia.war and deploy it into the server.

Get a list of all SwitchYard MBeans

All SwitchYard MBeans are registered under the org.switchyard.admin JMX domain name, as per thedocumentation. So we can get a list of what we have:

http://localhost:8080/jolokia/list/org.switchyard.admin

or a description of an MBean:

http://localhost:8080/jolokia/list/org.switchyard.admin/name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding

As it is mentioned on the documentation, there are different types of MBeans:

• Application: Management interface for a SwitchYard application.
• Service: Management interface for a composite service in a SwitchYard application. One MBean is registered per composite service.
• Reference: Management interface for a composite reference in a SwitchYard application. One MBean is registered per composite reference.
• Binding: Management interface for a gateway binding attached to a composite service or reference. One MBean is registered per binding instance on an application’s composite services and references.
• ComponentService: Management interface for a component service in a SwitchYard application. One MBean is registered per component service.
• ComponentReference: Management interface for a component reference in a SwitchYard application. One MBean is registered per component reference.
• Transformer: Management interface for a transformer in a SwitchYard application. One MBean is registered per transformer.
• Validator: Management interface for a validator in a SwitchYard application. One MBean is registered per validator.
• Throttling: Management interface for throttling a service in a SwitchYard application. One ThrottlingMBean is registered per composite service instance.

There are two additional MBean objects, that are superclasses, that define custom behavior.

• Lifecycle: Supertype of BindingMXBean which provides operations related to lifecycle control for service and reference bindings.
• Metris: Supertype of multiple MBeans providing message metrics information.

Starting/Stopping bindings

As service and reference bindings extends the Lifecycle MXBean, we can start or stop a binding, and know in what state they are:

• Check the state

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/State

• Stop the binding

http://localhost:8080/jolokia/exec/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/start

• Check the state

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/State

• Start the binding

http://localhost:8080/jolokia/exec/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/start

• Check the state

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/State

Geting metrics

If you want to get metrics, it is very simple, the only thing is that you need to know which metrics are worth for you, as every component, composite and binding provides with many metrics. Once you know what information you need, you can use jolokia to get the information, and maybe use that information to feed an ElasticSearch or InfluxDB database, and use Kibana/Graphana to view the information in a graphical way, and explore this information. Also RTGov is available.

• Get all the information available for a binding

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding

• Get the TotalCount for a binding

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/TotalCount

Getting metrics from multiple MBeans

You might want to get some metrics for more than one MBean. You can use wildcards for this, and knowing which types of MBeans and information you want is very easy.

http://localhost:8080/jolokia/read/org.switchyard.admin:name=*,service=*,type=Binding/MinProcessingTime

• More complex pattern

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22*soap*%22,service=*,type=Binding/MinProcessingTime

Search the MBeans you care for

When you have many apps deployed, you might not know which MBeans are there, and their ObjectNames. You can search for them:

http://localhost:8080/jolokia/search/org.switchyard.admin:type=Binding,*

Demo

If you want to test this, I have created a Dockerfile that you can use right away, based on the latest SwitchYard image. It is available here.

You just need to get this file, and build the image:

curl https://raw.githubusercontent.com/jorgemoralespou/fsw-demo/master/monitoring-with-jolokia/Dockerfile -o Dockerfile
docker build --rm -t "switchyard-with-jolokia"

And then run it:

docker run -it --rm -p 8080:8080 -p 9990:9990 switchyard-with-jolokia

...where to bundle SwitchYard application’s dependencies

The one problem, though, that I’ve constantly see is where to package your dependencies. We constantly fail to get the common classes used by many of the applications in a proper place.

Package common classes and model classes

If a class is going to be used by 2 or more SwitchYard applications, this class needs to be placed in a place where both applications will load it using the same classloader.

In a JEE world if the same class is loaded with different classloaders, it is not the same class.

Let’s take for example an application consisting of 3 SwitchYard applications, 2 of them (Sy app 1 and Sy app 2) packaged in an .ear file and another one (Sy app 3) packaged as a jar. These applications use some common classes (JAXB model, Entity beans, utilities, BaseMessageComposers, …​) that are packaged in two jar files, dependency A and dependency B.

If you have a request that initiated through Sy app 1 and calls to Sy app 2, you can safely use any class that is bundled in dependencyA or dependencyB, as long as for dependencyB you have a reference to the appropriate module (This dependency can be deployed either as a dynamic module, if left in the deployments folder, or as a static module, if registered as a module in modules folder). A reference to a module can be specified in the META-INF/MANIFEST file or in jboss-deployment-structure.xml file. (See JBoss documentation for how to do this).

If you have a requets that initiated through Sy app 1 and calls to Sy app 2, you can only use classes that are bundled in dependencyB for request object, otherwise you’ll most probably have a ClassCastException due to the class being loaded with two different class loaders. Any other use of classes from dependency A or B should be safe as long as it doesn’t use any object that is going from Sy app1 to Sy app 2, or viceversa, on the response path.

There are also some bugs in SY 1.1.1 (FSW 6.0) that raises some other similar and related problems, due to classloaders not properly propagated, and the incorrect use of some static code in serialization framework.

...How to setup JBoss EAP with RHEL 7 (systemd linuxes)

JBoss EAP (or Wildfly) has an init.d script that does not play well with systemd startup systems. To configure it, it is as easy as following this simple steps.

1- Create a group and user for the JBoss EAP process (username, uid, gid, and home to your preferences)

groupadd -r jboss -g 1000
useradd -u 1000 -r -g jboss -m -d /opt/jboss -s /sbin/nologin -c "JBoss user" jboss

2- Create the home folder for the user

chown -R jboss:jboss /opt/jboss

3- Create configuration directory for the JBoss EAP instance, create the configuration file for the EAP instance (of course, your values here), and then set appropriate permissions for the used folders.

mkdir /etc/jboss-as

cat > /etc/jboss-as/jboss-as.conf <<EOF
JBOSS_USER=jboss
STARTUP_WAIT=30
SHUTDOWN_WAIT=30
JBOSS_CONSOLE_LOG=/var/log/jboss-as/console.log
JBOSS_HOME=/usr/share/jboss-as/jboss-eap-6.X
EOF

mkdir /var/log/jboss-as
mkdir /var/run/jboss-as
chown -R jboss:jboss /var/log/jboss-as
chown -R jboss:jboss /var/run/jboss-as

4- Create the service file

cat > /etc/systemd/system/jboss-as-standalone.service <<EOF
[Unit]
Description=Jboss Application Server
After=syslog.target network.target

[Service]
Type=forking
ExecStart=/usr/share/jboss-as/bin/init.d/jboss-as-standalone.sh start
ExecStop=/usr/share/jboss-as/bin/init.d/jboss-as-standalone.sh stop

[Install]
WantedBy=multi-user.target
EOF

5- Restart the systemctl daemon, start the service, verify it’s status and enable the service

systemctl daemon-reload
systemctl start jboss-as-standalone.service
systemctl status jboss-as-standalone.service
systemctl enable jboss-as-standalone.service

6- Additionally, if you need to create a firewalld rules for the EAP, do:

cat > /etc/firewalld/services/jboss-as-standalone.xml
<?xml version="1.0" encoding="utf-8"?>
<service version="1.0">
     <short>jboss-as-standalone</short>
     <port port="8080" protocol="tcp"/>
     <port port="8443" protocol="udp"/>
     <port port="8009" protocol="tcp"/>
     <port port="4447" protocol="tcp"/>
     <port port="9990" protocol="udp"/>
     <port port="9999" protocol="tcp"/>
</service>
EOF

firewall-cmd --zone=public --add-service=jboss-as-standalone
firewall-cmd --permanent --zone=public --add-service=jboss-as-standalone
firewall-cmd --zone=public --list-services
firewall-cmd --permanent --zone=public --list-services