...JBoss projects in docker containers with an external RDBMS

I’ve been recently asked this:

Our project recently switched to using RDBMS + JPA as the backend storage. As such, the distro ships with DDL that need to be installed in Wildfly/EAP. What would you recommend?

I think this is a very interesting question, so I’ll post the answer here. Of course, it’s open for discussion :-D

There are different options here, so I will summarize them.

When we talk about containers, one of the "advices" that are usually given is "1 process 1 container". As such, if you want to provide your application with an RDBMS you’ll end up with 2 processes (EAP/Wildfly and RDBMS), unless you use embedded H2, which is an in memory database, executed in the same process as the Application server.

Use H2 as RDBMS

H2 can be configured as an embedded in memory database server. This option only works if you don’t plan to use your application server (your projects really) in a clustered way, as data will be local to the application server.

When using H2 you can:

• Let Hibernate create the schema for you. This option works when you only have to create a database schema, but not populate it with data. If you have to do that, you’ll have to provide a way of creating this master data once the container is run, and only for the first time. At the end, if this is the case, it would be a hack. Anyway, from my point of view, not a recommended approach for containers.
• Precreate the schema using a DDL and also load the data. An option is to create the H2 database with the data at build time, and fed the application server with the H2 DB data files already created (schema + master data). This process of creating the database can be part of your build process. I personally don’t like it, but it works, and it seems an option for H2.

Use a full RDBMS (Mysql/Postgresql)

For this you need to have 2 containers, one for the EAP/Wildfly and another for the RDBMS. Again multiple options.

• Care only for your container. You have to prepare your application server with the appropriate RDBMS driver and datasource configuration, and configure it to inject the values through ENV variables at container creation time. This option is good, but you leave the user with a lot of boilerplate to do, as he’ll have to create the RDBMS container and load the DDL (schema + data). From my point of view, an option only if you provide with good instructions on how to do it.
• Plain docker orchestration (fig, docker-compose). Good option for single machine and to use and test your project. You have 2 containers (EAP/wildfly and RDBMS). Typically, you have to ensure that the RDBMS starts before the EAP/Wildfly process so the DDL is fully executed into the database. You can see an apiman example here. I like this approach, as you do both containers, and provide with a single command for the user to execute, but not very scalable, unless using docker-swarm.
• Kubernetes, OpenShiftv3 (The Red Hat way :-D). Good option to let people try the project on a PaaS, once v3 is available. (Free of charge on the OSE Online version). You will need for this to create all the pods, deploymentConfiguration,etc…​ required by Openshift. From my point of view, it will be the option in the coming months

Any of the options will work, but there are some constrains either on the build process (for H2) or on the runtime order (for orchestration). I like more the orchestrated containers option, although I have to admit that H2 is much easier and probably integrates better with the development process. On the other hand, the other options help you craft your project in such a way where you’ll be testing things like installation process, authentication, HA, …​ in a very early stage.

So in summary, a matter of taste.

Examples from the JBoss projects official site:

• Keycloak postgres HA (the Kube/Openshift way)
• Keycloak postgres (the stanadlone way, ready to integrate in docker compose, or kubernetes)
• Keycloak (I guess that using H2)
• jbpm using H2
• Unofficial apiman use cases

I HOPE IT HELPS

...use Infinispan caches with SwitchYard

Sometimes, when developing SwitchYard services you might need to develop certain functionality where a cache could probably provide you with great benefit, things like:

• Clustered configuration
• Clustered storage of information
• Clustered synchronization service

We are lucky that we can use Infinispan cache with SwitchYard, as it is in the application server, and if you are using FSW you are entitled to use it for your applications, and you do not need an additional entitlement for JDG.

Here, I’m going to explain very briefly what are the parts you need to take into ocnsideration in order to make use of Infinispan. The rest, what to do with the cache, then falls down on your side.

Configuration

Wildfly and EAP brings into their configuration the infinispan subsystem, where you can define you own cache containers. A cache container is a logical grouping of caches, that will be registered and accesible through JNDI for your application to use. There are multiple configuration that you can set per container, and per cache in a container, and you should check Infinispan configuration for all the available options, but the section you can/should configure is like this:

<subsystem xmlns="urn:jboss:domain:infinispan:1.4">
   ....
   <cache-container name="switchyard" default-cache="default" start="EAGER">
       <transport lock-timeout="60000"/>
       <replicated-cache name="default" mode="SYNC" start="EAGER" batching="true">
           <locking isolation="REPEATABLE_READ"/>
       </replicated-cache>
   </cache-container>
   <cache-container name="mycustomcache" default-cache="cacheA" start="EAGER">
       <transport lock-timeout="60000"/>
       <distributed-cache name="cacheA" l1-lifespan="1000" mode="ASYNC" batching="true">
           <eviction strategy="LRU" max-entries="1000"/>
       </distributed-cache>
       <distributed-cache name="cacheB" l1-lifespan="0" mode="ASYNC" batching="true">
           <eviction strategy="LRU" max-entries="10000"/>
           <file-store/>
       </distributed-cache>
   </cache-container>
</subsystem>

Keep in mind, that you will be required to start the server with a -ha profile to have replication and jgroups started, otherwise you will only have local caches.

Usage

The first thing you need to do in your application is to inject the CacheContainer. As the CacheContainer is registered in JNDI, it can easily be injected as a Resource in the java:jboss/infinispan/container/CACHE_CONTAINER name.

@Resource(lookup = "java:jboss/infinispan/container/switchyard")
private CacheContainer container;

Once you have the cache container, you need the concrete cache for your use. In the example configuration above, there are 2 caches defined (cacheA, cacheB). You can get a reference to the cache through the CacheManager. This can be done once, if you set your component as ApplicationScoped, or every time, or using a Singleton, or any other pattern.

private Cache<String, String> cache;
...
this.cache = this.container.getCache();

And now you can use your cache to store/retrieve information.

cache.put(KEY, value);
cache.putIfAbsent(KEY, value, 10L, TimeUnit.SECONDS);
cache.get(KEY);
cache.remove(KEY);
....

Check out the complete Infinispan documentation or the API.

Remember to check the version of Infinispan for the application server you are using. If FSW, this is 5.2.7.Final.

Check out some sample application.

...using jolokia to monitor/manage SwitchYard

Install jolokia

Get the latest jolokia war file from their website, rename it to jolokia.war and deploy it into the server.

Get a list of all SwitchYard MBeans

All SwitchYard MBeans are registered under the org.switchyard.admin JMX domain name, as per thedocumentation. So we can get a list of what we have:

http://localhost:8080/jolokia/list/org.switchyard.admin

or a description of an MBean:

http://localhost:8080/jolokia/list/org.switchyard.admin/name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding

As it is mentioned on the documentation, there are different types of MBeans:

• Application: Management interface for a SwitchYard application.
• Service: Management interface for a composite service in a SwitchYard application. One MBean is registered per composite service.
• Reference: Management interface for a composite reference in a SwitchYard application. One MBean is registered per composite reference.
• Binding: Management interface for a gateway binding attached to a composite service or reference. One MBean is registered per binding instance on an application’s composite services and references.
• ComponentService: Management interface for a component service in a SwitchYard application. One MBean is registered per component service.
• ComponentReference: Management interface for a component reference in a SwitchYard application. One MBean is registered per component reference.
• Transformer: Management interface for a transformer in a SwitchYard application. One MBean is registered per transformer.
• Validator: Management interface for a validator in a SwitchYard application. One MBean is registered per validator.
• Throttling: Management interface for throttling a service in a SwitchYard application. One ThrottlingMBean is registered per composite service instance.

There are two additional MBean objects, that are superclasses, that define custom behavior.

• Lifecycle: Supertype of BindingMXBean which provides operations related to lifecycle control for service and reference bindings.
• Metris: Supertype of multiple MBeans providing message metrics information.

Starting/Stopping bindings

As service and reference bindings extends the Lifecycle MXBean, we can start or stop a binding, and know in what state they are:

• Check the state

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/State

• Stop the binding

http://localhost:8080/jolokia/exec/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/start

• Check the state

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/State

• Start the binding

http://localhost:8080/jolokia/exec/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/start

• Check the state

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/State

Geting metrics

If you want to get metrics, it is very simple, the only thing is that you need to know which metrics are worth for you, as every component, composite and binding provides with many metrics. Once you know what information you need, you can use jolokia to get the information, and maybe use that information to feed an ElasticSearch or InfluxDB database, and use Kibana/Graphana to view the information in a graphical way, and explore this information. Also RTGov is available.

• Get all the information available for a binding

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding

• Get the TotalCount for a binding

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22_OrderService_soap_1%22,service=%22%7Burn:switchyard-quickstart:bean-service:0.1.0%7DOrderService%22,type=Binding/TotalCount

Getting metrics from multiple MBeans

You might want to get some metrics for more than one MBean. You can use wildcards for this, and knowing which types of MBeans and information you want is very easy.

http://localhost:8080/jolokia/read/org.switchyard.admin:name=*,service=*,type=Binding/MinProcessingTime

• More complex pattern

http://localhost:8080/jolokia/read/org.switchyard.admin:name=%22*soap*%22,service=*,type=Binding/MinProcessingTime

Search the MBeans you care for

When you have many apps deployed, you might not know which MBeans are there, and their ObjectNames. You can search for them:

http://localhost:8080/jolokia/search/org.switchyard.admin:type=Binding,*

Demo

If you want to test this, I have created a Dockerfile that you can use right away, based on the latest SwitchYard image. It is available here.

You just need to get this file, and build the image:

curl https://raw.githubusercontent.com/jorgemoralespou/fsw-demo/master/monitoring-with-jolokia/Dockerfile -o Dockerfile
docker build --rm -t "switchyard-with-jolokia"

And then run it:

docker run -it --rm -p 8080:8080 -p 9990:9990 switchyard-with-jolokia

...where to bundle SwitchYard application’s dependencies

The one problem, though, that I’ve constantly see is where to package your dependencies. We constantly fail to get the common classes used by many of the applications in a proper place.

Package common classes and model classes

If a class is going to be used by 2 or more SwitchYard applications, this class needs to be placed in a place where both applications will load it using the same classloader.

In a JEE world if the same class is loaded with different classloaders, it is not the same class.

Let’s take for example an application consisting of 3 SwitchYard applications, 2 of them (Sy app 1 and Sy app 2) packaged in an .ear file and another one (Sy app 3) packaged as a jar. These applications use some common classes (JAXB model, Entity beans, utilities, BaseMessageComposers, …​) that are packaged in two jar files, dependency A and dependency B.

If you have a request that initiated through Sy app 1 and calls to Sy app 2, you can safely use any class that is bundled in dependencyA or dependencyB, as long as for dependencyB you have a reference to the appropriate module (This dependency can be deployed either as a dynamic module, if left in the deployments folder, or as a static module, if registered as a module in modules folder). A reference to a module can be specified in the META-INF/MANIFEST file or in jboss-deployment-structure.xml file. (See JBoss documentation for how to do this).

If you have a requets that initiated through Sy app 1 and calls to Sy app 2, you can only use classes that are bundled in dependencyB for request object, otherwise you’ll most probably have a ClassCastException due to the class being loaded with two different class loaders. Any other use of classes from dependency A or B should be safe as long as it doesn’t use any object that is going from Sy app1 to Sy app 2, or viceversa, on the response path.

There are also some bugs in SY 1.1.1 (FSW 6.0) that raises some other similar and related problems, due to classloaders not properly propagated, and the incorrect use of some static code in serialization framework.

...How to setup JBoss EAP with RHEL 7 (systemd linuxes)

JBoss EAP (or Wildfly) has an init.d script that does not play well with systemd startup systems. To configure it, it is as easy as following this simple steps.

1- Create a group and user for the JBoss EAP process (username, uid, gid, and home to your preferences)

groupadd -r jboss -g 1000
useradd -u 1000 -r -g jboss -m -d /opt/jboss -s /sbin/nologin -c "JBoss user" jboss

2- Create the home folder for the user

chown -R jboss:jboss /opt/jboss

3- Create configuration directory for the JBoss EAP instance, create the configuration file for the EAP instance (of course, your values here), and then set appropriate permissions for the used folders.

mkdir /etc/jboss-as

cat > /etc/jboss-as/jboss-as.conf <<EOF
JBOSS_USER=jboss
STARTUP_WAIT=30
SHUTDOWN_WAIT=30
JBOSS_CONSOLE_LOG=/var/log/jboss-as/console.log
JBOSS_HOME=/usr/share/jboss-as/jboss-eap-6.X
EOF

mkdir /var/log/jboss-as
mkdir /var/run/jboss-as
chown -R jboss:jboss /var/log/jboss-as
chown -R jboss:jboss /var/run/jboss-as

4- Create the service file

cat > /etc/systemd/system/jboss-as-standalone.service <<EOF
[Unit]
Description=Jboss Application Server
After=syslog.target network.target

[Service]
Type=forking
ExecStart=/usr/share/jboss-as/bin/init.d/jboss-as-standalone.sh start
ExecStop=/usr/share/jboss-as/bin/init.d/jboss-as-standalone.sh stop

[Install]
WantedBy=multi-user.target
EOF

5- Restart the systemctl daemon, start the service, verify it’s status and enable the service

systemctl daemon-reload
systemctl start jboss-as-standalone.service
systemctl status jboss-as-standalone.service
systemctl enable jboss-as-standalone.service

6- Additionally, if you need to create a firewalld rules for the EAP, do:

cat > /etc/firewalld/services/jboss-as-standalone.xml
<?xml version="1.0" encoding="utf-8"?>
<service version="1.0">
     <short>jboss-as-standalone</short>
     <port port="8080" protocol="tcp"/>
     <port port="8443" protocol="udp"/>
     <port port="8009" protocol="tcp"/>
     <port port="4447" protocol="tcp"/>
     <port port="9990" protocol="udp"/>
     <port port="9999" protocol="tcp"/>
</service>
EOF

firewall-cmd --zone=public --add-service=jboss-as-standalone
firewall-cmd --permanent --zone=public --add-service=jboss-as-standalone
firewall-cmd --zone=public --list-services
firewall-cmd --permanent --zone=public --list-services

...intro to apiman

I’ll be introducing apiman in a series of blogs, and I’ll try to provide insight into the usually not very well documented parts, like deploying, scaling and providing HA. But for this purpose, I need to provide some details about apiman, the concepts, and the basic architecture.

Description of API Management

I will not write what is already written, so a good description is here provided:

API management is the process of publishing, promoting and overseeing application programming interfaces (APIs) in a secure, scalable environment. It also includes the creation of end user support resources that define and document the API.

The goal of API management is to allow an organization that publishes an API to monitor the interface’s lifecycle and make sure the needs of developers and applications using the API are being met.

API management software tools typically provide the following functions:

1-Automate and control connections between an API and the applications that use it.

2-Ensure consistency between multiple API implementations and versions.

3-Monitor traffic from individual apps.

4-Provide memory management and caching mechanisms to improve application performance.

5-Protect the API from misuse by wrapping it in security procedures and policies.

— http://searchcloudapplications.techtarget.com/definition/API-management

Concepts

Although, there is a very nice and short video that can be easily seen introducing the apiman concepts, I will provide here with a human readable description of the core concepts.

• Organization: Anyone that wants to expose their APIs. The owner of the APIs.
• Service: Usually, the APIs to be exposed. A service represents an external API that is being governed by the API management system.
• Application: An application represents a consumer of an API. Typical API consumers are things like mobile applications and B2B applications.
• Plan: conditions/contraints that govern all the services for an organization, a concrete service or an application. These can be rate-limiting, security, white/black listing, caching,…​ A plan is a set of policies that define a level of service.
• Service contract: A service contract is simply a link between an application and a service through a plan offered by that service. This is the only way that an application can consume a service.

There is much more to it, that can be read in the docs. Although a work in progress, the concepts are already written.

Use cases

API Management can be used in different scenarios.

Within an organization (On premise)

An Organization wants to have control of their APIs, and can use an API management solution to expose and control these APIs consumption, whether internally or externally. In these cases, the internal department in of the APIs will be the Organization, in apiman terms.

In the cloud

An Organization wants to expose their APIs, and they rely on a public cloud API Management solution that will provide with all the requirements to control/govern/monitor/monetize the APIs being exposed. In these cases, the apiman Organization will map to this Organization.

Basic architecture

apiman is composed of 4 architectural pieces:

• APIManager UI: User interface layer for API Management. It is only a frontend, and makes REST calls to the APIManager backend which is the one having the information.
• APIManager backend: It exposes a set of REST interfaces for managing the APIs. It is the one holding the API Management information (data model). Every time an API has to be published, it communicates via REST with the APIGAteway Config.
• APIGateway Config: It is the layer for managing an APIGateway. It exposes a REST interface for management, and stores the management/configuration state.
• APIGateway Runtime: The APIGateway Runtime will be primary runtime component of apiman, and will proxy requests from consumers to the services, applying a set of policies for delivering the service. This is the endpoint that will be known by applications. This will be the single source of requests to the services (APIs).

There will be different implementations for the APIGateway, in order to fit different architectures and technologies. Currently available are:

• Undertow
• Vertx
• Servlet
• Wildfly8 War

Consuming services

Whenever apiman exposes a private service, it will hide the real endpoint, and uses a apikey for the application to identify the service contract. The enpoint used in a private service will be:

http://gatewayhost:port/apiman-gateway/{organizationid}/{serviceid}/{version}/

Anything after that will get passed through to the service impl.

So if you created a service named MyOrg / MyService version 1.0, and set the service implementation endpoint to http://myhost:8080/myservice Then your client would make managed calls to this endponit:

http://gatewayhost:port/apiman-gateway/MyOrg/MyService/1.0/path/to/resource?query=12345

And the gateway would proxy that request to the back service at this endpoint:

http://myhost:8080/myservice/path/to/resource?query=12345

Requests to managed endpoints must include the API Key so that the Gateway knows which Contract is being used to invoke the Service. The API Key can be sent in one of the following ways:

• As an HTTP Header named X-API-Key
• As a URL query parameter named apikey

All HTTP headers and all query parameters (except for the API Key) will also be proxied to the back-end service.