Wednesday, July 18, 2012

Configuring BPEL SE Clustering on Non-Clustered Glassfish

Originally posted on:

[Updated July 1, 09: Note: The following solution is not supported in Glassfish ESB and is merely suggested as workaround for testing/verification purposes only. Please check release notes for Glassfish ESB for support of cluster profile. This workaround may not work in future releases.]

Glassfish comes with Cluster Support (Refer this page for instructions for installing Glassfish Cluster).

Multiple BPEL SE's running on standalone or clustered Glassfish can be configured to behave as if participating in cluster by configuring some system properties. All the cluster related support of BPEL SE, such as Fail-over, and correlation support will be available in this mode.

Following two System properties need to be configured:
NOTE: The engine id need to be unique to each BPEL SE.

Detailed Instructions for environment setup:

- Refer the Resources Section for information on where to download the bits. The following was tested with Glassfish V2 UR2 b04 Build and Open JBI Components Build as of June 27, 2008.
- The following discussion is regarding standalone glassfish. Similar configuration is required for BPEL SE on clustered Glassfish.

First install Glassfish on separate boxes. Start the database. The following discussion assumes Derby for which BPEL SE automatically creates the data sources and connection pool and also installs the BPEL SE Schema. For oracle, you have to do this step manually.
Install BPEL SE on each one of those. Run the persistence on; verify that there is one entry in the database in the ENGINE Table for this engine. The engine id should be the default value - engine1.
Now, through Netbeans services tab (you can also edit the domain.xml directly) - expand the glassfish node and right click on JVM's. Then click JVMOptions and open the JVM Options editor as shown in Figure 1 below. Here add two properties as highlighted - [com.sun.jbi.bpelse.isClustered=true] and [com.sun.jbi.bpelse.isClustered.engineId=<engineid>]. Note that the engine ids should be unique for each Glassfish instance.
Restart the engine. Check the ENGINE table. Now you should see one entry with the engine id as supplied in the System Property (previous step). Refer Figure 2 below. Also, you can verify the server log, this should say - [Engine Installed on Application Server Cluster : false] and [Engine Configured to run in Cluster Profile : true] (Figure 3).
Refresh the Table after around 40 seconds, you should notice that the engine id (system property passed) LASTUPDATETIME should update. Refer figure 4.
Repeat steps 4 and  5 for each Glassfish stand alone instance.
Now you need to point all the BPEL SEs to point to the same schema. You can do this by changing the Connection pool to point to common database. Refer Figure 5, which shows that you just need to change the host name. Make sure to ping to verify the connection. No change is needed for the data source. Do this for all Glassfish instances.
Now shutdown and restart the BPEL SE for all Glassfish instances where the connection pool info was changed. Verify the ENGINE table that it now contains entries for all the engines. Also, check that the LASTUPDATETIME is also updating. Refer Figure 6.
Deploying and Testing Sample Project

Deploy the Sample Project #1 available on the wiki page on each Glassfish (Note: Also attached in Resources section below).
You can configure the load balancer to feed messages to the HTTP BCs. Refer this page for details on how to configure HTTP Load Balancer. You can also reference this page for additional tips for configuring load balancer. You can also feed messages directly to HTTP BC running on Glassfish instance. See the resources section (at the end of this entry) to download two Soap UI Projects created for feeding messages. Make sure that you point to correct HTTP BC port and support the server info (refer this page for details)
Feed one message to one of the server. Verify that instance exists in the STATE table. Wait 30 seconds and verify that the instance is shown as DONE in the STATE table. Also, verify that output is produced at C:\\temp location. Refer figure 7.
Repeat the above step and now shutdown the engine (you can also shutdown Glassfish). Notice that the instance is completed by other engine. Refer Figure 8.
Start the stopped engine in previous step.
Figures 9 and 10 shows where Ten messages are fed to each engine and one of the engines is shutdown. All the instances of the shutdown engine is picked and completed by live engine. Besides demonstrating Failover, this step also demonstrates that when the stopped engine is brought back to service, it continues to process new requests.
Figure 11 indicates the error message that will be shown to the user if the user fails to supply engine id as system property, if only the cluster override property is set.

Open ESB Installer (Contains Glassfish):
Open JBI Components Latest Build (for Shared Libraries, BPEL SE, HTTP BC and File BC):
Sample Project: Click to download from here
Soap UI Projects: Project 1 and Project 2

No comments: