Setting up a Jboss 7.1 Cluster

Sadly, setting up clustering in jboss 7.1.1 is not as straight forward as it should be.  After hours of googling and numerous false starts, I finally got it all working.  Here are some of the gotcha's which tripped me up.

My starting assumption is that we're running a domain.  Out of the box, the Master node defines 3 servers:
 Server-1    - main-server-group
 Server-2    - other-server-group
 Server-3    - main-server-group

Server-1 and Server-2 are configured to start automatically.  In a cluster these are of no use - disable them.  We're only interested in Server-3 which uses the full-ha profile.

Before we go on, a quick heads up:

• You only need to amend the domain.xml on the master node

Some helpfull links:

Jboss 7.1.1 Cluster guide

Discussion on "No EJBReceiver error message"

Mod Cluster 1.2 docs

Gotcha #1

When you start up the nodes using the domain.bat scripts, always provide the bind ip and ports.  If you don't then you seem to get some strange behaviours:

domain.sh-b 192.168.0.24 -bmanagement 192.168.0.24     (this is the master)
and
domain.sh-b 192.168.0.25 -bmanagement 192.168.0.25

Gotcha #2

Server-3 does not start cleanly out of the box, due to a mismatch between the full-ha progile and the socket bindings.  In domain.xml, amend as follows:


<server-group name="other-server-group" profile="full-ha">
  <jvm name="default"
    <heap size="64m" max-size="512m"/>
  </jvm>
  <socket-binding-group ref="full-ha-sockets"/>
</server-group>


This will allow Server-2 to start cleanly.

Gotcha #3

The hornetq-server needs configuring with a user name and password.  If you don't you will get an exception along the lines of:

[Server:server-three] 16:24:36,875 ERROR [org.hornetq.core.protocol.core.impl.Ho

rnetQPacketHandler] (Old I/O server worker (parentId: 2128830893, [id: 0x7ee361a

d, /192.168.0.24:5695])) Failed to create session : HornetQException[errorCode=1

05 message=Unable to validate user: HORNETQ.CLUSTER.ADMIN.USER]

Update domain.xml with:
<subsystem xmlns="urn:jboss:domain:messaging:1.1">
  <hornetq-server>                
  <cluster-user>fred</cluster-user>
  <cluster-password>password</cluster-password>


It looks as though you can use any user and password.  It doesn't appear to be authenticated against anything!

Gotcha #4

Use the correct version of @Clustered annotation!   If you don't then in a failover situation you will get the "No EJBReceiver" exception.  The version of @Clustered your'e after is org.jboss.ejb3.annotation.Clustered.  This comes from the jboss-ejb3-ext-api dependency, which is only available from the jboss repo.  Assuming that you are using maven add the following to the pom.xml:

<dependencies>
 ...

  <dependency>
    <groupId>org.jboss.ejb3</groupId>
    <artifactId>jboss-ejb3-ext-api</artifactId>
    <version>2.0.0-beta-3</version>
    <scope>provided</scope>
  </dependency>

<dependencies>
...
<repositories>
  <repository>
    <id>JBoss repository</id>
    <url>http://repository.jboss.org/nexus/content/groups/public/</url>
  </repository>
<repositories>

Then correctly annotate the bean:

import org.jboss.ejb3.annotation.Clustered;

@Stateful
@Clustered
@Remote( coder36.demo.service.MyService.class )
public class MyServiceImpl implements MyService {
...

Gotcha #5

When injecting the EJB do not use the @Inject annotation!   Use @EJB on it's own.  No JNDI name is required:


@ManagedBean
public class PersonController implements Serializable {
  @EJB
  private MyService myService;


Note - the @EJB annotation is available via the following dependency:


<dependency>
  <groupId>org.jboss.spec.javax.ejb</groupId>
  <artifactId>jboss-ejb-api_3.1_spec</artifactId>
</dependency>

Gotcha #6

To make the web application cluster-able ensure that webapp/WEB-INF/web.xml exists, and has the <distributable/> tag:


<web-app xmlns="http://java.sun.com/xml/ns/javaee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/javaee 
    http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
    version="3.0"> 

  <distributable/> 
</web-app>


To be extra helpful, I've provided the web.xml namespaces :)

Gotcha #7

Assuming you have downloaded and installed mod_cluster 1,2.Final from here.

Out of the box, mod_cluster does't  not work with jboss 7.1.1. A few tweaks are needed.  Keeping things as simple as possible, I've stripped the provided httpd.conf down the bare minimum:


LoadModule authz_host_module modules/mod_authz_host.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
LoadModule manager_module modules/mod_manager.so
LoadModule slotmem_module modules/mod_slotmem.so
LoadModule advertise_module modules/mod_advertise.so

Listen 192.168.0.24:80
Listen 192.168.0.24:10001

<VirtualHost 192.168.0.24:10001>
  <Location />
    Order deny,allow
    Deny from all
    Allow from all
  </Location>

  KeepAliveTimeout 300
  MaxKeepAliveRequests 0

  EnableMCPMReceive

  <Location /mod_cluster_manager>
    SetHandler mod_cluster-manager
    Order deny,allow
    Deny from all
    Allow from all
  </Location>
</VirtualHost>
ErrorLog "logs/error_log"


I used port 10001 as the default port 6666 didn't work for me.  This httpd.conf should be enough to get you going.

The domain.xml also needs updating to link it up with the mod_cluster instance:


<subsystem xmlns="urn:jboss:domain:modcluster:1.0">
  <mod-cluster-config advertise-socket="modcluster" 
                      proxy-list="192.168.0.24:10001">
  ...


and

<subsystem xmlns="urn:jboss:domain:web:1.1" 
            default-virtual-server="default-host" 
            instance-id="${jboss.node.name}" 
            native="false">
...

Testing

To test everything is working, fire up the domain, deploy an application and start httpd (on windows 7 I needed to run it as administrator otherwise it complained about file permissions).  Navigate to:  http://192.168.0.24:10001/mod_cluster_manager.  You should see:

The web application can be now accessed through the mod_cluster proxy sitting on http://192.168.0.24 and http://192.168.0.24:10001:

ie. http://192.168.0.24/demo/index/jsf