Wednesday, 14 November 2012

Setting up a Jboss 7.1 Cluster

Sadly, setting up clustering in jboss 7.1.1 is not as straight forward as it should be.  After hours of googling and numerous false starts, I finally got it all working.  Here are some of the gotcha's which tripped me up.

My starting assumption is that we're running a domain.  Out of the box, the Master node defines 3 servers:
 Server-1    - main-server-group
 Server-2    - other-server-group
 Server-3    - main-server-group

Server-1 and Server-2 are configured to start automatically.  In a cluster these are of no use - disable them.  We're only interested in Server-3 which uses the full-ha profile.

Before we go on, a quick heads up:

  • You only need to amend the domain.xml on the master node
Some helpfull links:

Gotcha #1

When you start up the nodes using the domain.bat scripts, always provide the bind ip and ports.  If you don't then you seem to get some strange behaviours: -bmanagement     (this is the master)
and -bmanagement

Gotcha #2

Server-3 does not start cleanly out of the box, due to a mismatch between the full-ha progile and the socket bindings.  In domain.xml, amend as follows:

<server-group name="other-server-group" profile="full-ha">
  <jvm name="default"
    <heap size="64m" max-size="512m"/>
  <socket-binding-group ref="full-ha-sockets"/>

This will allow Server-2 to start cleanly.

Gotcha #3

The hornetq-server needs configuring with a user name and password.  If you don't you will get an exception along the lines of:

[Server:server-three] 16:24:36,875 ERROR [org.hornetq.core.protocol.core.impl.Ho
rnetQPacketHandler] (Old I/O server worker (parentId: 2128830893, [id: 0x7ee361a
d, /])) Failed to create session : HornetQException[errorCode=1
05 message=Unable to validate user: HORNETQ.CLUSTER.ADMIN.USER]

Update domain.xml with:
<subsystem xmlns="urn:jboss:domain:messaging:1.1">

It looks as though you can use any user and password.  It doesn't appear to be authenticated against anything!

Gotcha #4

Use the correct version of @Clustered annotation!   If you don't then in a failover situation you will get the "No EJBReceiver" exception.  The version of @Clustered your'e after is org.jboss.ejb3.annotation.Clustered.  This comes from the jboss-ejb3-ext-api dependency, which is only available from the jboss repo.  Assuming that you are using maven add the following to the pom.xml:



    <id>JBoss repository</id>

Then correctly annotate the bean:

import org.jboss.ejb3.annotation.Clustered;

@Remote( coder36.demo.service.MyService.class )
public class MyServiceImpl implements MyService {

Gotcha #5

When injecting the EJB do not use the @Inject annotation!   Use @EJB on it's own.  No JNDI name is required:

public class PersonController implements Serializable {
  private MyService myService;

Note - the @EJB annotation is available via the following dependency:


Gotcha #6

To make the web application cluster-able ensure that webapp/WEB-INF/web.xml exists, and has the <distributable/> tag:

<web-app xmlns=""


To be extra helpful, I've provided the web.xml namespaces :)

Gotcha #7

Assuming you have downloaded and installed mod_cluster 1,2.Final from here.

Out of the box, mod_cluster does't  not work with jboss 7.1.1. A few tweaks are needed.  Keeping things as simple as possible, I've stripped the provided httpd.conf down the bare minimum:

LoadModule authz_host_module modules/
LoadModule proxy_module modules/
LoadModule proxy_ajp_module modules/
LoadModule proxy_http_module modules/
LoadModule proxy_cluster_module modules/
LoadModule manager_module modules/
LoadModule slotmem_module modules/
LoadModule advertise_module modules/


  <Location />
    Order deny,allow
    Deny from all
    Allow from all

  KeepAliveTimeout 300
  MaxKeepAliveRequests 0


  <Location /mod_cluster_manager>
    SetHandler mod_cluster-manager
    Order deny,allow
    Deny from all
    Allow from all
ErrorLog "logs/error_log"

I used port 10001 as the default port 6666 didn't work for me.  This httpd.conf should be enough to get you going.

The domain.xml also needs updating to link it up with the mod_cluster instance:

<subsystem xmlns="urn:jboss:domain:modcluster:1.0">
  <mod-cluster-config advertise-socket="modcluster" 


<subsystem xmlns="urn:jboss:domain:web:1.1" 


To test everything is working, fire up the domain, deploy an application and start httpd (on windows 7 I needed to run it as administrator otherwise it complained about file permissions).  Navigate to:  You should see:

The web application can be now accessed through the mod_cluster proxy sitting on and


Monday, 12 November 2012

How to set up a Jboss 7.1 domain

With the new JBOSS 7.1 domain controller, a cluster of jboss instances can be administered via a single console.  In this tutorial we will step through the simple process of setting up a domain controller, and 3 nodes.

Target Architecture:

Simplistically we are aiming for the following architecture:

I've assumed IP addresses, passwords and names to keep thing simple.

Later on we will need base64 version of the identity.  I used this website to generate the following base64 strings:

Identity     Base64
node1id      bm9kZTFpZA==
node2id      bm9kZTJpZA==
node3id      bm9kZTNpZA==

1) Configure the ManagementRealm

The nodes authenticates with the domain controller (DC) by passing it's node name and password.  The DC does a name/password lookup against the ManagementRealm.

so... the first step is to configure the ManagementRealm.

On the domain controller run:
add-user node1    (when prompted set password to node1id)
add-user node2    (when prompted set password to node1id)
add-user root      (when prompted set password to password)

(The root user is used for logging into the jboss console.)

Start up the domain controller:
domain -b -bmanagement

2) Configure Node 1

Open host.xml (under jboss-as-7.1.1.Final\domain\configuration) and edit as follows:

a) Update <host tag to read:
<host name="node1" xmlns="urn:jboss:domain:1.2">

b) Add server-identities section to <security-realm name="ManagementRealm">

<security-realm name="ManagementRealm">
    <secret value="bm9kZTFpZA=="/>


The secret value is the base64 encoded version of the Node password.

c) Edit the <domain-controller> section to read:


  <remote host="${jboss.domain.master.address:}"

Don't forget to add the the security-realm attribute otherwise you will get messages like:

[Host Controller] 16:26:06,454 ERROR [org.jboss.remoting.remote.connection] (Rem
oting "host1:MANAGEMENT" read-1) JBREM000200: Remote connection failed:
curity.sasl.SaslException: Authentication failed: all available authentication m
echanisms failed
[Host Controller] 16:26:06,469 ERROR [] (Controller
Boot Thread) JBAS010901: Could not connect to master. Aborting. Error was: java.
lang.IllegalStateException: JBAS010942: Unable to connect due to authentication

d) Start up the node
domain.bat -b -bmanagment

3) Configure Node 2

Configure as above,  setting <host name="node2" and <secret value="bm9kZTJpZA=="

4) Configure Node 3

Configure as above,  setting <host name="node3" and <secret value="bm9kZTNpZA=="


If everything is working, then open a browser and navigate to: using root/password when prompted.  On the left hand side, there should be a list of nodes.

Friday, 9 November 2012

How to enable security logging in Jboss 7

To view what's happening within the jboss 7 security subsystem, you need to enable logging of

Assuming standalone.xml, update the <subsystem xmlns="urn:jboss:domain:logging:1.1"> section, adding a new console-handler which deals with TRACE level messages only:

<console-handler name="CONSOLE_TRC">
  <level name="TRACE"/>
    <pattern-formatter pattern="%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n"/>

and link it up to a new org.jboss.securty logger:

<logger category="">
  <level name="TRACE"/>
    <handler name="CONSOLE_TRC"/>

Finally, restart the jboss server :)

Thursday, 8 November 2012

Securing JBoss 7 management console using SSL

This tutorial describes how to secure the JBOSS 7 web management console and the CLI management interface.   For clarity, it assumes you're running a stand alone jboss server.

Create a java keystore

Start by creating a java keystore with a self signed certificate:
mkdir ssl
cd ssl
keytool -genkey -keyalg RSA -alias mycert -keystore keystore.jks -storepass   password -keypass password -validity 360 -keysize 2048 -dname ", O=Coder36 L=Newcastle, S=England, C=GB"

The keystore password is: password and the private key password is: password

View the certificate using:
keytool -list -v -keystore keystore.jks -alias mycert -storepass password

c:/ssl/keystore.jks will now  contain your private key and a self signed certificate.

Enable SSL

The next step is to enable SSL on the management interfaces.  Edit standalone.xml.  Search for the <management> tags and update.  I've highlighted what's changed from out of the box:

    <security-realm name="ManagementRealm">
        <ssl protocol="TLSv1">
          <keystore path="C:/ssl/keystore.jks" password="password"/>
        <properties path="" relative-to="jboss.server.config.dir"/>
    <security-realm name="ApplicationRealm">
        <properties path="" relative-to="jboss.server.config.dir"/>
    <native-interface security-realm="ManagementRealm">
      <socket-binding native="management-native"/>
    <http-interface security-realm="ManagementRealm">
      <socket-binding https="management-https"/>


Restart the JBoss standalone server and test:
Open a web and navigate to: https://localhost:9443/console. You will be presented with the usual warnings about using untrusted certificates   Interestingly, the https console does not work with Google Chrome, but works fine with Internet Explorer, and what ever is built into eclipse.

Test the CLI using: -c

Using this solution we can prove that the jboss server we are calling is who we think we are calling.  It's also more secure in that it encrypts traffic to and from the server.


With a few changes, it straight forward to get the management console protected with SSL.

I'm of the opinion that java key stores are a bit clunky and unfriendly.  I much prefer the .pem format with openssl etc.  From my commercial experience, it's generally a bad idea to use java to do the SSL work.  Architecturally it would be better to offload the SSL termination to dedicated hardware (CISCO provides ACE modules to do this kind of work), or dedicated software like apache - see one of my earlier posts for how this could be done.


Saturday, 3 November 2012

TVHeadend on the Raspberry PI


Tvheadend is a linux service which converts a dvb-t feed into a network stream.  This  enables you to stream live tv feed from your raspberry pi to any device on your LAN.  The aim of this tutorial to set up a Tvheadend streaming server on the Raspberry PI, then watch it via an XBMC client on the Nexus 7.

Pre-requisites - Hardware

1) Raspberry PI  (I've got the newer 512MB version)
2) Powered USB hub (New Link 4 Port USB Hub with Mains Adaptor from modmypi )
3) USB DVB-T Tuner (Hauppauge WinTV Nova-TD)
4) USB WIFI adapter (Edimax EW-7811UN Wireless)
5) Memory card (Lexar class 10 32GB micro SD inside an SD card adapter)
6) Keuboard and mouse (I've got a wireless "MINI ULTRA SLIM DESIGN LAPTOP USB KEYBOARD MOUSE COMBO" from here)

Pre-requisistes OS

The operating system of choice is Raspbian (Wheezy), available from here.  Expand the root partition to fill up the SD card, otherwise you''ll run out of space.


so lets begin....

sudo apt-get update

sudo apt-get install libcurl4-openssl-dev git

Set up the NOVA-TD DVB-T TV tuner

Plug in the tv-tuner then run

You will see something along the lines of:

[ 1138.657011] usb 1-1.3.2: new high-speed USB device number 7 using dwc_otg
[ 1138.758550] usb 1-1.3.2: New USB device found, idVendor=2040, idProduct=9580
[ 1138.758581] usb 1-1.3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1138.758599] usb 1-1.3.2: Product: NovaT 500Stick
[ 1138.758611] usb 1-1.3.2: Manufacturer: Hauppauge
[ 1138.758624] usb 1-1.3.2: SerialNumber: 4027809413
[ 1138.865068] IR NEC protocol handler initialized
[ 1138.896509] IR RC5(x) protocol handler initialized
[ 1138.929688] IR RC6 protocol handler initialized
[ 1138.957795] IR JVC protocol handler initialized
[ 1138.968073] dib0700: loaded with support for 21 different device-types
[ 1138.968456] dvb-usb: found a 'Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity' in cold state, will try to load a firmware
[ 1138.991871] IR Sony protocol handler initialized
[ 1139.025138] IR MCE Keyboard/mouse protocol handler initialized
[ 1139.026512] dvb-usb: did not find the firmware file. (dvb-usb-dib0700-1.20.fw) Please see linux/Documentation/dvb/ for more details on firmware-problems. (-2)
[ 1139.026723] usbcore: registered new interface driver dvb_usb_dib0700
[ 1139.057895] lirc_dev: IR Remote Control driver registered, major 251
[ 1139.062582] IR LIRC bridge handler initialized

The problem here is that the dvb-usb module failed to find the nova-td firmware file (dvb-usb-dib0700-1.20.fw)  A quick google reveals that the best place to find tv tuner firmware for linux is here

cd ~
sudo cp dvb-usb-dib0700-1.20.fw /lib/firmware

or more elegantly ...

sudo apt-get install firmware-linux-nonfree

Pull out the tv-tuner, and push back in.


[ 1660.721801] usb 1-1.3.2: USB disconnect, device number 7

[ 1663.006468] usb 1-1.3.2: new high-speed USB device number 8 using dwc_otg

[ 1663.107981] usb 1-1.3.2: New USB device found, idVendor=2040, idProduct=9580
[ 1663.108013] usb 1-1.3.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1663.108029] usb 1-1.3.2: Product: NovaT 500Stick
[ 1663.108042] usb 1-1.3.2: Manufacturer: Hauppauge
[ 1663.108054] usb 1-1.3.2: SerialNumber: 4027809413
[ 1663.118347] dvb-usb: found a 'Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity' in cold state, will try to load a firmware
[ 1663.173435] dvb-usb: downloading firmware from file 'dvb-usb-dib0700-1.20.fw'
[ 1663.378063] dib0700: firmware started successfully.
[ 1663.886683] dvb-usb: found a 'Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity' in warm state.
[ 1663.889868] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.
[ 1663.890139] DVB: registering new adapter (Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity)
[ 1664.159908] DVB: registering adapter 0 frontend 0 (DiBcom 7000PC)...
[ 1664.201931] MT2266: successfully identified
[ 1664.373180] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.
[ 1664.373438] DVB: registering new adapter (Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity)
[ 1664.552728] DVB: registering adapter 1 frontend 0 (DiBcom 7000PC)...
[ 1664.557289] MT2266: successfully identified
[ 1664.766474] Registered IR keymap rc-dib0700-rc5
[ 1664.767378] input: IR-receiver inside an USB DVB receiver as /devices/platform/bcm2708_usb/usb1/1-1/1-1.3/1-1.3.2/rc/rc0/input3
[ 1664.767916] rc0: IR-receiver inside an USB DVB receiver as /devices/platform/bcm2708_usb/usb1/1-1/1-1.3/1-1.3.2/rc/rc0
[ 1664.769460] dvb-usb: schedule remote query interval to 50 msecs.
[ 1664.769497] dvb-usb: Hauppauge Nova-TD Stick/Elgato Eye-TV Diversity successfully initialized and connected.

That was easy enough!  This could easily have been a nightmare, but by the looks of it Raspbian has the correct kernel modules loaded.   Now that the tv tuner has been detected and initialised we can proceed with building tvheadend. 

Building tvheadend from source

cd ~
git clone -b release/3.2
cd tvheadend
sudo make install

This will install the tvheadend binary: /usr/local/bin/tvheadend
This targets release 3.2, but by leaving off -b release/3.2 you can target the bleeding edge!

Make it auto start at boot

Create tvheadend user home and group

sudo groupadd tvheadend
sudo useradd -g tvheadend -G video -m tvheadend

Create file named /etc/init.d/tvheadend



case "$1" in
    echo "Starting tvheadend"
    start-stop-daemon --start --user ${TVHUSER} --exec ${TVHBIN} -- \
                -u ${TVHUSER} -g ${TVHGROUP} -f -C
    echo "Stopping tvheadend"
    start-stop-daemon --stop --quiet --name ${TVHNAME} --signal 2
    echo "Restarting tvheadend"

    start-stop-daemon --stop --quiet --name ${TVHNAME} --signal 2

    start-stop-daemon --start --user ${TVHUSER} --exec ${TVHBIN} -- \
                -u ${TVHUSER} -g ${TVHGROUP} -f -C

    echo "Usage: tvheadend {start|stop|restart}"
    exit 1
exit 0

Set the permissions to make it runnable:
sudo chmod 755 /etc/init.d/tvheadend

Enable start during boot process
sudo update-rc.d tvheadend defaults

Finally start it manually so we can configure it:
sudo /etc/init.d/tvheadend start 


Navigate to http://<rasppi_ip>:9981

Configuration -> General : Set the language (and hit 'Save Changes')

Configuration->TV Adapters: Select the Adapter 

Hit Add DVB Network by location and select Generic->auto_Default 

Now wait while the Muxes have been scanned for services.  This will take a long time.  The right hand panel will update as channels (services) are discovered.

Hit Map DVB services to channels (once it becomes enable) and wait.  In the meantime hit the services tab, to see what's been found.  If everything is working as it should, there should be a list of TV and radio channels.

Select Configuration->Channels to view mapped channels

XBMC On the Nexus 7

All that's left to do is to install XBMC on the nexus 7.  The easiest way to do this, is by installing the 'nightlies' from the XDA forums (for the nexus 7 it's the non-neon build you need).  

To connect to TVHeadend, add the "Tvheadend HTSP Client" Addon:
System -> Settings -> Add-ons -> Search  
Type: Tvheadend

Enable and Configureas follows:

Tvheadend hostname or IP address :  <RaspPI+IP>
HTTP port:  9981
HTSP port: 9982

System -> Settings -> Live TV -> General    : set to Enabled

That's it... Go to Live-TV -> TV Channels

The XBMC tvheadend client is not the finished article.   The EPG looks a bit screwed up, as do some of the channel names.  Also not all the channels appear to be present.  Apart from the above niggles, the XBMC client looks amazing and seems to fit the bill.


Available from here.  It's not in the google play store, so you need to  download it and install it by hand. In comparison to XBMC, this seems to be the more finished product although the interface could do with some polish - its fairly basic!  The EPG works, although is a bit industrial.  The configuration is intuitive, and as a bonus it can use an external player like the amazing MXPlayer

Until the bugs in XBMC can be resolved, I'll be sticking with TVHGuide.