Configure Broker Fail-over in a WSO2 Message Broker and WSO2 ESB setup

What is Broker fail-over?

High availability (HA) is an important aspect associated with any server implementation that ensures a certain degree of operational continuity when unplanned downtime events impact parts of a system. JMS (Java Message Service)  1.1 does not define such instead most of the time they are vendor specific.

The idea behind fail-over is stopping single-point-of-failure in a system. As the broker is the middle man storing and forwarding the messages, if that server goes down entire message flowing of the system will go down no matter what other servers and functions are involved. Thus in order to make a robust messaging system it is mandatory to have a fail-over mechanism.

That is there are a few instances of Message Broker servers up and running in the system, and system generally uses one broker. If that broker goes down it automatically switch to the second broker and continue messaging. If the second fails it will try next and so on. Thus as a whole system will not have a downtime.

Fail-over scenario with WSO2 Message Broker and WSO2 ESB

Setting Up MB Instances In a Single Machine

  • Download MB 2.1.0 from here and extract it into a folder. 
  • Make exact copy of MB to three places and rename them as MB1, MB2 and MB3. Those will be 3 separate MB instances. 
  • Navigate to MB1/repository/conf/carbon.xml file and define an offset 1. 
            <!-- Ports offset. This entry will set the value of the ports defined below to
             the define value + Offset.
             e.g. Offset=2 and HTTPS port=9443 will set the effective HTTPS port to 9445
  • Navigate to MB2/repository/conf/carbon.xml file and define an offset 2 as in the previous step. 
  • Navigate to MB3/repository/conf/carbon.xml file and define an offset 3 as in the previous step. 
  • Now our 3 MB instances are ready. Go to /bin folder of each MBx folder and start servers by typing in console (or wso2server.bat in Windows).
  • Above instances will start on ports 5673,5674 and 5675 respectively. 

Note that in the above setup it will start 3 separate MB servers along with Cassandra servers dedicated for each. Those Cassandra servers will run in ports 9160, 9161 and 9162 respectively.

Setting Up MB Instances In different Machines

  • Download MB 2.1.0 from here and extract it into a folder.
  • Make exact copy of that folder in three machines. We will say those are MB1 in machine with IP1, MB2 in machine with IP2 and MB3 in machine with IP3.
  • Start all above instances by navigating to /bin folder and  typing in console (or wso2server.bat in Windows). All severs will start with port offset 0, which is 5672.

Setting Up WSO2 ESB Instance

  We will start ESB in default port (port offset 0). We can do that as we started all other servers with different port offsets in the singe machine setup. If you are using different machines use another machine to start WSO2 ESB.

  • For this you need to follow the WSO2 article on "Configuring WSO2 ESB with WSO2 Message Broker".
  • There is only a single difference to enable fail-over across the three brokers we have setup. That is when specifying 
    • connectionfactory.ConnectionFactory
    • connectionfactory.QueueConnectionFactory
    • connectionfactory.TopicConnectionFactory
  • For an example, if you hope to use single machine MB setup described above

connectionfactory.QueueConnectionFactory = amqp://admin:admin@clientID/carbon?failover='roundrobin'&cyclecount='2'&brokerlist='tcp://localhost:5673 retries='5'&connectdelay='50';tcp://localhost:5674?retries='5'&connectdelay='50';tcp://localhost:5675?retries='5'&connectdelay='50''
  • If you hope to use the broker setup made across several machines as described above
    connectionfactory.QueueConnectionFactory = amqp://admin:admin@clientID/carbon?failover='roundrobin'&cyclecount='2'&brokerlist='tcp://IP1:5672 retries='5'&connectdelay='50';tcp://IP2:5672?retries='5'&connectdelay='50';tcp://IP3:5672?retries='5'&connectdelay='50''
  • We will look into the parameters in the connection url above.

Brokerlist option

brokerlist='<broker url>[;<broker url>]'
The broker list defines the various brokers that can be used for this connection. A minimum of one broker url is required additional URLs are semi-colon(';') delimited.

Broker URL format


retries1The number of times to retry connection to this Broker
sslfalseUse ssl on the connection
connecttimeout30000How long in (milliseconds) to wait for the connection to succeed
connectdelaynoneHow long in (milliseconds) to wait before attempting to reconnect

Brokerlist failover option


singlebrokerThis will only use the first broker in the list.
roundrobinThis method tries each broker in turn.
nofailover[New in 0.5] This method disables all retry and failover logic.

The current defaults are naturally to use the 'singlebroker' when only one broker is present and the 'roundrobin' method with multiple brokers. The '''method''' value in the URL may also be any valid class on the classpath that implements the FailoverMethod interface.

cyclecount1The number of times to loop through the list of available brokers before failure.

Testing setup

  • When sending messages (you can configure SOAP UI to send messages continuously one after the other) to a queue using setup described in above article, kill MB1. You will notice fail-over happens at ESB to the second broker i.e MB2. Logs will be there at ESB side indicating fail-over occurred. 
  • If some previous messages were there at MB1, they will not be delivered. 

Most of the time solution to the above and many other problems is broker clustering. Read "Why Broker Fail-over is not Enough? Broker Clustering With WSO2 Message Broker" to know the advantages of clustering and WSO2 Message Broker's support on clustering. 

Hasitha Hiranya

No comments:

Post a Comment