What is Broker fail-over?
High availability (HA) is an important aspect associated with any server implementation that ensures a certain degree of operational continuity when unplanned downtime events impact parts of a system. JMS (Java Message Service) 1.1 does not define such instead most of the time they are vendor specific.The idea behind fail-over is stopping single-point-of-failure in a system. As the broker is the middle man storing and forwarding the messages, if that server goes down entire message flowing of the system will go down no matter what other servers and functions are involved. Thus in order to make a robust messaging system it is mandatory to have a fail-over mechanism.
That is there are a few instances of Message Broker servers up and running in the system, and system generally uses one broker. If that broker goes down it automatically switch to the second broker and continue messaging. If the second fails it will try next and so on. Thus as a whole system will not have a downtime.
Fail-over scenario with WSO2 Message Broker and WSO2 ESB
Setting Up MB Instances In a Single Machine
- Download MB 2.1.0 from here and extract it into a folder.
- Make exact copy of MB to three places and rename them as MB1, MB2 and MB3. Those will be 3 separate MB instances.
- Navigate to MB1/repository/conf/carbon.xml file and define an offset 1.
<Ports> <!-- Ports offset. This entry will set the value of the ports defined below to the define value + Offset. e.g. Offset=2 and HTTPS port=9443 will set the effective HTTPS port to 9445 --> <Offset>0</Offset> - Navigate to MB2/repository/conf/carbon.xml file and define an offset 2 as in the previous step.
- Navigate to MB3/repository/conf/carbon.xml file and define an offset 3 as in the previous step.
- Now our 3 MB instances are ready. Go to /bin folder of each MBx folder and start servers by typing in console wso2sever.sh (or wso2server.bat in Windows).
- Above instances will start on ports 5673,5674 and 5675 respectively.
Setting Up MB Instances In different Machines
- Download MB 2.1.0 from here and extract it into a folder.
- Make exact copy of that folder in three machines. We will say those are MB1 in machine with IP1, MB2 in machine with IP2 and MB3 in machine with IP3.
- Start all above instances by navigating to /bin folder and typing in console wso2sever.sh (or wso2server.bat in Windows). All severs will start with port offset 0, which is 5672.
Setting Up WSO2 ESB Instance
We will start ESB in default port (port offset 0). We can do that as we started all other servers with different port offsets in the singe machine setup. If you are using different machines use another machine to start WSO2 ESB.
- For this you need to follow the WSO2 article on "Configuring WSO2 ESB with WSO2 Message Broker".
- There is only a single difference to enable fail-over across the three brokers we have setup. That is when specifying
connectionfactory.ConnectionFactory
connectionfactory.QueueConnectionFactory
connectionfactory.TopicConnectionFactory
- For an example, if you hope to use single machine MB setup described above
connectionfactory.QueueConnectionFactory = amqp://admin:admin@clientID/carbon?failover='roundrobin'&cyclecount='2'&brokerlist='tcp://localhost:5673 retries='5'&connectdelay='50';tcp://localhost:5674?retries='5'&connectdelay='50';tcp://localhost:5675?retries='5'&connectdelay='50''
- If you hope to use the broker setup made across several machines as described above
connectionfactory.QueueConnectionFactory = amqp://admin:admin@clientID/carbon?failover='roundrobin'&cyclecount='2'&brokerlist='tcp://IP1:5672 retries='5'&connectdelay='50';tcp://IP2:5672?retries='5'&connectdelay='50';tcp://IP3:5672?retries='5'&connectdelay='50''
- We will look into the parameters in the connection url above.
Brokerlist option
brokerlist='<broker url>[;<broker url>]'
The broker list defines the various brokers that can be used for this connection. A minimum of one broker url is required additional URLs are semi-colon(';') delimited.
Broker URL format
<transport>://<host>[:<port>][?<option>='<value>'[&<option>='<value>']]
| Option | Default | Description |
|---|---|---|
| retries | 1 | The number of times to retry connection to this Broker |
| ssl | false | Use ssl on the connection |
| connecttimeout | 30000 | How long in (milliseconds) to wait for the connection to succeed |
| connectdelay | none | How long in (milliseconds) to wait before attempting to reconnect |
Brokerlist failover option
failover='<method>[?<options>]'
| Method | Description |
|---|---|
| singlebroker | This will only use the first broker in the list. |
| roundrobin | This method tries each broker in turn. |
| nofailover | [New in 0.5] This method disables all retry and failover logic. |
The current defaults are naturally to use the 'singlebroker' when only one broker is present and the 'roundrobin' method with multiple brokers. The '''method''' value in the URL may also be any valid class on the classpath that implements the FailoverMethod interface.
Options
| Option | Default | Description |
|---|---|---|
| cyclecount | 1 | The number of times to loop through the list of available brokers before failure. |
Testing setup
- When sending messages (you can configure SOAP UI to send messages continuously one after the other) to a queue using setup described in above article, kill MB1. You will notice fail-over happens at ESB to the second broker i.e MB2. Logs will be there at ESB side indicating fail-over occurred.
- If some previous messages were there at MB1, they will not be delivered.
Most of the time solution to the above and many other problems is broker clustering. Read "Why Broker Fail-over is not Enough? Broker Clustering With WSO2 Message Broker" to know the advantages of clustering and WSO2 Message Broker's support on clustering.
