WSO2 ESB Internals - Outgoing Message Path

0 Comments
WSO2 ESB supports many transports such as HTTP, JMS, FIX, VFS etc. HTTP protocol is one of main transports used (maybe most used).

Every message flow does not want to touch the payload of the message.  Some messages flows which includes message manipulation etc need that but a in a great deal  of scenarios there is no such requirement. WSO2 ESB has a special transport called Passthrough HTTP transport which intelligently decides whether to build the message or not  depending on the mediators involved and share internal read/write buffer to speed up message writing out while it is being read.

Following post discusses about code path followed when a HTTP message is sent out using "send mediator" of ESB.

Threads inside ESB

There is a several thread pools inside ESB to do message mediation and sending. Out of these following are two important thread groups. 

1. IO Reactor Threads - these reads messages coming into ESB from socket buffers. They do not have heavy jobs, just message dispatching is done. At message sender side also these involve to send messages out (to back-ends, back to client). 

2. ServerWorker Threads - these threads do message mediation inside ESB


SourceHandler and TargetHandler

These two classes are the implementations of NHTTPServerEventHandler and NHTTPClientEventHandler interfaces provided by HTTPcore library to handle HTTP events. Both are implemented as a part of passThrough HTTP transport.

1. SourceHandler - impl of NHTTPServerEventHandler - (server handler - for NHTTP transport impl)
2. TargetHandler - impl of NHTTPClientEventHandler - (client handler - for NHTTP transport impl)

Both classes maintain a state model while receiving events related to messages from HTTP core (REQUEST_READY, REQUEST_DONE etc). SourceHandler has a container called source context to gather events belonging to a particular message and similarly TargetHandler uses target context.


Debugging these two handlers are the entry point where message is received/sent out by ESB


Code path when sending a message out


Done by server Worker thread. 

  • When "send mediator" is used, FlexibleMEPClient send out the message from synapse layer down to Axis2 layer. 
  • This call lands onto the HTTP sender mentioned in axis2.xml - by defalut PasthrougHTTPSender. It has invike() method. It is hit. 
  • PasthrougHTTPSender passes the messageContext to deliveryAgent to send message. 
  • Message is added to a queue here until a connection is free to send this up. 

    /**
     * This Map holds the messages that need to be delivered. But at the moment maximum
     * number of connections to the host:pair is being used. So these messages has to wait
     * until a new connection is available.
     */
    private Map<HttpRoute, Queue<MessageContext>> waitingMessages =
            new ConcurrentHashMap<HttpRoute, Queue<MessageContext>>();

serverHandler Thread returns here to pick up next one. 

msgContext.getOperationContext().setProperty(
                Constants.RESPONSE_WRITTEN, Constants.VALUE_TRUE);
return InvocationResponse.CONTINUE;



Done by IO Reactor threads 



  • Above messages in waiting queue is written to the wire using IO Reactoer threads. Target handler’s outputReady method is called. 
public void outputReady(NHttpClientConnection conn, ContentEncoder encoder) {…} 



  • Get a connection from pool otherwise IORactor(host,port) thread ends here. There is no limit to num of connections here. If queue is flushed suddenly here BE rate can go high.
  • Then CONNECT is called on newly created connection. 
  • When response is coming back TargetHandler's following method is hit. 
public void responseReceived(NHttpClientConnection conn) {..}


PS:

While debug saw these two issues. This is due to callback timeout when response lands to ESB. 

2017-05-31 23:23:10,131]  WARN - TargetHandler http-outgoing-1: Connection time out while in state: REQUEST_DONE
[2017-05-31 23:23:10,172]  WARN - SynapseCallbackReceiver Synapse received a response for the request with message Id : urn:uuid:743bea80-fd1e-47ba-9bef-ab2d518c5fa8 But a callback is not registered (anymore) to process this response 

Hasitha Hiranya

No comments:

Post a Comment

Instagram