AU2010201275A1

AU2010201275A1 - A high performance and low latency messaging system

Info

Publication number: AU2010201275A1
Application number: AU2010201275A
Authority: AU
Inventors: Rajesh Mansharamani; Manoj Nambiar; Sricharan Samudrala
Original assignee: Tata Consultancy Services Ltd
Current assignee: Tata Consultancy Services Ltd
Priority date: 2009-04-13
Filing date: 2010-03-31
Publication date: 2010-10-28
Also published as: CN101866303B; CN101866303A

Description

TITLE A HIGH PERFORMANCE LOW LATENCY MESSAGING SYSTEM FIELD OF THE INVENTION The present invention relates to the field of communication. Particularly, the present invention relates to the field of communication between two or more distributed computers. BACKGROUND OF THE PRESENT INVENTION In computing, one of the most crucial factors for providing the high speed operation of applications, softwares and instantaneous response of web applications is the messaging system. Use of an efficient messaging system can substantially increase the performance of the software and applications using it and in turn increases over all performance of the computer. Messaging systems basically are used for inter process communication between various processes associated with an application and also processes which are involved in inter operation of various applications. Messaging systems transfer messages containing data to be exchanged between processes thus speeding up computational processing. 1 Various messaging systems used till date have locked queues used for storing messages coming from various processes, and as a result suffer from the drawback of slow operation. US patent no. 6029205 discloses various systems for shared memory inter process communication and message passing techniques between executing processes but does not overcome the problem of locked queues.. US patent application 20050021622 discloses a system for dynamic message routing on a topic between publishing nodes and subscribing nodes includes a plurality of message queues, at least one topic node table, a subscribing module, a publishing module, and other modules to send messages between one or more publisher and one or more subscribers. This document only focuses on the publish subscribe mechanism and is not applicable in case of point to point queues. . This application also does not overcome the problem of locked queues. US patent no. 6029205 discloses a system and method for inter process communication between concurrently executing, cooperating sequential processes using shared memory queue as a mechanism for message passing and process synchronization. The sending process stores the message as a new entry in queue (en-queue operation), which is visible in the virtual address space of a first process, and a receiving process reads the message from the queue and performs the de-queue operation. But there is no specific mention/claim of achieving low latency using the shared memory queue. There is therefore felt a need for a system and method which: 2 * enables multiple processes to communicate simultaneously; * increases the throughput of the messaging system; * is easy to deploy on existing systems; * requires minimum infrastructure; and e provides efficient message passing technique for critical applications. OBJECTS OF THE INVENTION It is an object of the present invention to provide an efficient messaging system and thereby reduce computational time. It is another object of the present invention to provide a cost effective messaging system. It is yet another object of the present invention to provide a system which enables multiple processes to communicate simultaneously. It is still another object of the present invention to provide a system which provides high throughput and low latency. It is still another object of the present invention to provide a system which can be deployed easily with existing systems. It is still another object of the present invention to provide a system that requires minimal infrastructure 3 It is still another object of present invention is to provide an efficient message passing technique for critical applications Further, an object of the present invention to be usable also on hardware & software infrastructure-with poor memory consistency model. SUMMARY OF THE INVENTION The present Invention envisages a system for messaging in inter-process communication between at least two processes running on at least one node, the system comprising: e a storage adapted to store inter process messages; * a queue of storage buffers in a shared memory in the storage concurrently accessible by plurality of processes; * a free pointing element associated with a process, adapted to point to a free storage buffer in the queue; * atleast one data pointing elements associated with a process, adapted to point to a storage buffer containing an inter process message; * message read/write means adapted to insert and fetch messages from the storage buffers in the queue simultaneously without locking the queue by allowing a process to insert messages in the free storage buffers of the queue pointed by the free pointing element and allowing at least one process to read an inserted message stored in a storage buffer containg an inter process message pointed by the data pointing element. 4 Typically, the system comprises updation means adapted to update the position of the free pointing element and the data pointing element. Typically, the shared memory is located on a local node or on a remote node. Also, the processes can be running at remote nodes run the processes which communicate using communicating means adapted to facilitate connections selected from the group consisting of TCP/IP connection, GPRS connection, WiFi connection, WiMax connection, and Edge connection. The read/write means includes a checking means to check if the queue is full for a write/insert operation and check if the queue is empty for read/fetch operation. In accordance with a preferred embodiment of the invention the system the queues are contained in files. Preferably, the files are mapped into the main memory space of the processor. Again in accordance with a preferred embodiment of the invention every storage buffer in a queue is linked to the next storage buffer and the last storage buffer links to the first storage buffer forming a circularly linked list for storing all the incoming messages into the storage buffers one after another. This system is preferably a financial trading system but also can be used for many other applications. 5 In accordance with another aspect of the invention there is provided a method for messaging in inter-process communication between at least two processes running on at least one node, said method comprising the following steps: 9 providing a storage to store inter process messages; * providing at least one queue of storage buffers in a shared memory in said storage; * providing a free pointing element associated with a process to point to a free storage buffer in said queue * providing atleast one data pointing element associated with a process pointing to said storage buffers containing said inter process message; * receiving atleast one inter process message from at least one process running on at least one node; * inserting said received inter process message into a storage buffer of said queue pointed by said free pointing element; e updating the position of free pointing element to next free storage buffer of said queue to accommodate the next inter process message; 9 fetching an interprocess message simultaneously by a plurality of processes from said storage buffer of said queue pointed by their corresponding data pointing elements; e updating the position of said data pointing element to a next storage buffer containg a message to be read by any of said processes. Typically, the step of updating the position of the free pointing element includes the steps of checking if the free pointing element is pointing to a 6 buffer pointed by any one of the data pointing elements, if true then waiting for the buffer to be free else updating the pointer position of free pointing element to next free storage buffer. And the step of fetching an interprocess message includes the steps of checking if the data pointing element is pointing to a buffer pointed by the free pointing element, if true then waiting for data to be inserted into the queue for fetching Typically, the step of inserting said received inter process message includes the step of copying said inserted message to main memory of said node for permanent/secondary storage. BRIEF DESCRIPTION OF ACCOMPANYING DRAWINGS The invention will now be described with reference to the accompanying drawings, in which: Figure 1 illustrates a system for lockless messaging in inter-process communication. Figure 2 illustrates a detailed view of queue which is a cyclically linked list of storage elements. Figure 3 illustrates a method for lockless messaging in inter-process communication. DETAILED DESCRIPTION OF THE INVENTION The prior art does not provide any system and method for lockless message passing of inter process messages on a platform with a poor memory consistency model. Thus, the present invention proposes an efficient system 7 and method for lockless message passing in inter process communication, even on platforms with poor memory consistency model. The messaging system envisaged by the present invention implements efficient inter-process communication between processes of the same server (local messaging) as well as processes on different servers (remote messaging). According to an embodiment of present invention the messaging system for inter process communication is implemented in files. These files contain message queues i.e. data structure storing the inter process messages. These files are mapped into the main memory space of the processor for speed of access. The queue comprises storage buffers and every storage buffer is linked to next storage buffer and the last storage buffer links to first storage buffer forming a circularly linked list for storing all the incoming messages into the storage buffers one after another. Also, the queue has two types of pointing elements i.e. free pointing element and data pointing element. The queue has a free pointing element which always points to a free storage buffer in which a next incoming message can be stored and is always associated with a process which is writing into the queue. The queue has at least one data pointing element which always points to the next storage buffer from which a pre stored message can be read and is always associated with a process which intends to read from the queue. In 8 case of multiple processes reading from the same queue multiple data pointing elements can be employed for reading messages from the queue where each of the data pointing element is associated with one reading process. Also, the position of free end pointing element and one or more data pointing elements is used to check whether the queue is empty i.e. has no message stored in any of the storage buffers or if it is full i.e. all the storage buffers of the queue have messages stored in them to be read by one or more processes. In the case of a queue having only one data pointing element i.e. the queue is point to point queue associated with only one reading and only one writing process, it is checked to be full or empty by locating the position of the free pointing element and the data pointing element and follows the procedure mentioned below: " If the Data pointing element and the Free pointing element points to the same storage buffer the queue is empty. * If the storage buffer pointed to by the Free pointing element has its next buffer pointed to by the Data Anchor then the queue is full. In that case the node pointed to by the free pointer is the next available node. In the case of a queue having multiple data pointing elements i.e. the queue is a publish subscribe queue which is associated with several reader processes reading messages from the same queue and every process has a 9 separate data pointing element and at any time only one process can write into the queue by associating itself to the free pointing element. Such a queue is checked to be full or empty by locating the position of the free pointing element and all the data pointing elements and following the below mentioned procedure: " If all the Data pointing elements and the free pointing element are pointing to the same storage buffer the queue is empty. " If the storage buffer pointed by the free pointing element has its next buffer pointed by any of the data pointing elements then the queue is full. In that case the node pointed to by the free pointer is the next available node. In accordance with the present invention the memory mapped files can be stored in a local machine i.e. a node or standalone computer running several processes also having message queues employed in the files and mapping the files into the main memory of the processor so as to facilitate inter process communication between processes running on the same standalone computer using either point to point queues between any two processes or by using a publish subscribe queue in which one process or several processes reads messages from one single queue. Also, in accordance with the present invention separate memory mapped files can be stored in remote machines i.e. any of the remotely present nodes communicably coupled with each other forming a network, to facilitate inter process communication between processes running on remote nodes using either point to point queues between any two processes running on remote 10 nodes or by using a publish subscribe queue in which one process or several processes running remote nodes reads messages from one single queue. In the above case writer processes running on a node of the network stores the messages into the message queue in its shared memory in which a message queue file is mapped and further this message is read by an intermediate sender process which then sends the message though the network to a second node running another intermediate receiver process which reads the message and stores it in the message queue mapped on the shared memory of this node so as to be read by the actual reading process. In accordance with present invention the messages can be written into the queues and read from the queues simultaneously without any kind of locking. As mentioned in the prior art, the locking of queues during a write operation on the queue by the writer process to lock the queue so that no other process can read till the write operation is finished and during a read operation on the queue by a reader process to lock the queue so that no process can write into the queue till the time reading operation is finished leads to very low throughput and higher latencies. Along with using locks the prior art processes also suspend on conditions of whether the queue is empty or full. This action of suspending the process and bringing the process back to execution incurs a lot of overhead and increases the average latency. 11 In order to eliminate this overhead the invention proposes continuous checking of the queues to be full or empty. This helps to keep very low latencies. To have such a lockless messaging system several constraints are also applied which are listed below: " Writer process can only write in the storage buffer of the queue which is pointed by free pointing element and reader process can only read the storage buffer which is pointed by data pointing element. " The ordering of the instructions of processes should not get modified by processor or chipset or Operating System for optimization purposes which may lead to reduced throughput. " The updation of free pointing element and the data pointing element should only be made after the processes have read or written into the queue. " Instructions and compiler directives for instance volatile keyword, memory barrier instructions are used to ensure that the message queuing works correctly in the hardware-software platforms with poor memory consistency. According to the preferred embodiment of the present invention the messaging system can be advantageously employed for facilitating financial trading of assets by traders, brokers, companies, and institutions spread across continents at a very high speed and reduced local and remote inter process communication latencies. 12 Referring to Figure 1, a system for lockless messaging in inter process communication between at least two processes is shown. The processes can be running at one stand alone node or else can be running on different nodes communicably coupled to each other in a network. Numeral 100 inside NODE A represents a storage which maintains queues 102 of storage buffers 104 to store inter process messages to be transferred from one process to another. As shown in Figure 2, the queue according to the preferred embodiment of the present invention is a cyclic linked list of storage buffers i.e. the every storage buffer points to the next storage buffer in the list and the last storage buffer points to the first storage buffer to make it cyclic. The queue 102 can be different point to point queue or publish subscribe queue from which several reader processes can read the messages using their process specific data pointing elements. In case of local inter process communication in which both reader and writer processes are running on the same node. The writer process enters a message into the storage buffer 104 of the queue 102 pointed by the free pointing element 106. After writing inside a storage buffer 104 the position of the free pointing element 106 is updated and it points to the next free storage buffer 104 of the queue in which another message can be written. Further, if the queue is of the Point to Point type, any one of the two processes associated with that queue can write a message in the storage 13 buffer 104 currently pointed by the free pointing element 106 and for a publish subscribe queue any process transferring messages through the queue can write into the storage buffer 104 currently pointed by the free pointing element 104. The reader process reads the messages stored in the storage buffers 104 of the queue 102 which is pointed by the data pointing element. Once a message is stored in the storage buffer 104, the storage buffer is read by the reader process. Consequently, the position of data pointing element is updated to point the next storage buffer 104 containg next message to be read. In case of a point to point queue any one of the two associated processes can read the message stored in the storage buffer 104 pointed by the data pointing element 108 of that queue and in case of publish subscribe messaging the any one of the processes communicating messages through that queue can read the messages. The difference here lies that every process will have a separate data pointing element any process wanting to read a message will read the message pointed by its own data pointing element 108. Figure 1 also illustrates two nodes NODE A and NODE B connected in a network which can run different reading and writing processes interacting with each other. In the same way, it is also within the scope of invention to have several nodes inter connected with each other using a TCP/IP connection or any other suitable connection running multiple processes using the system and method as envisaged by this invention for inter process communication. 14 Referring to Figure 3, a method for lockless messaging in inter-process communication between at least two processes is shown, the method comprises following steps: * providing a storage to store inter process messages, 1001; * providing a queue of storage buffers in a shared memory in the storage, 1003; * providing a free pointing element associated with a process to point to a free storage buffer in the queue, 1005; * providing at least one data pointing element associated with a process pointing to the storage buffers containing said inter process message, 1007; e receiving at least one inter process message from at least one process running on at least one node, 1009; e inserting the received inter process message into a storage buffer of said queue pointed by said free pointing element, 1011; * updating the position of free pointing element to next free storage buffer of the queue to accommodate the next inter process message, 1013; * fetching an interprocess message simultaneously by a plurality of processes from the storage buffer of the queue pointed by their corresponding data pointing elements, 1015; and * updating the position of the data pointing element to a next storage buffer containg a message to be read by any of the processes, 1017. 15 Also, instructions and compiler directives for instance volatile keyword and memory barrier instructions are used in a predefined order to ensure that the contents of the storage buffer are flushed from the cache to the main memory (RAM) after the step of inserting the received inter process message into a storage buffer of the queue pointed by the free pointing element. TECHNICAL ADVANCEMENTS The technical advancements of the present invention include in providing: * an efficient messaging system for reducing the computational time. e a cost effective messaging system. * a system which enables multiple processes to communicate simultaneously. * a system which provides high throughput and low latency. e a system which can be deployed easily with the existing systems. * a system that requires minimal infrastructure e an efficient message passing technique for critical applications. While considerable emphasis has been placed herein on the components and component parts of the preferred embodiments, it will be appreciated that many embodiments can be made and that many changes can be made in the preferred embodiments without departing from the principles of the invention. These and other changes in the preferred embodiment as well as other embodiments of the invention will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the invention and not as a limitation. 16

Claims

1. A system for messaging in inter-process communication between at least two processes running on at least one node, said system comprising: " a storage adapted to store inter process messages; " a queue of storage buffers in a shared memory in said storage concurrently accessible by plurality of processes; * a free pointing element associated with a process adapted to point to a free storage buffer in said queue; * at least one data pointing elements associated with a process adapted to point to a storage buffer containing an inter process message; and * message read/write means adapted to insert and fetch messages from said storage buffers in said queue simultaneously without locking the queue by allowing a process to insert messages in said free storage buffer pointed by said free pointing element and allowing at least one process to read an inserted message stored in a storage buffer containing an inter process message pointed by said data pointing element.

2. The system as claimed in claim 1, which includes an updation means adapted to update the position of said free pointing element and said data pointing element.

3. The system as claimed in claim 1, in which the shared memory is located on a local node.

4. The system as claimed in claim 1, in which the shared memory is located on a remote node. 17

5. The system as claimed in claim 1, in which the queue contains a cyclically linked list of storage buffers.

6. The system as claimed in claim 1, where in remote nodes are adapted to run the processes which communicate using communicating means adapted to facilitate connections selected from the group consisting of TCP/IP connection, GPRS connection, WiFi connection, WiMax connection and EDGE connection.

7. The system as claimed in claim 1, wherein said read/write means includes a checking means to check if the queue is full for a write/insert operation and check if the queue is empty for read/fetch operation.

8. The system as claimed in claim 1, in which the queues are contained in files.

9. The system as claimed in claim 1, in which the queues are contained in files which are mapped into the main memory space of a processor.

10. The system as claimed in claim 1, in which every storage buffer in a queue is linked to the next storage buffer and the last storage buffer links to the first storage buffer forming a circularly linked list for storing all the incoming messages into the storage buffers one after another. 18

11. The system as claimed in all the precedent claims, wherein said system is adapted to be used with financial trading applications.

12.. A method for lockless messaging in inter-process communication between at least two processes running on at least one node, said method comprising the following steps: " providing a storage to store inter process messages; " providing a queue of storage buffers in a shared memory in said storage; " providing a free pointing element associated with a process to point to a free storage buffer in said queue; * providing at least one data pointing element associated with a process pointing to said storage buffers containing said inter process message; * receiving at least one inter process message from at least one process running on at least one node; " inserting said received inter process message into a storage buffer of said queue pointed by said free pointing element; * updating the position of free pointing element to next free storage buffer of said queue to accommodate the next inter process message; " fetching an interprocess message simultaneously by a plurality of processes from said storage buffer of said queue pointed by their corresponding data pointing elements; and " updating the position of said data pointing element to a next storage buffer containg a message to be read by any of said processes.

13. The method as claimed in claim 12, wherein the step of updating the position of the free pointing element includes the steps of checking 19 if the free pointing element is pointing to a buffer pointed by a data pointing element, if true then waiting for the buffer to be free else updating the pointer position of free pointing element to next free storage buffer.

14. The method as claimed in claim 12, wherein the step of fetching an interprocess message includes the steps of checking if the data pointing element is pointing to a buffer pointed by the free pointing element, if true then waiting for data to be inserted into the queue for fetching.

15. The method as claimed in claim 12, wherein the step of inserting said received inter process message includes the step of copying said inserted message to main memory of said node.

16. The method as claimed in claim 12, wherein the step of inserting said received inter process message includes the step of executing instructions and compiler directives in a predefined order. 20