Based on low time delay FAST market decoding device and the method for pipelined architecture
Technical field
The invention belongs to areas of information technology, it is specifically related to a kind of low time delay FAST market based on pipelined architecture decoding device and method.
Background technology
The form that (multicast) broadcasts with group in finance exchange issues up-to-date market conditions data (marketdata) to participant in the market in real time. Market conditions data packet contains the up-to-date information such as up-to-date bid and offered quotation and demand, the conclusion of the business record (such as opening price, maxivalence, bottom price, present price, exchange hand, transaction value etc.) of financial product, order state. The financial market data of access are carried out real-time parsing by softwares such as Reading the Tape software or algorithm transaction by finance participant, make finance trade decision (such as dealing financial product) according to up-to-date market conditions. Therefore, the timely parsing of finance market data is most important for participant in the market (especially algorithm transaction and high frequency dealer). The participant in the market of up-to-date acquisition market conditions state preferentially can obtain the market profit of moment in other participants in the market.
Algorithm transaction (algorithmtrading) refer to automatically realized fast by computer program, the order of low cost performs and conclusion of the business. It is specifically related to determine that the best of order performs path, execution time, strike price and number of executions by program. Algorithm transaction is widely used in pension fund, common fund, hedge fund and other buyer organization investment person etc. Block trade can be divided into many penny antes by algorithm transaction and deal with the market risk and impact, can be market simultaneously and mobility is provided.
High frequency transaction (HighFrequencyTrading, HFT) refers to seeks, from the unserviceable very of short duration turn of the market of those people, the Computerized transaction that price difference realizes arbitrage by high performance computing platform. High frequency transaction is to the response time delay of marketing data at Microsecond grade, and the time of every time holding position is extremely short, substantially keeps closing a position during closing quotation. By accumulating, frequent slim profit repeatedly realizes making a profit in high frequency transaction. High frequency transaction is widely used in doing city, and it provides mobility for market.High frequency transaction has accounted for the 70% of US stock market total volume, accounts for 45% in Europe, accounts for 40% in Japan. High frequency transaction also starts to rise in emerging market, if all there is application the aspect such as commodity future, ETF and warrant at home.
Financial market exchanges agreement (SecuritiesTradingExchangeProtocol, STEP) mainly through security bargain data and transmits financal messaging between transaction mechanism and participant in the market at home, comprises order data, market data etc. Financial information exchange agreement (FinancialInformationExchange, FIX) in the completely compatible external financial market of STEP message. STEP/FIX message is strictly made up of the basic structure of multiple " tag=value ". This basic structure is called a territory or a field (field). Separate by field segmentation symbol " SOH " between field. Wherein " tag " is Field ID or field name, represents concrete field, and it implies the information such as the type of value, interval, and " value " is the value of this field. Being 270 as field " 270=342 " represents label, the value of field of (representing " market entry price " in STEP) is 342; SOH is non-printable character, and its ASCII code equals 1.
In order to reduce market data transfer bandwidth demand and transmission time delay, market body release can select streaming FIX agreement (FIXAdaptingforSTreaming, FAST) that STEP/FIX market message is carried out streaming encoding compression. FAST coding method reduces the size of data stream in two aspects. First, make it possible to utilize the dependency of data in stream by the concept of " field operations symbol ", eliminate redundant data. Secondly, make use of in the serial of binary coding to remaining data and the field that whether exists of the field length (stop bit encoding mechanism) of self-described and indication field can there is bitmap (PresenceMap, PMap). Coding basis is called that the control texture of " template " carries out. Template is by the regulation order of field and structure, field operations symbol, and the binary coding representation method used controls the coding of a part of convection current. Details refers to FIX/STEP and FAST agreement specification document more specifically. For convenience of description, the bit that each field of FIX/STEP obtains after FAST encodes is stated to be the encoded radio of field by the present invention.
Along with the open-development day by day of China's financial market, the financial application such as high frequency transaction from now on, algorithm transaction inherently occupy the bigger market share in China's financial market. The timely parsing of finance market data is the indispensable precondition of the financial application such as algorithm transaction, high frequency transaction, financial risks monitoring. In financial market, competition is day by day fierce, financial risks on the even social aspect of mechanism itself and even whole financial market affect ratio before become bigger. Therefore urgently need to study the FAST market decoding technique of a kind of low time delay, to support that financial risks monitoring, algorithm transaction etc. require the application demand of low time delay market process class. The mobility in financial market can also be promoted simultaneously.
Current FAST market decoding mainly realizes based on software approach, as adopt increase income market software OpenFAST (sourceforge.net/projects/openfast/), QuickFAST (www.ociweb.com/products/quickfast/), or enterprise oneself exploitation market decode system. Coding/decoding method based on software introduces extra data processing time delay.On the one hand, market network packet resolves the software network protocol stack time delay introduced: twice internal memory copy time delay and waiting system interrupt handling time delay; On the other hand, the system shake time delay that operating system is introduced, comprises multi-process compete for system resources, interrupts wait etc. The market processing delay of usual software is in millisecond rank. The market decoding time delay of Millisecond is difficult to meet such as the real-time application of high frequency transaction, market risk monitoring class.
On the other hand, FAST message has very strong data to be correlated with. FAST market message after coding is binary data stream, is all identified successively by PMap and stop bit encoding mechanism between field, between message. Namely the FAST encoded radio only having read a field from input stream could determine the position of next field in input market stream; With reason, after only having read a FAST message, next FAST message could be read in from input stream. The data dependence of FAST message limits the parallel decoding of FAST market. Published document does not also realize the work of FAST market message parallel decoding.
Summary of the invention
It is an object of the invention to provide a kind of device and the method for accelerating FAST market data decode based on specialized hardware, it is possible to the effective speed accelerating market decoding, for the financial application such as algorithm transaction, high frequency transaction, market risk monitoring provide support.
The technical solution used in the present invention is as follows:
A kind of low time delay FAST market based on pipelined architecture decoding device, comprise the field decoding operator of each field in inner bus, controller and FAST market data, each field decoding operator is connected to described inner bus respectively, completes the decoding of each field in FAST market data under the control of described controller successively; Described inner bus is divided into data bus and control bus, described data bus realizes FAST market data input stream buffer memory device and the data transmission between each field decoding operator and FIX message buffer, and described control bus is responsible for controlling the decode operation of each field.
Further, described field decoding operator is the decoding operator of syllogic, and is connected the FAST market decoding realizing streamline formula by bus; The decoding operator of described syllogic comprises reading certificate, field decoding, decoded result export three parts, carries out intermediate result buffer memory by buffer memory device between three parts. Wherein, read the encoded radio that data component is responsible for reading field from FAST market data input stream buffer memory device, field decoding parts are responsible for concrete decoding according to the rule of field operations symbol, and result output block is responsible for exporting the field value of decoding to exporting FIX message buffer. The FAST market decoding of described streamline formula comprises three independent streamlines altogether: the reading data component of all field decoding operators is connected to inner bus respectively, and is connected with Read Controller and forms time data stream waterline; The decoding device of all field decoding operators is connected to another inner bus respectively, is connected with decode controller and forms decoded stream waterline; The result output block of all field decoding operators is connected to another inner bus respectively, is connected with o controller and forms viewing pipeline.
Further, described controller adopts the finite state plane mechanism of band data path to realize, controller inside be divided into control path and data path, two paths inside realizes by finite state machine; Described control path is the field decoding task dispatcher of a top layer, sends, by it, the instruction that each field decoding operator starts decode operation successively; Described data path comprises the concrete steering logic of each decoding operator; After the decoding operator of a field completes decoding, data path returns decoding to control layer and terminates signal.
A kind of low time delay FAST market coding/decoding method based on pipelined architecture adopting said apparatus, field decoding operator is divided into reading certificate, field decoding, decoded result export three parts, and the FAST market decoding realizing streamline formula is connected by bus, comprise the steps:
1) the reading data component of all field decoding operators is connected to inner bus respectively, and is connected with Read Controller and forms time data stream waterline;From FAST market data input stream buffer memory device, the encoded radio of field is read by reading data component;
2) decoding device of all field decoding operators is connected to another inner bus respectively, is connected with decode controller and forms decoded stream waterline; It is responsible for concrete decoding according to the rule of field operations symbol by field decoding parts;
3) the result output block of all field decoding operators is connected to another inner bus respectively, is connected with o controller and forms viewing pipeline; The field value being responsible for exporting decoding by result output block is to exporting FIX message buffer.
Utilize device provided by the invention and method process FAST market data decode, have the following advantages:
1, the FAST market data processing speed of pole low time delay is obtained. The time delay of decoding FAST market is 0.1��1 delicate rank. Realize the decoding of FAST market based on specialized hardware, the extra process time delay that software system are introduced can be avoided.
2, based on the FAST market data decode treater of bus architecture, there is good extendability, support that field upgrades flexibly. Add new operator and only operator extension need to be loaded onto bus, delete operator and only need to unload operator bus. This point is very important to financial application, because FAST market template may often upgrade.
3, solving, based on the FAST market decoding processor of streamline, the difficult point that FAST market data rely on, solving FAST market can not the difficult point of parallel decoding, it is achieved that the parallel decoding between multiple field, multiple messages; And the reasonable representation of the intermediate result between each stage of streamline, the overall representation adopting " value exists mark+value ". Compared with the FAST market treater not using pipelined architecture, in theory, performance can obtain the acceleration rate of 3 times; Measured result shows, performance improves 1.8 times.
4, controller realizes based on the finite state machine of band data path, simplifies control and upgrades logic.
5, the present invention is applicable to the differential decoding agreement of the similar FAST agreement structure of task.
Actual measurement environment above-mentioned 1,3 is: FPGA chip is: XilinxZynq-7000chip (XC7Z020-3CLG484); Article one, FAST template comprises 12 fields.
Accompanying drawing explanation
Fig. 1 is the FAST market decoding processor schematic diagram based on bus architecture.
Fig. 2 is the field decoding device schematic diagram of syllogic streamline.
Fig. 3 is the FAST market decoding processor schematic diagram of syllogic streamline.
Fig. 4 is the controller principle schematic diagram based on FSMD.
Fig. 5 is the FAST market source codec treater schematic diagram realized based on FPGA.
Embodiment
For enabling above-mentioned purpose, the feature and advantage of the present invention more become apparent, below by specific embodiments and the drawings, the present invention will be further described.
In the present invention, specialized hardware is adopted to accelerate FAST market data decode, specialized hardware is such as FPGA (FieldProgrammableGateArray, field-programmable gate array), ASIC (ApplicationSpecificIntegratedCircuit, application specific integrated circuit) etc.
First, devise the FAST market decoding processor based on bus architecture. For concrete FAST market template, first the demoder (also referred to as decoding operator) of each field of definition in template is connected to respectively the inside bus of decoding processor. Inner bus is divided into data bus and control bus two class. The decoding of each field is completed successively under the control of decode controller.
Thereafter, the FAST market decoding processor of bus architecture is optimized improvement, the FAST market decoding processor of a design syllogic streamline.By the pipeline design cleverly, it is achieved that between FAST field, between FAST message, data, parallel decoding, Serial output decoded data are read in serial. Greatly improve the speed of FAST market decoding. The decoding operator being all designed to syllogic based on each field decoding operator of the FAST market decoding processor of streamline, then each decoding operator connects the overall FAST market demoder realizing streamline formula by bus.
1. based on the FAST market decoding processor of bus architecture
First design realizes all field operations symbol that FAST agreement is supported, these operational signs are embodied as decoding operator storehouse. For the decoding of concrete FAST template, first according to each field of definition in template, each field of instantiation from decoding operator storehouse. Fig. 1 describes the FAST market decoding processor based on bus architecture. The field decoding device that each is instantiated is connected to the inside bus of decoding processor respectively. Inner bus is divided into data bus and control bus two class. Data bus realizes FAST market data input stream buffer memory device (FAST-MSGFIFO) and each field decoding device, and the data transmission between FIX message buffer (FIX-MSGFIFO); Control bus is responsible for controlling the decode operation of each field. When the non-sky of FAST market data input stream buffer memory device, first controller activates the encoded radio that PMap field decoding device (PMapDecoder) reads PMap field from input stream, obtains PMap vector. Then the decoding (TIDDecoder in figure) of template ID is started. Thereafter, more successively activate the decode operation (fld in figure of follow-up field1Decoder��fldnDecoder). A front field decoding terminates, and could start next field decoding. Being correlated with owing to FAST message exists data, specifically the decoding of each field needs to judge whether there is field value in input data stream according to the PMap position of this field. The FAST market decoding processor based on bus structure that the present invention proposes has good extendability. Add a new field, only new decoding operator need to be hung and be loaded onto bus, and the steering logic of change control device; Delete a field, only this decoding operator need to be unloaded bus, and the steering logic of change control device. Extendability is especially important for often needing the financial application upgraded.
2. based on the FAST market decoding processor of streamline
Fig. 2 describes the field decoding operator of syllogic flowing water framework. This field decoding operator is divided into reading to export (writer) three parts according to (reader), field decoding (decoder), decoded result. Intermediate result buffer memory is carried out by buffer memory device (FIFO) between three parts. Read the encoded radio that data component is responsible for reading field from input stream (FAST-MSGFIFO); Field decoding parts are responsible for concrete decoding according to the rule of field operations symbol; Result output block is responsible for exporting the field value of decoding to exporting FIX message buffer (FIX-MSGFIFO).
Fig. 3 describes the framework of parallel FAST market decoding processor. This treater is divided into three independent streamlines. Each field is a syllogic field decoding operator, is namely divided into three big parts: reading certificate, field decoding, decoded result export. The reading data component that all (field) decodes operator is connected to inner bus (inside is divided into data bus and control bus) respectively, and be connected with Read Controller (readingcontrol, RC) and form the time data stream waterline of FAST market treater.Wherein Read Controller controls the concrete reading operation of each decoding operator. Similar, the decoding device of all decoding operators is connected to another inner bus respectively, and being connected with decode controller (decodingcontrol, DC) forms the decoded stream waterline of FAST market treater; The result output block of all decoding operators is connected to another inner bus respectively, and being connected with o controller (writingcontrol, WC) forms the viewing pipeline of FAST market treater. The concrete working mechanism that each streamline is described below.
A) time data stream waterline:
The read component realization order activating each field successively reads in the encoded radio of field. When inputting FAST message buffer (FAST-MSGFIFO) and be not empty, read data controller RC and send the reading data component (PMapreader) reading data enable signal to PMap decoding operator. These parts activate its subsequent parts (Template ID field, tidreader) and read data after having read data. Analogizing successively, Read Controller activates the reading data component of follow-up field successively according to the field order of PMap state and the definition of FAST template. After last field has read data, notice RC FAST message has been read. Now, if input FAST data flow buffer is not empty, then RC restarts the read work of a new FAST message; Otherwise enter idle waiting state.
Encoded radio write code field value buffer memory device xxx_in_fifo (the concrete field name in " xxx " the character representation figure occurred in the present invention that the read component of each operator will read in, encoded radio buffer memory device such as the first character section in FAST template is " fld1_in_fifo ", see Fig. 3), the basic unit of storage form of this buffer memory device is: " [pmap_bit] [value] ". Pmap_bit is the PMap zone bit of field, and field decoding parts need to utilize this mark determines how to encode. Namely demoder needs to know current field without encoded radio in input stream, recovers field value the need of the front value of utilization or initial value, when being labeled as 0 as copied the PMap of (COPY) operational sign field.
B) decoded stream waterline
What the decoding device of each field was parallel performs decoding. Reading the encoded radio of a field from code field value buffer memory device, it is also possible to do not need to read code field value, the rule accorded with by field operations and field PMap state determine. Then according to the rule execution field decoding of field operations symbol. Finally, decoded result is exported to field value buffer memory device (xxx_out_fifo).
The basic unit of storage structure of field value buffer memory device (xxx_out_fifo) is: " presence_flag [value] ". When this field has value, presence_flag=1; When this field is without presence_flag=0 during value, and " value " is empty. When the current field of message is empty, field output block needs to know that field is without value, and it does not need to write any data to output state. " value " is field value. Elongated String field value taking ' 0 ' as end mark.
C) viewing pipeline
The activation mechanism of viewing pipeline is same as time data stream waterline, activates the output block (xxx_writer) of each field successively. After each output block receives the output data enable signal that o controller sends, read the field value in corresponding field value buffer memory device (xxx_out_fifo), if it is 0 that the code field value read exists mark (presence_flag), indicate and need to export without field value, then terminate the output work of this field;Otherwise read a field value from field value buffer memory device, then by its byte-by-byte write FIX message buffer (FIX-MSGFIFO). After end of output, o controller notifies that follow-up output block continues to export field value.
3. controller
Controller in the present invention adopts finite state machine (FiniteStateMachinewithDatapath, the FSMD) mechanism of band data path to realize, as shown in Figure 4. Controller inside be divided into control path and data path, two paths inside realizes by finite state machine (FSM). Control path is the field decoding task dispatcher of a top layer, sends, by it, the instruction (xxx_dec_begin) that each field decoding operator starts decode operation successively; Data path comprises the concrete steering logic of each decoding operator. After the decoding operator of a field completes decoding, data path can return decoding to control layer and terminate signal (xxx_dec_done).
4. based on the low time delay market decoding processor of FPGA
The market decoding framework that the present invention proposes, can realize on the specialized hardwares such as FPGA (FieldProgrammableGateArray), ASIC (ApplicationSpecificIntegratedCircuit). This section take FPGA platform as row, provides concrete realization.
Fig. 5 gives the framework figure of the FAST market source codec treater realized based on FPGA. Wherein, finance market treater is the FAST market decoding processor based on streamline that the present invention proposes. A data exchange module is had, the fast data exchange of this module in charge market treater and ambient systems at FPGA chip internal. On the one hand, data exchange module is responsible for receiving the present quotation data stream issued from exchange, the data packet received being carried out Xie Bao and further parsing and extracts FAST market data, buffer memory is to FAST market data buffer memory device (FAST-MSG-FIFO). On the other hand, data exchange module is responsible for decoded FIX transmission of messages to financial application system (comprising necessary package). Due to the mode real-time release market that exchange is broadcast by group, therefore data exchange module is by express network interface access market data, as by 10/40GPYH+QSFP++GMAC core etc.; Decoded FIX market transmission of messages can be express network interface to financial application system, it is also possible to realized the quick transmission of market by PCIExpress interface+DMA signalling methods.
Above embodiment only in order to the technical scheme of the present invention to be described but not be limited; the technical scheme of the present invention can be modified or equivalent replacement by the those of ordinary skill of this area; and not departing from the spirit and scope of the present invention, protection scope of the present invention should to be as the criterion described in claim book.