CN106557974B - IMIX protocol data processing method and system - Google Patents
IMIX protocol data processing method and system Download PDFInfo
- Publication number
- CN106557974B CN106557974B CN201611027572.4A CN201611027572A CN106557974B CN 106557974 B CN106557974 B CN 106557974B CN 201611027572 A CN201611027572 A CN 201611027572A CN 106557974 B CN106557974 B CN 106557974B
- Authority
- CN
- China
- Prior art keywords
- data
- file
- serialized
- frequency
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/02—Banking, e.g. interest calculation or account maintenance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/14—Tree-structured documents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/149—Adaptation of the text data for streaming purposes, e.g. Efficient XML Interchange [EXI] format
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Development Economics (AREA)
- General Engineering & Computer Science (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Multimedia (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method for processing IMIX protocol data, which comprises the following steps: receiving IMIX protocol data generated in the transaction process; analyzing the IMIX protocol data to generate a transaction data object; analyzing the transaction data object to obtain a high-frequency processing domain; writing the high-frequency processing domain into a file stream with an agreed format, and restarting and serializing the file stream written into the high-frequency processing domain; and writing the serialized transaction data object serving as a sequence message field into an additional file stream to generate a serialized file. The invention can ensure the safety and stability of the data processing process. The invention also discloses a system for processing the IMIX protocol data.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a method and a system for processing IMIX (Inter-bank Market Information eXchange) protocol data.
Background
The IMIX protocol is used for financial information transmission in the domestic and foreign exchange markets between banks, and is currently widely applied to the domestic inter-bank market. The protocol is composed of three parts, namely a standard message header, a message body and a standard message tail, each part comprises a plurality of domains, domain blocks, components, chunks and the like, and the message structure of the specific protocol is shown in figure 1.
Financial information of the bank-to-bank currency market and the foreign exchange market is transmitted by using an IMIX protocol, and then subsequent transaction, settlement and other processing are carried out, so that a large amount of data needs to be converted and stored in a database of an application system for subsequent processing. As the transaction data amount related to the domestic currency market and the foreign exchange market among banks is huge, the safety and the stability of the data processing process must be ensured.
At present, when data of an IMIX protocol is processed, a conventional storage method is to build a table in a database according to a structure of the IMIX protocol, for example, a domain or a domain block in an IMIX message is mapped to a table and a field in the database, then the IMIX protocol is analyzed, and an analyzed result is directly stored in a corresponding table and a corresponding field in the database for subsequent system processing. On one hand, the existing processing mode has complex and various contents related to the IMIX protocol, and each part often contains nested relation and checking relation, so that a database table directly designed according to the system structure of the IMIX protocol is necessarily very complex, causes performance loss in the subsequent sequence extraction and analysis process, and is difficult to maintain; on the other hand, the IMIX protocol is continuously upgraded and evolved according to the change of the financial market, and a data storage mode in a database form designed based on the structural body of the protocol necessarily needs to be correspondingly modified, so that the loss of the security and the stability of the system is inevitably caused by the frequent change of the underlying data structure.
Disclosure of Invention
The invention provides a processing method of IMIX protocol data, which can ensure the safety and stability of the data processing process.
The invention provides a method for processing IMIX protocol data, which comprises the following steps:
receiving IMIX protocol data generated in the transaction process;
analyzing the IMIX protocol data to generate a transaction data object;
analyzing the transaction data object to obtain a high-frequency processing domain;
writing the high-frequency processing domain into a file stream in an agreed format, and restarting and serializing the file stream written into the high-frequency processing domain;
and writing the serialized transaction data object serving as a sequence message field into an additional file stream to generate a serialized file.
Preferably, the analyzing the transaction data object to obtain a high-frequency processing domain includes:
analyzing the high-frequency query content in the transaction data based on a dictionary, and extracting the value of the field of interest;
and assembling the interest field values into a high-frequency query condition extensible markup language header meeting extensible markup language standards as a high-frequency processing domain.
Preferably, after the analyzing the IMIX protocol data to generate the transaction data object, the method further includes:
and storing the transaction data object based on the service requirement and the system configuration.
Preferably, the writing the serialized transaction data object into the additional file stream as a sequence message field, and generating the serialized file further includes:
and storing the serialized file.
Preferably, after storing the serialized file, the method further includes:
assembling the query message based on the query and processing conditions;
analyzing the query message, retrieving and reading the service date file from the serialized file storage server according to the service date, and loading the service date file into a memory;
fully matching the remaining query conditions with conditions in all high-frequency query domains in the file object, and if the remaining query conditions are matched with the conditions in all high-frequency query domains in the file object, taking out the serialized object content in the serialized file domain corresponding to the domain;
deserializing the serialized object content to generate a data result or object;
and transmitting the data result or the object to a service processing server for service data processing based on the data check domain.
A system for processing IMIX protocol data, comprising:
the message receiving and analyzing server is used for receiving IMIX protocol data generated in the transaction process and analyzing the IMIX protocol data to generate a transaction data object;
and the serialization engine is used for analyzing the transaction data object to obtain a high-frequency processing domain, writing the high-frequency processing domain into a file stream in an agreed format, restarting and serializing the file stream written into the high-frequency processing domain, writing the serialized transaction data object into an additional file stream as a sequence message domain, and generating a serialized file.
Preferably, the serialization engine is specifically configured to:
analyzing the high-frequency query content in the transaction data based on a dictionary, and extracting the value of the field of interest;
and assembling the interest field values into a high-frequency query condition extensible markup language header meeting extensible markup language standards as a high-frequency processing domain.
Preferably, the system further comprises:
and the database is used for storing the transaction data object based on the service requirement and the system configuration.
Preferably, the system further comprises:
and the serialized file storage server is used for storing the serialized file.
Preferably, the system further comprises a service handler configured to:
assembling the query message based on the query and processing conditions;
analyzing the query message, retrieving and reading the service date file from the serialized file storage server according to the service date, and loading the service date file into a memory;
fully matching the remaining query conditions with conditions in all high-frequency query domains in the file object, and if the remaining query conditions are matched with the conditions in all high-frequency query domains in the file object, taking out the serialized object content in the serialized file domain corresponding to the domain;
deserializing the serialized object content to generate a data result or object;
and transmitting the data result or the object to a service processing server for service data processing based on the data check domain.
According to the scheme, when the IMIX protocol data is processed, firstly, IMIX protocol data generated in a transaction process is received, then, the IMIX protocol data is analyzed to generate a transaction data object, then, the transaction data object is analyzed to obtain a high-frequency processing domain, then, the high-frequency processing domain is written into a file stream with an agreed format, and the file stream written into the high-frequency processing domain is restarted and serialized; and writing the serialized transaction data object serving as a sequence message field into an additional file stream to generate a serialized file. The IMIX message is processed by the serialization engine, so that the encryption and persistence of data can be realized, the serialization protocol of the front-end stage can improve the universality, the robustness and the safety of the system and optimize the performance of the system, and the system is easier to debug and is convenient to expand.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a protocol packet structure;
fig. 2 is a flowchart of an embodiment 1 of a method for processing IMIX protocol data according to the present disclosure;
fig. 3 is a flowchart of an embodiment 2 of a method for processing IMIX protocol data according to the present disclosure;
fig. 4 is a flowchart of an embodiment 3 of a method for processing IMIX protocol data according to the present disclosure;
fig. 5 is a schematic structural diagram of an embodiment 1 of an IMIX protocol data processing system disclosed in the present invention;
fig. 6 is a schematic structural diagram of an embodiment 2 of an IMIX protocol data processing system disclosed in the present invention;
fig. 7 is a schematic structural diagram of an embodiment 3 of an IMIX protocol data processing system disclosed in the present invention;
fig. 8 is a schematic structural diagram of a package specification disclosed in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 2, the flowchart of embodiment 1 of the method for processing the IMIX protocol data disclosed in the present invention includes the following steps:
s101, receiving IMIX protocol data generated in a transaction process;
when the IMIX protocol data needs to be processed, IMIX protocol data generated during a transaction is first received. The IMIX protocol is used for the transmission of financial information of the domestic bank and foreign exchange markets, provides service layer messages and strong session layer messages covering the domestic bank and domestic bank markets before, during and after transaction, and provides a convenient channel for interaction of financial data of the bank and foreign exchange markets.
S102, analyzing the IMIX protocol data to generate a transaction data object;
and then analyzing the received IMIX protocol data generated in the transaction process to generate a transaction data object.
S103, analyzing the transaction data object to obtain a high-frequency processing domain;
and then analyzing the necessary high-frequency query content in the transaction data object to obtain a high-frequency processing domain.
S104, writing the high-frequency processing domain into a file stream with an agreed format, and restarting and serializing the file stream written into the high-frequency processing domain;
and then writing the constructed high-frequency processing domain into a file stream in an agreed format, restarting the written file stream after writing, and serializing the file stream.
And S105, writing the serialized transaction data object into an additional file stream as a sequence message domain to generate a serialized file.
And finally, writing the serialized transaction data object serving as a serialized message domain into the additional file stream, and generating a serialized file after two times of writing.
In summary, in the above embodiment, when the IMIX protocol data is processed, the IMIX protocol data generated in the transaction process is received first, then the IMIX protocol data is analyzed to generate a transaction data object, then the transaction data object is analyzed to obtain a high-frequency processing domain, then the high-frequency processing domain is written into the agreed format file stream, and the file stream written into the high-frequency processing domain is restarted and serialized; and writing the serialized transaction data object serving as a sequence message field into an additional file stream to generate a serialized file. The IMIX message is processed by the serialization engine, so that the encryption and persistence of data can be realized, the serialization protocol of the front-end stage can improve the universality, the robustness and the safety of the system and optimize the performance of the system, and the system is easier to debug and is convenient to expand.
As shown in fig. 3, is a flowchart of an embodiment 2 of a method for processing IMIX protocol data disclosed in the present invention, and the method includes the following steps:
s201, receiving IMIX protocol data generated in a transaction process;
when the IMIX protocol data needs to be processed, IMIX protocol data generated during a transaction is first received. The IMIX protocol is used for the transmission of financial information of the domestic bank and foreign exchange markets, provides service layer messages and strong session layer messages covering the domestic bank and domestic bank markets before, during and after transaction, and provides a convenient channel for interaction of financial data of the bank and foreign exchange markets.
S202, analyzing the IMIX protocol data to generate a transaction data object;
and then analyzing the received IMIX protocol data generated in the transaction process to generate a transaction data object.
S203, storing the transaction data object based on the service requirement and the system configuration;
and carrying out service preprocessing on the transaction data object according to the service requirement and the system configuration, and then storing the transaction data object after service preprocessing.
S204, analyzing the high-frequency query content in the transaction data based on the dictionary, and extracting the value of the field of interest;
and analyzing the content of the high-frequency query necessary in the received transaction data according to a dictionary provided by foreign exchange, and extracting the value of the field of interest.
S205, assembling the concerned field values into a high-frequency query condition extensible markup language header meeting extensible markup language standards as a high-frequency processing domain;
and assembling the extracted interested field values into a high-frequency query condition extensible markup language header meeting the extensible markup language standard as a high-frequency processing domain, wherein the base code adopts GB 2312.
S206, writing the high-frequency processing domain into a file stream with an agreed format, and restarting and serializing the file stream written into the high-frequency processing domain;
and then writing the constructed high-frequency processing domain into a file stream in an agreed format, restarting the written file stream after writing, and serializing the file stream.
S207, writing the serialized transaction data object into an additional file stream as a sequence message domain to generate a serialized file;
writing the serialized transaction data object serving as a serialized message domain into the additional file stream, and generating a serialized file after two times of writing.
And S208, storing the serialized file.
And after the two times of writing, entering a queue to be written or directly writing the serialized file into a serialized file storage server for storage without entering the queue according to configuration.
In summary, in the above embodiment, when the IMIX protocol data is processed, the IMIX protocol data generated in the transaction process is received first, then the IMIX protocol data is analyzed to generate a transaction data object, then the transaction data object is analyzed to obtain a high-frequency processing domain, then the high-frequency processing domain is written into the agreed format file stream, and the file stream written into the high-frequency processing domain is restarted and serialized; and writing the serialized transaction data object serving as a sequence message field into an additional file stream to generate a serialized file. The IMIX message is processed by the serialization engine, so that the encryption and persistence of data can be realized, the serialization protocol of the front-end stage can improve the universality, the robustness and the safety of the system and optimize the performance of the system, and the system is easier to debug and is convenient to expand.
As shown in fig. 4, is a flowchart of embodiment 3 of a method for processing IMIX protocol data, which is disclosed in the present invention, and the method includes the following steps:
s301, receiving IMIX protocol data generated in the transaction process;
when the IMIX protocol data needs to be processed, IMIX protocol data generated during a transaction is first received. The IMIX protocol is used for the transmission of financial information of the domestic bank and foreign exchange markets, provides service layer messages and strong session layer messages covering the domestic bank and domestic bank markets before, during and after transaction, and provides a convenient channel for interaction of financial data of the bank and foreign exchange markets.
S302, analyzing the IMIX protocol data to generate a transaction data object;
and then analyzing the received IMIX protocol data generated in the transaction process to generate a transaction data object.
S303, storing the transaction data object based on the service requirement and the system configuration;
and carrying out service preprocessing on the transaction data object according to the service requirement and the system configuration, and then storing the transaction data object after service preprocessing.
S304, analyzing the high-frequency query content in the transaction data based on the dictionary, and extracting the value of the concerned field;
and analyzing the content of the high-frequency query necessary in the received transaction data according to a dictionary provided by foreign exchange, and extracting the value of the field of interest.
S305, assembling the concerned field values into a high-frequency query condition extensible markup language header meeting extensible markup language standards as a high-frequency processing domain;
and assembling the extracted interested field values into a high-frequency query condition extensible markup language header meeting the extensible markup language standard as a high-frequency processing domain, wherein the base code adopts GB 2312.
S306, writing the high-frequency processing domain into a file stream in an agreed format, and restarting and serializing the file stream written into the high-frequency processing domain;
and then writing the constructed high-frequency processing domain into a file stream in an agreed format, restarting the written file stream after writing, and serializing the file stream.
S307, writing the serialized transaction data object into an additional file stream as a sequence message domain to generate a serialized file;
writing the serialized transaction data object serving as a serialized message domain into the additional file stream, and generating a serialized file after two times of writing.
S308, storing the serialized file;
and after the two times of writing, entering a queue to be written or directly writing the serialized file into a serialized file storage server for storage without entering the queue according to configuration.
S309, assembling the query message based on the query and processing conditions;
according to the data characteristics and the service characteristics of financial information of the bank-to-bank note market and the foreign exchange market, a set of extensible markup language packaging specifications is designed, high-frequency use data and other serialized objects are combined and stored in a class extensible markup language file, effective extraction of key data is achieved, the purpose of quick retrieval is achieved, the data except for high-frequency processing is separately stored in a serialized message domain, and effectiveness, integrity and consistency of the data are guaranteed through a data check domain. The specific package specification is shown in fig. 8.
S310, analyzing the query message, retrieving and reading the service date file from the serialized file storage server according to the service date, and loading the service date file into a memory;
the processor sends the query message to the serialization engine, the engine analyzes the query message, and first, according to the service date, the date file is retrieved from the serialization file storage server and read out, and the date file is loaded into the memory. If the date file object exists in the memory, the step is skipped.
S311, fully matching the residual query conditions with the conditions in all the high-frequency query domains in the file object, and if the residual query conditions are matched with the conditions in all the high-frequency query domains in the file object, entering S312:
s312, extracting the serialized object content in the serialized file domain corresponding to the domain;
and completely matching the residual query conditions with the conditions in all the high-frequency query domains in the file object, and if the conditions are hit, taking out the contents of the serialized objects in the serialized file domains corresponding to the domains, so that the larger deserialization operation of the system is really performed.
S313, performing deserialization on the serialized object content to generate a data result or object;
and S314, transmitting the data result or the object to a service processing server for service data processing based on the data check domain.
And then the deserialized object is called a service processing method or is sent to a corresponding service processing server for service data processing. And finally, the safety, integrity and consistency of message transmission data are ensured through the data check domain.
In summary, in the above embodiments, the serialization engine is used to process the IMIX message data, store the serialized data, and perform deserialization processing when using the data, so that the data storage does not depend on the database, and the storage security of the data and the flexibility of the system are ensured;
according to the data characteristics and the service characteristics of financial information of the bank-to-bank currency market and the foreign exchange market, a set of extensible markup language packaging specifications is designed, high-frequency use data and other serialized objects are combined and stored in extensible markup language files, effective extraction of key data is achieved, the purpose of quick retrieval is achieved, the data except for high-frequency processing is separately stored in a serialized message domain, and effectiveness, integrity and consistency of the data are guaranteed through a data check domain.
As shown in fig. 5, a schematic structural diagram of an embodiment 1 of the system for processing the IMIX protocol data disclosed in the present invention is shown, and the system includes:
the message receiving and analyzing server 501 is configured to receive the IMIX protocol data generated in the transaction process, and analyze the IMIX protocol data to generate a transaction data object;
the Imix message receiving and analyzing server is a receiving server which completes self-developed 7X 24-hour background silent operation based on ImixAPI provided by the Chinese foreign exchange transaction center. The Agent is divided into an Agent and a Client, the Agent completes the connection work with the CSTP server through a private network, and the links between a plurality of clients and the CSTP server are maintained and managed. In addition, the Agent also ensures that the received transaction data is forwarded to each Client terminal, and the Client terminal mainly works as two parts after receiving the data as well: the data analysis and restoration system is used for analyzing and restoring data into transaction data objects, and storing the transaction data objects into corresponding databases according to business needs and system configuration. And secondly, converting the transaction data into objects and then uniformly transmitting the objects to a subsequent serialization engine server.
The serialization engine 502 is configured to analyze the transaction data object to obtain a high-frequency processing domain, write the high-frequency processing domain into a file stream in an agreed format, restart serialization of the file stream written into the high-frequency processing domain, write the serialized transaction data object into an additional file stream as a sequence message domain, and generate a serialized file.
The serialization engine is a passive receiving type response, after receiving a transaction data object actively sent by a Client terminal, the necessary high-frequency query content in the transaction data object is analyzed to obtain a high-frequency processing domain, the constructed high-frequency processing domain is written into a file stream with an appointed format, the written file stream is restarted after the writing is finished, the file stream is serialized, and finally the serialized transaction data object is written into an additional file stream as a serialized message domain to generate a serialized file after two times of writing.
In summary, in the above embodiment, when the IMIX protocol data is processed, the IMIX protocol data generated in the transaction process is received first, then the IMIX protocol data is analyzed to generate a transaction data object, then the transaction data object is analyzed to obtain a high-frequency processing domain, then the high-frequency processing domain is written into the agreed format file stream, and the file stream written into the high-frequency processing domain is restarted and serialized; and writing the serialized transaction data object serving as a sequence message field into an additional file stream to generate a serialized file. The IMIX message is processed by the serialization engine, so that the encryption and persistence of data can be realized, the serialization protocol of the front-end stage can improve the universality, the robustness and the safety of the system and optimize the performance of the system, and the system is easier to debug and is convenient to expand.
As shown in fig. 6, which is a schematic structural diagram of an embodiment 2 of the system for processing IMIX protocol data disclosed in the present invention, the system includes:
the message receiving and analyzing server 601 is configured to receive the IMIX protocol data generated in the transaction process, and analyze the IMIX protocol data to generate a transaction data object;
the Imix message receiving and analyzing server is a receiving server which completes self-developed 7X 24-hour background silent operation based on ImixAPI provided by the Chinese foreign exchange transaction center. The Agent is divided into an Agent and a Client, the Agent completes the connection work with the CSTP server through a private network, and the links between a plurality of clients and the CSTP server are maintained and managed. In addition, the Agent also ensures that the received transaction data is forwarded to each Client terminal, and the Client terminal mainly works as two parts after receiving the data as well: the data analysis and restoration system is used for analyzing and restoring data into transaction data objects, and storing the transaction data objects into corresponding databases according to business needs and system configuration. And secondly, converting the transaction data into objects and then uniformly transmitting the objects to a subsequent serialization engine server.
A database 602 for storing the transaction data objects based on business requirements and system configuration;
and carrying out service preprocessing on the transaction data object according to the service requirement and the system configuration, and then storing the transaction data object after service preprocessing.
A serialization engine 603, configured to parse the high-frequency query content in the deal data based on a dictionary, and extract a field value of interest; assembling the concerned field values into a high-frequency query condition extensible markup language header meeting extensible markup language standards as a high-frequency processing domain; writing the high-frequency processing domain into a file stream with an agreed format, restarting and serializing the file stream written into the high-frequency processing domain, writing a serialized transaction data object into an additional file stream as a sequence message domain, and generating a serialized file;
and analyzing the content of the high-frequency query necessary in the received transaction data according to a dictionary provided by foreign exchange, and extracting the value of the field of interest. And assembling the extracted interested field values into a high-frequency query condition extensible markup language header meeting the extensible markup language standard as a high-frequency processing domain, wherein the base code adopts GB 2312. And then writing the constructed high-frequency processing domain into a file stream in an agreed format, restarting the written file stream after writing, and serializing the file stream. Writing the serialized transaction data object serving as a serialized message domain into the additional file stream, and generating a serialized file after two times of writing.
A serialized file storage server 604 for storing the serialized file.
And after the two times of writing, entering a queue to be written or directly writing the serialized file into a serialized file storage server for storage without entering the queue according to configuration.
In summary, in the above embodiment, when the IMIX protocol data is processed, the IMIX protocol data generated in the transaction process is received first, then the IMIX protocol data is analyzed to generate a transaction data object, then the transaction data object is analyzed to obtain a high-frequency processing domain, then the high-frequency processing domain is written into the agreed format file stream, and the file stream written into the high-frequency processing domain is restarted and serialized; and writing the serialized transaction data object serving as a sequence message field into an additional file stream to generate a serialized file. The IMIX message is processed by the serialization engine, so that the encryption and persistence of data can be realized, the serialization protocol of the front-end stage can improve the universality, the robustness and the safety of the system and optimize the performance of the system, and the system is easier to debug and is convenient to expand.
Fig. 7 is a schematic structural diagram of an embodiment 3 of an IMIX protocol data processing system disclosed in the present invention, where the system includes:
the message receiving and analyzing server 701 is used for receiving the IMIX protocol data generated in the transaction process and analyzing the IMIX protocol data to generate a transaction data object;
the Imix message receiving and analyzing server is a receiving server which completes self-developed 7X 24-hour background silent operation based on ImixAPI provided by the Chinese foreign exchange transaction center. The Agent is divided into an Agent and a Client, the Agent completes the connection work with the CSTP server through a private network, and the links between a plurality of clients and the CSTP server are maintained and managed. In addition, the Agent also ensures that the received transaction data is forwarded to each Client terminal, and the Client terminal mainly works as two parts after receiving the data as well: the data analysis and restoration system is used for analyzing and restoring data into transaction data objects, and storing the transaction data objects into corresponding databases according to business needs and system configuration. And secondly, converting the transaction data into objects and then uniformly transmitting the objects to a subsequent serialization engine server.
A database 702 for storing the transaction data objects based on business requirements and system configuration;
and carrying out service preprocessing on the transaction data object according to the service requirement and the system configuration, and then storing the transaction data object after service preprocessing.
A serialization engine 703 for analyzing the high-frequency query content in the transaction data based on a dictionary, and extracting a value of interest field; assembling the concerned field values into a high-frequency query condition extensible markup language header meeting extensible markup language standards as a high-frequency processing domain; writing the high-frequency processing domain into a file stream with an agreed format, restarting and serializing the file stream written into the high-frequency processing domain, writing a serialized transaction data object into an additional file stream as a sequence message domain, and generating a serialized file;
and analyzing the content of the high-frequency query necessary in the received transaction data according to a dictionary provided by foreign exchange, and extracting the value of the field of interest. And assembling the extracted interested field values into a high-frequency query condition extensible markup language header meeting the extensible markup language standard as a high-frequency processing domain, wherein the base code adopts GB 2312. And then writing the constructed high-frequency processing domain into a file stream in an agreed format, restarting the written file stream after writing, and serializing the file stream. Writing the serialized transaction data object serving as a serialized message domain into the additional file stream, and generating a serialized file after two times of writing.
A serialized file storage server 704 for storing the serialized file;
and after the two times of writing, entering a queue to be written or directly writing the serialized file into a serialized file storage server for storage without entering the queue according to configuration.
A service processor 705, configured to assemble a query message based on query and processing conditions; analyzing the query message, retrieving and reading the service date file from the serialized file storage server according to the service date, and loading the service date file into a memory; fully matching the remaining query conditions with conditions in all high-frequency query domains in the file object, and if the remaining query conditions are matched with the conditions in all high-frequency query domains in the file object, taking out the serialized object content in the serialized file domain corresponding to the domain; deserializing the serialized object content to generate a data result or object; and transmitting the data result or the object to a service processing server for service data processing based on the data check domain.
According to the data characteristics and the service characteristics of financial information of the bank-to-bank note market and the foreign exchange market, a set of extensible markup language packaging specifications is designed, high-frequency use data and other serialized objects are combined and stored in a class extensible markup language file, effective extraction of key data is achieved, the purpose of quick retrieval is achieved, the data except for high-frequency processing is separately stored in a serialized message domain, and effectiveness, integrity and consistency of the data are guaranteed through a data check domain. The specific package specification is shown in fig. 8.
The processor sends the query message to the serialization engine, the engine analyzes the query message, and first, according to the service date, the date file is retrieved from the serialization file storage server and read out, and the date file is loaded into the memory. If the date file object exists in the memory, the step is skipped.
And completely matching the residual query conditions with the conditions in all the high-frequency query domains in the file object, and if the conditions are hit, taking out the contents of the serialized objects in the serialized file domains corresponding to the domains, so that the larger deserialization operation of the system is really performed.
And then the deserialized object is called a service processing method or is sent to a corresponding service processing server for service data processing. And finally, the safety, integrity and consistency of message transmission data are ensured through the data check domain.
In summary, in the above embodiments, the serialization engine is used to process the IMIX message data, store the serialized data, and perform deserialization processing when using the data, so that the data storage does not depend on the database, and the storage security of the data and the flexibility of the system are ensured;
according to the data characteristics and the service characteristics of financial information of the bank-to-bank currency market and the foreign exchange market, a set of extensible markup language packaging specifications is designed, high-frequency use data and other serialized objects are combined and stored in extensible markup language files, effective extraction of key data is achieved, the purpose of quick retrieval is achieved, the data except for high-frequency processing is separately stored in a serialized message domain, and effectiveness, integrity and consistency of the data are guaranteed through a data check domain.
The functions described in the method of the present embodiment, if implemented in the form of software functional units and sold or used as independent products, may be stored in a storage medium readable by a computing device. Based on such understanding, part of the contribution of the embodiments of the present invention to the prior art or part of the technical solution may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computing device (which may be a personal computer, a server, a mobile computing device, a network device, or the like) to execute all or part of the steps of the method described in the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (6)
1. A processing method of IMIX protocol data is characterized by comprising the following steps:
receiving IMIX protocol data generated in the transaction process;
analyzing the IMIX protocol data to generate a transaction data object;
analyzing the high-frequency query content in the transaction data, and extracting the value of the field of interest;
assembling the concerned field values into a high-frequency query condition extensible markup language header meeting extensible markup language standards as a high-frequency processing domain;
writing the high-frequency processing domain into a file stream in an agreed format, and restarting and serializing the file stream written into the high-frequency processing domain;
writing the serialized transaction data object serving as a sequence message field into an additional file stream, generating a serialized file, and storing the serialized file;
assembling the query message based on the query and processing conditions;
analyzing the query message, retrieving and reading the service date file from the serialized file storage server according to the service date, and loading the service date file into a memory;
fully matching the remaining query conditions with conditions in all high-frequency query domains in the file object, and if the remaining query conditions are matched with the conditions in all high-frequency query domains in the file object, taking out the serialized object content in the serialized file domain corresponding to the domain;
deserializing the serialized object content to generate a data result or object;
transmitting the data result or the object to a service processing server based on the data check domain to perform service data processing;
in the processing process, the serialization engine is used for processing the IMIX protocol data and storing the serialized data, the high-frequency use data and other serialization objects are combined and stored in the extensible markup language file, the data except for the data needing high-frequency processing are separately stored in the serialization message field, and the deserialization processing is performed when the data are used, so that the data are not stored in a database.
2. The method of claim 1, wherein parsing the high frequency query content in the deal data and extracting values of fields of interest comprises:
analyzing the high-frequency query content in the transaction data based on a dictionary, and extracting the value of the field of interest.
3. The method of claim 2, wherein the parsing the IMIX protocol data to generate a transaction data object further comprises:
and storing the transaction data object based on the service requirement and the system configuration.
4. A system for processing IMIX protocol data, comprising:
the message receiving and analyzing server is used for receiving IMIX protocol data generated in the transaction process and analyzing the IMIX protocol data to generate a transaction data object;
the serialization engine is used for analyzing high-frequency query contents in the transaction data, extracting the field values of interest, assembling the field values of interest into a high-frequency query condition extensible markup language header meeting extensible markup language standards as a high-frequency processing domain, writing the high-frequency processing domain into a file stream in an agreed format, restarting and serializing the file stream written into the high-frequency processing domain, writing a serialized transaction data object into an additional file stream as a sequence message domain, and generating a serialized file;
the serialized file storage server is used for storing the serialized file;
a service handler to: assembling the query message based on the query and processing conditions; analyzing the query message, retrieving and reading the service date file from the serialized file storage server according to the service date, and loading the service date file into a memory; fully matching the remaining query conditions with conditions in all high-frequency query domains in the file object, and if the remaining query conditions are matched with the conditions in all high-frequency query domains in the file object, taking out the serialized object content in the serialized file domain corresponding to the domain; deserializing the serialized object content to generate a data result or object; transmitting the data result or the object to a service processing server based on the data check domain to perform service data processing;
in the processing process of the system, a serialization engine is used for processing IMIX protocol data and storing the serialized data, the high-frequency use data and other serialization objects are combined and stored in an extensible markup language file, the data except for the data needing high-frequency processing are separately stored in a serialization message domain, and the deserialization processing is performed when the data are used, so that the data are not dependent on a database.
5. The system of claim 4, wherein the serialization engine parses high frequency query content in the deal data, extracting values of fields of interest, including:
analyzing the high-frequency query content in the transaction data based on a dictionary, and extracting the value of the field of interest.
6. The system of claim 5, further comprising:
and the database is used for storing the transaction data object based on the service requirement and the system configuration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611027572.4A CN106557974B (en) | 2016-11-21 | 2016-11-21 | IMIX protocol data processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611027572.4A CN106557974B (en) | 2016-11-21 | 2016-11-21 | IMIX protocol data processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106557974A CN106557974A (en) | 2017-04-05 |
| CN106557974B true CN106557974B (en) | 2020-11-20 |
Family
ID=58444420
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611027572.4A Active CN106557974B (en) | 2016-11-21 | 2016-11-21 | IMIX protocol data processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106557974B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107370753A (en) * | 2017-08-23 | 2017-11-21 | 上海斐讯数据通信技术有限公司 | A kind of data pack protocol parses field processing method and system |
| CN110020412B (en) * | 2019-04-02 | 2023-04-28 | 中汇信息技术（上海）有限公司 | Method and device for generating IMIX standard message and electronic equipment |
| CN110753050B (en) * | 2019-10-22 | 2022-03-04 | 网易（杭州）网络有限公司 | Method and device for generating protocol document, computer storage medium and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007064876A2 (en) * | 2005-12-01 | 2007-06-07 | Firestar Software, Inc. | System and method for exchanging information among exchange applications |
| CN101807207A (en) * | 2010-03-22 | 2010-08-18 | 北京大用科技有限责任公司 | Method for sharing document based on content difference comparison |
| CN102868712A (en) * | 2011-07-06 | 2013-01-09 | 阿里巴巴集团控股有限公司 | Method and system for data transmission |
| CN104281970A (en) * | 2014-10-13 | 2015-01-14 | 中国外汇交易中心 | Message processing method, message processing device and server platform |
-
2016
- 2016-11-21 CN CN201611027572.4A patent/CN106557974B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007064876A2 (en) * | 2005-12-01 | 2007-06-07 | Firestar Software, Inc. | System and method for exchanging information among exchange applications |
| CN101807207A (en) * | 2010-03-22 | 2010-08-18 | 北京大用科技有限责任公司 | Method for sharing document based on content difference comparison |
| CN102868712A (en) * | 2011-07-06 | 2013-01-09 | 阿里巴巴集团控股有限公司 | Method and system for data transmission |
| CN104281970A (en) * | 2014-10-13 | 2015-01-14 | 中国外汇交易中心 | Message processing method, message processing device and server platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106557974A (en) | 2017-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9703675B2 (en) | Structured logging and instrumentation framework | |
| EP1676210B1 (en) | Method and apparatus for handling text and binary mark up languages in a computing device | |
| CN105550206B (en) | The edition control method and device of structured query sentence | |
| CN106557974B (en) | IMIX protocol data processing method and system | |
| CN112307512A (en) | Log desensitization method, device and storage medium | |
| EP3940550A1 (en) | Data compression methods and systems based on key-value store | |
| CN110795697B (en) | Method and device for acquiring logic expression, storage medium and electronic device | |
| CN112925954B (en) | Method and device for querying data in graph database | |
| CN104063401A (en) | Webpage style address merging method and device | |
| WO2022127259A1 (en) | Data cleaning method, apparatus and device, and storage medium | |
| CN112906024A (en) | Data desensitization method, device, storage medium and server | |
| CN111881094B (en) | Method, device, terminal and storage medium for extracting key information in log | |
| CN112256776A (en) | Form repeated submission detection method and device, computer equipment and storage medium | |
| CN108345648B (en) | Method and device for acquiring log information based on columnar storage | |
| CN117171108B (en) | Virtual model mapping method and system | |
| CN113742231A (en) | Protobuf protocol dynamic analysis-based number making method and device | |
| CN112883088B (en) | Data processing method, device, equipment and storage medium | |
| CN112463533A (en) | Log data analysis method and device, electronic device and storage medium | |
| CN107122417B (en) | Database auditing method and device | |
| CN110471918A (en) | A kind of relevant database is synchronized to the method and device of block chain | |
| CN114186958A (en) | Method, computing device and storage medium for exporting list data as spreadsheet | |
| WO2022048464A1 (en) | Data masking method, data masking apparatus and storage device | |
| CN114201376A (en) | Log analysis method and device based on artificial intelligence, terminal equipment and medium | |
| CN112528339A (en) | Data desensitization method based on Cach é database and electronic equipment | |
| CN113704648A (en) | Page data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221116 Address after: 100005 No. 69, inner main street, Dongcheng District, Beijing, Jianguomen Patentee after: AGRICULTURAL BANK OF CHINA Patentee after: Agricultural Bank of China Financial Technology Co.,Ltd. Address before: 100005 No. 69, inner main street, Dongcheng District, Beijing, Jianguomen Patentee before: AGRICULTURAL BANK OF CHINA |
|
| TR01 | Transfer of patent right |