CN109086319A - For the high concurrent data processing method and system of transaction data - Google Patents
For the high concurrent data processing method and system of transaction data Download PDFInfo
- Publication number
- CN109086319A CN109086319A CN201810705182.0A CN201810705182A CN109086319A CN 109086319 A CN109086319 A CN 109086319A CN 201810705182 A CN201810705182 A CN 201810705182A CN 109086319 A CN109086319 A CN 109086319A
- Authority
- CN
- China
- Prior art keywords
- data
- user
- request
- trading server
- transaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
Abstract
The invention discloses a kind of high concurrent data processing methods and system for transaction data, this method comprises: receiving user's request that request is traded；Trading server is sent by user's request to handle, and in sending user's request process, user's request is shunted, to send the request of the same user to same trading server and be handled；The transaction feedback data that trading server returns are received, and notify corresponding user；Wherein, the trading server has multiple, and each trading server is respectively connected with a transaction data base for store transaction data, and each transaction data base is only associated with a unique trading server.Present invention can ensure that transaction data uniformity for the treatment of and timeliness, occur inconsistent probability so as to greatly reduce data, can be widely applied in data processing field.
Description
Technical field
The present invention relates to data field of synchronization, more particularly to the high concurrent data processing method for transaction data and are
System.
Background technique
With the development of internet, the quantity of various transaction Internet-based is in explosive growth, in process of exchange,
How to synchronize to transaction data is considerable content.In the treatment process of transaction data, if using database
The mode of affairs is handled, then there are the following problems: 1, under high concurrent scene, deadlock easy to form；2, database
Affairs itself have biggish performance cost, need to expend more resource, and in high concurrent, this performance bottleneck can be more
Add obvious；3, when some affairs cannot in time at the end of, will cause request associated therewith all can not correctly return for a long time, hold
Easily cause avalanche effect；4, when occur multiple servers it is across services transaction when, multiple clothes are unable to satisfy using db transaction
Coordination Treatment requirement between business device.Another mode is handled by the way of distributed transaction, and this mode is deposited
It in the big problem of development difficulty, needs first to be finely divided interface, is abstracted into multiple interfaces, then each interface needs full
The requirement of sufficient idempotence, to adapt to timing-compensation requirement, and that equally there is also network overheads is big for this mode, need to expend compared with
The problems such as more resources.
Generally speaking, current traditional processing mode to transaction data, nothing poor to the process performance of high concurrent data
High concurrent data in method Coordination Treatment process of exchange, it is difficult to guarantee the consistency and timeliness of data.
Explanation of nouns:
Redis:Redis be being write using ANSI C language an of open source, support network, it is memory-based also can be lasting
Log type, the Key-Value database of change, and the application programming interface of multilingual is provided；
Redis pipeline:Redis pipe technology, a function of Redis, for carrying out data stream transmitting.
Socket: two programs on network realize the exchange of data, this connection by a two-way communication connection
One end be known as a socket.
Nginx: being a high performance HTTP and Reverse Proxy and an IMAP/POP3/SMTP service
Device.
Summary of the invention
In order to solve the above technical problems, the object of the present invention is to provide the high concurrent data processings for being directed to transaction data
Method and system.
The present invention solves the first technical solution used by its technical problem:
For the high concurrent data processing method of transaction data, comprising the following steps:
Receive user's request that request is traded；It include User ID and transaction data in user's request；
Trading server is sent by user's request to handle, and in sending user's request process, user is requested
It is shunted, to send the request of the same user to same trading server and be handled；
The transaction feedback data that trading server returns are received, and notify corresponding user；
Wherein, the trading server has multiple, and each trading server is respectively connected with one for store transaction data
Transaction data base, and each transaction data base is only associated with a unique trading server.
Further, trading server is sent in the process of processing for user's request described, especially by data
Multiple users are requested Batch sending to handle to trading server by the form of stream.
Further, the transaction data base uses Redis database, and the method also includes following steps:
Redis pipeline tool is first passed through in advance, establishes the pipeline of data flow for carrying out data stream transmitting.
Further, the transaction feedback data returned in the reception trading server, and the step of notifying corresponding user
Later, further comprising the steps of:
According to the transaction feedback data that trading server returns, the inconsistent situation of data is judged whether there is, if so,
After receiving Client-initiated inquiry request, the inconsistent corresponding transaction data of transaction feedback data of the data is obtained, into
Row retries.
Further, the transaction feedback data returned in the reception trading server, and the step of notifying corresponding user
Later, further comprising the steps of:
When the transaction feedback data returned according to trading server, when judgement has that data are inconsistent, carry out different
Reason condition record, and in the preset time in next morning point, it is recorded according to abnormal conditions, obtains the inconsistent transaction of the data
After the corresponding transaction data of feedback data, asynchronous retry is carried out.
Further, described by way of data flow, request Batch sending at trading server multiple users
The step of reason, specifically:
User's request is successively received, and when the user's request received reaches predetermined number, default will received
Several user's requests, Batch sending is handled to trading server by way of data flow.
Further, described by way of data flow, request Batch sending at trading server multiple users
The step of reason, specifically:
User's request is successively received, and after judgement reaches prefixed time interval, by institute received in prefixed time interval
There is user's request, Batch sending is handled to trading server by way of data flow.
Further, described to send user's request in the step of trading server is handled, by by each user
After User ID in request carries out consistency Hash operation, request to distribute corresponding trading server for each user.
The present invention solves the second technical solution used by its technical problem:
For the high concurrent data processing system of transaction data, including processing server, multiple client, multiple transaction clothes
Business device and multiple transaction data bases, the processing server are connect with multiple client and multiple trading servers respectively, respectively
The trading server is respectively connected with a transaction data base for store transaction data, and each transaction data base is only and uniquely
A trading server it is associated；
The client is used for: receiving user's request that the request of user's input is traded, and user is requested to send
To processing server；
The processing server, for performing the following operations:
Receive user's request that request is traded；
Trading server is sent by user's request to handle, and in sending user's request process, user is requested
It is shunted, to send the request of the same user to same trading server and be handled；
The transaction feedback data that trading server returns are received, and notify corresponding user；
The trading server is used for: after handling the user's request received, by corresponding transaction feedback data
It is sent to processing server.
The present invention solves third technical solution used by its technical problem:
For the high concurrent data processing system of transaction data, comprising:
At least one processor；
At least one processor, for storing at least one program；
When at least one described program is executed by least one described processor, so that at least one described processor is realized
The high concurrent data processing method for transaction data.
The beneficial effects of the present invention are: the present invention, by the request of the same user, is sent out by shunting to user's request
It is sent to same trading server to be handled, the same data can be operated to avoid multiple servers, it is ensured that handed over
The consistency and timeliness of easy data processing, occur inconsistent probability so as to greatly reduce data.
Detailed description of the invention
Fig. 1 is the flow chart of the high concurrent data processing method for transaction data of the invention；
Fig. 2 is the system block diagram of the embodiment one of the high concurrent data processing system for transaction data of the invention；
Fig. 3 is the system block diagram of the embodiment two of the high concurrent data processing system for transaction data of the invention.
Specific embodiment
Referring to Fig.1, the present invention provides a kind of high concurrent data processing methods for transaction data, including following step
It is rapid:
S1, user's request that request is traded is received；It include User ID and transaction data in user's request；
S2, it sends user's request to trading server and handles, and in sending user's request process, user is asked
It asks and is shunted, to send the request of the same user to same trading server and be handled；
S3, the transaction feedback data that trading server returns are received, and notifies corresponding user；
Wherein, the trading server has multiple, and each trading server is respectively connected with one for store transaction data
Transaction data base, and each transaction data base is only associated with a unique trading server.
In the present solution, the relationship of unique association is mutually constituted between trading server and transaction data base, to guarantee every
A trading server has dedicated transaction data base, avoids multiple trading servers and carries out together to the same transaction data base
When operation caused by data it is inconsistent, and the present invention is by shunting user's request, by the request of the same user,
It is sent to same trading server to be handled, the same data can be operated to avoid multiple servers, it is ensured that
So as to greatly reduce data inconsistent probability occurs for transaction data uniformity for the treatment of and timeliness.
It is further used as preferred embodiment, step S2 sends user's request at trading server described
During reason, especially by the form of data flow, Batch sending is requested to handle to trading server multiple users.
By way of data flow, multiple users can be requested to batch, be disposably sent at trading server
Reason, avoid network problem or socket problem from causing data inconsistent, and pipeline of data flow reduce it is frequent with database
Transaction, round-trip delay is extremely low, and greatly avoids the probability of error, and performance becomes higher, to further ensure the one of transaction
Cause property and timeliness.
It is further used as preferred embodiment, the transaction data base uses Redis database, the method also includes
Following steps:
S0, Redis pipeline tool is first passed through in advance, establishes the pipeline of data flow for carrying out data stream transmitting.
The present embodiment establishes pipeline of data flow by Redis pipeline tool, can satisfy data-stream form batch and sends out
The requirement for sending user to request, it is simple and fast.
It is further used as preferred embodiment, further comprising the steps of after the step S3:
According to the transaction feedback data that trading server returns, the inconsistent situation of data is judged whether there is, if so,
After receiving Client-initiated inquiry request, the inconsistent corresponding transaction data of transaction feedback data of the data is obtained, into
Row retries.
In the present embodiment, the mechanism retried is compensated using trigger-type, when the transaction feedback number returned according to trading server
According to, judge whether there is data it is inconsistent when, after judgement receives Client-initiated inquiry request, just retried, if not having
Client-initiated inquiry request is received, then without retrying, to greatly reduce workload and difficulty that compensation retries
Degree, improves the working efficiency of this method.
It is further used as preferred embodiment, further comprising the steps of after the step S3:
When the transaction feedback data returned according to trading server, when judgement has that data are inconsistent, carry out different
Reason condition record, and in the preset time in next morning point, it is recorded according to abnormal conditions, obtains the inconsistent transaction of the data
After the corresponding transaction data of feedback data, asynchronous retry is carried out.
In the present embodiment, judge whether to deposit when the transaction feedback data returned according to trading server using clearance mechanism
When data is inconsistent, line asynchronous is clicked through in the preset time in second day morning and is retried, go to compensate to retry and clear with currency,
It realizes final consistency, also avoids peak period, the working resource of this method can be saved.
It is further used as preferred embodiment, it is described by way of data flow, multiple users are requested into Batch sending
The step of being handled to trading server, specifically:
User's request is successively received, and when the user's request received reaches predetermined number, default will received
Several user's requests, Batch sending is handled to trading server by way of data flow.
It is by batches, disposably being sent out in the form of data flow after reception predetermined number user request in the present embodiment
Trading server is sent to be handled.
It is further used as preferred embodiment, it is described by way of data flow, multiple users are requested into Batch sending
The step of being handled to trading server, specifically:
User's request is successively received, and after judgement reaches prefixed time interval, by institute received in prefixed time interval
There is user's request, Batch sending is handled to trading server by way of data flow.
In the present embodiment, be after intercepting multiple users request by prefixed time interval, in the form of data flow batch,
Disposably trading server is sent to be handled.The method user above-mentioned for choosing predetermined number requests to carry out batch hair
The mode sent can realize the purpose that data flow is sent.Further, two schemes be can be combined with and carry out data intercept
Stream, such as the preferred user's request for choosing predetermined number, the unselected user's request for getting predetermined number after prefixed time interval
When, the received all user's requests of prefixed time interval are sent by data-stream form.
Be further used as preferred embodiment, described in the step S2 by user request be sent to trading server into
In the step of row processing, after the User ID in each user request is carried out consistency Hash operation, asked for each user
Seek the corresponding trading server of distribution.By carrying out consistency Hash operation to User ID, obtained for each user unique corresponding
Character string after, user can be assigned randomly to a trading server and handled according to the character string, subsequent processes
In, it is allocated according to allocation result of the first time to the user.
System embodiment one
Referring to Fig. 2, for the high concurrent data processing system of transaction data, including it is processing server, multiple client, more
A trading server and multiple transaction data bases, the processing server respectively with multiple client and multiple trading servers
Connection, each trading server are respectively connected with a transaction data base for store transaction data, and each transaction data base
It is only associated with a unique trading server；In Fig. 2, n client, m trading server and m transaction data are shared
Library, every group mutually constitutes unique association relationship between trading server and transaction data base；
The client is used for: receiving user's request that the request of user's input is traded, and user is requested to send
To processing server；
The processing server, for performing the following operations:
Receive user's request that request is traded；
Trading server is sent by user's request to handle, and in sending user's request process, user is requested
It is shunted, to send the request of the same user to same trading server and be handled；
The transaction feedback data that trading server returns are received, and notify corresponding user；
The trading server is used for: after handling the user's request received, by corresponding transaction feedback data
It is sent to processing server.
In the present embodiment, trading server preferably uses Nginx server.
The high concurrent data processing system for transaction data of the present embodiment, executable embodiment of the present invention method are mentioned
The high concurrent data processing method for transaction data supplied, any combination implementation steps of executing method embodiment have
The corresponding function of this method and beneficial effect.
System embodiment two
Referring to Fig. 3, for the high concurrent data processing system of transaction data, comprising:
At least one processor 100；
At least one processor 200, for storing at least one program；
When at least one described program is executed by least one described processor 100, so that at least one described processor
100 realize the high concurrent data processing method for transaction data.
The high concurrent data processing system for transaction data of the present embodiment, executable embodiment of the present invention method are mentioned
The high concurrent data processing method for transaction data supplied, any combination implementation steps of executing method embodiment have
The corresponding function of this method and beneficial effect.
It is to be illustrated to preferable implementation of the invention, but the invention is not limited to the implementation above
Example, those skilled in the art can also make various equivalent variations on the premise of without prejudice to spirit of the invention or replace
It changes, these equivalent variation or replacement are all included in the scope defined by the claims of the present application.
Claims (10)
1. being directed to the high concurrent data processing method of transaction data, which comprises the following steps:
Receive user's request that request is traded；It include User ID and transaction data in user's request；
Trading server is sent by user's request to handle, and in sending user's request process, user is requested to carry out
It shunts, to send the request of the same user to same trading server and be handled；
The transaction feedback data that trading server returns are received, and notify corresponding user；
Wherein, the trading server has multiple, and each trading server is respectively connected with a friendship for store transaction data
Easy database, and each transaction data base is only associated with a unique trading server.
2. the high concurrent data processing method according to claim 1 for transaction data, which is characterized in that incited somebody to action described
User's request is sent to trading server in the process of processing, and especially by the form of data flow, multiple users are requested
Batch sending is handled to trading server.
3. the high concurrent data processing method according to claim 1 for transaction data, which is characterized in that the transaction
Database uses Redis database, and the method also includes following steps:
Redis pipeline tool is first passed through in advance, establishes the pipeline of data flow for carrying out data stream transmitting.
4. the high concurrent data processing method according to claim 1 for transaction data, which is characterized in that connect described
The transaction feedback data that trading server returns are received, and after the step of notifying corresponding user, further comprising the steps of:
According to the transaction feedback data that trading server returns, the inconsistent situation of data is judged whether there is, if so, connecing
After receiving Client-initiated inquiry request, the inconsistent corresponding transaction data of transaction feedback data of the data is obtained, carries out weight
Examination.
5. the high concurrent data processing method according to claim 1 for transaction data, which is characterized in that connect described
The transaction feedback data that trading server returns are received, and after the step of notifying corresponding user, further comprising the steps of:
When the transaction feedback data returned according to trading server, when judgement has that data are inconsistent, abnormal feelings are carried out
Condition record, and in the preset time in next morning point, it is recorded according to abnormal conditions, obtains the inconsistent transaction feedback of the data
After the corresponding transaction data of data, asynchronous retry is carried out.
6. the high concurrent data processing method according to claim 2 for transaction data, which is characterized in that described to pass through
The form of data flow, the step of multiple users request Batch sending is handled to trading server, specifically:
User's request is successively received, and when the user's request received reaches predetermined number, by the predetermined number received
User's request, Batch sending is handled to trading server by way of data flow.
7. the high concurrent data processing method according to claim 2 for transaction data, which is characterized in that described to pass through
The form of data flow, the step of multiple users request Batch sending is handled to trading server, specifically:
User's request is successively received, and after judgement reaches prefixed time interval, institute received in prefixed time interval is useful
Family request, Batch sending is handled to trading server by way of data flow.
8. the high concurrent data processing method according to claim 1 for transaction data, which is characterized in that described to use
Family request is sent in the step of trading server is handled, by the way that the User ID in each user request is carried out consistency
After Hash operation, request to distribute corresponding trading server for each user.
9. be directed to transaction data high concurrent data processing system, which is characterized in that including processing server, multiple client,
Multiple trading servers and multiple transaction data bases, the processing server respectively with multiple client and multiple transactional services
Device connection, each trading server are respectively connected with a transaction data base for store transaction data, and each transaction data
Library is only associated with a unique trading server；
The client is used for: receiving user's request that the request of user's input is traded, and user is requested to send everywhere
Manage server；
The processing server, for performing the following operations:
Receive user's request that request is traded；
Trading server is sent by user's request to handle, and in sending user's request process, user is requested to carry out
It shunts, to send the request of the same user to same trading server and be handled；
The transaction feedback data that trading server returns are received, and notify corresponding user；
The trading server is used for: after handling the user's request received, corresponding transaction feedback data being sent
To processing server.
10. being directed to the high concurrent data processing system of transaction data characterized by comprising
At least one processor；
At least one processor, for storing at least one program；
When at least one described program is executed by least one described processor, so that at least one described processor is realized as weighed
Benefit requires 1-8 described in any item for the high concurrent data processing method of transaction data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810705182.0A CN109086319A (en) | 2018-07-02 | 2018-07-02 | For the high concurrent data processing method and system of transaction data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810705182.0A CN109086319A (en) | 2018-07-02 | 2018-07-02 | For the high concurrent data processing method and system of transaction data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109086319A true CN109086319A (en) | 2018-12-25 |
Family
ID=64836883
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810705182.0A Pending CN109086319A (en) | 2018-07-02 | 2018-07-02 | For the high concurrent data processing method and system of transaction data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109086319A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783205A (en) * | 2019-01-03 | 2019-05-21 | 山东浪潮通软信息科技有限公司 | A kind of data final consistency method based on event compensation mechanism |
| CN110009796A (en) * | 2019-04-11 | 2019-07-12 | 北京邮电大学 | Invoice category recognition methods, device, electronic equipment and readable storage medium storing program for executing |
| CN114465974A (en) * | 2022-01-20 | 2022-05-10 | 中国工商银行股份有限公司 | Push message processing method and device, processor and electronic equipment |
-
2018
- 2018-07-02 CN CN201810705182.0A patent/CN109086319A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783205A (en) * | 2019-01-03 | 2019-05-21 | 山东浪潮通软信息科技有限公司 | A kind of data final consistency method based on event compensation mechanism |
| CN110009796A (en) * | 2019-04-11 | 2019-07-12 | 北京邮电大学 | Invoice category recognition methods, device, electronic equipment and readable storage medium storing program for executing |
| CN114465974A (en) * | 2022-01-20 | 2022-05-10 | 中国工商银行股份有限公司 | Push message processing method and device, processor and electronic equipment |
| CN114465974B (en) * | 2022-01-20 | 2024-01-26 | 中国工商银行股份有限公司 | Push message processing method and device, processor and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7966383B2 (en) | Client-server systems and methods for accessing metadata information across a network using proxies | |
| US7415470B2 (en) | Capturing and re-creating the state of a queue when migrating a session | |
| US6976258B1 (en) | Providing quality of service guarantees to virtual hosts | |
| US7606839B2 (en) | Systems and methods for providing client connection fail-over | |
| US8095504B2 (en) | N-way synchronization of computer databases | |
| US7680848B2 (en) | Reliable and scalable multi-tenant asynchronous processing | |
| EP3296870A1 (en) | Cdn-based content management system | |
| US7756949B2 (en) | System of handling a web service call | |
| US7694178B2 (en) | Method, apparatus and computer program product for transaction recovery | |
| US20120284228A1 (en) | User-Defined Parallelization in Transactional Replication of In-Memory Database | |
| CN109086319A (en) | For the high concurrent data processing method and system of transaction data | |
| US20040162836A1 (en) | System and method for altering database requests and database responses | |
| CN103793485B (en) | Client is based on the data cached method for realizing inquiry network data | |
| US7836123B2 (en) | System and method for non-HTTP session based publish/subscribe support using pre-emptive subscriptions | |
| US11741291B2 (en) | Systems and methods for providing error recovery in data transmissions | |
| US20070282899A1 (en) | System and method for managing and distributing assets over a network | |
| CN108881485A (en) | The method for ensureing the high concurrent system response time under big data packet | |
| CN109241194A (en) | The load-balancing method and device of Database Systems based on High-Performance Computing Cluster distribution | |
| CN109361778A (en) | A kind of method and terminal managing session | |
| CN110798495A (en) | Method and server for end-to-end message push in cluster architecture mode | |
| CN105915587B (en) | Content delivery method, system and cache server | |
| CN111131367B (en) | Distributed access service processing method and device | |
| CN110290196A (en) | A kind of content distribution method and device | |
| US6912586B1 (en) | Apparatus for journaling during software deployment and method therefor | |
| US11829810B2 (en) | Single device independent backend interface to a frontend interface of a microservice |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181225 |