CN112822091A - Message processing method and device - Google Patents

Message processing method and device Download PDF

Info

Publication number
CN112822091A
CN112822091A CN201911125238.6A CN201911125238A CN112822091A CN 112822091 A CN112822091 A CN 112822091A CN 201911125238 A CN201911125238 A CN 201911125238A CN 112822091 A CN112822091 A CN 112822091A
Authority
CN
China
Prior art keywords
message
transaction
state
preparation
writing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911125238.6A
Other languages
Chinese (zh)
Other versions
CN112822091B (en
Inventor
林德强
何小锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN201911125238.6A priority Critical patent/CN112822091B/en
Publication of CN112822091A publication Critical patent/CN112822091A/en
Application granted granted Critical
Publication of CN112822091B publication Critical patent/CN112822091B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/07User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail characterised by the inclusion of specific contents
    • H04L51/18Commands or executable codes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/56Provisioning of proxy services
    • H04L67/568Storing data temporarily at an intermediate stage, e.g. caching
    • H04L67/5682Policies or rules for updating, deleting or replacing the stored data

Abstract

The application provides a message processing method, which comprises the following steps: scanning a preparation message in a buffer queue; if the overtime preparation message is determined, inquiring the transaction state corresponding to the preparation message from the generating equipment; the transaction state is a transaction state recorded when the generation equipment processes the local transaction after sending the transaction message; when the inquired transaction state is determined to be a success state, writing the submission message into a log database; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue; when the inquired transaction state is determined to be a failure state, writing a rollback message into a log database; and deleting the prepare message in the buffer queue. The method can keep the transaction states on the generating device and the intermediate device consistent.

Description

Message processing method and device
Technical Field
The present invention relates to the field of information processing technologies, and in particular, to a method and an apparatus for processing a message.
Background
At present, data interaction is carried out among different applications, in order to achieve the purposes of application decoupling and the like, a message middleware is introduced, an application A is deployed in a first computer room and serves as a producer of a message, and an application B is deployed in a second computer room and serves as a consumer of the message. The producer sends the message to BROKER of the message middleware for temporary storage, and the consumer obtains the message from BROKER to execute corresponding business logic.
On the one hand, the sender needs to be able to store the message to BROKER quickly and reliably, and on the other hand, it needs to be able to support the transaction property and local transaction combination. If the local transaction is successful, the transaction at the BROKER end is submitted, and if the local transaction is failed, the transaction at the BROKER end is rolled back.
The ROKETMQ provides DB-based transaction message functionality. By writing a transaction message sent by a client into the DB first, if the transaction is committed, the message is made visible to the consumer, and if the transaction is rolled back, the message is deleted. This implementation meets the user's needs for transaction messages to some extent.
However, the above implementation has no perfect transaction compensation mechanism: when the local transaction is successful, the transaction is submitted to MQ, and the situation of state inconsistency can be caused if the scene such as failure occurs. The method specifically comprises the following steps: the local transaction is successful, the MQ transaction is unknown, and finally only some default operations can be executed to terminate the transaction, the default operations can be either commit or rollback, and the transaction is inconsistent.
Disclosure of Invention
In view of this, the present application provides a message processing method and apparatus, which can keep the transaction states on the generating device and the intermediate device consistent.
In order to solve the technical problem, the technical scheme of the application is realized as follows:
a method of message processing, the method comprising:
scanning a preparation message in a buffer queue;
if the overtime preparation message is determined, inquiring the transaction state corresponding to the preparation message from the generating equipment; the transaction state is a transaction state recorded when the generation equipment processes the local transaction after sending the transaction message;
when the inquired transaction state is determined to be a success state, writing the submission message into a log database; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue;
when the inquired transaction state is determined to be a failure state, writing a rollback message into a log database; and deleting the prepare message in the buffer queue.
A message processing apparatus, the apparatus comprising: the device comprises a scanning unit, an acquisition unit, a determination unit and a processing unit;
the scanning unit is used for scanning the preparation message in the buffer queue;
the acquiring unit is configured to query, from the generating device, a transaction state corresponding to the preparation message if it is determined that the preparation message scanned by the scanning unit has a timeout preparation message; the transaction state is a transaction state recorded when the generation equipment processes the local transaction after sending the transaction message;
the determining unit is configured to determine whether the transaction state acquired by the acquiring unit is a success state or a failure state;
the processing unit is used for writing a submission message into a log database when the determination unit determines that the inquired transaction state is a success state; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue; when the determining unit determines that the inquired transaction state is a failure state, writing a rollback message into a log database; and deleting the prepare message in the buffer queue.
An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the message processing method when executing the program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the message processing method.
According to the technical scheme, the processing state of the corresponding transaction is obtained by performing timeout processing on the Prepare message in the cache queue, and the corresponding transaction message is processed according to the processing state, so that a transaction compensation mechanism is perfected, and the consistency of the transaction states on the generating device and the intermediate device can be kept.
Drawings
FIG. 1 is a schematic diagram of a message processing flow in a first embodiment of the present application;
FIG. 2 is a schematic diagram of a message processing flow in a second embodiment of the present application;
FIG. 3 is a schematic diagram of an apparatus for implementing the above technique in an embodiment of the present application;
fig. 4 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clearly apparent, the technical solutions of the present invention are described in detail below with reference to the accompanying drawings and examples.
The embodiment of the application provides a message processing method, which obtains the processing state of a corresponding transaction by performing timeout processing on a preparation (Prepare) message in a cache queue, processes the corresponding transaction message according to the processing state, perfects a transaction compensation mechanism, and can keep the consistency of the transaction states on a generating device and an intermediate device.
The message processing system in the embodiment of the application comprises: generating equipment, intermediate equipment and consuming equipment; the production equipment is used as a producer of the message, the intermediate equipment is used as a temporary storage equipment of the message, the consumer is used as a consumer of the message, and the intermediate equipment is provided with a brooker.
The producer generates a message and then sends the message to the intermediate equipment, and the message is temporarily stored by the intermediate equipment; the consuming equipment acquires the message from the intermediate equipment and executes corresponding service logic.
In the embodiment of the application, after the producer sends the transaction message to the sender, the producer processes the local transaction, and when the local transaction is processed, the producer sends a Commit (Commit) message or a rollback (rollback) message to the intermediate device according to a processing result; the processing result of the local transaction is also recorded locally, namely, the processing result is recorded as a transaction state, and the transaction state comprises: success and failure.
In a specific implementation, multiple transactions may need to be maintained, and the transaction state of the transaction is correspondingly recorded by using the transaction ID responded by the intermediate device:
as transaction 1, the state is: success is achieved;
transaction 2, status is failure.
In a specific implementation, the recording may be performed in a table manner or a log manner, and as for a manner of recording a corresponding relationship between the transaction ID and the transaction status, the embodiment of the present application is not limited.
When the intermediate equipment is used for receiving the received message sent by the production equipment, reading and writing in sequence; and copying and storing the data stored to the local to the backup device.
The problem of slow writing performance is solved by sequentially storing the local files and adding the system image file, so that the writing of a single brooker transaction message into the TPS can reach 3W +, which is about 10 times of the existing scheme, and the performance can be linearly improved by transversely expanding the number of brookers for one topic. The copying of files among brokers ensures high availability of the system, and when one Broker has a problem, the system can be switched to another Broker to continue providing services in time.
A conventional message processing procedure is given below, that is, a processing procedure in which the intermediate device normally receives a Commit message or a rollback message sent by the generating device, specifically:
first, the generating device sends a Prepare message to the intermediate device.
Secondly, when the intermediate device receives the Prepare message, storing the Prepare message into a cache queue, and writing the Prepare message into a log database; and when sending the message responding to the Prepare message to the generating device, carrying the transaction ID allocated to the generating device.
And thirdly, when the production equipment receives the response aiming at the Prepare message, the production equipment sends the transaction message by using the transaction ID carried by the Prepare message.
And fourthly, when the intermediate equipment receives the transaction message sent by the generating equipment, writing the transaction message into a log database, and sending a response message to the generating equipment.
And fifthly, after the generating device sends the transaction message, performing local transaction processing, recording the transaction processing result as the transaction state of the transaction, and sending a Commit message or a rollback message to the intermediate device according to the processing result.
When the processing result is successful, sending a Commit message; and when the processing result is failure, sending a rollback message.
Sixthly, when receiving the Commit message sent by the generating equipment, the intermediate equipment writes the Commit message into a log database; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the Prepare message in the cache queue;
when receiving the rollback message, writing the rollback message into a log database; and deleting the Prepare message in the buffer queue.
An index is created for a transaction message, i.e., can be retrieved by a consuming device for relevant processing.
At this point, a completed message processing flow ends.
The following describes in detail a transaction message processing procedure under an abnormal condition in an embodiment of the present application, with reference to the accompanying drawings:
example one
In this embodiment, a case where the middleware does not receive the Commit message and the rollback message and the middleware normally operates is taken as an example.
Referring to fig. 1, fig. 1 is a schematic view illustrating a message processing flow in a first embodiment of the present application. The method comprises the following specific steps:
step 101, the intermediate device scans the Prepare message in the buffer queue.
Step 102, if the intermediate device determines that there is a timeout Prepare message, querying a transaction state corresponding to the Prepare message from the generating device.
Here, determining that the Prepare message times out may be: by setting a timer for each Prepare message in advance, when the timer timing is expired, the Prepare message is still in the buffer queue, and the Prepare message is determined to be expired.
Inquiring the transaction state corresponding to the Prepare message from the generating device, which specifically includes:
the intermediate equipment sends a transaction state inquiry request to the generating equipment; the inquiry request carries a transaction ID of the corresponding transaction.
And when the generation equipment receives the query request, acquiring the transaction state corresponding to the transaction ID stored locally, and responding to the intermediate equipment.
And when the intermediate device receives the response transaction state of the generating device, acquiring the transaction state.
The transaction state is a transaction state recorded when the generation equipment finishes processing a local transaction after sending a transaction message; including success and failure states.
103, when the intermediate device determines that the inquired transaction state is a success state, writing a Commit message into a log database; and creating an index for the transaction, acquiring by the equipment to be consumed, deleting the Prepare message in the cache queue, and ending the process.
Step 104, when the intermediate device determines that the inquired transaction state is a failure state, writing a rollback message into a log database; and deleting the Prepare message in the buffer queue.
No matter what the transaction state is, the corresponding message needs to be written into the log database, and the corresponding Prepare message is deleted, and for the transaction of which the transaction state is a successful state, an index needs to be created for the transaction, where the created index is the position of each message in the log, that is, the position for the consumer device to obtain the transaction message.
The process of specifically establishing the index and the consumer obtaining the corresponding message is the same as the existing implementation and is not described in detail here.
By the compensation mechanism, the consistency of the transaction states in the scene of transaction submission failure can be ensured.
Example two
In this embodiment, the case where the middleware fails and receives no Commit message or rollback message is taken as an example.
Referring to fig. 2, fig. 2 is a schematic view illustrating a message processing flow in the second embodiment of the present application. The method comprises the following specific steps:
step 201, after the failure is recovered, the intermediate device obtains the Prepare message corresponding to the transaction in which the rolback message and the Commit message are not written from the log database, and stores the Prepare message in the cache queue.
If the intermediate equipment fails, the Prepare message in the buffer queue is lost;
after the device failure is recovered, the Prepare message corresponding to the transaction message that is not determined whether the transaction message can be acquired by the consuming device needs to be placed in the cache queue.
Because all the messages corresponding to each transaction are stored in the log database, which Prepare message, no corresponding Commit message or rollback message, can be determined to determine whether the processing of the related message of the transaction is finished, and if no corresponding Commit message or rollback message exists, it is determined that the processing is not finished, and the corresponding Prepare message needs to be put into the cache queue for continuous processing; if there is a corresponding Commit message, or rollback message, it is determined that the process is finished.
In step 202, the intermediate device queries the generating device for the transaction status corresponding to the Prepare message.
Inquiring the transaction state corresponding to the Prepare message from the generating device, which specifically includes:
the intermediate equipment sends a transaction state inquiry request to the generating equipment; the inquiry request carries a transaction ID of the corresponding transaction.
And when the generation equipment receives the query request, acquiring the transaction state corresponding to the transaction ID stored locally, and responding to the intermediate equipment.
And when the intermediate device receives the response transaction state of the generating device, acquiring the transaction state.
The transaction state is a transaction state recorded when the generation equipment finishes processing a local transaction after sending a transaction message; including success and failure states.
Step 203, when the intermediate device determines that the inquired transaction state is successful, writing the Commit message into a log database; and creating an index for the transaction, acquiring by the equipment to be consumed, deleting the Prepare message in the cache queue, and ending the process.
Step 204, when the intermediate device determines that the inquired transaction state is a failure state, writing a rollback message into a log database; and deleting the Prepare message in the buffer queue.
No matter what the transaction state is, the corresponding message needs to be written into the log database, and the corresponding Prepare message is deleted, and for the transaction of which the transaction state is a successful state, an index needs to be created for the transaction, where the created index is the position of each message in the log, that is, the position for the consumer device to obtain the transaction message.
The process of specifically establishing the index and the consumer obtaining the corresponding message is the same as the existing implementation and is not described in detail here.
By the compensation mechanism, the consistency of the transaction state in the scene of transaction submission failure caused by recovery of the failure of the intermediate equipment can be ensured.
Based on the implementation, the intermediate device in the embodiment of the application can provide the capability of transaction messages, reduce external dependence and ensure high availability of transmission; the BROKER transaction message processing performance of the intermediate device is improved, the sending delay is reduced, and the resource utilization rate is improved; and provides a sophisticated transaction compensation mechanism.
In summary, the present application completes a transaction compensation mechanism by performing timeout processing on the Prepare message in the cache queue, or storing the Prepare message corresponding to the unprocessed message in the log database into the cache queue, and acquiring the processing state of the corresponding transaction, and processing the corresponding transaction message according to the processing state, so as to maintain the consistency of the transaction states on the generating device and the intermediate device.
Based on the same inventive concept, the embodiment of the application also provides a message processing device. Referring to fig. 3, fig. 3 is a schematic structural diagram of an apparatus applied to the above technology in the embodiment of the present application. The device includes: a scanning unit 301, an acquisition unit 302, a determination unit 303, and a processing unit 304;
a scanning unit 301, configured to scan a preparation message in a buffer queue;
an obtaining unit 302, configured to query, if it is determined that a preparation message scanned by the scanning unit 301 has a timeout preparation message, a transaction state corresponding to the preparation message from a generating device; the transaction state is a transaction state recorded when the generation equipment processes the local transaction after sending the transaction message;
a determining unit 303, configured to determine whether the transaction status acquired by the acquiring unit 302 is a success status or a failure status;
a processing unit 304, configured to write a commit message into the log database when the determining unit 303 determines that the queried transaction state is a success state; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue; when the determining unit 303 determines that the queried transaction state is a failure state, writing a rollback message into the log database; and deleting the prepare message in the buffer queue.
Preferably, the apparatus further comprises: a receiving unit 305;
the receiving unit is used for receiving a preparation message sent by the generating equipment; receiving a transaction message sent by a generating device;
a processing unit 304, configured to, when the receiving unit receives a preparation message sent by a generating device, store the preparation message in a buffer queue, and write the preparation message into a log database; when the receiving unit receives a transaction message sent by the generating device, the transaction message is written into a log database.
Preferably, the apparatus further comprises: a receiving unit 305;
a receiving unit 305, configured to receive a commit message sent by a generating device, and receive a rollback message sent by the generating device;
the processing unit 304 is further configured to, when the receiving unit 305 receives a commit message sent by the generating device, write the commit message into the log database; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue; when a rollback message sent by a generating device is received, writing the rollback message into a log database; and deleting the prepare message in the buffer queue.
Preferably, the first and second liquid crystal films are made of a polymer,
an obtaining unit 302, configured to obtain, from the log database, a preparation message corresponding to a transaction in which a rollback message and a commit message are not written after a failure of a device in which the unit is located is recovered; and storing the data into a buffer queue; inquiring the transaction state corresponding to the preparation message from the generating equipment;
a determining unit 303, configured to determine whether the transaction status acquired by the acquiring unit 302 is a success status or a failure status;
a processing unit 304, configured to write a commit message into the log database when the determining unit 303 determines that the queried transaction state is a success state; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue; when the determining unit 303 determines that the queried transaction state is a failure state, writing a rollback message into the log database; and deleting the prepare message in the buffer queue.
Preferably, the first and second liquid crystal films are made of a polymer,
the processing unit 304 is further configured to sequentially read and write when the receiving unit 305 receives the message sent by the sender; and copying and storing the data stored to the local to the backup device.
The units of the above embodiments may be integrated into one body, or may be separately deployed; may be combined into one unit or further divided into a plurality of sub-units.
In addition, an electronic device is further provided in an embodiment of the present application, and includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the message processing method when executing the program.
In addition, a computer-readable storage medium is provided in an embodiment of the present application, and a computer program is stored on the computer-readable storage medium, where the computer program is used to implement the steps of the message processing method when executed by a processor.
Fig. 4 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention. As shown in fig. 4, the electronic device may include: a processor (processor)410, a communication Interface 420, a memory (memory)430 and a communication bus 440, wherein the processor 410, the communication Interface 420 and the memory 430 are communicated with each other via the communication bus 440. Processor 510 may call logic instructions in memory 430 to perform the following method:
scanning a preparation message in a buffer queue;
if the overtime preparation message is determined, inquiring the transaction state corresponding to the preparation message from the generating equipment; the transaction state is a transaction state recorded when the generation equipment processes the local transaction after sending the transaction message;
when the inquired transaction state is determined to be a success state, writing a submitted and submitted message into a log database; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue;
when the inquired transaction state is determined to be a failure state, writing a rollback message into a log database; and deleting the prepare message in the buffer queue.
In addition, the logic instructions in the memory 430 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments. The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (12)

1. A method for message processing, the method comprising:
scanning a preparation message in a buffer queue;
if the overtime preparation message is determined, inquiring the transaction state corresponding to the preparation message from the generating equipment; the transaction state is a transaction state recorded when the generation equipment processes the local transaction after sending the transaction message;
when the inquired transaction state is determined to be a success state, writing the submission message into a log database; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue;
when the inquired transaction state is determined to be a failure state, writing a rollback message into a log database; and deleting the prepare message in the buffer queue.
2. The method of claim 1, further comprising:
when a preparation message sent by a generating device is received, storing the preparation message into a buffer queue, and writing the preparation message into a log database;
and when receiving a transaction message sent by the generating equipment, writing the transaction message into a log database.
3. The method of claim 1, further comprising:
when a submission message sent by a generating device is received, writing the submission message into a log database; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue;
when a rollback message sent by a generating device is received, writing the rollback message into a log database; and deleting the prepare message in the buffer queue.
4. The method of claim 1, further comprising:
after the fault is recovered, acquiring preparation messages corresponding to transactions in which the rollback messages and the commit messages are not written from the log database, and storing the preparation messages into a cache queue;
inquiring the transaction state corresponding to the preparation message from the generating equipment;
when the inquired transaction state is determined to be a success state, writing the submission message into a log database; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue;
when the inquired transaction state is determined to be a failure state, writing a rollback message into a log database; and deleting the prepare message in the buffer queue.
5. The method according to any one of claims 1-4, wherein the method further comprises:
when receiving the message sent by the sender, reading and writing in sequence;
and copying and storing the data stored to the local to the backup device.
6. A message processing apparatus, characterized in that the apparatus comprises: the device comprises a scanning unit, an acquisition unit, a determination unit and a processing unit;
the scanning unit is used for scanning the preparation message in the buffer queue;
the acquiring unit is configured to query, from the generating device, a transaction state corresponding to the preparation message if it is determined that the preparation message scanned by the scanning unit has a timeout preparation message; the transaction state is a transaction state recorded when the generation equipment processes the local transaction after sending the transaction message;
the determining unit is configured to determine whether the transaction state acquired by the acquiring unit is a success state or a failure state;
the processing unit is used for writing a submission message into a log database when the determination unit determines that the inquired transaction state is a success state; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue; when the determining unit determines that the inquired transaction state is a failure state, writing a rollback message into a log database; and deleting the prepare message in the buffer queue.
7. The apparatus of claim 6, further comprising: a receiving unit;
the receiving unit is used for receiving a preparation message sent by the generating equipment; receiving a transaction message sent by a generating device;
the processing unit is used for storing the preparation message into a buffer queue and writing the preparation message into a log database when the receiving unit receives the preparation message sent by the generating equipment; when the receiving unit receives a transaction message sent by the generating device, the transaction message is written into a log database.
8. The apparatus of claim 6, further comprising: a receiving unit;
the receiving unit is used for receiving the submission message sent by the generating equipment and receiving the rollback message sent by the generating equipment;
the processing unit is further used for writing a submission message into a log database when the receiving unit receives the submission message sent by the generating equipment; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue; when a rollback message sent by a generating device is received, writing the rollback message into a log database; and deleting the prepare message in the buffer queue.
9. The apparatus of claim 6,
the obtaining unit is further configured to obtain, from the log database, a preparation message corresponding to a transaction in which a rollback message and a commit message are not written after the failure of the device in which the obtaining unit is located is recovered; and storing the data into a buffer queue; inquiring the transaction state corresponding to the preparation message from the generating equipment;
the determining unit is configured to determine whether the transaction state acquired by the acquiring unit is a success state or a failure state;
the processing unit is used for writing a submission message into a log database when the determination unit determines that the inquired transaction state is a success state; creating an index for the transaction, acquiring the index by the equipment to be consumed, and deleting the preparation message in the cache queue; when the determining unit determines that the inquired transaction state is a failure state, writing a rollback message into a log database; and deleting the prepare message in the buffer queue.
10. The apparatus according to any one of claims 6 to 9,
the processing unit is further used for sequentially reading and writing when the receiving unit receives the message sent by the sender; and copying and storing the data stored to the local to the backup device.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1-5 when executing the program.
12. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 6.
CN201911125238.6A 2019-11-18 2019-11-18 Message processing method and device Active CN112822091B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911125238.6A CN112822091B (en) 2019-11-18 2019-11-18 Message processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911125238.6A CN112822091B (en) 2019-11-18 2019-11-18 Message processing method and device

Publications (2)

Publication Number Publication Date
CN112822091A true CN112822091A (en) 2021-05-18
CN112822091B CN112822091B (en) 2023-05-30

Family

ID=75852162

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911125238.6A Active CN112822091B (en) 2019-11-18 2019-11-18 Message processing method and device

Country Status (1)

Country Link
CN (1) CN112822091B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113626457A (en) * 2021-08-19 2021-11-09 福建天晴数码有限公司 Method and system for realizing database and cache consistency by cache deletion retry mechanism
CN113742354A (en) * 2021-09-18 2021-12-03 中国银行股份有限公司 Message confirmation method and device, storage medium and electronic equipment
CN114116262A (en) * 2021-12-02 2022-03-01 北京宇信科技集团股份有限公司 Processing method, device, medium and equipment for distributed asynchronous data communication

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0578406A1 (en) * 1992-07-06 1994-01-12 Digital Equipment Corporation Distributed transaction processing using two-phase commit protocol with presumed-commit without log force
US5781910A (en) * 1996-09-13 1998-07-14 Stratus Computer, Inc. Preforming concurrent transactions in a replicated database environment
US20050262077A1 (en) * 2004-05-21 2005-11-24 Bea Systems, Inc. Logging last resource system
CN103312549A (en) * 2013-06-26 2013-09-18 华为技术有限公司 Transaction management method, device and system
US20160299948A1 (en) * 2015-04-07 2016-10-13 International Business Machines Corporation Database statistics based on transaction state
WO2016168530A1 (en) * 2015-04-16 2016-10-20 Nuodb, Inc. Backup and restore in a distributed database utilizing consistent database snapshots
CN106462601A (en) * 2014-03-31 2017-02-22 亚马逊科技公司 Atomic writes for multiple-extent operations
CN106503257A (en) * 2016-11-15 2017-03-15 北京京东金融科技控股有限公司 Distributed transaction server method and system based on binlog compensation mechanism
US20180101563A1 (en) * 2016-10-10 2018-04-12 Alibaba Group Holding Limited Database processing method, apparatus, and electronic device
CN108009027A (en) * 2017-11-23 2018-05-08 北京百度网讯科技有限公司 Implementation method, device, equipment and the storage medium of queue message uniformity
US20180150503A1 (en) * 2016-11-29 2018-05-31 International Business Machines Corporation QUERY PROCESSING WITH BOUNDED STALENESS FOR TRANSACTIONAL MUTATIONS IN NoSQL DATABASE

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0578406A1 (en) * 1992-07-06 1994-01-12 Digital Equipment Corporation Distributed transaction processing using two-phase commit protocol with presumed-commit without log force
US5781910A (en) * 1996-09-13 1998-07-14 Stratus Computer, Inc. Preforming concurrent transactions in a replicated database environment
US20050262077A1 (en) * 2004-05-21 2005-11-24 Bea Systems, Inc. Logging last resource system
CN103312549A (en) * 2013-06-26 2013-09-18 华为技术有限公司 Transaction management method, device and system
CN106462601A (en) * 2014-03-31 2017-02-22 亚马逊科技公司 Atomic writes for multiple-extent operations
US20160299948A1 (en) * 2015-04-07 2016-10-13 International Business Machines Corporation Database statistics based on transaction state
WO2016168530A1 (en) * 2015-04-16 2016-10-20 Nuodb, Inc. Backup and restore in a distributed database utilizing consistent database snapshots
US20180101563A1 (en) * 2016-10-10 2018-04-12 Alibaba Group Holding Limited Database processing method, apparatus, and electronic device
CN106503257A (en) * 2016-11-15 2017-03-15 北京京东金融科技控股有限公司 Distributed transaction server method and system based on binlog compensation mechanism
US20180150503A1 (en) * 2016-11-29 2018-05-31 International Business Machines Corporation QUERY PROCESSING WITH BOUNDED STALENESS FOR TRANSACTIONAL MUTATIONS IN NoSQL DATABASE
CN108009027A (en) * 2017-11-23 2018-05-08 北京百度网讯科技有限公司 Implementation method, device, equipment and the storage medium of queue message uniformity

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
猎人在吃肉: "《RocketMQ 4.3正式发布后,RocketMQ 分布式事务的介绍》", 《CSDN》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113626457A (en) * 2021-08-19 2021-11-09 福建天晴数码有限公司 Method and system for realizing database and cache consistency by cache deletion retry mechanism
CN113742354A (en) * 2021-09-18 2021-12-03 中国银行股份有限公司 Message confirmation method and device, storage medium and electronic equipment
CN113742354B (en) * 2021-09-18 2024-02-09 中国银行股份有限公司 Message confirmation method and device, storage medium and electronic equipment
CN114116262A (en) * 2021-12-02 2022-03-01 北京宇信科技集团股份有限公司 Processing method, device, medium and equipment for distributed asynchronous data communication
CN114116262B (en) * 2021-12-02 2022-08-23 北京宇信科技集团股份有限公司 Processing method, device, medium and equipment for distributed asynchronous data communication

Also Published As

Publication number Publication date
CN112822091B (en) 2023-05-30

Similar Documents

Publication Publication Date Title
US11614867B2 (en) Distributed storage system-based data processing method and storage device
US9778998B2 (en) Data restoration method and system
US7624112B2 (en) Asynchronously storing transaction information from memory to a persistent storage
CN110968586B (en) Distributed transaction processing method and device
CN112822091B (en) Message processing method and device
US20070078911A1 (en) Replicating data across the nodes in a cluster environment
CN108833479B (en) Data synchronization method and device
CN110753084B (en) Uplink data reading method, cache server and computer readable storage medium
US20080115128A1 (en) Method, system and computer program product for implementing shadow queues for recovery of messages
CN112597249B (en) Synchronous distribution storage method and system for service data
CN110888718A (en) Method and device for realizing distributed transaction
CN111475583B (en) Transaction processing method and device
CA3041211C (en) Utilizing nonce table to resolve concurrent blockchain transaction failure
CN109684128B (en) Cluster overall fault recovery method of message middleware, server and storage medium
WO2021082925A1 (en) Transaction processing method and apparatus
CN109271367A (en) Distributed file system multinode snapshot rollback method and system
CN107168642B (en) Data storage method and system
US8805942B2 (en) Storing and partitioning email messaging data
CN111753141A (en) Data management method and related equipment
JP4280306B2 (en) Log-based data architecture for transaction message queuing systems
CN110175179B (en) Data transmission method and system, service node and storage device
CN112860746B (en) Cache reduction-based method, equipment and system
CN116069868B (en) Distributed flexible transaction processing method and device based on enterprise microservice
CN116991333B (en) Distributed data storage method, device, electronic equipment and storage medium
WO2023246654A1 (en) Data management method and apparatus, system, and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant