CN109274604B - Message processing method and system - Google Patents
Message processing method and system Download PDFInfo
- Publication number
- CN109274604B CN109274604B CN201811145086.1A CN201811145086A CN109274604B CN 109274604 B CN109274604 B CN 109274604B CN 201811145086 A CN201811145086 A CN 201811145086A CN 109274604 B CN109274604 B CN 109274604B
- Authority
- CN
- China
- Prior art keywords
- message
- cluster
- equipment
- message forwarding
- forwarding device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/50—Queue scheduling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Abstract
An embodiment of the present specification provides a method and a system for processing a message, where the method includes: a distributed message processing equipment cluster in a message processing system splits and/or assembles a message from an external service system to obtain a target message and sends the target message to a message forwarding equipment cluster in the message processing system; the message forwarding equipment cluster stores the target message into a message queue of the message forwarding equipment cluster, and provides messages to the outside based on the message sequence of the message queue; the message forwarding equipment cluster sends a message writing request to a first storage equipment cluster in the message processing system, wherein the message writing request is used for requesting the first storage equipment cluster to write the target message; and the first storage equipment cluster responds to the message write-in request and writes in the target message.
Description
Technical Field
The present application relates to the field of electronic technologies, and in particular, to a method and a system for processing a packet.
Background
In the process of building commercial banks, the second generation payment system of the central bank is required to be accessed, and the fund exchange information interaction is carried out with other commercial banks through the second generation payment system of the central bank. However, because the internal system architecture of the commercial bank is greatly different from the system architecture of the second generation payment system, a message processing middleware (also called "message processing system") needs to be added between the commercial bank and the second generation payment system, and a message generated by the business system of the commercial bank is processed by the message processing middleware and then is sent to the second generation payment system. The currently used message processing middleware has long work time and high cost.
In order to solve the above technical problem, a message processing method needs to be provided.
Disclosure of Invention
An object of the embodiments of the present specification is to provide a method and a system for processing a message, where the embodiments of the present specification are implemented as follows:
in a first aspect, a method for processing a packet is provided, where the method includes:
a distributed message processing equipment cluster in a message processing system splits and/or assembles a message from an external service system to obtain a target message and sends the target message to a message forwarding equipment cluster in the message processing system;
the message forwarding equipment cluster stores the target message into a message queue of the message forwarding equipment cluster, and provides messages to the outside based on the message sequence of the message queue;
the message forwarding equipment cluster sends a message writing request to a first storage equipment cluster in the message processing system, wherein the message writing request is used for requesting the first storage equipment cluster to write the target message;
and the first storage equipment cluster responds to the message write-in request and writes in the target message.
In a second aspect, a message processing system is provided, which includes: the distributed message processing device cluster, the message forwarding device cluster and the first storage device cluster;
the distributed message processing equipment cluster is used for splitting and/or assembling a message from an external service system to obtain a target message and sending the target message to the message forwarding equipment cluster;
the message forwarding device cluster is used for storing the target message into a message queue of the message forwarding device cluster and providing messages to the outside based on the message sequence of the message queue;
the message forwarding device cluster is further configured to send a message write request to the first storage device cluster, where the message write request is used to request the first storage device cluster to write the target message;
and the first storage device cluster is used for responding to the message write-in request and writing the target message.
As can be seen from the technical solutions provided in the embodiments of the present specification, the efficiency of message processing can be improved under the condition that normal delivery of a message is ensured by mutual cooperation between the distributed message processing device cluster, the message forwarding device cluster, and the first storage device cluster, and since a database and a dedicated storage device are not required to be used, the processing time and the processing cost can be reduced.
Drawings
In order to more clearly illustrate the embodiments of the present specification or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can be obtained according to the drawings without any creative effort.
FIG. 1 is an architecture diagram of a message processing system according to one embodiment of the present description;
FIG. 2 is a flow diagram of a message processing method according to one embodiment of the present description;
FIG. 3 is a block diagram of a message processing system according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a message processing system according to another embodiment of the present specification;
fig. 5 is a schematic structural diagram of a message processing system according to another embodiment of the present specification.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present specification without any creative effort shall fall within the protection scope of the present specification.
The embodiment of the specification provides a message processing method and a message processing system.
For ease of understanding, some concepts and message processing systems involved in embodiments of the present description will be described first.
Message: a byte stream which is compiled in a convention format (such as XML, JSON and the like) and used for bearing service information is convenient for information interaction among different organizations.
Message Queue (MQ): the software program is a software program for asynchronously transmitting data and is used for storing messages.
The architecture of the message processing system: as shown in fig. 1, the message processing system may include: the distributed message processing device cluster comprises a scheduling device and a message processing device, the message forwarding device cluster comprises a plurality of message forwarding devices, the first storage device cluster comprises a plurality of storage devices, and the cooperative work flow among the devices is specifically introduced in the following message processing method.
Next, a message processing method provided in an embodiment of the present specification is described.
Fig. 2 is a flowchart of a message processing method according to an embodiment of the present specification, and as shown in fig. 2, the method may include the following steps: step 202, step 204, and step 206, wherein,
in step 202, the distributed message processing device cluster in the message processing system splits and/or assembles the message from the external service system to obtain a target message, and sends the target message to the message forwarding device cluster in the message processing system.
In this embodiment, the message processing system is located between the business system of the commercial bank and the second-generation payment system in the central bank, and the message processing system is responsible for receiving the message generated by the business system of the commercial bank and processing the message into a message that can be received by the second-generation payment system, so as to ensure that the second-generation payment system can timely and reliably obtain the message. The external business system refers to a business system of a commercial bank, and the message processing system processes messages generated by the business system of the commercial bank and provides the processed messages to a second-generation payment system of a central bank.
In this embodiment, the distributed packet processing device cluster may be responsible for receiving a packet generated by an external service system, splitting and reassembling the packet from the external service system, assembling the packet into a packet (i.e., a target packet) required by a second-generation payment system, and providing the packet to the packet forwarding device cluster, where the packet is provided to the second-generation payment system by the packet forwarding device cluster.
In this embodiment, the message processing system may further include: and the system inlet equipment is specially responsible for receiving the message from the external service system and forwarding the message to the distributed message processing equipment cluster.
In this embodiment of this specification, a distributed packet processing device cluster may include: the system comprises a scheduling device and a plurality of message processing devices, wherein the scheduling device is used for scheduling the messages from the external service system to the corresponding message processing devices, and the message processing devices are used for splitting and/or assembling the scheduled messages to obtain target messages and sending the target messages to the message forwarding device cluster.
In this embodiment of the present description, in consideration of that a single scheduling device fails once, the whole distributed packet processing device cluster cannot work normally, and in order to improve the disaster tolerance capability of the distributed packet processing device cluster, the distributed packet processing device cluster may include: a scheduling device cluster and a packet processing device cluster, where the scheduling device cluster includes a plurality of scheduling devices, and the packet processing device cluster includes a plurality of packet processing devices, and at this time, the step 202 may specifically include the following steps:
the dispatching equipment cluster dispatches the message from the external service system to the corresponding message processing equipment in the message processing equipment cluster; the message processing equipment in the message processing equipment cluster splits and/or assembles the message from the external service system to obtain a target message, and sends the target message to the message forwarding equipment cluster in the message processing system.
In this case, if one or more scheduling devices in the scheduling device cluster fail, the scheduling device may also perform scheduling work through other scheduling devices in the scheduling device cluster, and the distributed packet processing device cluster may still continue to work without affecting the processing of the packet, and may also perform a load balancing function when scheduling the packet.
In this embodiment of the present specification, an addressing table is stored in a distributed packet processing device cluster, where the addressing table records addresses of available packet forwarding devices in the packet forwarding device cluster and priority information of each address. After the message processing device processes the target message, the message processing device determines available message forwarding devices in the message forwarding device cluster according to the address recorded in the addressing table, and sends the target message to the available message forwarding devices.
In this embodiment of the present description, the addressing table may be dynamically updated according to the available state of the packet forwarding device in the packet forwarding device cluster, so that the available packet forwarding device can be found at the first time, and timely and reliable delivery of the packet is ensured.
In this embodiment, the priority information recorded in the addressing table may include: the message processing equipment preferentially sends the target message to the message forwarding equipment in the same physical machine room.
In step 204, the message forwarding device cluster stores the target message into a message queue of the message forwarding device cluster, and provides the message to the outside based on the message sequence of the message queue; and sending a message write-in request to a first storage device cluster in the message processing system, wherein the message write-in request is used for requesting the first storage device cluster to write in a target message.
In this embodiment of the present description, a packet forwarding device cluster includes a plurality of packet forwarding devices, and each packet forwarding device in the packet forwarding device cluster may correspond to a separate message queue (that is, each packet forwarding device has a message queue), or a plurality of packet forwarding devices may share a message queue.
In order to avoid a single machine failure and a single room failure that may cause the message forwarding system to fail to operate normally, and improve the disaster tolerance capability, in this embodiment of the present specification, a message forwarding device cluster may be designed as a dual-group multi-machine inter-backup, specifically, the message forwarding device cluster may include: the system comprises a first message forwarding device cluster and a second message forwarding device cluster, wherein at least two message forwarding devices in the first message forwarding device cluster are deployed in a first machine room, at least one message forwarding device in the first message forwarding device cluster is deployed in a second machine room, at least two message forwarding devices in the second message forwarding device cluster are deployed in the second machine room, and at least one message forwarding device in the second message forwarding device cluster is deployed in the first machine room.
Further, the first machine room may include: the main message forwarding device and the warm standby message forwarding device in the first message forwarding device cluster may include: a cold standby message forwarding device in the first message forwarding device cluster; the main message forwarding device is responsible for daily operation, the warm-standby message forwarding device is in a task-free operation state and takes over the failed main message forwarding device at any time, and the cold-standby message forwarding device is started when the main message forwarding device and the warm-standby message forwarding device both fail.
The main message forwarding equipment sends a working normal signal to the warm standby message forwarding equipment at regular time in the running process, and the working normal signal is used for indicating that the working state of the main message forwarding equipment is normal; the warm standby message forwarding device sends a detection signal to the main message forwarding device at regular time, and if an abnormal response aiming at the detection signal is received, the unfinished task of the main message forwarding device is executed, wherein the detection signal is used for detecting the working state of the main message forwarding device.
In one example, the first packet forwarding device cluster includes: the main message forwarding device a1, the warm standby message forwarding device a2 and the cold standby message forwarding device a3, and the second message forwarding device cluster comprises: the main message forwarding device B1, the warm standby message forwarding device B2 and the cold standby message forwarding device B3 are arranged in the physical machine room a, and the main message forwarding device a1, the warm standby message forwarding device a2 and the cold standby message forwarding device B3 are arranged in the physical machine room B, and the main message forwarding device B1, the warm standby message forwarding device B2 and the cold standby message forwarding device a3 are arranged in the physical machine room B.
Taking the first packet forwarding device cluster as an example, the main packet forwarding device a1 is responsible for daily operation, and sends a normal operation signal to the warm-standby packet forwarding device a2 to indicate that it is operating normally while processing services. When the main message forwarding device a1 does not send a normal working signal according to the convention, the warm standby message forwarding device a2 will try to send a detection signal to the main message forwarding device a1, and if the main message forwarding device a1 makes a normal response to the detection signal, it indicates that the main message forwarding device a1 can still work only in a short-term abnormality; if the main packet forwarding device a1 makes an abnormal response to the detection signal, it indicates that the main packet forwarding device a1 cannot work normally, and at this time, the warm-up packet forwarding device a2 takes over the work of the main packet forwarding device a 1. Therefore, the warm standby message forwarding equipment can take over the work of the main message forwarding equipment under the condition that the main message forwarding equipment fails.
In order to avoid that the message processing system can still normally operate when the message forwarding device fails, in this embodiment of the present specification, when the message forwarding device cluster stores the target message in the message queue, a message write request may be sent to all storage devices in the first storage device cluster, and the storage device is requested to write the target message in the storage device, so that when the failure of the message forwarding device cluster is recovered, the target message may be obtained from the storage device, and a message service is continuously provided to the outside.
In step 206, the first storage device cluster writes the target packet in response to the packet write request.
In order to avoid that a message processing system cannot normally work due to storage related failures and improve the disaster tolerance capability of the message processing system, in this embodiment of the present specification, a redundancy design may be performed on a first storage device cluster, and correspondingly, the first storage device cluster may include: the number of the storage devices in the first storage device cluster is odd; at this time, when the main storage device determines that more than half of the storage devices in the first storage device cluster finish writing the target message, the main storage device sends a message of successful writing to the message forwarding device cluster; if the storage device in the first storage device cluster fails, all the storage devices in the first storage device cluster stop writing the messages until the main storage device election is completed.
Specifically, a first storage device cluster is built by selecting an odd number (for example, 3 or 5) of storage devices, when the message forwarding device cluster stores a target message in a message queue, a message write request is sent to all the storage devices in the first storage device cluster, and if and only if more than half of the storage devices write the same data, the whole write is completed, and a message with a successful write is returned to the message forwarding device cluster. When a single storage device fails, all the storage devices stop external service at the same time, and election of main storage devices is performed, before the election is completed, the message forwarding device cluster cannot request the storage devices to write data, and then the message processing device cluster is refused to send target messages, and the affected messages are overstocked and are not delivered. The process of selecting primary storage devices ends at the slowest minute level, after which the entire first cluster of storage devices works out again.
It is considered that if the packet processing device cluster and the packet forwarding device cluster fail in the process of processing the packet, the packet needs to be requested again to the external service system. In order to implement the decoupling with the external service system, in this embodiment of the present specification, the message processing system may further include: the second storage equipment cluster is arranged between the external service system and the distributed message processing equipment cluster; and the second storage equipment cluster receives the message from the external service system, stores the message into a distributed file system of the second storage equipment cluster, and sends the message from the external service system to the message processing equipment cluster.
In this embodiment, the message processing system may further include: monitoring the equipment cluster; the monitoring device cluster can collect the working state information of the distributed message processing device cluster, the message forwarding device cluster and the first storage device cluster, and execute corresponding preset type operation according to the collected working state information.
In this embodiment, the preset type of operation may include: sending an early warning message, or sending the available state of each message forwarding device in the message forwarding device cluster to the message processing device cluster so that the message processing device cluster can update the addressing list, or sending a starting instruction to the cold standby message forwarding device when the main message forwarding device and the warm standby message forwarding device both have faults.
As can be seen from the above embodiments, in this embodiment, by the mutual cooperation between the distributed packet processing device cluster, the packet forwarding device cluster, and the first storage device cluster, the packet processing efficiency can be improved while ensuring that the packet can be delivered normally.
Fig. 3 is a schematic structural diagram of a message processing system according to an embodiment of the present disclosure, and as shown in fig. 3, the message processing system 300 may include: a distributed message processing device cluster 302, a message forwarding device cluster 304, and a first storage device cluster 306; wherein the content of the first and second substances,
the distributed packet processing device cluster 302 is configured to split and/or assemble a packet from an external service system to obtain a target packet, and send the target packet to the packet forwarding device cluster 304;
the packet forwarding device cluster 304 is configured to store the target packet into a message queue of the packet forwarding device cluster 304, and provide a packet to the outside based on a packet sequence of the message queue;
the message forwarding device cluster 304 is further configured to send a message write request to the first storage device cluster 306, where the message write request is used to request the first storage device cluster 306 to write the target message;
the first storage device cluster 306 is configured to write the target packet in response to the packet write request.
As can be seen from the above embodiments, in this embodiment, by the mutual cooperation between the distributed packet processing device cluster, the packet forwarding device cluster, and the first storage device cluster, the packet processing efficiency can be improved while ensuring that the packet can be delivered normally.
Optionally, as an embodiment, the distributed packet processing device cluster 302 may include: scheduling equipment clusters and message processing equipment clusters;
the dispatching equipment cluster is used for dispatching the message from the external service system to the corresponding message processing equipment in the message processing equipment cluster;
and the message processing equipment in the message processing equipment cluster is used for splitting and/or assembling the message from the external service system to obtain a target message and sending the target message to the message forwarding equipment cluster in the message processing system.
Optionally, as an embodiment, the packet forwarding device cluster 304 may include: the system comprises a first message forwarding device cluster and a second message forwarding device cluster, wherein at least two message forwarding devices in the first message forwarding device cluster are deployed in a first machine room, at least one message forwarding device in the first message forwarding device cluster is deployed in a second machine room, at least two message forwarding devices in the second message forwarding device cluster are deployed in the second machine room, and at least one message forwarding device in the second message forwarding device cluster is deployed in the first machine room.
Optionally, as an embodiment, the first machine room includes: a main message forwarding device and a warm standby message forwarding device in the first message forwarding device cluster;
the main message forwarding equipment is used for sending a working normal signal to the warm standby message forwarding equipment at regular time in the operation process, and the working normal signal is used for indicating that the working state of the main message forwarding equipment is normal;
the warm standby message forwarding device is configured to send a detection signal to the main message forwarding device at regular time, and if an abnormal response to the detection signal is received, execute a task that the main message forwarding device has not completed, where the detection signal is used to detect a working state of the main message forwarding device.
Optionally, as an embodiment, an addressing table is stored in the distributed packet processing device cluster, and addresses of available packet forwarding devices in the packet forwarding device cluster and priority information of each address are recorded in the addressing table.
Optionally, as an embodiment, the first storage device cluster 306 may include: the number of the storage devices in the first storage device cluster is odd;
the main storage device is configured to send a message that writing is successful to the message forwarding device cluster when it is determined that more than half of the storage devices in the first storage device cluster complete writing of the target message;
if the storage device in the first storage device cluster fails, all the storage devices in the first storage device cluster stop writing messages until the main storage device election is completed.
Optionally, as an embodiment, each of the packet forwarding devices in the packet forwarding device cluster corresponds to a separate message queue.
Fig. 4 is a schematic structural diagram of a message processing system according to an embodiment of the present disclosure, and as shown in fig. 4, the message processing system 400 may include: a second storage device cluster 402, a distributed message processing device cluster 404, a message forwarding device cluster 406, and a first storage device cluster 408; wherein the content of the first and second substances,
the second storage device cluster 402 is configured to receive a packet from an external service system, store the packet in a distributed file system of the second storage device cluster, and send the packet from the external service system to the packet processing device cluster;
the distributed packet processing device cluster 404 is configured to split and/or assemble a packet from an external service system to obtain a target packet, and send the target packet to the packet forwarding device cluster 406;
the packet forwarding device cluster 406 is configured to store the target packet into a message queue of the packet forwarding device cluster 406, and provide a packet to the outside based on a packet sequence of the message queue;
the packet forwarding device cluster 406 is further configured to send a packet write request to the first storage device cluster 408, where the packet write request is used to request the first storage device cluster 408 to write the target packet;
the first storage device cluster 408 is configured to write the target packet in response to the packet write request.
Fig. 5 is a schematic structural diagram of a message processing system according to another embodiment of the present specification, and as shown in fig. 5, the message processing system 500 may include: a distributed message processing device cluster 502, a message forwarding device cluster 504, a first storage device cluster 506, and a monitoring device cluster 508; wherein the content of the first and second substances,
the distributed packet processing device cluster 502 is configured to split and/or assemble a packet from an external service system to obtain a target packet, and send the target packet to the packet forwarding device cluster 504;
the packet forwarding device cluster 504 is configured to store the target packet into a message queue of the packet forwarding device cluster 504, and provide a packet to the outside based on a packet sequence of the message queue;
the message forwarding device cluster 504 is further configured to send a message write request to the first storage device cluster 506, where the message write request is used to request the first storage device cluster 506 to write the target message;
the first storage device cluster 506 is configured to respond to the message write request and write the target message;
the monitoring device cluster 508 is configured to collect the working state information of the distributed packet processing device cluster 502, the packet forwarding device cluster 504, and the first storage device cluster 506, and execute a corresponding operation of a preset type according to the collected working state information.
In short, the above description is only a preferred embodiment of the present disclosure, and is not intended to limit the scope of the present disclosure. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present specification shall be included in the protection scope of the present specification.
The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. One typical implementation device is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Claims (18)
1. A method of message processing, the method comprising:
a distributed message processing equipment cluster in a message processing system splits and/or assembles a message from an external service system to obtain a target message and sends the target message to a message forwarding equipment cluster in the message processing system;
the message forwarding equipment cluster stores the target message into a message queue of the message forwarding equipment cluster, and provides messages to the outside based on the message sequence of the message queue;
the message forwarding equipment cluster sends a message writing request to a first storage equipment cluster in the message processing system, wherein the message writing request is used for requesting the first storage equipment cluster to write the target message;
and the first storage equipment cluster responds to the message write-in request and writes in the target message.
2. The method of claim 1, the cluster of distributed packet processing devices comprising: scheduling equipment clusters and message processing equipment clusters;
the distributed message processing equipment cluster in the message processing system splits and/or assembles the message from the external service system to obtain a target message, and sends the target message to the message forwarding equipment cluster in the message processing system, and the method comprises the following steps:
the dispatching equipment cluster dispatches the message from the external service system to the corresponding message processing equipment in the message processing equipment cluster;
and the message processing equipment in the message processing equipment cluster splits and/or assembles the message from the external service system to obtain a target message, and sends the target message to the message forwarding equipment cluster in the message processing system.
3. The method of claim 1, the cluster of packet forwarding devices comprising: the system comprises a first message forwarding device cluster and a second message forwarding device cluster, wherein at least two message forwarding devices in the first message forwarding device cluster are deployed in a first machine room, at least one message forwarding device in the first message forwarding device cluster is deployed in a second machine room, at least two message forwarding devices in the second message forwarding device cluster are deployed in the second machine room, and at least one message forwarding device in the second message forwarding device cluster is deployed in the first machine room.
4. The method of claim 3, comprising, within the first machine room: a main message forwarding device and a warm standby message forwarding device in the first message forwarding device cluster; the method further comprises the following steps:
the main message forwarding equipment sends a working normal signal to the warm standby message forwarding equipment at regular time in the running process, and the working normal signal is used for indicating that the working state of the main message forwarding equipment is normal;
the warm standby message forwarding device sends a detection signal to the main message forwarding device at regular time, and if an abnormal response aiming at the detection signal is received, the unfinished task of the main message forwarding device is executed, wherein the detection signal is used for detecting the working state of the main message forwarding device.
5. The method of claim 1, wherein an addressing table is stored in the distributed cluster of packet processing devices, and addresses of available packet forwarding devices in the cluster of packet forwarding devices and priority information of each address are recorded in the addressing table.
6. The method of claim 1, the message processing system further comprising: a second cluster of storage devices; the method further comprises the following steps:
and the second storage equipment cluster receives the message from the external service system, stores the message into a distributed file system of the second storage equipment cluster, and sends the message from the external service system to the message processing equipment cluster.
7. The method of claim 1, the first cluster of storage devices comprising: the number of the storage devices in the first storage device cluster is odd; the method further comprises the following steps:
when the main storage device determines that more than half of the storage devices in the first storage device cluster finish writing the target message, the main storage device sends a message of successful writing to the message forwarding device cluster;
if the storage device in the first storage device cluster fails, all the storage devices in the first storage device cluster stop writing messages until the main storage device election is completed.
8. The method of claim 1, the message processing system further comprising: monitoring the equipment cluster; the method further comprises the following steps:
the monitoring equipment cluster collects the working state information of the distributed message processing equipment cluster, the message forwarding equipment cluster and the first storage equipment cluster, and executes corresponding preset type operation according to the collected working state information.
9. The method of claim 1, wherein each message forwarding device in the cluster of message forwarding devices corresponds to a separate message queue.
10. A message processing system, the message processing system comprising: the distributed message processing device cluster, the message forwarding device cluster and the first storage device cluster;
the distributed message processing equipment cluster is used for splitting and/or assembling a message from an external service system to obtain a target message and sending the target message to the message forwarding equipment cluster;
the message forwarding device cluster is used for storing the target message into a message queue of the message forwarding device cluster and providing messages to the outside based on the message sequence of the message queue;
the message forwarding device cluster is further configured to send a message write request to the first storage device cluster, where the message write request is used to request the first storage device cluster to write the target message;
and the first storage device cluster is used for responding to the message write-in request and writing the target message.
11. The message processing system of claim 10, the cluster of distributed message processing devices comprising: scheduling equipment clusters and message processing equipment clusters;
the dispatching equipment cluster is used for dispatching the message from the external service system to the corresponding message processing equipment in the message processing equipment cluster;
and the message processing equipment in the message processing equipment cluster is used for splitting and/or assembling the message from the external service system to obtain a target message and sending the target message to the message forwarding equipment cluster in the message processing system.
12. The packet processing system of claim 10, the packet forwarding device cluster comprising: the system comprises a first message forwarding device cluster and a second message forwarding device cluster, wherein at least two message forwarding devices in the first message forwarding device cluster are deployed in a first machine room, at least one message forwarding device in the first message forwarding device cluster is deployed in a second machine room, at least two message forwarding devices in the second message forwarding device cluster are deployed in the second machine room, and at least one message forwarding device in the second message forwarding device cluster is deployed in the first machine room.
13. The message processing system of claim 12, comprising within the first machine room: a main message forwarding device and a warm standby message forwarding device in the first message forwarding device cluster;
the main message forwarding equipment is used for sending a working normal signal to the warm standby message forwarding equipment at regular time in the operation process, and the working normal signal is used for indicating that the working state of the main message forwarding equipment is normal;
the warm standby message forwarding device is configured to send a detection signal to the main message forwarding device at regular time, and if an abnormal response to the detection signal is received, execute a task that is not completed by the main message forwarding device, where the detection signal is used to detect a working state of the main message forwarding device.
14. The message processing system of claim 10, the distributed cluster of message processing devices having an addressing table stored therein, the addressing table having recorded therein addresses of available message forwarding devices in the cluster of message forwarding devices and priority information for each address.
15. The message processing system of claim 10, further comprising: a second cluster of storage devices;
the second storage device cluster is configured to receive a message from an external service system, store the message in a distributed file system of the second storage device cluster, and send the message from the external service system to the message processing device cluster.
16. The message processing system of claim 10, the first cluster of storage devices comprising: the number of the storage devices in the first storage device cluster is odd;
the main storage device is configured to send a message that writing is successful to the message forwarding device cluster when it is determined that more than half of the storage devices in the first storage device cluster complete writing of the target message;
if the storage device in the first storage device cluster fails, all the storage devices in the first storage device cluster stop writing messages until the main storage device election is completed.
17. The message processing system of claim 10, further comprising: monitoring the equipment cluster;
the monitoring device cluster is used for collecting the working state information of the distributed message processing device cluster, the message forwarding device cluster and the first storage device cluster, and executing corresponding preset type operation according to the collected working state information.
18. The message processing system of claim 10, wherein each message forwarding device of the cluster of message forwarding devices corresponds to a separate message queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811145086.1A CN109274604B (en) | 2018-09-29 | 2018-09-29 | Message processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811145086.1A CN109274604B (en) | 2018-09-29 | 2018-09-29 | Message processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109274604A CN109274604A (en) | 2019-01-25 |
| CN109274604B true CN109274604B (en) | 2021-12-07 |
Family
ID=65198811
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811145086.1A Active CN109274604B (en) | 2018-09-29 | 2018-09-29 | Message processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109274604B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110413686B (en) * | 2019-04-26 | 2024-02-13 | 财付通支付科技有限公司 | Data writing method, device, equipment and storage medium |
| CN111010299B (en) * | 2019-12-17 | 2023-05-26 | 杭州迪普科技股份有限公司 | Method and device for recording message forwarding flow |
| CN112037002B (en) * | 2020-09-03 | 2024-04-23 | 中国银行股份有限公司 | Defragmentation message processing method and defragmentation message processing system based on distributed payment system |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN106878473A (en) * | 2017-04-20 | 2017-06-20 | 腾讯科技（深圳）有限公司 | A kind of message treatment method, server cluster and system |
| CN106953901A (en) * | 2017-03-10 | 2017-07-14 | 重庆邮电大学 | A kind of trunked communication system and its method for improving message transmission performance |
| CN107493308A (en) * | 2016-06-12 | 2017-12-19 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus and distributed apparatus group system for sending message |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150249548A1 (en) * | 2014-02-28 | 2015-09-03 | Tyco Fire & Security Gmbh | Establishing Links Between Sub-Nets |
-
2018
- 2018-09-29 CN CN201811145086.1A patent/CN109274604B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107493308A (en) * | 2016-06-12 | 2017-12-19 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus and distributed apparatus group system for sending message |
| CN106453564A (en) * | 2016-10-18 | 2017-02-22 | 北京京东尚科信息技术有限公司 | Elastic cloud distributed massive request processing method, device and system |
| CN106953901A (en) * | 2017-03-10 | 2017-07-14 | 重庆邮电大学 | A kind of trunked communication system and its method for improving message transmission performance |
| CN106878473A (en) * | 2017-04-20 | 2017-06-20 | 腾讯科技（深圳）有限公司 | A kind of message treatment method, server cluster and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109274604A (en) | 2019-01-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109274604B (en) | Message processing method and system | |
| US20080091895A1 (en) | Cache data backup method and system for computer clustering systems | |
| CN102088490B (en) | Data storage method, device and system | |
| CN108737132B (en) | Alarm information processing method and device | |
| CN107861691B (en) | Load balancing method and device of multi-control storage system | |
| EP3813335B1 (en) | Service processing methods and systems based on a consortium blockchain network | |
| CN113595773B (en) | Information processing method, device, electronic equipment and storage medium | |
| CN111913807A (en) | Event processing method, system and device based on multiple storage areas | |
| CN115378799B (en) | Election method and device in equipment cluster based on PaxosLease algorithm | |
| CN111309746A (en) | Asynchronous parallel data synchronization method and device | |
| CN105868038B (en) | Memory error processing method and electronic equipment | |
| US20080250421A1 (en) | Data Processing System And Method | |
| CN110018986B (en) | Abnormal snapshot identification method and device | |
| CN113596195B (en) | Public IP address management method, device, main node and storage medium | |
| CN114064343A (en) | Abnormal handling method and device for block chain | |
| CN114979188A (en) | Self-healing method and device for edge equipment, electronic equipment and storage medium | |
| CN114666201A (en) | High-availability distributed micro-service architecture | |
| CN114661526A (en) | Data backup method and device | |
| CN114116202A (en) | Parallel computing method and system based on micro-service framework | |
| CN112799835A (en) | Method and system for processing metadata of distributed database system | |
| CN112463514A (en) | Monitoring method and device for distributed cache cluster | |
| JPH05314085A (en) | System for waiting operation mutually among plural computers | |
| CN116860516A (en) | Disaster recovery processing method for big data service and related equipment | |
| CN107038077B (en) | Message flood storage method and device | |
| CN117950921A (en) | Memory fault processing method, memory expansion control device, electronic device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20201010 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman, British Islands Applicant after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman, British Islands Applicant before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20201010 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman, British Islands Applicant after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Applicant before: Alibaba Group Holding Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |