CN114500443B - Message pushing method, device, system, electronic equipment and storage medium - Google Patents

Message pushing method, device, system, electronic equipment and storage medium Download PDF

Info

Publication number
CN114500443B
CN114500443B CN202111619013.3A CN202111619013A CN114500443B CN 114500443 B CN114500443 B CN 114500443B CN 202111619013 A CN202111619013 A CN 202111619013A CN 114500443 B CN114500443 B CN 114500443B
Authority
CN
China
Prior art keywords
message
transaction
target
database
service node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111619013.3A
Other languages
Chinese (zh)
Other versions
CN114500443A (en
Inventor
白建民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Baidu Netcom Science and Technology Co Ltd
Original Assignee
Beijing Baidu Netcom Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Baidu Netcom Science and Technology Co Ltd filed Critical Beijing Baidu Netcom Science and Technology Co Ltd
Priority to CN202111619013.3A priority Critical patent/CN114500443B/en
Publication of CN114500443A publication Critical patent/CN114500443A/en
Application granted granted Critical
Publication of CN114500443B publication Critical patent/CN114500443B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/466Transaction processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/546Message passing systems or structures, e.g. queues
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Transfer Between Computers (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The disclosure provides a message pushing method, device, system, electronic equipment and storage medium, and relates to the fields of intelligent searching, augmented reality and the like. The specific implementation scheme is as follows: caching at least one transaction message generated by any service node in the distributed system to at least one first database, monitoring the cached transaction message in the at least one first database, determining candidate messages meeting a first setting condition from the cached transaction messages, and writing the candidate messages into a second database; and acquiring the target message meeting the second set condition from the candidate message in the second database, placing the target message into a target message queue, and pushing the target message in the target message queue to a service node subscribing the subject of the target message in the distributed system. Therefore, the stable writing of the information in the second database can be ensured, abnormal conditions such as downtime and the like of equipment can be avoided, the situation that the transaction information is lost is avoided, and the timely and effective pushing of the transaction information is ensured.

Description

Message pushing method, device, system, electronic equipment and storage medium
Technical Field
The disclosure relates to the field of computers, in particular to the technical fields of big data, cloud computing and the like, and particularly relates to a message pushing method, a device, a system, electronic equipment and a storage medium.
Background
Message consistency is widely available in distributed systems and is also a strongly dependent application scenario in distributed systems, while transaction messages are a general solution to message consistency. How to push transaction messages produced by message producers in a distributed system to message consumers in the distributed system is very important.
Disclosure of Invention
The disclosure provides a message pushing method, a message pushing device, a system electronic device and a storage medium.
According to an aspect of the present disclosure, there is provided a message pushing method, including:
acquiring at least one transaction message generated by any service node in a distributed system, and caching the at least one transaction message to at least one first database;
monitoring the cached transaction messages in the at least one first database, determining candidate messages meeting a first setting condition from the cached transaction messages, and writing the candidate messages into a second database;
Obtaining target information meeting a second set condition from the candidate information in the second database, and placing the target information into a target information queue;
pushing the target message in the target message queue to a service node in the distributed system subscribing to the subject of the target message.
According to another aspect of the present disclosure, there is provided a message pushing system, including:
at least one first service node in the distributed system for generating at least one transaction message and transmitting the at least one transaction message;
at least one second service node in the distributed system, configured to obtain the at least one transaction message sent by the at least one first service node, and cache the at least one transaction message to a first database;
at least one third service node in the distributed system, configured to monitor the transaction messages cached in the first database in the at least one second service node, to determine candidate messages that meet a first setting condition from the cached transaction messages, and to write the candidate messages into a second database;
at least one fourth service node in the distributed system is configured to obtain, from candidate messages in the second database in the at least one third service node, a target message that meets a second set condition, and place the target message in a target message queue, so as to push the target message in the target message queue to a fifth service node in the distributed system that subscribes to a topic of the target message.
According to still another aspect of the present disclosure, there is provided a message pushing apparatus, including:
the first acquisition module is used for acquiring at least one transaction message generated by any service node in the distributed system;
a caching module, configured to cache the at least one transaction message to at least one first database;
the monitoring module is used for monitoring the cached transaction messages in the at least one first database, determining candidate messages meeting a first setting condition from the cached transaction messages, and writing the candidate messages into a second database;
the second acquisition module is used for acquiring target messages meeting a second set condition from the candidate messages in the second database and placing the target messages into a target message queue;
and the pushing module is used for pushing the target message in the target message queue to a service node subscribing the subject of the target message in the distributed system.
According to still another aspect of the present disclosure, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a message pushing method set forth in the above aspect of the disclosure.
According to yet another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium of computer instructions for causing the computer to perform the message pushing method set forth in the above aspect of the present disclosure.
According to a further aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the message pushing method set forth in the above aspect of the present disclosure.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic diagram of pushing a transaction message by a RocketMQ implementation;
FIG. 2 is a push diagram of a Kafka implementation transaction message;
fig. 3 is a flowchart of a message pushing method according to an embodiment of the disclosure;
fig. 4 is a flow chart of a message pushing method according to a second embodiment of the disclosure;
FIG. 5 is a schematic diagram of an interaction process between a message producer and a message server in an embodiment of the present disclosure;
Fig. 6 is a flow chart of a message pushing method according to a third embodiment of the disclosure;
fig. 7 is a flow chart of a message pushing method according to a fourth embodiment of the present disclosure;
fig. 8 is a flow chart of a message pushing method provided in a fifth embodiment of the disclosure;
fig. 9 is a schematic structural diagram of a message pushing system according to a sixth embodiment of the present disclosure;
fig. 10 is a schematic structural diagram of a message pushing system according to a seventh embodiment of the present disclosure;
fig. 11 is a schematic structural diagram of a message pushing system according to an embodiment eight of the present disclosure;
fig. 12 is a schematic structural diagram of a message pushing device according to a ninth embodiment of the disclosure;
FIG. 13 illustrates a schematic block diagram of an example electronic device that may be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the related art, the pushing of the transaction message may be implemented by the RocketMQ (a distributed message system with a open source, based on a high available distributed cluster technology, providing a low-latency and high-reliability message publishing and subscribing service) in FIG. 1, or by the Kafka (a distributed message queue) in FIG. 2.
Wherein, brooker in fig. 1 refers to: the sharded storage node, brooker master refers to: the sharded storage master node, a brooker slave, refers to: the sharded storage slave node, brooker discover, refers to: a sharded storage node service discovery mechanism; nameserver refers to: and a service discovery server. The zookeeper in FIG. 2 is a distributed, open source distributed application coordination service.
However, the pushing mechanisms of the two transaction messages are both dependent on Nameserver for strong consistency coordination, and the instance migration often needs to rely on manual intervention, so that the operation and maintenance cost is high.
In addition, the zookeeper cluster needs to be maintained, a certain manpower is required to be consumed for a business research and development team, and the whole service is a stateful service and cannot be subjected to rapid horizontal capacity expansion.
In view of at least one of the above problems, the present disclosure proposes a message pushing method, a device, an electronic apparatus, and a storage medium.
Message pushing methods, devices, electronic equipment and storage media of the embodiments of the present disclosure are described below with reference to the accompanying drawings.
Fig. 3 is a flowchart of a message pushing method according to an embodiment of the present disclosure.
The embodiment of the disclosure is exemplified by the message pushing method being configured in a message pushing device, and the message pushing device can be applied to any electronic equipment with computing capability so that the electronic equipment can execute the message pushing function.
As shown in fig. 3, the message pushing method may include the following steps:
step 301, obtaining at least one transaction message generated by any service node in the distributed system, and caching the at least one transaction message in at least one first database.
In the embodiment of the disclosure, the first database may be a stable database, for example, the first database may be a Bolt database (a simple, fast and stable database using Key-Value Key/Value database developed by Go language). Of course, the first database may be another database, which is not limited in this disclosure.
In embodiments of the present disclosure, at least one transaction message generated by any service node in the distributed system, which may be referred to as a message producer, may be obtained. After the transaction message is acquired, the transaction message may be cached in at least one first database.
Step 302, monitoring cached transaction messages in at least one first database to determine candidate messages satisfying a first set condition from the cached transaction messages, and writing the candidate messages into a second database.
In the embodiment of the present disclosure, the first setting condition may be preset according to an actual scene and a requirement.
In an embodiment of the present disclosure, the second database is a different database than the first database, and is used for storing transaction messages to be pushed. For example, the second database may be a centrally stored database.
In the embodiment of the disclosure, the transaction messages cached in the first database may be monitored to determine candidate messages meeting the first setting condition from the transaction messages cached in the first database, and the candidate messages are written into the second database.
As a possible implementation manner, in order to avoid the situation that the message is lost due to an abnormal reason such as a downtime of the device, the second databases may be redundantly set, that is, the number of the second databases may be at least one, for example, the second databases in step 302 may be the master second database, and the message written in the master second database may be sent to the slave second database for backup. So that when the primary second database is abnormal, the message can be obtained from the secondary database later.
And 303, acquiring the target message meeting the second setting condition from the candidate messages in the second database, and placing the target message into a target message queue.
In the embodiment of the present disclosure, the second setting condition is preset.
In the embodiment of the disclosure, the target message meeting the second setting condition may be obtained from the candidate messages in the second database, and the target message is put into the constructed target message queue.
Step 304, pushing the target message in the target message queue to a service node in the distributed system subscribing to the topic of the target message.
In embodiments of the present disclosure, a target message in a target message queue may be pushed to a service node (which may be referred to as a message consumer) in a distributed system that subscribes to the subject of the target message.
As an application scenario, taking the example that user 1 purchases XX member on the client, after user 1 completes the payment operation, the member service node may open member for user 1, and the member service node may act as a message producer, send a transaction message to the target message queue, where the transaction message is used to indicate that user 1 has become a member. After ensuring that user 1 in the transaction message is a member and the database in the meeting service node is aligned, the transaction message in the target message queue can be pushed to the marketing service node (message consumer), so that the marketing service node can push movie tickets, various benefits and the like to the client logged in by user 1.
According to the message pushing method, at least one transaction message generated by any service node in a distributed system is cached to at least one first database, the cached transaction message in the at least one first database is monitored, so that candidate messages meeting a first setting condition are determined from the cached transaction messages, and the candidate messages are written into a second database; and acquiring the target message meeting the second set condition from the candidate message in the second database, placing the target message into a target message queue, and pushing the target message in the target message queue to a service node subscribing the subject of the target message in the distributed system. Therefore, before the transaction information is written into the second database, each transaction information is cached through the first database, so that the stable writing of the information in the second database can be ensured, and the situation that the second database cannot be normally used due to the fact that the information with higher flow is written into the second database in unit time in a high concurrency scene is avoided. And the transaction information is stored by the second database, so that abnormal conditions such as downtime and the like of equipment can be avoided, and the situation that the transaction information is lost is avoided, and timely and effective pushing of the transaction information is ensured.
It should be noted that, in the technical solution of the present disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing, etc. of the personal information of the user are all performed on the premise of proving the consent of the user, and all conform to the rules of the related laws and regulations, and do not violate the popular regulations of the public order.
In order to clearly illustrate how the transaction messages are cached in the above embodiments of the present disclosure, the present disclosure also proposes a message pushing method.
Fig. 4 is a flow chart of a message pushing method according to a second embodiment of the disclosure.
As shown in fig. 4, the message pushing method may include the steps of:
step 401, obtaining at least one transaction message generated by any service node in a distributed system.
The implementation of step 401 may refer to the implementation of any embodiment of the present disclosure, which is not described herein.
Step 402, for each acquired transaction message, setting an identification of the transaction message as waiting, and storing the transaction message identified as waiting in at least one first database.
In the embodiment of the disclosure, for each transaction message acquired from any service node, the identification of the transaction message may be set to wait, at this time, the transaction message may be called a half message, and the transaction message identified as waiting is stored in at least one first database.
Step 403, a transaction review request is sent to any service node, where the review request is used to query whether the transaction message can be pushed.
In an embodiment of the present disclosure, a transaction review request may be sent to any of the service nodes, where the transaction review request is used to query whether the transaction message can be pushed.
And step 404, receiving a transaction review response sent by any service node, wherein the transaction review response is generated by any service node in response to the transaction review request according to the execution result of the target transaction on which the transaction message depends.
In the embodiment of the present disclosure, after receiving the transaction review request by any one of the service nodes, a transaction review response may be generated according to an execution result of a target transaction on which the transaction message depends in response to the transaction review request, and the transaction review response is sent.
Step 405, in the case that the transaction review response includes the first indication information, updating the identifier of the transaction message in at least one first database to be deliverable according to the first indication information, where the first indication information is generated after any service node determines that the target transaction execution is successful, and is used to indicate that the transaction message can be pushed.
In this embodiment of the present disclosure, after receiving a transaction reply response sent by any one of the service nodes, it may be determined whether the transaction reply response includes first indication information, and in the case that the transaction reply response includes the first indication information, an identifier of the transaction message in at least one first database may be updated to be deliverable, where the first indication information is generated after the any one of the service nodes determines that the target transaction is executed successfully, and is used to indicate that the transaction message may be pushed.
In a possible implementation manner of the embodiment of the present disclosure, when any service node determines that the target transaction on which the transaction message depends fails to execute, the any service node may generate second indication information, where the second indication information is used to indicate that the transaction message cannot be pushed, and correspondingly, in a case where the transaction review response includes the second indication information, the transaction message in the first data may be deleted. Therefore, when the execution of the target transaction on which the transaction message depends fails, the transaction message is deleted, namely the transaction message is not pushed to the message consumer, so that the processing burden can be reduced, and the processing speed can be increased.
Step 406, monitoring the cached transaction messages in the at least one first database to determine candidate messages satisfying the first set condition from the cached transaction messages, and writing the candidate messages to the second database.
Step 407, obtaining the target message meeting the second setting condition from the candidate messages in the second database, and placing the target message in the target message queue.
In any of the embodiments of the present disclosure, the target message identified as deliverable may be obtained from the candidate message in the second database. That is, the transaction message identified as waiting and/or the transaction message identified as deliverable may be included in the second database, which in the present disclosure may be identified as a deliverable transaction message as a target message to be pushed.
In any of the embodiments of the present disclosure, in order to ensure the integrity and consistency of the data in the second database, the target message identified as deliverable may be obtained from the candidate messages in the second database based on a transaction isolation mechanism.
Step 408, pushing the target message in the target message queue to a service node in the distributed system subscribing to the topic of the target message.
The execution of steps 406 to 408 may refer to the execution of any embodiment of the present disclosure, and will not be described herein.
As an application scenario, the method is applied to a message server for illustration, and the interaction procedure between a message producer and the message server can be as shown in fig. 5. After the message producer sends a transaction message, the message producer may execute logic of a target transaction (or referred to as a local transaction) on which the transaction message depends, and after receiving the transaction message and persistence success of the transaction message, the message server may send a confirmation message to the message producer, where the confirmation message is used to indicate that the sending was successful, and at this time, the transaction message is a half message.
The message producer submits a secondary acknowledgement (submit Commit state or Rollback state) to the message server according to the execution result of the target transaction, the message server marks the transaction message as deliverable when receiving the submit state, the message consumer can receive the transaction message marked as deliverable, and the message server deletes the transaction message when receiving the Rollback state, and the message consumer will not receive the transaction message.
In the special cases of network disconnection, application restarting and the like of the message producer, the secondary confirmation submitted by the message producer does not reach the message server, at this time, the message server can wait for a set time period, then send a transaction review request to the message producer, after receiving the transaction review request, the message producer can check the execution result of the target transaction on which the transaction message depends, if the message producer determines that the execution result of the target transaction is successful in executing the target transaction, a transaction review response (for example, the first indication information can comprise a Commit command state) containing first indication information is sent to the message server, the message server can mark the transaction message as deliverable after receiving the transaction review response, and the message consumer can receive the transaction message marked as deliverable.
If the message producer determines that the execution result of the target transaction is that the execution of the target transaction fails, a transaction review response containing second indication information (for example, the second indication information may include a Rollback status) is sent to the message server, and after receiving the transaction review response, the message server may delete the transaction message, and the message consumer will not receive the transaction message.
According to the message pushing method, for each acquired transaction message, the identification of the transaction message is set to be waiting, and the transaction message identified as waiting is stored in the at least one first database; sending a transaction review request to any service node, wherein the review request is used for inquiring whether the transaction message can be pushed; receiving a transaction review response sent by any service node, wherein the transaction review response is generated by the any service node according to an execution result of a target transaction on which the transaction message depends in response to the transaction review request; and under the condition that the transaction review response comprises first indication information, updating the identification of the transaction message in the at least one first database to be deliverable according to the first indication information, wherein the first indication information is generated after the any service node determines that the target transaction is successfully executed and is used for indicating that the transaction message can be pushed. Therefore, through a transaction review mechanism, whether the target transaction relied on by the transaction message is successfully executed or not can be effectively determined, so that the transaction message is stored under the condition that the execution is successful, and the effective storage and pushing of the transaction message can be ensured.
In order to clearly illustrate how candidate information is written into the second database in the present disclosure, the present disclosure also proposes a message pushing method.
Fig. 6 is a flow chart of a message pushing method according to a third embodiment of the present disclosure.
As shown in fig. 6, the message pushing method may include the steps of:
step 601, at least one transaction message generated by any service node in the distributed system is obtained, and the at least one transaction message is cached in at least one first database.
The execution of step 601 may refer to the execution of any embodiment of the present disclosure, which is not described herein.
In step 602, transaction messages cached in at least one first database are monitored, and in response to reaching a set point in time, transaction messages not higher than a set number are acquired from the transaction messages cached in the at least one first database and used as candidate messages.
In the embodiment of the present disclosure, the set number is preset, for example, the set number may be 100, 200, 300, or the like.
In the embodiment of the disclosure, the transaction messages cached in the first database can be monitored, and in response to reaching a set time point, the transaction messages with the quantity not higher than the set quantity are acquired from the transaction messages cached in the first database and are taken as candidate messages.
As an example, a set number of transaction messages not higher than a set number may be acquired from the transaction messages cached in the at least one first database periodically (e.g., every second) and used as candidate messages.
Step 603, starting a first asynchronous thread.
Step 604, by the first asynchronous thread, writes the candidate message to the second database.
In an embodiment of the present disclosure, a first asynchronous thread may be started by which candidate messages are written to a second database.
As an example, taking the first asynchronous thread as an asynchronous aggregation and warehousing thread, candidate messages meeting the first set condition (fixed time + fixed number) can be written into the second database in batches by starting the asynchronous aggregation and warehousing thread.
Step 605, obtain the target message meeting the second set condition from the candidate messages in the second database, and put the target message into the target message queue.
Step 606, pushing the target message in the target message queue to a service node in the distributed system subscribing to the topic of the target message.
The execution of steps 605 to 606 may refer to the execution of any embodiment of the present disclosure, and will not be described herein.
According to the message pushing method, the transaction messages, which are not higher than the set number, in the transaction messages cached in the first database are written into the second database at the set time point, so that the writing flow of the second database can be reduced, and the stable writing of the messages in the second database is ensured. In addition, the transaction information is written into the second database through the asynchronous thread, so that the writing efficiency and writing performance of the information can be improved.
In order to clearly illustrate how a target message is pushed to a service node subscribing to the subject of the target message in the present disclosure, the present disclosure also proposes a message pushing method.
Fig. 7 is a flow chart of a message pushing method according to a fourth embodiment of the present disclosure.
As shown in fig. 7, the message pushing method may include the following steps:
step 701, obtaining at least one transaction message generated by any service node in the distributed system, and caching the at least one transaction message in at least one first database.
Step 702, monitoring the cached transaction messages in at least one first database to determine candidate messages satisfying a first set condition from the cached transaction messages, and writing the candidate messages to a second database.
Step 703, obtaining the target message meeting the second setting condition from the candidate messages in the second database, and placing the target message in the target message queue.
The execution of steps 701 to 703 may refer to the execution of any embodiment of the present disclosure, and will not be described herein.
Step 704, determining a topic corresponding to each target message in the target message queue.
In embodiments of the present disclosure, each message may have a corresponding topic (topic), and the topic corresponding to each target message in the target message queue may be determined.
Step 705, the second asynchronous thread is started.
Step 706, pushing, by the second asynchronous thread, each target message in the target message queue to a service node subscribing to the topic of each target message.
In embodiments of the present disclosure, a second asynchronous thread may be started by which each target message in the target message queue is pushed to a service node subscribing to the subject of each target message.
As an example, taking the second asynchronous thread as an asynchronous push thread for example, each target message in the target message queue may be pushed to a service node subscribing to the topic of the corresponding target message by starting the asynchronous push thread.
According to the message pushing method, the target message is sent in the form of the message queue, only one thread is occupied, memory occupation can be reduced, and the condition that the target message is sent in a disordered mode can be prevented. In addition, through asynchronous threads, each target message in the target message queue is pushed, so that the pushing efficiency of the message can be improved.
In one possible implementation manner of the embodiment of the present disclosure, in order to ensure that the target message is effectively and timely pushed to the service node subscribed to the target message, when the target message fails to be sent, the target message may be pushed to the service node again, and the detailed description of the foregoing procedure is described below in connection with fig. 8.
Fig. 8 is a flow chart of a message pushing method provided in a fifth embodiment of the disclosure.
As shown in fig. 8, the message pushing method may include the steps of:
step 801, at least one transaction message generated by any service node in the distributed system is obtained, and the at least one transaction message is cached in at least one first database.
Step 802, monitoring cached transaction messages in at least one first database to determine candidate messages satisfying a first set condition from the cached transaction messages, and writing the candidate messages to a second database.
Step 803, obtaining the target message meeting the second setting condition from the candidate messages in the second database, and placing the target message in the target message queue.
Step 804, pushing the target message in the target message queue to a service node in the distributed system subscribing to the topic of the target message.
The execution of steps 801 to 804 may refer to the execution of any of the above embodiments, and will not be described herein.
Step 805, for any target message in the target message queue, after pushing any target message to the target service node subscribed to any target message, counting the pushed duration of any target message, and marking the push status of any target message as the pushed status.
In the embodiment of the present disclosure, for any target message in the target message queue, after the target service node subscribed to the any target message is noted as pushing the any target message in the present disclosure, the pushed duration of the any target message may be counted, and the pushing state of the any target message may be marked as the pushed state.
Step 806, if the pushed time reaches the time threshold corresponding to any target message, determining that the pushing of any target message fails if the response message sent by the target service node is not received, where the response message is used to indicate that any target message is received.
In the embodiment of the disclosure, each target message corresponds to a duration threshold, wherein the duration threshold is related to the number of times of pushing failure of the corresponding target message.
In the embodiment of the present disclosure, under the condition that the pushed time of any one of the target messages reaches a time duration threshold corresponding to any one of the target messages, whether a response message sent by the target service node is received may be determined, where the response message is used to indicate that the target service node receives any one of the target messages.
In the embodiment of the present disclosure, if a response message sent by a target service node is received, it is determined that any one of the target messages is successfully sent, at this time, any one of the target messages may be deleted, and if a response message sent by the target service node is not received, it is determined that any one of the target messages is failed to be sent.
In one possible implementation manner of the embodiment of the present disclosure, the duration threshold corresponding to each target message may be in a forward relationship with the number of times that the target message fails to be pushed, that is, in the present disclosure, the number of times that any target message fails to be pushed may be determined, and according to the number of times, the duration threshold corresponding to any target message is updated, where the updated duration threshold is in a forward relationship with the number of times.
As an example, when the target message is pushed for the first time, the duration threshold corresponding to the target message may be set, when the number of times of pushing the target message is 1, the duration threshold corresponding to the target message may be extended, when the number of times of pushing the target message is 2, the duration threshold corresponding to the target message may be extended again, and so on.
That is, in this disclosure, considering that the processing capacities of different message consumers are different, for example, message consumer 1 can process ten thousand messages in one second, message consumer 2 can process one thousand messages in one second, and push a message to a message consumer with lower processing capacity, a situation that the message consumer cannot receive the message in a short time may occur, which leads to timeout of message transmission or failure of transmission. Therefore, in order to ensure that the target message is effectively and timely pushed to the service node subscribed to the target message, the duration threshold corresponding to the target message may be in a forward relationship with the number of times the target message fails to be pushed.
Step 807, update the push status of any target message to a push failure status.
In the embodiment of the present disclosure, in the case that any one of the target messages fails to be pushed, the push status of the any one of the target messages may be updated to be a push failure status.
Step 808, re-adding any target message in the failed push state to the target message queue to re-push any target message.
In the embodiment of the disclosure, any target message that fails to be pushed may be re-added to the target message queue, so as to be pushed again. For example, the target service node may be re-pushed with any target message, or may send any target message to a service node of the same type as the target service node, which is not limited in this disclosure.
According to the message pushing method, when the target message fails to be pushed, the target message which fails to be pushed is added to the target message queue again, so that the target message can be effectively and timely pushed to the service node subscribing the target message.
In order to implement the above embodiment, the present disclosure further provides a message pushing system.
Fig. 9 is a schematic structural diagram of a message pushing system according to a sixth embodiment of the present disclosure.
As shown in fig. 9, the message pushing system may include:
at least one first service node 910 in the distributed system is configured to generate at least one transaction message and transmit the at least one transaction message.
At least one second service node 920 in the distributed system is configured to obtain at least one transaction message sent by at least one first service node, and cache the at least one transaction message in the first database.
At least one third service node 930 in the distributed system is configured to monitor the cached transaction messages in the first database in the at least one second service node to determine candidate messages from the cached transaction messages that satisfy the first set condition, and write the candidate messages to the second database.
At least one fourth service node 940 in the distributed system, configured to obtain a target message meeting the second setting condition from the candidate messages in the second database in the at least one third service node, and place the target message in a target message queue, so as to push the target message in the target message queue to a fifth service node 950 in the distributed system subscribing to the subject of the target message.
In fig. 9, the number of the first service node 910, the second service node 920, the third service node 930, the fourth service node 940, and the fifth service node 950 is only 1, and in actual application, there may be one or more service nodes, which is not limited in this disclosure.
It should be noted that, in any of the foregoing embodiments of fig. 3 to fig. 8, the explanation of the message pushing method is also applicable to the message pushing system, and the implementation principle is similar, which is not repeated herein.
As an example, taking the first service node as a message producer and the fifth service node as a message consumer as an example, the structure of the message push system may be as shown in fig. 10.
Wherein the transaction message generated by the message producer is randomly sent to the second service nodes, and each second service node has a first database (such as a database, DB) built therein to temporarily and permanently store the transaction message. Meanwhile, an asynchronous aggregation and warehousing thread can be started, and transaction messages (fixed time and fixed number) meeting the first set condition in the first database are written into the second database in the third service node in batches.
The fourth service node may obtain messages (i.e. full messages) satisfying the second set condition in batch from the second database, and construct a push queue of FIFO (First In First Out ) in the memory of the second service node, which is referred to as a target message queue in the present disclosure, and start an asynchronous push thread to push the target message in the target message queue to the fifth service node subscribing to the subject of the target message.
As a possible implementation manner, the second service node and the fourth service node in fig. 10 are stateless services, so that service operation and maintenance of the whole message pushing system are facilitated, and horizontal expansion of the message pushing system is facilitated. For example, the structure of the message push system may become as shown in fig. 11.
The second service node randomly sends the received transaction information to other second service nodes (more than 2 service nodes can be configured in the configuration file) to perform data copy, so that abnormal situations such as dead halt and the like of a machine where the second service node is located can be prevented, and the situation of data loss can be prevented.
The messages which are pulled in batches from the second database by the fourth service node can be stored in the memory, and if the machine in which the fourth service node is located is in a dead halt abnormal condition to cause the condition of data loss, the messages can be obtained from the memory as much as possible again, and the data loss can not be really caused. The fourth service node is provided with an asynchronous thread, and the message with overtime pushing or failed pushing can be reset to be pushed through the asynchronous thread.
The message pushing system of the embodiment of the disclosure caches at least one transaction message generated by any service node in a distributed system to at least one first database, monitors the cached transaction message in the at least one first database, determines candidate messages meeting a first setting condition from the cached transaction messages, and writes the candidate messages into a second database; and acquiring the target message meeting the second set condition from the candidate message in the second database, placing the target message into a target message queue, and pushing the target message in the target message queue to a service node subscribing the subject of the target message in the distributed system. Therefore, before the transaction information is written into the second database, each transaction information is cached through the first database, so that the stable writing of the information in the second database can be ensured, and the situation that the second database cannot be normally used due to the fact that the information with higher flow is written into the second database in unit time in a high concurrency scene is avoided. And the transaction information is stored by the second database, so that abnormal conditions such as downtime and the like of equipment can be avoided, and the situation that the transaction information is lost is avoided, and timely and effective pushing of the transaction information is ensured.
Corresponding to the message pushing method provided by the embodiments of fig. 3 to 8, the present disclosure further provides a message pushing device, and since the message pushing device provided by the embodiments of the present disclosure corresponds to the message pushing method provided by the embodiments of fig. 3 to 8, the implementation of the message pushing method is also applicable to the message pushing device provided by the embodiments of the present disclosure, which is not described in detail in the embodiments of the present disclosure.
Fig. 12 is a schematic structural diagram of a message pushing device according to a ninth embodiment of the disclosure.
As shown in fig. 12, the message pushing apparatus 1200 may include: the first acquisition module 1210, the caching module 1220, the monitoring module 1230, the second acquisition module 1240, and the pushing module 1250.
The first acquiring module 1210 is configured to acquire at least one transaction message generated by any service node in the distributed system.
The caching module 1220 is configured to cache at least one transaction message to at least one first database.
The monitoring module 1230 is configured to monitor the cached transaction messages in the at least one first database, determine candidate messages that satisfy the first setting condition from the cached transaction messages, and write the candidate messages to the second database.
The second obtaining module 1240 is configured to obtain a target message that meets the second setting condition from the candidate messages in the second database, and place the target message in the target message queue.
A pushing module 1250 is configured to push the target message in the target message queue to a service node in the distributed system that subscribes to the topic of the target message.
In one possible implementation of the embodiment of the disclosure, the buffering module 1220 may include:
and the processing unit is used for setting the identification of the transaction message as waiting for each acquired transaction message and storing the transaction message identified as waiting into at least one first database.
And the sending unit is used for sending a transaction review request to any service node, wherein the review request is used for inquiring whether the transaction message can be pushed.
The receiving unit is used for receiving a transaction review response sent by any service node, wherein the transaction review response is generated by any service node according to the execution result of a target transaction relied on by the transaction message in response to the transaction review request.
And the updating unit is used for updating the identification of the transaction message in at least one first database into deliverable according to the first indication information when the transaction review response contains the first indication information, wherein the first indication information is generated after any service node determines that the target transaction execution is successful and is used for indicating that the transaction message can be pushed.
In one possible implementation of the embodiment of the disclosure, the buffering module 1220 may further include:
and the deleting unit is used for deleting the transaction message in at least one first database under the condition that the transaction review response contains second instruction information, wherein the second instruction information is generated after any service node determines that the target transaction fails to execute and is used for indicating that the transaction message cannot be pushed.
In one possible implementation manner of the embodiment of the present disclosure, the second obtaining module 1240 is specifically configured to:
based on the transaction isolation mechanism, a target message identified as deliverable is obtained from the candidate messages in the second database.
In one possible implementation of the embodiments of the present disclosure, the monitoring module 1230 is specifically configured to:
monitoring transaction messages cached in at least one first database, and responding to the arrival of a set time point, acquiring transaction messages not higher than a set number from the transaction messages cached in the at least one first database and taking the transaction messages as candidate messages; starting a first asynchronous thread; candidate messages are written to the second database by the first asynchronous thread.
In one possible implementation of the embodiments of the present disclosure, the pushing module 1250 is specifically configured to: determining a theme corresponding to each target message in the target message queue; starting a second asynchronous thread; and pushing each target message in the target message queue to a service node subscribing to the topic of each target message through a second asynchronous thread.
In one possible implementation of the embodiment of the present disclosure, the message pushing apparatus 1200 may further include:
the statistics module is used for counting the pushed duration of any target message after pushing any target message to a target service node subscribing any target message aiming at any target message in the target message queue, and marking the pushing state of any target message as the pushed state.
The determining module is configured to determine that pushing of any target message fails if a response message sent by the target service node is not received under a condition that a time length of the pushed message reaches a time length threshold corresponding to any target message, where the response message is used to indicate that any target message is received.
And the updating module is used for updating the pushing state of any target message into a pushing failure state.
And the adding module is used for re-adding any target message in the push failure state to the target message queue so as to re-push any target message.
In a possible implementation manner of the embodiment of the present disclosure, the determining module is further configured to determine a number of times that any target message fails to be pushed.
And the updating module is also used for updating the time length threshold corresponding to any target message according to the times, wherein the updated time length threshold and the times form a forward relation.
The message pushing device of the embodiment of the disclosure caches at least one transaction message generated by any service node in a distributed system to at least one first database, monitors the cached transaction message in the at least one first database, determines candidate messages meeting a first setting condition from the cached transaction messages, and writes the candidate messages into a second database; and acquiring the target message meeting the second set condition from the candidate message in the second database, placing the target message into a target message queue, and pushing the target message in the target message queue to a service node subscribing the subject of the target message in the distributed system. Therefore, before the transaction information is written into the second database, each transaction information is cached through the first database, so that the stable writing of the information in the second database can be ensured, and the situation that the second database cannot be normally used due to the fact that the information with higher flow is written into the second database in unit time in a high concurrency scene is avoided. And the transaction information is stored by the second database, so that abnormal conditions such as downtime and the like of equipment can be avoided, and the situation that the transaction information is lost is avoided, and timely and effective pushing of the transaction information is ensured.
To achieve the above embodiments, the present disclosure also provides an electronic device that may include at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the message pushing method according to any one of the above embodiments of the present disclosure.
To implement the above embodiments, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the message pushing method set forth in any one of the above embodiments of the present disclosure.
To achieve the above embodiments, the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements a message pushing method as set forth in any of the above embodiments of the present disclosure.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
FIG. 13 illustrates a schematic block diagram of an example electronic device that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 13, the electronic device 1300 includes a computing unit 1301 that can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1302 or a computer program loaded from a storage unit 1308 into a RAM (Random Access Memory, random access/Memory) 1303. In the RAM 1303, various programs and data required for the operation of the electronic device 1300 can also be stored. The computing unit 1301, the ROM 1302, and the RAM 1303 are connected to each other through a bus 1304. An I/O (Input/Output) interface 1305 is also connected to bus 1304.
Various components in electronic device 1300 are connected to I/O interface 1305, including: an input unit 1306 such as a keyboard, a mouse, or the like; an output unit 1307 such as various types of displays, speakers, and the like; storage unit 1308, such as a magnetic disk, optical disk, etc.; and a communication unit 1309 such as a network card, a modem, a wireless communication transceiver, or the like. The communication unit 1309 allows the electronic device 1300 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The computing unit 1301 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1301 include, but are not limited to, a CPU (Central Processing Unit ), GPU (Graphic Processing Units, graphics processing unit), various dedicated AI (Artificial Intelligence ) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor ), and any suitable processor, controller, microcontroller, etc. The computing unit 1301 performs the various methods and processes described above, such as the message pushing method described above. For example, in some embodiments, the message pushing method described above may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1308. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 1300 via the ROM 1302 and/or the communication unit 1309. When the computer program is loaded into the RAM 1303 and executed by the computing unit 1301, one or more steps of the message pushing method described above may be performed. Alternatively, in other embodiments, the computing unit 1301 may be configured to perform the message pushing method described above by any other suitable means (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit System, FPGA (Field Programmable Gate Array ), ASIC (Application-Specific Integrated Circuit, application-specific integrated circuit), ASSP (Application Specific Standard Product, special-purpose standard product), SOC (System On Chip ), CPLD (Complex Programmable Logic Device, complex programmable logic device), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory, erasable programmable read-Only Memory) or flash Memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display ) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network ), WAN (Wide Area Network, wide area network), internet and blockchain networks.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service (Virtual Private Server, virtual special servers) are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be noted that, artificial intelligence is a subject of studying a certain thought process and intelligent behavior (such as learning, reasoning, thinking, planning, etc.) of a computer to simulate a person, and has a technology at both hardware and software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge graph technology and the like.
Cloud computing (cloud computing) refers to a technical system that accesses an elastically extensible shared physical or virtual resource pool through a network, wherein resources can include servers, operating systems, networks, software, applications, storage devices and the like, and can be deployed and managed in an on-demand and self-service manner. Through cloud computing technology, high-efficiency and powerful data processing capability can be provided for technical application such as artificial intelligence and blockchain, and model training.
According to the technical scheme of the embodiment of the disclosure, at least one transaction message generated by any service node in a distributed system is cached to at least one first database, the cached transaction message in the at least one first database is monitored, so that candidate messages meeting a first setting condition are determined from the cached transaction messages, and the candidate messages are written into a second database; and acquiring the target message meeting the second set condition from the candidate message in the second database, placing the target message into a target message queue, and pushing the target message in the target message queue to a service node subscribing the subject of the target message in the distributed system. Therefore, before the transaction information is written into the second database, each transaction information is cached through the first database, so that the stable writing of the information in the second database can be ensured, and the situation that the second database cannot be normally used due to the fact that the information with higher flow is written into the second database in unit time in a high concurrency scene is avoided. And the transaction information is stored by the second database, so that abnormal conditions such as downtime and the like of equipment can be avoided, and the situation that the transaction information is lost is avoided, and timely and effective pushing of the transaction information is ensured.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions presented in the present disclosure are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (15)

1. A message pushing method, the method comprising:
acquiring at least one transaction message generated by any service node in a distributed system, and caching the at least one transaction message to at least one first database;
monitoring the cached transaction messages in the at least one first database, determining candidate messages meeting a first setting condition from the cached transaction messages, and writing the candidate messages into a second database, wherein the second database is different from the first database, and the second database is used for storing the transaction messages to be pushed;
Based on a transaction isolation mechanism, acquiring a target message which is identified as deliverable from candidate messages in the second database, and placing the target message into a target message queue;
pushing the target message in the target message queue to a service node in the distributed system subscribing to the subject of the target message;
wherein said caching said at least one transaction message to at least one first database comprises:
setting an identification of the transaction message to wait for each acquired transaction message, and storing the transaction message identified as waiting to the at least one first database;
sending a transaction review request to any service node, wherein the review request is used for inquiring whether the transaction message can be pushed;
receiving a transaction review response sent by any service node, wherein the transaction review response is generated by the any service node according to an execution result of a target transaction on which the transaction message depends in response to the transaction review request;
and under the condition that the transaction review response comprises first indication information, updating the identification of the transaction message in the at least one first database to be deliverable according to the first indication information, wherein the first indication information is generated after the any service node determines that the target transaction is successfully executed and is used for indicating that the transaction message can be pushed.
2. The method of claim 1, wherein after the receiving the review response sent by the any service node, the method further comprises:
and deleting the transaction message in the at least one first database under the condition that the transaction review response contains second instruction information, wherein the second instruction information is generated after the fact that any service node determines that the target transaction fails to execute and is used for indicating that the transaction message cannot be pushed.
3. The method of claim 1, wherein the monitoring the cached transaction messages in the at least one first database to determine candidate messages from the cached transaction messages that satisfy a first set condition and writing the candidate messages to a second database comprises:
monitoring the transaction messages cached in the at least one first database, and responding to the arrival of a set time point, acquiring the transaction messages which are not higher than a set number from the transaction messages cached in the at least one first database and taking the transaction messages as the candidate messages;
starting a first asynchronous thread;
and writing the candidate message into the second database through the first asynchronous thread.
4. The method of claim 1, wherein the pushing the target message in the target message queue to a service node in the distributed system that subscribes to a topic of the target message comprises:
determining a theme corresponding to each target message in the target message queue;
starting a second asynchronous thread;
pushing, by the second asynchronous thread, each of the target messages in the target message queue to a service node subscribing to a topic of each of the target messages.
5. The method of claim 4, wherein the pushing, by the second asynchronous thread, each of the target messages in the target message queue to a service node subscribing to a subject of each of the target messages further comprises:
counting the pushed duration of any target message after pushing the target message to a target service node subscribed to the target message aiming at any target message in the target message queue, and marking the pushing state of the target message as a pushed state;
if the pushed duration reaches a duration threshold corresponding to any target message, determining that the pushing of any target message fails if a response message sent by the target service node is not received, wherein the response message is used for indicating that any target message is received;
Updating the push state of any target message to be a push failure state;
and re-adding the any target message in the push failure state to the target message queue so as to re-push the any target message.
6. The method of claim 5, wherein the method further comprises:
determining the times of failed pushing of any target message;
and updating a time length threshold corresponding to any target message according to the times, wherein the updated time length threshold and the times form a forward relation.
7. A message push system, the system comprising:
at least one first service node in the distributed system for generating at least one transaction message and transmitting the at least one transaction message;
at least one second service node in the distributed system, configured to obtain the at least one transaction message sent by the at least one first service node, and cache the at least one transaction message to a first database;
at least one third service node in the distributed system, configured to monitor a transaction message cached in the first database in the at least one second service node, to determine a candidate message that meets a first setting condition from the cached transaction message, and to write the candidate message into a second database, where the second database is a different database from the first database, and the second database is configured to store a transaction message to be pushed;
At least one fourth service node in the distributed system, configured to obtain, based on a transaction isolation mechanism, a target message identified as deliverable from candidate messages in the second database in the at least one third service node, and place the target message in a target message queue, so as to push the target message in the target message queue to a fifth service node in the distributed system that subscribes to a topic of the target message;
wherein said caching said at least one transaction message to the first database comprises:
setting the identification of the transaction message to wait for each acquired transaction message, and storing the transaction message identified as waiting to the first database;
sending a transaction review request to any service node, wherein the review request is used for inquiring whether the transaction message can be pushed;
receiving a transaction review response sent by any service node, wherein the transaction review response is generated by the any service node according to an execution result of a target transaction on which the transaction message depends in response to the transaction review request;
And under the condition that the transaction review response comprises first indication information, updating the identification of the transaction message in the first database to be deliverable according to the first indication information, wherein the first indication information is generated after the any service node determines that the target transaction is successfully executed and is used for indicating that the transaction message can be pushed.
8. A message pushing apparatus, the apparatus comprising:
the first acquisition module is used for acquiring at least one transaction message generated by any service node in the distributed system;
a caching module, configured to cache the at least one transaction message to at least one first database;
the monitoring module is used for monitoring the cached transaction messages in the at least one first database, determining candidate messages meeting a first setting condition from the cached transaction messages, and writing the candidate messages into a second database; the second database is a database different from the first database, and is used for storing transaction messages to be pushed;
the second acquisition module is used for acquiring target information marked as deliverable from candidate information in the second database based on a transaction isolation mechanism, and placing the target information into a target information queue;
A pushing module, configured to push the target message in the target message queue to a service node in the distributed system that subscribes to a topic of the target message;
wherein, the cache module includes:
a processing unit, configured to set, for each acquired transaction message, an identifier of the transaction message to wait, and store the transaction message identified as waiting to the at least one first database;
a sending unit, configured to send a transaction review request to the any service node, where the review request is used to query whether the transaction message can be pushed;
the receiving unit is used for receiving a transaction review response sent by any service node, wherein the transaction review response is generated by the any service node according to an execution result of a target transaction on which the transaction message depends in response to the transaction review request;
the updating unit is configured to update, according to first indication information, an identifier of the transaction message in the at least one first database to be deliverable when the transaction review response includes the first indication information, where the first indication information is generated after the any service node determines that the target transaction is successfully executed, and is used to indicate that the transaction message can be pushed.
9. The apparatus of claim 8, wherein the cache module further comprises:
and the deleting unit is used for deleting the transaction message in the at least one first database under the condition that the transaction review response contains second instruction information, wherein the second instruction information is generated after the any service node determines that the target transaction fails to execute and is used for indicating that the transaction message cannot be pushed.
10. The apparatus of claim 8, wherein the monitoring module is specifically configured to:
monitoring the transaction messages cached in the at least one first database, and responding to the arrival of a set time point, acquiring the transaction messages which are not higher than a set number from the transaction messages cached in the at least one first database and taking the transaction messages as the candidate messages;
starting a first asynchronous thread;
and writing the candidate message into the second database through the first asynchronous thread.
11. The device of claim 8, wherein the pushing module is specifically configured to:
determining a theme corresponding to each target message in the target message queue;
starting a second asynchronous thread;
pushing, by the second asynchronous thread, each of the target messages in the target message queue to a service node subscribing to a topic of each of the target messages.
12. The apparatus of claim 11, wherein the apparatus further comprises:
the statistics module is used for counting the pushed duration of any target message after pushing the target message to a target service node subscribed to the target message aiming at any target message in the target message queue, and marking the pushing state of the target message as the pushed state;
a determining module, configured to determine that, if a response message sent by the target service node is not received, the pushing of the target message fails if the pushed duration reaches a duration threshold corresponding to the target message, where the response message is used to indicate that the target message is received;
the updating module is used for updating the pushing state of any target message into a pushing failure state;
and the adding module is used for re-adding any target message in the push failure state to the target message queue so as to re-push any target message.
13. The apparatus of claim 12, wherein,
the determining module is further configured to determine a number of times that the pushing of the arbitrary target message fails;
The updating module is further configured to update a duration threshold corresponding to the any one of the target messages according to the times, where the updated duration threshold and the times form a forward relationship.
14. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
15. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-6.
CN202111619013.3A 2021-12-27 2021-12-27 Message pushing method, device, system, electronic equipment and storage medium Active CN114500443B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111619013.3A CN114500443B (en) 2021-12-27 2021-12-27 Message pushing method, device, system, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111619013.3A CN114500443B (en) 2021-12-27 2021-12-27 Message pushing method, device, system, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN114500443A CN114500443A (en) 2022-05-13
CN114500443B true CN114500443B (en) 2024-03-29

Family

ID=81496965

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111619013.3A Active CN114500443B (en) 2021-12-27 2021-12-27 Message pushing method, device, system, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN114500443B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115237637A (en) * 2022-08-10 2022-10-25 中国建设银行股份有限公司 Message sending system, method and device, electronic equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105592117A (en) * 2014-10-23 2016-05-18 阿里巴巴集团控股有限公司 Method and device for processing transaction message
CN107609086A (en) * 2017-09-07 2018-01-19 同程网络科技股份有限公司 A kind of APP method for pushing and its automotive engine system
CN108009027A (en) * 2017-11-23 2018-05-08 北京百度网讯科技有限公司 Implementation method, device, equipment and the storage medium of queue message uniformity
CN111147355A (en) * 2019-12-25 2020-05-12 北京五八信息技术有限公司 Message sending method and device, electronic equipment and storage medium
CN111158933A (en) * 2019-12-31 2020-05-15 易票联支付有限公司 Distributed transaction processing method and system based on message queue
CN112492019A (en) * 2020-11-24 2021-03-12 上海优扬新媒信息技术有限公司 Message pushing method and device, electronic equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105592117A (en) * 2014-10-23 2016-05-18 阿里巴巴集团控股有限公司 Method and device for processing transaction message
CN107609086A (en) * 2017-09-07 2018-01-19 同程网络科技股份有限公司 A kind of APP method for pushing and its automotive engine system
CN108009027A (en) * 2017-11-23 2018-05-08 北京百度网讯科技有限公司 Implementation method, device, equipment and the storage medium of queue message uniformity
CN111147355A (en) * 2019-12-25 2020-05-12 北京五八信息技术有限公司 Message sending method and device, electronic equipment and storage medium
CN111158933A (en) * 2019-12-31 2020-05-15 易票联支付有限公司 Distributed transaction processing method and system based on message queue
CN112492019A (en) * 2020-11-24 2021-03-12 上海优扬新媒信息技术有限公司 Message pushing method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN114500443A (en) 2022-05-13

Similar Documents

Publication Publication Date Title
CN113364877B (en) Data processing method, device, electronic equipment and medium
CN113961510B (en) File processing method, device, equipment and storage medium
KR20210083222A (en) Method, apparatus, device and storage medium for processing voice data
CN114500443B (en) Message pushing method, device, system, electronic equipment and storage medium
CN113010535B (en) Cache data updating method, device, equipment and storage medium
CN112883041B (en) Data updating method and device, electronic equipment and storage medium
CN113986936A (en) Data processing method and device, electronic equipment and storage medium
CN114518848B (en) Method, device, equipment and medium for processing stored data
CN115510036A (en) Data migration method, device, equipment and storage medium
CN113986135B (en) Method, device, equipment and storage medium for processing request
CN113360689B (en) Image retrieval system, method, related device and computer program product
CN113641688B (en) Node updating method, related device and computer program product
CN115357641A (en) Data query method and device, electronic equipment and storage medium
CN114780022B (en) Method and device for realizing additional writing operation, electronic equipment and storage medium
CN113779080B (en) Method and device for transmitting data
CN113553216B (en) Data recovery method and device, electronic equipment and storage medium
CN118301215A (en) Processing method and device for client collection resources, electronic equipment and storage medium
CN113934581A (en) Method and device for acquiring disk fault information of storage system, electronic equipment and medium
CN117632545A (en) Data sharing method and device of business system and electronic equipment
CN115587119A (en) Database query method and device, electronic equipment and storage medium
CN117608879A (en) Message queue processing method and device, electronic equipment and storage medium
CN116737126A (en) Data processing method, device, electronic equipment and storage medium
CN115757959A (en) Method, apparatus, device, medium and product for removing duplicate
CN114661738A (en) Method, device and equipment for adding index and storage medium
CN116455968A (en) Message middleware management method and related equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant