CN109783258B - Message processing method and device and server - Google Patents
Message processing method and device and server Download PDFInfo
- Publication number
- CN109783258B CN109783258B CN201910129887.7A CN201910129887A CN109783258B CN 109783258 B CN109783258 B CN 109783258B CN 201910129887 A CN201910129887 A CN 201910129887A CN 109783258 B CN109783258 B CN 109783258B
- Authority
- CN
- China
- Prior art keywords
- message
- backlog
- restored
- messages
- cached
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The embodiment of the invention provides a message processing method, a message processing device and a server, which relate to the technical field of computers and aim to realize that a message consumer acquires messages according to the sequence of the messages reaching a message system. The embodiment of the application comprises the following steps: receiving a message restoring instruction carrying designated parameters, wherein the designated parameters are used for identifying designated cache tags which are used for identifying backlog messages to be restored; then, acquiring backlog information to be restored corresponding to the specified cache tag in a database; and sequencing the backlog messages to be restored from the morning to the evening according to the receiving time, and sequentially sending each backlog message to be restored to the message system according to the sequencing result, wherein the receiving time is the time when the message system receives the backlog messages.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a message processing method, an apparatus, and a server.
Background
The message system is a system public storage space for storing messages, a message producer can send a message to be sent to the message system, and a message consumer takes the message out of the message system when needing to obtain the message.
At present, if the messages in the message system need to be cached in the database, in order to increase the caching speed, a multi-thread parallel cache is usually adopted, and because the processing speed of each thread is different, the order of the messages arriving at the message system is different from the order of the message cache. When the messages in the database are restored to the message system, the messages are generally restored according to the sequence of message cache, so that the order of the restored messages is different from the order of the messages reaching the message system, and a message consumer cannot acquire the messages according to the order of the messages reaching the message system.
Disclosure of Invention
The embodiment of the invention aims to provide a message processing method, a message processing device and a message processing server, so that a message consumer can acquire messages according to the sequence of the messages reaching a message system. The specific technical scheme is as follows:
in a first aspect, a message processing method is provided, which is applied to a target server, where the target server may perform information interaction with a message system and a database, and the method includes:
receiving a message restoring instruction carrying designated parameters, wherein the designated parameters are used for identifying designated cache tags which are used for identifying backlog messages to be restored;
obtaining backlog information to be restored corresponding to the specified cache tag in the database;
and sequencing the backlog messages to be restored from the morning to the evening according to the receiving time, and sequentially sending each backlog message to be restored to the message system according to the sequencing result, wherein the receiving time is the time when the message system receives the backlog messages.
Optionally, the method further includes:
receiving a message caching instruction, wherein the message caching instruction comprises: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
determining a cache label corresponding to each preset time character string according to each preset time character string, the cluster identification and the queue identification, wherein the preset time character strings are used for representing the time period of the receiving time of the backlog message to be cached;
and correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
Optionally, the backlog messages to be restored are sorted from morning to evening according to the receiving time, and the backlog messages to be restored are sequentially sent to the message system according to the sorting result, including:
sequencing the designated cache tags according to the sequence of the time periods represented by the preset time character strings from morning to evening;
sequencing backlog messages to be restored corresponding to each appointed cache tag according to the sequence of the receiving time from morning to evening;
and sequentially sending the backlog information to be restored to the information system according to the sequence of the receiving time from morning to evening.
Optionally, the method further includes:
receiving a message caching instruction, wherein the message caching instruction comprises: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
determining a cache label corresponding to each preset hash value according to each preset hash value, the cluster identifier and the queue identifier, wherein the preset hash value is used for representing the hash value corresponding to the identifier of the backlog message to be cached;
and correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
Optionally, the backlog messages to be restored are sorted from morning to evening according to the receiving time, and the backlog messages to be restored are sequentially sent to the message system according to the sorting result, including:
according to the sequence of the receiving time from morning to evening, respectively carrying out internal sequencing on backlog information to be restored corresponding to each appointed cache tag;
merging and sequencing backlog messages to be restored corresponding to the designated cache tags according to the internal sequencing result of the backlog messages to be restored corresponding to the designated cache tags and the sequence of the receiving time from morning to evening;
and sequentially sending each backlog message to be restored to the message system according to the merging and sorting result of the backlog messages to be restored corresponding to each appointed cache tag.
In a second aspect, a message processing apparatus is provided, which is applied to a target server, where the target server can perform information interaction with a message system and a database, and the apparatus includes:
the receiving module is used for receiving a message restoring instruction carrying specified parameters, wherein the specified parameters are used for identifying specified cache tags, and the specified cache tags are used for identifying backlog messages to be restored;
the acquisition module is used for acquiring the backlog information to be restored corresponding to the specified cache tag received by the receiving module from a database;
and the sending module is used for sequencing the backlog messages to be restored acquired by the acquiring module from morning to evening according to the receiving time, and sequentially sending each backlog message to be restored to the message system according to the sequencing result, wherein the receiving time is the time when the message system receives the backlog messages.
Optionally, the apparatus further comprises: the device comprises a first determining module and a first saving module;
the receiving module is further configured to receive a message caching instruction, where the message caching instruction includes: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
the first determining module is configured to determine a cache tag corresponding to each preset time character string according to each preset time character string, the cluster identifier and the queue identifier received by the receiving module, where the preset time character string is used to indicate a time period in which a receiving time of the backlog message to be cached is located;
and the first storage module is used for correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
Optionally, the sending module is specifically configured to:
sequencing the designated cache tags according to the sequence of the time periods represented by the preset time character strings from morning to evening; sequencing backlog messages to be restored corresponding to each appointed cache tag according to the sequence of the receiving time from morning to evening; and sequentially sending the backlog information to be restored to the information system according to the sequence of the receiving time from morning to evening.
Optionally, the apparatus further comprises: a second determining module and a second saving module;
the receiving module is further configured to receive a message caching instruction, where the message caching instruction includes: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
the second determining module is configured to determine, according to each preset hash value, the cluster identifier and the queue identifier received by the receiving module, a cache tag corresponding to each preset hash value, where the preset hash value is used to represent a hash value corresponding to an identifier of a backlog message to be cached;
and the second storage module is used for correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
Optionally, the sending module is specifically configured to:
according to the sequence of the receiving time from morning to evening, respectively carrying out internal sequencing on backlog information to be restored corresponding to each appointed cache tag; merging and sequencing backlog messages to be restored corresponding to the designated cache tags according to the internal sequencing result of the backlog messages to be restored corresponding to the designated cache tags and the sequence of the receiving time from morning to evening; and sequentially sending each backlog message to be restored to the message system according to the merging and sorting result of the backlog messages to be restored corresponding to each appointed cache tag.
In a third aspect, a message processing system is provided, which includes the target server in the second aspect, the database and the message system related to the second aspect;
the database is used for receiving the cache tags sent by the target server and the backlog information to be cached corresponding to the cache tags, and correspondingly storing the backlog information to be cached corresponding to each cache tag and the cache tag into one cache unit.
In a fourth aspect, a server is provided, which includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
and a processor for implementing any of the message processing method steps described above when executing the program stored in the memory.
In a fifth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when being executed by a processor, the computer program implements any of the message processing method steps described above.
In a sixth aspect, embodiments of the present invention further provide a computer program product including instructions, which when run on a computer, cause the computer to perform any of the message processing method steps described above.
According to the message processing method, the message processing device and the message processing server, when the backlog messages to be restored are restored, the backlog messages to be restored can be sequenced from the early to the late of the receiving time, and then the backlog messages to be restored are sequentially sent to the message system according to the sequencing result of the backlog messages to be restored. Therefore, the invention can restore the backlog information to be restored to the information system according to the sequence of the information generated to the information system, so the invention can realize the unification of the sequence of the restored information and the sequence of the information reaching the information system, and the information consumer can obtain the information according to the sequence of the information reaching the information system.
Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a schematic structural diagram of a message processing system according to an embodiment of the present invention;
fig. 2 is a flowchart of a message processing method according to an embodiment of the present invention;
fig. 3 is an exemplary schematic diagram of a message processing method according to an embodiment of the present invention;
fig. 4 is an exemplary schematic diagram of a message processing method according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a server according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.
The message processing method provided by the embodiment of the invention can be applied to a message processing system shown in fig. 1, and the system comprises the following steps: a messaging system, a target server, and a database. The connections between the messaging system, the target server and the database may all be wireless connections.
The message system comprises a plurality of system servers, and is used for receiving and storing messages sent by message producers. And adds cluster identification and queue identification to the received message. The connection between the various system servers in the messaging system may be a wireless connection.
The target server includes: the buffer and recovery module is used for determining backlog information to be buffered according to the cluster identifier and the queue identifier of the backlog information when the backlog information in the information system needs to be buffered in the database, and then buffering the backlog information to be buffered in the information system in the database; when the backlog information in the database needs to be restored to the message system, the backlog information to be restored is sent to the message system from the database according to the cache tag of the backlog information.
The database may be a distributed storage system (HBase) and is configured to receive cache tags sent by a target server and backlog messages to be cached corresponding to the cache tags, and store the cache tags and the backlog messages to be cached corresponding to the cache tags in one cache unit (Block).
In another embodiment, the database may be integrated in the target server.
Referring to fig. 1, an embodiment of the present invention provides a message processing method, which is applied to a target server, where the target server may perform information interaction with a message system and a database, and as shown in fig. 2, the method includes:
In the embodiment of the present invention, in a case that a backlog message exists in a message system and is cached in a database, when the backlog message in a queue of the message system is reduced to a preset threshold, a target server may receive a message restoring instruction, so that the target server restores the backlog message of the queue cached in the database to the message system.
Alternatively, backlog messages in an old cluster may be stored in the database when there is a server failure in the old cluster, or when the old cluster needs to change the geographical location of the cluster, resulting in a need to deploy a new cluster to replace the old cluster. When the new cluster is deployed and the backlog message in the old cluster needs to be restored, the target server can receive the message restoring instruction, so that the target server can restore the backlog message in the old cluster cached in the database to the new cluster.
The designated parameter is used for identifying a designated cache tag; the designated cache tag is used to identify the backlog message to be restored. The specified parameters include: cluster identification and queue identification. For example, the cluster identifier may be a cluster Identity (ID) and the queue identifier may be a queue name.
In one embodiment, the target server may receive a message restoring instruction, and the message restoring instruction may include a specified parameter or a cache TAG (TAG). The TAG comprises: cluster identification, queue identification, timestamp, and reason for the cache operation. For example: the reason for the cache operation of the TAG in TAG is: ClusterMigration, which indicates that backlog messages in old clusters need to be cached due to cluster migration.
The timestamp is used to mark the time of the cache, for example, the timestamp may be a time when a message caching instruction is received, may be a time when a backlog message to be cached is acquired from a message system, or may be a time when the backlog message to be cached is stored in a database.
The method for acquiring the backlog information to be restored comprises the following steps: determining a designated cache tag according to designated parameters; and then acquiring backlog information to be restored from the database according to the specified cache tag.
The manner of obtaining the specified cache tag may include the following three ways:
in the first mode, a cache tag (MD5) corresponding to the cluster ID and the queue name may be obtained from the target server according to the cluster ID and the queue name (when the backlog message is cached, the target server may store the cluster ID, the queue name and MD5 in a relational database management system (MySQL) of the target server). Or the target server may compute MD5 directly from the cluster ID and the queue name.
In the second way, if the backlog information of the same queue in the same cluster is stored in the database for multiple times, the TAG with the timestamp closest to the current restoration time in the TAGs corresponding to the cluster ID and the queue can be obtained from MySQL according to the cluster ID and the queue name, and the MD5 corresponding to the TAG can be obtained.
In a third mode, the MD5 corresponding to the TAG can be obtained from MySQL according to the TAG in the message restoring instruction.
After the MD5 is obtained, each backlog message to be restored corresponding to the MD5 is obtained from the HBase (when the backlog messages to be cached are cached, the backlog messages to be cached corresponding to the MD5 and the MD5 may be correspondingly stored in the HBase).
And 203, sequencing the backlog messages to be restored from morning to evening according to the receiving time, and sequentially sending each backlog message to be restored to the message system according to the sequencing result.
Wherein, the receiving time is the time when the message system receives the backlog message.
In one embodiment, the receiving time of each backlog message to be restored may be obtained, then the backlog messages to be restored are sorted from the morning to the evening according to the receiving time (that is, the backlog messages to be restored are sorted according to the sequence from the message production to the message system), and then the backlog messages to be restored are sequentially sent to the message system according to the sorting result.
For example: the sequence of the three backlog messages to be restored after sequencing according to the sequence of the receiving time from morning to evening is as follows: message 1, message 2, and message 3. The target server may send message 1 to the messaging system, then message 2 to the messaging system, and then message 3 to the messaging system.
According to the message processing method provided by the embodiment of the invention, when the backlog message to be restored is restored, the backlog message to be restored can be sorted from the morning to the evening according to the receiving time, and then the backlog message to be restored is sent to the message system in sequence according to the sorting result of the backlog message to be restored. Therefore, the invention can restore each backlog message to be restored to the message system according to the sequence of producing the message to the message system. Therefore, the invention can realize the unification of the order of the restored messages and the order of the messages reaching the message system, and the message consumers can obtain the messages according to the order of the messages reaching the message system.
It can be understood that, on the premise that the backlog message is restored, the backlog message is already cached in the database, in order to support the message restoration process of the embodiment corresponding to fig. 2, the embodiment of the present invention further provides two message caching manners, one message restoration manner is provided corresponding to each message caching manner, and the following respectively describes the two caching manners and the restoration manners corresponding to the two caching manners:
the first method is as follows: with reference to fig. 3, when backlog messages are cached, T time strings are preset (the preset time strings are used to indicate a time period of a receiving time of the backlog message to be cached), each time string corresponds to one MD5, and each backlog message to be cached and the MD5 corresponding to the backlog message to be cached are stored correspondingly according to the receiving time of the backlog message to be cached. When the backlog messages to be restored are restored, the backlog messages to be restored corresponding to the MDs 5 are sorted from morning to evening according to the receiving time, and the backlog messages to be restored are sequentially sent to the message system according to the sorting result.
Specifically, the first message caching method includes the following three steps:
first, a message caching instruction is received. Wherein, the message cache instruction comprises: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs. The cluster identification may be a cluster ID and the queue identification may be a queue name.
In this embodiment of the present invention, when there are too many backlog messages in a queue in the message system, the target server may receive a message caching instruction, so that the target server caches the backlog messages in the queue in the database.
Alternatively, when there is a server failure in an old cluster, or when the old cluster needs to change the geographical location of the cluster, resulting in a new cluster being deployed to replace the old cluster, the target server may receive a message caching instruction, so that the target server caches the backlog message in the old cluster in the database.
In an embodiment, the backlog message to be cached may be obtained from a queue corresponding to the queue identifier in a cluster corresponding to the cluster identifier in the message system.
For example, if the cluster identifier in the message caching instruction is 1 and the queue identifier is a, the target server obtains the backlog message to be cached from the queue a included in the cluster 1 of the message system, which can be understood that the messages included in the current queue a are all the backlog messages to be cached.
Optionally, the message caching instruction may further include a time parameter, for example, the time parameter is 2018010105, and then, in the cluster corresponding to the cluster identifier, messages received by the message system in the queue corresponding to the queue identifier in the time period from 2018, 1 month, 1 day 05:00 to 2018, 1 month, 1 day 05:59 are backlog messages to be cached.
And secondly, determining a cache label corresponding to each preset time character string according to each preset time character string, the cluster identification and the queue identification.
In the embodiment of the present invention, 24 time character strings may be preset to distinguish 24 hours of a day. And determining the MD5 corresponding to each preset time character string according to the preset time character string, the cluster identifier and the queue identifier. For example: the time string is: 2018080101, the receiving time of the backlog message to be buffered is in the time period of 2018, 8, month 1, 01:00-2018, 8, month 1, 01: 59.
Or, the receiving time T1 of the first backlog message to be buffered in the queue corresponding to the backlog message to be buffered and the receiving time T2 of the last backlog message to be buffered may also be obtained, and the number of the preset time character strings is determined according to the difference between T2 and T1. For example: t1, T2-T1, T3 time strings may be set, where T1 is 2018, 8, 1, 01:12, T2 is 2018, 8, 1, 03:22, T2-T1 are 2 hours and 20 minutes, and the respective time strings are: 2018080101, 2018080102, and 2018080103. Wherein 2018080101 denotes: the receiving time of the backlog message to be cached is in the time period of 8 month 1 day 01:00 in 2018 to 8 month 1 day 01:59 in 2018. 2018080102 denotes: the receiving time of the backlog message to be cached is within the time period of 8/1/02: 00 in 2018-1/8/59 in 2018. 2018080103 denotes: the receiving time of the backlog message to be cached is within the time period of 2018, 8 month, 1 day 03:00-2018, 8 month, 1 day 03: 59.
In one embodiment, the cluster identifier, the queue identifier and the preset time string may be concatenated, and the cache tag of the backlog message may be calculated according to the concatenation result (MD 5). From MD5 and the ID of each backlog message to be cached, a row key (RowKey) is determined, i.e., RowKey ═ MD5+ MessageID. And correspondingly storing the RowKey and the backlog information to be cached corresponding to the RowKey in the HBase.
In another embodiment, a cache tag of the backlog message to be cached can be determined according to the cluster identifier, the queue identifier, the preset time character string and the timestamp; and correspondingly storing the cache tag, the backlog information identification to be cached and backlog information to be cached corresponding to the backlog information identification to be cached in a database.
In the embodiment of the invention, because the RowKey in the HBase consists of MD5 and MessageID, one MD5 corresponds to one Block, a plurality of blocks form a Region (Region), and different regions are distributed in different Region services (Region servers), the purpose of load balancing can be achieved. And the MessageID in the RowKey is used for distinguishing each backlog message, so that the conflict is avoided.
In the embodiment of the present invention, after the cache TAG, each backlog message identifier to be cached, and the backlog message to be cached corresponding to the backlog message identifier to be cached are correspondingly stored in the database, the cluster identifier, the queue identifier, and the timestamp may be used as a cache TAG (TAG), and the TAG and the MD5 are correspondingly stored in a relational database management system (MySQL) of the target server. Optionally, the TAG may further include a reason for the cache operation.
And thirdly, correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into a database.
In one embodiment, the receiving time of each backlog message to be cached is obtained, then the backlog message to be cached corresponding to each cache tag is determined according to the time period in which the receiving time is, and then each cache tag and the backlog message to be cached corresponding to the cache tag are correspondingly stored in the database.
Corresponding to the first message caching mode, the first message restoring mode comprises the following four steps:
the method comprises the steps of firstly, receiving a message restoring instruction carrying designated parameters, then determining designated cache tags according to the designated parameters, and then obtaining backlog messages to be restored corresponding to the designated cache tags in a database. The designated parameter is used for identifying a designated cache tag; the designated cache tag is used to identify the backlog message to be restored.
In one embodiment, the specified parameters may be: the cluster identifier may be a cluster ID, and the queue identifier may be a queue name. Since the cluster ID, queue name and MD5 are stored in MySQL when the message is cached, the target server can obtain MD5 corresponding to the cluster ID and queue name from MySQL when the message is restored.
For example: if 24 preset time character strings are provided when backlog information to be restored corresponding to the cluster ID and the queue name is cached, 24 MDs 5 can be obtained according to the cluster ID and the queue identification when the information is restored.
In another embodiment, the specified parameter may be a TAG. Because the cluster ID, the queue name, the MD5 and the TAG are correspondingly stored in MySQL when the message is cached, N MDs 5 correspond to unique TAGs. Therefore, in message reduction, the target server can obtain N MDs 5 corresponding to the TAG from MySQL.
For example: if 24 preset time character strings are provided when the backlog message to be restored corresponding to the TAG is cached, 24 MDs 5 can be obtained according to the TAG when the message is restored.
And secondly, sequencing the designated cache tags according to the sequence of the time periods represented by the preset time character strings from morning to evening.
For example: obtain 3 MDs 5 according to the specified parameters in the message restore instruction: MD5(1), MD5(2), and MD5(3), the time string in MD5(1) being: 2018080102, the time string in MD5(2) is: 2018080103, the time string in MD5(3) is: 2018080101. then the 3 MDs 5 are sorted in the order of the time periods represented by the time strings from the morning to the evening in the following order: MD5(3), MD5(1), and MD5 (2).
And thirdly, sequencing the backlog messages to be restored corresponding to each appointed cache tag according to the sequence of the receiving time from morning to evening.
For example: the designated cache tag corresponds to 3 backlog messages to be restored, which are respectively: message 1, message 2, and message 3. The reception time of message 1 is: 201808010005, the moment of reception of message 2 is: 201808010003, the moment of reception of message 3 is: 201808010001. then, the 3 backlog messages to be restored are sorted according to the sequence of the receiving time from morning to evening, and the sequence is as follows: message 3, message 2, and message 1.
In the embodiment of the present invention, the second step and the third step of the first message recovery mode may be executed simultaneously, or may be executed in an order of executing the second step first and then executing the third step, or may be executed in an order of executing the third step first and then executing the second step, and the present application does not limit the execution order of the two steps.
And fourthly, sequentially sending the backlog information to be restored to the information system according to the sequence of the receiving time from morning to evening.
In one embodiment, the designated cache tags may be sorted first, then the backlog messages to be restored corresponding to the first designated cache tag are sorted in the order from the morning to the evening of the receiving time, and then the backlog messages to be restored corresponding to the first designated cache tag are sequentially sent to the message system according to the sorting result. And then sequentially executing the above operations on backlog information to be restored corresponding to other specified cache tags.
In another embodiment, the designated cache tags may be sorted, and then the backlog messages to be restored corresponding to the designated cache tags are sorted. And after the sorting is finished, sequentially sending each backlog message to be restored to the message system according to the sorting result.
The second method comprises the following steps: with reference to fig. 4, when the backlog message is cached, N hash values are preset (the preset hash value is used to indicate a hash value corresponding to the identifier of the backlog message to be cached), each hash value corresponds to one MD5, and each backlog message to be cached and the MD5 corresponding to the message to be cached are stored correspondingly according to the hash value corresponding to the identifier of the message to be cached. When the backlog messages to be restored are restored, the backlog messages to be restored corresponding to the MD5 are sorted from morning to evening according to the receiving time, then the backlog messages to be restored are stored into a data memory (DataLog) according to the sorting result, and then the backlog messages to be restored are taken out from the DataLog and sent to a message system in sequence.
Specifically, the second message caching method includes the following three steps:
first, a message caching instruction is received. Wherein, the message cache instruction comprises: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs.
This step is the same as the first step of the first message caching method, and the description related to the first step of the first message caching method may be referred to, which is not described herein again.
And secondly, determining a cache label corresponding to each preset hash value according to each preset hash value, the cluster identifier and the queue identifier.
In the embodiment of the present invention, N hash values may be preset. And calculating the MD5 corresponding to each preset hash value according to the preset hash value, the cluster identifier and the queue identifier. For example: the preset hash value is 1, which means that the hash value corresponding to the ID of the backlog message to be cached is 1, that is, the backlog message to be cached identified by the MD5 calculated according to the hash value 1 is: a backlog message with a hash value of 1 for the message ID.
Or, the number of the hash values may also be determined according to the number of the backlog messages to be cached, the size of the backlog messages to be cached, and the size of the memory of the target server.
In one embodiment, the cluster identifier, the queue identifier and the preset hash value may be concatenated, and a cache tag of the backlog message to be cached is calculated according to the concatenation result (MD 5). From MD5 and the ID of each backlog message to be cached, a row key (RowKey) is determined, i.e., RowKey ═ MD5+ MessageID. And correspondingly storing the RowKey and the backlog information to be cached corresponding to the RowKey in the HBase.
In another embodiment, a cache tag of the backlog message to be cached can be determined according to the cluster identifier, the queue identifier, the preset hash value and the timestamp; and correspondingly storing the cache tag, the backlog information identification to be cached and backlog information to be cached corresponding to the backlog information identification to be cached in a database.
And thirdly, correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into a database.
In one embodiment, the hash value of each backlog message to be cached is calculated, then the cache tag corresponding to each backlog message to be cached is determined, and each cache tag and the backlog message to be cached corresponding to the cache tag are correspondingly stored in the database.
Corresponding to the second message caching mode, the second message restoring mode comprises the following four steps:
the method comprises the steps of firstly, receiving a message restoring instruction carrying designated parameters, then determining designated cache tags according to the designated parameters, and then obtaining backlog messages to be restored corresponding to the designated cache tags in a database. The designated parameter is used for identifying a designated cache tag; the designated cache tag is used to identify the backlog message to be restored.
In one embodiment, the specified parameters may be: the cluster identifier may be a cluster ID, and the queue identifier may be a queue name. Since the cluster ID, queue name and MD5 are stored in MySQL when the message is cached, the target server can obtain MD5 corresponding to the cluster ID and queue name from MySQL when the message is restored.
For example: if N preset hash values are set when backlog information to be restored corresponding to the cluster ID and the queue name is cached, N MDs 5 can be obtained according to the cluster ID and the queue identifier when the information is restored.
In another embodiment, the specified parameter may be a TAG. The cluster ID, the queue name, the MD5 and the TAG are correspondingly stored in MySQL when the message is cached. Wherein the N MDs 5 correspond to unique TAGs. Therefore, in message reduction, the target server can obtain N MDs 5 corresponding to the TAG from MySQL.
For example: if N preset hash values are set when the backlog message to be restored corresponding to the TAG is cached, N MDs 5 can be obtained according to the TAG when the message is restored.
And secondly, respectively carrying out internal sequencing on the backlog information to be restored corresponding to each appointed cache tag according to the sequence of the receiving time from morning to evening.
In the embodiment of the present invention, since the number of backlog messages to be restored corresponding to N MDs 5 is large, all backlog messages to be restored cannot be simultaneously transferred to the memory for sorting, and therefore, only through external sorting, each backlog message to be restored can be sorted. Wherein the external ordering comprises: inner sorting and merge sorting.
The inner ordering refers to: and calling backlog information to be restored corresponding to each appointed cache tag into a memory, and sequencing according to the sequence of the receiving time from morning to evening.
For example: the designated cache tag corresponds to 3 backlog messages to be restored, which are respectively: message 1, message 2, and message 3. The reception time of message 1 is: 201808010005, the moment of reception of message 2 is: 201808010003, the moment of reception of message 3 is: 201808010001. then, the 3 backlog messages to be restored are sorted according to the sequence of the receiving time from morning to evening, and the sequence after the inner sorting is as follows: message 3, message 2, and message 1.
And thirdly, merging and sequencing the backlog information to be restored corresponding to each appointed cache tag according to the internal sequencing result of the backlog information to be restored corresponding to each appointed cache tag and the sequence of the receiving time from morning to evening.
For example: there are 3 designated cache tags, which are: tag 1, tag 2 and tag 3, tag 1 corresponding to message 1 and message 2, tag 2 corresponding to message 3 and message 4, and tag 3 corresponding to message 5 and message 6. Then respectively taking the first messages corresponding to the three labels: message 1, message 3, and message 5, which are ordered according to the order of the time of receipt from morning to evening. Assuming that message 1 was received earlier than messages 3 and 5, message 1 is ordered first and a message is taken from tag 1: message 2. And sequencing the message 2, the message 3 and the message 5 according to the sequence of the receiving time from morning to evening until the backlog messages to be restored corresponding to the 3 specified labels are sequenced.
And fourthly, sequentially sending the backlog information to be restored to the information system according to the merging and sorting result of the backlog information to be restored corresponding to each appointed cache tag.
In one embodiment, the merged and sorted backlog messages to be restored are sequentially stored in the DataLog, and a location information (offset) is maintained in the database and indicates the location of the next backlog message to be restored, which needs to be acquired. When restoring the backlog messages to be restored, the target server can sequentially acquire each backlog message to be restored according to the offset and send each backlog message to be restored to the message system.
By applying the embodiment of the invention, the backlog messages to be restored can be sequenced according to the time when the messages reach the message system, and then the backlog messages to be restored are sent to the message system according to the sequencing result. It can be ensured that the order in which the messages are restored to the messaging system is uniform with the order in which the messages arrive at the messaging system.
In the prior art, when a message is restored by using a multi-thread parallel cache, the order of restoring the message to a message system is different from the order of arrival of the message to the message system, so that an operation order of an information consumer operating according to the obtained message is possibly wrong, or the obtained message by the information consumer is inaccurate. For example: the message system receives two operation messages containing operation instructions for the object A in sequence, namely a message 1 and a message 2.
The message 1 contains the following operation instructions: multiplying a by 2, the message 2 contains the operation instructions: add a to 10. If the two messages are cached in parallel by adopting multithreading, the sequence of the two messages is changed into that: message 2, message 1, then the message consumer will acquire message 2 first and then message 1 when acquiring two messages for object a. Causing to be performed on object a: first add 10 and then multiply by 2. It can be seen that the order of message 1 and message 2 changes, which may result in an incorrect order of operations performed by the message consumer.
For another example, there are 5 messages for purchasing article a in the messaging system, and the sequence is: message 1, message 2, message 3, message 4, and message 5. If the five messages cached in the multithread parallel mode are restored to the message system when the messages are restored, the sequence of restoring to the message system is as follows: message 5, message 4, message 3, message 2, and message 1. At this time, the message consumer needs to obtain the first 3 messages for purchasing the article a, which should originally obtain the message 1, the message 2 and the message 3, but after the message is restored, the message 5, the message 4 and the message 3 are obtained. Therefore, the prior art adopts the multithreading parallel cache to restore the message, which may cause the message consumer to receive the message inaccurately.
In the embodiment of the invention, the backlog messages to be restored can be ordered according to the sequence from the message production to the message system when the messages are restored, and the backlog messages to be restored are sent to the message system in sequence according to the ordering result, so that the sequence of the messages received by the message consumer can be unified with the sequence from the message production to the message system. Therefore, the method and the device can improve the accuracy of the operation sequence of the message consumer according to the acquired message and improve the accuracy of the message received by the message consumer.
Corresponding to the above method embodiment, as shown in fig. 5, an embodiment of the present invention provides a message processing apparatus, which is applied to a target server, where the target server may perform information interaction with a message system and a database, and the apparatus includes: a receiving module 501, an obtaining module 502 and a sending module 503.
A receiving module 501, configured to receive a message restoring instruction carrying an appointed parameter, where the appointed parameter is used to identify an appointed cache tag, and the appointed cache tag is used to identify a backlog message to be restored;
an obtaining module 502, configured to obtain, in a database, a backlog to be restored corresponding to the specified cache tag received by the receiving module;
the sending module 503 is configured to sort the backlog messages to be restored acquired by the acquiring module in the order from the morning to the evening according to the receiving time, and send each backlog message to be restored to the message system in sequence according to the sorting result, where the receiving time is the time when the message system receives the backlog messages.
Optionally, the apparatus may further include: the device comprises a first determining module and a first saving module;
the receiving module 501 is further configured to receive a message caching instruction, where the message caching instruction includes: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
the first determining module is used for determining a cache label corresponding to each preset time character string according to each preset time character string, the cluster identifier and the queue identifier received by the receiving module, wherein the preset time character strings are used for representing the time period of the receiving time of the backlog message to be cached;
and the first storage module is used for correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
Optionally, the sending module 503 may be specifically configured to:
sequencing the designated cache tags according to the sequence of the time periods represented by the preset time character strings from morning to evening; sequencing backlog messages to be restored corresponding to each appointed cache tag according to the sequence of the receiving time from morning to evening; and sequentially sending the backlog information to be restored to the information system according to the sequence of the receiving time from morning to evening.
Optionally, the apparatus may further include: a second determining module and a second saving module;
the receiving module 501 is further configured to receive a message caching instruction, where the message caching instruction includes: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
the second determining module is used for determining a cache label corresponding to each preset hash value according to each preset hash value, the cluster identifier and the queue identifier received by the receiving module, wherein the preset hash value is used for representing the hash value corresponding to the identifier of the backlog message to be cached;
and the second storage module is used for correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
Optionally, the sending module 503 may be specifically configured to:
according to the sequence of the receiving time from morning to evening, respectively carrying out internal sequencing on backlog information to be restored corresponding to each appointed cache tag; merging and sequencing backlog messages to be restored corresponding to the designated cache tags according to the internal sequencing result of the backlog messages to be restored corresponding to the designated cache tags and the sequence of the receiving time from morning to evening; and sequentially sending each backlog message to be restored to the message system according to the merging and sorting result of the backlog messages to be restored corresponding to each appointed cache tag.
According to the message processing device provided by the embodiment of the invention, when the backlog message to be restored is restored, the backlog message to be restored can be sorted from the morning to the evening according to the receiving time, and then the backlog message to be restored is sent to the message system in sequence according to the sorting result of the backlog message to be restored. Therefore, the invention can restore the backlog information to be restored to the information system according to the sequence of the information generated to the information system, so the invention can realize the unification of the sequence of the restored information and the sequence of the information reaching the information system, and the information consumer can obtain the information according to the sequence of the information reaching the information system.
The embodiment of the present invention further provides a server, as shown in fig. 6, including a processor 601, a communication interface 602, a memory 603, and a communication bus 604, where the processor 601, the communication interface 602, and the memory 603 complete mutual communication through the communication bus 604,
a memory 603 for storing a computer program;
the processor 601 is configured to implement the steps executed by the target server in the above method embodiments when executing the program stored in the memory 603.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In yet another embodiment provided by the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program realizes the steps of any of the above message processing methods when executed by a processor.
In yet another embodiment, a computer program product containing instructions is provided, which when run on a computer causes the computer to perform any of the message processing methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (12)
1. A message processing method is applied to a target server, the target server can carry out information interaction with a message system and a database, and the method comprises the following steps:
receiving a message restoring instruction carrying designated parameters from a message system, wherein the designated parameters are used for identifying designated cache tags which are used for identifying backlog messages to be restored;
obtaining the backlog information to be restored corresponding to the appointed cache tag in the database, wherein the appointed parameters comprise a cluster identifier and a queue identifier;
sequencing backlog messages to be restored from the morning to the evening according to the receiving time, and sequentially sending each backlog message to be restored to the message system according to the sequencing result so that a message consumer can operate according to the sequence of the obtained messages, wherein the receiving time is the time when the message system receives the backlog messages;
the message system is used for receiving and storing messages sent by a message producer, and adding cluster identifiers and queue identifiers to the received messages; the target server comprises a cache module, and the cache module is used for determining backlog information to be cached according to the cluster identification and the queue identification of the backlog information and then caching the backlog information to be cached in the information system into the database when the backlog information in the information system needs to be cached in the database.
2. The method of claim 1, further comprising:
receiving a message caching instruction, wherein the message caching instruction comprises: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
determining a cache label corresponding to each preset time character string according to each preset time character string, the cluster identification and the queue identification, wherein the preset time character strings are used for representing the time period of the receiving time of the backlog message to be cached;
and correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
3. The method according to claim 2, wherein the backlog messages to be restored are sorted in the order from the morning to the evening of the receiving time, and the backlog messages to be restored are sequentially sent to the message system according to the sorting result, including:
sequencing the designated cache tags according to the sequence of the time periods represented by the preset time character strings from morning to evening;
sequencing backlog messages to be restored corresponding to each appointed cache tag according to the sequence of the receiving time from morning to evening;
and sequentially sending the backlog information to be restored to the information system according to the sequence of the receiving time from morning to evening.
4. The method of claim 1, further comprising:
receiving a message caching instruction, wherein the message caching instruction comprises: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
determining a cache label corresponding to each preset hash value according to each preset hash value, the cluster identifier and the queue identifier, wherein the preset hash value is used for representing the hash value corresponding to the identifier of the backlog message to be cached;
and correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
5. The method according to claim 4, wherein the backlog messages to be restored are sorted in the order from the morning to the evening of the receiving time, and the backlog messages to be restored are sequentially sent to the message system according to the sorting result, including:
according to the sequence of the receiving time from morning to evening, respectively carrying out internal sequencing on backlog information to be restored corresponding to each appointed cache tag;
merging and sequencing backlog messages to be restored corresponding to the designated cache tags according to the internal sequencing result of the backlog messages to be restored corresponding to the designated cache tags and the sequence of the receiving time from morning to evening;
and sequentially sending each backlog message to be restored to the message system according to the merging and sorting result of the backlog messages to be restored corresponding to each appointed cache tag.
6. A message processing apparatus applied to a target server, the target server being capable of information interaction with a message system and a database, the apparatus comprising:
the receiving module is used for receiving a message restoring instruction which carries specified parameters and is from a message system, wherein the specified parameters are used for identifying specified cache tags, and the specified cache tags are used for identifying backlog messages to be restored;
the acquisition module is used for acquiring the backlog information to be restored corresponding to the specified cache tag received by the receiving module from a database, wherein the specified parameters comprise cluster identification and queue identification;
the sending module is used for sequencing the backlog messages to be restored acquired by the acquiring module from morning to evening according to the receiving time, and sequentially sending each backlog message to be restored to the message system according to the sequencing result so that a message consumer can operate according to the sequence of the acquired messages, wherein the receiving time is the time when the message system receives the backlog messages;
the message system is used for receiving and storing messages sent by a message producer, and adding cluster identifiers and queue identifiers to the received messages; the target server comprises a cache module, and the cache module is used for determining backlog information to be cached according to the cluster identification and the queue identification of the backlog information and then caching the backlog information to be cached in the information system into the database when the backlog information in the information system needs to be cached in the database.
7. The apparatus of claim 6, further comprising: the device comprises a first determining module and a first saving module;
the receiving module is further configured to receive a message caching instruction, where the message caching instruction includes: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
the first determining module is configured to determine a cache tag corresponding to each preset time character string according to each preset time character string, the cluster identifier and the queue identifier received by the receiving module, where the preset time character string is used to indicate a time period in which a receiving time of the backlog message to be cached is located;
and the first storage module is used for correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
8. The apparatus of claim 7, wherein the sending module is specifically configured to:
sequencing the designated cache tags according to the sequence of the time periods represented by the preset time character strings from morning to evening; sequencing backlog messages to be restored corresponding to each appointed cache tag according to the sequence of the receiving time from morning to evening; and sequentially sending the backlog information to be restored to the information system according to the sequence of the receiving time from morning to evening.
9. The apparatus of claim 6, further comprising: a second determining module and a second saving module;
the receiving module is further configured to receive a message caching instruction, where the message caching instruction includes: queue identification of a queue to which backlog information to be cached belongs and cluster identification of a cluster to which the queue belongs;
the second determining module is configured to determine, according to each preset hash value, the cluster identifier and the queue identifier received by the receiving module, a cache tag corresponding to each preset hash value, where the preset hash value is used to represent a hash value corresponding to an identifier of a backlog message to be cached;
and the second storage module is used for correspondingly storing each cache tag and the backlog information to be cached corresponding to the cache tag into the database.
10. The apparatus of claim 9, wherein the sending module is specifically configured to:
according to the sequence of the receiving time from morning to evening, respectively carrying out internal sequencing on backlog information to be restored corresponding to each appointed cache tag; merging and sequencing backlog messages to be restored corresponding to the designated cache tags according to the internal sequencing result of the backlog messages to be restored corresponding to the designated cache tags and the sequence of the receiving time from morning to evening; and sequentially sending each backlog message to be restored to the message system according to the merging and sorting result of the backlog messages to be restored corresponding to each appointed cache tag.
11. A message processing system comprising a target server as claimed in claims 6 to 10, and a database and messaging system as claimed in claims 6 to 10;
the database is used for receiving the cache tags sent by the target server and the backlog information to be cached corresponding to the cache tags, and correspondingly storing the backlog information to be cached corresponding to each cache tag and the cache tag into one cache unit.
12. A server is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any one of claims 1 to 5 when executing a program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910129887.7A CN109783258B (en) | 2019-02-21 | 2019-02-21 | Message processing method and device and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910129887.7A CN109783258B (en) | 2019-02-21 | 2019-02-21 | Message processing method and device and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109783258A CN109783258A (en) | 2019-05-21 |
| CN109783258B true CN109783258B (en) | 2021-02-23 |
Family
ID=66485857
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910129887.7A Active CN109783258B (en) | 2019-02-21 | 2019-02-21 | Message processing method and device and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109783258B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111683140B (en) * | 2020-06-05 | 2024-03-19 | 北京百度网讯科技有限公司 | Method and apparatus for distributing messages |
| CN111813370B (en) * | 2020-07-08 | 2023-10-31 | 上海雪湖科技有限公司 | Multi-path parallel merging and sequencing system based on FPGA |
| CN113778701A (en) * | 2021-01-07 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Message processing method and device, electronic device and medium |
| CN113296977B (en) * | 2021-02-24 | 2023-04-07 | 阿里巴巴集团控股有限公司 | Message processing method and device |
| CN113489635B (en) * | 2021-06-18 | 2023-08-18 | 深圳软牛科技有限公司 | WeChat-based message recovery method and related equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102469033A (en) * | 2010-11-01 | 2012-05-23 | 阿里巴巴集团控股有限公司 | Message subscription system and message sending method |
| CN108228363A (en) * | 2017-12-22 | 2018-06-29 | 中国银联股份有限公司 | A kind of message method and device |
| CN108270662A (en) * | 2017-12-26 | 2018-07-10 | 微梦创科网络科技（中国）有限公司 | A kind of message distributing method, device and equipment |
| CN108664337A (en) * | 2018-03-23 | 2018-10-16 | 北京奇艺世纪科技有限公司 | A kind of the message decompression method and device of distributed system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9544261B2 (en) * | 2013-08-27 | 2017-01-10 | International Business Machines Corporation | Data communications in a distributed computing environment |
| US9807015B2 (en) * | 2014-03-19 | 2017-10-31 | Dell Products L.P. | Message processing using dynamic load balancing queues in a messaging system |
| US10346226B2 (en) * | 2017-08-07 | 2019-07-09 | Time Warner Cable Enterprises Llc | Methods and apparatus for transmitting time sensitive data over a tunneled bus interface |
-
2019
- 2019-02-21 CN CN201910129887.7A patent/CN109783258B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102469033A (en) * | 2010-11-01 | 2012-05-23 | 阿里巴巴集团控股有限公司 | Message subscription system and message sending method |
| CN108228363A (en) * | 2017-12-22 | 2018-06-29 | 中国银联股份有限公司 | A kind of message method and device |
| CN108270662A (en) * | 2017-12-26 | 2018-07-10 | 微梦创科网络科技（中国）有限公司 | A kind of message distributing method, device and equipment |
| CN108664337A (en) * | 2018-03-23 | 2018-10-16 | 北京奇艺世纪科技有限公司 | A kind of the message decompression method and device of distributed system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109783258A (en) | 2019-05-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109783258B (en) | Message processing method and device and server | |
| US11513846B1 (en) | Distributed data acquisition, indexing and search system | |
| CN107748790B (en) | Online service system, data loading method, device and equipment | |
| CN106815254B (en) | Data processing method and device | |
| CN109471783B (en) | Method and device for predicting task operation parameters | |
| CN112579692B (en) | Data synchronization method, device, system, equipment and storage medium | |
| US20170109409A1 (en) | Event log versioning, synchronization, and consolidation | |
| CN112115200B (en) | Data synchronization method, device, electronic equipment and readable storage medium | |
| CN111338834B (en) | Data storage method and device | |
| CN106815223B (en) | Mass picture management method and device | |
| CN112686418A (en) | Method and device for predicting performance timeliness | |
| CN111444148B (en) | Data transmission method and device based on MapReduce | |
| US9026988B2 (en) | Code conversion method, apparatus, storage medium and request remaining time response method | |
| CN111291127B (en) | Data synchronization method, device, server and storage medium | |
| CN114401239A (en) | Metadata transmission method and device, computer equipment and storage medium | |
| CN114049065A (en) | Data processing method, device and system | |
| CN114168607A (en) | Global serial number generation method, device, equipment, medium and product | |
| CN112148705A (en) | Data migration method and device | |
| CN112799863A (en) | Method and apparatus for outputting information | |
| CN115793968A (en) | Data migration method and device and electronic equipment | |
| CN113553097B (en) | Model version management method and device | |
| CN112732757B (en) | Method, system, device, equipment and storage medium for processing degraded data | |
| CN117609257A (en) | Database data synchronous updating method, device, equipment and storage medium | |
| CN110019162B (en) | Method and device for realizing attribute normalization | |
| CN117742875A (en) | Container scheduling method, device, electronic equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |