CN116320010A - Message processing method, device, equipment and storage medium - Google Patents

Message processing method, device, equipment and storage medium Download PDF

Info

Publication number
CN116320010A
CN116320010A CN202310149798.5A CN202310149798A CN116320010A CN 116320010 A CN116320010 A CN 116320010A CN 202310149798 A CN202310149798 A CN 202310149798A CN 116320010 A CN116320010 A CN 116320010A
Authority
CN
China
Prior art keywords
message
subject
sequence
target
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310149798.5A
Other languages
Chinese (zh)
Inventor
陈燕颖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jingdong Technology Information Technology Co Ltd
Original Assignee
Jingdong Technology Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jingdong Technology Information Technology Co Ltd filed Critical Jingdong Technology Information Technology Co Ltd
Priority to CN202310149798.5A priority Critical patent/CN116320010A/en
Publication of CN116320010A publication Critical patent/CN116320010A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/2866Architectures; Arrangements
    • H04L67/30Profiles

Abstract

The present disclosure provides a message processing method, which can be applied to the internet field and the cloud computing field, and the method includes: determining a target subject message from a subject message sequence of the message queue; determining a boundary message index file from a message index file sequence according to a target message index file corresponding to the target subject message, wherein the boundary message index file is adjacent to the target message index file in time sequence and is positioned in front of the target message index file; inquiring a boundary message log file from a message log file sequence by utilizing a boundary message index which satisfies a preset time sequence rule at the message index storage moment in the boundary message index file; and deleting static message log files arranged in front of the boundary message log files in the message log file sequence to obtain a target message queue containing dynamic message log files. The present disclosure also provides a message processing apparatus, a device, a storage medium, and a program product.

Description

Message processing method, device, equipment and storage medium
Technical Field
The present disclosure relates to the field of internet and the field of cloud computing, and more particularly, to a message processing method, apparatus, device, storage medium, and program product.
Background
With the rapid development of internet technology, the demand for information interaction has increased dramatically. The message queue is used as an intermediate medium for information interaction, can store the messages (i.e. the information in the interaction process) and ensures the transmission of the messages. For example, when a message is sent where the recipient is not available, the message queue may hold the message until the message can be successfully delivered.
In the process of implementing the disclosed concept, the inventor finds that at least the following problems exist in the related art: with the frequency of message interaction and the increasing number of messages in a message queue, the operation performance of the message queue is greatly negatively affected, while the processing method for the messages in the message queue in the related art is difficult to improve the performance of the message queue while guaranteeing the consumption requirement of a consumption end, and the information interaction requirement of the consumption end is negatively affected to a certain extent.
Disclosure of Invention
In view of this, the present disclosure provides a message processing method, apparatus, device, storage medium, and program product.
One aspect of the present disclosure provides a message processing method, including:
determining target subject messages from a subject message sequence of a message queue, wherein initial subject messages in the subject message sequence are arranged according to the time sequence of respective message storage moments;
Determining a boundary message index file from a message index file sequence according to a target message index file corresponding to the target subject message, wherein the boundary message index file is adjacent to the target message index file in time sequence and is positioned in front of the target message index file;
inquiring a boundary message log file from a message log file sequence by utilizing the boundary message index which satisfies a preset time sequence rule at the message index storage moment in the boundary message index file; and
and deleting static message log files arranged in front of the boundary message log files in the message log file sequence to obtain a target message queue containing dynamic message log files.
According to an embodiment of the present disclosure, the target message queue includes a first message node, where the first message node stores a node topic message set corresponding to a topic identifier;
the message processing method further comprises the following steps:
screening a target node theme message set from the first message node under the condition that the occupied space of the first node of the first message node is larger than a node occupied space threshold value; and
and migrating the subject message set of the target node to the target idle message node in an idle state to obtain a second message node.
According to an embodiment of the present disclosure, the target message queue includes a plurality of message nodes;
the message processing method further comprises the following steps:
screening the target idle message nodes from the target message queue according to the node resource idle attribute information of each of the message nodes;
wherein the node resource idle attribute information includes at least one of:
node memory free information, node computing resource free information, node storage space free information.
According to an embodiment of the present disclosure, determining a target subject message from a sequence of subject messages of a message queue includes:
screening a first initial topic message with a consumption completion state from the topic message sequence to obtain a first initial topic message subsequence;
determining a first initial topic message meeting the preset time sequence rule in the first initial topic message subsequence as a first candidate topic message; and
and determining the target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence.
According to an embodiment of the present disclosure, the configuration retention attribute information includes a configuration retention time length;
Wherein determining a target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence comprises:
screening out outdated subject messages with the storage time longer than the configuration retention time from the subject message sequence;
screening out candidate expired subject messages with the latest storage time of the expired messages from a plurality of the expired subject messages under the condition that the number of the expired subject messages is plural;
comparing the time sequence relation between the first candidate message storage time of the first candidate subject message and the expiration message storage time of the candidate expiration subject message to obtain a first time sequence comparison result; and
and determining the target subject message from the first candidate subject message and the candidate outdated subject message according to the first time sequence comparison result.
According to an embodiment of the present disclosure, the configuration retention attribute information includes a configuration retention space;
wherein determining a target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence comprises:
Determining a second candidate topic message from the topic message sequence according to the configuration reserved space, wherein the topic message storage space formed by the initial topic message at the tail of the arrangement of the second candidate topic message in the topic message sequence is matched with a configuration reserved space threshold corresponding to the configuration reserved space;
comparing the time sequence relation between the first candidate message storage time of the first candidate subject message and the second candidate message storage time of the second candidate subject message to obtain a second time sequence comparison result; and
and determining the target subject message from the first candidate subject message and the second candidate subject message according to the second time sequence comparison result.
According to an embodiment of the present disclosure, the configuration retention attribute information includes a configuration retention time length and a configuration retention space;
wherein determining a target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence comprises:
screening out outdated subject messages with the storage time longer than the configuration retention time from the subject message sequence;
Screening out candidate expired subject messages with the latest storage time of the expired messages from a plurality of the expired subject messages under the condition that the number of the expired subject messages is plural;
determining a second candidate topic message from the topic message sequence according to the configuration reserved space, wherein the topic message storage space formed by the initial topic message at the tail of the arrangement of the second candidate topic message in the topic message sequence is matched with a configuration reserved space threshold corresponding to the configuration reserved space;
comparing the time sequence relation among the first candidate message storage time of the first candidate subject message, the expiration message storage time of the candidate expiration subject message and the second candidate message storage time of the second candidate subject message to obtain a third time sequence comparison result; and
and determining the target subject message from the first candidate subject message, the candidate outdated subject message and the second candidate subject message according to the third time sequence comparison result.
According to an embodiment of the present disclosure, using the boundary message index in the above-mentioned boundary message index file, where the message index storage time satisfies a preset timing rule, querying the boundary message log file from the message log file sequence includes:
Inquiring boundary information related to the boundary information index with the latest information index storage time in the boundary information index file; and
and determining a message log file storing the boundary message as the boundary message log file.
According to an embodiment of the present disclosure, before deleting the static message log file, the message processing method further includes:
and deleting the boundary message index file and the static message index file arranged in front of the boundary message index file in the message index file sequence respectively to obtain a dynamic message index file.
Another aspect of the present disclosure also provides a message processing apparatus, including:
the system comprises a target topic message determining module, a target topic message sending module and a target topic message receiving module, wherein the target topic message determining module is used for determining target topic messages from a topic message sequence of a message queue, and initial topic messages in the topic message sequence are arranged according to the time sequence of respective message storage moments;
the boundary message index file determining module is used for determining a boundary message index file from a message index file sequence according to a target message index file corresponding to the target subject message, wherein the boundary message index file is adjacent to the target message index file in time sequence and is positioned in front of the target message index file;
The boundary message log file inquiring module is used for inquiring the boundary message log file from the message log file sequence by utilizing the boundary message index which satisfies the preset time sequence rule at the message index storing moment in the boundary message index file; and
and the first deleting module is used for deleting static message log files arranged in front of the boundary message log files in the message log file sequence to obtain a target message queue containing dynamic message log files.
Another aspect of the present disclosure provides an electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the message processing method as described above.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, are configured to implement a message processing method as described above.
Another aspect of the present disclosure provides a computer program product comprising computer executable instructions which, when executed, are for implementing a message processing method as described above.
According to the embodiment of the disclosure, the technical means that the boundary message index and the corresponding boundary message log file are queried by determining the target subject message in the subject message sequence and according to the target subject message, and the boundary message log file is used as the boundary between the static message log file and the dynamic message log file in the differential message log file sequence is adopted, so that the technical problem of improving the accuracy of identifying the consumed demand of the message log file in the message queue is at least partially solved, and the technical effects of improving the message cleaning efficiency, improving the storage space utilization rate of the message queue and improving the operation performance can be achieved by deleting the static log file, saving the storage space in the message queue.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings in which:
FIG. 1 schematically illustrates an exemplary system architecture to which message processing methods, apparatuses may be applied, according to embodiments of the present disclosure;
FIG. 2 schematically illustrates a flow chart of a message processing method according to an embodiment of the disclosure;
FIG. 3 schematically illustrates a flow diagram for determining a target subject message from a sequence of subject messages of a message queue, according to an embodiment of the disclosure;
FIG. 4 schematically illustrates a flow chart of a message processing method according to another embodiment of the present disclosure;
fig. 5 schematically illustrates a flow chart of a message processing method according to a further embodiment of the present disclosure;
fig. 6 schematically illustrates a block diagram of a message processing apparatus according to an embodiment of the disclosure; and
fig. 7 schematically illustrates a block diagram of an electronic device adapted to implement a message processing method according to an embodiment of the disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.
Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
For clarity of presentation of the technical solutions of the embodiments of the present disclosure, the details of the embodiments of the present disclosure may be understood with reference to the following technical term explanations.
Topic or Topic identification, i.e. Topic, is used for recording or distinguishing the classification identifications of different messages, and it is generally required to store the messages according to the Topic in a classification manner, and the messages in the same Topic can form a Topic message sequence.
The message log file, which stores the subject message, may be the base unit of message queue communications.
The message node, i.e. Broker, may be a server or a cluster of servers in a message queue. The same message node can store the topic messages corresponding to different topics respectively, and accordingly, topic messages in the same topic message sequence can also be stored in different message nodes.
Message queues are commonly used as a message middleware for decoupling service system messages, message transfer and the like. As user data grows, the message logs stored in the message queues grow, and the storage pressure brought to the message queues also grows. How to timely and automatically clean the expired message log becomes a key item for measuring the carrying and operation and maintenance capabilities of the message queue.
In the related art, most of the clearing strategies or methods for the messages in the message queue refer to the time dimension attribute of the message to delete the corresponding message log file, and the interface tool of the message node is usually required to be called to complete the clearing work of the message. This may result in that messages for which there is still a consumed demand are deleted, and it is difficult to delete inactive states to the maximum extent, i.e. static messages no longer having a consumed demand, resulting in a larger memory space occupation of the message nodes, affecting the overall operation performance of the message queue.
Embodiments of the present disclosure provide a message processing method, apparatus, device, storage medium, and program product. The message processing method comprises the following steps: determining target topic messages from a topic message sequence of a message queue, wherein initial topic messages in the topic message sequence are arranged according to the time sequence of respective message storage moments; determining a boundary message index file from a message index file sequence according to a target message index file corresponding to the target subject message, wherein the boundary message index file is adjacent to the target message index file in time sequence and is positioned in front of the target message index file; inquiring a boundary message log file from a message log file sequence by utilizing a boundary message index which satisfies a preset time sequence rule at the message index storage moment in the boundary message index file; and deleting static message log files arranged in front of the boundary message log files in the message log file sequence to obtain a target message queue containing dynamic message log files.
According to the embodiment of the disclosure, the target topic message in the topic message sequence is determined, the boundary message index and the corresponding boundary message log file are queried according to the target topic message, and the boundary message log file is used as the boundary between the static message log file and the dynamic message log file in the differential message log file sequence, so that the accuracy of identifying the consumed demand of the message log file in the message queue is at least partially improved, the message cleaning efficiency is improved, and therefore, the storage space in the message queue is saved by deleting the static log file, and the technical effects of improving the storage space utilization rate and the running performance of the message queue are realized.
In the technical scheme of the disclosure, the authorization or consent of the user is obtained before the personal information of the user is obtained or acquired.
In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing, applying and the like of the personal information of the user all conform to the regulations of related laws and regulations, necessary security measures are adopted, and the public order harmony is not violated.
Fig. 1 schematically illustrates an exemplary system architecture to which message processing methods, apparatuses may be applied according to embodiments of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.
As shown in fig. 1, a system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, a message queue 110, and consumer servers 121, 123. Message queue 110 may include message nodes 111, 112, and 113.
The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the message queue 110. The network 104 may include various connection types, such as wired and/or wireless communication links, and the like.
A user may interact with the message queue 110 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients and/or social platform software, to name a few.
The terminal devices 101, 102, 103 may be a variety of electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
It should be appreciated that the terminal devices 101, 102, 103 may be producers of subject messages and that the message queue 110 may obtain subject messages from the terminal devices 101, 102, 103 and store the subject messages as subject matter in any of the message nodes 111, 112, 113. The message nodes 111, 112, 113 may comprise one server or may comprise a server cluster comprising a plurality of servers.
The consumer servers 121, 123 may be servers providing various services, such as a background management server (by way of example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and process the received data such as the user request, and feed back the processing result (e.g., the web page, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that, the message processing method provided in the embodiments of the present disclosure may be generally performed by the message queue 110. Accordingly, the message processing apparatus provided by the embodiments of the present disclosure may be generally disposed in the message queue 110. The message processing method provided by the embodiments of the present disclosure may also be performed by a server or a server cluster that is different from the message queue 110 and is capable of communicating with the terminal devices 101, 102, 103 and/or the message queue 110. Accordingly, the message processing apparatus provided by the embodiments of the present disclosure may also be provided in a server or a server cluster different from the message queue 110 and capable of communicating with the terminal devices 101, 102, 103 and/or the message queue 110.
It should be understood that the number of end devices, networks, message nodes, message queues, and consumption servers in fig. 1 are merely illustrative. There may be any number of terminal devices, networks, message nodes, message queues, and consumption servers, as desired for implementation.
Fig. 2 schematically illustrates a flow chart of a message processing method according to an embodiment of the present disclosure.
As shown in fig. 2, the message processing method includes operations S210 to S240.
In operation S210, a target subject message is determined from a subject message sequence of a message queue, wherein initial subject messages in the subject message sequence are arranged in a time sequence of respective message storage moments.
According to the embodiment of the disclosure, the topic message sequence may be messages stored in a message queue according to topic categories, topic messages in the same topic message sequence have the same topic type, for example, may have the same topic identifier, so that a consuming end can conveniently call messages in the corresponding topic message sequence from the message queue, and a consuming process of the messages is realized.
According to an embodiment of the present disclosure, the message storing time may be a time when the message producer submits the subject message to the message queue, i.e. a time when the subject message is stored in the message queue. It is understood that the consuming side may consume the subject message in time sequence with respect to the message storage time. The target subject message may thus still be borderline by the consumed demand as subject messages in the sequence of subject messages.
It should be appreciated that message storage times may be represented by respective time stamps of subject messages.
In operation S220, a boundary message index file is determined from the sequence of message index files according to the target message index file corresponding to the target subject message, wherein the boundary message index file is adjacent to the target message index file in time sequence and is located before the target message index file.
According to embodiments of the present disclosure, the target message index file may contain a target subject message index that can be associated with a target subject message. By determining the boundary index file from the target message index file, the message index file that was generated and/or updated prior to the target message index file may be clearly queried.
In operation S230, the boundary message log file is queried from the sequence of message log files using the boundary message index in which the message index storage time satisfies the preset timing rule.
According to an embodiment of the present disclosure, the boundary message index satisfying the preset timing rule may be a message index satisfying a preset time period attribute (e.g., generated during a preset time period) in the boundary message index file, or may also be a message index satisfying a preset time attribute (e.g., generated at a preset time) in the boundary message index file. The embodiment of the disclosure does not limit a specific design manner of the preset timing rule, and a person skilled in the art can design according to actual requirements, so long as the time attribute correlation can be satisfied.
The message index file sequence and the message log file sequence may be file sequences arranged according to time sequence attributes of the message index file and the message log file, respectively.
In operation S240, the static message log file arranged before the boundary message log file in the message log file sequence is deleted, so as to obtain the target message queue containing the dynamic message log file.
According to embodiments of the present disclosure, a static message log file may be composed of static messages that may be predicted to be less or not as consumed demand messages. The dynamic message log file may be made up of dynamic messages that may be predicted to still have consumed demand or messages that are more likely to be consumed. By deleting the static message log file, the static messages can be quickly deleted in batches, so that the storage space in the message queue is saved.
It should be noted that, in embodiments of the present disclosure, the number of subject message sequences in the message queue is not limited, and the message queue may include any number of subject message sequences. Accordingly, the static message log files of one or more subject message sequences can be deleted based on the same or similar method, so that the messages in the message queue are cleared as much as possible, and the free storage space of the message queue is improved.
According to the embodiment of the disclosure, the target topic message in the topic message sequence is determined, the boundary message index and the corresponding boundary message log file are queried according to the target topic message, and the boundary message log file is used as the boundary between the static message log file and the dynamic message log file in the differential message log file sequence, so that the accuracy of identifying the consumed demand of the message log file in the message queue is at least partially improved, the message cleaning efficiency is improved, and therefore, the storage space in the message queue is saved by deleting the static log file, and the technical effects of improving the storage space utilization rate and the running performance of the message queue are realized.
Fig. 3 schematically illustrates a flow diagram for determining a target subject message from a sequence of subject messages of a message queue, according to an embodiment of the disclosure.
As shown in fig. 3, operation S210, determining a target subject message from the subject message sequence of the message queue may include operations S310 to S330.
In operation S310, a first initial subject message having a consumption completion status is filtered out of the subject message sequence to obtain a first initial subject message subsequence.
In operation S320, the first initial subject message satisfying the preset timing rule among the first initial subject message sub-sequences is determined as the first candidate subject message.
In operation S330, a target subject message is determined from the sequence of subject messages according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the sequence of subject messages.
According to the embodiment of the disclosure, the consumption completion status may indicate whether the message is completely consumed by the consuming end, for example, may indicate that the message is consumed at least once by all the consuming ends, or may also indicate that the message is consumed by the consuming end having the corresponding topic consumption right for a preset frequency, or may also update the consumption completion status of the topic message in the topic message sequence through one or more consuming ends. The specific design manner of the consumer completion status is not limited in the embodiments of the present disclosure, and may be selected by those skilled in the art according to actual needs.
In one embodiment of the present disclosure, the consumption state of the first initial subject message may be determined as a consumption completion state in a case where all consumption terminals connected to the message queue consume the first initial subject message.
In one embodiment of the present disclosure, the last sorting in the first initial topic message sub-sequence, that is, the first initial topic message with the latest message storage time, may be determined as the first candidate topic message, so as to improve the prediction accuracy of the subsequent static message log file.
According to embodiments of the present disclosure, the configuration retention attribute information may correspond to storage requirements of the sequence of subject messages, such as storage size requirements, storage duration requirements, subject message attribute type requirements, and the like. Those skilled in the art may determine the configuration retention attribute information according to actual requirements, and the embodiment of the present disclosure does not limit a specific information type of the configuration retention attribute information, so long as the corresponding storage requirement can be satisfied.
According to the embodiment of the disclosure, the consumption completion state of the subject message and the configuration retention attribute information of the subject message sequence are comprehensively considered to determine the target subject information, so that the cleaning strategy of the message queue can be determined from multiple dimensions, the same consumption requirement on the subject message and the same consumption queue message cleaning requirement are realized, and the actual data processing requirement is met.
According to an embodiment of the present disclosure, configuring the persistence attribute information may include configuring a persistence duration.
In operation S330, determining the target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence may include the following operations.
Screening out outdated subject messages with the storage time longer than the configuration retention time from the subject message sequence; screening out candidate expired subject messages with the latest storage time of the expired messages from the plurality of expired subject messages under the condition that the number of the expired subject messages is multiple; comparing the time sequence relation between the first candidate message storage time of the first candidate subject message and the expiration message storage time of the candidate expiration subject message to obtain a first time sequence comparison result; and determining a target subject message from the first candidate subject message and the candidate outdated subject message according to the first time sequence comparison result.
According to embodiments of the present disclosure, the configuration persistence duration may characterize the storage requirements of the sequence of subject messages. The message storage time of each subject message in the subject message sequence can be calculated. The method comprises the steps of comparing respective message storage moments of a first candidate topic message and a candidate expired topic message, and determining the topic message as a target topic message according to the topic message with an earlier time sequence relationship represented by a first time sequence comparison result, such as a topic message with an earlier time sequence relationship represented by a timestamp, so as to avoid mistaken deletion of topic messages with consumed demands, and improve the accuracy of message cleaning.
According to an embodiment of the present disclosure, configuring the persistence attribute information includes configuring a persistence space.
In operation S330, determining the target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence includes the following operations:
determining a second candidate topic message from the topic message sequence according to the configuration retention space, wherein the topic message storage space formed by the initial topic message from the second candidate topic message to the tail of the arrangement in the topic message sequence is matched with a configuration retention space threshold corresponding to the configuration retention space; comparing the time sequence relation between the first candidate message storage time of the first candidate subject message and the second candidate message storage time of the second candidate subject message to obtain a second time sequence comparison result; and determining a target subject message from the first candidate subject message and the second candidate subject message according to the second time sequence comparison result.
According to embodiments of the present disclosure, the configuration persistence space may characterize the data volume requirements for subject messages having the same subject, e.g., the configuration persistence space may be set by setting a minimum stored data volume for a sequence of subject messages. The configuration persistence space threshold may be a storage space threshold corresponding to the configuration persistence space, for example, when the configuration persistence space is 1GB, the configuration persistence space threshold may be set to 0.9GB to 1.1GB. In the case that a topic message memory space formed by one topic message in the topic message sequence and the initial topic message at the end of the arrangement is 0.95GB, the topic message can be determined as a second candidate topic message.
According to an embodiment of the present disclosure, the second candidate subject message may be determined by traversing forward from the end of the sequence of subject messages.
According to the embodiment of the disclosure, the topic message with the earlier message storage time in the second candidate topic message and the first candidate topic message can be determined as the target topic message, so that the unification of the configuration reserved space and the consumption state of the cleaned topic message sequence is realized, and further, the consumption requirement of a consumption end is met while the working performance of the message queue is improved.
According to an embodiment of the present disclosure, the configuration persistence attribute information includes a configuration persistence duration and a configuration persistence space.
In operation S330, determining the target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence may include the operations of:
screening out outdated subject messages with the storage time longer than the configuration retention time from the subject message sequence; screening out candidate expired subject messages with the latest storage time of the expired messages from the plurality of expired subject messages under the condition that the number of the expired subject messages is multiple; determining a second candidate topic message from the topic message sequence according to the configuration retention space, wherein the topic message storage space formed by the initial topic message from the second candidate topic message to the tail of the arrangement in the topic message sequence is matched with a configuration retention space threshold corresponding to the configuration retention space; comparing the time sequence relation between the first candidate message storage time of the first candidate subject message, the expiration message storage time of the candidate expiration subject message and the second candidate message storage time of the second candidate subject message to obtain a third time sequence comparison result; and determining a target subject message from the first candidate subject message, the candidate expired subject message and the second candidate subject message according to the third time sequence comparison result.
According to the embodiment of the disclosure, the topic message with the earlier message storage time in the candidate expired topic message, the second candidate topic message and the first candidate topic message can be determined as the target topic message, so that the unification of the retention time requirement, the retention space requirement and the consumption state of the cleaned topic message sequence is realized, and further, the consumption requirement of a consumption end is met while the working performance of the message queue is improved.
According to an embodiment of the present disclosure, in operation S230, using the boundary message index in the boundary message index file, in which the message index storage time satisfies the preset timing rule, querying the boundary message log file from the message log file sequence may include the operations of:
inquiring boundary information associated with the boundary information index with the latest information index storage time in the boundary information index file; and determining a message log file storing the boundary message as a boundary message log file.
According to the embodiment of the disclosure, when the message index file is ordered according to the generation time of the message index, the boundary message index with the latest message index storage time can be the last message index ordered in the boundary message index file, so that the rate of searching the boundary message index can be improved, and the overall efficiency of message cleaning of the message queue is further improved.
According to an embodiment of the present disclosure, before deleting the static message log file in operation S240, the message processing method may further include the operations of:
and respectively deleting the boundary message index file and static message index files arranged in front of the boundary message index file in the message index file sequence to obtain a dynamic message index file.
According to the embodiment of the disclosure, the static message index file is deleted before the static message log file is deleted, so that the spare storage space of the message queue can be further enlarged, meanwhile, the error reporting situations such as the empty index and the like in the message query process caused by the undeleted message index can be avoided, and the running stability of the message queue is improved, so that the running stability of the message queue is improved.
Fig. 4 schematically illustrates a flow chart of a message processing method according to another embodiment of the present disclosure.
As shown in fig. 4, the message processing method in this embodiment may include operations S401 to S417.
After the start operation S401 is performed, operation 402 may be performed to determine whether there is a forced deletion attribute. I.e., whether the subject message sequence has an attribute that can be forcedly deleted.
In the case where the result of the determination of operation S402 is no, operation S403 may be performed to query the first candidate subject message identifier P0 in the subject message sequence.
In the case that the result of the determination in operation S402 is yes, operation S404 may be performed to query the last subject message, that is, to query the subject message identification at the end of the sequence arrangement of the subject messages. Then, operation S405 is performed again.
It should be noted that P0 may be a first candidate topic message identifier, topic messages in the topic message sequence may be arranged according to respective message storage moments, and accordingly, a timing relationship between different topic messages may be represented according to respective message identifiers.
For example, in the sequence of topic messages, adjacent topic messages may have topic message identifications 111 and 112, respectively, and then it may be understood that the topic message having topic identification 111 is stored earlier in time than the topic message having topic identification 112.
In operation S405, it is determined whether the subject message sequence has a configuration persistence duration. In the case that the result of the determination of operation S405 is yes, operation S406 may be performed to query the candidate expired subject message identifier P1 in the subject message sequence. Accordingly, in case the determination result of operation S405 is no, operation S407 may be performed to determine that the candidate outdated subject message identification P1 is the same as the message identification P0 of the first candidate subject message.
Then, operation S408 is performed to determine whether there is a configuration save space. In the case where the result of the judgment of operation S408 is yes, operation S409 may be performed, otherwise operation S410 is performed.
In operation S409, the second candidate subject message identification P2 in the subject message sequence is queried. Accordingly, in operation S410, the second candidate subject message identity P2 may be determined as the first candidate subject message identity, i.e., p2=p0 may be determined.
According to the execution results of operations S401 to S410, the target subject message identity P may be determined according to the second candidate subject message identity P2, the candidate expired subject message identity P1, and the first candidate subject message identity in operation S411. The target subject message identity P may be determined, for example, according to equation (1).
P=min(P0,P1,P2);(1)
Operation S412 may then be performed to query the target message index file according to the target subject message. For example, from a sequence of message index files, a target index is queried that can index into a target subject message, and then a target message index file is queried that contains the target message index.
In operation S413, a boundary message index file is determined from the message index file sequence according to the target message index file. I.e., a message index file that is adjacent to the timing of the target message index file and that is located before the target message index file may be queried.
Then, operation S414 is performed to query the boundary message log file using the boundary message index in the boundary message index file. For example, the message index at the end of the sorting in the boundary message index file can be determined as the boundary message index, and then the boundary message log file associated with the boundary message index is queried.
In operation S415, the boundary message index file is deleted, and the static message index file in the message index file sequence.
In operation S416, the static message log file arranged before the boundary message log file in the message log file sequence is deleted. The message queue will typically create a message index file of the message log file at the time of creation. Accordingly, the message index file is loaded into the cache of the message node and is not deleted arbitrarily. Therefore, by deleting the boundary message index file and the static message index file, the condition that the message queue can inquire the empty message log file can be avoided, and meanwhile, the storage space in the message queue can be further cleaned, so that the operation efficiency and the operation stability of the message queue are improved.
After deleting the corresponding boundary message index file, static message index file, and static message log file, operation S417 may be performed to finish the message processing procedure for one cycle.
It should be understood that the same or similar method may be used to perform the message processing procedure for a plurality of periods on the message queue, and the number of periods of the message processing procedure in the message processing method according to the embodiments of the present disclosure is not limited, and may be selected by those skilled in the art according to actual needs.
According to an embodiment of the present disclosure, a target message queue includes a first message node storing a set of node topic messages corresponding to a topic identification.
Fig. 5 schematically shows a flow chart of a message processing method according to a further embodiment of the present disclosure.
As shown in fig. 5, the message processing method of this embodiment may further include operations S510 to S520.
In operation S510, a subject message set of a target node is screened from the first message node if the first node footprint of the first message node is greater than the node footprint threshold.
In operation S520, the subject message set of the target node is migrated to the target idle message node in the idle state, to obtain a second message node.
According to embodiments of the present disclosure, subject messages in a subject message sequence may be stored in one or more message nodes of a message queue, and thus in a target message queue cleaned up of static message log files, the same subject identification corresponding dynamic message log file may be stored in one or more first message nodes. The dynamic message log file stored in the same first message node may constitute a set of node topic messages for the same topic identification. In the case where the node topic message set includes a plurality, the target node topic message set may be any one of the plurality of node topic message sets.
According to the embodiment of the disclosure, in the case that the occupied space of the first node is larger than the occupied space threshold value of the node, the first message node still needs to be further cleaned up. The storage space of the first message node can be further emptied by migrating the target node subject message set with the largest occupied space to the target idle message node, for example, copying the target node subject message set from the first message node to the target idle node, and then deleting the target node subject message set from the first message node, and the running stability and the running performance of the first message node are improved by improving the spare storage space.
In one embodiment of the present disclosure, the target node topic message set may be a node topic message set occupying the largest storage space in the first message node. Therefore, the cleaning efficiency of the first message node can be improved by migrating the subject message set of the target node.
It should be noted that, after the target node theme message set is migrated to the target idle node, if the occupied space of the first node of the first message node is still greater than the node occupied space threshold, a new target node theme message set in the first message node may be iteratively cleaned until the occupied space of the first node is less than or equal to the node occupied space threshold, thereby obtaining the first message node meeting the storage space requirement. The embodiments of the present disclosure do not limit the frequency of the subject message set of the migration target node, and those skilled in the art may design according to actual requirements.
According to the embodiment of the disclosure, the target idle message node may be one message node in the target message queue, or may also be a message node constructed by other manners, for example, a server or a server cluster may be utilized to construct a new target idle message node, and the first message node of the target message queue establishes a communication connection, so as to realize migration of a subsequent target node theme message. The embodiment of the present disclosure does not limit the specific manner of determining the target idle message node, and those skilled in the art may select according to actual requirements.
According to an embodiment of the present disclosure, a target message queue includes a plurality of message nodes.
The message processing method further comprises the following operations:
and screening out the target idle message nodes from the target message queue according to the node resource idle attribute information of each of the message nodes.
Wherein the node resource idle attribute information includes at least one of:
node memory free information, node computing resource free information, node storage space free information.
According to embodiments of the present disclosure, the message node may include a first message node in the target message queue, or may further include a new idle message node constructed, and the specific type of the message node is not limited in the embodiments of the present disclosure.
According to embodiments of the present disclosure, the node memory idle information may include idle resources of memory in a server or a server cluster in the message node, such as a real-time memory idle rate, an average memory idle rate, and the like.
According to embodiments of the present disclosure, the node computing resource idle information may characterize the computing resource idle condition of the node, such as real-time computing resource idle rate, average computing resource idle rate, and so on.
According to the embodiment of the disclosure, the node storage space idle information can represent the idle storage space of the message node in real time and/or in a preset time period, and a person skilled in the art can set the node storage space idle information according to actual requirements.
According to the embodiment of the disclosure, the target idle message node is determined according to the node resource idle attribute information, so that the target idle message node with more abundant node resources can be provided for the subsequent migration target subject message set, and the problem that the node resource of the second message node is insufficient after the migration target subject message set is avoided, thereby realizing the technical effects of improving the operation performance and the operation stability of the target message queue.
Fig. 6 schematically illustrates a block diagram of a message processing apparatus according to an embodiment of the disclosure.
As shown in fig. 6, the message processing apparatus 600 includes a target subject message determination module 610, a boundary message index file determination module 620, a boundary message log file query module 630, and a first deletion module 640.
The target topic message determination module 610 is configured to determine a target topic message from a topic message sequence of a message queue, where initial topic messages in the topic message sequence are arranged according to a time sequence of respective message storage moments.
The boundary message index file determining module 620 is configured to determine a boundary message index file from a sequence of message index files according to a target message index file corresponding to a target subject message, where the boundary message index file is adjacent to the target message index file in time sequence and is located before the target message index file.
The boundary message log file querying module 630 is configured to query a boundary message log file from the sequence of message log files by using a boundary message index in the boundary message index file, where the message index storage time satisfies a preset timing rule.
The first deleting module 640 is configured to delete a static message log file arranged before a boundary message log file in the message log file sequence, so as to obtain a target message queue containing dynamic message log files.
According to an embodiment of the present disclosure, a target message queue includes a first message node storing a set of node topic messages corresponding to a topic identification.
The message processing apparatus further includes: the system comprises a target node theme message set screening module and a migration module.
The target node topic message set screening module is used for screening the target node topic message set from the first message node under the condition that the occupied space of the first node of the first message node is larger than the occupied space threshold value of the node.
The migration module is used for migrating the subject message set of the target node to the target idle message node in the idle state to obtain a second message node.
According to an embodiment of the present disclosure, a target message queue includes a plurality of message nodes.
The message processing apparatus further includes a target idle message node screening module.
The target idle message node screening module is used for screening the target idle message nodes from the target message queue according to the node resource idle attribute information of each of the message nodes.
Wherein the node resource idle attribute information includes at least one of:
node memory free information, node computing resource free information, node storage space free information.
According to an embodiment of the present disclosure, the target subject message determination module 610 includes: the device comprises a first screening unit, a first determining unit and a second determining unit.
The first screening unit is used for screening a first initial topic message with a consumption completion state from the topic message sequence to obtain a first initial topic message subsequence.
The first determining unit is used for determining a first initial topic message meeting a preset time sequence rule in the first initial topic message subsequence as a first candidate topic message.
The second determining unit is used for determining a target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence.
According to an embodiment of the present disclosure, configuring the persistence attribute information includes configuring a persistence duration.
The second determination unit includes: the first screening subunit, the second screening subunit, the first comparing subunit, and the first determining subunit.
The first screening subunit is configured to screen out expired topic messages with a storage time longer than the configuration retention time from the topic message sequence.
The second screening subunit is configured to screen, when the number of expired subject messages is multiple, candidate expired subject messages with the latest storage time of the expired messages from the multiple expired subject messages.
The first comparing subunit is configured to compare the time sequence relationship between the first candidate message storage time of the first candidate subject message and the expiration message storage time of the candidate expiration subject message, and obtain a first time sequence comparison result.
The first determining subunit is configured to determine, according to the first timing comparison result, a target subject message from the first candidate subject message and the candidate expired subject message.
According to an embodiment of the present disclosure, configuring the persistence attribute information includes configuring a persistence space.
The second determination unit includes: a second candidate topic message determination subunit, a second comparison subunit, and a second determination subunit.
The second candidate topic message determining subunit is configured to determine a second candidate topic message from the topic message sequence according to the configuration reserved space, where a topic message storage space formed by the initial topic message from the second candidate topic message to the end of the arrangement in the topic message sequence matches with a configuration reserved space threshold corresponding to the configuration reserved space.
The second comparing subunit is configured to compare the time sequence relationship between the first candidate message storage time of the first candidate subject message and the second candidate message storage time of the second candidate subject message, so as to obtain a second time sequence comparison result.
The second determining subunit is configured to determine, according to the second timing comparison result, a target subject message from the first candidate subject message and the second candidate subject message.
According to an embodiment of the present disclosure, the configuration persistence attribute information includes a configuration persistence duration and a configuration persistence space.
Wherein the second determining unit includes: a third screening subunit, a fourth screening subunit, a third determination subunit, a third comparison subunit, and a fourth determination subunit.
And the third screening subunit is used for screening out expired subject messages with the storage time longer than the configuration retention time from the subject message sequence.
The fourth screening subunit is configured to screen, when there are multiple expired subject messages, a candidate expired subject message with the latest storage time of the expired message from the multiple expired subject messages.
And the third determining subunit is configured to determine a second candidate topic message from the topic message sequence according to the configuration reserved space, where the topic message storage space formed by the initial topic message from the second candidate topic message to the end of the arrangement in the topic message sequence matches with the configuration reserved space threshold corresponding to the configuration reserved space.
The third comparing subunit is configured to compare the time sequence relationship between the first candidate message storage time of the first candidate subject message, the expiration message storage time of the candidate expiration subject message, and the second candidate message storage time of the second candidate subject message, to obtain a third time sequence comparison result.
The fourth determining subunit is configured to determine, according to the third timing comparison result, the target subject message from the first candidate subject message, the candidate expired subject message, and the second candidate subject message.
According to an embodiment of the present disclosure, the boundary message log file querying module 630 includes: a first query unit and a boundary message log file determination unit.
The first query unit is used for querying boundary messages associated with the boundary message index with the latest message index storage time in the boundary message index file.
The boundary message log file determining unit is configured to determine a message log file storing boundary messages as a boundary message log file.
According to an embodiment of the present disclosure, the message processing apparatus further includes a second deletion module.
The second deleting module is used for deleting the boundary message index file and the static message index file arranged in front of the boundary message index file in the message index file sequence respectively to obtain the dynamic message index file.
Any number of the modules, units, sub-units, or at least some of the functionality of any number of the modules, units, sub-units, or sub-units according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, units, sub-units according to embodiments of the present disclosure may be implemented as split into multiple modules. Any one or more of the modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware in any other reasonable manner of integrating or packaging the circuits, or in any one of or in any suitable combination of three of software, hardware, and firmware. Alternatively, one or more of the modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as computer program modules, which when executed, may perform the corresponding functions.
For example, any of the target subject message determination module 610, the boundary message index file determination module 620, the boundary message log file query module 630, and the first deletion module 640 may be combined in one module/unit/sub-unit or any of the modules/units/sub-units may be split into multiple modules/units/sub-units. Alternatively, at least some of the functionality of one or more of these modules/units/sub-units may be combined with at least some of the functionality of other modules/units/sub-units and implemented in one module/unit/sub-unit. According to embodiments of the present disclosure, at least one of the targeted subject message determination module 610, the boundary message index file determination module 620, the boundary message log file query module 630, and the first deletion module 640 may be implemented, at least in part, as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system-on-chip, a system-on-substrate, a system-on-package, an Application Specific Integrated Circuit (ASIC), or as hardware or firmware in any other reasonable manner of integrating or packaging circuitry, or as any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the targeted topic message determination module 610, the boundary message index file determination module 620, the boundary message log file query module 630, and the first delete module 640 may be implemented, at least in part, as a computer program module that, when executed, performs the corresponding functions.
It should be noted that, in the embodiment of the present disclosure, the message processing apparatus portion corresponds to the message processing method portion in the embodiment of the present disclosure, and the description of the message processing apparatus portion specifically refers to the message processing method portion and is not described herein again.
Fig. 7 schematically illustrates a block diagram of an electronic device adapted to implement a message processing method according to an embodiment of the disclosure. The electronic device shown in fig. 7 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
As shown in fig. 7, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. The processor 701 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 701 may also include on-board memory for caching purposes. The processor 701 may comprise a single processing unit or a plurality of processing units for performing different actions of the method flows according to embodiments of the disclosure.
In the RAM703, various programs and data necessary for the operation of the electronic apparatus 700 are stored. The processor 701, the ROM702, and the RAM703 are connected to each other through a bus 704. The processor 701 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM702 and/or the RAM 703. Note that the program may be stored in one or more memories other than the ROM702 and the RAM 703. The processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, the electronic device 700 may further include an input/output (I/O) interface 705, the input/output (I/O) interface 705 also being connected to the bus 704. The system 700 may also include one or more of the following components connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
According to embodiments of the present disclosure, the method flow according to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 702 and/or RAM703 and/or one or more memories other than ROM 702 and RAM703 described above.
Embodiments of the present disclosure also include a computer program product comprising a computer program comprising program code for performing the methods provided by the embodiments of the present disclosure, the program code for causing an electronic device to implement the message processing methods provided by the embodiments of the present disclosure when the computer program product is run on the electronic device.
The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 701. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.
In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed over a network medium in the form of signals, downloaded and installed via the communication section 709, and/or installed from the removable medium 711. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.
The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims (13)

1. A message processing method, comprising:
determining target topic messages from a topic message sequence of a message queue, wherein initial topic messages in the topic message sequence are arranged according to the time sequence of respective message storage moments;
determining a boundary message index file from a message index file sequence according to a target message index file corresponding to the target subject message, wherein the boundary message index file is adjacent to the target message index file in time sequence and is positioned in front of the target message index file;
inquiring a boundary message log file from a message log file sequence by utilizing boundary message indexes which meet a preset time sequence rule at the message index storage moment in the boundary message index file; and
And deleting static message log files arranged in front of the boundary message log files in the message log file sequence to obtain a target message queue containing dynamic message log files.
2. The method of claim 1, wherein the target message queue comprises a first message node storing a set of node topic messages corresponding to topic identifications;
the message processing method further comprises the following steps:
screening a target node theme message set from the first message node under the condition that the occupied space of the first node of the first message node is larger than a node occupied space threshold value; and
and migrating the subject message set of the target node to the target idle message node in an idle state to obtain a second message node.
3. The method of claim 2, wherein the target message queue comprises a plurality of message nodes;
the message processing method further comprises the following steps:
screening the target idle message nodes from the target message queue according to the node resource idle attribute information of each of the message nodes;
wherein the node resource idle attribute information includes at least one of:
Node memory free information, node computing resource free information, node storage space free information.
4. The method of claim 1, wherein determining a target subject message from a sequence of subject messages of a message queue comprises:
screening a first initial topic message with a consumption completion state from the topic message sequence to obtain a first initial topic message subsequence;
determining a first initial topic message meeting the preset time sequence rule in the first initial topic message subsequence as a first candidate topic message; and
and determining a target subject message from the subject message sequence according to the first candidate message storage time of the first candidate subject message and the configuration retention attribute information of the subject message sequence.
5. The method of claim 4, wherein the configuration retention attribute information comprises a configuration retention time period;
wherein determining a target subject message from the subject message sequence according to a first candidate message storage time of the first candidate subject message and configuration retention attribute information of the subject message sequence comprises:
screening out outdated subject messages with the storage time longer than the configuration retention time from the subject message sequence;
Screening out candidate expired subject messages with the latest storage time of the expired messages from a plurality of expired subject messages under the condition that the number of the expired subject messages is plural;
comparing the time sequence relation between the first candidate message storage time of the first candidate subject message and the expiration message storage time of the candidate expiration subject message to obtain a first time sequence comparison result; and
and determining the target subject message from the first candidate subject message and the candidate outdated subject message according to the first time sequence comparison result.
6. The method of claim 4, wherein the configuration persistence attribute information comprises a configuration persistence space;
wherein determining a target subject message from the subject message sequence according to a first candidate message storage time of the first candidate subject message and configuration retention attribute information of the subject message sequence comprises:
determining a second candidate topic message from the topic message sequence according to the configuration reserved space, wherein the topic message storage space formed by the initial topic message from the second candidate topic message to the tail of the arrangement in the topic message sequence is matched with a configuration reserved space threshold corresponding to the configuration reserved space;
Comparing the time sequence relation between the first candidate message storage time of the first candidate subject message and the second candidate message storage time of the second candidate subject message to obtain a second time sequence comparison result; and
and determining the target subject message from the first candidate subject message and the second candidate subject message according to the second time sequence comparison result.
7. The method of claim 4, wherein the configuration persistence attribute information comprises a configuration persistence duration and a configuration persistence space;
wherein determining a target subject message from the subject message sequence according to a first candidate message storage time of the first candidate subject message and configuration retention attribute information of the subject message sequence comprises:
screening out outdated subject messages with the storage time longer than the configuration retention time from the subject message sequence;
screening out candidate expired subject messages with the latest storage time of the expired messages from a plurality of expired subject messages under the condition that the number of the expired subject messages is plural;
determining a second candidate topic message from the topic message sequence according to the configuration reserved space, wherein the topic message storage space formed by the initial topic message from the second candidate topic message to the tail of the arrangement in the topic message sequence is matched with a configuration reserved space threshold corresponding to the configuration reserved space;
Performing time sequence relation comparison on the first candidate message storage time of the first candidate subject message, the expiration message storage time of the candidate expiration subject message and the second candidate message storage time of the second candidate subject message to obtain a third time sequence comparison result; and
and determining the target subject message from the first candidate subject message, the candidate outdated subject message and the second candidate subject message according to the third time sequence comparison result.
8. The method of claim 1, wherein querying a boundary message log file from a sequence of message log files using a boundary message index in the boundary message index file for which a message index storage time satisfies a preset timing rule comprises:
inquiring boundary information associated with the boundary information index with the latest information index storage time in the boundary information index file; and
and determining a message log file storing the boundary message as the boundary message log file.
9. The method of any of claims 1 to 8, wherein prior to deleting the static message log file, the message processing method further comprises:
And deleting the boundary message index file and the static message index file arranged in front of the boundary message index file in the message index file sequence respectively to obtain a dynamic message index file.
10. A message processing apparatus, comprising:
the system comprises a target topic message determining module, a target topic message sending module and a target topic message receiving module, wherein the target topic message determining module is used for determining target topic messages from a topic message sequence of a message queue, and initial topic messages in the topic message sequence are arranged according to the time sequence of respective message storage moments;
the boundary message index file determining module is used for determining a boundary message index file from a message index file sequence according to a target message index file corresponding to the target subject message, wherein the boundary message index file is adjacent to the target message index file in time sequence and is positioned in front of the target message index file;
the boundary message log file inquiring module is used for inquiring the boundary message log file from the message log file sequence by utilizing the boundary message index meeting the preset time sequence rule at the message index storing moment in the boundary message index file; and
and the first deleting module is used for deleting static message log files arranged in front of the boundary message log files in the message log file sequence to obtain a target message queue containing dynamic message log files.
11. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1 to 9.
12. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to implement the method of any of claims 1 to 9.
13. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.
CN202310149798.5A 2023-02-13 2023-02-13 Message processing method, device, equipment and storage medium Pending CN116320010A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310149798.5A CN116320010A (en) 2023-02-13 2023-02-13 Message processing method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310149798.5A CN116320010A (en) 2023-02-13 2023-02-13 Message processing method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116320010A true CN116320010A (en) 2023-06-23

Family

ID=86778972

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310149798.5A Pending CN116320010A (en) 2023-02-13 2023-02-13 Message processing method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116320010A (en)

Similar Documents

Publication Publication Date Title
US11580107B2 (en) Bucket data distribution for exporting data to worker nodes
CN105740048A (en) Image management method, device and system
CN109582470B (en) Data processing method and data processing device
WO2022257604A1 (en) Method and apparatus for determining user tag
CN112783887A (en) Data processing method and device based on data warehouse
CN113760638A (en) Log service method and device based on kubernets cluster
CN116383207A (en) Data tag management method and device, electronic equipment and storage medium
CN113609168B (en) Data export method, device, terminal and readable storage medium
CN116320010A (en) Message processing method, device, equipment and storage medium
CN113986833A (en) File merging method, system, computer system and storage medium
US20180165018A1 (en) Partial storage of large files in distinct storage systems
CN113760176A (en) Data storage method and device
CN113672671A (en) Method and device for realizing data processing
CN112862554A (en) Order data processing method and device
CN112783925B (en) Paging retrieval method and device
CN115759233B (en) Model training method, graph data processing device and electronic equipment
CN116436855B (en) Data information processing method, device, electronic equipment and medium
CN114063879B (en) Method for processing operation command, electronic device and storage medium
CN110716885B (en) Data management method and device, electronic equipment and storage medium
CN109710673B (en) Work processing method, device, equipment and medium
CN110213314B (en) Method, device and server for determining storage node
CN117950850A (en) Data transmission method, device, electronic equipment and computer readable medium
CN117608879A (en) Message queue processing method and device, electronic equipment and storage medium
CN117633111A (en) ClickHouse-based request processing method, device, equipment and medium
CN115858472A (en) Data processing method, device, server and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination