CN113204392A - Message processing method, device and storage medium - Google Patents
Message processing method, device and storage medium Download PDFInfo
- Publication number
- CN113204392A CN113204392A CN202110385322.2A CN202110385322A CN113204392A CN 113204392 A CN113204392 A CN 113204392A CN 202110385322 A CN202110385322 A CN 202110385322A CN 113204392 A CN113204392 A CN 113204392A
- Authority
- CN
- China
- Prior art keywords
- message
- processed
- messages
- ith
- batch processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 claims abstract description 17
- 238000010923 batch production Methods 0.000 description 13
- 238000004364 calculation method Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000010365 information processing Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
- G06F9/4482—Procedural
- G06F9/4484—Executing subprograms
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention provides a message processing method, a message processing device and a storage medium; the method comprises the following steps: acquiring the total number of messages to be processed, the number of unprocessed residual messages and the number of processed messages; and obtaining a batch processing number based on the total number of the messages, the number of the residual messages and the number of the processed messages, and carrying out batch processing on the messages based on the batch processing number. The invention can dynamically calculate the batch processing number according to the actual situation, thereby improving the processing efficiency.
Description
Technical Field
The present invention relates to information processing technologies, and in particular, to a method and an apparatus for processing a packet, and a storage medium.
Background
With the development of network communication technology, Batch processing (Batch) technology is increasingly used in information processing. The batch processing technology is adopted in network input and output (I/O), and a plurality of messages can be processed by one-time function call in packet receiving and sending and message processing flows, so that the hit rate of a Cache (Cache) can be obviously improved, the overhead of context switching of a stack/register group during function call is shared, the call time is shorter, and the system load is lower.
In the existing batch processing technology, the batch processing number is often set as a static value, so that the requirements of network throughput and time delay under different flow models cannot be met simultaneously, the batch processing number is not matched with the actual situation, and the batch processing efficiency is low.
Disclosure of Invention
Embodiments of the present invention are intended to provide a method, an apparatus, and a storage medium for processing a packet, which can dynamically calculate a batch processing number according to an actual situation, thereby improving processing efficiency.
The technical scheme of the invention is realized as follows:
the embodiment of the invention provides a message processing method, which comprises the following steps:
acquiring the total number of messages to be processed, the number of unprocessed residual messages and the number of processed messages;
and obtaining a batch processing number based on the total number of the messages, the number of the residual messages and the number of the processed messages, and carrying out message batch processing based on the batch processing number.
In the foregoing solution, the total number of the messages includes: the total number of ith messages; the total number of the ith message is the total number of messages to be processed in the ith polling period; the ith polling cycle is a polling cycle to be processed, and i is a positive integer;
the remaining message quantity includes: the number of the ith residual message is up to the number of the ith plus (T-1) residual message; wherein T is the period number of the polling period to be processed, and T is an integer greater than or equal to 2;
the number of processed packets includes: the number of the ith processed message is up to the number of the ith plus (T-1) processed message;
the acquiring of the total number of the messages to be processed, the number of unprocessed remaining messages and the number of processed messages includes:
acquiring the total number of the ith messages in the ith polling period;
under the limitation of the j-1 th batch processing number, carrying out batch processing on the messages to be processed to obtain the number of unprocessed ith residual messages and the number of processed ith processed messages; wherein j is an integer greater than or equal to 2;
under the limitation of the jth batch processing number, continuing batch processing of an i +1 th polling cycle until an i + (T-1) th polling cycle, acquiring the i + (T-1) th processed message quantity and the unprocessed i + (T-1) th residual message quantity, thereby completing acquisition of the total message quantity of the messages to be processed, the unprocessed residual message quantity and the processed message quantity; wherein, T is the period number of the polling period to be processed, and T is an integer greater than or equal to 2.
In the foregoing solution, the obtaining a batch processing number based on the total number of the messages, the number of the remaining messages, and the number of the processed messages, and performing packet batch processing based on the batch processing number includes:
determining the (i + T) th batch processing number corresponding to the (i + T) th polling period to the (i + 2) th (T-1) th polling period according to the (i + (T-1) th residual message number, the (i) th message total number, and the (i + processed) th message number to the (i + (T-1) th processed message number, so as to be used in subsequent batch processing, thereby completing batch processing based on the message total number, the residual message number and the processed message number, and performing message batch processing based on the batch processing number.
In the foregoing scheme, obtaining the total number of the ith packet in the ith polling cycle includes:
if i is larger than 1, acquiring the i-1 th residual message quantity of the i-1 st polling cycle aiming at the i-th polling cycle;
receiving the ith group of messages, obtaining the ith number of received messages of the ith group of messages, and determining the sum of the (i-1) th residual message number and the ith number of received messages as the ith total number of messages of the messages to be processed in the ith polling period.
In the foregoing solution, the determining, according to the i + (T-1) th remaining packet quantity, the i th packet total quantity, and the i + th processed packet quantity to the i + (T-1) th processed packet quantity, the j +1 th batch processing number corresponding to the i + T th polling cycle to the i +2(T-1) th polling cycle includes:
subtracting the total number of the ith message from the number of the ith (T-1) remaining messages, adding the sum of the number of the ith processed message to the number of the ith (T-1) processed message, and dividing the obtained result by the cycle number T of the polling cycle to be processed to obtain the (j + 1) th batch processing number corresponding to the (i + T) th polling cycle to the (i + 2) th (T-1) polling cycle.
In the above scheme, the method further comprises:
if i is equal to 1, acquiring the total quantity of the 1 st message of the messages to be processed in the 1 st polling period;
under the limit of a preset initial batch processing number, carrying out batch processing on the messages to be processed to obtain the number of unprocessed 1 st residual messages and the number of processed 1 st processed messages;
under the limit of the preset initial batch processing number, continuing batch processing in the 2 nd polling period until the T-th polling period, and acquiring the T-th processed message quantity and the T-th unprocessed residual message quantity; wherein T is the period number of the polling period to be processed, and T is more than or equal to 2;
and determining a 2 nd batch processing number corresponding to a 1+ T polling period to a 2T-1 st polling period according to the Tth residual message number, the 1 st message total number and the 1 st to Tth processed message numbers for subsequent batch processing.
In the foregoing scheme, the obtaining of the total number of 1 st packet of the packets to be processed in the 1 st polling period includes:
receiving the 1 st group of messages, acquiring the 1 st number of the 1 st group of messages, and determining the 1 st number of the messages as the 1 st total number of the messages to be processed in the 1 st polling period.
In the foregoing solution, the determining a 2 nd batch processing number corresponding to a 1+ T-th polling period to a 2T-1 th polling period according to the T-th remaining packet number, the 1 st packet total number, and the 1 st to T-th processed packet numbers includes:
subtracting the total quantity of the 1 st message from the quantity of the Tth residual message, adding the sum of the quantity of the 1 st processed message to the quantity of the Tth processed message, and dividing the obtained result by the period number T of the polling period to be processed to obtain the 2 nd batch processing number corresponding to the 1+ T polling period to the 1+2(T-1) polling period.
In the foregoing scheme, the performing batch processing on the packets to be processed under the limitation of the jth batch processing number to obtain the number of unprocessed ith remaining packets and the number of processed ith processed packets includes:
if the preset number of the processed messages in the ith polling cycle is larger than the jth-1 batch processing number, reducing the preset number to the jth-1 batch processing number;
carrying out batch processing on messages meeting the preset number in the messages to be processed;
and acquiring the number of unprocessed ith residual messages and the number of processed ith messages.
An embodiment of the present invention further provides a packet processing apparatus, including:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring the total number of messages to be processed, the number of unprocessed residual messages and the number of processed messages;
a determining unit, configured to obtain a batch processing number based on the total number of the packets, the remaining number of the packets, and the processed number of the packets;
and the batch processing unit is used for carrying out message batch processing based on the batch processing number.
In the foregoing solution, the total number of the messages includes: the total number of ith messages is the total number of messages to be processed in an ith polling period, the ith polling period is the polling period to be processed, and i is a positive integer; the remaining message quantity includes: the number of the ith residual message is up to the number of the ith plus (T-1) residual message; wherein T is the period number of the polling period to be processed, and T is an integer greater than or equal to 2; the number of processed packets includes: the number of the ith processed message is up to the number of the ith plus (T-1) processed message;
the acquiring unit is further configured to acquire the total number of ith messages in an ith polling period; obtaining the unprocessed ith residual message quantity and the processed ith processed message quantity; acquiring the number of the i + (T-1) th processed messages and the number of unprocessed i + (T-1) th residual messages, wherein T is the period number of a polling period to be processed, and T is more than or equal to 2;
the batch processing unit is further configured to perform batch processing on the to-be-processed packet in the ith polling cycle under the limitation of the j-1 th batch processing number; under the limitation of the jth batch processing number, continuing to perform batch processing of the (i + 1) th polling cycle until reaching the (i + (T-1) th polling cycle;
the determining unit is further configured to determine, according to the i + (T-1) th remaining packet quantity, the i th packet total quantity, and the i + th processed packet quantity to the i + (T-1) th processed packet quantity, a j +1 th batch processing quantity corresponding to an i + T-th polling period to an i +2(T-1) th polling period, for use in subsequent batch processing.
An embodiment of the present invention further provides a packet processing apparatus, including:
a memory for storing executable instructions;
and the processor is used for realizing the message processing method in the scheme when the processor executes the executable instructions stored in the memory.
The embodiment of the invention also provides a storage medium, which stores executable instructions and is used for causing the processor to execute so as to realize the message processing method in the scheme.
Therefore, embodiments of the present invention provide a message processing method, a message processing apparatus, and a storage medium, which can obtain a batch processing number based on the total number of messages, the number of remaining messages, and the number of processed messages after obtaining the total number of messages to be processed, the number of unprocessed remaining messages, and the number of processed messages, thereby performing batch processing on messages based on the batch processing number. Therefore, the batch processing number can be determined and updated according to the actual condition of batch processing, so that the batch processing number is adaptively adjusted to be matched with the actual condition, and the processing efficiency is improved.
Drawings
Fig. 1 is a schematic view of an application scenario of a message processing method according to an embodiment of the present invention;
fig. 2 is a first flowchart of a message processing method according to an embodiment of the present invention;
fig. 3 is a second flowchart of a message processing method according to an embodiment of the present invention;
fig. 4 is a third flowchart of a message processing method according to an embodiment of the present invention;
fig. 5 is a fourth flowchart of a message processing method according to an embodiment of the present invention;
fig. 6 is a fifth flowchart of a message processing method according to an embodiment of the present invention;
fig. 7 is a sixth flowchart of a message processing method according to an embodiment of the present invention;
fig. 8 is a seventh flowchart of a message processing method according to an embodiment of the present invention;
fig. 9 is an eighth flowchart of a message processing method according to an embodiment of the present invention;
fig. 10 is a schematic effect diagram of a message processing method according to an embodiment of the present invention;
fig. 11 is a first schematic structural diagram of a message processing apparatus according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention are further described in detail with reference to the drawings and the embodiments, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.
To the extent that similar descriptions of "first/second" appear in this patent document, the description below will be added, where reference is made to the term "first \ second \ third" merely to distinguish between similar objects and not to imply a particular ordering with respect to the objects, it being understood that "first \ second \ third" may be interchanged either in a particular order or in a sequential order as permitted, to enable embodiments of the invention described herein to be practiced in other than the order illustrated or described herein.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.
Fig. 1 is a schematic view of an application scenario provided in an embodiment of the present invention, and as shown in fig. 1, a message processing apparatus may be applied in a server 300; the server 300 is in communication connection with the cloud platform 301, the client 1 and the client 2, so that the server 300 can receive messages to be processed from the cloud platform 301, the client 1 and the client 2, and further perform batch processing on the messages to be processed.
Fig. 2 is an optional flowchart of the message processing method according to the embodiment of the present invention, which will be described with reference to the steps shown in fig. 2.
S101, acquiring the total number of messages to be processed, the number of unprocessed residual messages and the number of processed messages.
In the embodiment of the present invention, a message (message) is a data unit exchanged and transmitted in a network, that is, a data block to be sent by a station at a time. The message processing apparatus may perform batch processing on the message in a Polling (Polling) manner, that is, the batch processing apparatus performs batch processing on the message at regular time, and a time period of performing batch processing each time is referred to as a Polling cycle.
In the embodiment of the present invention, the total number of messages to be processed includes: the total number of ith messages; the total quantity of the ith message is the total quantity of messages to be processed in the ith polling period; the ith polling cycle is a polling cycle to be processed, and i is a positive integer;
the remaining message quantity includes: the number of the ith residual message is up to the number of the ith plus (T-1) residual message; the number of the ith residual message to the number of the ith + (T-1) residual message are respectively the number of the residual messages after batch processing in the ith polling period to the ith + (T-1) polling period; t is the period number of the polling period to be processed, and T is an integer greater than or equal to 2;
the number of processed messages includes: the number of the ith processed message is up to the number of the ith plus (T-1) processed message; the batch processing method comprises the steps of obtaining the number of processed messages from ith to ith + (T-1), wherein the number of processed messages from ith to ith is the number of messages to be batch processed from ith to ith + (T-1) polling cycles.
The message processing device can obtain the total number of the ith message in the ith polling period; then, under the limitation of the j-1 th batch processing number, batch processing is carried out on the messages to be processed, and the number of unprocessed ith residual messages and the number of processed ith processed messages are obtained; wherein j is an integer greater than or equal to 2; and finally, under the limitation of the jth batch processing number, continuously carrying out batch processing in the (i + 1) th polling period until the (i + (T-1) th polling period is reached, and acquiring the (i + (T-1) th processed message quantity and the unprocessed (i + (T-1) th residual message quantity, thereby finishing acquiring the total message quantity of the messages to be processed, the unprocessed residual message quantity and the processed message quantity.
S102, obtaining a batch processing number based on the total number of the messages, the residual message number and the processed message number, and carrying out message batch processing based on the batch processing number.
In the embodiment of the invention, after acquiring the i + (T-1) th residual message quantity, the i < th > message total quantity, and the i < th > processed message quantity to the i + (T-1) th processed message quantity, the message processing device can determine the j +1 th batch processing quantity corresponding to the i + T-th polling period to the i +2(T-1) th polling period according to the message quantities, and the batch processing quantity is used for subsequent batch processing, so that the batch processing quantity is obtained based on the message total quantity, the residual message quantity and the processed message quantity, and the message batch processing is performed based on the batch processing quantity.
In some embodiments of the invention, the total number of messages comprises: the total number of ith messages; the total quantity of the ith message is the total quantity of messages to be processed in the ith polling period; the ith polling cycle is a polling cycle to be processed, and i is a positive integer; the remaining message quantity includes: the number of the ith residual message is up to the number of the ith plus (T-1) residual message; wherein T is the period number of the polling period to be processed, and T is an integer greater than or equal to 2; the number of processed messages includes: the number of the ith processed message is up to the number of the ith plus (T-1) processed message; s101 shown in fig. 2 may be implemented by S201 to S203 shown in fig. 3, and will be described with reference to the steps.
S201, obtaining the total quantity of the ith message in the ith polling period.
In the embodiment of the invention, the message processing device can receive a certain amount of newly arrived messages after the end of the last polling period; after the polling period begins, carrying out batch processing on part or all of all messages in the polling period, wherein all messages in the polling period comprise the remaining messages which are not processed in the previous polling period and the newly arrived messages in the polling period; and after the batch processing of the polling period is completed, the polling period is ended. For example, after the batch processing is performed in the 2 nd polling cycle, 4 unprocessed messages remain; after the 2 nd polling period is finished, 2 newly arrived messages are received; after the 3 rd polling period starts, all messages in the polling period, including the remaining 4 unprocessed messages in the 2 nd polling period and the 2 newly arrived messages received after the 2 nd polling period ends, are processed in batch for part or all of 6 messages; and finishing the 3 rd polling period after the 3 rd polling period is finished in batch.
In the embodiment of the present invention, the message processing apparatus may preset the cycle number T of the to-be-processed polling cycle, where T is an integer greater than or equal to 2, and determine the T-1 th polling cycle after one to-be-processed polling cycle as the next to-be-processed polling cycle, and recur in sequence. For example, if T is preset to be 4, and the 4 th polling cycle is determined as the first to-be-processed polling cycle, the 3 rd polling cycle after the 4 th polling cycle, that is, the 7 th polling cycle, may be determined as the next to-be-processed polling cycle; the 3 rd polling period after the 7 th polling period, namely the 10 th polling period, is the next polling period to be processed; the sequential recursion can obtain that when i is 1+3N (N >0), the ith polling cycle is the polling cycle to be processed. For another example, if T is preset to be 2, and the 1 st polling cycle is determined as the first to-be-processed polling cycle, the 1 st polling cycle after the 1 st polling cycle may be determined, that is, the 2 nd polling cycle is the next to-be-processed polling cycle; the 1 st polling cycle after the 2 nd polling cycle, namely the 3 rd polling cycle is the next polling cycle to be processed; the sequential recursion can obtain that when i ═ N (N >0), the ith polling cycle is the pending polling cycle.
In the embodiment of the present invention, the total number of ith packets of the to-be-processed packet in the ith polling cycle includes: the i-1 th residual message quantity of the i-1 st polling period and the i-th received message quantity of the i-th group of received messages. And determining the sum of the ith-1 residual message quantity and the ith received message quantity as the total quantity of the ith message.
It can be understood that the polling cycles to be processed are periodically set so that the polling cycles to be processed are uniformly distributed, the uniformity of the acquired data is ensured when the batch processing number is calculated each time, missing acquisition or repeated acquisition of the data is avoided, and the calculated batch processing number is more stable and reliable.
S202, under the limitation of the jth-1 batch processing number, batch processing is carried out on the messages to be processed, and the number of unprocessed ith residual messages and the number of processed ith processed messages are obtained; wherein j is an integer of 2 or more.
In the embodiment of the invention, the message processing device determines the j-1 batch processing number in the last polling period to be processed before the ith polling period, wherein j is an integer greater than 1. Then, under the limitation of the j-1 batch processing number, batch processing is carried out on the messages to be processed in the ith polling period, and the unprocessed ith residual message quantity and the processed ith processed message quantity are obtained.
The ith (T-1) polling cycle is the last polling cycle to be processed before the ith polling cycle, the jth-1 batch processing number is determined in the ith (T-1) polling cycle, and the jth-1 batch processing number is used for limiting the ith (T-1) +1 polling cycle until the batch processing in the ith polling cycle. That is, each pending polling cycle determines a batch number, and the determined batch number is used to limit the next polling cycle after the pending polling cycle until the batch processing performed in the next pending polling cycle after the pending polling cycle. For example, if T is 4, and the 7 th polling cycle is a pending polling cycle, the determined batch processing number in the 7 th polling cycle is used to limit the 8 th polling cycle until the 10 th polling cycle (i.e. the next pending polling cycle after the 7 th polling cycle, which can be derived by substituting it into i + (T-1)).
Wherein, the unprocessed ith residual message quantity is the residual message quantity after batch processing in the ith polling cycle; the processed ith processed message number is the message number for batch processing in the ith polling period.
In the embodiment of the present invention, the message processing apparatus may determine the predetermined number of messages processed in the ith polling cycle before batch processing of the messages in the ith polling cycle. If the predetermined number of the processed messages in the ith polling period is larger than the jth-1 batch processing number, the message processing device reduces the predetermined number to the jth-1 batch processing number. That is, the number of batch processes limits the upper limit of the number of batch process messages in one polling cycle, and when the predetermined number of process messages is greater than the number of batch processes, the predetermined number is reduced to the number of batch processes.
It is to be understood that the determined number of batches in each pending polling period is used to limit the next polling period after the pending polling period until the batch in the next pending polling period after the pending polling period. Therefore, the periodic dynamic update of the batch processing number can be realized, and the dynamic adjustment of the batch processing process is ensured.
S203, under the limitation of the jth batch processing number, continuously carrying out batch processing in the (i + 1) th polling cycle until the (i + (T-1) th polling cycle, and acquiring the (i + (T-1) th processed message quantity and the unprocessed (i + (T-1) th residual message quantity, thereby completing the acquisition of the total quantity of the messages to be processed, the unprocessed residual message quantity and the processed message quantity.
In the embodiment of the invention, the message processing device continues to perform batch processing in the (i + 1) th polling cycle under the limitation of the jth batch processing number, and simultaneously obtains the (i + 1) th processed message quantity and the (i + 1) th unprocessed residual message quantity. And when the polling period reaches the i + (T-1), carrying out batch processing of the i + (T-1) polling period, and acquiring the number of the i + (T-1) processed messages and the number of unprocessed i + (T-1) remaining messages, wherein T is the period number of the polling period to be processed, and T is an integer greater than or equal to 2. Thus, under the limitation of the jth batch processing number, batch processing of each polling period from the (i + 1) th polling period to the (i + (T-1) th polling period is performed, and the number of processed messages and the number of unprocessed residual messages in each polling period are obtained.
And the jth batch processing number is the batch processing number determined by the message processing device in the ith polling period to be processed. The message processing device determines the jth batch processing number in the ith polling period, wherein the jth batch processing number is used for limiting the (i + 1) th polling period until the batch processing is carried out in the (i + (T-1) th polling period.
The unprocessed (i + 1) th residual message quantity is the residual message quantity after batch processing in the (i + 1) th polling cycle; the processed (i + 1) th processed message quantity is the batch processing message quantity in the (i + 1) th polling cycle. The unprocessed i + (T-1) th residual message quantity is the residual message quantity after batch processing in the i + (T-1) th polling period; the processed message quantity of the ith + (T-1) th is the message quantity of batch processing in the ith + (T-1) th polling period. The ith polling cycle and the ith (T-1) polling cycle are two adjacent polling cycles to be processed. That is, the message processing apparatus acquires the data related to batch processing including two adjacent polling periods to be processed and all polling periods between the two polling periods, to prepare for determining the next batch processing number.
It can be understood that obtaining the number of processed messages and the number of unprocessed remaining messages for each of the (i + 1) th polling period to the (i + (T-1) th polling period can accumulate a sufficient data base for the batch processing number determination, so that the determined batch processing number is close to practical and more reliable.
In some embodiments of the present invention, S102 shown in fig. 2 may be implemented by S204 shown in fig. 3, which will be described in conjunction with various steps.
S204, determining the (j + 1) th batch processing number corresponding to the (i + T) th polling period to the (i + 2) th (T-1) th polling period according to the (i + (T-1) th residual message number, the (i) th message total number, and the (i + 1) th processed message number to the (i + (T-1) th processed message number, for use in subsequent batch processing, thereby completing batch processing based on the message total number, the residual message number and the processed message number to obtain the batch processing number, and performing message batch processing based on the batch processing number.
In the embodiment of the invention, the message processing device determines the j +1 th batch processing number according to the i + (T-1) th residual message number, the i (T-1) th total message number and the i (i) th processed message number to the i + (T-1) th processed message number, and the j +1 th batch processing number is used for batch processing from the i + T (th) polling period to the i +2(T-1) th polling period.
In the embodiment of the invention, the manager of the message processing device can finish determining and updating the batch processing number at any time; the cycle upper limit may be set, and when the number of times of determining and updating the number of batches reaches the cycle upper limit, the determination and updating of the number of batches may be ended. And are not intended to be limiting herein.
It can be understood that, according to the embodiments of the present invention, the j +1 th batch processing number for limiting the (i + T) th polling cycle to the (i + 2) th (T-1) th polling cycle is determined according to the data for batch processing from the ith polling cycle to the (i + (T-1) th polling cycle, that is, according to the preset number of cycles, the batch processing number of the next polling cycle is determined according to the data for batch processing from a historical polling cycle. Therefore, the batch processing number is updated dynamically according to the most similar historical data regularly, the batch processing number is adjusted in a self-adaptive mode, the batch processing number is matched with the actual situation, and the processing efficiency is improved.
In some embodiments of the present invention, S201 shown in fig. 3 may be implemented by S2011-S2012 shown in fig. 4, which will be described in conjunction with the steps.
S2011, if i is larger than 1, acquiring the i-1 th residual message quantity of the i-1 th polling cycle aiming at the i-th polling cycle.
In the embodiment of the invention, the message processing device acquires the i-1 st residual message quantity in the i-1 st polling period and is used for determining the total quantity of the messages to be processed in the i-th polling period. Wherein, the i-1 th residual message quantity of the i-1 st polling cycle is the residual message quantity after batch processing in the i-1 st polling cycle.
S2012, receiving the ith group of messages, obtaining the ith number of received messages of the ith group of messages, and determining the sum of the (i-1) th residual message number and the ith number of received messages as the ith total number of messages of the messages to be processed in the ith polling period.
In the embodiment of the invention, a message processing device receives an ith group of messages and acquires the ith message receiving quantity of the ith group of messages; and then adding the obtained i-1 th residual message quantity and the i-th received message quantity, and determining the obtained added sum as the i-th message total quantity of the messages to be processed in the i-th polling period. That is, the total number of messages in a polling cycle is the sum of the remaining number of messages in the previous polling cycle and the number of messages newly received in the current polling cycle.
In some embodiments of the present invention, S204 shown in fig. 3 may be implemented by S2041 shown in fig. 5, which will be described in conjunction with various steps.
S2041, subtracting the total quantity of the ith message from the quantity of the ith (T-1) remaining messages, adding the sum of the quantity of the ith processed message to the quantity of the ith (T-1) processed message, and dividing the obtained result by the cycle number T of the polling cycle to be processed to obtain the j +1 batch processing number corresponding to the (i + T) polling cycle to the (i + 2) polling cycle.
In this embodiment of the present invention, the message processing apparatus may implement the foregoing S2041 by using a calculation formula, where an exemplary specific calculation formula (1) is:
wherein, Bj+1Is the j +1 th batch number, li+(T-1)Is the i + (T-1) th residual message quantity, LiIs the total number of ith messages, baIs the a-th number of processed messages, and T is the number of cycles of the pending polling cycle.
It is understood that the number of batch processes for limiting the (i + T) th polling period to the (i + 2) th (T-1) th polling period is determined according to the data for batch processes from the ith polling period to the (i + (T-1) th polling period, that is, the number of batch processes for the next polling period is determined according to the data for batch processes from a historical polling period at regular intervals by a preset number of cycles. Therefore, the batch processing number is updated dynamically according to the most similar historical data regularly, the batch processing number is matched with the actual situation, and the processing efficiency is improved.
In some embodiments of the present invention, if i is equal to 1, S201 to S204 shown in fig. 3 can be implemented by S301 to S304 shown in fig. 6, which will be described with reference to the steps.
S301, if i is equal to 1, acquiring the total quantity of the 1 st message in the 1 st polling period.
In the embodiment of the present invention, a message processing apparatus obtains the total number of 1 st messages in a 1 st polling cycle, where the total number of 1 st messages is the total number of messages to be processed in the 1 st polling cycle.
S302, under the limit of the preset initial batch processing number, batch processing is carried out on the messages to be processed, and the number of unprocessed 1 st residual messages and the number of processed 1 st processed messages are obtained.
In the embodiment of the present invention, for the 1 st polling cycle to the T th polling cycle, the message processing apparatus needs to use a preset initial batch processing number to limit batch processing because there is not enough batch processing history data. The message processing device carries out batch processing on the messages to be processed in the 1 st polling period under the limit of the preset initial batch processing number, and obtains the number of unprocessed 1 st residual messages and the number of processed 1 st processed messages.
S303, under the limit of the preset initial batch processing number, continuing to perform batch processing in the 2 nd polling period until the T-th polling period, and acquiring the T-th processed message quantity and the unprocessed T-th residual message quantity; wherein, T is the period number of the polling period to be processed, and T is more than or equal to 2.
In the embodiment of the present invention, the packet processing apparatus continues batch processing in the 2 nd polling cycle under the limitation of the preset initial batch processing number, and simultaneously obtains the 2 nd processed packet number and the unprocessed 2 nd remaining packet number in the 2 nd polling cycle. And when the T polling period is reached, carrying out batch processing on the T polling period, and acquiring the T processed message quantity and the unprocessed T residual message quantity, wherein T is the period number of the polling period to be processed, and T is more than or equal to 2.
S304, determining the 2 nd batch processing number corresponding to the 1 st + T polling period to the 2 nd T-1 st polling period according to the Tth residual message number, the 1 st message total number and the 1 st to Tth processed message numbers for use in subsequent batch processing.
In the embodiment of the invention, the message processing device determines the 2 nd batch processing number according to the Tth residual message number, the 1 st total message number and the 1 st to Tth processed message numbers, and is used for batch processing from the 1+ T polling period to the 2T-1 polling period.
It will be appreciated that there is insufficient historical batch data for the 1 st to tth polling cycles, so that the number of batches cannot be determined by calculation. Therefore, the preset initial batch processing number is set to limit the 1 st polling period to the T th polling period, so that the batch processing from the 1 st polling period to the T th polling period can be effectively carried out.
In some embodiments of the present invention, S301 shown in fig. 6 may be implemented by S3011 shown in fig. 7, and will be described with reference to steps.
S3011, receiving the 1 st group of messages, obtaining the 1 st number of the received messages of the 1 st group of messages, and determining the 1 st number of the received messages as the 1 st total number of the messages to be processed in the 1 st polling period.
In the embodiment of the invention, a message processing device receives a 1 st group of messages and obtains the 1 st received message quantity of the 1 st group of messages; and then determining the number of the obtained 1 st received messages as the total number of the 1 st messages of the messages to be processed in the 1 st polling period.
It is understood that for the 1 st polling period, there are no remaining messages that were not processed in the last polling period. Therefore, the 1 st received message quantity is determined as the 1 st message total quantity of the messages to be processed in the 1 st polling period.
In some embodiments of the present invention, S304 illustrated in fig. 6 may be implemented by S3041 illustrated in fig. 8, which will be described in conjunction with various steps.
S3041, subtracting the total number of the 1 st message from the number of the T th remaining message, adding the sum of the number of the 1 st processed message and the number of the T th processed message, and dividing the obtained result by the cycle number T of the polling cycle to be processed to obtain the 2 nd batch processing number corresponding to the 1+ T polling cycle to the 1+2(T-1) th polling cycle.
In this embodiment of the present invention, the message processing apparatus may implement the foregoing S3041 by using a calculation formula, where an exemplary specific calculation formula (2) is:
wherein, B2For the 2 nd batch number, lTIs the Tth remaining message quantity, L1Is the total number of the 1 st message, baIs the a-th number of processed messages, and T is the number of cycles of the pending polling cycle.
In some embodiments of the present invention, S202 shown in fig. 3 may be implemented by S2021-S2023 shown in fig. 9, and will be described in conjunction with various steps.
S2021, if the preset number of the processed messages in the ith polling period is larger than the jth-1 batch processing number, reducing the preset number to the jth-1 batch processing number.
In the embodiment of the present invention, the message processing apparatus may determine the predetermined number of messages processed in the ith polling cycle before batch processing of the messages in the ith polling cycle. If the predetermined number of the processed messages in the ith polling period is larger than the jth-1 batch processing number, the message processing device reduces the predetermined number to the jth-1 batch processing number. That is, the number of batch processes limits the upper limit of the number of batch process messages in one polling cycle, and when the predetermined number of process messages is greater than the number of batch processes, the predetermined number is reduced to the number of batch processes.
S2022, batch processing is carried out on the messages meeting the preset number in the messages to be processed.
In the embodiment of the invention, the message processing device acquires a batch of messages meeting the preset quantity in the messages to be processed, and then carries out batch processing on the acquired batch of messages.
S2023, obtaining the number of unprocessed ith residual messages and the number of processed ith messages.
In the embodiment of the invention, after the batch processing of the ith polling cycle is completed, the message processing device acquires the unprocessed ith residual message quantity and the processed ith processed message quantity in the ith polling cycle.
The following describes a message processing method according to an application scenario, as shown in fig. 10, the method includes the following steps:
in a Network Input/Output System (NetBIOS), messages are batch processed, that is, a function is called to process a plurality of messages in a packet receiving and sending process and a message processing process. According to the 'time/space locality principle', batch processing can remarkably improve the hit rate of a processor Cache (CPU Cache) and share the overhead of context switching of a stack/register set during function calling. However, the smaller the batch processing number is set, the more the single-packet processing overhead is increased, the processing capacity of the processor is reduced, the network throughput is reduced, but the single-packet processing delay can be reduced, and the network delay is reduced; the larger the batch processing number is set, the lower the single-packet processing overhead, the higher the processing capacity of the processor, and the higher the network throughput, but the longer the single-packet processing delay and the longer the network delay are increased. Therefore, the static batch processing number cannot simultaneously meet the requirements of high throughput and low delay of the network under different traffic models.
S401 sets the cycle number T of the pending polling cycle to 4, and sets the polling cycle count T to 1.
S402, in the 1 st polling cycle, batch-processing the messages to be processed, confirming that the polling cycle count T has not reached the cycle number T of 4, and changing the polling cycle count T to 2.
In the embodiment of the present invention, as shown in fig. 10, in the 1 st polling cycle, the number of newly arrived packets in the 1 st polling cycle is 7, the number of batch packets in the 1 st polling cycle is 5, and the number of unprocessed packets in the 1 st polling cycle is 2. After the message processing apparatus completes the batch processing, if it is confirmed that the polling cycle count T has not reached the cycle number T of the pending polling cycle is 4, the message processing apparatus changes the polling cycle count T to 2.
S403, in the 2 nd polling cycle, batch-processing the messages to be processed, confirming that the polling cycle count T does not reach the cycle number T of 4, and changing the polling cycle count T to 3.
In the embodiment of the present invention, as shown in fig. 10, in the 2 nd polling period, the number of newly arrived packets in the 2 nd polling period is 2, the number of batch processed packets in the 2 nd polling period is 2, and the number of unprocessed packets in the 2 nd polling period is 2. After the message processing apparatus completes the batch processing, if it is confirmed that the polling cycle count T has not reached the cycle number T of the pending polling cycle is 4, the polling cycle count T is changed to 3.
S404, in the 3 rd polling cycle, batch-processing the messages to be processed, confirming that the polling cycle count T does not reach the cycle number T equal to 4, and changing the polling cycle count T to 4.
In the embodiment of the present invention, as shown in fig. 10, in the 3 rd polling period, the number of newly arrived packets in the 3 rd polling period is 2, the number of batch packets in the 3 rd polling period is 2, and the number of unprocessed packets in the 3 rd polling period is 2. After the message processing apparatus completes the batch processing, if it is confirmed that the polling cycle count T has not reached the cycle number T of the pending polling cycle is 4, the polling cycle count T is changed to 4.
And S405, in the 4 th polling cycle, carrying out batch processing on the message to be processed, confirming that the polling cycle count T reaches a cycle number T which is 4, calculating and updating the batch processing number, and changing the polling cycle count T into 1.
In the embodiment of the present invention, as shown in fig. 10, in the 4 th polling period, the number of newly arrived packets in the 4 th polling period is 0, the number of batch packets in the 4 th polling period is 2, and the number of unprocessed packets in the 4 th polling period is 0. After the message processing apparatus completes batch processing, it is confirmed that the polling cycle count T has reached the cycle number T of the to-be-processed polling cycle, which is 4, and then the batch processing number is calculated and updated on the basis of the batch processing data from the 1 st polling cycle to the 4 th polling cycle. After the message processing device completes the calculation, the polling cycle count t is changed to 1 for the next polling cycle to count again.
In the embodiment of the invention, the message processing device determines the residual message quantity l of the 4 th polling period on the basis of the batch processing data from the 1 st polling period to the 4 th polling period4The total number L of the 1 st packet of the 1 st polling cycle is 01The number of processed messages from the 1 st polling cycle to the 4 th polling cycle is b1＝5，b2＝2，b3＝2，b42, and mixing the above numbersSubstituting the values into formula (1) to obtain a calculation formula (3), and calculating to obtain a 2 nd batch processing number B corresponding to the 5 th to 7 th polling periods2：
Fig. 11 is an alternative structural schematic diagram of a message processing apparatus according to an embodiment of the present invention. As shown in fig. 11, an embodiment of the present invention further provides a message processing apparatus 800, including: an obtaining unit 804, a determining unit 805, and a batch processing unit 806, wherein:
an obtaining unit 804, configured to obtain a total number of messages to be processed, an unprocessed remaining number of messages, and a processed number of messages;
a determining unit 805 configured to obtain a batch processing number based on the total number of the messages, the number of remaining messages, and the number of processed messages;
and a batch processing unit 806, configured to perform batch processing on the message based on the batch processing number.
In some embodiments of the invention, the total number of messages comprises: the total number of ith messages; the total number of the ith message is the total number of messages to be processed in the ith polling period, the ith polling period is the polling period to be processed, and i is a positive integer; the remaining message quantity includes: the number of the ith residual message is up to the number of the ith plus (T-1) residual message; wherein T is the period number of the polling period to be processed, and T is an integer greater than or equal to 2; the number of processed messages includes: the number of the ith processed message is up to the number of the ith plus (T-1) processed message;
the obtaining unit 804 is further configured to obtain the total number of the ith packet in the ith polling period; acquiring the number of unprocessed ith residual messages and the number of processed ith messages; acquiring the number of the i + (T-1) th processed messages and the number of unprocessed i + (T-1) th residual messages;
the batch processing unit 806 is further configured to perform batch processing on the to-be-processed packet in the ith polling cycle under the limitation of the j-1 th batch processing number; under the limitation of the jth batch processing number, continuing to perform batch processing of the (i + 1) th polling cycle until reaching the (i + (T-1) th polling cycle;
the determining unit 805 is further configured to determine, according to the i + (T-1) th remaining packet quantity, the i th total packet quantity, and the i th to i + (T-1) th processed packet quantities, a j +1 th batch processing quantity corresponding to an i + T th to i +2(T-1) th polling cycles, for use in subsequent batch processing.
In some embodiments of the present invention, the obtaining unit 804 is further configured to obtain, for the ith polling cycle, an i-1 th remaining packet quantity of the i-1 st polling cycle if i is greater than 1; and receiving the ith group of messages, acquiring the ith received message quantity of the ith group of messages, and determining the sum of the (i-1) th residual message quantity and the ith received message quantity as the ith message total quantity of the messages to be processed in the ith polling period.
In some embodiments of the present invention, the determining unit 805 is further configured to subtract the total number of the ith packet from the number of the ith (T-1) remaining packets, add the sum of the number of the ith processed packet to the number of the ith (T-1) processed packet, and divide the obtained result by the cycle number T of the polling cycle to be processed to obtain the j +1 th batch processing number corresponding to the (i + T) th polling cycle to the (i + 2) th (T-1) polling cycle.
In some embodiments of the present invention, the obtaining unit 804 is further configured to obtain a total number of 1 st packets of the to-be-processed packets in the 1 st polling period if i is equal to 1; acquiring the number of unprocessed 1 st residual messages and the number of processed 1 st messages; acquiring the quantity of the T processed messages and the quantity of the T unprocessed residual messages, wherein T is the period number of a polling period to be processed, and T is more than or equal to 2;
the batching unit 806 is further configured to batch process the to-be-processed packet in the 1 st polling cycle under the limit of a preset initial batching number; under the limit of the preset initial batch processing number, continuing to perform batch processing in the 2 nd polling period until the T-th polling period;
the determining unit 805 is further configured to determine, according to the tth remaining packet quantity, the total quantity of the 1 st packet, and the number of the 1 st processed packet to the tth processed packet quantity, a 2 nd batch processing number corresponding to the 1+ T polling period to the 2T-1 st polling period, for use in subsequent batch processing.
In some embodiments of the present invention, the obtaining unit 804 is further configured to receive the 1 st group of packets, obtain the 1 st number of received packets of the 1 st group of packets, and determine the 1 st number of received packets as the 1 st total number of packets of the to-be-processed packet in the 1 st polling cycle.
In some embodiments of the invention, the determining unit 805 is further configured to subtract the total number of the 1 st remaining packet from the total number of the 1 st packet, add the sum of the number of the 1 st processed packet to the number of the tth processed packet, and divide the obtained result by the cycle number T of the polling cycle to be processed, so as to obtain the 2 nd batch processing number corresponding to the 1+ T polling cycle to the 1+2(T-1) th polling cycle.
In some embodiments of the present invention, the batch processing unit 806 is further configured to reduce the predetermined number to the j-1 th batch processing number if the predetermined number of the processed messages in the ith polling period is greater than the j-1 th batch processing number; and carrying out batch processing on messages meeting the preset number in the messages to be processed.
It should be noted that fig. 12 is an optional schematic structural diagram of the message processing apparatus according to the embodiment of the present invention, and as shown in fig. 12, a hardware entity of the message processing apparatus 800 includes: a processor 801, a communication interface 802, and a memory 803, wherein:
the processor 801 generally controls the overall operation of the message processing apparatus 800.
The communication interface 802 may enable the message processing apparatus 800 to communicate with other apparatuses or devices via a network.
The Memory 803 is configured to store instructions and applications executable by the processor 801, and may also buffer data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by each module in the message processing apparatus 800 and the processor 801, and may be implemented by a flash Memory (FLAS H) or a Random Access Memory (RAM).
It should be noted that, in the embodiment of the present invention, if the method for detecting malicious behavior is implemented in the form of a software functional module, and is sold or used as a standalone product, it may also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be substantially implemented or a part of the technical solutions contributing to the related art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling the message processing apparatus 800 (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
Correspondingly, the embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the above-mentioned method.
Claims (13)
1. A message processing method is characterized by comprising the following steps:
acquiring the total number of messages to be processed, the number of unprocessed residual messages and the number of processed messages;
and obtaining a batch processing number based on the total number of the messages, the number of the residual messages and the number of the processed messages, and carrying out message batch processing based on the batch processing number.
2. The message processing method according to claim 1, wherein the total number of messages comprises: the total number of ith messages; the total number of the ith message is the total number of messages to be processed in the ith polling period; the ith polling cycle is a polling cycle to be processed, and i is a positive integer;
the remaining message quantity includes: the number of the ith residual message is up to the number of the ith plus (T-1) residual message; wherein T is the period number of the polling period to be processed, and T is an integer greater than or equal to 2;
the number of processed packets includes: the number of the ith processed message is up to the number of the ith plus (T-1) processed message;
the acquiring of the total number of the messages to be processed, the number of unprocessed remaining messages and the number of processed messages includes:
acquiring the total number of the ith messages in the ith polling period;
under the limitation of the j-1 th batch processing number, carrying out batch processing on the messages to be processed to obtain the number of unprocessed ith residual messages and the number of processed ith processed messages; wherein j is an integer greater than or equal to 2;
under the limitation of the jth batch processing number, continuing batch processing of the (i + 1) th polling cycle until the (i + (T-1) th polling cycle, acquiring the (i + (T-1) th processed message quantity and the unprocessed (i + (T-1) th residual message quantity, thereby completing acquisition of the total message quantity, the unprocessed residual message quantity and the processed message quantity of the to-be-processed message.
3. The message processing method according to claim 2, wherein the obtaining a batch processing number based on the total number of messages, the remaining number of messages, and the processed number of messages, and performing message batch processing based on the batch processing number includes:
determining the (i + T) th batch processing number corresponding to the (i + T) th polling period to the (i + 2) th (T-1) th polling period according to the (i + (T-1) th residual message number, the (i) th message total number, and the (i + processed) th message number to the (i + (T-1) th processed message number, so as to be used in subsequent batch processing, thereby completing batch processing based on the message total number, the residual message number and the processed message number, and performing message batch processing based on the batch processing number.
4. The message processing method according to claim 2 or 3, wherein the obtaining of the i-th total number of messages in the i-th polling period includes:
if i is larger than 1, acquiring the i-1 th residual message quantity of the i-1 st polling cycle aiming at the i-th polling cycle;
receiving the ith group of messages, obtaining the ith number of received messages of the ith group of messages, and determining the sum of the (i-1) th residual message number and the ith number of received messages as the total number of the ith messages in the ith polling period.
5. The message processing method according to claim 3, wherein the determining the j +1 th batch processing number corresponding to the i + T-th polling period to the i +2(T-1) th polling period according to the i + (T-1) th remaining message number, the i-th message total number, and the i-th processed message number to the i + (T-1) th processed message number comprises:
subtracting the total number of the ith message from the number of the ith (T-1) remaining messages, adding the sum of the number of the ith processed message to the number of the ith (T-1) processed message, and dividing the obtained result by the cycle number T of the polling cycle to be processed to obtain the (j + 1) th batch processing number corresponding to the (i + T) th polling cycle to the (i + 2) th (T-1) polling cycle.
6. The message processing method according to any of claims 2 to 5, characterized in that the method further comprises:
if i is equal to 1, acquiring the total quantity of the 1 st message in the 1 st polling period;
under the limit of a preset initial batch processing number, carrying out batch processing on the messages to be processed to obtain the number of unprocessed 1 st residual messages and the number of processed 1 st processed messages;
under the limit of the preset initial batch processing number, continuing batch processing in the 2 nd polling period until the T-th polling period, and acquiring the T-th processed message quantity and the T-th unprocessed residual message quantity; wherein T is the period number of the polling period to be processed, and T is more than or equal to 2;
and determining a 2 nd batch processing number corresponding to a 1+ T polling period to a 2T-1 st polling period according to the Tth residual message number, the 1 st message total number and the 1 st to Tth processed message numbers for subsequent batch processing.
7. The message processing method according to claim 6, wherein the obtaining of the total number of 1 st messages of the messages to be processed in the 1 st polling cycle comprises:
receiving a 1 st group of messages, acquiring the 1 st number of the 1 st group of messages, and determining the 1 st number of the messages as the 1 st total number of the messages to be processed in the 1 st polling period.
8. The message processing method according to claim 6 or 7, wherein the determining the 2 nd batch processing number corresponding to the 1+ T-th polling period to the 2T-1 st polling period according to the tth remaining message number, the 1 st message total number, and the 1 st to tth processed message numbers comprises:
subtracting the total quantity of the 1 st message from the quantity of the Tth residual message, adding the sum of the quantity of the 1 st processed message to the quantity of the Tth processed message, and dividing the obtained result by the period number T of the polling period to be processed to obtain the 2 nd batch processing number corresponding to the 1+ T polling period to the 1+2(T-1) polling period.
9. The message processing method according to any one of claims 2 to 8, wherein the performing batch processing on the messages to be processed under the limitation of the jth batch processing number to obtain an unprocessed ith remaining message quantity and a processed ith processed message quantity comprises:
if the preset number of the processed messages in the ith polling cycle is larger than the jth-1 batch processing number, reducing the preset number to the jth-1 batch processing number;
carrying out batch processing on messages meeting the preset number in the messages to be processed;
and acquiring the unprocessed ith residual message quantity and the processed ith processed message quantity.
10. A message processing apparatus, comprising:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring the total number of messages to be processed, the number of unprocessed residual messages and the number of processed messages;
a determining unit, configured to obtain a batch processing number based on the total number of the packets, the remaining number of the packets, and the processed number of the packets;
and the batch processing unit is used for carrying out message batch processing based on the batch processing number.
11. The message processing apparatus of claim 10, wherein the total number of messages comprises: the total number of ith messages; the total number of the ith message is the total number of messages to be processed in an ith polling period, the ith polling period is the polling period to be processed, and i is a positive integer; the remaining message quantity includes: the number of the ith residual message is up to the number of the ith plus (T-1) residual message; wherein T is the period number of the polling period to be processed, and T is an integer greater than or equal to 2; the number of processed packets includes: the number of the ith processed message is up to the number of the ith plus (T-1) processed message;
the acquiring unit is further configured to acquire the total number of ith messages in an ith polling period; obtaining the unprocessed ith residual message quantity and the processed ith processed message quantity; acquiring the number of the i + (T-1) th processed messages and the number of the i + (T-1) th residual messages which are not processed;
the batch processing unit is further configured to perform batch processing on the to-be-processed packet in the ith polling cycle under the limitation of the j-1 th batch processing number; under the limitation of the jth batch processing number, continuing to perform batch processing of the (i + 1) th polling cycle until reaching the (i + (T-1) th polling cycle;
the determining unit is further configured to determine, according to the i + (T-1) th remaining packet quantity, the i th packet total quantity, and the i + th processed packet quantity to the i + (T-1) th processed packet quantity, a j +1 th batch processing quantity corresponding to an i + T-th polling period to an i +2(T-1) th polling period, for use in subsequent batch processing.
12. A message processing apparatus, comprising:
a memory for storing executable instructions;
a processor for implementing the method of any one of claims 1 to 9 when executing executable instructions stored in the memory.
13. A storage medium having stored thereon executable instructions for causing a processor to perform the method of any one of claims 1 to 9 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110385322.2A CN113204392B (en) | 2021-04-09 | 2021-04-09 | Message processing method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110385322.2A CN113204392B (en) | 2021-04-09 | 2021-04-09 | Message processing method, device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113204392A true CN113204392A (en) | 2021-08-03 |
| CN113204392B CN113204392B (en) | 2024-04-09 |
Family
ID=77026488
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110385322.2A Active CN113204392B (en) | 2021-04-09 | 2021-04-09 | Message processing method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113204392B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9632850B1 (en) * | 2016-05-05 | 2017-04-25 | International Business Machines Corporation | Polling parameter adjustment |
| CN108259377A (en) * | 2018-02-13 | 2018-07-06 | 中国联合网络通信集团有限公司 | Queue assignment method and device |
| CN109299049A (en) * | 2018-10-11 | 2019-02-01 | 郑州云海信息技术有限公司 | A kind of processing method and processing device of file access request |
| CN111461583A (en) * | 2019-01-18 | 2020-07-28 | 北京京东尚科信息技术有限公司 | Inventory counting method and device |
| CN112612607A (en) * | 2020-12-16 | 2021-04-06 | 平安消费金融有限公司 | Data processing method, device, computer equipment and medium based on message queue |
-
2021
- 2021-04-09 CN CN202110385322.2A patent/CN113204392B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9632850B1 (en) * | 2016-05-05 | 2017-04-25 | International Business Machines Corporation | Polling parameter adjustment |
| CN108259377A (en) * | 2018-02-13 | 2018-07-06 | 中国联合网络通信集团有限公司 | Queue assignment method and device |
| CN109299049A (en) * | 2018-10-11 | 2019-02-01 | 郑州云海信息技术有限公司 | A kind of processing method and processing device of file access request |
| CN111461583A (en) * | 2019-01-18 | 2020-07-28 | 北京京东尚科信息技术有限公司 | Inventory counting method and device |
| CN112612607A (en) * | 2020-12-16 | 2021-04-06 | 平安消费金融有限公司 | Data processing method, device, computer equipment and medium based on message queue |
Non-Patent Citations (1)
| Title |
|---|
| 杜永斐: "内容分发系统中关键策略研究", 《中国优秀硕士论文 信息科技》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113204392B (en) | 2024-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022105805A1 (en) | Data processing method and in-memory computing chip | |
| KR102198020B1 (en) | Video frame coding method, terminal and storage medium | |
| CN111049756B (en) | Request response method and device, electronic equipment and computer readable storage medium | |
| US20200146074A1 (en) | Method for random access and terminal device | |
| CN108259685B (en) | Call processing method and device | |
| CN111245732B (en) | Flow control method, device and equipment | |
| US8201070B2 (en) | System and method for pre-calculating checksums | |
| CN107948084B (en) | Current limiting method and device | |
| CN108023938B (en) | Message sending method and server | |
| CN108600794B (en) | Frame loss rate correction method, device and terminal | |
| US7689798B2 (en) | Method and device for determining size of memory frames | |
| CN107920028B (en) | Data packet sending method, device and equipment | |
| CN113204392A (en) | Message processing method, device and storage medium | |
| US10467161B2 (en) | Dynamically-tuned interrupt moderation | |
| CN114780228B (en) | Hybrid cloud resource creation method and system | |
| CN112491573A (en) | Network parameter configuration method and device | |
| CN111181875A (en) | Bandwidth adjusting method and device | |
| CN111858019B (en) | Task scheduling method and device and computer readable storage medium | |
| CN110535785B (en) | Control method and device for sending frequency and distributed system | |
| CN112866496A (en) | Outbound line routing method and device based on concurrency control and electronic equipment | |
| CN113781214A (en) | Packaging method, device and equipment for promoting transaction TPS (transaction transfer protocol) based on PoW (PoW) | |
| CN107707383B (en) | Put-through processing method and device, first network element and second network element | |
| CN113453145A (en) | Method for predicting air interface state and wireless coverage equipment | |
| CN107800648B (en) | Data packet scheduling method and device | |
| Antonova | The study of wireless network resources while transmitting heterogeneous traffic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |