CN112988417A

CN112988417A - Message processing method and device, electronic equipment and computer readable medium

Info

Publication number: CN112988417A
Application number: CN202110240311.5A
Authority: CN
Inventors: 姜春志
Original assignee: Changsha Youheng Network Technology Co Ltd
Current assignee: Changsha Youheng Network Technology Co Ltd
Priority date: 2021-03-04
Filing date: 2021-03-04
Publication date: 2021-06-18

Abstract

The invention belongs to the technical field of computer information processing, and provides a message processing method, a message processing device, electronic equipment and a computer readable medium. The method can be applied to a server side in a distributed service cluster, and comprises the following steps: the server side acquires a plurality of messages from a plurality of client sides; generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics; when the total message amount is within a preset threshold value interval, extracting a target message topic based on the message topic ranking; and discarding a plurality of messages corresponding to the target message subject. According to the message processing method, the message processing device, the electronic equipment and the computer readable medium, under the normal network state and the extreme network state, the subject range of the appropriate speed limiting message can be determined, so that the speed of the excessive message is limited, the network risk condition is avoided, and the cost of manpower and material resources is reduced.

Description

Message processing method and device, electronic equipment and computer readable medium

Technical Field

The present disclosure relates to the field of computer information processing, and in particular, to a method and an apparatus for processing a message, an electronic device, and a computer-readable medium.

Background

Large companies often use distributed clustering to handle daily traffic, and in distributed cluster systems, message middleware is often used as a message distribution decoupling component. With the development of large company services, the number of service lines accessing a distributed cluster is gradually increased, and the message volume of the system reaches hundreds of billions. Meanwhile, the machine resources allocated to the distributed system are limited, and for a certain company, there are 33 physical machines at present, and 18 clusters are deployed altogether, and the total number of service nodes is 106. In order to improve the resource utilization rate, different nodes need to share a physical machine, topics of different service lines need to be deployed in a mixed manner, and when the quantity of topic messages of some service lines is abnormal (such as pressure measurement), server resources are exhausted, so that a server is down, and other topic messages processed by a cluster where the server is located are affected.

In the prior art, the subject message of the abnormal message volume of the message middleware can be limited in speed to avoid the above situation. The speed limit of the client needs the service line to be matched and upgraded, and the upgrading period is long, so that the speed limit is carried out at the server of the distributed cluster.

The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosure and therefore it may contain information that does not constitute prior art that is already known to a person of ordinary skill in the art.

Disclosure of Invention

In view of this, the present disclosure provides a message processing method, a device, an electronic device, and a computer readable medium, which can determine a subject range of a proper speed-limiting message in both a normal network state and an extreme network state, so as to limit the speed of an excessive message, avoid the occurrence of a network risk condition, and reduce the costs of manpower and material resources.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

According to an aspect of the present disclosure, a message processing method is provided, which is applicable to a server in a distributed service cluster, and the method includes: the server side acquires a plurality of messages from a plurality of client sides; generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics; when the total message amount is within a preset threshold value interval, extracting a target message topic based on the message topic ranking; and discarding a plurality of messages corresponding to the target message subject.

In an exemplary embodiment of the present disclosure, before the server obtains the plurality of messages from the plurality of clients, the method further includes: receiving a speed limit subject set sent by the registration center, wherein the speed limit subject set comprises message subjects; receiving a plurality of messages from a plurality of clients; determining message topics corresponding to the plurality of messages; and when the message theme is consistent with the message theme in the speed limit theme set, discarding the message corresponding to the message theme.

In an exemplary embodiment of the present disclosure, before receiving the set of speed limiting topics sent by the registry, the method further includes: the server side obtains a plurality of message themes to be processed currently and the number of messages corresponding to the message themes; and sending the plurality of message themes and the corresponding message number to a registration center in the distributed service cluster at regular time.

In an exemplary embodiment of the present disclosure, before the server obtains the plurality of messages from the plurality of clients, the method further includes: obtaining an available memory of a heap memory corresponding to a message queue of the server; receiving the plurality of messages when the available space threshold is less than a first space threshold.

In an exemplary embodiment of the present disclosure, further comprising: when the available space is larger than a first space threshold and smaller than a second space threshold, generating a notification message to enable the client to send a subsequent message to other servers; discarding the plurality of messages when the available space is greater than a second space threshold.

In an exemplary embodiment of the present disclosure, acquiring an available memory of a heap memory corresponding to a message queue of the server includes: acquiring an actual maximum memory based on a memory management function; acquiring a memory occupied by a tracelog queue and a memory occupied by basic information; and determining the available memory based on the actual maximum memory, the memory occupied by the tracelog queue and the memory occupied by the basic information.

In an exemplary embodiment of the disclosure, receiving the plurality of messages when the available space threshold is less than a first space threshold comprises: receiving the plurality of messages when the available space threshold is less than a first space threshold and the number of times of disk-swiping operations is less than a disk-swiping threshold.

In an exemplary embodiment of the present disclosure, further comprising: and determining a preset threshold interval according to the maximum bearing capacity of the server.

In an exemplary embodiment of the present disclosure, when the total amount of messages is within a preset threshold interval, extracting a target message topic based on the message topic ranking includes: when the total message amount is within a first threshold interval, taking the first position in the message topic ranking as the target message topic; when the total message amount is within a second threshold interval, taking the first five bits in the message topic ranking as the target message topic; and when the total message amount is within a third threshold interval, all of the message topic ranks are taken as the target message topics.

In an exemplary embodiment of the present disclosure, discarding a plurality of messages corresponding to the target message topic includes: and after a preset time interval, discarding the plurality of messages corresponding to the target message theme.

According to an aspect of the present disclosure, a message processing apparatus is provided, which is applicable to a server in a distributed service cluster, and the apparatus includes: the acquisition module is used for the server side to acquire a plurality of messages from a plurality of clients; the calculation module is used for generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics; the target module is used for extracting a target message topic based on the message topic ranking when the total message amount is within a preset threshold interval; and the discarding module is used for discarding a plurality of messages corresponding to the target message subject.

In an exemplary embodiment of the present disclosure, further comprising: the collection module is used for receiving a speed limit subject collection sent by the registration center, wherein the speed limit subject collection comprises message subjects; a pre-receiving module for receiving a plurality of messages from a plurality of clients; the theme module is used for determining message themes corresponding to the plurality of messages; and the discarding module is further used for discarding the message corresponding to the message theme when the message theme is consistent with the message theme in the speed limit theme set.

In an exemplary embodiment of the present disclosure, further comprising: the memory module is used for acquiring the available memory of the heap memory corresponding to the message queue of the server; a receiving module for receiving the plurality of messages when the available space threshold is less than a first space threshold.

According to an aspect of the present disclosure, an electronic device is provided, the electronic device including: one or more processors; storage means for storing one or more programs; when executed by one or more processors, cause the one or more processors to implement a method as above.

According to an aspect of the disclosure, a computer-readable medium is proposed, on which a computer program is stored, which program, when being executed by a processor, carries out the method as above.

According to the message processing method, the message processing device, the electronic equipment and the computer readable medium, the server side obtains a plurality of messages from a plurality of clients; generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics; when the total message amount is within a preset threshold value interval, extracting a target message topic based on the message topic ranking; the method for discarding the plurality of messages corresponding to the target message theme can determine the theme range of the appropriate speed-limiting message in both the normal network state and the extreme network state, thereby limiting the speed of the excessive messages, avoiding the occurrence of network risk conditions and reducing the cost of manpower and material resources.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings. The drawings described below are merely some embodiments of the present disclosure, and other drawings may be derived from those drawings by those of ordinary skill in the art without inventive effort.

Fig. 1 is a system block diagram illustrating a message processing method and apparatus according to an example embodiment.

Fig. 2 is a flow chart illustrating a method of message processing according to an example embodiment.

Fig. 3 is a flow chart illustrating a message processing method according to another exemplary embodiment.

Fig. 4 is a flow chart illustrating a message processing method according to another exemplary embodiment.

Fig. 5 is a flowchart illustrating a message processing method according to another example embodiment.

Fig. 6 is a block diagram illustrating a message processing apparatus according to an example embodiment.

Fig. 7 is a block diagram illustrating a message processing apparatus according to another exemplary embodiment.

FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.

FIG. 9 is a block diagram illustrating a computer-readable medium in accordance with an example embodiment.

Detailed Description

In describing particular embodiments, specific details of structures, properties, effects, or other features are set forth in order to provide a thorough understanding of the embodiments by one skilled in the art. However, it is not excluded that a person skilled in the art may implement the invention in a specific case without the above-described structures, performances, effects or other features.

The flow chart in the drawings is only an exemplary flow demonstration, and does not represent that all the contents, operations and steps in the flow chart are necessarily included in the scheme of the invention, nor does it represent that the execution is necessarily performed in the order shown in the drawings. For example, some operations/steps in the flowcharts may be divided, some operations/steps may be combined or partially combined, and the like, and the execution order shown in the flowcharts may be changed according to actual situations without departing from the gist of the present invention.

The block diagrams in the figures generally represent functional entities and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and/or processing unit devices and/or microcontroller devices.

The same reference numerals denote the same or similar elements, components, or parts throughout the drawings, and thus, a repetitive description thereof may be omitted hereinafter. It will be further understood that, although the terms first, second, third, etc. may be used herein to describe various elements, components, or sections, these elements, components, or sections should not be limited by these terms. That is, these phrases are used only to distinguish one from another. For example, a first device may also be referred to as a second device without departing from the spirit of the present invention. Furthermore, the term "and/or", "and/or" is intended to include all combinations of any one or more of the listed items.

Fig. 1 is a system block diagram illustrating a message processing method, apparatus, electronic device and computer readable medium according to an example embodiment.

As shown in fig. 1, the system architecture 10 may include

clients

101, 102, 103,

servers

104, 105, a registry 106, and may further include a network to provide a medium for communication links between the

clients

101, 102, 103 and the

servers

104, 105, registry 106. The network may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

A user may interact with a

server

104, 105 over a

network using clients

101, 102, 103 to receive or send messages, etc. The

clients

101, 102, 103 may have various communication client applications installed thereon, such as shopping applications, web browser applications, search applications, instant messaging tools, mailbox clients, social platform software, and the like.

Clients

101, 102, 103 may be a variety of electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablets, laptop and desktop computers, and the like.

The

servers

104 and 105 may be servers in a distributed cluster, and the

servers

104 and 105 may perform distributed processing on messages from the

clients

101, 102, and 103 and generate results to return to the clients.

The registry 106 and the

servers

104 and 105 may form a distributed cluster together, the registry 106 may be configured to manage and control messages processed by the

servers

104 and 105, and the registry 106 may determine whether the

servers

104 and 105 should limit the speed according to the current message traffic, and may also determine which subject messages the

servers

104 and 105 should limit the speed.

The

server

104, 105 may, for example, retrieve a plurality of messages from the

client

101, 102, 103; the

server

104, 105 may generate a total amount of messages and a ranking of message topics in real time, for example, based on the message topics of the plurality of messages and their corresponding message quantities; the

server

104, 105 may extract a target message topic based on the message topic ranking, for example, when the total amount of messages is within a preset threshold interval; the

servers

104 and 105 may discard a plurality of messages corresponding to the target message topic, for example.

The

server

104, 105 can also receive a speed limit subject set sent by the registration center 106, for example, where the speed limit subject set includes message subjects; the

server

104, 105 may also, for example, receive a plurality of messages from the

client

101, 102, 103; the

server

104, 105 may also determine message topics corresponding to the plurality of messages, for example; the

server

104, 105 may also discard the message corresponding to the message topic, for example, when the message topic is consistent with the message topic in the speed limit topic set.

The

server

104, 105 may also, for example, obtain an available memory of a heap memory corresponding to a message queue of the server; the

server

104, 105 may also receive the plurality of messages, for example, when the available space threshold is less than a first space threshold.

The

server

104, 105 may be a server of one entity, and may also be composed of a plurality of servers, for example, it should be noted that the message processing method provided by the embodiment of the present disclosure may be executed by the

server

104, 105 and the registration center 106, and accordingly, the message processing apparatus may be disposed in the

server

104, 105 and the registration center 106.

Fig. 2 is a flow chart illustrating a method of message processing according to an example embodiment. The message processing method 20 is applicable to a server in a distributed service cluster, and at least includes steps S202 to S208.

As shown in fig. 2, in S202, the server acquires a plurality of messages from a plurality of clients.

In S204, a total number of messages and a ranking of message topics are generated in real time based on the message topics of the plurality of messages and the number of messages corresponding thereto. And counting the total data of all messages to be processed of the server at the current time and the number of the messages in each message topic.

In one embodiment, for optimal efficiency, the server may count the topic of the message amount top 5 every 10 ms.

In S206, when the total amount of messages is within a preset threshold interval, a target message topic is extracted based on the message topic ranking. The preset threshold interval can be determined according to the maximum bearing capacity of the server. The pressure test can be carried out before the server is on line, and the maximum bearing capacity of the server is measured.

When the total message amount is within a first threshold interval, taking the first position in the message topic ranking as the target message topic; the topic with the largest message amount can be discarded when the message amount reaches 80% of the maximum bearing capacity.

When the total message amount is within a second threshold interval, taking the first five bits of the message topic ranking as the target message topic, and discarding the message amount top 5 topic when the message amount reaches 90% of the maximum bearing capacity.

And when the total message amount is within a third threshold interval, all of the message topic ranks are taken as the target message topics. The full discard can be done when the message volume reaches 95% of the maximum bearer capacity.

In S208, a plurality of messages corresponding to the target message topic are discarded. For example, after a preset time interval, the plurality of messages corresponding to the target message topic are discarded.

In an actual scenario, when a certain topic flow is abnormal, 80% of the single machine carrying capacity may be rapidly occupied, at this time, only 15% of the distance total is discarded, and if the topic sending amount of the normal flow exceeds 15%, the total discarding may be caused, resulting in a false discarding. The reason for this problem is that the node only limits the speed according to the current second summary result, and the topic with abnormal flow has a lot of messages participating in statistics every second, and then is limited by the speed, resulting in the shortage of the normal topic quota. The solution is to limit the speed according to the statistical result of the last second.

More specifically, the proportional speed limit has the risk of mistakenly throwing the normal theme, and if the speed limit can be realized according to the statistical result in the previous second, the theme with abnormal flow can not be received in the next second, so that the full receiving of the normal theme is protected. More specifically, according to the currently acquired target message theme, the main message in the next 2 seconds is subjected to speed limitation.

According to the message processing method disclosed by the invention, a server side acquires a plurality of messages from a plurality of clients; generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics; when the total message amount is within a preset threshold value interval, extracting a target message topic based on the message topic ranking; the method for discarding the plurality of messages corresponding to the target message theme can determine the theme range of the appropriate speed-limiting message in both the normal network state and the extreme network state, thereby limiting the speed of the excessive messages, avoiding the occurrence of network risk conditions and reducing the cost of manpower and material resources.

The inventor of the present disclosure finds that, when the single machine bearing limit is reached, the topic with the maximum speed is generally an overspeed topic, so in the message processing method of the present disclosure, a mode of counting the topic with the maximum speed as soon as possible is selected, so that the topic is discarded when a certain proportion of the single machine bearing limit is reached, and the service load can be effectively reduced. Other topics may continue to grow, and in order to reduce the influence of topic discarding on a service line, the scope of the topic discarding should be reduced as much as possible, and at the same time, the service load can be reduced to the maximum extent, and the message amount top n is selected for discarding. When the single machine bearing limit is reached to a certain proportion, if the single machine bearing limit can not be increased, the whole quantity is discarded, and the service downtime is avoided.

It should be clearly understood that this disclosure describes how to make and use particular examples, but the principles of this disclosure are not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.

Fig. 3 is a flow chart illustrating a message processing method according to another exemplary embodiment. The message processing method 30 includes at least steps S302 to S308.

As shown in fig. 3, in S302, a speed limit subject set sent by the registry is received, where the speed limit subject set includes a message subject.

Before receiving the speed limit subject set sent by the registry, the method further comprises the following steps: the server side obtains a plurality of message themes to be processed currently and the number of messages corresponding to the message themes; and sending the plurality of message themes and the corresponding message number to a registration center in the distributed service cluster at regular time. The service end can report the theme QPS to the registration center every 100ms, and each theme reports the message format: topic _ second timestamp _ qps, for example, topic 201 reports 100 messages at the time represented by millisecond 1508317365363: 201_1508317365000_ 100.

The registration center counts a summary theme QPS in a stand-alone memory, if the summary theme QPS exceeds the sending quantity applied by the client, the speed limit identifier is pushed to the server, and the message format can be as follows: subject _ second level timestamp, if subject 201 needs to be rate limited at the time represented by millisecond 1508317365363, then the message format may be: 201_ 1508317365000.

The server can store the speed limit identification by using the theme speed limit set and clean the speed limit identification regularly.

In S304, a plurality of messages from a plurality of clients are received.

In S306, message topics corresponding to the plurality of messages are determined.

In S308, when the message topic is consistent with the message topic in the speed limit topic set, the message corresponding to the message topic is discarded. And the server receives the message sent by the client, analyzes the theme, inquires whether the theme is in the speed limit theme set, and discards the message if the theme is in the speed limit theme set.

The server side can be used as a speed limit period every second, the server side reports the subject speed to the registration center every 100ms, the registration center collects the speed, and if the speed is overspeed, the speed limit identification, the subject and the second where the speed limit is located are pushed to the server side so that the server side can discard the subsequent received messages of the specified subject in the second.

In the present disclosure, in addition to limiting the speed according to the sending speed, the speed can also be limited according to the current heap memory usage. After receiving the message, the server puts the entity encapsulating the message into a queue, and finally takes the entity out of the queue by a processing thread for disk refreshing. Because the service nodes are deployed on the physical machine in a mixed manner, a plurality of nodes may affect each other, the speed of disk refreshing is reduced, messages are accumulated in the queue, and if the size of the messages in the queue exceeds the size of the messages which can be stored in the heap area, service downtime is caused, so that speed limitation is required to be performed according to the total size of the messages in the message queue and the disk refreshing delay. The method comprises the following specific steps:

fig. 4 is a flow chart illustrating a message processing method according to another exemplary embodiment. The message processing method 40 includes at least steps S402 to S408.

As shown in fig. 4, in S402, the available memory of the heap memory corresponding to the message queue of the server is obtained. Specifically, for example, the actual maximum memory is obtained based on the memory management function; acquiring a memory occupied by a tracelog queue and a memory occupied by basic information; and determining the available memory based on the actual maximum memory, the memory occupied by the tracelog queue and the memory occupied by the basic information.

The tracelog queues and other underlying information (including objects that connect the objects to the Netty framework itself, etc.) may now be included in the heap. The calculation method is as follows:

the Tracelog comprises 3 parts (product, store and control), wherein the total number of each part in a Tracelog queue is not more than 20W, the content of the Tracelog is a line log, each line is not more than 150 bytes, and the total consumption is as follows: 200000 x 3 x 150 ═ 90M.

The method for testing the memory occupied by the basic information comprises the following steps:

starting the single node service, executing System.gc () every 2 seconds to start FGC, and then obtaining total memory and free memory through Runtime to obtain the difference value between the two.

100 topics are deployed, 1 sender and 2 subscribers per topic (only start, no message sent).

Obtaining max memory through Runtime, and subtracting the memory occupied by the transcolog queue and the memory occupied by the basic information to obtain the maximum memory usable by the message queue.

In S404, the plurality of messages are received when the available space threshold is less than a first space threshold. Specifically, for example, when the available space threshold is smaller than the first space threshold and the number of times of the disk-swiping operation is smaller than the disk-swiping threshold, the plurality of messages are received.

In S406, when the available space is greater than the first space threshold and less than the second space threshold, a notification message is generated so that the client sends a subsequent message to the other server. When the message amount in the queue reaches 80% of the available space of the heap memory and the disk refreshing operation is continued for 3 times for more than 2 seconds, the server side informs all the senders to send the messages to other service nodes of the cluster where the subject is located.

In S408, the plurality of messages are discarded when the available space is greater than a second space threshold. The amount of messages in the queue reaches 100% of the available space of the heap memory, the disk refreshing operation is continued for 3 times for more than 2 seconds, and the server discards all received messages.

Fig. 5 is a flowchart illustrating a message processing method according to another example embodiment. The flow 50 shown in fig. 5 is a detailed description of the overall process of the message processing method of the present disclosure.

As shown in fig. 5, in S501, whether the memory limit is exceeded.

In S502, whether to flush a recovery period in the message queue.

In S503, whether the subject application threshold is exceeded.

In S504, whether the speed-limiting subject is the speed-limiting subject counted in the last second.

In S505, whether the current second needs to be limited in proportion or not is judged.

In S506, it is determined whether 80% of the maximum message amount of the individual is exceeded.

In S507, whether it is the most-subject message.

In S508, it is determined whether 90% of the maximum message amount of the stand-alone is exceeded.

In S509, it is a top 5 subject message.

In S510, it is determined whether 95% of the maximum message amount of the stand-alone is exceeded.

In S511, it is stored.

In S512, it is discarded.

The server side takes every second as a speed limit period, the server side reports the subject speed to the registration center every 100ms, the registration center collects the speed, if the speed is overspeed, the speed limit identification, the subject and the speed limit time are pushed to the server side, and the server side discards the subsequently received messages of the specified subject within the second.

Besides the speed limit according to the sending speed, the speed limit can also be according to the current heap memory use condition. After receiving the message, the server puts the entity encapsulating the message into a queue, and finally takes the entity out of the queue by a processing thread for disk refreshing. Because the service nodes are deployed on the physical machine in a mixed manner, a plurality of nodes may affect each other, the speed of disk refreshing is reduced, messages are accumulated in the queue, and if the size of the messages in the queue exceeds the size of the messages which can be stored in the heap area, service downtime is caused, so that speed limitation is required to be performed according to the total size of the messages in the message queue and the disk refreshing delay.

The server counts the topic of the message volume top 5 every 10 ms. The current limiting strategy is executed after the message volume reaches a certain proportion of the maximum bearing capacity: up to 80%, discarding the subject with the largest message amount; up to 90%, discarding the message volume top 5 topic; up to 95%, the whole amount is discarded.

Those skilled in the art will appreciate that all or part of the steps implementing the above embodiments are implemented as computer programs executed by a CPU. When executed by the CPU, performs the functions defined by the above-described methods provided by the present disclosure. The program may be stored in a computer readable storage medium, which may be a read-only memory, a magnetic or optical disk, or the like.

Furthermore, it should be noted that the above-mentioned figures are only schematic illustrations of the processes involved in the methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods. For details not disclosed in the embodiments of the apparatus of the present disclosure, refer to the embodiments of the method of the present disclosure.

Fig. 6 is a block diagram illustrating a message processing apparatus according to another exemplary embodiment. As shown in fig. 6, the message processing apparatus 60 is applicable to a server in a distributed service cluster, and includes: an acquisition module 602, a calculation module 604, a target module 606, and a discard module 608.

The obtaining module 602 is configured to obtain, by a server, a plurality of messages from a plurality of clients;

the calculation module 604 is configured to generate a total amount of messages and a ranking of message topics in real time based on the message topics of the plurality of messages and the number of messages corresponding to the message topics;

the target module 606 is configured to extract a target message topic based on the message topic ranking when the total message amount is within a preset threshold interval; the preset threshold interval can be determined according to the maximum bearing capacity of the server.

The discarding module 608 is configured to discard a plurality of messages corresponding to the target message topic. For example, after a preset time interval, the discarding module 608 discards a plurality of messages corresponding to the target message topic. The discarding module 608 is further configured to discard the message corresponding to the message topic when the message topic is consistent with the message topic in the speed limit topic set.

Fig. 7 is a block diagram illustrating a message processing apparatus according to an example embodiment. As shown in fig. 7, the message processing apparatus 70 may be applied to a server in a distributed service cluster, and includes: a collection module 702, a pre-receiving module 704, a subject module 706, a memory module 708, and a receiving module 710.

The aggregation module 702 is configured to receive a speed limit subject aggregation sent by the registry, where the speed limit subject aggregation includes a message subject;

the pre-receiving module 704 is configured to receive a plurality of messages from a plurality of clients;

the topic module 706 is configured to determine message topics corresponding to the plurality of messages;

the memory module 708 is configured to obtain an available memory of a heap memory corresponding to the message queue of the server; the memory module 708 is further configured to obtain an actual maximum memory based on the memory management function; acquiring a memory occupied by a tracelog queue and a memory occupied by basic information; and determining the available memory based on the actual maximum memory, the memory occupied by the tracelog queue and the memory occupied by the basic information.

The receiving module 710 is configured to receive the plurality of messages when the available space threshold is less than a first space threshold.

According to the message processing device disclosed by the invention, a server side acquires a plurality of messages from a plurality of clients; generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics; when the total message amount is within a preset threshold value interval, extracting a target message topic based on the message topic ranking; the method for discarding the plurality of messages corresponding to the target message theme can determine the theme range of the appropriate speed-limiting message in both the normal network state and the extreme network state, thereby limiting the speed of the excessive messages, avoiding the occurrence of network risk conditions and reducing the cost of manpower and material resources.

Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, which includes a processor and a memory, where the memory stores a computer-executable program, and when the computer program is executed by the processor, the processor executes a vehicle intelligent assistance pushing method based on rotation angle monitoring.

As shown in fig. 8, the electronic device is in the form of a general purpose computing device. The processor can be one or more and can work together. The invention also does not exclude that distributed processing is performed, i.e. the processors may be distributed over different physical devices. The electronic device of the present invention is not limited to a single entity, and may be a sum of a plurality of entity devices.

The memory stores a computer executable program, typically machine readable code. The computer readable program may be executed by the processor to enable an electronic device to perform the method of the invention, or at least some of the steps of the method.

The memory may include volatile memory, such as Random Access Memory (RAM) and/or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

Optionally, in this embodiment, the electronic device further includes an I/O interface, which is used for data exchange between the electronic device and an external device. The I/O interface may be a local bus representing one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, and/or a memory storage device using any of a variety of bus architectures.

It should be understood that the electronic device shown in fig. 8 is only one example of the present invention, and elements or components not shown in the above example may be further included in the electronic device of the present invention. For example, some electronic devices further include a display unit such as a display screen, and some electronic devices further include a human-computer interaction element such as a button, a keyboard, and the like. Electronic devices are considered to be covered by the present invention as long as the electronic devices are capable of executing a computer-readable program in a memory to implement the method of the present invention or at least a part of the steps of the method.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, as shown in fig. 9, the technical solution according to the embodiment of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, or a network device, etc.) to execute the above method according to the embodiment of the present disclosure.

The software product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

The computer readable medium carries one or more programs which, when executed by a device, cause the computer readable medium to perform the functions of: : the server side acquires a plurality of messages from a plurality of client sides; generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics; when the total message amount is within a preset threshold value interval, extracting a target message topic based on the message topic ranking; and discarding a plurality of messages corresponding to the target message subject. Those skilled in the art will appreciate that the modules described above may be distributed in the apparatus according to the description of the embodiments, or may be modified accordingly in one or more apparatuses unique from the embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

Exemplary embodiments of the present disclosure are specifically illustrated and described above. It is to be understood that the present disclosure is not limited to the precise arrangements, instrumentalities, or instrumentalities described herein; on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A message processing method can be applied to a server side in a distributed service cluster, and is characterized by comprising the following steps:

the server side acquires a plurality of messages from a plurality of client sides;

generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics;

when the total message amount is within a preset threshold value interval, extracting a target message topic based on the message topic ranking;

and discarding a plurality of messages corresponding to the target message subject.

2. The message processing method of claim 1, wherein before the server obtains the plurality of messages from the plurality of clients, further comprising:

receiving a speed limit subject set sent by the registration center, wherein the speed limit subject set comprises message subjects;

receiving a plurality of messages from a plurality of clients;

determining message topics corresponding to the plurality of messages;

and when the message theme is consistent with the message theme in the speed limit theme set, discarding the message corresponding to the message theme.

3. The message processing method of claim 2, wherein before receiving the set of rate limiting topics sent by the registry, further comprising:

the server side obtains a plurality of message themes to be processed currently and the number of messages corresponding to the message themes;

and sending the plurality of message themes and the corresponding message number to a registration center in the distributed service cluster at regular time.

4. The message processing method of claim 1, wherein before the server obtains the plurality of messages from the plurality of clients, further comprising:

obtaining an available memory of a heap memory corresponding to a message queue of the server;

receiving the plurality of messages when the available space threshold is less than a first space threshold.

5. The message processing method of claim 4, further comprising:

when the available space is larger than a first space threshold and smaller than a second space threshold, generating a notification message to enable the client to send a subsequent message to other servers;

discarding the plurality of messages when the available space is greater than a second space threshold.

6. The message processing method according to claim 1, wherein obtaining the available memory of the heap memory corresponding to the message queue of the server comprises:

acquiring an actual maximum memory based on a memory management function;

acquiring a memory occupied by a tracelog queue and a memory occupied by basic information;

and determining the available memory based on the actual maximum memory, the memory occupied by the tracelog queue and the memory occupied by the basic information.

7. The message processing method of claim 4, wherein receiving the plurality of messages when the available space threshold is less than a first space threshold comprises:

receiving the plurality of messages when the available space threshold is less than a first space threshold and the number of times of disk-swiping operations is less than a disk-swiping threshold.

8. The message processing method of claim 1, further comprising:

and determining a preset threshold interval according to the maximum bearing capacity of the server.

9. The message processing method of claim 8, wherein extracting a target message topic based on the message topic ranking when the total number of messages is within a preset threshold interval comprises:

when the total message amount is within a first threshold interval, taking the first position in the message topic ranking as the target message topic;

when the total message amount is within a second threshold interval, taking the first five bits in the message topic ranking as the target message topic;

and when the total message amount is within a third threshold interval, all of the message topic ranks are taken as the target message topics.

10. The message processing method according to claim 1, wherein discarding the plurality of messages corresponding to the target message topic comprises:

and after a preset time interval, discarding the plurality of messages corresponding to the target message theme.

11. A message processing apparatus, applicable to a server in a distributed service cluster, comprising:

the acquisition module is used for the server side to acquire a plurality of messages from a plurality of clients;

the calculation module is used for generating a total message amount and a message topic ranking in real time based on the message topics of the plurality of messages and the number of the messages corresponding to the message topics;

the target module is used for extracting a target message topic based on the message topic ranking when the total message amount is within a preset threshold interval;

and the discarding module is used for discarding a plurality of messages corresponding to the target message subject.

12. The message processing apparatus of claim 11, further comprising:

the collection module is used for receiving a speed limit subject collection sent by the registration center, wherein the speed limit subject collection comprises message subjects;

a pre-receiving module for receiving a plurality of messages from a plurality of clients;

the theme module is used for determining message themes corresponding to the plurality of messages;

and the discarding module is further used for discarding the message corresponding to the message theme when the message theme is consistent with the message theme in the speed limit theme set.

13. The message processing apparatus of claim 1, further comprising:

the memory module is used for acquiring the available memory of the heap memory corresponding to the message queue of the server;

a receiving module for receiving the plurality of messages when the available space threshold is less than a first space threshold.

14. An electronic device, comprising:

one or more processors;

storage means for storing one or more programs;

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-10.

15. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-10.