CN110727507A - Message processing method and device, computer equipment and storage medium - Google Patents
Message processing method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN110727507A CN110727507A CN201911001012.5A CN201911001012A CN110727507A CN 110727507 A CN110727507 A CN 110727507A CN 201911001012 A CN201911001012 A CN 201911001012A CN 110727507 A CN110727507 A CN 110727507A
- Authority
- CN
- China
- Prior art keywords
- queue
- message
- user
- processing
- messages
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
Abstract
The invention relates to a message processing method, a message processing device, computer equipment and a storage medium, wherein the method comprises the following steps: receiving a message sent by a user in a session; distributing the message to a queue corresponding to the user; calling a thread corresponding to the queue to send the message of the user in the queue to other users in the conversation; determining that the thread has processed all messages of the user in the queue; detecting a state of the queue; and transferring the user's message from the queue to other queues according to the state. The method ensures the sequency of the messages of the user dimension, and allocates the messages of the users in different queues according to the states of the queues, thereby reducing the load of part of the queues, improving the utilization rate of other queues and corresponding threads, leading the load of each queue to tend to be balanced and realizing the dynamic allocation of resources.
Description
Technical Field
The embodiment of the invention relates to the technology of instant messaging, in particular to a message processing method, a message processing device, computer equipment and a storage medium.
Background
With the development of internet technology, Instant Messenger (IM) is rapidly popularized due to its advantages of real-time performance, low cost, high efficiency, etc. Instant messaging is an internet-based instant messaging service that allows two or more people to communicate text messages, files, voice and video in real time using a network.
In instant messaging, when a certain user sends a message, the server needs to push the message to other users, and at the moment, the message is diffused into a plurality of messages to flow in each system of the server, so that the diffusion coefficient is large, and the pressure of the server is high.
In the current internet technology architecture, in order to deal with the instant messaging scenes with high concurrency and large flow, a distributed technical scheme is generally adopted. By adopting the distributed scheme, the throughput and the concurrency of the whole architecture can be effectively improved in a horizontal and horizontal extension mode.
Currently, a message is shunted according to a certain dimension to ensure that the message of each dimension is only fixedly processed by the same thread to ensure the sequence of message processing of the dimension, but the thread fixedly processes the message of a certain dimension to cause low resource utilization rate.
Disclosure of Invention
Embodiments of the present invention provide a method and an apparatus for processing a message, a computer device, and a storage medium, so as to solve the problem of ensuring both the order of processing the dimension message and the resource utilization rate.
In a first aspect, an embodiment of the present invention provides a method for processing a message, including:
receiving a message sent by a user in a session;
distributing the message to a queue corresponding to the user;
calling a thread corresponding to the queue to send the message of the user in the queue to other users in the conversation;
determining that the thread has processed all messages of the user in the queue;
detecting a state of the queue;
and transferring the user's message from the queue to other queues according to the state.
Optionally, the allocating the message to the queue corresponding to the user includes:
generating message sequence numbers for the messages according to a preset sequence;
updating the latest sequence number corresponding to the user into the message sequence number of the message in a statistical list;
determining a queue to process the user's message;
writing the message to the queue.
Optionally, after the updating the latest sequence number corresponding to the user in the statistics list to the message sequence number of the message, the method further includes:
adding a sharing lock to the user;
after the determining a queue to process the user's messages, further comprising:
releasing the sharing lock for the user.
Optionally, the invoking a thread corresponding to the queue to send the message of the user in the queue to other users in the session includes:
determining a thread corresponding to the queue;
calling the thread to send the message of the user in the queue to other users in the conversation;
if the message is successfully sent, updating the processing sequence number corresponding to the user in the statistical list to be the message sequence number of the message;
the processing times corresponding to the users in the statistical list are increased once;
and increasing the total processing times corresponding to the queues in the statistical list once.
Optionally, the determining that the thread has processed all messages of the user in the queue includes:
inquiring the latest serial number and the processing serial number corresponding to the user in the statistical list;
and if the latest sequence number is equal to the processing sequence number, determining that the thread has processed all the messages of the user in the queue.
Optionally, the detecting the state of the queue includes:
inquiring the total processing times corresponding to the queues in the statistical list;
if the total processing times are larger than a preset first threshold value, determining that the queue is in a busy state;
if the total processing times are smaller than a preset second threshold value, determining that the queue is idle;
wherein the first threshold is greater than the second threshold.
Optionally, the transferring the user's message from the queue to another queue according to the status includes:
if the queue is busy, searching other queues with idle states;
transferring the user's message from the queue to the other queue.
Optionally, before the transferring the message of the user from the queue to the other queue, further comprising:
adding an exclusive lock to the user;
after the transferring the user's message from the queue to the other queue, further comprising:
releasing the exclusive lock to the user.
Optionally, the transferring the user's message from the queue to the other queue includes:
in a statistical list, transferring the latest sequence number, the processing sequence number and the processing times corresponding to the user from the queue to the other queues;
subtracting the processing times on the basis of the total processing times corresponding to the queue;
and adding the processing times on the basis of the total processing times corresponding to the other queues.
In a second aspect, an embodiment of the present invention further provides a device for processing a message, including:
the message receiving module is used for receiving messages sent by users in the session;
the message distribution module is used for distributing the message to a queue corresponding to the user;
the message processing module is used for calling the thread corresponding to the queue to send the messages of the users in the queue to other users in the conversation;
a processing completion module for determining that the thread has processed all messages of the user in the queue;
the state detection module is used for detecting the state of the queue;
and the message transfer module is used for transferring the message of the user from the queue to other queues according to the state.
Optionally, the message distribution module includes:
the message sequence number generation submodule is used for generating a message sequence number for the message according to a preset sequence;
a latest sequence number updating submodule for updating the latest sequence number corresponding to the user to the message sequence number of the message in a statistical list;
a queue determination submodule for determining a queue for processing the user's message;
and the message writing submodule is used for writing the message into the queue.
Optionally, the message distribution module further includes:
the sharing lock adding submodule is used for adding a sharing lock to the user;
and the sharing lock releasing submodule is used for releasing the sharing lock for the user.
Optionally, the message processing module includes:
the thread determining submodule is used for determining a thread corresponding to the queue;
the message sending submodule is used for calling the thread to send the messages of the user in the queue to other users in the conversation;
a processing sequence number updating submodule, configured to update, if the sending is successful, the processing sequence number corresponding to the user in the statistical list to the message sequence number of the message;
a processing frequency increasing submodule for increasing the processing frequency corresponding to the user once in the statistical list;
and the total processing time increasing submodule is used for increasing the total processing time corresponding to the queue in the statistical list once.
Optionally, the processing completion module includes:
a serial number query submodule for querying the latest serial number and the processing serial number corresponding to the user in the statistical list;
and the equality determining submodule is used for determining that the thread has processed all the messages of the user in the queue if the latest sequence number is equal to the processing sequence number.
Optionally, the state detection module includes:
a total processing times inquiry submodule for inquiring the total processing times corresponding to the queues in the statistical list;
a busy queue determining submodule, configured to determine that the queue is busy if the total processing time is greater than a preset first threshold;
an idle queue determining submodule, configured to determine that the queue is idle if the total processing time is less than a preset second threshold;
wherein the first threshold is greater than the second threshold.
Optionally, the message transfer module includes:
the idle queue searching submodule is used for searching other queues with idle states if the queues are busy;
and the busy queue transferring submodule is used for transferring the message of the user from the queue to the other queues.
Optionally, the message transfer module further includes:
an exclusive lock adding submodule, configured to add an exclusive lock to the user;
and the exclusive lock releasing submodule is used for releasing the exclusive lock for the user.
Optionally, the busy queue transfer sub-module includes:
a parameter transferring unit, configured to transfer, in a statistical list, the latest sequence number, the processing sequence number, and the processing frequency corresponding to the user from the queue to the other queues;
a total processing number subtracting unit, configured to subtract the processing number based on the total processing number corresponding to the queue;
and the total processing number value adding unit is used for adding the processing number on the basis of the total processing number corresponding to the other queues.
In a third aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a method of processing a message as in any one of the first aspects.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for processing a message according to any one of the first aspect.
In this embodiment, a message sent by a user in a session is received, the message is distributed to a queue corresponding to the user, a thread corresponding to the queue is called to send the message of the user in the queue to other users in the session, all messages of the user in the queue processed by the thread are determined, the state of the queue is detected, the message of the user is transferred from the queue to other queues according to the state, because the message of a certain user dimension enters one queue and is processed by the certain queue, the condition that a plurality of threads simultaneously process the message of the same user dimension can not occur, thereby ensuring the sequentiality of the messages of the user dimension, meanwhile, allocating the messages of the user in different queues according to the state of the queues, the method can reduce the load of part of queues, improve the utilization rate of other queues and corresponding threads, balance the load of each queue and realize dynamic resource allocation.
Drawings
Fig. 1 is a flowchart of a message processing method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an instant messaging system according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a message processing apparatus according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device according to a third embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Currently, the session scenario of instant messaging mainly includes single chat and group chat.
The two conversation scenes have high requirements on the processing sequence of the messages, and if the sequence is disordered, the chat semantics can be wrong. In an instant messaging system with a low concurrency, the occurrence probability of the situation is low, but in an instant messaging system with a large number of users, the occurrence probability of the situation is high.
Particularly, for group chat, there are many members in the group, and when any one member sends a message in the group, the message becomes multiple messages and is pushed to other members in the group respectively. Then, it has been one of the technical problems in the industry to ensure that the order of messages received by all members is the same.
For example, in a group of 500 people, A, B, C are three of them. At some point A, B, C three members have sent three messages simultaneously, respectively: message 1, message 2, message 3, it is necessary to ensure that the order of messages seen by all members is the same (i.e., message 1, message 2, message 3).
The business has the following schemes for the order of messages in instant messaging:
(1) the first scheme is as follows: aiming at single chat and group chat, a sequence number is added to a message, and the message is displayed according to the sequence of the sequence number after a user at the opposite side receives the message (generally, the sequence number is larger, the message is newer), if a certain message arrives slowly, the message needs to be inserted into the middle of a chat record for display, and the correctness of the whole chat semantic meaning is ensured.
In the first scheme, only the message sequence of the message displayed finally is guaranteed to be correct, but in the process of receiving the message, as the whole framework is distributed, a plurality of messages of the same user can be processed by different systems, the processing speeds of the different systems can be high and low, and a thread pool, a message queue and the like are often adopted in the system to improve the concurrent processing efficiency of the message, the situation that the message suddenly appears in the middle of the chat record can occur, the message display is more impressive for some instant communication systems with lower efficiency, and the user experience is poorer.
(2) Scheme II: for single chat and group chat, the timestamp of the server is added to the message, and then displayed in the order of the size of the timestamp at the client (generally, the larger the timestamp, the newer the message).
However, in a distributed architecture, a unified clock is lacked (that is, the time of different servers is not exactly the same, and there is a time difference), so that the scheme still has a situation that the order of messages is inaccurate, and the accuracy is low.
(3) The third scheme is as follows: and aiming at single chat and group chat, the timestamp of the message sent by the client is used as the basis for displaying the client receiving the message.
However, the time on the client can be modified at will, and if a user modifies the time maliciously and sends a message again, the trouble is brought to a message receiver, the user experience is poor, and the influence is bad.
(4) And the scheme is as follows: and aiming at the group chat scene, processing the messages in the group in a pull mode. That is, only one copy of the message is provided for the same group, and all members in the group periodically pull the message to the server in increments through the client.
The fourth scheme is poor in instantaneity, if the number of inactive groups is large, the server query is easily wasted due to the fact that messages are pulled regularly, namely when some inactive groups are queried for many times, no new message exists, the query is still executed, and server pressure is increased.
(5) And a fifth scheme: for single chat and group chat, the request is shunted according to a certain dimension (for example, according to the dimension of a chat session, the dimension of a user, the dimension of a group ID, and the like) so as to ensure that messages of each dimension are only fixedly processed by the same thread, and thus the sequence of the messages of the dimension is ensured.
Solution five can better guarantee the sequentiality of the messages, but solution five has a defect that resources (e.g., queues, threads, etc.) cannot be dynamically allocated to cope with the change of traffic (e.g., originally active user messages, become inactive, inactive user messages become active, etc.), and the resources are not fully utilized.
For an instant messaging product, there may be situations where a small percentage of users have particularly high activity and a large percentage of users have relatively low activity. Most of the traffic of the whole architecture comes from users with extremely high activity, if the scheme is adopted, part of resources are frequently used (for example, part of threads are busy), and part of resources are in an idle state (for example, part of threads are idle), and the activity of different users changes continuously, and the message traffic of corresponding users also changes.
In view of the fifth aspect, a scheme for dynamically allocating resources is provided, which can ensure that resources can be dynamically allocated while guaranteeing the sequence of messages in the framework of the distributed instant messaging system, thereby achieving the effect of fully utilizing resources.
Example one
Fig. 1 is a flowchart of a message processing method according to an embodiment of the present invention, where the method is applicable to a case where a resource is dynamically allocated to process a message of a user, and the method may be executed by a message processing apparatus, and the message processing apparatus may be implemented by software and/or hardware and may be configured in a computer device, such as a server, a workstation, and the like.
In this embodiment, as shown in fig. 2, a plurality of queues and a plurality of threads are maintained in the computer device, and generally, one queue stores messages of one or more users, one message of one user is stored in one queue, and one queue binds one thread (thread).
The thread is the smallest unit that the operating system can perform operation scheduling. It is included in the process and is the actual unit of operation in the process. A thread refers to a single sequential control flow in a process, multiple threads can be concurrently executed in a process, and each thread executes different tasks in parallel. Unix System V and SunOS are also called lightweight processes (light processes), but lightweight processes are more referred to as kernel threads (kernel threads) and user threads (user threads) are called threads.
In addition, a statistical list is maintained in the computer device, and the statistical list records the condition of processing the user message in each queue, and specifically comprises columns such as a queue ID, a user ID, a latest sequence number, a processing sequence number, processing times and total processing times.
Here, the "latest sequence number" refers to the latest number assigned to the message of the user.
The "processing sequence number" refers to a message sequence number after the thread performs logic processing on a message of a certain user.
The "number of times of processing" is the number of times a message of a certain user is processed in a unit of time.
The "total processing times" refers to the total times of the messages of all users being processed in a time unit on a certain queue.
When a certain thread finishes processing a certain message, the corresponding recorded 'processing times' and 'total processing times' are added with 1, and when the time unit is exceeded, the numerical values of the 'processing times' and the 'total processing times' are reset to 0 at regular time.
In one example, the statistical list is as follows:
in this example, user 1's message, is saved to queue 1; the information of the user 2 is stored in the queue 1; the information of the user 3 is stored in the queue 1; the information of the user 4 is stored in the queue 2; the message of the user 5 is stored in the queue 2; the message of the user 6 is stored in the queue 2.
As shown in fig. 1, the method specifically includes the following steps:
In a specific implementation, the instant messaging system has components such as a connection system, a chat system, a database, a connection system, and the like.
The connection system has a plurality of access devices, each access device is used for connecting one or more users, providing access functions for the users, and maintaining network connection, such as long connection of HTTP (Hyper Text transfer Protocol), and simultaneously maintaining connection with each downstream chat system.
It should be noted that the user refers to an instant messaging client, such as a browser, an independent instant messaging tool, an application with an instant messaging function (e.g., a payment application, a short video application, a live application, etc.), and the like, and a user account registered in advance in an instant messaging system may be logged in the client to indicate the user, and operations of sending and receiving messages, etc. refer to operations executed by the client.
The chat system comprises a single chat system for processing single chat and a group chat system for processing group chat, and is mainly responsible for processing group chat service logic, such as validity check of user identity, validity check of message content, inquiry of group member list, message storage in a database, and the like.
The connection system is mainly responsible for managing the connection information of the users.
In this embodiment, the user inputs a message in different session scenarios (including single chat and group chat), sends the message to the instant messaging system, and receives the message by the access device connected to the first user.
Wherein the content of the message may be text information, image data, audio data, video data, etc.
And 102, distributing the message to a queue corresponding to the user.
In this embodiment, a user to which a message belongs may be searched, and a queue for managing the user is determined, so that the message is written in the queue and waits for processing.
Further, the user ID of the user to which the message belongs may be determined, and the queue ID corresponding to the user ID is searched for in the statistical list, so that the message may be written into the queue to which the queue ID belongs.
In a specific implementation, as shown in fig. 2, the message sequence numbers may be generated for the messages according to a preset sequence, and in general, the sequence is incremented, that is, the value of the message sequence number is incremented each time.
The message sequence number and the user ID of the message are routed to a statistical list for updating, the latest sequence number corresponding to the user is updated to the message sequence number of the message in the statistical list, a queue for processing the message of the user is determined, and the message is written into the queue through the routing.
For example, for the above example, if the message of the user 3 is received, and the latest sequence number corresponding to the user 3 in the statistical list is 87, the message sequence number 88 may be generated for the message, and the latest sequence number 88 may be updated, and at this time, the statistical list is as follows:
further, after updating the latest sequence number, a shared lock may be added to the user (user ID), and after determining the queue (queue ID), the shared lock may be released to the user
So-called shared locks, also called read locks, multiple threads can simultaneously acquire the same shared lock to read data concurrently. If an exclusive lock is desired, then wait for all threads that acquire the shared lock to release the shared lock.
The sharing lock is added to the ID dimension of the user, the granularity of the lock is fine, the lock cannot compete with other users, the lock is shared, the lock cannot compete with the read operation recorded by the same user, and therefore the influence on the performance is small.
The purpose of the shared lock is to read dirty data (i.e., the old "queue ID") in order to prevent concurrent conflicts with dynamic deployment of resources.
In this embodiment, as shown in fig. 2, a thread that processes messages in the queue may be searched, so that the thread is concurrently invoked to process messages of each user in the queue in order, and update the statistical list.
In a certain conversation, the message enters one of the task queues and is processed by a certain thread, so that the message serial processing in the conversation (namely, the message in any one conversation is processed one by one) can be achieved, and the effect of the whole conversation message parallel processing (namely, the messages in different conversations enter different queues and are processed by different threads concurrently) can be achieved, thereby not only ensuring the sequence of the messages in the conversation, but also improving the overall architecture height.
Further, the thread service logic processes, including performing validity check on the user (identity) and/or the message, such as whether the first user belongs to a group, is forbidden, and the like, and whether the message contains sensitive information (such as a file containing a virus, a URL (Uniform Resource Locator) of an illegal web page, and the like).
If the validity check is not passed, the message is ignored.
If the validity is verified, the message is stored in a database to realize persistence, so that the message roaming and other processing are carried out, and the message is sent to an access system which sends the message to the user.
In a specific implementation, a thread corresponding to the queue can be determined, and the thread is called to send messages of the user in the queue to other users in the session in sequence.
And if the transmission is successful, updating the processing sequence number corresponding to the user in the statistical list to be the message sequence number of the message.
The number of times of processing corresponding to the user is increased once in the statistical list, and the total number of times of processing corresponding to the queue is increased once in the statistical list.
For example, for the above example, if the calling thread processes the message with the message number 88 in the queue of user 3, the processing number 88 corresponding to the user 3, the processing frequency 23(22+1) corresponding to the updated user 3, and the total processing period frequency 53(52+1) of the updated queue 1 may be updated, and at this time, the statistical list is as follows:
and 104, determining that the thread processes all the messages of the user in the queue.
In this embodiment, each time a message of a certain user is processed, whether all messages of the user in the queue are processed can be detected. If yes, the dynamic resource allocation is started asynchronously, otherwise, the process is ended.
In a specific implementation, the latest sequence number and the processing sequence number corresponding to the user in the statistical list can be queried and compared.
And if the latest sequence number is equal to the processing sequence number, determining that all the messages of the user in the queue are processed by the thread.
And if the latest sequence number is smaller than the processing sequence number, determining that all messages of the user in the thread unprocessed queue are not processed.
In this embodiment, the status of each queue can be detected, the operation status thereof can be determined, and the resources can be dynamically allocated according to the operation status.
In a specific implementation, the total number of processing times corresponding to the queue in the statistical list may be queried.
And comparing the total processing times with a preset first threshold and a preset second threshold, wherein the first threshold is greater than the second threshold, and for example, the first threshold is 50 and the second threshold is 20.
And if the total processing times are larger than a preset first threshold value, determining that the state of the queue is busy.
And if the total processing times are less than a preset second threshold value, determining the state of the queue to be idle.
Of course, the above-mentioned manner of detecting the state of the queue is only an example, and when the embodiment of the present invention is implemented, other manners of detecting the state of the queue may be set according to actual situations, for example, the state of the queue is detected according to resource occupancy of a thread corresponding to the queue (e.g., Central Processing Unit (CPU) occupancy, memory occupancy, and the like), and the embodiment of the present invention is not limited thereto. In addition, besides the above-mentioned manner of detecting the state of the queue, a person skilled in the art may also adopt other manners of detecting the state of the queue according to actual needs, and the embodiment of the present invention is not limited to this.
And 106, transferring the message of the user from the queue to other queues according to the state.
In this embodiment, the processed messages of the user are transferred among the queues according to the states of the queues, so as to implement dynamic allocation of resources.
In a specific implementation, if the queue is busy, searching for another queue in an idle state, and transferring the message of the user from the queue to the other queue, wherein the resource occupancy rate of the queue in the busy state is greater than that of the queue in the idle state.
Further, when transferring the queue, the latest number, the processing number, and the number of times of processing corresponding to the user are transferred from the queue to another queue in the statistical list.
And subtracting the processing times on the basis of the total processing times corresponding to the queue to obtain a new total processing time.
And adding the processing times on the basis of the total processing times corresponding to other queues to be used as new total processing times.
For example, for the above example, all messages for user 3 have completed processing, with queue 1 having a total number of processing 53 > 50, the status is busy, queue 2 having a total number of processing 12 < 20, and the status is free.
At this time, the message of the user 3 is transferred from the queue 1 to the queue 2, the total processing time of the queue 1 is 33(53-23), and the total processing time of the queue 2 is 35(12+23), at this time, the statistical list is as follows:
further, an exclusive lock is added to the user before the transfer message, and the exclusive lock is released to the user after the transfer message.
The exclusive lock is also called a write lock or an exclusive lock, and only one thread can obtain the exclusive lock, and in the process of obtaining the exclusive lock by a certain thread, if other threads want to obtain a shared lock or an exclusive lock, the exclusive lock must wait until the thread obtaining the exclusive lock releases the exclusive lock.
Adding an exclusive lock to the user ID prevents dirty data (i.e., old queue ID) from being obtained when querying records.
In this embodiment, a message sent by a user in a session is received, the message is distributed to a queue corresponding to the user, a thread corresponding to the queue is called to send the message of the user in the queue to other users in the session, all messages of the user in the queue processed by the thread are determined, the state of the queue is detected, the message of the user is transferred from the queue to other queues according to the state, because the message of a certain user dimension enters one queue and is processed by the certain queue, the condition that a plurality of threads simultaneously process the message of the same user dimension can not occur, thereby ensuring the sequentiality of the messages of the user dimension, meanwhile, allocating the messages of the user in different queues according to the state of the queues, the method can reduce the load of part of queues, improve the utilization rate of other queues and corresponding threads, balance the load of each queue and realize dynamic resource allocation.
Example two
Fig. 3 is a schematic structural diagram of a message processing apparatus according to a fifth embodiment of the present invention, where the apparatus may specifically include the following modules:
a message receiving module 301, configured to receive a message sent by a user in a session;
a message distribution module 302, configured to distribute the message to a queue corresponding to the user;
a message processing module 303, configured to invoke a thread corresponding to the queue to send the message of the user in the queue to another user in the session;
a process complete module 304, configured to determine that the thread has processed all messages of the user in the queue;
a status detection module 305 for detecting the status of the queue;
a message transfer module 306, configured to transfer the user's message from the queue to another queue according to the status.
In a preferred embodiment of the present invention, the message distribution module 302 includes:
the message sequence number generation submodule is used for generating a message sequence number for the message according to a preset sequence;
a latest sequence number updating submodule for updating the latest sequence number corresponding to the user to the message sequence number of the message in a statistical list;
a queue determination submodule for determining a queue for processing the user's message;
and the message writing submodule is used for writing the message into the queue.
In a preferred embodiment of the present invention, the message distribution module 302 further includes:
the sharing lock adding submodule is used for adding a sharing lock to the user;
and the sharing lock releasing submodule is used for releasing the sharing lock for the user.
In a preferred embodiment of the present invention, the message processing module 303 includes:
the thread determining submodule is used for determining a thread corresponding to the queue;
the message sending submodule is used for calling the thread to send the messages of the user in the queue to other users in the conversation;
a processing sequence number updating submodule, configured to update, if the sending is successful, the processing sequence number corresponding to the user in the statistical list to the message sequence number of the message;
a processing frequency increasing submodule for increasing the processing frequency corresponding to the user once in the statistical list;
and the total processing time increasing submodule is used for increasing the total processing time corresponding to the queue in the statistical list once.
In a preferred embodiment of the present invention, the processing completion module 304 includes:
a serial number query submodule for querying the latest serial number and the processing serial number corresponding to the user in the statistical list;
and the equality determining submodule is used for determining that the thread has processed all the messages of the user in the queue if the latest sequence number is equal to the processing sequence number.
In a preferred embodiment of the present invention, the status detection module 305 includes:
a total processing times inquiry submodule for inquiring the total processing times corresponding to the queues in the statistical list;
a busy queue determining submodule, configured to determine that the queue is busy if the total processing time is greater than a preset first threshold;
an idle queue determining submodule, configured to determine that the queue is idle if the total processing time is less than a preset second threshold;
wherein the first threshold is greater than the second threshold.
In a preferred embodiment of the present invention, the message transfer module 306 includes:
the idle queue searching submodule is used for searching other queues with idle states if the queues are busy;
and the busy queue transferring submodule is used for transferring the message of the user from the queue to the other queues.
In a preferred embodiment of the present invention, the message transfer module 306 further includes:
an exclusive lock adding submodule, configured to add an exclusive lock to the user;
and the exclusive lock releasing submodule is used for releasing the exclusive lock for the user.
In a preferred embodiment of the present invention, the busy queue transfer sub-module includes:
a parameter transferring unit, configured to transfer, in a statistical list, the latest sequence number, the processing sequence number, and the processing frequency corresponding to the user from the queue to the other queues;
a total processing number subtracting unit, configured to subtract the processing number based on the total processing number corresponding to the queue;
and the total processing number value adding unit is used for adding the processing number on the basis of the total processing number corresponding to the other queues.
The message processing device provided by the embodiment of the invention can execute the message processing method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE III
Fig. 4 is a schematic structural diagram of a computer device according to a third embodiment of the present invention. As shown in fig. 4, the computer apparatus includes a processor 400, a memory 401, a communication module 402, an input device 403, and an output device 404; the number of processors 400 in the computer device may be one or more, and one processor 400 is taken as an example in fig. 4; the processor 400, the memory 401, the communication module 402, the input device 403 and the output device 404 in the computer apparatus may be connected by a bus or other means, and fig. 6 illustrates an example of connection by a bus.
The memory 401 is used as a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as modules corresponding to the message processing method in the present embodiment (for example, the message receiving module 301, the message allocating module 302, the message processing module 303, the processing completion module 304, the state detection module 305, and the message transfer module 306 in the message processing apparatus shown in fig. 3). The processor 400 executes various functional applications of the computer device and data processing, i.e., implements the above-described message processing method, by executing software programs, instructions, and modules stored in the memory 401.
The memory 401 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the computer device, and the like. Further, the memory 401 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 401 may further include memory located remotely from processor 400, which may be connected to a computer device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
And the communication module 402 is used for establishing connection with the display screen and realizing data interaction with the display screen. The input device 403 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer apparatus.
The computer device provided in this embodiment may execute the message processing method provided in any embodiment of the present invention, and its corresponding functions and advantages are described in detail.
Example four
A fourth embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a method for processing a message, and the method includes:
receiving a message sent by a user in a session;
distributing the message to a queue corresponding to the user;
calling a thread corresponding to the queue to send the message of the user in the queue to other users in the conversation;
determining that the thread has processed all messages of the user in the queue;
detecting a state of the queue;
and transferring the user's message from the queue to other queues according to the state.
Of course, the computer program of the computer-readable storage medium provided in the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the message processing method provided in any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the message processing apparatus, the included units and modules are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (12)
1. A method for processing a message, comprising:
receiving a message sent by a user in a session;
distributing the message to a queue corresponding to the user;
calling a thread corresponding to the queue to send the message of the user in the queue to other users in the conversation;
determining that the thread has processed all messages of the user in the queue;
detecting a state of the queue;
and transferring the user's message from the queue to other queues according to the state.
2. The method of claim 1, wherein the allocating the message to the queue corresponding to the user comprises:
generating message sequence numbers for the messages according to a preset sequence;
updating the latest sequence number corresponding to the user into the message sequence number of the message in a statistical list;
determining a queue to process the user's message;
writing the message to the queue.
3. The method of claim 2,
after the updating the latest sequence number corresponding to the user in the statistical list to the message sequence number of the message, the method further includes:
adding a sharing lock to the user;
after the determining a queue to process the user's messages, further comprising:
releasing the sharing lock for the user.
4. The method according to claim 1, wherein the invoking the thread corresponding to the queue sends the message of the user in the queue to other users in the session, and includes:
determining a thread corresponding to the queue;
calling the thread to send the message of the user in the queue to other users in the conversation;
if the message is successfully sent, updating the processing sequence number corresponding to the user in the statistical list to be the message sequence number of the message;
the processing times corresponding to the users in the statistical list are increased once;
and increasing the total processing times corresponding to the queues in the statistical list once.
5. The method of claim 1, wherein the determining that the thread has processed all messages for the user in the queue comprises:
inquiring the latest serial number and the processing serial number corresponding to the user in the statistical list;
and if the latest sequence number is equal to the processing sequence number, determining that the thread has processed all the messages of the user in the queue.
6. The method of claim 1, wherein the detecting the status of the queue comprises:
inquiring the total processing times corresponding to the queues in the statistical list;
if the total processing times are larger than a preset first threshold value, determining that the queue is in a busy state;
if the total processing times are smaller than a preset second threshold value, determining that the queue is idle;
wherein the first threshold is greater than the second threshold.
7. The method of any of claims 1-6, wherein said transferring the user's message from the queue to another queue according to the status comprises:
if the queue is busy, searching other queues with idle states;
transferring the user's message from the queue to the other queue.
8. The method of claim 7, further comprising, prior to said transferring said user's message from said queue to said other queue:
adding an exclusive lock to the user;
after the transferring the user's message from the queue to the other queue, further comprising:
releasing the exclusive lock to the user.
9. The method of claim 7, wherein transferring the user's messages from the queue to the other queue comprises:
in a statistical list, transferring the latest sequence number, the processing sequence number and the processing times corresponding to the user from the queue to the other queues;
subtracting the processing times on the basis of the total processing times corresponding to the queue;
and adding the processing times on the basis of the total processing times corresponding to the other queues.
10. An apparatus for processing a message, comprising:
the message receiving module is used for receiving messages sent by users in the session;
the message distribution module is used for distributing the message to a queue corresponding to the user;
the message processing module is used for calling the thread corresponding to the queue to send the messages of the users in the queue to other users in the conversation;
a processing completion module for determining that the thread has processed all messages of the user in the queue;
the state detection module is used for detecting the state of the queue;
and the message transfer module is used for transferring the message of the user from the queue to other queues according to the state.
11. A computer device, characterized in that the computer device comprises:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a method of processing a message as claimed in any one of claims 1-9.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a method for processing a message according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911001012.5A CN110727507B (en) | 2019-10-21 | 2019-10-21 | Message processing method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911001012.5A CN110727507B (en) | 2019-10-21 | 2019-10-21 | Message processing method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110727507A true CN110727507A (en) | 2020-01-24 |
| CN110727507B CN110727507B (en) | 2022-09-09 |
Family
ID=69220458
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911001012.5A Active CN110727507B (en) | 2019-10-21 | 2019-10-21 | Message processing method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110727507B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111740897A (en) * | 2020-08-06 | 2020-10-02 | 上海众旦信息科技有限公司 | Method for information processing, electronic device, and storage medium |
| US11528232B1 (en) | 2021-08-27 | 2022-12-13 | Hong Kong Applied Science and Technology Research Institute Company Limited | Apparatus and method for handling real-time tasks with diverse size based on message queue |
| CN115842691A (en) * | 2023-02-21 | 2023-03-24 | 北京友友天宇系统技术有限公司 | Message sequence guarantee method, device and equipment for distributed group communication |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6493745B1 (en) * | 1999-09-03 | 2002-12-10 | Microsoft Corporation | Message processing technique to improve client computer response to user input |
| US20100142696A1 (en) * | 2007-03-05 | 2010-06-10 | China Mobile Communications Corporation | Message-based notifying method for notifying waiting queue and application server |
| US20100293312A1 (en) * | 2009-04-27 | 2010-11-18 | Sonnier David P | Network Communications Processor Architecture |
| CN102368700A (en) * | 2011-10-25 | 2012-03-07 | 曙光信息产业(北京)有限公司 | Transmission method of massages in distributed system |
| CN102479106A (en) * | 2010-11-23 | 2012-05-30 | 金蝶软件（中国）有限公司 | Method and system for acquiring messages from multi-queue nodes |
| US20160205054A1 (en) * | 2015-01-14 | 2016-07-14 | Linkedin Corporation | Conditional delivery of electronic messages |
| CN106708607A (en) * | 2015-11-12 | 2017-05-24 | 阿里巴巴集团控股有限公司 | Congestion control method and apparatus for message queue |
| US20170228233A1 (en) * | 2016-02-09 | 2017-08-10 | Intel Corporation | Methods, apparatus, and instructions for user-level thread suspension |
| CN107168794A (en) * | 2017-05-11 | 2017-09-15 | 腾讯科技（深圳）有限公司 | The treating method and apparatus of request of data |
| CN108446183A (en) * | 2018-04-13 | 2018-08-24 | 广东亿迅科技有限公司 | Processing method and processing device based on message distribution |
| CN108667719A (en) * | 2018-04-26 | 2018-10-16 | 广州品唯软件有限公司 | A kind of real-time Message Passing method and system |
| CN109729119A (en) * | 2017-10-30 | 2019-05-07 | 中兴通讯股份有限公司 | A kind of method, apparatus, equipment and the storage medium of coordination data synchrodata |
| CN110545237A (en) * | 2019-10-08 | 2019-12-06 | 广州欢聊网络科技有限公司 | Instant messaging method, device, system, computer equipment and storage medium |
| CN110750566A (en) * | 2019-10-24 | 2020-02-04 | 广州欢聊网络科技有限公司 | Data processing method and device, cache system and cache management platform |
-
2019
- 2019-10-21 CN CN201911001012.5A patent/CN110727507B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6493745B1 (en) * | 1999-09-03 | 2002-12-10 | Microsoft Corporation | Message processing technique to improve client computer response to user input |
| US20100142696A1 (en) * | 2007-03-05 | 2010-06-10 | China Mobile Communications Corporation | Message-based notifying method for notifying waiting queue and application server |
| US20100293312A1 (en) * | 2009-04-27 | 2010-11-18 | Sonnier David P | Network Communications Processor Architecture |
| CN102479106A (en) * | 2010-11-23 | 2012-05-30 | 金蝶软件（中国）有限公司 | Method and system for acquiring messages from multi-queue nodes |
| CN102368700A (en) * | 2011-10-25 | 2012-03-07 | 曙光信息产业(北京)有限公司 | Transmission method of massages in distributed system |
| US20160205054A1 (en) * | 2015-01-14 | 2016-07-14 | Linkedin Corporation | Conditional delivery of electronic messages |
| CN106708607A (en) * | 2015-11-12 | 2017-05-24 | 阿里巴巴集团控股有限公司 | Congestion control method and apparatus for message queue |
| US20170228233A1 (en) * | 2016-02-09 | 2017-08-10 | Intel Corporation | Methods, apparatus, and instructions for user-level thread suspension |
| CN107168794A (en) * | 2017-05-11 | 2017-09-15 | 腾讯科技（深圳）有限公司 | The treating method and apparatus of request of data |
| CN109729119A (en) * | 2017-10-30 | 2019-05-07 | 中兴通讯股份有限公司 | A kind of method, apparatus, equipment and the storage medium of coordination data synchrodata |
| CN108446183A (en) * | 2018-04-13 | 2018-08-24 | 广东亿迅科技有限公司 | Processing method and processing device based on message distribution |
| CN108667719A (en) * | 2018-04-26 | 2018-10-16 | 广州品唯软件有限公司 | A kind of real-time Message Passing method and system |
| CN110545237A (en) * | 2019-10-08 | 2019-12-06 | 广州欢聊网络科技有限公司 | Instant messaging method, device, system, computer equipment and storage medium |
| CN110750566A (en) * | 2019-10-24 | 2020-02-04 | 广州欢聊网络科技有限公司 | Data processing method and device, cache system and cache management platform |
Non-Patent Citations (3)
| Title |
|---|
| RASHID MAHMOOD等: "A SIP DoS Flooding Attack Defense Mechanism Based on Custom Weighted Fair Queue Scheduling", 《2010 INTERNATIONAL CONFERENCE ON MULTIMEDIA TECHNOLOGY》 * |
| 刘云: "企业IM架构的自由扩展性研究", 《计算机应用与软件》 * |
| 张乐: "基于AMQP的即时通讯系统的实现", 《电子技术与软件工程》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111740897A (en) * | 2020-08-06 | 2020-10-02 | 上海众旦信息科技有限公司 | Method for information processing, electronic device, and storage medium |
| US11528232B1 (en) | 2021-08-27 | 2022-12-13 | Hong Kong Applied Science and Technology Research Institute Company Limited | Apparatus and method for handling real-time tasks with diverse size based on message queue |
| CN115842691A (en) * | 2023-02-21 | 2023-03-24 | 北京友友天宇系统技术有限公司 | Message sequence guarantee method, device and equipment for distributed group communication |
| CN115842691B (en) * | 2023-02-21 | 2023-05-23 | 北京友友天宇系统技术有限公司 | Message sequence assurance method, device and equipment for distributed group communication |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110727507B (en) | 2022-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10244066B2 (en) | Push notification delivery system | |
| US20200264942A1 (en) | Message management method and device, and storage medium | |
| CN107766129B (en) | Task processing method, device and system | |
| CN110727507B (en) | Message processing method and device, computer equipment and storage medium | |
| US8028085B2 (en) | Optimizing message transmission and delivery in a publisher-subscriber model | |
| CN109547511B (en) | WEB message real-time pushing method, server, client and system | |
| US11080090B2 (en) | Method and system for scalable job processing | |
| CN107241281B (en) | Data processing method and device | |
| WO2019223596A1 (en) | Method, device, and apparatus for event processing, and storage medium | |
| US8984530B2 (en) | Queued message dispatch | |
| US20130067015A1 (en) | Counting and reseting broadcast system badge counters | |
| WO2017185615A1 (en) | Method for determining service status of service processing device and scheduling device | |
| US8930518B2 (en) | Processing of write requests in application server clusters | |
| CN111200606A (en) | Deep learning model task processing method, system, server and storage medium | |
| CN111741110B (en) | Message pushing system and method | |
| CN113641410A (en) | Netty-based high-performance gateway system processing method and system | |
| KR20140063688A (en) | Scale-out system to acquire event data | |
| CN111475315A (en) | Server and subscription notification push control and execution method | |
| CN116319810A (en) | Flow control method, device, equipment, medium and product of distributed system | |
| US11870746B2 (en) | Method for chatting messages by topic based on subscription channel reference in server and user device | |
| CN109639795B (en) | Service management method and device based on AcitveMQ message queue | |
| CN110545237A (en) | Instant messaging method, device, system, computer equipment and storage medium | |
| CN110990213A (en) | Method and device for monitoring user logs in cluster environment in real time | |
| US11722451B2 (en) | Distributing event messages from a first service to additional services using a message store | |
| US11789777B2 (en) | Resource utilization method, electronic device, and computer program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |