CN109766200A - A kind of message queue processing method, device, equipment and storage medium - Google Patents
A kind of message queue processing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN109766200A CN109766200A CN201811650061.7A CN201811650061A CN109766200A CN 109766200 A CN109766200 A CN 109766200A CN 201811650061 A CN201811650061 A CN 201811650061A CN 109766200 A CN109766200 A CN 109766200A
- Authority
- CN
- China
- Prior art keywords
- message
- priority
- consumed
- high priority
- goka
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Computer And Data Communications (AREA)
Abstract
The invention discloses a kind of message queue processing method, device, equipment and storage mediums.It include: initial message queue；Receive new information；Obtain the priority and delay information of message；Judge the priority and delay information of the message；If the priority is low priority and the delay information is zero, it is wait state that the message, which is arranged,；If the priority is high priority or the priority is low priority and the delay information is zero, the message is consumed.Message queue processing method proposed by the present invention based on goka, it is preferentially consumed by the priority of message in setting queue or according to queue priority, and judge to postpone information simultaneously, carrying out which message in comprehensive descision queue should be by priority processing, and then ensure that in practical application under the excessive scene of more Message Payloads, important messages being capable of not quickly consumption in time.
Description
Technical field
The present invention relates to technical field of computer information processing, and in particular to a kind of message queue processing side based on Goka
Method, device, computer equipment and readable storage medium storing program for executing.
Background technique
Kafka is that a kind of distributed post of high-throughput subscribes to message system, it can handle the net of consumer's scale
Everything flow data in standing.This movement (web page browsing, the action of search and other users) is on modern network
One key factor of many social functions.These data are often as the requirement of handling capacity and pass through processing log and log
Polymerization is to solve.For the daily record data and off-line analysis system as Hadoop, but the limitation handled in real time is required,
This is a feasible solution.The purpose of Kafka is by the loaded in parallel mechanism of Hadoop come on unified line and offline
Message Processing, also for providing real-time message by cluster.
In Kafka, the communication between client and server be it is simple by one, it is high performance, with language without
What the Transmission Control Protocol of pass was completed.This agreement is versioned, and keeps the backward compatibility with legacy version.We mention for Kafka
A Java client is supplied, but multilingual can be used in client.
Kafka gives we provide a record stream-theme (topic), theme be will record the classification that is published to or
It submits the article source name.The topic of Kafka always multi-user；That is a theme can have 0, and one or more is subscribed to and write
Enter the consumer of data.For each theme, Kafka cluster safeguards a subregion log.
The producer issues the theme that data are selected to them.Which record the producer is responsible for selecting distribute to which in theme
A subregion.This can be completed in a circulating manner, be intended merely to balance the load, or can be according to certain semantic sectoring functions
(such as based on some key in record) is completed.More about use subregion second.
Consumer marks oneself with consumer's group name, and each record for being published to theme is passed to each subscription
Consumer's example in consumer's group.Consumer's example can in different processes or on different machines.If
All consumer's example consumer's groups having the same, then record and will effectively carry out load balancing in consumer's example.Such as
All consumer's examples of fruit have different consumer's groups, then each record will be broadcast to all consumer process.
In the distributed stream processing library of existing Apache Kafka in the processing mode of message queue, it is all made of sequence
Queue addition element is given, then takes out element from queue again, and is sent the element taken out from queue (message) to specified
Path, i.e., the Message Processing mode of typical FIFO.Under the mode under the excessive scene of more message, the important message in part
Consumption can be very slow or be blocked, and treatment effeciency is extremely low.
Summary of the invention
The present invention, which provides, a kind of is related to a kind of message queue processing method based on Goka, device, computer equipment and can
Read storage medium.
On the one hand the embodiment of the present invention provides a kind of message queue processing method based on goka, comprising:
Initial message queue；
Receive new information；
The message number that the priority and high priority for obtaining new information are not consumed；
If the priority is that the message number that low priority and the high priority are not consumed is greater than zero, it is arranged described new
Message is wait state；
If the priority is high priority or the priority is the message that low priority and high priority are not consumed
Quantity is zero, then consumes the message.
With the method for the invention it is preferred to, the initial message queue specially defines queue structure body topic
And it initializes.
With the method for the invention it is preferred to, the priority for obtaining the message, specially using in structural body
Priority determines the priority of every message.
With the method for the invention it is preferred to, what the priority and high priority of the judgement message were not consumed disappears
Before ceasing quantity, the method also includes:
A periodically preset time,
After reaching the preset time, the message number information that high priority is not consumed is calculated.
With the method for the invention it is preferred to, after the message number that the calculating high priority is not consumed, the method is also
Include:
Judge that the high priority does not consume the size of message number；
If the message number that the high priority is not consumed is greater than zero, low priority is stated described in holding do not consume message and be etc.
To state；
If the message number that the high priority is not consumed is equal to zero, low priority is stated described in unlock and does not consume message, is disappeared
Take the low priority and does not consume message.
On the other hand the embodiment of the present invention provides a kind of message queue processing unit based on goka, comprising:
Initialization unit is used for initial message queue；
Receiving unit, for receiving new information；
Acquiring unit: the message number that priority and high priority for obtaining new information are not consumed；
First judging unit: the message number that priority and high priority for judging the message are not consumed；
First setting unit: if being the message number that low priority and the high priority are not consumed for the priority
Big by zero, it is wait state that the message, which is arranged,；
Consumer unit: if being high priority for the priority or the priority is low priority and the Gao You
The message number that first grade is not consumed is zero, then consumes the message.
The apparatus according to the invention, it is preferred that further include:
Timing unit, for one preset time of timing；
Computing unit calculates the message number that high priority is not consumed after reaching the preset time.
The apparatus according to the invention, it is preferred that further include:
Second judgment unit, for judging the size of message number that the high priority is not consumed；
Holding unit states low priority described in holding if being greater than zero for the message number that the high priority is not consumed
Not consuming message is wait state；
Unlocking unit states low priority described in unlock if being equal to zero for the message number that the high priority is not consumed
Message is not consumed；
The consumer unit is also used to consume the low priority and does not consume message.
Another embodiment of the present invention provides a kind of computer equipment, the computer equipment includes processor, the processing
A kind of such as message queue based on Goka of above-mentioned any one is realized when device is for executing the computer program stored in memory
The step of processing method.
Another embodiment of the present invention provides a kind of computer readable storage mediums, are stored thereon with computer program and (refer to
Enable), a kind of such as message team based on Goka of above-mentioned any one is realized when the computer program (instruction) is executed by processor
The step of column processing method.
The invention discloses a kind of message queue processing method, device, computer equipment and readable storage based on Goka
Medium.It include: initial message queue；Receive new information；The message count that the priority and high priority for obtaining message are not consumed
Amount；Judge the priority of the message and the message number that high priority is not consumed；If the priority is low priority and institute
Stating the message number that high priority is not consumed is zero, and it is wait state that the message, which is arranged,；If the priority is high priority
Or it is zero that the priority, which is the message number that low priority and the high priority are not consumed, then consumes the message.This
Invent propose the message queue processing method based on Goka, by setting queue in message priority or according to queue it is excellent
First grade is preferentially consumed, and judges the message number that high priority is not consumed simultaneously, comes which message in comprehensive descision queue is answered
This ensure that in practical application under the excessive scene of more Message Payloads that important messages can be not fast in time by priority processing
Speed consumption.
Message queue processing method disclosed by the invention based on Goka has high efficiency and flexibility: according to demand to every
A topic sets priority, it is ensured that preferential to consume the higher information queue of priority under the premise of not influencing system performance
Object increases a priority parameter newly to determine topic's.Allow being waited for of low priority when consumption, to
After high priority has been consumed, low priority is automatically waken up, it is ensured that can be in the information for receiving high priority at the first time.
Other features and advantages of the present invention will be illustrated in the following description, also, partly becomes from specification
It obtains it is clear that understand through the implementation of the invention.The objectives and other advantages of the invention can be by written explanation
Specifically noted structure is achieved and obtained in book, claims and attached drawing.
Detailed description of the invention
Fig. 1 is that the present invention is based on the one embodiment flow diagrams of message queue processing method of Goka；
Fig. 2 is another embodiment flow diagram of message queue processing method of Goka of the present invention；
Fig. 3 is the message queue processing unit schematic diagram of three Goka of the embodiment of the present invention；
Fig. 4 is computer equipment embodiment of the present invention；
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention
Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
Description and claims of this specification and term " first ", " second ", " third ", " in above-mentioned attached drawing
The (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manage
The data that solution uses in this way are interchangeable under appropriate circumstances, so that the embodiments described herein can be in addition to illustrating herein
Or the sequence other than the content of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that
Cover it is non-exclusive include, for example, containing the process, method, system, product or equipment of a series of steps or units need not limit
In step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, produce
The other step or units of product or equipment inherently.
Kafka is the open source stream process platform developed by Apache Software Foundation, by Scala and written in Java.
Kafka is that a kind of distributed post of high-throughput subscribes to message system, it can handle the institute in the website of consumer's scale
There is movement flow data.This movement (web page browsing, the action of search and other users) is many societies on modern network
One key factor of function.These data are often as the requirement of handling capacity and are solved by processing log and log aggregation
Certainly.For the daily record data and off-line analysis system as Hadoop, but the limitation handled in real time is required, this is one
Feasible solution.The purpose of Kafka is by the loaded in parallel mechanism of Hadoop come on unified line and at offline message
Reason, also for providing real-time message by cluster.It should be noted that the proprietary term explanation in the application field
It is as follows:
Broker
Kafka cluster includes one or more servers, and this server is referred to as broker [5]
Topic
Every message for being published to Kafka cluster has a classification, this classification is referred to as Topic.It is (physically different
The message of Topic is stored separately, in logic the message of a Topic be although stored on one or more broker but user only
Need to specify the Topic of message can produce or consumption data without be concerned about data where be stored in)
Partition
Partition is concept physically, and each Topic includes one or more Partition.
Producer
It is responsible for giving out information to Kafka broker
Consumer
Message consumer reads the client of message to Kafka broker.
Consumer Group
Each Consumer, which belongs to a specific Consumer Group, (can specify group for each Consumer
Name belongs to the group of default if not specified group name).
In the distributed stream processing library of existing Apache Kafka in the processing mode of message queue, it is all made of sequence
Queue addition element is given, then takes out element from queue again, and is sent the element taken out from queue (message) to specified
Path, i.e., the Message Processing mode of typical FIFO.Under the mode under the excessive scene of more message, the important message in part
Consumption can be very slow or be blocked, and treatment effeciency is extremely low.The embodiment of the present invention provides a kind of message queue processing method based on Goka,
It is preferentially consumed by the priority of message in setting queue or according to queue priority, and judges that high priority does not disappear simultaneously
The message number taken, carrying out which message in comprehensive descision queue should be by priority processing, and then ensure that more in practical application
Under the excessive scene of Message Payload, important messages being capable of not quickly consumption in time.
Specifically, providing a kind of message queue processing method based on Goka in the embodiment of the present invention one, comprising:
Step 101: initial message queue.
In the embodiment of the present invention, it should be noted that in a kind of preferred mode, the initial message queue, specifically
To define queue structure's body topic and initializing.
Preferably a kind of implementation is to define queue structure's body topic for the initialization of queue in the embodiment of the present invention one.
Queue connects structural body and defines:
Priority in structural body determines the priority of each topic, has only done the differentiation of height two-stage at present, has consumed
It can be according to the value of this field as one of Rule of judgment when with waiting.
Step 103: receiving new information.
The present invention defines set of the QueueList as the topic message of all initialization.
Step 105: the message number that the priority and high priority for obtaining message are not consumed.
Newly-built queue simultaneously constructs start method, and the processor in Goka can be called when constructing start method
The Start method of object is reconstructed the Start method of processor herein, and newly-increased parameter (pri int) is for receiving
The priority of topic.
Step 107 judges the message number that the priority of the message and high priority are not consumed.
Preferably, in the embodiment of the present invention one, what the priority and high priority of the acquisition message were not consumed disappears
Quantity is ceased, the priority of every message is specially determined using the Priority in structural body.
If the priority is that the message number that low priority and the high priority are not consumed is zero, the method is also wrapped
Include step 109: it is wait state that the message, which is arranged,.
The Start method of processor in reconstruct open source packet Goka and the run method of inside.
In run method, cond object is created, first by cond object acquisition thread lock, judges whether there is Gao You
The topic of first grade and non-consumption information number Lag > 0 then allow low priority if it does, calling the waiting method of cond object
Topic waits notice, temporary block, if it does not exist, then discharging thread lock, the non-consumption information number of juxtaposition by cond object
Lag is 0.
It is discharged and is locked according to condition, it is ensured that the problem of being not in deadlock increases QueueLock (queue newly in processor
Lock) method:
The Information Number that initialization high priority topic is not consumed is 0；
Judge whether the Information Number that high priority topic is not consumed is less than limiting value, is if it is obtained by cond object
Thread lock is taken, the non-consumption information number Lag of juxtaposition is 0, then by cond object reference thread broadcasting method, passes through cond object
Release lock.
If the priority is high priority or the priority is low priority and the high priority is not consumed
Message number is zero, and the method also includes steps 111: consuming the message.
It should be noted that the distributed post that Kafka is a kind of high-throughput subscribes to message system, it, which can handle, disappears
Everything flow data in the website of the person's of expense scale.Kafka is that message management is carried out by topic, and each topic includes
The corresponding logic log of multiple partition, each parttition, is made of multiple segment.It is deposited in each segment
It storing up multiple messages (following figure), message id is determined by its logical place, i.e., the storage location of message can be directly targeted to from message id,
Avoid id to the additional mapping of position.Each part corresponds to an index in memory, records first in each segment
Message offset.The message that publisher is dealt into some topic can be evenly dispersed on multiple partition (or according to
The specified routing rule in family is distributed), broker receives the last one segment to give out information toward corresponding partition
Upper addition message, when the message bar number on some segment reaches Configuration Values or the news release time is more than threshold value,
Message on segment can be by flush to disk, and the message subscribing person on only flush to disk can just subscribe to,
Segment will not write data toward the segment again after reaching certain size, and broker will create new segment.
The message queue processing method based on Goka proposed in the embodiment of the present invention one passes through message in setting queue
Priority is preferentially consumed according to queue priority, and judges the message number that high priority is not consumed simultaneously, to integrate
Judge which message should be by priority processing in queue, and then ensure that in practical application under the excessive scene of more Message Payloads,
Important messages being capable of not quickly consumption in time.The message queue processing method based on Goka in the present embodiment have high efficiency and
Flexibility: priority is set to each topic according to demand, it is ensured that under the premise of not influencing system performance, preferentially consume excellent
The first higher information queue object of grade increases a priority parameter newly to determine topic's.It is allowed when consumption low preferential
Being waited for for grade, after having consumed to high priority, automatically wakes up low priority, it is ensured that can receive at the first time
The information of high priority.
In the embodiment of the present invention two, a kind of message queue processing method based on Goka includes: as shown in Figure 2
Step 201: initial message queue；
It should be noted that in the embodiment of the present invention, it should be noted that in a kind of preferred mode, the initialization
Message queue specially defines queue structure body topic and initializes.
Preferably a kind of implementation is a fixed queue structure body topic for the initialization of queue in the embodiment of the present invention one,
Priority in structural body determines the priority of each topic, has only done the differentiation of height two-stage at present, has consumed and waiting
When can be according to the value of this field as one of Rule of judgment.
Queue connects structural body and defines:
Priority in structural body determines the priority of each topic, has only done the differentiation of height two-stage at present, has consumed
It can be according to the value of this field as one of Rule of judgment when with waiting.
Newly-built queue simultaneously constructs start method, and the processor in Goka can be called when constructing start method
The Start method of object is reconstructed the Start method of processor herein, and newly-increased parameter (pri int) is for receiving
The priority of topic.
Step 203: receiving new information；
Step 205: the message number that the priority and high priority for obtaining message are not consumed；
It should be noted that the message number that the priority for obtaining the message and high priority are not consumed, specifically
For the priority for using the Priority in structural body to determine every message.
Step 207: judging the message number that the priority of the message and high priority are not consumed；
If the priority is that the message number that low priority and high priority are not consumed is greater than zero, the method includes steps
Rapid 209: it is wait state that the message, which is arranged,；
If the priority is high priority or the priority is low priority and the high priority is not consumed
Message number is zero, and the method also includes steps 211: consuming the message.
It should be noted that Consumer is message consumer in kafka mechanism, usually read to Kafka broker
The client of message.For consumer, it needs to save the offset of consumption message, preservation for offset and makes
With having consumer to control；When consumer normally consumes message, offset will " linear " drive forwards, that is, disappear
Breath will successively by consumption, in fact random order consumption message be can be used in consumer to sequence, it is only needed offset weight
It is set to arbitrary value.
In the present embodiment, in step 207: judging the message number that the priority of the message and high priority are not consumed
Before, the method also includes: step 213: one preset time of timing after reaching the preset time, calculates high priority and does not disappear
The message number taken；
It should be noted that timing (can be configured in configuration file, unit is minute) calculates Gao You in step 213
The Information Number of the first non-message of grade topic, and call QueueLock method above.
Step 215: judging that the high priority does not consume the size of the quantity of message；
If the quantity that the high priority does not consume message is greater than zero, the method also includes steps 217: keeping described low
It is wait state that priority, which does not consume message,；
If the message number that the high priority is not consumed is equal to zero, the method also includes steps 219: unlocking the height
Low elder generation's grade does not consume message, consumes the low priority and does not consume message.
The message queue processing method based on Goka proposed in the embodiment of the present invention two passes through message in setting queue
Priority is preferentially consumed according to queue priority, and judges the message number that high priority is not consumed simultaneously, to integrate
Judge which message should be by priority processing in queue, and then ensure that in practical application under the excessive scene of more Message Payloads,
Important messages being capable of not quickly consumption in time.The message queue processing method based on Goka in the present embodiment have high efficiency and
Flexibility: priority is set to each topic according to demand, it is ensured that under the premise of not influencing system performance, preferentially consume excellent
The first higher information queue object of grade increases a priority parameter newly to determine topic's.It is allowed when consumption low preferential
Being waited for for grade, after having consumed to high priority, automatically wakes up low priority, it is ensured that can receive at the first time
The information of high priority.
The embodiment of the present invention three provides a kind of message queue processing unit 300 based on Goka, comprising:
Initialization unit 301 is used for initial message queue；
It should be noted that in the embodiment of the present invention, it should be noted that in a kind of preferred mode, the initialization
Message queue specially defines queue structure body topic and initializes.
Preferably a kind of implementation is a fixed queue structure body topic for the initialization of queue in the embodiment of the present invention one,
Priority in structural body determines the priority of each topic, has only done the differentiation of height two-stage at present, has consumed and waiting
When can be according to the value of this field as one of Rule of judgment.
Queue connects structural body and defines:
Newly-built queue simultaneously constructs start method, and the processor in Goka can be called when constructing start method
The Start method of object is reconstructed the Start method of processor herein, and newly-increased parameter (pri int) is for receiving
The priority of topic.
Receiving unit 303, for receiving new information；
Acquiring unit 305: the message number that priority and high priority for obtaining message are not consumed；
First judging unit 307: the message number that priority and high priority for judging the message are not consumed；
First setting unit 309: if being the message that low priority and the high priority are not consumed for the priority
Quantity is zero, and it is wait state that the message, which is arranged,；
Consumer unit 315: if being high priority for the priority or the priority is low priority and described
The message number that high priority is not consumed is zero, then consumes the message.It should be noted that in the present embodiment, the return
Condition is, the data packet serial number adjacent data packet serial number of returned data packet, i.e., the data packet currently judged with
Whether the serial number of the upper data packet returned is adjacent, the upper number that the current data packet is thought if adjacent and has been returned
It is sequence according to packet, i.e., current data packet can be returned and is normally placed on behind a data packet, be integrally formed alphabetic data
Packet, if the data packet that current data packet has been returned with upper one is not connected, what current data packet had been returned with upper one
Data packet is simultaneously non-sequential, and current data packet should not be returned.
It should be noted that Consumer is message consumer in the present embodiment in kafka mechanism, usually to Kafka
The client of broker reading message.For consumer, it needs to save the offset of consumption message, for offset
Preservation and use, have consumer to control；When consumer normally consumes message, offset will " linear " forward
Driving, i.e., message will successively sequence by consumption in fact consumer can be used random order consumption message, it only need by
Offset resets to arbitrary value.
Timing unit 311, for one preset time of timing；
Computing unit 315 calculates the message number that high priority is not consumed after reaching the preset time.
Second judgment unit 313, for judging the size of message number that the high priority is not consumed；
Holding unit 317 is stated low preferential if being greater than zero for the message number that the high priority is not consumed described in holding
It is wait state that grade, which does not consume message,；
Unlocking unit 319 is stated low preferential if being equal to zero for the quantity that the high priority does not consume message described in unlock
Grade does not consume message；
Preferably, consumer unit 315 described in message queue processing unit 300 of the embodiment of the present invention based on Goka, also
Message is not consumed for consuming the low priority.
The message queue processing unit based on Goka proposed in the embodiment of the present invention three, obtains queue by acquiring unit
The priority of middle message is preferentially consumed according to queue priority, and judges the message count that high priority is not consumed simultaneously
Amount, carrying out which message in comprehensive descision queue should be by priority processing, and then ensure that more Message Payload mistakes in practical application
Under more scenes, important messages being capable of not quickly consumption in time.The message queue processing method tool based on Goka in the present embodiment
Standby high efficiency and flexibility: priority is set to each topic according to demand, it is ensured that under the premise of not influencing system performance,
The preferential consumption higher information queue object of priority increases a priority parameter newly to determine topic's.Consumption when
It waits and allows being waited for of low priority, after having consumed to high priority, automatically wake up low priority, it is ensured that can be first
Time receives the information of high priority.
Fig. 4 is the schematic diagram for the computer equipment that an embodiment of the present invention provides.As shown in figure 4, computer equipment 1 to
It less include: processor 20, memory 30, the computer that is stored in the memory 30 and can be run on the processor 20
Program 40 (such as control program of data processing method).
Wherein, the computer equipment 1 can be smart phone, tablet computer, laptop, desktop computer, service
Device computer, personal digital assistant etc. have the computer equipment of shooting function.It will be understood by those skilled in the art that the signal
Fig. 3 is only the example of computer equipment 1, does not constitute the restriction to computer equipment 1, may include more or more than illustrating
Few component perhaps combines certain components or different components, such as the computer equipment 1 can also include that input is defeated
Equipment, network access equipment, bus etc. out.
The processor 20 realizes a kind of above-mentioned message queue processing based on Goka when executing the computer program 40
Step in mode, for example, it is a kind of message queue processing method based on Goka shown in FIG. 1, shown in Fig. 2 based on Goka's
Message queue processing method.The processor 20 is realized each in above-mentioned each device embodiments when executing the computer program 40
The function of each unit in module/unit, such as attached drawing 3.
Illustratively, the computer program 40 can be divided into one or more module/units, it is one or
Multiple module/units are stored in the memory 30, and are executed by the processor 20, to complete the present invention.Described one
A or multiple module/units can be the series of computation machine program instruction section that can complete specific function, and described instruction section is used
In implementation procedure of the description computer program 40 in the terminal 1.The concrete function of each unit refers to the tool of front
Body introduction, for the sake of saving space and avoiding repetition, details are not described herein again.
Alleged processor 20 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor 20 is also possible to any conventional processing
Device etc., the processor 20 are the control of computer equipment described in a kind of message queue processing unit 300/ based on Goka
Center utilizes various interfaces and a kind of entire 300/ computer equipment 1 of message queue processing unit based on Goka of connection
Various pieces.
For the memory 30 for storing the computer program 40 and/or module/unit, the processor 20 passes through fortune
Row executes the computer program and/or module/unit being stored in the memory 30, and calls and be stored in the storage
Data in device 30 realize a kind of various functions of 300/ computer equipment 1 of message queue processing unit based on Goka.
The memory 30 can mainly include storing program area and storage data area, wherein storing program area can storage program area, extremely
Application program (such as sound-playing function, image player function etc.) needed for a few function etc.；Storage data area can store
Created data etc. are used according to computer equipment 1.In addition, the memory 30 may include high random access storage
Device can also include nonvolatile memory, such as hard disk, memory, plug-in type hard disk, intelligent memory card (Smart Media
Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash Card), at least one magnetic disk storage
Part, flush memory device or other volatile solid-state parts.
The embodiment of the present invention also provides a kind of computer readable storage medium, is stored thereon with computer program, the meter
A kind of message queue processing method based on Goka described in above embodiment is realized when calculation machine program is executed by processor
The step of.
If a kind of integrated module/unit of message queue 300//computer equipment of processing unit based on Goka
It is realized in the form of SFU software functional unit and when sold or used as an independent product, can store computer-readable at one
It takes in storage medium.Based on this understanding, the present invention realizes all or part of the process in above embodiment method, can also
It is completed with instructing relevant hardware by computer program, the computer program can be stored in one and computer-readable deposit
In storage media, the computer program is when being executed by processor, it can be achieved that the step of above-mentioned each method implementation.Its
In, the computer program includes computer program code, and the computer program code can be source code form, object generation
Code form, executable file or certain intermediate forms etc..The computer readable storage medium may include: can carry it is described
Any entity or device of computer program code, recording medium, USB flash disk, mobile hard disk, magnetic disk, CD, computer storage,
Read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electricity
Carrier signal, telecommunication signal and software distribution medium etc..It should be noted that the content that the computer-readable medium includes
Can according to making laws in jurisdiction and the requirement of patent practice carries out increase and decrease appropriate, such as in certain jurisdictions,
It does not include electric carrier signal and telecommunication signal according to legislation and patent practice, computer-readable medium.
In several specific embodiments provided by the present invention, it should be understood that disclosed terminal and method, it can be with
It realizes by another way.For example, termini embodiment described above is only schematical, for example, the module
Division, only a kind of logical function partition, there may be another division manner in actual implementation.
It, can also be in addition, each functional module in each embodiment of the present invention can integrate in same treatment module
It is that modules physically exist alone, can also be integrated in equal modules with two or more modules.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also realize in the form of hardware adds software function module.
It is obvious to a person skilled in the art that the embodiment of the present invention is not limited to the details of above-mentioned exemplary embodiment,
And without departing substantially from the spirit or essential attributes of the embodiment of the present invention, this hair can be realized in other specific forms
Bright embodiment.Therefore, in all respects, the present embodiments are to be considered as illustrative and not restrictive, this
The range of inventive embodiments is indicated by the appended claims rather than the foregoing description, it is intended that being equal for claim will be fallen in
All changes in the meaning and scope of important document are included in the embodiment of the present invention.It should not be by any attached drawing mark in claim
Note is construed as limiting the claims involved.Furthermore, it is to be understood that one word of " comprising " does not exclude other units or steps, odd number is not excluded for
Plural number.Multiple units, module or the device stated in system, device or terminal claim can also be by the same units, mould
Block or device are implemented through software or hardware.The first, the second equal words are used to indicate names, and are not offered as any specific
Sequence.
Finally it should be noted that embodiment of above is only to illustrate the technical solution of the embodiment of the present invention rather than limits,
Although the embodiment of the present invention is described in detail referring to the above better embodiment, those skilled in the art should
Understand, can modify to the technical solution of the embodiment of the present invention or equivalent replacement should not all be detached from the skill of the embodiment of the present invention
The spirit and scope of art scheme.
Claims (10)
1. a kind of message queue processing method based on goka characterized by comprising
Initial message queue；
Receive new information；
The message number that the priority and high priority for obtaining new information are not consumed；
If the priority is that the message number that low priority and the high priority are not consumed is greater than zero, the new information is set
For wait state；
If the priority is high priority or the priority is the message number that low priority and high priority are not consumed
It is zero, then consumes the message.
2. the message queue processing method based on goka as described in claim 1, it is characterised in that:
The initial message queue specially defines queue structure body topic and initializes.
3. the message queue processing method based on goka as claimed in claim 2, which is characterized in that described to obtain the message
Priority, the priority of every message is specially determined using the Priority in structural body.
4. the message queue processing method as described in any one of claims 1-3 based on goka, which is characterized in that the judgement
Before the message number that the priority and high priority of the message are not consumed, the method also includes:
A periodically preset time,
After reaching the preset time, the message number information that high priority is not consumed is calculated.
5. the message queue processing method based on goka as claimed in claim 4, which is characterized in that the calculating high priority
After the message number that do not consume, the method also includes:
Judge that the high priority does not consume the size of message number；
If the message number that the high priority is not consumed is greater than zero, low priority is stated described in holding and does not consume message to wait shape
State；
If the message number that the high priority is not consumed is equal to zero, low priority is stated described in unlock and does not consume message, consumes institute
It states low priority and does not consume message.
6. a kind of message queue processing unit based on goka characterized by comprising
Initialization unit is used for initial message queue；
Receiving unit, for receiving new information；
Acquiring unit: the message number that priority and high priority for obtaining new information are not consumed；
First judging unit: the message number that priority and high priority for judging the message are not consumed；
First setting unit: if being that the message number that low priority and the high priority are not consumed is big for the priority
Zero, it is wait state that the message, which is arranged,；
Consumer unit: if being high priority for the priority or the priority is low priority and the high priority
The message number that do not consume is zero, then consumes the message.
7. the message queue processing unit as described in any one of claim 6 based on goka, which is characterized in that further include:
Timing unit, for one preset time of timing；
Computing unit calculates the message number that high priority is not consumed after reaching the preset time.
8. the message queue processing unit based on goka as claimed in claim 7, which is characterized in that further include:
Second judgment unit, for judging the size of message number that the high priority is not consumed；
Holding unit states low priority described in holding and does not disappear if being greater than zero for the message number that the high priority is not consumed
Taking message is wait state；
Unlocking unit states low priority described in unlock and does not disappear if being equal to zero for the message number that the high priority is not consumed
Take message；
The consumer unit is also used to consume the low priority and does not consume message.
9. a kind of computer equipment, which is characterized in that the computer equipment includes processor, and the processor is deposited for executing
It realizes when the computer program stored in reservoir such as the message queue processing method of claim 1-5 based on goka any one of
The step of.
10. a kind of computer readable storage medium is stored thereon with computer program (instruction), it is characterised in that: the calculating
It realizes when machine program (instruction) is executed by processor such as the message queue processing side of claim 1-5 based on goka any one of
The step of method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811650061.7A CN109766200A (en) | 2018-12-31 | 2018-12-31 | A kind of message queue processing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811650061.7A CN109766200A (en) | 2018-12-31 | 2018-12-31 | A kind of message queue processing method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109766200A true CN109766200A (en) | 2019-05-17 |
Family
ID=66453159
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811650061.7A Pending CN109766200A (en) | 2018-12-31 | 2018-12-31 | A kind of message queue processing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109766200A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111049761A (en) * | 2019-12-25 | 2020-04-21 | 哈尔滨新中新电子股份有限公司 | Data processing method of wireless water controller communication module |
| CN112256458A (en) * | 2020-11-09 | 2021-01-22 | 北京沃东天骏信息技术有限公司 | Message enqueuing method and device, electronic equipment and computer readable medium |
| CN112449247A (en) * | 2019-08-30 | 2021-03-05 | 海信视像科技股份有限公司 | Data refreshing method and display device |
| CN112468404A (en) * | 2020-11-24 | 2021-03-09 | 苏宁云计算有限公司 | Method, device and system for controlling message traffic |
| CN112787958A (en) * | 2021-01-05 | 2021-05-11 | 北京字跳网络技术有限公司 | Delay message processing method and device |
| CN112905318A (en) * | 2021-02-04 | 2021-06-04 | 方新 | Message priority queue based distributed switching network element calling method and system |
| CN112966005A (en) * | 2021-03-08 | 2021-06-15 | 平安科技（深圳）有限公司 | Timing message sending method and device, computer equipment and storage medium |
| CN113138883A (en) * | 2021-05-14 | 2021-07-20 | 北京字节跳动网络技术有限公司 | Message processing method and device, storage medium and electronic equipment |
| CN113467969A (en) * | 2021-06-22 | 2021-10-01 | 上海星融汽车科技有限公司 | Method for processing message accumulation |
| CN113535426A (en) * | 2021-06-16 | 2021-10-22 | 福建天泉教育科技有限公司 | Message issuing optimization method and server |
| CN113590641A (en) * | 2021-08-06 | 2021-11-02 | 上海金仕达软件科技有限公司 | Distributed processing method and device of automatic bank bond quotation system |
| KR102373562B1 (en) * | 2021-08-03 | 2022-03-10 | 이지케어텍(주) | Method, Apparatus, System and Computer Program for ensuring message processing order |
| CN115211092A (en) * | 2020-03-02 | 2022-10-18 | 深圳市欢太科技有限公司 | Message pulling method and device and computer storage medium |
| CN116048828A (en) * | 2022-05-26 | 2023-05-02 | 荣耀终端有限公司 | Method for processing drawing message, electronic device and storage medium |
| CN117992250A (en) * | 2024-01-08 | 2024-05-07 | 广州市玄武无线科技股份有限公司 | Consumption priority implementation method, system, device and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080028031A1 (en) * | 2006-07-25 | 2008-01-31 | Byron Lewis Bailey | Method and apparatus for managing instant messaging |
| CN101996099A (en) * | 2010-11-17 | 2011-03-30 | 山东中创软件工程股份有限公司 | Method and system for processing information |
-
2018
- 2018-12-31 CN CN201811650061.7A patent/CN109766200A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080028031A1 (en) * | 2006-07-25 | 2008-01-31 | Byron Lewis Bailey | Method and apparatus for managing instant messaging |
| CN101996099A (en) * | 2010-11-17 | 2011-03-30 | 山东中创软件工程股份有限公司 | Method and system for processing information |
Non-Patent Citations (3)
| Title |
|---|
| MIRAN: "Kafka Consumer - topic(s) with higher priority", 《HTTPS://STACKOVERFLOW.COM/QUESTIONS/45139586/KAFKA-CONSUMER-TOPICS-WITH-HIGHER-PRIORITY》 * |
| SKY ETC: "Does Kafka support priority for topic or message", 《HTTPS://STACKOVERFLOW.COM/QUESTIONS/30655361/DOES-KAFKA-SUPPORT-PRIORITY-FOR-TOPIC-OR-MESSAGE》 * |
| UNCLE_Y: "Golang Cond同步机制", 《HTTPS://BLOG.CSDN.NET/WENTYOON/ARTICLE/DETAILS/81174288》 * |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112449247A (en) * | 2019-08-30 | 2021-03-05 | 海信视像科技股份有限公司 | Data refreshing method and display device |
| CN111049761A (en) * | 2019-12-25 | 2020-04-21 | 哈尔滨新中新电子股份有限公司 | Data processing method of wireless water controller communication module |
| CN115211092B (en) * | 2020-03-02 | 2023-12-22 | 深圳市欢太科技有限公司 | Message pulling method and device and computer storage medium |
| CN115211092A (en) * | 2020-03-02 | 2022-10-18 | 深圳市欢太科技有限公司 | Message pulling method and device and computer storage medium |
| CN112256458A (en) * | 2020-11-09 | 2021-01-22 | 北京沃东天骏信息技术有限公司 | Message enqueuing method and device, electronic equipment and computer readable medium |
| CN112468404B (en) * | 2020-11-24 | 2022-10-11 | 深圳市云网万店科技有限公司 | Method, device and system for controlling message traffic |
| CN112468404A (en) * | 2020-11-24 | 2021-03-09 | 苏宁云计算有限公司 | Method, device and system for controlling message traffic |
| CN112787958A (en) * | 2021-01-05 | 2021-05-11 | 北京字跳网络技术有限公司 | Delay message processing method and device |
| CN112905318A (en) * | 2021-02-04 | 2021-06-04 | 方新 | Message priority queue based distributed switching network element calling method and system |
| CN112966005B (en) * | 2021-03-08 | 2023-07-25 | 平安科技（深圳）有限公司 | Timing message sending method, device, computer equipment and storage medium |
| CN112966005A (en) * | 2021-03-08 | 2021-06-15 | 平安科技（深圳）有限公司 | Timing message sending method and device, computer equipment and storage medium |
| CN113138883A (en) * | 2021-05-14 | 2021-07-20 | 北京字节跳动网络技术有限公司 | Message processing method and device, storage medium and electronic equipment |
| CN113138883B (en) * | 2021-05-14 | 2024-02-02 | 北京字节跳动网络技术有限公司 | Message processing method and device, storage medium and electronic equipment |
| CN113535426A (en) * | 2021-06-16 | 2021-10-22 | 福建天泉教育科技有限公司 | Message issuing optimization method and server |
| CN113535426B (en) * | 2021-06-16 | 2023-11-03 | 福建天泉教育科技有限公司 | Message issuing optimization method and server |
| CN113467969A (en) * | 2021-06-22 | 2021-10-01 | 上海星融汽车科技有限公司 | Method for processing message accumulation |
| CN113467969B (en) * | 2021-06-22 | 2024-01-23 | 上海星融汽车科技有限公司 | Method for processing message accumulation |
| KR102373562B1 (en) * | 2021-08-03 | 2022-03-10 | 이지케어텍(주) | Method, Apparatus, System and Computer Program for ensuring message processing order |
| CN113590641A (en) * | 2021-08-06 | 2021-11-02 | 上海金仕达软件科技有限公司 | Distributed processing method and device of automatic bank bond quotation system |
| CN116048828A (en) * | 2022-05-26 | 2023-05-02 | 荣耀终端有限公司 | Method for processing drawing message, electronic device and storage medium |
| CN116048828B (en) * | 2022-05-26 | 2023-10-20 | 荣耀终端有限公司 | Method for processing drawing message, electronic device and storage medium |
| CN117992250A (en) * | 2024-01-08 | 2024-05-07 | 广州市玄武无线科技股份有限公司 | Consumption priority implementation method, system, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109766200A (en) | A kind of message queue processing method, device, equipment and storage medium | |
| CN105337755B (en) | The system of the method and processing business of client/server server and its processing business | |
| US20170192819A1 (en) | Method and electronic device for resource allocation | |
| CN110808922B (en) | Message processing method and device, storage medium and electronic equipment | |
| CN107370667A (en) | Multi-threading parallel process method and apparatus, computer-readable recording medium and storage control | |
| CN111181976B (en) | Method and system for managing rank authority based on message queue, electronic equipment and storage medium | |
| CN108696594A (en) | A kind of the big data traffic load equalization methods and device of market surpervision block chain | |
| CN111506434B (en) | Task processing method and device and computer readable storage medium | |
| CN109800067A (en) | Database connection optimization method, device and relevant device based on cloud monitoring | |
| CN105955807A (en) | System and method for processing task | |
| CN108520471A (en) | It is overlapped community discovery method, device, equipment and storage medium | |
| CN109800261A (en) | Dynamic control method, device and the relevant device of double data library connection pool | |
| CN103577469B (en) | Database connection multiplexing method and apparatus | |
| CN110069533B (en) | Event subscription method and device based on blockchain | |
| CN105554069B (en) | A kind of big data processing distributed cache system and its method | |
| CN110290009A (en) | A kind of data dispatching method, device and computer readable storage medium | |
| CN109597825A (en) | Regulation engine call method, device, equipment and computer readable storage medium | |
| CN103442034B (en) | A kind of stream media service method based on cloud computing technology and system | |
| CN111131512B (en) | Equipment information processing method and device, storage medium and processor | |
| CN111813529B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN108875035A (en) | The date storage method and relevant device of distributed file system | |
| CN109951532B (en) | DPDK-based automatic flow model conversion device | |
| CN110349076A (en) | The processing method and processing device of data | |
| CN106408793B (en) | A kind of Service Component sharing method and system suitable for ATM business | |
| CN115563160A (en) | Data processing method, data processing device, computer equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190517 |
|
| RJ01 | Rejection of invention patent application after publication |