CN110502402A - Message treatment method, equipment and terminal device - Google Patents
Message treatment method, equipment and terminal device Download PDFInfo
- Publication number
- CN110502402A CN110502402A CN201910801083.7A CN201910801083A CN110502402A CN 110502402 A CN110502402 A CN 110502402A CN 201910801083 A CN201910801083 A CN 201910801083A CN 110502402 A CN110502402 A CN 110502402A
- Authority
- CN
- China
- Prior art keywords
- message
- offset
- subregion
- detected
- consumer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
Landscapes
- Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the present invention provides a kind of message treatment method, equipment and terminal device, this method comprises: obtaining the corresponding first consumption offset of subregion to be detected and the first production offset, determine whether subregion to be detected is message heap integrated area according to the first consumption offset and the first production offset, if subregion to be detected is message heap integrated area, new consumer's example is then created so that message in new consumer's instance processes message heap integrated area, it realizes the automatic detection of subregion and realizes the adjust automatically of message handling ability, improve regulated efficiency, without being manually monitored to subregion and manual creation consumer embodiment adjusts message handling ability, reduce cost of labor.
Description
Technical field
The present embodiments relate to computer application technology more particularly to a kind of message treatment methods, equipment and end
End equipment.
Background technique
Kafka is a kind of high distributed post subscription message system handled up, and can substitute traditional message queue and be used for
Decoupling data processing caches untreated message etc., while having higher throughput.Message is number most basic in kafka
According to unit, theme is the logical concept for storing message, is considered as a massage set, and each theme can divide multiple
Subregion, the producer's example PUSH message, consumer's example can also consume the message in subregion into subregion.Work as the producer
When the message of example push is excessive, correspondingly, consumer's example is also required to the message of over-consumption, at consumer's instance data
When managing scarce capacity, the message in subregion can excessively be overstock.
In the prior art, in order to prevent since consumer's instance processes scarce capacity causes the message in subregion excessively long-pending
Pressure, the usually message in related personnel monitored partition are determining the message mistake in subregion with the presence or absence of excessively overstocked phenomenon
When degree is overstock, the new consumer's example of manual creation increases the thread of processing message to handle message in the subregion, to mention
The speed of message in the high disposal subregion.
However, it is found by the inventors that at least there are the following problems in the prior art: due to needing disappearing in artificial monitored partition
Breath needs consumer's example that manual creation is new with the presence or absence of excessively overstocked phenomenon, and when determining that message is excessively overstock
Message handling ability is adjusted, regulated efficiency is low, and high labor cost.
Summary of the invention
The embodiment of the present invention provides a kind of message treatment method, equipment and terminal device, to solve to adjust in the prior art
The problem of message handling ability low efficiency and high labor cost.
In a first aspect, the embodiment of the present invention provides a kind of message treatment method, comprising:
Obtain the corresponding first consumption offset of subregion to be detected and the first production offset；
Determine whether the subregion to be detected is to disappear according to the first consumption offset and the first production offset
The integrated area Xi Dui；
If the subregion to be detected is message heap integrated area, new consumer's example is created so that the new consumer
Message in message heap integrated area described in instance processes.
It is described true according to the first consumption offset and the first production offset in a kind of possible design
Whether the fixed subregion to be detected is message heap integrated area, comprising:
Calculate the first difference of the first production offset and the first consumption offset；
If first difference is greater than or equal to the first preset difference value, it is determined that the subregion to be detected is message heap integral
Area；
If first difference is less than first preset difference value, it is determined that the subregion to be detected is not message heap integral
Area.
In a kind of possible design, new consumer's example includes the processing thread of the first preset quantity；
After the new consumer's example of the creation, further includes:
Obtain the corresponding second consumption offset in message heap integrated area and the second production offset；
Calculate the second difference of the second production offset and the second consumption offset；
Judge whether to adjust the corresponding processing thread of new consumer's example and/or described according to second difference
New consumer's example.
It is described to be judged whether to adjust new consumer's example according to second difference in a kind of possible design
Corresponding processing thread and/or new consumer's example, comprising:
If second difference is greater than or equal to the second preset difference value, by the new corresponding processing of consumer's example
The quantity of thread increases to the second preset quantity.
If second difference is less than second preset difference value, new consumer's example is closed.
In a kind of possible design, the acquisition message heap integrated area it is corresponding second consumption offset it
Before, comprising:
Receive the second consumption offset that new consumer's example is sent by pipeline.
Second aspect, the embodiment of the present invention provide a kind of message processing device, comprising:
Offset obtains module, for obtaining the corresponding first consumption offset of subregion to be detected and the first production offset
Amount；
Subarea detecting module, for according to the first consumption offset and the first production offset determine it is described to
Whether sense partitions are message heap integrated area；
Message processing module creates new consumer's example if being message heap integrated area for the subregion to be detected
So that the message in message heap integrated area described in new consumer's instance processes.
In a kind of possible design, the Subarea detecting module is specifically used for: calculate it is described first production offset with
First difference of the first consumption offset；
If first difference is greater than or equal to the first preset difference value, it is determined that the subregion to be detected is message heap integral
Area；
If first difference is less than first preset difference value, it is determined that the subregion to be detected is not message heap integral
Area.
In a kind of possible design, new consumer's example includes the processing thread of the first preset quantity；
The equipment further includes example adjustment module；
The example adjusts module, for obtaining the message heap integral after the new consumer's example of the creation
The corresponding second consumption offset in area and the second production offset；
Calculate the second difference of the second production offset and the second consumption offset；
Judge whether to adjust the corresponding processing thread of new consumer's example and/or described according to second difference
New consumer's example.
In a kind of possible design, the example adjustment module is specifically used for: if second difference is greater than or equal to
The quantity of the corresponding processing thread of new consumer's example is then increased to the second preset quantity by the second preset difference value.
If second difference is less than second preset difference value, new consumer's example is closed.
In a kind of possible design, the example adjustment module is also used to: in the acquisition message heap integrated area
Before corresponding second consumption offset, the second consumption offset that new consumer's example is sent by pipeline is received.
The third aspect, the embodiment of the present invention provide a kind of terminal device, comprising: at least one processor and memory；
The memory stores computer executed instructions；
At least one described processor executes the computer executed instructions of memory storage so that it is described at least one
Processor executes such as the described in any item message treatment methods of first aspect.
Fourth aspect, the embodiment of the present invention provide a kind of computer readable storage medium, the computer-readable storage medium
It is stored with computer executed instructions in matter, when processor executes the computer executed instructions, realizes as first aspect is any
Message treatment method described in.
The embodiment of the present invention provides a kind of message treatment method, equipment and terminal device, and this method is according to subregion to be detected
Corresponding first consumption offset and the first production offset determine whether the subregion to be detected is message heap integrated area, that is, determine
The subregion to be detected then automatically creates new consumer's example if message heap integrated area with the presence or absence of message packing phenomenon, should
New consumer's example is used to handle the message in message heap integrated area, so that the speed for handling the message in the subregion is improved,
It realizes the automatic detection of subregion and realizes the adjust automatically of message handling ability, regulated efficiency is improved, without manually to subregion
It is monitored and manual creation consumer embodiment adjusts message handling ability, reduce cost of labor.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention without any creative labor, may be used also for those of ordinary skill in the art
To obtain other drawings based on these drawings.
Fig. 1 is the configuration diagram one of message handling system provided in an embodiment of the present invention；
Fig. 2 is the flow chart one of message treatment method provided in an embodiment of the present invention；
Fig. 3 is the configuration diagram two of message handling system provided in an embodiment of the present invention；
Fig. 4 is the flowchart 2 of message treatment method provided in an embodiment of the present invention；
Fig. 5 is the configuration diagram three of message handling system provided in an embodiment of the present invention；
Fig. 6 is the structural schematic diagram one of consumption processing equipment provided in an embodiment of the present invention；
Fig. 7 is the structural schematic diagram two of consumption processing equipment provided in an embodiment of the present invention；
Fig. 8 is the hardware structural diagram of terminal device provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
Description and claims of this specification and term " first ", " second ", " third " " in above-mentioned attached drawing
The (if present)s such as four " are to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should manage
The data that solution uses in this way are interchangeable under appropriate circumstances, so that the embodiment of the present invention described herein for example can be to remove
Sequence other than those of illustrating or describe herein is implemented.In addition, term " includes " and " having " and theirs is any
Deformation, it is intended that cover it is non-exclusive include, for example, containing the process, method of a series of steps or units, system, production
Product or equipment those of are not necessarily limited to be clearly listed step or unit, but may include be not clearly listed or for this
A little process, methods, the other step or units of product or equipment inherently.
Fig. 1 is the configuration diagram of message handling system provided in an embodiment of the present invention.As shown in Figure 1, producer's cluster
In include 4 producer's examples, corresponding 4 subregions of theme 1 are subregion 0, subregion 1, subregion 2 and subregion 3, consumer's collection respectively
Group includes consumer's example, i.e. consumer's example 0, and consumer's example 0 holds 4 processing threads, and 4 producer's examples are to master
PUSH message in the subregion of topic 1, for consumer's example 0 by the message in processing 4 subregions of thread process, consumer's example can be with
The message in one or more subregions is handled simultaneously, but a subregion is at most by consumer's instance processes, it is real in consumer
During example 0 handles the message in subregion, i.e., during the processing thread process message in consumer's example 0, consumer
Example 0 determines the corresponding production offset of the corresponding each subregion of the theme and consumes offset, and is uploaded to where observer
Terminal device, which is directed to each subregion, using the subregion as subregion to be detected, and according to the corresponding life of subregion to be detected
It produces offset and consumption offset determines that subregion to be detected whether there is message packing phenomenon, if it exists message packing phenomenon, then
New consumer's example is created to handle the subregion to be detected there are message packing phenomenon, to improve Message Processing speed, is realized
The automatic dynamic of message handling ability adjusts, and improves the efficiency of adjustment, and reduce cost of labor.
When each message is when being added to subregion, it can be all assigned an offset offset, it is message in this point
Unique number in area guarantees that sequence of the message in subregion, i.e. message in subregion are ordered into storage by offset.It is raw
It produces offset and refers to offset corresponding to most newly added message in subregion, be equivalent to most newly added message in the subregion
Position, consumption offset refer to offset corresponding to newest processed message in subregion, be equivalent to newest processed
Position of the message in the subregion.
Technical solution of the present invention is described in detail with specifically embodiment below.These specific implementations below
Example can be combined with each other, and the same or similar concept or process may be repeated no more in some embodiments.
Fig. 2 is the flow chart one of message treatment method provided in an embodiment of the present invention, and the executing subject of the present embodiment can be with
For the terminal device where observer as shown in Figure 1, as shown in Fig. 2, the method for the present embodiment, may include:
S201, the corresponding first consumption offset of subregion to be detected and the first production offset are obtained.
In the present embodiment, the corresponding consumption offset of subregion to be detected and production offset are obtained, and respectively as the
One consumption offset and the first production offset.
Wherein, the first consumption offset refers to offset corresponding to most newly added message in subregion to be detected, first
Production offset refers to offset corresponding to newest processed message in subregion to be detected.
Wherein, subregion to be detected can be any one subregion in the subregion of storage message.
Optionally, before obtaining the corresponding first consumption offset of subregion to be detected and the first production offset, packet
It includes:
It receives the consumption offset for the initial consumer's example transmission for handling message in subregion to be detected and receives to be checked
It surveys subregion and sends production offset.
In the present embodiment, subregion to be detected will record when saving the message of producer's example push as the message point
The offset matched, the terminal device that then offset can be sent to where observer.Initial consumer's example consumption to
When message in sense partitions, it will record the offset of currently processed message, the offset be then sent to observer institute
In terminal device.
In order to reduce the consumption of Internet resources, subregion to be detected or initial consumer's example are sent to terminal in offset and set
When standby, after specified time being spaced, then by offset corresponding to most newly added message or newest processed message institute
Corresponding offset is sent to terminal device.
Wherein, initial consumer's example can be sent when that will consume offset and be sent to terminal device by pipeline.
S202, determine whether subregion to be detected is message heap integral according to the first consumption offset and the first production offset
Area.
In the present embodiment, being determined according to the corresponding first consumption offset of subregion to be detected with the first production offset should
Subregion to be detected whether there is message packing phenomenon, that is, determine whether the subregion to be detected is message heap integrated area, when to be detected
When subregion is message heap integrated area, indicate that the speed of the message of consumer's processing is less than the producer and pushes into the subregion to be detected
The speed of message.
The subregion to be detected is being determined with the first production offset according to the corresponding first consumption offset of subregion to be detected
When for message heap integrated area, the size of offset directly can be produced with first according to the first consumption offset to determine, it can also
It is determined with producing the difference of offset with first according to the first consumption offset.
Wherein, the difference of offset is produced with first to determine whether subregion to be detected is to disappear according to the first consumption offset
The process of the integrated area Xi Dui includes:
Calculate the first difference of the first production offset with the first consumption offset；
If the first difference is greater than or equal to the first preset difference value, it is determined that subregion to be detected is message heap integrated area；
If the first difference is less than the first preset difference value, it is determined that subregion to be detected is not message heap integrated area.
Wherein, the difference of the first production offset and the first consumption offset is that the first production offset subtracts the first consumption
The difference that offset obtains.
In the present embodiment, when the first difference be greater than or equal to the first preset difference value when, i.e., when first production offset with
When the difference of first consumption offset is excessive, the location of most newly added message and to be detected point are indicated in subregion to be detected
The location of newest processed message is apart from excessive in area, the partitioned storage to be detected in more not processed message,
The speed of initial consumer's instance processes message is unable to satisfy business demand, and there are message packing phenomenons for the subregion to be detected.
When the first difference is greater than or equal to the first preset difference value, i.e., when the first production offset and the first consumption offset
Difference it is smaller when, indicate in subregion to be detected the location of most newly added message with it is newest processed in subregion to be detected
The location of message distance it is smaller, the not processed message in the subregion to be detected is less, and initial consumer's example disappears
The speed of expense message can satisfy demand, and message packing phenomenon is not present in subregion to be detected.
If S203, subregion to be detected are message heap integrated area, new consumer's example is created so that new consumer is real
Message in example processing message heap integrated area.
In the present embodiment, when determining subregion to be detected is message heap integrated area, expression needs to increase new consumer
Example handles the message in the subregion to be detected, to improve the processing capacity of message, reduces the backlog of message, therefore, wound
New consumer's example is built so that new consumer's example consumes the message in the subregion to be detected.Wherein, new consumer is real
Example includes the processing thread of the first preset quantity.
In the present embodiment, when consumer's instance processes message, really its internal processing thread is handling message,
Therefore, when creating new consumer's example, it is thus necessary to determine that the quantity for the processing thread that new consumer's example includes.
Wherein, when creating new consumer's example, it can use association's journey technology, create lightweight consumer example.
Above-mentioned application scenarios are accepted, as shown in figure 3, initial consumer's example is consumer's example 0, when determining that subregion 2 is
When message heap integrated area, new consumer's example 1 is created to handle the message in subregion 2, which includes at 4
Lineation journey, since a subregion is at most consumed by consumer's example, after consumer's example 1 handles subregion 2, disappear
Expense person's example 0 does not just continue to processing subregion 2.
As can be seen from the above description, being determined according to the corresponding first consumption offset of subregion to be detected and the first production offset
Whether the subregion to be detected is message heap integrated area, that is, determines the subregion to be detected with the presence or absence of message packing phenomenon, if disappearing
The integrated area Xi Dui then automatically creates new consumer's example, which is used to handle in message heap integrated area
Message realizes automatic monitoring and the realization message handling ability of subregion to improve the speed for handling the message in the subregion
Adjust automatically, regulated efficiency is improved, without being manually monitored to subregion and manual creation consumer embodiment adjusts
Message handling ability reduces cost of labor.
The message in the subregion to be detected there are message packing phenomenon is handled in order to avoid creating new consumer's example
When, the speed of newly created consumer's instance processes message is not able to satisfy current business demand still, leads to the subregion to be detected still
There are message packing phenomenons, after creating new consumer's example to handle the message in message heap integrated area, continue to determine and be somebody's turn to do
Whether there are still message packing phenomenons for message heap integrated area, if needing the consumer new to this there are still message packing phenomenon
Example carries out secondary adjustment, carries out secondary adjustment process to new consumer's example below with reference to a specific embodiment and carries out
Detailed description.
Fig. 4 is the flowchart 2 of message treatment method provided in an embodiment of the present invention, base of the present embodiment in Fig. 1 embodiment
On plinth, the specific implementation process for carrying out secondary adjustment to new consumer's example is described in detail, as shown in figure 4, the party
Method includes:
S401, the corresponding first consumption offset of subregion to be detected and the first production offset are obtained.
S402, determine whether subregion to be detected is message heap integral according to the first consumption offset and the first production offset
Area.
If S403, subregion to be detected are message heap integrated area, new consumer's example is created so that new consumer is real
Message in example processing message heap integrated area.
The specific embodiment of the S401 to S403 of the present embodiment, in above-described embodiment S101 and S103 it is similar, this
Place repeats no more.
S404, the corresponding second consumption offset in message heap integrated area and the second production offset are obtained.
In the present embodiment, it handles in the subregion to be detected for message heap integrated area creating new consumer's example
After message, behind interval preset time (for example, 5 minutes), continue to obtain the corresponding message offset amount of the subregion to be detected and production
Offset, and respectively as second message offset and the second production offset.
Optionally, before obtaining the corresponding second consumption offset in message heap integrated area, further includes:
Receive the second consumption offset that new consumer's example is sent by pipeline.
Wherein, the acquisition process and above-mentioned acquisition first message offset of second message offset and the second production offset
Similar with the first production process of offset, details are not described herein again.
S405, the second difference for calculating the second production offset and the second consumption offset.
In the present embodiment, it is calculated as corresponding to the subregion to be detected of message heap integrated area the second production offset and the
The difference of two consumption offsets, obtains the second difference.
Wherein, the difference of the second production offset and the second consumption offset is that the second production offset subtracts the second consumption
The difference that offset obtains.
S406, judge whether to adjust the corresponding processing thread of new consumer's example according to the second difference and/or new disappear
The person's of expense example.
In the present embodiment, the second production offset and the according to corresponding to the subregion to be detected for message heap integrated area
Whether there are still message packing phenomenons to determine the subregion to be detected for the difference of two consumption offsets, so that dynamic adjusts new disappear
The person's of expense example realizes the secondary adjustment of newly created consumer's example.
Optionally, judge whether to adjust the corresponding processing thread of new consumer's example and/or new according to the second difference
Consumer's example, comprising:
If the second difference is greater than or equal to the second preset difference value, by the number of the new corresponding processing thread of consumer's example
Amount increases to the second preset quantity.
If the second difference closes new consumer's example less than the second preset difference value.
Wherein, the second preset quantity is greater than the first preset quantity.
When the second difference is greater than or equal to the second preset difference value, in the subregion to be detected that increases new consumer's example
There are still message packing phenomenons, even if indicating to increase new consumer's example to handle the message in the subregion, the subregion
The speed of consumer's example consumption message is still unable to meet demand, i.e. consuming capacity is still insufficient, it is still necessary to improve the subregion pair
Therefore the consuming capacity answered increases the corresponding processing thread of new consumer's example, i.e., by the processing line of new consumer's example
The quantity of journey increases to the second preset quantity by the first preset quantity, so as to can have more processing threads to handle the subregion
Interior message improves the processing capacity of new consumer's example, that is, improves the corresponding consuming capacity of the subregion, realizes that consumer is real
The dynamic adjustment of the quantity of processing thread inside example.
Wherein, when increasing the quantity of the corresponding processing thread of new consumer's example, observer sends line by pipeline
Cheng Zengjia is requested to new consumer's example, and it includes subject thread quantity which, which increases request, which exists
After receiving thread increase request, the quantity for the processing thread that itself is held increases to subject thread quantity.
Above-mentioned application scenarios are accepted, as shown in figure 5, when determining that subregion 2 there are still when message packing phenomenon, need to improve new
Consumer's example 1 handles the ability of message, i.e., need to increase the quantity of the processing thread of consumer's example 1, by consumer's example 1
The quantity of processing thread increases to 6 by 4, so as to have more processing threads to handle the message in subregion 2, it is real to improve consumer
The consumption message capability of example 1.
When the second difference is less than the second preset difference value, indicate that this is that the subregion to be detected of message heap integrated area no longer exists
Message packing phenomenon, the i.e. subregion to be detected are no longer message heap integrated area, and producer's example is to subregion PUSH message to be detected
Speed lower, in order to reduce the waste of resource, new consumer's example can be closed, continued with by initial consumer's example
Message in the subregion to be detected.
After due to closing new consumer's example, disappearing in initial consumer's instance processes subregion to be detected
The speed of breath is still unable to satisfy demand, which occurs message packing phenomenon again, when the second difference is default less than second
When difference, further determine that whether second difference is greater than third preset difference value, if more than third preset difference value, i.e. the second difference
When less than the second preset difference value and being greater than third preset difference value, the corresponding ability and life for consuming message of the subregion to be detected is indicated
The ability matching of message is produced, i.e., the speed of new consumer's instance processes message disappears with producer's example to subregion to be detected push
The speeds match of breath, without being adjusted to new consumer's example.
If the second difference is less than third preset difference value, indicate that the speed of new consumer's instance processes message is greater than the producer
The speed of example PUSH message into subregion to be detected can close new consumer's example to reduce the consumption of resource, by
Raw consumer person's example continues with the message in the subregion to be detected.
In the present embodiment, after determining subregion to be detected is message heap integrated area, new for the partition creating to be detected
Consumer's example handles the message in the subregion to be detected, is then deviated again according to the second production in the subregion to be detected
The difference of amount and the second consumption offset determines the subregion to be detected, and whether there are still message packing phenomenons, that is, determine new consumption
Whether the speed of person's instance processes message can be with demand, if being unable to meet demand, and to this, new consumer's example is held
Processing thread is adjusted, and to improve the speed of new consumer's instance processes message, reduces the accumulating amount of message, if can be with
Meet demand then closes new consumer's example, reduces the consumption of resource, and it is corresponding can dynamically to adjust subregion to be detected in time
Consumer's embodiment consuming capacity, without being manually adjusted.
Fig. 6 is the structural schematic diagram one of message processing device provided in an embodiment of the present invention, as shown in fig. 6, the present embodiment
The message processing device 600 of offer may include: that offset obtains module 601, Subarea detecting module 602 and Message Processing mould
Block 603.
Wherein, offset obtains module 601, for obtaining the corresponding first consumption offset and first of subregion to be detected
Produce offset.
Subarea detecting module 602, for determining subregion to be detected according to the first consumption offset and the first production offset
It whether is message heap integrated area.
Message processing module 603, if for subregion to be detected be message heap integrated area, create new consumer's example with
Make the message in new consumer's instance processes message heap integrated area.
In a kind of possible design, Subarea detecting module is specifically used for: calculating the first production offset and the first consumption
First difference of offset.
If the first difference is greater than or equal to the first preset difference value, it is determined that subregion to be detected is message heap integrated area.
If the first difference is less than the first preset difference value, it is determined that subregion to be detected is not message heap integrated area.
The message treatment method of Fig. 2 embodiment may be implemented in message processing device provided in an embodiment of the present invention, realizes
Principle is similar with technical effect, and details are not described herein again.
Fig. 7 is the structural schematic diagram two of message processing device provided in an embodiment of the present invention, and new consumer's example includes
The processing thread of first preset quantity, as shown in fig. 7, message processing device provided in this embodiment 600, can also include: reality
Example adjustment module 604.
Wherein, example adjusts module 604, for obtaining message heap integrated area pair after creating new consumer's example
The the second consumption offset and the second production offset answered.
Calculate the second difference of the second production offset and the second consumption offset.
Judge whether to adjust the corresponding processing thread of new consumer's example according to the second difference and/or new consumer is real
Example.
In a kind of possible design, example adjustment module is specifically used for: presetting if the second difference is greater than or equal to second
The quantity of the new corresponding processing thread of consumer's example is then increased to the second preset quantity by difference.
If the second difference closes new consumer's example less than the second preset difference value.
In a kind of possible design, example adjustment module is also used to: being disappeared obtaining message heap integrated area corresponding second
Before taking offset, the second consumption offset that new consumer's example is sent by pipeline is received.
The message treatment method of Fig. 4 embodiment may be implemented in message processing device provided in an embodiment of the present invention, realizes
Principle is similar with technical effect, and details are not described herein again.
Fig. 8 is the hardware structural diagram of terminal device provided in an embodiment of the present invention.As shown in fig. 6, the present embodiment mentions
The terminal device 800 of confession includes: at least one processor 801 and memory 802.Wherein, processor 801, memory 802 pass through
Bus 803 connects.
During specific implementation, at least one processor 801 executes the computer execution that the memory 802 stores and refers to
It enables, so that at least one processor 801 executes the message treatment method in above method embodiment.
The specific implementation process of processor 801 can be found in above method embodiment, and it is similar that the realization principle and technical effect are similar,
Details are not described herein again for the present embodiment.
In above-mentioned embodiment shown in fig. 5, it should be appreciated that processor can be central processing unit (English:
Central Processing Unit, referred to as: CPU), can also be other general processors, digital signal processor (English:
Digital Signal Processor, referred to as: DSP), specific integrated circuit (English: Application Specific
Integrated Circuit, referred to as: ASIC) etc..General processor can be microprocessor or the processor is also possible to
Any conventional processor etc..Hardware processor can be embodied directly in conjunction with the step of invention disclosed method to have executed
At, or in processor hardware and software module combination execute completion.
Memory may include high speed RAM memory, it is also possible to and it further include non-volatile memories NVM, for example, at least one
Magnetic disk storage.
Bus can be industry standard architecture (Industry Standard Architecture, ISA) bus, outer
Portion's apparatus interconnection (Peripheral Component, PCI) bus or extended industry-standard architecture (Extended
Industry Standard Architecture, EISA) bus etc..Bus can be divided into address bus, data/address bus, control
Bus etc..For convenient for indicating, the bus in illustrations does not limit only a bus or a type of bus.
The embodiment of the present invention also provides a kind of computer readable storage medium, stores in the computer readable storage medium
There are computer executed instructions, when processor executes the computer executed instructions, realizes at the message of above method embodiment
Reason method.
Above-mentioned computer readable storage medium, above-mentioned readable storage medium storing program for executing can be by any kind of volatibility or non-
Volatile storage devices or their combination realize that, such as static random access memory (SRAM), electrically erasable is only
It reads memory (EEPROM), Erasable Programmable Read Only Memory EPROM (EPROM), programmable read only memory (PROM) is read-only to deposit
Reservoir (ROM), magnetic memory, flash memory, disk or CD.Readable storage medium storing program for executing can be general or specialized computer capacity
Any usable medium enough accessed.
A kind of illustrative readable storage medium storing program for executing is coupled to processor, to enable a processor to from the readable storage medium storing program for executing
Information is read, and information can be written to the readable storage medium storing program for executing.Certainly, readable storage medium storing program for executing is also possible to the composition portion of processor
Point.Processor and readable storage medium storing program for executing can be located at specific integrated circuit (Application Specific Integrated
Circuits, referred to as: ASIC) in.Certainly, processor and readable storage medium storing program for executing can also be used as discrete assembly and be present in equipment
In.
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above-mentioned each method embodiment can lead to
The relevant hardware of program instruction is crossed to complete.Program above-mentioned can be stored in a computer readable storage medium.The journey
When being executed, execution includes the steps that above-mentioned each method embodiment to sequence；And storage medium above-mentioned include: ROM, RAM, magnetic disk or
The various media that can store program code such as person's CD.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme.
Claims (10)
1. a kind of message treatment method characterized by comprising
Obtain the corresponding first consumption offset of subregion to be detected and the first production offset；
Determine whether the subregion to be detected is message heap according to the first consumption offset and the first production offset
Integrated area；
If the subregion to be detected is message heap integrated area, new consumer's example is created so that new consumer's example
Handle the message in message heap integrated area.
2. the method according to claim 1, wherein described according to the first consumption offset and described the
One production offset determines whether the subregion to be detected is message heap integrated area, comprising:
Calculate the first difference of the first production offset and the first consumption offset；
If first difference is greater than or equal to the first preset difference value, it is determined that the subregion to be detected is message heap integrated area；
If first difference is less than first preset difference value, it is determined that the subregion to be detected is not message heap integrated area.
3. the method according to claim 1, wherein new consumer's example includes the first preset quantity
Handle thread；
After the new consumer's example of the creation, further includes:
Obtain the corresponding second consumption offset in message heap integrated area and the second production offset；
Calculate the second difference of the second production offset and the second consumption offset；
Judge whether to adjust the corresponding processing thread of new consumer's example and/or described new according to second difference
Consumer's example.
4. according to the method described in claim 3, it is characterized in that, described judge whether described in adjustment according to second difference
The corresponding processing thread of new consumer's example and/or new consumer's example, comprising:
If second difference is greater than or equal to the second preset difference value, by the corresponding processing thread of new consumer's example
Quantity increase to the second preset quantity；
If second difference is less than second preset difference value, new consumer's example is closed.
5. according to the method described in claim 3, it is characterized in that, in the acquisition message heap integrated area corresponding second
Before consumption offset, comprising:
Receive the second consumption offset that new consumer's example is sent by pipeline.
6. a kind of message processing device characterized by comprising
Offset obtains module, for obtaining the corresponding first consumption offset of subregion to be detected and the first production offset；
Subarea detecting module, it is described to be detected for being determined according to the first consumption offset and the first production offset
Whether subregion is message heap integrated area；
Message processing module, if for the subregion to be detected be message heap integrated area, create new consumer's example so that
Message in message heap integrated area described in new consumer's instance processes.
7. equipment according to claim 6, which is characterized in that the Subarea detecting module is specifically used for: calculating described the
First difference of one production offset and the first consumption offset；
If first difference is greater than or equal to the first preset difference value, it is determined that the subregion to be detected is message heap integrated area；
If first difference is less than first preset difference value, it is determined that the subregion to be detected is not message heap integrated area.
8. equipment according to claim 6, which is characterized in that new consumer's example includes the first preset quantity
Handle thread；
The equipment further includes example adjustment module；
The example adjusts module, for obtaining message heap integrated area pair after the new consumer's example of the creation
The the second consumption offset and the second production offset answered；
Calculate the second difference of the second production offset and the second consumption offset；
Judge whether to adjust the corresponding processing thread of new consumer's example and/or described new according to second difference
Consumer's example.
9. a kind of terminal device characterized by comprising at least one processor and memory；
The memory stores computer executed instructions；
At least one described processor executes the computer executed instructions of the memory storage, so that at least one described processing
Device executes such as message treatment method described in any one of claim 1 to 5.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer in the computer readable storage medium
It executes instruction, when processor executes the computer executed instructions, realizes such as message described in any one of claim 1 to 5
Processing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910801083.7A CN110502402B (en) | 2019-08-28 | 2019-08-28 | Message processing method and device and terminal device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910801083.7A CN110502402B (en) | 2019-08-28 | 2019-08-28 | Message processing method and device and terminal device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110502402A true CN110502402A (en) | 2019-11-26 |
| CN110502402B CN110502402B (en) | 2023-03-21 |
Family
ID=68588491
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910801083.7A Active CN110502402B (en) | 2019-08-28 | 2019-08-28 | Message processing method and device and terminal device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110502402B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111124710A (en) * | 2019-12-17 | 2020-05-08 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN112131005A (en) * | 2020-09-25 | 2020-12-25 | 新华三大数据技术有限公司 | Resource adjustment strategy determination method and device |
| CN112612606A (en) * | 2020-12-16 | 2021-04-06 | 平安消费金融有限公司 | Message theme processing method and device, computer equipment and readable storage medium |
| CN113535368A (en) * | 2021-09-07 | 2021-10-22 | 武汉中科通达高新技术股份有限公司 | Task processing method and device and electronic equipment |
| CN113872888A (en) * | 2021-09-27 | 2021-12-31 | 南威软件股份有限公司 | Method, system and medium for Kafka backlog queue pursuit |
| CN114827049A (en) * | 2022-03-02 | 2022-07-29 | 厦门服云信息科技有限公司 | Accumulated data consumption method based on kafka, terminal equipment and storage medium |
| CN115211092A (en) * | 2020-03-02 | 2022-10-18 | 深圳市欢太科技有限公司 | Message pulling method and device and computer storage medium |
| CN115567477A (en) * | 2022-09-19 | 2023-01-03 | 上海星起源新能源科技有限公司 | Method, equipment and storage medium for processing message accumulation |
| CN115567477B (en) * | 2022-09-19 | 2024-05-31 | 上海星起源新能源科技有限公司 | Method, equipment and storage medium for processing message accumulation |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101599027A (en) * | 2009-06-30 | 2009-12-09 | 中兴通讯股份有限公司 | A kind of thread pool management method and system thereof |
| CN105786909A (en) * | 2014-12-25 | 2016-07-20 | 北京东方通科技股份有限公司 | Message queue backlog load self-adaptive application triggering method and system |
| US20170272516A1 (en) * | 2016-03-17 | 2017-09-21 | International Business Machines Corporation | Providing queueing in a log streaming messaging system |
| CN107291585A (en) * | 2016-03-31 | 2017-10-24 | 阿里巴巴集团控股有限公司 | Disaster recovery method and device, server |
| CN107509119A (en) * | 2017-07-11 | 2017-12-22 | 北京潘达互娱科技有限公司 | A kind of monitoring alarm method and device |
| CN108848039A (en) * | 2018-04-24 | 2018-11-20 | 平安科技（深圳）有限公司 | The method and storage medium that server, message are distributed |
-
2019
- 2019-08-28 CN CN201910801083.7A patent/CN110502402B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101599027A (en) * | 2009-06-30 | 2009-12-09 | 中兴通讯股份有限公司 | A kind of thread pool management method and system thereof |
| CN105786909A (en) * | 2014-12-25 | 2016-07-20 | 北京东方通科技股份有限公司 | Message queue backlog load self-adaptive application triggering method and system |
| US20170272516A1 (en) * | 2016-03-17 | 2017-09-21 | International Business Machines Corporation | Providing queueing in a log streaming messaging system |
| CN107291585A (en) * | 2016-03-31 | 2017-10-24 | 阿里巴巴集团控股有限公司 | Disaster recovery method and device, server |
| CN107509119A (en) * | 2017-07-11 | 2017-12-22 | 北京潘达互娱科技有限公司 | A kind of monitoring alarm method and device |
| CN108848039A (en) * | 2018-04-24 | 2018-11-20 | 平安科技（深圳）有限公司 | The method and storage medium that server, message are distributed |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111124710A (en) * | 2019-12-17 | 2020-05-08 | 联想(北京)有限公司 | Data processing method and electronic equipment |
| CN115211092A (en) * | 2020-03-02 | 2022-10-18 | 深圳市欢太科技有限公司 | Message pulling method and device and computer storage medium |
| CN115211092B (en) * | 2020-03-02 | 2023-12-22 | 深圳市欢太科技有限公司 | Message pulling method and device and computer storage medium |
| CN112131005A (en) * | 2020-09-25 | 2020-12-25 | 新华三大数据技术有限公司 | Resource adjustment strategy determination method and device |
| CN112131005B (en) * | 2020-09-25 | 2022-09-20 | 新华三大数据技术有限公司 | Resource adjustment strategy determination method and device |
| CN112612606A (en) * | 2020-12-16 | 2021-04-06 | 平安消费金融有限公司 | Message theme processing method and device, computer equipment and readable storage medium |
| CN113535368A (en) * | 2021-09-07 | 2021-10-22 | 武汉中科通达高新技术股份有限公司 | Task processing method and device and electronic equipment |
| CN113872888A (en) * | 2021-09-27 | 2021-12-31 | 南威软件股份有限公司 | Method, system and medium for Kafka backlog queue pursuit |
| CN113872888B (en) * | 2021-09-27 | 2023-05-19 | 南威软件股份有限公司 | Method, system and medium for catch-up of Kafka backlog queue |
| CN114827049A (en) * | 2022-03-02 | 2022-07-29 | 厦门服云信息科技有限公司 | Accumulated data consumption method based on kafka, terminal equipment and storage medium |
| CN115567477A (en) * | 2022-09-19 | 2023-01-03 | 上海星起源新能源科技有限公司 | Method, equipment and storage medium for processing message accumulation |
| CN115567477B (en) * | 2022-09-19 | 2024-05-31 | 上海星起源新能源科技有限公司 | Method, equipment and storage medium for processing message accumulation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110502402B (en) | 2023-03-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110502402A (en) | Message treatment method, equipment and terminal device | |
| CN110457382A (en) | Method for processing business and equipment | |
| CN111427859B (en) | Message processing method and device, electronic equipment and storage medium | |
| CN106095589B (en) | A kind of method, apparatus and system for distributing subregion | |
| CN106598760B (en) | Consumption method and device of message queue | |
| CN110018899B (en) | Method and device for recycling memory | |
| CN109977129A (en) | Multi-stage data caching method and equipment | |
| CN111309644B (en) | Memory allocation method and device and computer readable storage medium | |
| CN109144791A (en) | Data conversion storage method, apparatus and data management server | |
| CN110727517A (en) | Memory allocation method and device based on partition design | |
| CN110737717B (en) | Database migration method and device | |
| US8615642B2 (en) | Automatic page promotion and demotion in multiple page size environments | |
| CN110221914B (en) | File processing method and device | |
| US7900010B2 (en) | System and method for memory allocation management | |
| CN113806102B (en) | Message queue processing method and device and computing equipment | |
| CN111309576A (en) | Pressure testing method and device | |
| CN116225610A (en) | Binding method and device of virtual machine, electronic equipment and scheduling system | |
| CN115391703A (en) | Interface information processing method and device | |
| CN106326143A (en) | Cache distribution, data access and data sending method, processor and system | |
| CN110674170B (en) | Data caching method, device, equipment and medium based on linked list reverse order access | |
| CN113468442A (en) | Resource bit flow distribution method, computing device and computer storage medium | |
| CN110458537B (en) | Method and device for processing cash-out transaction | |
| KR20170037851A (en) | Method and device for operating a many-core-system | |
| CN107038021B (en) | For accessing the methods, devices and systems of random access memory ram | |
| CN112087501A (en) | Transmission method and system for keeping data consistency |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |