CN109815082A - KAFKA theme monitoring method, device, electronic equipment and storage medium - Google Patents
KAFKA theme monitoring method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN109815082A CN109815082A CN201811537835.5A CN201811537835A CN109815082A CN 109815082 A CN109815082 A CN 109815082A CN 201811537835 A CN201811537835 A CN 201811537835A CN 109815082 A CN109815082 A CN 109815082A
- Authority
- CN
- China
- Prior art keywords
- kafka
- topics
- configuration
- current
- theme
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The present invention provides a kind of KAFKA theme monitoring method, device, electronic equipment and storage medium.The KAFKA theme monitoring method includes: to call kafka-topics source code using configuration script when receiving the monitoring instruction of KAFKA theme;Obtain KAFKA subject name;According to the KAFKA subject name, the kafka-topics source code is executed, obtains current KAFKA number of topics;Obtain configuration KAFKA number of topics;Compare the current KAFKA number of topics and the configuration KAFKA number of topics;When the current KAFKA number of topics is less than the configuration KAFKA number of topics, warning message is exported.The invention can avoid there is KAFKA theme deficient phenomena, realizes timely early warning, improve O&M efficiency.
Description
Technical field
The present invention relates to software supervision technical fields more particularly to a kind of KAFKA theme monitoring method, device, electronics to set
Standby and storage medium.
Background technique
KAFKA is that a kind of distributed post of high-throughput subscribes to message system, it can handle the net of consumer's scale
Everything flow data in standing.In KAFKA, the affiliated class of data is divided using a category attribute, divides data
This class, then referred to as topic.If KAFKA is seen as a database, topic be can be understood as in database
One table, the name of topic are table name.
Therefore, if the topic in KAFKA is lacked, the loss of data and confusion of KAFKA will be caused, is influencing KAFKA just
Often work.
Summary of the invention
In view of the foregoing, it is necessary to a kind of KAFKA theme monitoring method, device, electronic equipment and storage medium are provided,
It is avoided that KAFKA theme deficient phenomena occur, realizes timely early warning, improve O&M efficiency.
A kind of KAFKA theme monitoring method, which comprises
When receiving the monitoring instruction of KAFKA theme, kafka-topics source code is called using configuration script;
Obtain KAFKA subject name;
According to the KAFKA subject name, the kafka-topics source code is executed, obtains current KAFKA number of topics;
Obtain configuration KAFKA number of topics;
Compare the current KAFKA number of topics and the configuration KAFKA number of topics;
When the current KAFKA number of topics is less than the configuration KAFKA number of topics, warning message is exported.
Preferred embodiment according to the present invention, the configuration script include kafka-run-class.sh script.
Preferred embodiment according to the present invention, after obtaining current KAFKA number of topics, the method also includes:
The current KAFKA number of topics is saved as into variable, for transferring.
Preferred embodiment according to the present invention, the acquisition configuration KAFKA number of topics include any of the following mode:
The configuration file for obtaining KAFKA, obtains the configuration KAFKA number of topics from the configuration file;Or
The history log for obtaining KAFKA obtains the KAFKA number of topics of last time operation from the history log
As the configuration KAFKA number of topics.
Preferred embodiment according to the present invention, the output warning message include the combination of following one or more kinds of modes:
Alarm mail is sent to designated person;And/or
It dials the police emergency number to the designated person;And/or
Warning message is sent to the designated person;And/or
Issue alarm sound.
Preferred embodiment according to the present invention, the output warning message further include:
Obtain current time and preset alarm text;
Show the current time, the preset alarm text and the current KAFKA number of topics.
Preferred embodiment according to the present invention, the method also includes:
Pressure test is carried out to KAFKA, obtains simulation KAFKA number of topics;
Calculate the difference of the current KAFKA number of topics and the simulation KAFKA number of topics;
When the absolute value of the difference is greater than preset value, sound an alarm.
A kind of KAFKA theme monitoring device, described device include:
Call unit, for calling kafka-topics using configuration script when receiving the monitoring instruction of KAFKA theme
Source code;
Acquiring unit, for obtaining KAFKA subject name;
Execution unit, for executing the kafka-topics source code, being worked as according to the KAFKA subject name
Preceding KAFKA number of topics;
The acquiring unit is also used to obtain configuration KAFKA number of topics;
Comparison unit, for comparing the current KAFKA number of topics and the configuration KAFKA number of topics;
Output unit, for when the current KAFKA number of topics is less than the configuration KAFKA number of topics, output to be alarmed
Information.
Preferred embodiment according to the present invention, the configuration script include kafka-run-class.sh script.
Preferred embodiment according to the present invention, described device further include:
Storage unit, for after obtaining current KAFKA number of topics, the current KAFKA number of topics to be saved as variable,
For transferring.
Preferred embodiment according to the present invention, the acquiring unit obtain configuration KAFKA number of topics and include any of the following
Mode:
The configuration file for obtaining KAFKA, obtains the configuration KAFKA number of topics from the configuration file;Or
The history log for obtaining KAFKA obtains the KAFKA number of topics of last time operation from the history log
As the configuration KAFKA number of topics.
Preferred embodiment according to the present invention, the output unit output warning message include following one or more kinds of modes
Combination:
Alarm mail is sent to designated person;And/or
It dials the police emergency number to the designated person;And/or
Warning message is sent to the designated person;And/or
Issue alarm sound.
Preferred embodiment according to the present invention, the output unit export warning message further include:
Obtain current time and preset alarm text;
Show the current time, the preset alarm text and the current KAFKA number of topics.
Preferred embodiment according to the present invention, described device further include:
Test cell obtains simulation KAFKA number of topics for carrying out pressure test to KAFKA;
Computing unit, for calculating the difference of the current KAFKA number of topics and the simulation KAFKA number of topics;
Issue unit, for sounding an alarm when the absolute value of the difference is greater than preset value.
A kind of electronic equipment, the electronic equipment include:
Memory stores at least one instruction;And
Processor executes the instruction stored in the memory to realize the KAFKA theme monitoring method.
A kind of computer readable storage medium is stored at least one instruction, institute in the computer readable storage medium
At least one instruction is stated to be executed by the processor in electronic equipment to realize the KAFKA theme monitoring method.
As can be seen from the above technical solutions, the present invention can be when receiving the monitoring instruction of KAFKA theme, using configuration
Script calls kafka-topics source code, and combines the KAFKA subject name obtained, executes the source kafka-topics generation
Code further obtains configuration KAFKA number of topics, and work as described in comparison to quickly and accurately obtain current KAFKA number of topics
Preceding KAFKA number of topics and the configuration KAFKA number of topics, when the current KAFKA number of topics is less than the configuration KAFKA theme
It when number, sounds an alarm, to avoid the occurrence of KAFKA theme deficient phenomena, realizes timely early warning, improve O&M efficiency.
Detailed description of the invention
Fig. 1 is the flow chart of the preferred embodiment of KAFKA theme monitoring method of the present invention.
Fig. 2 is the functional block diagram of the preferred embodiment of KAFKA theme monitoring device of the present invention.
Fig. 3 is the structural schematic diagram of the electronic equipment for the preferred embodiment that the present invention realizes KAFKA theme monitoring method.
Main element symbol description
Electronic equipment | 1 |
Memory | 12 |
Processor | 13 |
KAFKA theme monitoring device | 11 |
Call unit | 110 |
Acquiring unit | 111 |
Execution unit | 112 |
Comparison unit | 113 |
Output unit | 114 |
Storage unit | 115 |
Test cell | 116 |
Computing unit | 117 |
Issue unit | 118 |
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, right in the following with reference to the drawings and specific embodiments
The present invention is described in detail.
As shown in Figure 1, being the flow chart of the preferred embodiment of KAFKA theme monitoring method of the present invention.According to different need
It asks, the sequence of step can change in the flow chart, and certain steps can be omitted.
The KAFKA theme monitoring method is applied in one or more electronic equipment, and the electronic equipment is a kind of
Can be according to the instruction for being previously set or storing, the automatic equipment for carrying out numerical value calculating and/or information processing, hardware include but
Be not limited to microprocessor, specific integrated circuit (Application Specific Integrated Circuit, ASIC), can
Program gate array (Field-Programmable Gate Array, FPGA), digital processing unit (Digital Signal
Processor, DSP), embedded device etc..
The electronic equipment can be any electronic product that human-computer interaction can be carried out with user, for example, personal meter
Calculation machine, tablet computer, smart phone, personal digital assistant (PersonalDigital Assistant, PDA), game machine, friendship
Mutual formula Web TV (Internet Protocol Television, IPTV), intellectual wearable device etc..
The electronic equipment can also include the network equipment and/or user equipment.Wherein, the network equipment includes, but
It is not limited to single network server, the server group of multiple network servers composition or based on cloud computing (Cloud
Computing the cloud being made of a large amount of hosts or network server).
Network locating for the electronic equipment include but is not limited to internet, wide area network, Metropolitan Area Network (MAN), local area network, it is virtual specially
With network (Virtual Private Network, VPN) etc..
S10, when receiving the monitoring instruction of KAFKA theme, the electronic equipment calls kafka- using configuration script
Topics source code.
In at least one embodiment of the present invention, KAFKA is that a kind of distributed post of high-throughput subscribes to message system
System, it can handle the everything flow data in the website of consumer's scale.In KAFKA, come using a category attribute
The affiliated class for dividing data, divides this class of data, then referred to as topic theme.If KAFKA is seen as a data
Library, then topic theme can be understood as a table in database, and topic subject name is table name.
Therefore, if the topic in KAFKA is lacked, the loss of data and confusion of KAFKA will be caused, is influencing KAFKA just
Often work.
In at least one embodiment of the present invention, the electronic equipment, which receives the monitoring of KAFKA theme and instructs, includes, but not
It is limited to the combination of following one or more:
(1) the KAFKA theme that the electronic equipment receives user's triggering monitors instruction.
Specifically, the user can trigger the KAFKA theme monitoring instruction, the triggering by clicking trigger key
Key can be virtual key or physical button etc..
The user can also input corresponding phonetic order, be instructed using monitoring as the KAFKA theme, the present invention is not
Limitation.
Either, the user can also input finger print information, to trigger described KAFKA theme monitoring instruction etc..
(2) signal that the electronic equipment triggers the KAFKA theme monitoring instruction every prefixed time interval is received.
Specifically, the prefixed time interval can be configured by the electronic equipment, can also by the user into
Row custom-configures, and the present invention does not limit.
Such as: the prefixed time interval can be every 1 minute etc..
(3) receive the electronic equipment configuration triggers the signal of the KAFKA theme monitoring instruction in preset time.
Specifically, the preset time can be configured by the electronic equipment, can also be carried out certainly by the user
Definition configuration, the present invention do not limit.
Such as: the preset time can be 10 points of the morning of December 1 etc..
(4) the KAFKA theme monitoring instruction that the electronic equipment triggers under configuration status is received.
Specifically, the configuration status includes, but are not limited to the state etc. of loss of data in KAFKA.
In at least one embodiment of the present invention, the electronic equipment is receiving the KAFKA theme monitoring instruction
Afterwards, the identity of user can also be authenticated, such as: the electronic equipment can be by the identity information of the user (as used
Name in an account book, password etc.) it is matched with the identity information of storage, wherein user corresponding to the identity information of the storage has
Supervision authority.
Preferably, the configuration script includes kafka-run-class.sh script.
Specifically, the kafka-run-class.sh script is joined for calling TopicCommand class, and according to corresponding
Number executes specified function, for example, in the present embodiment, the kafka-run-class.sh script is described for calling
Kafka-topics source code, to get current KAFKA number of topics.
Further, the electronic equipment calls institute by the configuration script kafka-run-class.sh script
It states kafka-topics source code, and by executing the kafka-topics source code, further gets and be currently running
KAFKA number of topics, i.e., the described current KAFKA number of topics.
S11, the electronic equipment obtain KAFKA subject name.
In at least one embodiment of the present invention, according to the KAFKA subject name, the electronic equipment is available
To the topic theme corresponding to the KAFKA subject name, and further count the current KAFKA number of topics.
Specifically, the electronic equipment acquisition KAFKA subject name includes, but are not limited to any one following mode:
(1) electronic equipment obtains the configuration file of KAFKA, and the KAFKA theme is obtained from the configuration file
Title.
Specifically, when installing KAFKA, while the configuration file of KAFKA is saved.
It specifically, include KAFKA cluster name, the performance parameter of KAFKA, configuration KAFKA theme in the configuration file
Number, KAFKA subject name etc..
(2) electronic equipment obtains the history log of KAFKA, and last time fortune is obtained from the history log
Capable subject name is as the KAFKA subject name.
By above embodiment, it can more meet the currently used demand of user.
S12, the electronic equipment execute the kafka-topics source code, obtain according to the KAFKA subject name
Current KAFKA number of topics.
In at least one embodiment of the present invention, the electronic equipment is according to the KAFKA subject name, and combines
The parameters such as zookeeperIP (zookeeper Internet Protocol), port numbers, execute the source kafka-topics
Code.
Such as: the electronic equipment executes the kafka-topics source code/wls/elis_ after filling parameter
racs_kafka/bin/kafka-topics.sh--list--zookeeper30.16.4.197:2181,30.16.4.208:
2181,30.16.4.219:2181/kafka|egrep"T-HADOOP-MSG-R2P1|T-SKYNET-DATA-R1P6|T-
SKYNET-DATA-R1P6-TAG | T-SKYNET-DATA-TAG-R1P6 | _ _ consumer_offsets "-c obtains described work as
Preceding KAFKA number of topics.Wherein, the zookeeperIP is 30.16.4.197, and the port numbers are 2181, the KAFKA master
Autograph is known as T-HADOOP-MSG-R2P1, T-SKYNET-DATA-R1P6 etc..
Preferably, after obtaining current KAFKA number of topics, the method also includes:
The current KAFKA number of topics is saved as variable by the electronic equipment, for transferring.
Under normal conditions, the current KAFKA number of topics can be stored in the running log of the KAFKA, but be worked as
When the running log runs up to certain volume, the speed of service of the KAFKA will seriously affect, therefore, in the KAFKA
Running log is periodically executed delete operation.
It is directed to above situation, in order to avoid subsequent the case where can not getting current KAFKA number of topics generation, institute
It states electronic equipment and the current KAFKA number of topics is saved as into variable, transferred at any time with facilitating, and have more flexibility.
S13, the electronic equipment obtain configuration KAFKA number of topics.
In at least one embodiment of the present invention, the configuration KAFKA number of topics is the master that the KAFKA should have
Inscribe number.
Specifically, the acquisition configuration KAFKA number of topics includes, but are not limited to any one following mode:
(1) electronic equipment obtains the configuration file of KAFKA, and the configuration KAFKA is obtained from the configuration file
Number of topics.
Specifically, when installing KAFKA, while the configuration file of KAFKA is saved.
It wherein, include KAFKA cluster name, the performance parameter of KAFKA, configuration KAFKA number of topics in the configuration file
Deng.
(2) electronic equipment obtains the history log of KAFKA, and last time fortune is obtained from the history log
Capable KAFKA number of topics is as the configuration KAFKA number of topics.
Specifically, the KAFKA number of topics that the electronic equipment runs last time is as the configuration KAFKA number of topics, energy
It is enough more to meet current monitoring demand.
S14, the electronic equipment compare the current KAFKA number of topics and the configuration KAFKA number of topics.
In at least one embodiment of the present invention, described current when deficient phenomena is not present in the KAFKA theme
KAFKA number of topics should be consistent with the configuration KAFKA number of topics.
Therefore, in the present embodiment, the electronic equipment is by the current KAFKA number of topics and the configuration KAFKA master
Topic number compares, so that it is determined that whether the current KAFKA number of topics is normal.
S15, when the current KAFKA number of topics is less than the configuration KAFKA number of topics, the electronic equipment output report
Alert information.
In at least one embodiment of the present invention, when the current KAFKA number of topics is equal to the configuration KAFKA theme
When number, the electronic equipment can determine that the current KAFKA number of topics is normal, and the electronic equipment is without alarm.
And when the current KAFKA number of topics is less than the configuration KAFKA number of topics, described in the electronic equipment determination
KAFKA theme missing, the electronic equipment export warning message.
Specifically, the output warning message includes, but are not limited to the combination of following one or more kinds of modes:
(1) electronic equipment sends alarm mail to designated person.
(2) electronic equipment is dialed the police emergency number to the designated person.
(3) electronic equipment sends warning message to the designated person.
(4) alarm sound is issued.
Specifically, the designated person includes, but are not limited to the combination of following one or more: operation maintenance personnel, project
Manager, developer etc..
By above embodiment, the electronic equipment can be realized timely early warning, and by configuring different alarm sides
Formula effectively improves alarm efficiency.
Further, the electronic equipment exports warning message further include:
The electronic equipment obtains current time and preset alarm text, shows the current time, the preset alarm
Text and the current KAFKA number of topics.
Specifically, the preset alarm text can be custom-configured by user, and the present invention does not limit, for example, institute
Stating preset alarm text may include " kafka topic is less than 5 " etc..
By above embodiment, the electronic equipment can show simultaneously event occur time and details (such as:
Described current KAFKA number of topics etc.), so that related personnel is become more apparent upon warning reason, and thus handled in time.
Preferably, the method also includes:
The electronic equipment carries out pressure test to KAFKA, obtains simulation KAFKA number of topics, and calculates described current
The difference of KAFKA number of topics and the simulation KAFKA number of topics, when the absolute value of the difference is greater than preset value, the electricity
Sub- equipment sounds an alarm.
Specifically, the pressure test refers to, by simulating the actual use scene of the electronic equipment, to realize to institute
State the simulation test of the actual motion index of electronic equipment.
Specifically, the preset value can assist the case where determining KAFKA number of topics as the benchmark compared, such as: institute
Stating preset value can be 3 etc..
Illustrated when actual conditions and analog case difference are larger there may be problem by above embodiment, it is described
The timely early warning of electronic equipment, to reduce risk.
In conclusion the present invention can call kafka- using configuration script when receiving the monitoring instruction of KAFKA theme
Topics source code, and combine the KAFKA subject name obtained, executes the kafka-topics source code, thus quickly, it is quasi-
Really obtain current KAFKA number of topics, further obtain configuration KAFKA number of topics, and compare the current KAFKA number of topics and
The configuration KAFKA number of topics is sounded an alarm when the current KAFKA number of topics is less than the configuration KAFKA number of topics,
To avoid the occurrence of KAFKA theme deficient phenomena, realizes timely early warning, improve O&M efficiency.
As shown in Fig. 2, being the functional block diagram of the preferred embodiment of KAFKA theme monitoring device of the present invention.The KAFKA
Theme monitoring device 11 includes call unit 110, acquiring unit 111, execution unit 112, comparison unit 113, output unit
114, storage unit 115, test cell 116, computing unit 117 and issue unit 118.So-called module/the unit of the present invention is
Refer to that one kind can be performed by processor 13, and the series of computation machine program segment of fixed function can be completed, is stored in
In memory 12.In the present embodiment, it will be described in detail in subsequent embodiment about the function of each module/unit.
When receiving the monitoring instruction of KAFKA theme, call unit 110 calls the source kafka-topics using configuration script
Code.
In at least one embodiment of the present invention, KAFKA is that a kind of distributed post of high-throughput subscribes to message system
System, it can handle the everything flow data in the website of consumer's scale.In KAFKA, come using a category attribute
The affiliated class for dividing data, divides this class of data, then referred to as topic theme.If KAFKA is seen as a data
Library, then topic theme can be understood as a table in database, and topic subject name is table name.
Therefore, if the topic in KAFKA is lacked, the loss of data and confusion of KAFKA will be caused, is influencing KAFKA just
Often work.
In at least one embodiment of the present invention, the electronic equipment, which receives the monitoring of KAFKA theme and instructs, includes, but not
It is limited to the combination of following one or more:
(1) the KAFKA theme that the electronic equipment receives user's triggering monitors instruction.
Specifically, the user can trigger the KAFKA theme monitoring instruction, the triggering by clicking trigger key
Key can be virtual key or physical button etc..
The user can also input corresponding phonetic order, be instructed using monitoring as the KAFKA theme, the present invention is not
Limitation.
Either, the user can also input finger print information, to trigger described KAFKA theme monitoring instruction etc..
(2) signal that the electronic equipment triggers the KAFKA theme monitoring instruction every prefixed time interval is received.
Specifically, the prefixed time interval can be configured by the electronic equipment, can also by the user into
Row custom-configures, and the present invention does not limit.
Such as: the prefixed time interval can be every 1 minute etc..
(3) receive the electronic equipment configuration triggers the signal of the KAFKA theme monitoring instruction in preset time.
Specifically, the preset time can be configured by the electronic equipment, can also be carried out certainly by the user
Definition configuration, the present invention do not limit.
Such as: the preset time can be 10 points of the morning of December 1 etc..
(4) the KAFKA theme monitoring instruction that the electronic equipment triggers under configuration status is received.
Specifically, the configuration status includes, but are not limited to the state etc. of loss of data in KAFKA.
In at least one embodiment of the present invention, the electronic equipment is receiving the KAFKA theme monitoring instruction
Afterwards, the identity of user can also be authenticated, such as: the electronic equipment can be by the identity information of the user (as used
Name in an account book, password etc.) it is matched with the identity information of storage, wherein user corresponding to the identity information of the storage has
Supervision authority.
Preferably, the configuration script includes kafka-run-class.sh script.
Specifically, the kafka-run-class.sh script is joined for calling TopicCommand class, and according to corresponding
Number executes specified function, for example, in the present embodiment, the kafka-run-class.sh script is described for calling
Kafka-topics source code, to get current KAFKA number of topics.
Further, the call unit 110 is called by the configuration script kafka-run-class.sh script
The kafka-topics source code, and by executing the kafka-topics source code, it further gets and is currently running
KAFKA number of topics, i.e., the described current KAFKA number of topics.
Acquiring unit 111 obtains KAFKA subject name.
In at least one embodiment of the present invention, according to the KAFKA subject name, the acquiring unit 111 can be with
The topic theme corresponding to the KAFKA subject name is got, and further counts the current KAFKA number of topics.
Specifically, the acquisition of the acquiring unit 111 KAFKA subject name includes, but are not limited to any one following mode:
(1) acquiring unit 111 obtains the configuration file of KAFKA, and the KAFKA master is obtained from the configuration file
Autograph claims.
Specifically, when installing KAFKA, while the configuration file of KAFKA is saved.
It specifically, include KAFKA cluster name, the performance parameter of KAFKA, configuration KAFKA theme in the configuration file
Number, KAFKA subject name etc..
(2) acquiring unit 111 obtains the history log of KAFKA, from obtaining in the history log
The subject name of secondary operation is as the KAFKA subject name.
By above embodiment, it can more meet the currently used demand of user.
Execution unit 112 executes the kafka-topics source code according to the KAFKA subject name, obtains current
KAFKA number of topics.
In at least one embodiment of the present invention, the execution unit 112 is according to the KAFKA subject name, and ties
The parameters such as zookeeperIP (zookeeper Internet Protocol), port numbers are closed, the kafka-topics is executed
Source code.
Such as: the execution unit 112 executes the kafka-topics source code/wls/elis_ after filling parameter
racs_kafka/bin/kafka-topics.sh--list--zookeeper30.16.4.197:2181,30.16.4.208:
2181,30.16.4.219:2181/kafka|egrep"T-HADOOP-MSG-R2P1|T-SKYNET-DATA-R1P6|T-
SKYNET-DATA-R1P6-TAG | T-SKYNET-DATA-TAG-R1P6 | _ _ consumer_offsets "-c obtains described work as
Preceding KAFKA number of topics.Wherein, the zookeeperIP is 30.16.4.197, and the port numbers are 2181, the KAFKA master
Autograph is known as T-HADOOP-MSG-R2P1, T-SKYNET-DATA-R1P6 etc..
Preferably, after obtaining current KAFKA number of topics, the method also includes:
The current KAFKA number of topics is saved as variable by storage unit 115, for transferring.
Under normal conditions, the current KAFKA number of topics can be stored in the running log of the KAFKA, but be worked as
When the running log runs up to certain volume, the speed of service of the KAFKA will seriously affect, therefore, in the KAFKA
Running log is periodically executed delete operation.
It is directed to above situation, in order to avoid subsequent the case where can not getting current KAFKA number of topics generation, institute
It states electronic equipment and the current KAFKA number of topics is saved as into variable, transferred at any time with facilitating, and have more flexibility.
The acquiring unit 111 obtains configuration KAFKA number of topics.
In at least one embodiment of the present invention, the configuration KAFKA number of topics is the master that the KAFKA should have
Inscribe number.
Specifically, the acquisition of the acquiring unit 111 configuration KAFKA number of topics includes, but are not limited to any one following side
Formula:
(1) acquiring unit 111 obtains the configuration file of KAFKA, and the configuration is obtained from the configuration file
KAFKA number of topics.
Specifically, when installing KAFKA, while the configuration file of KAFKA is saved.
It wherein, include KAFKA cluster name, the performance parameter of KAFKA, configuration KAFKA number of topics in the configuration file
Deng.
(2) acquiring unit 111 obtains the history log of KAFKA, from obtaining in the history log
The KAFKA number of topics of secondary operation is as the configuration KAFKA number of topics.
Specifically, the KAFKA number of topics that the acquiring unit 111 runs last time as the configuration KAFKA number of topics,
Current monitoring demand can more be met.
Comparison unit 113 compares the current KAFKA number of topics and the configuration KAFKA number of topics.
In at least one embodiment of the present invention, described current when deficient phenomena is not present in the KAFKA theme
KAFKA number of topics should be consistent with the configuration KAFKA number of topics.
Therefore, in the present embodiment, the comparison unit 113 is by the current KAFKA number of topics and the configuration KAFKA
Number of topics compares, so that it is determined that whether the current KAFKA number of topics is normal.
When the current KAFKA number of topics is less than the configuration KAFKA number of topics, output unit 114 exports alarm signal
Breath.
In at least one embodiment of the present invention, when the current KAFKA number of topics is equal to the configuration KAFKA theme
When number, the output unit 114 can determine that the current KAFKA number of topics is normal, and the output unit 114 is without report
It is alert.
And when the current KAFKA number of topics is less than the configuration KAFKA number of topics, the output unit 114 determines
The KAFKA theme missing, the output unit 114 export warning message.
Specifically, the output of output unit 114 warning message includes, but are not limited to following one or more kinds of modes
Combination:
(1) output unit 114 sends alarm mail to designated person.
(2) output unit 114 is dialed the police emergency number to the designated person.
(3) output unit 114 sends warning message to the designated person.
(4) output unit 114 issues alarm sound.
Specifically, the designated person includes, but are not limited to the combination of following one or more: operation maintenance personnel, project
Manager, developer etc..
By above embodiment, the output unit 114 can be realized timely early warning, and by configuring different alarms
Mode effectively improves alarm efficiency.
Further, the output unit 114 exports warning message further include:
The output unit 114 obtains current time and preset alarm text, shows the current time, the default report
Alert text and the current KAFKA number of topics.
Specifically, the preset alarm text can be custom-configured by user, and the present invention does not limit, for example, institute
Stating preset alarm text may include " kafka topic is less than 5 " etc..
By above embodiment, the output unit 114 can show the time and details that event occurs simultaneously
(such as: the current KAFKA number of topics), makes related personnel become more apparent upon warning reason, and thus handled in time.
Preferably, the method also includes:
Test cell 116 carries out pressure test to KAFKA, obtains simulation KAFKA number of topics, and computing unit 117 calculates institute
The difference for stating current KAFKA number of topics and the simulation KAFKA number of topics, when the absolute value of the difference is greater than preset value,
Issue unit 118 sounds an alarm.
Specifically, the pressure test refers to, by simulating the actual use scene of the electronic equipment, to realize to institute
State the simulation test of the actual motion index of electronic equipment.
Specifically, the preset value can assist the case where determining KAFKA number of topics as the benchmark compared, such as: institute
Stating preset value can be 3 etc..
Illustrated when actual conditions and analog case difference are larger there may be problem by above embodiment, it is described
The timely early warning of electronic equipment, to reduce risk.
In conclusion the present invention can call kafka- using configuration script when receiving the monitoring instruction of KAFKA theme
Topics source code, and combine the KAFKA subject name obtained, executes the kafka-topics source code, thus quickly, it is quasi-
Really obtain current KAFKA number of topics, further obtain configuration KAFKA number of topics, and compare the current KAFKA number of topics and
The configuration KAFKA number of topics is sounded an alarm when the current KAFKA number of topics is less than the configuration KAFKA number of topics,
To avoid the occurrence of KAFKA theme deficient phenomena, realizes timely early warning, improve O&M efficiency.
As shown in figure 3, being that the present invention realizes that the structure of electronic equipment of the preferred embodiment of KAFKA theme monitoring method is shown
It is intended to.
The electronic equipment 1 be it is a kind of can according to the instruction for being previously set or store, automatic progress numerical value calculating and/or
The equipment of information processing, hardware include but is not limited to microprocessor, specific integrated circuit (Application Specific
Integrated Circuit, ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), number
Word processing device (Digital Signal Processor, DSP), embedded device etc..
The electronic equipment 1, which can also be but not limited to any one, to pass through keyboard, mouse, remote controler, touching with user
The modes such as template or voice-operated device carry out the electronic product of human-computer interaction, for example, personal computer, tablet computer, smart phone,
Personal digital assistant (PersonalDigital Assistant, PDA), game machine, Interactive Internet TV (Internet
Protocol Television, IPTV), intellectual wearable device etc..
The electronic equipment 1 can also be that the calculating such as desktop PC, notebook, palm PC and cloud server are set
It is standby.
Network locating for the electronic equipment 1 include but is not limited to internet, wide area network, Metropolitan Area Network (MAN), local area network, it is virtual specially
With network (Virtual Private Network, VPN) etc..
In one embodiment of the invention, the electronic equipment 1 includes, but are not limited to memory 12, processor 13,
And it is stored in the computer program that can be run in the memory 12 and on the processor 13, such as KAFKA theme prison
Control program.
It will be understood by those skilled in the art that the schematic diagram is only the example of electronic equipment 1, not structure paired electrons
The restriction of equipment 1 may include perhaps combining certain components or different components, example than illustrating more or fewer components
Such as described electronic equipment 1 can also include input-output equipment, network access equipment, bus.
Alleged processor 13 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor
Deng the processor 13 is arithmetic core and the control centre of the electronic equipment 1, entire using various interfaces and connection
The various pieces of electronic equipment 1, and execute the operating system of the electronic equipment 1 and types of applications program, the program of installation
Code etc..
The processor 13 executes the operating system of the electronic equipment 1 and the types of applications program of installation.The place
Reason device 13 executes the application program to realize the step in above-mentioned each KAFKA theme monitoring method embodiment, such as Fig. 1 institute
Step S10, S11, S12, S13, S14, the S15 shown.
Alternatively, the processor 13 realizes each module in above-mentioned each Installation practice/mono- when executing the computer program
The function of member, such as: when receiving the monitoring instruction of KAFKA theme, kafka-topics source code is called using configuration script;
Obtain KAFKA subject name;According to the KAFKA subject name, the kafka-topics source code is executed, is obtained current
KAFKA number of topics;Obtain configuration KAFKA number of topics;Compare the current KAFKA number of topics and the configuration KAFKA number of topics;
When the current KAFKA number of topics is less than the configuration KAFKA number of topics, warning message is exported.
Illustratively, the computer program can be divided into one or more module/units, one or more
A module/unit is stored in the memory 12, and is executed by the processor 13, to complete the present invention.It is one
Or multiple module/units can be the series of computation machine program instruction section that can complete specific function, the instruction segment is for retouching
State implementation procedure of the computer program in the electronic equipment 1.For example, the computer program can be divided into tune
It is single with unit 110, acquiring unit 111, execution unit 112, comparison unit 113, output unit 114, storage unit 115, test
Member 116, computing unit 117 and issue unit 118.
The memory 12 can be used for storing the computer program and/or module, the processor 13 by operation or
The computer program and/or module being stored in the memory 12 are executed, and calls the data being stored in memory 12,
Realize the various functions of the electronic equipment 1.The memory 12 can mainly include storing program area and storage data area,
In, storing program area can application program needed for storage program area, at least one function (such as sound-playing function, image
Playing function etc.) etc.;Storage data area, which can be stored, uses created data (such as audio data, phone directory according to mobile phone
Deng) etc..In addition, memory 12 may include high-speed random access memory, it can also include nonvolatile memory, such as firmly
Disk, memory, plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital,
SD) block, flash card (Flash Card), at least one disk memory, flush memory device or other volatile solid-states
Part.
The memory 12 can be the external memory and/or internal storage of electronic equipment 1.Further, described
Memory 12 can be the circuit with store function for not having physical form in integrated circuit, such as RAM (Random-Access
Memory, random access memory), FIFO (First In First Out) etc..Alternatively, the memory 12 is also possible to
Memory with physical form, such as memory bar, TF card (Trans-flash Card).
If the integrated module/unit of the electronic equipment 1 is realized in the form of SFU software functional unit and as independent
Product when selling or using, can store in a computer readable storage medium.Based on this understanding, the present invention is real
All or part of the process in existing above-described embodiment method, can also instruct relevant hardware come complete by computer program
At the computer program can be stored in a computer readable storage medium, which is being executed by processor
When, it can be achieved that the step of above-mentioned each embodiment of the method.
Wherein, the computer program includes computer program code, and the computer program code can be source code
Form, object identification code form, executable file or certain intermediate forms etc..The computer-readable medium may include: can
Carry any entity or device, recording medium, USB flash disk, mobile hard disk, magnetic disk, CD, computer of the computer program code
Memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access
Memory), electric carrier signal, telecommunication signal and software distribution medium etc..It should be noted that the computer-readable medium
The content for including can carry out increase and decrease appropriate according to the requirement made laws in jurisdiction with patent practice, such as in certain departments
Method administrative area does not include electric carrier signal and telecommunication signal according to legislation and patent practice, computer-readable medium.
In conjunction with Fig. 1, the memory 12 in the electronic equipment 1 stores multiple instruction to realize a kind of KAFKA theme
The multiple instruction can be performed to realize in monitoring method, the processor 13: when receiving the monitoring instruction of KAFKA theme,
Kafka-topics source code is called using configuration script;Obtain KAFKA subject name;According to the KAFKA subject name, hold
The row kafka-topics source code, obtains current KAFKA number of topics;Obtain configuration KAFKA number of topics;It compares described current
KAFKA number of topics and the configuration KAFKA number of topics;When the current KAFKA number of topics is less than the configuration KAFKA number of topics
When, export warning message.
Preferred embodiment according to the present invention, the configuration script include kafka-run-class.sh script.
Preferred embodiment according to the present invention, the processor 13 also execute multiple instruction and include:
The current KAFKA number of topics is saved as into variable, for transferring.
Preferred embodiment according to the present invention, the processor 13 also execute multiple instruction and include:
The configuration file for obtaining KAFKA, obtains the configuration KAFKA number of topics from the configuration file;Or
The history log for obtaining KAFKA obtains the KAFKA number of topics of last time operation from the history log
As the configuration KAFKA number of topics.
Preferred embodiment according to the present invention, the processor 13 also execute multiple instruction and include:
Alarm mail is sent to designated person;And/or
It dials the police emergency number to the designated person;And/or
Warning message is sent to the designated person;And/or
Issue alarm sound.
Preferred embodiment according to the present invention, the processor 13 also execute multiple instruction and include:
Obtain current time and preset alarm text;
Show the current time, the preset alarm text and the current KAFKA number of topics.
Preferred embodiment according to the present invention, the processor 13 also execute multiple instruction and include:
Pressure test is carried out to KAFKA, obtains simulation KAFKA number of topics;
Calculate the difference of the current KAFKA number of topics and the simulation KAFKA number of topics;
When the absolute value of the difference is greater than preset value, sound an alarm.
Specifically, the processor 13 can refer to the concrete methods of realizing of above-metioned instruction related in Fig. 1 corresponding embodiment
The description of step, this will not be repeated here.
In several embodiments provided by the present invention, it should be understood that disclosed system, device and method can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the module
It divides, only a kind of logical function partition, there may be another division manner in actual implementation.
The module as illustrated by the separation member may or may not be physically separated, aobvious as module
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.Some or all of the modules therein can be selected to realize the mesh of this embodiment scheme according to the actual needs
's.
It, can also be in addition, each functional module in each embodiment of the present invention can integrate in one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of hardware adds software function module.
It is obvious to a person skilled in the art that invention is not limited to the details of the above exemplary embodiments, Er Qie
In the case where without departing substantially from spirit or essential attributes of the invention, the present invention can be realized in other specific forms.
Therefore, in all respects, the present embodiments are to be considered as illustrative and not restrictive, this
The range of invention is indicated by the appended claims rather than the foregoing description, it is intended that the equivalent requirements of the claims will be fallen in
All changes in meaning and scope are included in the present invention.Any attached associated diagram label in claim should not be considered as limit
Claim involved in making.
Furthermore, it is to be understood that one word of " comprising " does not exclude other units or steps, odd number is not excluded for plural number.In system claims
The multiple units or device of statement can also be implemented through software or hardware by a unit or device.Second equal words are used
It indicates title, and does not indicate any particular order.
Finally it should be noted that the above examples are only used to illustrate the technical scheme of the present invention and are not limiting, although reference
Preferred embodiment describes the invention in detail, those skilled in the art should understand that, it can be to of the invention
Technical solution is modified or equivalent replacement, without departing from the spirit and scope of the technical solution of the present invention.
Claims (10)
1. a kind of KAFKA theme monitoring method, which is characterized in that the described method includes:
When receiving the monitoring instruction of KAFKA theme, kafka-topics source code is called using configuration script;
Obtain KAFKA subject name;
According to the KAFKA subject name, the kafka-topics source code is executed, obtains current KAFKA number of topics;
Obtain configuration KAFKA number of topics;
Compare the current KAFKA number of topics and the configuration KAFKA number of topics;
When the current KAFKA number of topics is less than the configuration KAFKA number of topics, warning message is exported.
2. KAFKA theme monitoring method as described in claim 1, which is characterized in that the configuration script includes kafka-
Run-class.sh script.
3. KAFKA theme monitoring method as described in claim 1, which is characterized in that after obtaining current KAFKA number of topics,
The method also includes:
The current KAFKA number of topics is saved as into variable, for transferring.
4. KAFKA theme monitoring method as described in claim 1, which is characterized in that the acquisition configures KAFKA number of topics packet
Include any one following mode:
The configuration file for obtaining KAFKA, obtains the configuration KAFKA number of topics from the configuration file;Or
The history log for obtaining KAFKA obtains the KAFKA number of topics conduct of last time operation from the history log
The configuration KAFKA number of topics.
5. KAFKA theme monitoring method as described in claim 1, which is characterized in that the output warning message includes following
The combination of one or more kinds of modes:
Alarm mail is sent to designated person;And/or
It dials the police emergency number to the designated person;And/or
Warning message is sent to the designated person;And/or
Issue alarm sound.
6. KAFKA theme monitoring method as described in claim 1, which is characterized in that the output warning message further include:
Obtain current time and preset alarm text;
Show the current time, the preset alarm text and the current KAFKA number of topics.
7. KAFKA theme monitoring method as described in claim 1, which is characterized in that the method also includes:
Pressure test is carried out to KAFKA, obtains simulation KAFKA number of topics;
Calculate the difference of the current KAFKA number of topics and the simulation KAFKA number of topics;
When the absolute value of the difference is greater than preset value, sound an alarm.
8. a kind of KAFKA theme monitoring device, which is characterized in that described device includes:
Call unit, for calling the source kafka-topics generation using configuration script when receiving the monitoring instruction of KAFKA theme
Code;
Acquiring unit, for obtaining KAFKA subject name;
Execution unit obtains current for executing the kafka-topics source code according to the KAFKA subject name
KAFKA number of topics;
The acquiring unit is also used to obtain configuration KAFKA number of topics;
Comparison unit, for comparing the current KAFKA number of topics and the configuration KAFKA number of topics;
Output unit, for exporting warning message when the current KAFKA number of topics is less than the configuration KAFKA number of topics.
9. a kind of electronic equipment, which is characterized in that the electronic equipment includes:
Memory stores at least one instruction;And
Processor executes the instruction stored in the memory to realize as claimed in any of claims 1 to 7 in one of claims
KAFKA theme monitoring method.
10. a kind of computer readable storage medium, it is characterised in that: be stored at least one in the computer readable storage medium
A instruction, at least one described instruction are executed by the processor in electronic equipment to realize such as any one of claim 1 to 7
The KAFKA theme monitoring method.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811537835.5A CN109815082B (en) | 2018-12-15 | 2018-12-15 | KAFKA theme monitoring method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811537835.5A CN109815082B (en) | 2018-12-15 | 2018-12-15 | KAFKA theme monitoring method and device, electronic equipment and storage medium |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109815082A true CN109815082A (en) | 2019-05-28 |
CN109815082B CN109815082B (en) | 2023-04-11 |
Family
ID=66602035
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811537835.5A Active CN109815082B (en) | 2018-12-15 | 2018-12-15 | KAFKA theme monitoring method and device, electronic equipment and storage medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109815082B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113608896A (en) * | 2021-08-12 | 2021-11-05 | 重庆紫光华山智安科技有限公司 | Method, system, medium and terminal for dynamically switching data stream |
CN114253807A (en) * | 2021-12-20 | 2022-03-29 | 深圳前海微众银行股份有限公司 | Alarm information notification method and device |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140289231A1 (en) * | 2013-03-25 | 2014-09-25 | Salesforce.Com, Inc. | Combining topic suggestions from different topic sources to assign to textual data items |
CN106778033A (en) * | 2017-01-10 | 2017-05-31 | 南京邮电大学 | A kind of Spark Streaming abnormal temperature data alarm methods based on Spark platforms |
CN107391279A (en) * | 2017-07-31 | 2017-11-24 | 山东浪潮云服务信息科技有限公司 | A kind of message queue container creation method, device and message queue container |
CN107590210A (en) * | 2017-08-25 | 2018-01-16 | 咪咕互动娱乐有限公司 | A kind of data processing method, device, system and computer-readable recording medium |
US20180032582A1 (en) * | 2016-07-28 | 2018-02-01 | International Business Machines Corporation | Cross object synchronization |
US20180063055A1 (en) * | 2016-08-26 | 2018-03-01 | Linkedin Corporation | Repartitioning a topic in a publish-subscribe message system |
-
2018
- 2018-12-15 CN CN201811537835.5A patent/CN109815082B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140289231A1 (en) * | 2013-03-25 | 2014-09-25 | Salesforce.Com, Inc. | Combining topic suggestions from different topic sources to assign to textual data items |
US20180032582A1 (en) * | 2016-07-28 | 2018-02-01 | International Business Machines Corporation | Cross object synchronization |
US20180063055A1 (en) * | 2016-08-26 | 2018-03-01 | Linkedin Corporation | Repartitioning a topic in a publish-subscribe message system |
CN106778033A (en) * | 2017-01-10 | 2017-05-31 | 南京邮电大学 | A kind of Spark Streaming abnormal temperature data alarm methods based on Spark platforms |
CN107391279A (en) * | 2017-07-31 | 2017-11-24 | 山东浪潮云服务信息科技有限公司 | A kind of message queue container creation method, device and message queue container |
CN107590210A (en) * | 2017-08-25 | 2018-01-16 | 咪咕互动娱乐有限公司 | A kind of data processing method, device, system and computer-readable recording medium |
Non-Patent Citations (3)
Title |
---|
ERNEST.WU: "Kafka常用命令之kafka-topics.sh", 《HTTPS://BLOG.CSDN.NET/QQ_29116427/ARTICLE/DETAILS/80202392?》 * |
ZFSZHANGYUAN: "kafka单机重启topic丢失问题排查", 《HTTPS://BLOG.CSDN.NET/ZFSZHANGYUAN/ARTICLE/DETAILS/53389916》 * |
苏洛尔寂: "Hadoop环境搭建笔记整理(六)——初识Kafka以及本阶段总结", 《HTTPS://ZHUANLAN.ZHIHU.COM/P/27851350》 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113608896A (en) * | 2021-08-12 | 2021-11-05 | 重庆紫光华山智安科技有限公司 | Method, system, medium and terminal for dynamically switching data stream |
CN113608896B (en) * | 2021-08-12 | 2023-09-08 | 重庆紫光华山智安科技有限公司 | Method, system, medium and terminal for dynamically switching data streams |
CN114253807A (en) * | 2021-12-20 | 2022-03-29 | 深圳前海微众银行股份有限公司 | Alarm information notification method and device |
CN114253807B (en) * | 2021-12-20 | 2023-04-07 | 深圳前海微众银行股份有限公司 | Alarm information notification method and device |
Also Published As
Publication number | Publication date |
---|---|
CN109815082B (en) | 2023-04-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109634818A (en) | Log analysis method, system, terminal and computer readable storage medium | |
CN110855473A (en) | Monitoring method, device, server and storage medium | |
CN109359026A (en) | Log reporting method, device, electronic equipment and computer readable storage medium | |
CN109669837A (en) | Equipment state method for early warning, system, computer installation and readable storage medium storing program for executing | |
CN109086182A (en) | The method and terminal device of database auto-alarming | |
JP2022118108A (en) | Log auditing method, device, electronic apparatus, medium and computer program | |
CN112070416B (en) | AI-based RPA flow generation method, apparatus, device and medium | |
CN111427748A (en) | Task warning method, system, equipment and storage medium | |
CN105227405A (en) | Method for supervising and system | |
CN106254137A (en) | The alarm root-cause analysis system and method for supervisory systems | |
CN101277218B (en) | Dynamic analysis system and method for network alarm | |
CN109800269A (en) | Data managing method, device, computer equipment and storage medium | |
CN109542737A (en) | Platform alert processing method, device, electronic device and storage medium | |
CN109815082A (en) | KAFKA theme monitoring method, device, electronic equipment and storage medium | |
CN109800124A (en) | CPU usage monitoring method, device, electronic equipment and storage medium | |
CN109284331A (en) | Accreditation information acquisition method, terminal device and medium based on business datum resource | |
CN111914859A (en) | Service multiplexing method, computing device and computer readable storage medium | |
CN109240916A (en) | Information output controlling method, device and computer readable storage medium | |
CN109766236A (en) | KAFKA message queue number monitoring method, device, electronic equipment and storage medium | |
CN114756301B (en) | Log processing method, device and system | |
CN109889399A (en) | RocketMQ client connection number monitoring method, device, electronic equipment and storage medium | |
CN110362294A (en) | Development task executes method, apparatus, electronic equipment and storage medium | |
Zhang et al. | Role of subgraphs in epidemics over finite-size networks under the scaled SIS process | |
CN105245380B (en) | Message propagation mode identification method and device | |
CN109840179A (en) | RocketMQ Thread Count monitoring method, device, electronic equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |