CN110515795A - A kind of monitoring method of big data component, device, electronic equipment - Google Patents
A kind of monitoring method of big data component, device, electronic equipment Download PDFInfo
- Publication number
- CN110515795A CN110515795A CN201910687478.9A CN201910687478A CN110515795A CN 110515795 A CN110515795 A CN 110515795A CN 201910687478 A CN201910687478 A CN 201910687478A CN 110515795 A CN110515795 A CN 110515795A
- Authority
- CN
- China
- Prior art keywords
- target element
- monitoring
- monitoring scheme
- big data
- scheme
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Fuzzy Systems (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
This application discloses a kind of monitoring methods of big data component, to solve in the prior art due to usually requiring to rely on artificial loading and deleting monitoring module, so as to cause the lower problem of monitoring process automation degree.Method includes: to monitor on server with the presence or absence of target element to be monitored；The server, for running the target element to be monitored；If listening on the server, there are target elements to be monitored, then judgement can in monitoring scheme list with the presence or absence of the monitoring scheme that matches with the target element, it is described can be with monitoring scheme list for storing preconfigured big data monitor component scheme；If so, obtaining the monitoring scheme to match with the target element and loading, to be monitored to the target element.Monitoring device, electronic equipment and the computer readable storage medium of a kind of big data component is also disclosed in the application.
Description
Technical field
This application involves the monitoring technology field of big data component more particularly to a kind of monitoring method of big data component,
Device, electronic equipment and computer readable storage medium.
Background technique
With the fast development of big data technology, all trades and professions can all generate mass data daily, in face of mass data
Operation, various big data serviced components come into being, such as Hadoop, ElasticSearch, ClickHouse, Kafka etc..
Big data component in the process of running, carries out tuning and fault diagnosis to big data component for convenience, in general,
It needs to be monitored the big data component of operation.However, in relevant big data monitor component technology, due to usually require according to
Rely artificial load and deletes monitoring scheme, it is lower so as to cause monitoring process automation degree, in addition, in big data component count
In huge situation, it is also possible to reduce the treatment effeciency of monitoring process.
Summary of the invention
The embodiment of the present application provides a kind of monitoring method of big data component, is needed to solve using usual in the prior art
It relies on artificial load and deletes monitoring scheme, so as to cause the lower problem of monitoring process automation degree.
The embodiment of the present application also provides a kind of monitoring device of big data component, a kind of electronic equipment and a kind of calculating
Machine readable storage medium storing program for executing.
The embodiment of the present application adopts the following technical solutions:
A kind of monitoring method of big data component, comprising:
It monitors and whether there is target element to be monitored on server；The server, it is described to be monitored for running
Target element；
If listening on the server there are target element to be monitored, judgement can in monitoring scheme list whether
It is described to be used to store preconfigured monitoring with monitoring scheme list in the presence of the monitoring scheme to match with the target element
Scheme；
If so, obtaining the monitoring scheme to match with the target element and loading, to the target element
It is monitored.
A kind of monitoring device of big data component, including monitor module, judgment module and obtain module, in which:
Module is monitored, for monitoring on server with the presence or absence of target element to be monitored；The server, for running
The target element to be monitored；
Judgment module, if judgement can use monitoring for listening to there are target element to be monitored on the server
It is described to be used to store with monitoring scheme list with the presence or absence of the monitoring scheme to match with the target element in scheme list
Preconfigured monitoring scheme；
Obtain module, if for it is described can be with there is the monitoring side that matches with the target element in monitoring scheme list
Case then obtains the monitoring scheme to match with the target element and loads, to be monitored to the target element.
A kind of electronic equipment, comprising: memory, processor and be stored on the memory and can be on the processor
The computer program of operation, the computer program realize the prison of big data component as described above when being executed by the processor
The step of prosecutor method.
A kind of computer readable storage medium, which is characterized in that calculating is stored on the computer readable storage medium
The step of machine program, the computer program realizes the monitoring method of big data component as described above when being executed by processor.
The embodiment of the present application use at least one above-mentioned technical solution can reach it is following the utility model has the advantages that
Method provided by the embodiments of the present application, firstly, being pre-configured with big data monitor component scheme and being stored in available prison
It controls in scheme list；Secondly, by the operating condition of monitoring server so that it is determined that target element to be monitored, and determine to
After the target element of monitoring, trigger the server judgement can in monitoring scheme list with the presence or absence of the prison that matches with target element
Prosecutor case, finally, if judgement can be with having the monitoring scheme to match with target element in monitoring scheme list, it can be automatic
It obtains corresponding monitoring scheme and loads, to be monitored to target element, it is seen then that entire monitoring process needs not rely on manually
Load, therefore, avoids in relevant big data monitor component technology, due to usually requiring to rely on artificial load and deleting monitoring
Scheme, it is lower so as to cause monitoring process automation degree, and in the huge situation of big data component count, it is also possible to it drops
The problem for the treatment of effeciency of low monitoring process.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present application, constitutes part of this application, this Shen
Illustrative embodiments and their description please are not constituted an undue limitation on the present application for explaining the application.In the accompanying drawings:
Fig. 1 is a kind of implementation process schematic diagram of the monitoring method of big data component provided by the embodiments of the present application；
Fig. 2 is a kind of implementation process schematic diagram of the monitoring method of big data component provided by the embodiments of the present application；
Fig. 3 is a kind of concrete structure schematic diagram of the monitoring device of big data component provided by the embodiments of the present application；
Fig. 4 is a kind of structural schematic diagram of the monitoring electronic equipment of big data component provided by the embodiments of the present application.
Specific embodiment
To keep the purposes, technical schemes and advantages of the application clearer, below in conjunction with the application specific embodiment and
Technical scheme is clearly and completely described in corresponding attached drawing.Obviously, described embodiment is only the application one
Section Example, instead of all the embodiments.Based on the embodiment in the application, those of ordinary skill in the art are not doing
Every other embodiment obtained under the premise of creative work out, shall fall in the protection scope of this application.
Below in conjunction with attached drawing, the technical scheme provided by various embodiments of the present application will be described in detail.
Embodiment 1
In order to solve in relevant big data monitor component technology, due to usually requiring to rely on artificial load and deleting monitoring
Scheme, it is lower so as to cause monitoring process automation degree, and in the huge situation of big data component count, it is also possible to it drops
The problem for the treatment of effeciency of low monitoring process, the embodiment of the present application 1 provide a kind of monitoring method of big data component.This method
Executing subject, can be, but not limited to the server cluster etc. for being server or being made of server.
For convenience of description, the application is introduced embodiments herein so that executing subject is server as an example.It can be with
Understand, the executing subject of this method is that server is a kind of illustrative explanation, is not construed as the restriction to this method.
It referring to Figure 1, is a kind of monitoring method of big data component provided by the embodiments of the present application, which includes following
Step:
Step 11, monitoring whether there is target element to be monitored on server；
The server, for running target element to be monitored；In the embodiment of the present application, server can be to be monitored
Target element it is online operation when, obtain the corresponding configuration file of the client release, with determine the client release client
Each target element to be monitored in end；Alternatively, it is corresponding to obtain online several client releases together
Configuration file, with each target element to be monitored in the client of each online client release of determination.
Further, since client update of today is very frequent, generally require to dispose in server-side for every kind of function
The big data component of the corresponding many versions of the function, it is more and more too fat to move that this would potentially result in server-side.In this way, technology people
Member usually requires to be monitored the big data component being each currently running on server, to realize to big data component, service
Device carries out tuning and fault diagnosis.Based on this, in the embodiment of the present application, the target element to be detected may include various
The big data component that can be run on the server, for example, Hadoop, ElasticSearch, ClickHouse, Kafka etc..
In the embodiment of the present application, monitor on server with the presence or absence of before target element to be monitored, can in advance for
The target element setting monitoring interface of monitoring, there are when target element to be monitored, to be adjusted on listening to server
Target element is monitored with pre-set monitoring interface.
Step 12, if listening to there are target element to be monitored on server, judgement can be in monitoring scheme list
With the presence or absence of the monitoring scheme to match with target element；
It is described to use monitoring scheme list, for storing preconfigured big data monitor component scheme.
The big data monitor component scheme may include when being monitored to big data component, and required deployment is adopted
Collect the program of data and the start command of program.In order to facilitate subsequent query, in the embodiment of the present application, can will be pre-configured with
Big data monitor component scheme uniformly store to a certain preset path, such as can store into available monitoring scheme list,
In one embodiment, it generates, causes due to having new big data monitor component scheme during subsequent operation
It is subsequent may need to preconfigured big data monitor component scheme carry out it is extending transversely, therefore, in the application, can also will
Preconfigured big data monitor component scheme is uploaded onto the server or can be written in configuration file with monitoring scheme list,
If subsequent generate new big data monitor component scheme, directly the monitoring scheme of newly generated big data component can be uploaded
Server, or the title of the service of the monitoring scheme of newly generated big data component is added in configuration file, to realize cross
To extension.
It, can be with if being listened on server by step 11 there are target element to be monitored in the embodiment of the present application
With the following method judgement can in monitoring scheme list with the presence or absence of the monitoring scheme that matches with target element:
Step 121, the relevant information of target element is obtained；
Wherein, the relevant information includes at least following one kind: the corresponding clothes of the service name of target element, target element
The address of service of business device, the corresponding Hostname of target element, the corresponding port information of target element；
Step 122, according to relevant information judgement can in monitoring scheme list with the presence or absence of matching with target element
Monitoring scheme.
In the embodiment of the present application step 121, the relevant information of target element is obtained, specifically can be and determine the client
The configuration information of each big data component in the client of version.
Wherein, the configuration information of big data component specifically may include the information of two aspects: 1, big data component is corresponding
Functional information；2, the version information of big data component, service name, the corresponding service of big data component such as big data component
The address of service of device, the corresponding Hostname of big data component, the corresponding port information of big data component and support sequence
Change agreement (Json, protobuf etc.), the service logic of code, the parameter of requirement or format etc..
Optionally, in the application, in order to improve the speed for obtaining relevant information, target element to be monitored can only be obtained
Version information, for example, the service of the service name of target element to be monitored, the corresponding server of target element to be monitored
Address, the corresponding Hostname of target element to be monitored, the corresponding port information of target element to be monitored.
Wherein, the service name of target element to be monitored is for determining monitoring scheme corresponding with target element；Wait supervise
The corresponding Hostname of the target element of control is used to determine the host of target element operation；The corresponding clothes of target element to be monitored
The address of service of business device is used to determine the address of service of the corresponding server of target element；The corresponding end of target element to be monitored
Message breath, for obtaining the relevant information of target element.
Specifically, it when obtaining the relevant information of target element, can be realized by java applet, for example, firstly, passing through
Runtime executes Linux command, obtains the first implementing result；Secondly, calling Runtime.getRuntime () .exec
(command) the first implementing result is executed, and executes completion using Wait Orders such as waitFor () methods；Wherein,
Runtime.exec method can produce the process of a local, and the example for returning to a Process subclass；Then, pass through
Process.getInputStream () method and Process.getOutputStream () method are to Process subclass
Input, the output of example execute operation respectively, to obtain the second implementing result after order executes；Finally, being executed according to second
As a result text resolution is carried out, can be obtained service name, the target element of the target element to be monitored run on server
The address of service of corresponding server, the corresponding Hostname of target element, the corresponding port information of target element.
It in one embodiment, usually can be to correlation in order to which the versatility and scalability of relevant information may be implemented
Information carries out standardized management, for example, the metadata of Mysql database can be written in the unified formatting of above-mentioned relevant information
In table.
Through the above steps 121 get the relevant information of target element after, can be believed according to correlation by following steps
Breath further judgement can in monitoring scheme list with the presence or absence of the monitoring scheme that matches with target element:
Traversal can use monitoring scheme list, judgement can in monitoring scheme list with the presence or absence of matching with the component list
Big data monitor component scheme.
It specifically, if can be consistent big with the service name of target element in the component list with existing in monitoring scheme list
Data package monitoring scheme, then can be with there is the big data monitor component that matches with the component list in judgement in monitoring scheme list
Scheme.
For example, it is assumed that after service processes starting, firstly, getting preconfigured big data by loading configuration file
The list A of monitor component scheme；Secondly, read Mysql database in metadata table, get be currently running on server to
Monitoring objective component, and the service name of target element is written in list B；Finally, traversing list A, judge whether deposit in A
With the consistent big data monitor component scheme of the service name of target element in B.
Step 13, if so, the monitoring scheme to match with target element and load are obtained, to supervise to target element
Control.
Step 131, the start command of monitoring scheme corresponding with target element is obtained；
Step 132, the start command of monitoring scheme is loaded into server corresponding with target element, to target element
It is monitored.
Examples detailed above is continued to use, if there is the consistent big data monitor component side of service name with the target element in B in A
Case then obtains the monitoring scheme, and obtains corresponding start command, and then the start command of monitoring scheme is loaded into and target
The corresponding server of component, to be monitored to target element.
It optionally, can be corresponding with target element by monitoring scheme after start command starting in the application reality example
The address of service of server and the corresponding port information of target element are keyword, the configuration file of write timing database
In, so as to subsequent query monitoring scheme.Wherein, the time series database, for storing the data of acquisition.
Optionally, in the embodiment of the present application, when being monitored to target element, server can will be supervised by monitoring interface
The status information feedback for each target element controlled gives the monitoring report page.Wherein, the information that monitoring report can be shown includes mesh
Service name, monitoring period and the monitor state information for marking component, for example, the operating status of target element.
In one embodiment, monitoring report can be stored into the time series database of server, so as in real time
The state of each component is monitored, the problem of timely discovery process is broken down, or even can select to reinstall some target element.
Specifically, due to the service name of target element to be monitored be it is unique, if subsequent need to report in monitoring
The information for accusing page interrogation target element, then the service name for only needing to input target element in time series database can be into
Row inquiry.
It should be noted that needing to locate certain data accordingly according to the property of itself before displaying
Reason, for example the total amount of data in display data library is wanted to add up it is necessary to do one to the data volume obtained on each server, such as
Sum () function, processing here can be by the built-in function of prometheus it is achieved which type of processing done
It is according to service logic come fixed, and displayed page exploitation is completed one key when deployment and imported.
Using method provided by the embodiments of the present application, firstly, being pre-configured with big data monitor component scheme and be stored in can
With in monitoring scheme list；Secondly, by the operating condition of monitoring server so that it is determined that target element to be monitored, and true
After fixed target element to be monitored, trigger the server judgement can be matched with whether there is in monitoring scheme list with target element
Monitoring scheme can be with finally, if judgement can be with having the monitoring scheme that matches with target element in monitoring scheme list
Automatically it obtains corresponding monitoring scheme and loads, to be monitored to target element, it is seen then that entire monitoring process needs not rely on
Artificial load, therefore, avoids in relevant big data monitor component technology, due to usually requiring to rely on artificial load and delete
Monitoring scheme, it is lower so as to cause monitoring process automation degree, and in the huge situation of big data component count, may be used also
The problem for the treatment of effeciency of monitoring process can be reduced.
In the embodiment of the present application, since client of today update is very frequent, generally require taking for every kind of function
The big data component of the corresponding many versions of the function is disposed on business end, therefore, for the big data component of many versions, service
Many corresponding monitoring schemes are also had on end, this would potentially result in, and server-side is more and more too fat to move, and then influences server
Service performance is based on this, the following method for deleting monitoring scheme is provided in the embodiment of the present application, also to improve the service of server
Performance:
After listening to target element completion monitoring, the monitoring scheme is deleted according to default script, to stop to described
Target element is monitored.
Specifically, after listening to target element completion monitoring, to the monitoring side to some target element to be monitored
Case is deleted, then can call directly deletion script, and the title of the corresponding monitoring scheme of target element to be monitored is passed
Pass deletion script, wherein after deleting the title that script receives monitoring scheme, can automatically call Uninstaller to remove first number
Which according to the relevant information for reading monitoring scheme in table, operated on server to obtain the monitoring scheme, then according to suitable
Sequence deletes the monitoring scheme of every server operation.
It, can be to avoid monitoring scheme be deleted due to usually requiring to rely in the related technology, so as to cause prison using this method
It is lower to control process automation degree, and in the huge situation of big data component count, it is also possible to reduce the place of monitoring process
The problem of managing efficiency.
Below by way of embodiment 2 is introduced, the monitoring method of big data component provided by the embodiments of the present application is made further
Explanation.
Embodiment 2
In order to solve in relevant big data monitor component technology, due to usually requiring to rely on artificial load and deleting monitoring
Scheme, it is lower so as to cause monitoring process automation degree, and in the huge situation of big data component count, it is also possible to it drops
The problem for the treatment of effeciency of low monitoring process, the embodiment of the present application provide a kind of big data monitor component method.
For convenience of description, below by taking executing subject is server as an example, the embodiment of the present application is introduced.Such as Fig. 2 institute
Show, the embodiment of the present application provides a kind of monitoring method of big data component, which includes the following steps:
Step 21, the big data component that server is currently running is determined；
Due in order to realize various service functions, it usually needs it is corresponding much to dispose the function in server-side
The big data component of version, in this way, it is more and more too fat to move to may cause server-side, big data component can not be carried out tuning with
And fault diagnosis, it is thus typically necessary to be monitored to the big data component being each currently running on server, to realize to big
Data package, server carry out tuning and fault diagnosis.Based on this, in the application, can first determine on server it is current just
In the big data component of operation, for example, Hadoop, ElasticSearch, ClickHouse, Kafka etc., to determine wait supervise
The target element of control.
In addition, in the embodiment of the present application, it is also necessary to obtain the relevant information of target element to be monitored, the relevant information
Including at least following one kind: the address of service of the corresponding server of the service name of target element, target element, target element pair
The corresponding port information of Hostname, target element answered.
Wherein, when obtaining the relevant information of target element to be monitored, it can use and be obtained as provided by above-described embodiment 1
The method for taking relevant information no longer illustrates herein to avoid repeating.
After the target element for determining big data component that server is currently running/to be monitored, following steps can be executed
Rapid 22, with it is determining with the big data component/the corresponding monitoring scheme of to be monitored target element, and then to it is currently running greatly
Data package/target element to be monitored is monitored.
The monitoring scheme may include the journey of the acquisition data of required deployment when being monitored to big data component
The start command of sequence and program.In the embodiment of the present application, preconfigured big data monitor component scheme can uniformly be deposited
Storage is into available monitoring scheme list, so as to subsequent query.
Step 22, it determines monitoring scheme corresponding with the big data component being currently running, and loads the monitoring automatically
Scheme, to realize the monitoring of big data component.
In the embodiment of the present application, it is possible, firstly, to which loading configuration file, gets preconfigured big data monitor component side
The list A of case；Secondly, reading the metadata table in Mysql database, gets and be currently running target group to be monitored on server
Part, and the service name of target element is written in list B；Finally, traversing list A, judge to whether there is and mesh in B in A
Mark the consistent big data monitor component scheme of service name of component.
If there is the consistent big data monitor component scheme of service name with target element in B, acquisition and target in A
Monitoring scheme that component matches and load, to be monitored to target element.
It should be noted that in order to avoid deleting monitoring scheme due to usually requiring to rely in the related technology, so as to cause
It is lower to monitor process automation degree, and in the huge situation of big data component count, it is also possible to reduce monitoring process
The problem for the treatment of effeciency, in the embodiment of the present application, after listening to target element and completing monitoring, deletes institute also according to default script
Monitoring scheme is stated, to stop being monitored the target element.
Since the embodiment of the present invention is conceived using application identical with above-described embodiment 1, it is also able to solve existing skill
The problems in art, details are not described herein again.
Embodiment 3
For inventive concept same as mentioned above, the embodiment of the present application also provides a kind of monitoring dress of big data component
It sets, for solving in relevant big data monitor component technology, due to usually requiring to rely on artificial load and deleting monitoring scheme,
It is lower so as to cause monitoring process automation degree, and in the huge situation of big data component count, it is also possible to reduce prison
The problem for the treatment of effeciency of control process.
The concrete structure schematic diagram of the monitoring device 30 of the big data component is as shown in figure 3, include monitoring module 31, judgement
Module 32 and acquisition module 33, wherein each functions of modules is as follows:
Module 31 is monitored, for monitoring on server with the presence or absence of target element to be monitored；The server, for transporting
The row target element to be monitored；
Judgment module 32, if judgement can use prison for listening to there are target element to be monitored on the server
It is described to be used to deposit with monitoring scheme list with the presence or absence of the monitoring scheme to match with the target element in control scheme list
Store up preconfigured big data monitor component scheme；
Module 33 is obtained, for that can exist and the matched monitoring side of the target element in monitoring scheme list when described
When case, obtains the monitoring scheme corresponding with the target element and load, to be monitored to the target element.
Wherein, the judgment module 31 specifically includes and obtains first unit and the first judging unit, the function of each unit
It is as follows:
Acquiring unit, for obtaining the relevant information of the target element；Wherein, the relevant information includes at least following
It is a kind of: the service name of the target element, the address of service of the corresponding server of the target element, the target element pair
Hostname, the corresponding port information of the target element answered；
Judging unit, for according to relevant information judgement can in monitoring scheme list with the presence or absence of with it is described
The monitoring scheme that target element matches.
Optionally, in the embodiment of the present application, acquiring unit specifically can be used for:
Obtain the relevant information of the target element, and by the service name of the target element in the relevant information
The component list is written；Wherein, the relevant information includes at least following one kind: the service name of the target element, the mesh
Mark address of service, the corresponding Hostname of the target element, the corresponding end of the target element of the corresponding server of component
Message breath；
Optionally, in the embodiment of the present application, judging unit specifically can be used for:
Monitoring scheme list can be used described in traversal, can be whether there is and the component in monitoring scheme list described in judgement
The big data monitor component scheme that list matches.
Specifically, if the service existed in monitoring scheme list with target element described in described the component list
The title of the consistent big data monitor component scheme of title, then described in judgement can with exist in monitoring scheme list with it is described
The big data monitor component scheme that the component list matches.
The acquisition module 33, specifically includes second acquisition unit and monitoring unit, and the function of each unit is as follows:
Second acquisition unit, for obtaining the start command of the monitoring scheme corresponding with the target element；
Monitoring unit, for the start command of the monitoring scheme to be loaded into service corresponding with the target element
Device, to be monitored to the target element.
Using device provided by the present application, firstly, being pre-configured with big data monitor component scheme and being stored in available monitoring
In scheme list；Secondly, module is monitored by monitoring the operating condition of server so that it is determined that target element to be monitored, and
For judgment module after determining target element to be monitored, trigger the server judgement can be with whether there is and mesh in monitoring scheme list
The monitoring scheme that matches of mark component, finally, if can be with there is the monitoring side to match with target element in monitoring scheme list
Case, corresponding monitoring scheme can then be obtained automatically and load by obtaining module, to be monitored to target element, entirely be monitored
Journey needs not rely on artificial load, therefore, avoids in relevant big data monitor component technology, due to usually requiring to rely on people
Work load and deletion monitoring scheme, it is lower and huge in big data component count so as to cause monitoring process automation degree
In the case where, it is also possible to the treatment effeciency for reducing monitoring process avoids in the prior art due to usually requiring to rely on artificial add
Monitoring scheme is carried and deletes, it is lower so as to cause monitoring process automation degree, and in the huge feelings of big data component count
Under condition, it is also possible to the problem of reducing the treatment effeciency of monitoring process.
Optionally, device provided by the embodiments of the present application can also include stopping monitoring module, be used for:
After listening to target element completion monitoring, the monitoring scheme is deleted according to default script, to stop to described
Target element is monitored.
Using device provided by the present application, can to avoid deleting monitoring scheme due to usually requiring to rely in the related technology,
It is lower so as to cause monitoring process automation degree, and in the huge situation of big data component count, it is also possible to reduce prison
The problem for the treatment of effeciency of control process.
Embodiment 4
The embodiment of the present application, also provides a kind of smart machine for equipment control, and the hardware configuration of the smart machine shows
Be intended to as shown in figure 4, the electronic equipment includes processor in hardware view, optionally further comprising internal bus, network interface,
Memory.Wherein, memory may include memory, such as high-speed random access memory (Random-Access Memory,
RAM), it is also possible to further include nonvolatile memory (non-volatile memory), for example, at least 1 magnetic disk storage etc..
Certainly, which is also possible that hardware required for other business.
Processor, network interface and memory can be connected with each other by internal bus, which can be ISA
(Industry Standard Architecture, industry standard architecture) bus, PCI (Peripheral
Component Interconnect, Peripheral Component Interconnect standard) bus or EISA (Extended Industry Standard
Architecture, expanding the industrial standard structure) bus etc..The bus can be divided into address bus, data/address bus, control always
Line etc..Only to be indicated with a four-headed arrow in Fig. 4, it is not intended that an only bus or a type of convenient for indicating
Bus.
Memory, for storing program.Specifically, program may include program code, and said program code includes calculating
Machine operational order.Memory may include memory and nonvolatile memory, and provide instruction and data to processor.
Processor is from the then operation into memory of corresponding computer program is read in nonvolatile memory, in logical layer
Data synchronization unit is formed on face.Processor executes the program that memory is stored, and is specifically used for executing following operation:
It monitors and whether there is target element to be monitored on server；The server, it is described to be monitored for running
Target element；
If listening on the server there are target element to be monitored, judgement can in monitoring scheme list whether
It is described to be used to store preconfigured big number with monitoring scheme list in the presence of the monitoring scheme to match with the target element
According to monitor component scheme；
If so, obtaining the monitoring scheme to match with the target element and loading, to the target element
It is monitored.
In one case, following operation can also be performed in processor: obtaining the relevant information of the target element；Its
In, the relevant information includes at least following one kind: the service name of the target element, the corresponding service of the target element
The address of service of device, the corresponding Hostname of the target element, the corresponding port information of the target element；
It can be whether there is and the target element phase in monitoring scheme list according to relevant information judgement
The monitoring scheme matched.
It specifically, if can be consistent big with the service name of target element in the component list with existing in monitoring scheme list
Data package monitoring scheme, then can be with there is the big data monitor component that matches with the component list in judgement in monitoring scheme list
Scheme.
Optionally, processor obtains monitoring scheme corresponding with target element and loads, to be monitored to target element,
It specifically includes:
Obtain the start command of monitoring scheme corresponding with target element；
The start command of monitoring scheme is loaded into server corresponding with target element, to supervise to target element
Control.
The monitoring method of big data component disclosed in the above-mentioned embodiment illustrated in fig. 4 such as the application can be applied to processor
In, or realized by processor.Processor may be a kind of IC chip, the processing capacity with signal.It was realizing
Each step of Cheng Zhong, the above method can be complete by the integrated logic circuit of the hardware in processor or the instruction of software form
At.Above-mentioned processor can be general processor, including central processing unit (Central Processing Unit, CPU),
Network processing unit (Network Processor, NP) etc.；It can also be digital signal processor (Digital Signal
Processor, DSP), it is specific integrated circuit (Application Specific Integrated Circuit, ASIC), existing
Field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device are divided
Vertical door or transistor logic, discrete hardware components.It may be implemented or execute and is in the embodiment of the present application disclosed each
Method, step and logic diagram.General processor can be microprocessor or the processor is also possible to any conventional place
Manage device etc..The step of method in conjunction with disclosed in the embodiment of the present application, can be embodied directly in hardware decoding processor and execute
At, or in decoding processor hardware and software module combination execute completion.Software module can be located at random access memory,
This fields such as flash memory, read-only memory, programmable read only memory or electrically erasable programmable memory, register maturation
In storage medium.The storage medium is located at memory, and processor reads the information in memory, completes above-mentioned side in conjunction with its hardware
The step of method.
Certainly, other than software realization mode, other implementations are not precluded in the electronic equipment of the application, for example patrol
Collect device or the mode of software and hardware combining etc., that is to say, that the executing subject of following process flow is not limited to each patrol
Unit is collected, hardware or logical device are also possible to.
In addition, Fig. 4 further includes some unshowned functional modules, details are not described herein.
The embodiment of the present invention also provides a kind of computer readable storage medium, and meter is stored on computer readable storage medium
Calculation machine program, the computer program realize each mistake of the monitoring method embodiment of above-mentioned big data component when being executed by processor
Journey, and identical technical effect can be reached, to avoid repeating, which is not described herein again.Wherein, the computer-readable storage medium
Matter, such as read-only memory (Read-Only Memory, abbreviation ROM), random access memory (Random Access
Memory, abbreviation RAM), magnetic or disk etc..
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/or
The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium
Example.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including described want
There is also other identical elements in the process, method of element, commodity or equipment.
The above description is only an embodiment of the present invention, is not intended to restrict the invention.For those skilled in the art
For, the invention may be variously modified and varied.All any modifications made within the spirit and principles of the present invention are equal
Replacement, improvement etc., should be included within scope of the presently claimed invention.
Claims (10)
1. a kind of monitoring method of big data component characterized by comprising
It monitors and whether there is target element to be monitored on server；The server, for running the target to be monitored
Component；
If listening to there are target element to be monitored on the server, judgement can be whether there is in monitoring scheme list
The monitoring scheme to match with the target element, it is described to be used to store preconfigured big data group with monitoring scheme list
Part monitoring scheme；
If so, obtaining the monitoring scheme to match with the target element and loading, to be carried out to the target element
Monitoring.
2. the method as described in claim 1, which is characterized in that judgement can be whether there is and the mesh in monitoring scheme list
The monitoring scheme that mark component matches, specifically includes:
Obtain the relevant information of the target element；Wherein, the relevant information includes at least following one kind: the target element
The service name, address of service of the corresponding server of the target element, the corresponding Hostname of the target element, described
The corresponding port information of target element；
According to relevant information judgement can in monitoring scheme list with the presence or absence of matching with the target element
Monitoring scheme.
3. method according to claim 2, which is characterized in that the relevant information for obtaining the target element specifically includes:
The relevant information of the target element is obtained, and the service name of the target element in the relevant information is written
The component list；Wherein, the relevant information includes at least following one kind: the service name of the target element, the target group
The address of service of the corresponding server of part, the corresponding Hostname of the target element, the corresponding port letter of the target element
Breath；
It is described to be whether there is and the target element phase in monitoring scheme list according to relevant information judgement
The monitoring scheme matched, specifically includes:
Monitoring scheme list can be used described in traversal, can be whether there is and described the component list in monitoring scheme list described in judgement
The big data monitor component scheme to match.
4. method as claimed in claim 3, which is characterized in that can be whether there is and institute in monitoring scheme list described in judgement
The big data monitor component scheme that the component list matches is stated, is specifically included:
If described consistent with the service name of target element described in described the component list with existing in monitoring scheme list
The big data monitor component scheme can then be matched with existing in monitoring scheme list with described the component list described in judgement
The big data monitor component scheme.
5. the method as described in claim 1, which is characterized in that obtain the monitoring scheme corresponding with the target element simultaneously
Load, to be monitored to the target element, specifically includes:
Obtain the start command of the monitoring scheme corresponding with the target element；
The start command of the monitoring scheme is loaded into server corresponding with the target element, to the target element
It is monitored.
6. the method as described in claim 1, which is characterized in that further include:
After listening to target element completion monitoring, the monitoring scheme is deleted according to default script, to stop to the target
Component is monitored.
7. a kind of monitoring device of big data component characterized by comprising
Module is monitored, for monitoring on server with the presence or absence of target element to be monitored；The server, it is described for running
Target element to be monitored；
Judgment module, if judgement can use monitoring scheme for listening to there are target element to be monitored on the server
It is described to be used to store in advance with monitoring scheme list with the presence or absence of the monitoring scheme to match with the target element in list
The big data monitor component scheme of configuration；
Module is obtained, when being used for when described with there is monitoring scheme matched with the target element in monitoring scheme list,
It obtains the monitoring scheme corresponding with the target element and loads, to be monitored to the target element.
8. the device as shown in claim 7, which is characterized in that the judgment module is specifically used for:
Obtain the relevant information of the target element；Wherein, the relevant information include: the target element service name,
The address of service of the corresponding server of the target element, Hostname, port information；
It can be whether there is and the matched prison of the target element in monitoring scheme list according to relevant information judgement
Prosecutor case.
9. a kind of electronic equipment characterized by comprising memory, processor and be stored on the memory and can be in institute
The computer program run on processor is stated, such as claim 1~6 is realized when the computer program is executed by the processor
Any one of described in big data component monitoring method the step of.
10. a kind of computer readable storage medium, which is characterized in that be stored with computer on the computer readable storage medium
Program realizes the big data component as described in any one of claim 1~6 when the computer program is executed by processor
Monitoring method the step of.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910687478.9A CN110515795B (en) | 2019-07-29 | 2019-07-29 | Big data component monitoring method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910687478.9A CN110515795B (en) | 2019-07-29 | 2019-07-29 | Big data component monitoring method and device and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110515795A true CN110515795A (en) | 2019-11-29 |
| CN110515795B CN110515795B (en) | 2023-03-28 |
Family
ID=68624797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910687478.9A Active CN110515795B (en) | 2019-07-29 | 2019-07-29 | Big data component monitoring method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110515795B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563019A (en) * | 2020-04-29 | 2020-08-21 | 厦门市美亚柏科信息股份有限公司 | Service assembly monitoring method, system and computer storage medium |
| CN112217885A (en) * | 2020-09-27 | 2021-01-12 | 普联国际有限公司 | Dynamic management method, device, equipment and storage medium for components |
| CN114168431A (en) * | 2022-02-10 | 2022-03-11 | 北京金堤科技有限公司 | Method and apparatus for automatically monitoring service, electronic device and storage medium |
| CN114900444A (en) * | 2022-04-25 | 2022-08-12 | 北京奇艺世纪科技有限公司 | Data monitoring method and device, electronic equipment and storage medium |
| CN115134270A (en) * | 2022-06-28 | 2022-09-30 | 北京奇艺世纪科技有限公司 | Code monitoring method, monitoring system, electronic device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105005527A (en) * | 2015-05-26 | 2015-10-28 | 北京中亦安图科技股份有限公司 | Server-side product monitoring method and device |
| CN107704362A (en) * | 2017-09-25 | 2018-02-16 | 浪潮软件股份有限公司 | A kind of method and device based on Ambari monitoring big data components |
| CN108845916A (en) * | 2018-07-03 | 2018-11-20 | 中国联合网络通信集团有限公司 | Platform monitoring and alarm method, device, equipment and computer readable storage medium |
| CN109766241A (en) * | 2018-12-29 | 2019-05-17 | 中国银行股份有限公司 | System monitoring method, apparatus, computer equipment and computer readable storage medium |
-
2019
- 2019-07-29 CN CN201910687478.9A patent/CN110515795B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105005527A (en) * | 2015-05-26 | 2015-10-28 | 北京中亦安图科技股份有限公司 | Server-side product monitoring method and device |
| CN107704362A (en) * | 2017-09-25 | 2018-02-16 | 浪潮软件股份有限公司 | A kind of method and device based on Ambari monitoring big data components |
| CN108845916A (en) * | 2018-07-03 | 2018-11-20 | 中国联合网络通信集团有限公司 | Platform monitoring and alarm method, device, equipment and computer readable storage medium |
| CN109766241A (en) * | 2018-12-29 | 2019-05-17 | 中国银行股份有限公司 | System monitoring method, apparatus, computer equipment and computer readable storage medium |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111563019A (en) * | 2020-04-29 | 2020-08-21 | 厦门市美亚柏科信息股份有限公司 | Service assembly monitoring method, system and computer storage medium |
| CN112217885A (en) * | 2020-09-27 | 2021-01-12 | 普联国际有限公司 | Dynamic management method, device, equipment and storage medium for components |
| CN112217885B (en) * | 2020-09-27 | 2024-06-04 | 普联国际有限公司 | Dynamic management method, device, equipment and storage medium for components |
| CN114168431A (en) * | 2022-02-10 | 2022-03-11 | 北京金堤科技有限公司 | Method and apparatus for automatically monitoring service, electronic device and storage medium |
| CN114168431B (en) * | 2022-02-10 | 2022-04-15 | 北京金堤科技有限公司 | Method and apparatus for automatically monitoring service, electronic device and storage medium |
| CN114900444A (en) * | 2022-04-25 | 2022-08-12 | 北京奇艺世纪科技有限公司 | Data monitoring method and device, electronic equipment and storage medium |
| CN115134270A (en) * | 2022-06-28 | 2022-09-30 | 北京奇艺世纪科技有限公司 | Code monitoring method, monitoring system, electronic device and storage medium |
| CN115134270B (en) * | 2022-06-28 | 2023-09-08 | 北京奇艺世纪科技有限公司 | Code monitoring method, monitoring system, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110515795B (en) | 2023-03-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110515795A (en) | A kind of monitoring method of big data component, device, electronic equipment | |
| CN107193750B (en) | Script recording method and device | |
| US20180365085A1 (en) | Method and apparatus for monitoring client applications | |
| WO2019051948A1 (en) | Method, apparatus, server, and storage medium for processing monitoring data | |
| CN108134812B (en) | Data processing method and device | |
| CN112581018B (en) | Method, system, device and storage medium for managing process tasks | |
| CN108255620A (en) | A kind of business logic processing method, apparatus, service server and system | |
| CN107066519A (en) | A kind of task detection method and device | |
| CN110471754A (en) | Method for exhibiting data, device, equipment and storage medium in job scheduling | |
| CN107633015A (en) | A kind of data processing method, device and equipment | |
| CN113656245A (en) | Data inspection method and device, storage medium and processor | |
| CN110941632A (en) | Database auditing method, device and equipment | |
| CN112306411A (en) | Data storage method and device, nonvolatile storage medium and processor | |
| CN111061802A (en) | Power data management processing method and device and storage medium | |
| CN110266555A (en) | Method for analyzing web site service request | |
| CN104123303A (en) | Method and device for providing data | |
| CN111078418A (en) | Operation synchronization method and device, electronic equipment and computer readable storage medium | |
| CN107193721B (en) | Method and device for generating log | |
| CN107894942B (en) | Method and device for monitoring data table access amount | |
| CN112346952A (en) | Method, equipment and electronic equipment for regression testing of test cases | |
| CN114157662A (en) | Cloud platform parameter adaptation method and device, terminal equipment and storage medium | |
| CN109150993B (en) | Method for obtaining network request tangent plane, terminal device and storage medium | |
| CN113641628A (en) | Data quality detection method, device, equipment and storage medium | |
| CN113342806A (en) | Big data processing method and device, storage medium and processor | |
| CN116028108B (en) | Method, device, equipment and storage medium for analyzing dependent package installation time |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100081 No.101, 1st floor, building 14, 27 Jiancai Chengzhong Road, Haidian District, Beijing Applicant after: Beijing PERCENT Technology Group Co.,Ltd. Address before: 100081 16 / F, block a, Beichen Century Center, building 2, courtyard 8, Beichen West Road, Chaoyang District, Beijing Applicant before: BEIJING BAIFENDIAN INFORMATION SCIENCE & TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |