CN112486766A - Problem troubleshooting method, device and system of file-based monitoring platform - Google Patents
Problem troubleshooting method, device and system of file-based monitoring platform Download PDFInfo
- Publication number
- CN112486766A CN112486766A CN202011339138.6A CN202011339138A CN112486766A CN 112486766 A CN112486766 A CN 112486766A CN 202011339138 A CN202011339138 A CN 202011339138A CN 112486766 A CN112486766 A CN 112486766A
- Authority
- CN
- China
- Prior art keywords
- application
- target problem
- information
- target
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 78
- 238000013024 troubleshooting Methods 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000036541 health Effects 0.000 claims abstract description 61
- 238000003860 storage Methods 0.000 claims description 30
- 230000002688 persistence Effects 0.000 claims description 12
- 238000013519 translation Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000008901 benefit Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000004931 aggregating effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to a problem troubleshooting method, a device and a system of a file-based monitoring platform, which are used for analyzing pre-stored data reporting information and query application information corresponding to each application and determining target problems, application health information and analysis result information corresponding to each application; according to the analysis result information, positioning a target problem application corresponding to the target problem, determining target problem host data and target problem inlet flow in the target problem application, determining target problem outlet flow corresponding to the target problem inlet flow and target problem resource data in the target problem outlet flow; and performing link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data, and determining and outputting the reason of the target problem. According to the scheme, the monitoring platform can monitor the problems and can automatically troubleshoot the reasons caused by the problems, and the accuracy of problem troubleshooting and the practicability of the monitoring platform are improved.
Description
Technical Field
The invention relates to the technical field of monitoring, in particular to a problem troubleshooting method, device and system of a monitoring platform based on files.
Background
In the prior art, various monitoring platforms are diversified, but the monitoring platform in the prior art can only realize monitoring and problem alarming, but cannot find out the cause of the problem, mainly relying on experienced workers to analyze and find the problem, and for a new person who enters a workplace or a person who is not familiar with the application of the monitoring platform, the problem cannot be accurately analyzed and found.
Therefore, how to implement the problem troubleshooting work of the monitoring platform, and improve the accuracy of the problem troubleshooting and the practicability of the monitoring platform are technical problems that the technical personnel in the field need to solve urgently.
Disclosure of Invention
In view of the above, the present invention provides a problem troubleshooting method, device and system for a file-based monitoring platform, so as to solve the problems that in the prior art, a worker with a rich experience is mainly relied on to perform problem analysis and troubleshooting, the problem cannot be automatically troubleshot, the accuracy of problem troubleshooting is low, and the practicability of the monitoring platform is low.
In order to achieve the purpose, the invention adopts the following technical scheme:
a problem troubleshooting method of a file-based monitoring platform comprises the following steps:
acquiring pre-stored data reporting information and query application information corresponding to each application;
analyzing the data reporting information and the query application information, and determining a target problem corresponding to each application, application health information corresponding to each application and analysis result information corresponding to each application;
positioning target problem applications corresponding to the target problems according to analysis result information corresponding to each application;
determining target problem host data and target problem inlet flow corresponding to the target problem in the target problem application according to the analysis result information;
determining target problem outlet flow corresponding to the target problem inlet flow and target problem resource data corresponding to the target problem in the target problem outlet flow according to the analysis result information;
and performing link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data corresponding to the target problem, determining a target problem reason corresponding to the target problem, and outputting the target problem and the target problem reason.
Further, in the method for troubleshooting a problem of the file-based monitoring platform, after analyzing the data reporting information and the query application information and determining a target problem corresponding to each application, application health information corresponding to each application, and analysis result information corresponding to each application, the method further includes:
analyzing the application health information, and judging whether the application health information has a target problem reason of the application corresponding to the application health information;
if the target problem reason of the application corresponding to the application health information exists in the application health information, outputting the target problem and the target problem reason corresponding to the application;
correspondingly, the positioning the target problem application corresponding to the target problem according to the analysis result information corresponding to each application includes:
and if the target problem reason of the application corresponding to the application health information does not exist in the application health information, positioning the target problem application corresponding to the target problem according to the analysis result information corresponding to each application.
Further, in the method for troubleshooting problems on a file-based monitoring platform, before the obtaining of the pre-stored data reporting information corresponding to each application and the query of the application information, the method further includes:
receiving initial reported data sent by a client;
converting the initial reporting data into standard reporting data of a standard protocol;
writing the standard reported data into a message queue in a memory so that a report in the memory performs message consumption on the standard reported data, and storing the standard reported data into a report engine in the memory;
and performing data persistence on the standard report data in the report engine to generate data report information, and storing the data report information to a file storage engine.
Further, in the method for troubleshooting problems on a file-based monitoring platform, before the obtaining of the pre-stored data reporting information corresponding to each application and the query of the application information, the method further includes:
receiving initial query information sent by the client;
performing SQL statement translation on the initial query information to generate an SQL query statement corresponding to the initial query information;
analyzing the SQL query statement by using an SQL engine to generate target query information corresponding to the initial query information;
according to the target query information, querying a target query result corresponding to the target query information from the report engine or the file storage engine;
and generating the query application information according to the initial query information, the target query information and the target query result.
Further, in the method for troubleshooting problems of the file-based monitoring platform, the step of storing the standard report data in a report engine in the memory includes:
and storing the standard reported data into a report engine in the memory according to a preset storage level and a preset minute storage mode.
The invention also provides a problem troubleshooting device of the file-based monitoring platform, which comprises the following components:
the acquisition module is used for acquiring pre-stored data reporting information and query application information corresponding to each application;
the analysis module is used for analyzing the data reporting information and the query application information and determining a target problem corresponding to each application, application health information corresponding to each application and analysis result information corresponding to each application;
the positioning module is used for positioning the target problem application corresponding to the target problem according to the analysis result information corresponding to each application;
a first determining module, configured to determine, according to the analysis result information, target problem host data and target problem inlet traffic corresponding to the target problem in the target problem application;
a second determining module, configured to determine, according to the analysis result information, a target problem outlet flow corresponding to the target problem inlet flow and target problem resource data corresponding to the target problem in the target problem outlet flow;
and the link tracking module is used for performing link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data corresponding to the target problem, determining a target problem reason corresponding to the target problem and outputting the target problem and the target problem reason.
Further, the problem troubleshooting device of the file-based monitoring platform further comprises: the device comprises a judging module and an output module;
the judging module is used for analyzing the application health information and judging whether the application health information has a target problem reason of the application corresponding to the application health information;
the output module is configured to output the target problem and the target problem reason corresponding to the application if the target problem reason of the application corresponding to the application health information exists in the application health information;
the positioning module is specifically configured to, if the target problem reason of the application corresponding to the application health information does not exist in the application health information, position the target problem application corresponding to the target problem according to analysis result information corresponding to each application.
Further, the problem troubleshooting device of the file-based monitoring platform further comprises: the device comprises a receiving module, a conversion module, a writing module and a data persistence module;
the receiving module is used for receiving initial reported data sent by a client;
the conversion module is used for converting the initial reporting data into standard reporting data of a standard protocol;
the writing module is used for writing the standard reported data into a message queue in a memory so that a report in the memory performs message consumption on the standard reported data, and storing the standard reported data into a report engine in the memory;
the data persistence module is used for performing data persistence on the standard report data in the report engine, generating data report information, and storing the data report information to a file storage engine.
Further, the problem troubleshooting device of the file-based monitoring platform further comprises: the system comprises a translation module, an analysis module, a query module and a generation module;
the receiving module is further configured to receive initial query information sent by the client;
the analysis module is used for carrying out SQL statement translation on the initial query information to generate an SQL query statement corresponding to the initial query information;
the translation module is used for analyzing the SQL query statement by utilizing an SQL engine to generate target query information corresponding to the initial query information;
the query module is used for querying a target query result corresponding to the target query information from the report engine or the file storage engine according to the target query information;
and the generating module is used for generating the query application information according to the initial query information, the target query information and the target query result.
The invention also provides a problem troubleshooting system of the file-based monitoring platform, which comprises the following components: the system comprises a processor and a memory connected with the processor;
the memory is used for storing a computer program, and the computer program is at least used for executing the problem troubleshooting method of the file-based monitoring platform;
the processor is used for calling and executing the computer program.
A problem troubleshooting method, device and system of a monitoring platform based on files are disclosed, which are used for acquiring pre-stored data reporting information and query application information corresponding to each application; analyzing the data reporting information and the query application information, and determining a target problem corresponding to each application, application health information corresponding to each application and analysis result information corresponding to each application; positioning target problem applications corresponding to the target problems according to the analysis result information corresponding to each application; determining target problem host data and target problem inlet flow corresponding to a target problem in the target problem application according to the analysis result information; determining target problem outlet flow corresponding to the target problem inlet flow and target problem resource data corresponding to the target problem in the target problem outlet flow according to the analysis result information; and performing link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data corresponding to the target problem, determining a target problem reason corresponding to the target problem, and outputting the target problem and the target problem reason. By adopting the technical scheme of the invention, the monitoring platform can monitor the problems and can automatically find out the reasons of the problems without the need of analyzing and checking by workers according to experience, and the reasons of the problems can be timely determined for new people or people who are not familiar with the monitoring platform, so that the accuracy of problem checking and the practicability of the monitoring platform are improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart provided by one embodiment of a method for troubleshooting problems with a file-based monitoring platform of the present invention;
FIG. 2 is a flow chart provided by another embodiment of a method for troubleshooting problems with a file-based monitoring platform of the present invention;
FIG. 3 is a flow chart provided by another embodiment of a method for troubleshooting problems with a file-based monitoring platform of the present invention;
FIG. 4 is a schematic structural diagram provided by an embodiment of the problem troubleshooting apparatus of the file-based monitoring platform of the present invention;
FIG. 5 is a schematic diagram of a problem troubleshooting system of a file-based monitoring platform according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the examples given herein without any inventive step, are within the scope of the present invention.
Fig. 1 is a flowchart provided in an embodiment of the method for troubleshooting problems of a file-based monitoring platform according to the present invention, and as shown in fig. 1, the method for troubleshooting problems of a file-based monitoring platform according to the present embodiment specifically includes the following steps:
s101, acquiring pre-stored data reporting information and query application information corresponding to each application;
in this embodiment, data reporting information and query application information corresponding to each application are stored in advance, and therefore, the data reporting information and the query application information corresponding to each application need to be obtained first, where the storage of the data reporting information and the query application information is preferably local storage. By adopting local storage, the monitoring process does not need to rely on a big data middleware, thereby improving the data transmission efficiency, reducing the overhead of a monitoring platform and having obvious real-time advantage and cost advantage.
S102, analyzing the data reporting information and the query application information, and determining a target problem corresponding to each application, application health information corresponding to each application and analysis result information corresponding to each application;
after the data reporting information and the query application information of each application are obtained, the data reporting information and the query application information need to be analyzed, so that a target problem corresponding to each application, application health information corresponding to each application and analysis result information corresponding to each application are determined. Wherein, the analysis result information corresponding to each application comprises: and the application overview data, the application core data, the IP core data and the like corresponding to each application.
S103, positioning target problem applications corresponding to the target problems according to analysis result information corresponding to each application;
and positioning the application generating the target problem according to the determined analysis result information corresponding to each application, and determining the target problem application corresponding to the target problem. In this embodiment, a plurality of target problems may be determined, and a target problem application corresponding to each target problem needs to be located.
S104, determining target problem host data and target problem inlet flow corresponding to a target problem in the target problem application according to the analysis result information;
and determining target problem host data and target problem inlet traffic corresponding to the target problem in the target problem application according to the determined analysis result information corresponding to each application, such as application overview data, application core data, IP core data and the like corresponding to each application in the analysis result information. The inlet flow comprises interface calling data, calling normal distribution and calling data. The target problem inlet flow is an inlet flow corresponding to the target problem inquired from the inlet flows.
S105, determining target problem outlet flow corresponding to the target problem inlet flow and target problem resource data corresponding to the target problem in the target problem outlet flow according to the analysis result information;
the outlet flow rate includes: downstream service resources, MySQL resources, ES resources, MQ resources, HBase resources, custom resources and the like. And determining target problem outlet flow corresponding to the target problem inlet flow according to the analysis result information, namely positioning the target problem outlet flow corresponding to the target problem from outlet flows such as downstream service resources, MySQL resources, ES resources, MQ resources, HBase resources, custom resources and the like corresponding to the target problem inlet flow. After the target problem outlet flow is determined, the target problem resource data corresponding to the target problem outlet flow is determined.
S106, performing link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data corresponding to the target problem, determining a target problem reason corresponding to the target problem, and outputting the target problem and the target problem reason.
By the steps, the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data corresponding to the target problem can be obtained, and the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data can be used for realizing link tracking of the target problem, so that the scene recovery corresponding to the target problem is realized, and the target problem reason corresponding to the target problem can be determined through the link tracking of the target problem. The link tracking needs to call a link, message sampling and problem sampling, so that the link tracking of the target problem is completed. And after the target problem reason corresponding to the target problem is determined, outputting the target problem and the target problem reason to enable a worker to check the target problem and correct the target problem in time according to the target problem reason.
In each application of this embodiment, a plurality of target problems may be determined, and therefore, a target problem application, target problem host data, target problem entry traffic, target problem exit traffic, and target problem resource data need to be located for each target problem, so that link tracking is performed on each target problem, and a target problem cause corresponding to each target problem is never determined.
The problem troubleshooting method for the file-based monitoring platform of the embodiment acquires pre-stored data reporting information and query application information corresponding to each application; analyzing the data reporting information and the query application information, and determining a target problem corresponding to each application, application health information corresponding to each application and analysis result information corresponding to each application; positioning target problem applications corresponding to the target problems according to the analysis result information corresponding to each application; determining target problem host data and target problem inlet flow corresponding to a target problem in the target problem application according to the analysis result information; determining target problem outlet flow corresponding to the target problem inlet flow and target problem resource data corresponding to the target problem in the target problem outlet flow according to the analysis result information; and performing link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data corresponding to the target problem, determining a target problem reason corresponding to the target problem, and outputting the target problem and the target problem reason. By adopting the technical scheme of the embodiment, the monitoring platform can not only monitor the problems, but also automatically troubleshoot the reasons of the problems, workers are not required to analyze and troubleshoot according to experience, and the reasons of the problems can be timely determined for new people or people who are not familiar with the monitoring platform, so that the accuracy of problem troubleshooting and the practicability of the monitoring platform are improved.
Further, after the step S102 is executed, the method for troubleshooting a problem of the file-based monitoring platform of the embodiment further includes the following steps:
firstly, analyzing the application health information, and judging whether the application health information has a target problem reason of an application corresponding to the application health information;
the target problem cause corresponding to the target problem may be directly displayed in the application health information determined through the above steps, and therefore, before the target problem cause is located, it is necessary to determine whether the target problem cause of the application corresponding to the application health information exists in the application health information.
Secondly, if the target problem reason of the application corresponding to the application health information exists in the application health information, outputting the target problem and the target problem reason corresponding to the application;
and if the target problem reason of the application corresponding to the application health information exists in the application health information is determined through the judgment, directly outputting the target problem and the target problem reason of the application to enable a worker to check the target problem and correct the target problem in time according to the target problem reason.
Correspondingly, step S103 specifically includes: and if the target problem reason of the application corresponding to the application health information does not exist in the application health information, positioning the target problem application corresponding to the target problem according to the analysis result information corresponding to each application.
Further, fig. 2 is a flowchart provided by another embodiment of the method for troubleshooting problems of the file-based monitoring platform of the present invention, and as shown in fig. 2, the method for troubleshooting problems of the file-based monitoring platform of the present embodiment further includes the following steps before executing step S101:
s201, receiving initial reported data sent by a client;
the client may report data through an HTTP protocol or a TCP protocol with better performance, and therefore, this embodiment needs to receive initial report data sent by the client.
S202, converting the initial reported data into standard reported data of a standard protocol;
s203, writing the standard reported data into a message queue in the memory so that the report in the memory performs message consumption on the standard reported data, and storing the standard reported data into a report engine in the memory;
after the standard report data is generated, the standard report data needs to be written into a message queue in the memory, so that the report in the memory performs message consumption on the standard report data, and the processed standard report data is stored in a report engine in the memory. The memory is preferably the memory of the local disk. The report forms for performing message consumption on the standard reported data in the embodiment include a heartbeat report form, an index report form, an event report form, a link report form, a problem report form, other report forms, and the like.
Further, the standard report data is stored in a report engine in the memory, and the report engine is specifically required to be stored according to a preset storage level and a minute storage mode. The storage levels comprise four levels of application, IP, index group and index name, and each level has independent summarized data. And the storage is carried out according to a minute storage mode, so that the minute-level query granularity at any time can be supported.
S204, performing data persistence on the standard report data in the report engine to generate data report information, and storing the data report information to the file storage engine.
Further, fig. 3 is a flowchart provided by another embodiment of the method for troubleshooting problems of the file-based monitoring platform of the present invention, and as shown in fig. 3, the method for troubleshooting problems of the file-based monitoring platform of the present embodiment further includes the following steps before executing step S101:
s301, receiving initial query information sent by a client;
when a user needs to query some information, the client is used to send initial query information containing query conditions, so that the embodiment needs to receive the initial query information sent by the client.
S302, SQL sentence translation is carried out on the initial query information to generate an SQL query sentence corresponding to the initial query information;
after the initial query information is received, the initial query information needs to be translated into an SQL statement as the SQL query statement corresponding to the initial query information.
S303, analyzing the SQL query statement by using the SQL engine to generate target query information corresponding to the initial query information;
and analyzing the generated SQL query statement by using an SQL engine, generating target query information corresponding to the initial query information, for example, translating into which report form needs to be queried, screening what the condition is, aggregating according to what dimension, such as RPC request query times, and aggregating according to IP.
S304, according to the target query information, querying a target query result corresponding to the target query information from a report engine or a file storage engine;
s305, generating query application information according to the initial query information, the target query information and the target query result.
According to the above steps, after the initial query information, the target query information and the target query result are determined, the initial query information, the target query information and the target query result can be collected, and the collected information is used as query application information.
Compared with Skywalk, the monitoring platform adopting the technical scheme of the embodiment does not depend on a large data middleware and an ES storage engine, all data are calculated and stored in a local disk in real time, and the monitoring platform has cost advantage in production; compared with CAT, the method has better read-write performance, more friendly use mode, no invasion of SDK and no perception access of users; compared with the conventional monitoring system in the industry, a relatively clear debugging thought is provided and is embodied in the system function, and a user can gradually judge the cause of the problem according to page guidance and analyze the application health condition.
In order to be more comprehensive, the application also provides a problem troubleshooting device of the file-based monitoring platform, which corresponds to the problem troubleshooting method of the file-based monitoring platform provided by the embodiment of the invention.
Fig. 4 is a schematic structural diagram provided by an embodiment of the problem troubleshooting apparatus for a file-based monitoring platform according to the present invention, and as shown in fig. 4, the problem troubleshooting apparatus for a file-based monitoring platform according to the present embodiment includes: an acquisition module 101, an analysis module 102, a positioning module 103, a first determination module 104, a second determination module 105, and a link tracking module 106.
An obtaining module 101, configured to obtain pre-stored data reporting information and query application information corresponding to each application;
the analysis module 102 is configured to analyze the data reporting information and the query application information, and determine a target problem corresponding to each application, application health information corresponding to each application, and analysis result information corresponding to each application;
the positioning module 103 is configured to position a target problem application corresponding to a target problem according to analysis result information corresponding to each application;
a first determining module 104, configured to determine, according to the analysis result information, target problem host data and target problem inlet traffic corresponding to a target problem in the target problem application;
a second determining module 105, configured to determine, according to the analysis result information, a target problem outlet flow corresponding to the target problem inlet flow and target problem resource data corresponding to the target problem in the target problem outlet flow;
and the link tracking module 106 is configured to perform link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet traffic, the target problem outlet traffic, and the target problem resource data corresponding to the target problem, determine a target problem cause corresponding to the target problem, and output the target problem and the target problem cause.
In the problem troubleshooting device of the file-based monitoring platform according to this embodiment, the obtaining module 101 obtains pre-stored data reporting information and query application information corresponding to each application; the analysis module 102 analyzes the data reporting information and the query application information, and determines a target problem corresponding to each application, application health information corresponding to each application, and analysis result information corresponding to each application; the positioning module 103 positions the target problem application corresponding to the target problem according to the analysis result information corresponding to each application; the first determining module 104 determines target problem host data and target problem inlet traffic corresponding to a target problem in the target problem application according to the analysis result information; the second determining module 105 determines target problem outlet traffic corresponding to the target problem inlet traffic and target problem resource data corresponding to the target problem in the target problem outlet traffic according to the analysis result information; the link tracking module 106 performs link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet traffic, the target problem outlet traffic, and the target problem resource data corresponding to the target problem, determines a target problem cause corresponding to the target problem, and outputs the target problem and the target problem cause. By adopting the technical scheme of the embodiment, the monitoring platform can not only monitor the problems, but also automatically troubleshoot the reasons of the problems, workers are not required to analyze and troubleshoot according to experience, and the reasons of the problems can be timely determined for new people or people who are not familiar with the monitoring platform, so that the accuracy of problem troubleshooting and the practicability of the monitoring platform are improved.
Further, the problem troubleshooting device of the file-based monitoring platform of the embodiment further comprises a judging module and an output module.
The judging module is used for analyzing the application health information and judging whether the application health information has an application target problem reason corresponding to the application health information;
the output module is used for outputting the target problem and the target problem reason corresponding to the application if the target problem reason of the application corresponding to the application health information exists in the application health information;
the positioning module 103 is specifically configured to, if there is no target problem reason of the application corresponding to the application health information in the application health information, position a target problem application corresponding to a target problem according to analysis result information corresponding to each application.
Further, the problem troubleshooting device of the file-based monitoring platform of the embodiment further includes: the device comprises a receiving module, a conversion module, a writing module and a data persistence module.
The receiving module is used for receiving initial reported data sent by the client;
the conversion module is used for converting the initial reported data into standard reported data of a standard protocol;
the writing module is used for writing the standard reported data into a message queue in the memory so that the report in the memory performs message consumption on the standard reported data, and storing the standard reported data into a report engine in the memory; the method for storing the standard reported data into the report engine in the memory comprises the following steps: and storing the standard reported data into a report engine in the memory according to a preset storage level and a minute storage mode.
And the data persistence module is used for performing data persistence on the standard report data in the report engine, generating data report information and storing the data report information to the file storage engine.
Further, the problem troubleshooting device of the file-based monitoring platform of the embodiment further includes: the device comprises a translation module, an analysis module, a query module and a generation module.
The receiving module is also used for receiving initial query information sent by the client;
the analysis module is used for carrying out SQL statement translation on the initial query information to generate an SQL query statement corresponding to the initial query information;
the translation module is used for analyzing the SQL query statement by utilizing the SQL engine and generating target query information corresponding to the initial query information;
the query module is used for querying a target query result corresponding to the target query information from the report engine or the file storage engine according to the target query information;
and the generating module is used for generating query application information according to the initial query information, the target query information and the target query result.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
FIG. 5 is a schematic diagram of a problem troubleshooting system of a file-based monitoring platform according to an embodiment of the present invention. As shown in fig. 5, the problem troubleshooting system of the file-based monitoring platform of the present embodiment includes a processor 21 and a memory 22 connected to the processor 21;
the memory 22 is used for storing a computer program at least for executing the problem troubleshooting method of the file-based monitoring platform of the above embodiment;
the processor 21 is used to call and execute the computer program.
According to the problem troubleshooting system based on the file monitoring platform, the monitoring platform can monitor problems and automatically troubleshoot reasons generated by the problems, workers do not need to analyze and troubleshoot according to experience, the reasons generated by the problems can be timely determined for new people or people unfamiliar with the monitoring platform, and the accuracy of problem troubleshooting and the practicability of the monitoring platform are improved.
It is understood that the same or similar parts in the above embodiments may be mutually referred to, and the same or similar parts in other embodiments may be referred to for the content which is not described in detail in some embodiments.
It should be noted that the terms "first," "second," and the like in the description of the present invention are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. Further, in the description of the present invention, the meaning of "a plurality" means at least two unless otherwise specified.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A problem troubleshooting method of a file-based monitoring platform is characterized by comprising the following steps:
acquiring pre-stored data reporting information and query application information corresponding to each application;
analyzing the data reporting information and the query application information, and determining a target problem corresponding to each application, application health information corresponding to each application and analysis result information corresponding to each application;
positioning target problem applications corresponding to the target problems according to analysis result information corresponding to each application;
determining target problem host data and target problem inlet flow corresponding to the target problem in the target problem application according to the analysis result information;
determining target problem outlet flow corresponding to the target problem inlet flow and target problem resource data corresponding to the target problem in the target problem outlet flow according to the analysis result information;
and performing link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data corresponding to the target problem, determining a target problem reason corresponding to the target problem, and outputting the target problem and the target problem reason.
2. The method of claim 1, wherein after analyzing the data reporting information and the query application information and determining the target problem corresponding to each application, the application health information corresponding to each application, and the analysis result information corresponding to each application, the method further comprises:
analyzing the application health information, and judging whether the application health information has a target problem reason of the application corresponding to the application health information;
if the target problem reason of the application corresponding to the application health information exists in the application health information, outputting the target problem and the target problem reason corresponding to the application;
correspondingly, the positioning the target problem application corresponding to the target problem according to the analysis result information corresponding to each application includes:
and if the target problem reason of the application corresponding to the application health information does not exist in the application health information, positioning the target problem application corresponding to the target problem according to the analysis result information corresponding to each application.
3. The method for troubleshooting a problem of a file-based monitoring platform according to claim 1, wherein before the obtaining the pre-stored data reporting information and the query application information corresponding to each application, further comprising:
receiving initial reported data sent by a client;
converting the initial reporting data into standard reporting data of a standard protocol;
writing the standard reported data into a message queue in a memory so that a report in the memory performs message consumption on the standard reported data, and storing the standard reported data into a report engine in the memory;
and performing data persistence on the standard report data in the report engine to generate data report information, and storing the data report information to a file storage engine.
4. The method for troubleshooting a problem of a file-based monitoring platform according to claim 3, wherein before the obtaining the pre-stored data reporting information and the query application information corresponding to each application, further comprising:
receiving initial query information sent by the client;
performing SQL statement translation on the initial query information to generate an SQL query statement corresponding to the initial query information;
analyzing the SQL query statement by using an SQL engine to generate target query information corresponding to the initial query information;
according to the target query information, querying a target query result corresponding to the target query information from the report engine or the file storage engine;
and generating the query application information according to the initial query information, the target query information and the target query result.
5. The method as claimed in claim 3, wherein the step of storing the standard report data in a report engine in the memory comprises:
and storing the standard reported data into a report engine in the memory according to a preset storage level and a preset minute storage mode.
6. A problem troubleshooting device of a file-based monitoring platform is characterized by comprising:
the acquisition module is used for acquiring pre-stored data reporting information and query application information corresponding to each application;
the analysis module is used for analyzing the data reporting information and the query application information and determining a target problem corresponding to each application, application health information corresponding to each application and analysis result information corresponding to each application;
the positioning module is used for positioning the target problem application corresponding to the target problem according to the analysis result information corresponding to each application;
a first determining module, configured to determine, according to the analysis result information, target problem host data and target problem inlet traffic corresponding to the target problem in the target problem application;
a second determining module, configured to determine, according to the analysis result information, a target problem outlet flow corresponding to the target problem inlet flow and target problem resource data corresponding to the target problem in the target problem outlet flow;
and the link tracking module is used for performing link tracking on the target problem by using the target problem application, the target problem host data, the target problem inlet flow, the target problem outlet flow and the target problem resource data corresponding to the target problem, determining a target problem reason corresponding to the target problem and outputting the target problem and the target problem reason.
7. The file-based monitoring platform troubleshooting apparatus of claim 6 further comprising: the device comprises a judging module and an output module;
the judging module is used for analyzing the application health information and judging whether the application health information has a target problem reason of the application corresponding to the application health information;
the output module is configured to output the target problem and the target problem reason corresponding to the application if the target problem reason of the application corresponding to the application health information exists in the application health information;
the positioning module is specifically configured to, if the target problem reason of the application corresponding to the application health information does not exist in the application health information, position the target problem application corresponding to the target problem according to analysis result information corresponding to each application.
8. The file-based monitoring platform troubleshooting apparatus of claim 6 further comprising: the device comprises a receiving module, a conversion module, a writing module and a data persistence module;
the receiving module is used for receiving initial reported data sent by a client;
the conversion module is used for converting the initial reporting data into standard reporting data of a standard protocol;
the writing module is used for writing the standard reported data into a message queue in a memory so that a report in the memory performs message consumption on the standard reported data, and storing the standard reported data into a report engine in the memory;
the data persistence module is used for performing data persistence on the standard report data in the report engine, generating data report information, and storing the data report information to a file storage engine.
9. The file-based monitoring platform troubleshooting apparatus of claim 8 further comprising: the system comprises a translation module, an analysis module, a query module and a generation module;
the receiving module is further configured to receive initial query information sent by the client;
the analysis module is used for carrying out SQL statement translation on the initial query information to generate an SQL query statement corresponding to the initial query information;
the translation module is used for analyzing the SQL query statement by utilizing an SQL engine to generate target query information corresponding to the initial query information;
the query module is used for querying a target query result corresponding to the target query information from the report engine or the file storage engine according to the target query information;
and the generating module is used for generating the query application information according to the initial query information, the target query information and the target query result.
10. A problem troubleshooting system for a file-based monitoring platform, comprising: the system comprises a processor and a memory connected with the processor;
the memory is used for storing a computer program at least for executing the problem troubleshooting method of the file-based monitoring platform according to any one of claims 1 to 5;
the processor is used for calling and executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011339138.6A CN112486766B (en) | 2020-11-25 | 2020-11-25 | File-based problem investigation method, device and system for monitoring platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011339138.6A CN112486766B (en) | 2020-11-25 | 2020-11-25 | File-based problem investigation method, device and system for monitoring platform |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112486766A true CN112486766A (en) | 2021-03-12 |
| CN112486766B CN112486766B (en) | 2024-08-09 |
Family
ID=74934723
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011339138.6A Active CN112486766B (en) | 2020-11-25 | 2020-11-25 | File-based problem investigation method, device and system for monitoring platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112486766B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103532739A (en) * | 2013-09-25 | 2014-01-22 | 上海斐讯数据通信技术有限公司 | Monitoring analysis system based on network service and application |
| CN106230939A (en) * | 2016-07-28 | 2016-12-14 | 税友软件集团股份有限公司 | A kind of application operation and monitoring method and system |
| CN109522287A (en) * | 2018-09-18 | 2019-03-26 | 平安科技（深圳）有限公司 | Monitoring method, system, equipment and the medium of distributed document storage cluster |
| US20200133814A1 (en) * | 2018-10-25 | 2020-04-30 | Capital One Services, Llc | Application performance analytics platform |
| CN111949705A (en) * | 2020-07-31 | 2020-11-17 | 上海中通吉网络技术有限公司 | Slow query statistical analysis method, device and system |
-
2020
- 2020-11-25 CN CN202011339138.6A patent/CN112486766B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103532739A (en) * | 2013-09-25 | 2014-01-22 | 上海斐讯数据通信技术有限公司 | Monitoring analysis system based on network service and application |
| CN106230939A (en) * | 2016-07-28 | 2016-12-14 | 税友软件集团股份有限公司 | A kind of application operation and monitoring method and system |
| CN109522287A (en) * | 2018-09-18 | 2019-03-26 | 平安科技（深圳）有限公司 | Monitoring method, system, equipment and the medium of distributed document storage cluster |
| US20200133814A1 (en) * | 2018-10-25 | 2020-04-30 | Capital One Services, Llc | Application performance analytics platform |
| CN111949705A (en) * | 2020-07-31 | 2020-11-17 | 上海中通吉网络技术有限公司 | Slow query statistical analysis method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112486766B (en) | 2024-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106301971B (en) | electric power application performance monitoring system based on flow analysis | |
| CN107273280B (en) | Log processing method and device, electronic equipment and storage medium | |
| CN111274095B (en) | Log data processing method, device, equipment and computer readable storage medium | |
| CN106027328B (en) | Cluster monitoring method and system based on application container deployment | |
| CN111459782B (en) | Method and device for monitoring service system, cloud platform system and server | |
| CN107704387B (en) | Method, device, electronic equipment and computer readable medium for system early warning | |
| CN110417575A (en) | Alarm method, device and the computer equipment of O&M monitor supervision platform | |
| Picoreti et al. | Multilevel observability in cloud orchestration | |
| CN102143507A (en) | Method and system for monitoring service quality, and analytical method and system therefor | |
| US8631280B2 (en) | Method of measuring and diagnosing misbehaviors of software components and resources | |
| CN112311617A (en) | Configured data monitoring and alarming method and system | |
| WO2018082340A1 (en) | Simulation test system and method | |
| CN101707632A (en) | Method for dynamically monitoring performance of server cluster and alarming real-timely | |
| CN112615742A (en) | Method, device, equipment and storage medium for early warning | |
| CN111124830B (en) | Micro-service monitoring method and device | |
| CN115034638A (en) | Digital twinning processing method and digital twinning system | |
| WO2021120544A1 (en) | Method and apparatus for debugging device | |
| CN111045911A (en) | Performance test method, performance test device, storage medium and electronic equipment | |
| CN113596078B (en) | Service problem positioning method and device | |
| CN112994972B (en) | Distributed probe monitoring platform | |
| CN109995555A (en) | Monitoring method, device, equipment and medium | |
| CN114039900A (en) | Efficient network data packet protocol analysis method and system | |
| CN110535972B (en) | Centralized control and communication system, equipment and readable storage medium for platform gas detection equipment | |
| CN113032218B (en) | Server fault detection method, system and computer readable storage medium | |
| CN114238330A (en) | Data processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |