CN111221702B - Log analysis-based exception handling method, system, terminal and medium - Google Patents
Log analysis-based exception handling method, system, terminal and medium Download PDFInfo
- Publication number
- CN111221702B CN111221702B CN201911127824.4A CN201911127824A CN111221702B CN 111221702 B CN111221702 B CN 111221702B CN 201911127824 A CN201911127824 A CN 201911127824A CN 111221702 B CN111221702 B CN 111221702B
- Authority
- CN
- China
- Prior art keywords
- log
- index
- analysis
- alarm
- indexes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000005856 abnormality Effects 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 7
- 238000010219 correlation analysis Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 abstract description 12
- 238000012544 monitoring process Methods 0.000 abstract description 10
- 230000002159 abnormal effect Effects 0.000 abstract description 5
- 230000004044 response Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000004393 prognosis Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
- G06F11/3082—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting the data filtering being achieved by aggregating or compressing the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides an exception handling method based on log analysis, which comprises the following steps: reading a log stored in a database; analyzing the log according to a preset log template to obtain an analysis log; extracting corresponding indexes in the analysis log according to preset log indexes; and when the index abnormality is detected, alarming. According to the method, the log can be analyzed, automatic abnormal data processing is realized by combining the log according to the abnormal condition of the operation and maintenance of the log analysis system, and the timeliness of the operation and maintenance monitoring of the system is improved.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to an exception handling method, system, terminal and medium based on log analysis.
Background
The system operation and maintenance monitoring mainly collects various monitoring indexes through various technical tools such as Tivoli, zabbix, APM, network packet capturing, application embedded point monitoring and the like, so as to monitor the system operation and maintenance. These metrics are typically stored as time series data (i.e., containing acquisition time and metric values).
Because the monitoring data is recorded in a log mode when the daily system is operated and maintained, a worker knows the operation condition of the system by checking the corresponding log, and when the system is abnormal, the abnormal condition is processed in time, but the abnormal processing method has lower efficiency, needs manual checking by the worker, and has poor instantaneity.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides an exception handling method, a system, a terminal and a medium based on log analysis, which can realize automatic exception data handling by combining with logs and improve the timeliness of system operation and maintenance monitoring.
In a first aspect, an exception handling method based on log analysis includes the steps of:
reading a log stored in a database;
analyzing the log according to a preset log template to obtain an analysis log;
extracting corresponding indexes in the analysis log according to preset log indexes;
and when the index abnormality is detected, alarming.
Preferably, the log includes a system log and an application log.
Preferably, the setting method of the log template comprises the following steps:
obtaining a log sample from an internal memory or network;
performing multi-row combination according to the head-of-line matching mode in the log sample to obtain a combination template;
extracting log indexes in the merging module through a regular expression to obtain an extraction template;
defining the extraction template as the log template.
Preferably, the categories of the log index include one or more combinations of:
application transaction classes, middleware classes, database classes, operating system classes, network classes, server classes, security device classes, and storage device classes.
Preferably, when the index abnormality is detected, the alerting specifically includes:
when the index extracted from the analysis log meets the preset alarm rule, alarming is carried out;
and acquiring the associated alarm associated with the alarm, and compressing the associated alarm.
Preferably, the method further comprises, after the alerting when the index anomaly is detected:
establishing a prediction model according to the linear relation between the index extracted from the analysis log and a preset standard index;
and carrying the analysis log into the prediction model, and predicting the index utilization rate of each time node according to time, wherein the index utilization rate of each time node exceeds the standard.
In a second aspect, an exception handling system based on log analysis includes:
the acquisition unit: for reading a log stored in a database;
an analysis unit: the method comprises the steps of analyzing the log according to a preset log template to obtain an analysis log;
extraction unit: extracting corresponding indexes in the analysis log according to preset log indexes;
and a processing unit: and the alarm is used for alarming when the index abnormality is detected.
In a third aspect, a terminal comprises a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being interconnected, wherein the memory is adapted to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method according to the first aspect.
In a fourth aspect, a computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of the first aspect.
According to the technical scheme, the exception handling method, the system, the terminal and the medium based on log analysis can analyze the log, realize automatic exception data handling by combining the log according to the exception condition of the operation and the maintenance of the log analysis system, and improve the timeliness of the operation and the maintenance monitoring of the system.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. Like elements or portions are generally identified by like reference numerals throughout the several figures. In the drawings, elements or portions thereof are not necessarily drawn to scale.
Fig. 1 is a flowchart of an exception handling method according to an embodiment of the present invention.
Fig. 2 is a block diagram of an exception handling system according to a second embodiment of the present invention.
Fig. 3 is a block diagram of a terminal according to a third embodiment of the present invention.
Detailed Description
Embodiments of the technical scheme of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical aspects of the present invention, and thus are merely examples, and are not intended to limit the scope of the present invention. It is noted that unless otherwise indicated, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in this specification and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Embodiment one:
an exception handling method based on log analysis, see fig. 1, comprises the following steps:
s1: reading a log stored in a database;
preferably, the log includes a system log and an application log. Specifically, the method may analyze a system log and an application log.
S2: analyzing the log according to a preset log template to obtain an analysis log;
preferably, the setting method of the log template comprises the following steps:
s11: obtaining a log sample from an internal memory or network;
s12: performing multi-row combination according to the head-of-line matching mode in the log sample to obtain a combination template;
s13: extracting log indexes in the merging module through a regular expression to obtain an extraction template;
s14: defining the extraction template as the log template.
Specifically, the method needs to determine a head-of-line matching mode in the log sample, and determine whether multi-line merging is needed.
S3: extracting corresponding indexes in the analysis log according to preset log indexes;
preferably, the categories of the log index include one or more combinations of:
application transaction classes, middleware classes, database classes, operating system classes, network classes, server classes, security device classes, and storage device classes.
Specifically, the application transaction classes mainly include time of transaction related logs, transaction operation codes, success-failure flags, transaction time-consuming, and the like. The middleware class mainly comprises time, log level, local IP, http transmission data volume, remote IP, request protocol, request mode, request URL and the like of logs such as Tomcat. Database classes mainly include log levels, instance names, hostnames, pid, etc. of database logs. The operating system class mainly comprises an operating system, virtualized related performance data, such as a CPU, a memory and the like; the network class mainly includes network-related data such as: source IP, destination IP, source port, destination port, protocol type, etc. The server class and the security device class mainly comprise hardware related performance data such as hardware temperature, SSL hardware performance and the like of the server, the security device and the like. Storage device classes mainly include storage related data such as: storage space usage, and so on.
S4: and when the index abnormality is detected, alarming.
Preferably, when the index abnormality is detected, the alerting specifically includes:
when the index extracted from the analysis log meets the preset alarm rule, alarming is carried out;
and acquiring the associated alarm associated with the alarm, and compressing the associated alarm.
Specifically, the alarm mainly takes care of the following points:
1) A rule engine;
the rule engine is mainly used for carrying out alarm management on unstructured data such as logs and the like, supporting defining alarm rules according to keyword search results, and determining whether to alarm and alarm level according to rule matching results.
2) Index management;
the index management is a precondition for setting a threshold, and includes, in addition to basic performance indexes, indexes aggregated based on basic data, such as the number of erroneous transactions in the last 5 minutes or the average transaction time of the last 5 minutes. The definition of the aggregate metrics is generated using a query interface provided by a query engine. Any statistical aggregate statement supported by the query engine can be stored as an index.
3) Threshold management;
the definition of the threshold depends on the index, the threshold management support sets different alarm values according to time periods, for example, the index of CPU utilization rate is set to be 0.8 for 8 am to 8 pm, and other time periods are 0.9, so that the user can conveniently and flexibly define the alarm threshold according to actual service conditions.
4) A scheduling engine;
after the alarm is started, the corresponding index or rule needs to be calculated in real time according to the time period when the alarm is defined, the index/rule of parallel real-time calculation which is required to be supported by the operation and maintenance big data platform reaches the millions, and the system needs to provide efficient dispatching and parallel real-time monitoring of the millions of indexes.
The framework of the alarm engine is realized based on AKKA Cluster, each started alarm management item is an Actor, the Actor is a lightweight parallel model, the weight of the Actor is lighter than that of threads, the rule of the alarm is required to be calculated, the index of the alarm, the threshold value of the alarm, the calculated period and other information are determined to be stored in the Actor when the Actor is created, each Actor is monitored and managed in real time, and when abnormality occurs, the Actor can be restarted or re-created automatically. A4 g virtual machine can easily create millions of level actors and support the improvement of the overall throughput of AKKA Cluster by adding nodes.
5) Alarm compression;
the actually running IT components have a close association relationship, besides the self-generated alarms, the anomalies of the bottom layer components can also cause the alarms of the upper layer components or services supported by the bottom layer components, if no alarms are compressed, alarm storms can be generated, the system needs to automatically identify the alarms of the bottommost layer according to the association relationship between the mastered and identified components, find out the associated alarms, compress the associated alarms, and maintain the association relationship between the components and store the association relationship in a Neo4j graph database.
6) Alarming and predicting;
depending on the feature model provided by the underlying analysis engine, intelligent prognosis can be provided for the overall operating condition of the system, and when the operation of the system does not conform to the model provided by the underlying analysis engine, alarm prediction can be performed.
7) Alarming by keywords;
after selection, the user can enter an index alarm interface, a fixed threshold configuration alarm of a certain index can be set, and the following diagram is set, namely, the threshold alarm of the maximum Duration index is set, and the situation that more than 3 times exceeds 300 seconds continuously occurs and is set as crisis alarm; setting a general alarm when more than 3 times of conditions exceeding 100 seconds continuously occur; setting up as an information alarm or the like more than 50 seconds for 3 times in succession.
According to the exception handling method based on log analysis, the log can be analyzed, automatic exception data processing is realized by combining the log according to the exception condition of the operation and maintenance of the log analysis system, and the timeliness of the operation and maintenance monitoring of the system is improved.
Preferably, the method further comprises, after the alerting when the index anomaly is detected:
establishing a prediction model according to the linear relation between the index extracted from the analysis log and a preset standard index;
and carrying the analysis log into the prediction model, and predicting the index utilization rate of each time node according to time, wherein the index utilization rate of each time node exceeds the standard.
Specifically, the capacity prediction is performed by analyzing the linear relationship between the indexes, creating a prediction model, and using a multiple linear regression model. Capacity prediction distinguishes between users, transactions, and workload of the system and business related metrics by capturing performance related data in existing systems. Using capacity prediction, the following results can be obtained:
1) Analyzing and predicting part of indexes according to time to predict the utilization rate of the indexes to reach a certain time node; how much time the index is predicted, when the point in time is.
2) Carrying out correlation analysis on part of important indexes and indexes, analyzing a mathematical formula between the two indexes, and estimating the utilization rate of the indexes according to total transaction amount of TPS and daily; the maximum transaction condition that the server can load can be estimated according to the index utilization rate.
Embodiment two:
an exception handling system based on log analysis, see fig. 2, comprising:
the acquisition unit: for reading a log stored in a database;
an analysis unit: the method comprises the steps of analyzing the log according to a preset log template to obtain an analysis log;
extraction unit: extracting corresponding indexes in the analysis log according to preset log indexes;
and a processing unit: and the alarm is used for alarming when the index abnormality is detected.
The exception handling system based on log analysis can analyze the log, and according to the exception condition of the operation and maintenance of the log analysis system, the automatic exception data processing is realized by combining the log, so that the timeliness of the operation and maintenance monitoring of the system is improved.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in this application, it should be understood that the disclosed system may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices, or elements, or may be an electrical, mechanical, or other form of connection.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present invention.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
For a brief description of the system provided by the embodiments of the present invention, reference may be made to the corresponding content in the foregoing method embodiments where the description of the embodiments is not mentioned.
Embodiment III:
a terminal, see fig. 3, comprising a processor 801, an input device 802, an output device 803 and a memory 804, the processor 801, the input device 802, the output device 803 and the memory 804 being interconnected by a bus 805, wherein the memory 804 is adapted to store a computer program comprising program instructions, the processor 801 being configured to invoke the program instructions to perform the method as described above.
It should be appreciated that in embodiments of the present invention, the processor 801 may be a central processing unit (Central Processing Unit, CPU) which may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 802 may include a touch pad, a fingerprint sensor (for collecting fingerprint information of a user and direction information of a fingerprint), a microphone, etc., and the output device 803 may include a display (LCD, etc.), a speaker, etc.
The memory 804 may include read only memory and random access memory and provides instructions and data to the processor 801. A portion of the memory 804 may also include non-volatile random access memory. For example, the memory 804 may also store information of device type.
For a brief description, the terminal provided in the embodiment of the present invention may refer to the corresponding content in the foregoing method embodiment, where the embodiment section is not mentioned.
Embodiment four:
a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method described above.
The computer readable storage medium may be an internal storage unit of the terminal according to any of the foregoing embodiments, for example, a hard disk or a memory of the terminal. The computer readable storage medium may also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the terminal. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the terminal. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
For a brief description, reference may be made to the corresponding contents of the foregoing method embodiments for the media provided in the embodiments of the present invention, where the description of the embodiments is not mentioned.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention, and are intended to be included within the scope of the appended claims and description.
Claims (7)
1. The exception handling method based on log analysis is characterized by comprising the following steps:
reading a log stored in a database;
analyzing the log according to a preset log template to obtain an analysis log;
extracting corresponding indexes in the analysis log according to preset log indexes;
when the index abnormality is detected, alarming is carried out;
and when the index abnormality is detected, specifically, alarming comprises the following steps:
when the index extracted from the analysis log meets the preset alarm rule, alarming is carried out; defining an alarm rule according to the keyword search result, and determining whether to alarm and an alarm level according to the rule matching result;
acquiring an associated alarm associated with the alarm, and compressing the associated alarm; according to the association relation between the mastered and identified components, automatically identifying the alarm at the bottommost layer, finding out the associated alarm, compressing the associated alarm, and maintaining the association relation between the components in a Neo4j graph database;
the method further comprises the following steps after the alarm is carried out when the index abnormality is detected:
establishing a prediction model according to the linear relation between the index extracted from the analysis log and a preset standard index;
and carrying the analysis log into the prediction model, predicting index utilization rate of each time node according to time, wherein the time node with the index utilization rate exceeding the standard comprises:
analyzing and predicting part of indexes according to time, and predicting the index utilization rate of a certain time node and the time node of a certain index;
or, carrying out correlation analysis between the trade condition and the index on part of the indexes, and carrying out estimation on the utilization rate of the indexes according to the total amount of TPS and daily trade; and estimating the maximum transaction condition of the server load according to the index utilization rate.
2. The method for exception handling based on log analysis according to claim 1, wherein,
the log includes a system log and an application log.
3. The method for exception handling based on log analysis according to claim 1, wherein,
the setting method of the log template comprises the following steps:
obtaining a log sample from an internal memory or network;
performing multi-row combination according to the head-of-line matching mode in the log sample to obtain a combination template;
extracting log indexes in the merging module through a regular expression to obtain an extraction template;
defining the extraction template as the log template.
4. The method for exception handling based on log analysis according to claim 1, wherein,
the categories of log indicators include one or more combinations of:
application transaction classes, middleware classes, database classes, operating system classes, network classes, server classes, security device classes, and storage device classes.
5. An exception handling system based on log analysis, comprising:
the acquisition unit: for reading a log stored in a database;
an analysis unit: the method comprises the steps of analyzing the log according to a preset log template to obtain an analysis log;
extraction unit: extracting corresponding indexes in the analysis log according to preset log indexes;
and a processing unit: the method is used for alarming when the index abnormality is detected, and specifically comprises alarming when the index extracted from the analysis log meets a preset alarming rule; defining an alarm rule according to the keyword search result, and determining whether to alarm and an alarm level according to the rule matching result;
acquiring an associated alarm associated with the alarm, and compressing the associated alarm; according to the association relation between the mastered and identified components, automatically identifying the alarm at the bottommost layer, finding out the associated alarm, compressing the associated alarm, and maintaining the association relation between the components in a Neo4j graph database;
after the alarming when the index abnormality is detected, the method further comprises the following steps:
establishing a prediction model according to the linear relation between the index extracted from the analysis log and a preset standard index;
and carrying the analysis log into the prediction model, predicting index utilization rate of each time node according to time, wherein the time node with the index utilization rate exceeding the standard comprises:
analyzing and predicting part of indexes according to time, and predicting the index utilization rate of a certain time node and the time node of a certain index;
or, carrying out correlation analysis between the trade condition and the index on part of the indexes, and carrying out estimation on the utilization rate of the indexes according to the total amount of TPS and daily trade; and estimating the maximum transaction condition of the server load according to the index utilization rate.
6. A terminal comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being interconnected, wherein the memory is adapted to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method of any of claims 1-4.
7. A computer readable storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911127824.4A CN111221702B (en) | 2019-11-18 | 2019-11-18 | Log analysis-based exception handling method, system, terminal and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911127824.4A CN111221702B (en) | 2019-11-18 | 2019-11-18 | Log analysis-based exception handling method, system, terminal and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111221702A CN111221702A (en) | 2020-06-02 |
| CN111221702B true CN111221702B (en) | 2024-02-27 |
Family
ID=70810157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911127824.4A Active CN111221702B (en) | 2019-11-18 | 2019-11-18 | Log analysis-based exception handling method, system, terminal and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111221702B (en) |
Families Citing this family (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111798237B (en) * | 2020-06-30 | 2023-08-25 | 中国工商银行股份有限公司 | Abnormal transaction diagnosis method and system based on application log |
| CN113297046A (en) * | 2020-08-03 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Early warning method and device for memory fault |
| CN112015653A (en) * | 2020-08-31 | 2020-12-01 | 中移（杭州）信息技术有限公司 | Problem positioning method, server and storage medium |
| CN112100047A (en) * | 2020-09-22 | 2020-12-18 | 北京思特奇信息技术股份有限公司 | Service performance monitoring and analyzing method and device |
| CN114518983B (en) * | 2020-11-02 | 2024-09-24 | 电科云(北京)科技有限公司 | Abnormal log alarming method and device |
| CN112269730A (en) * | 2020-11-05 | 2021-01-26 | 北京小米松果电子有限公司 | Abnormal log detection method, abnormal log detection device, and storage medium |
| CN112559300B (en) * | 2020-12-14 | 2024-03-01 | 中国工商银行股份有限公司 | Fault cause determining system, method and device |
| CN112667464A (en) * | 2020-12-22 | 2021-04-16 | 国网甘肃省电力公司 | Intelligent analysis method, system and equipment for information system state |
| CN113448798A (en) * | 2020-12-30 | 2021-09-28 | 北京新氧科技有限公司 | Log data monitoring method and related equipment |
| CN112905429B (en) * | 2021-02-20 | 2024-07-05 | 北京物芯科技有限责任公司 | Monitoring method and device for system simulation |
| CN113760645B (en) * | 2021-03-10 | 2024-09-24 | 京东科技控股股份有限公司 | System operation log monitoring method and device, electronic equipment and storage medium |
| CN113220543B (en) * | 2021-04-15 | 2024-02-23 | 新浪技术（中国）有限公司 | Service automatic alarm method and device |
| CN113190415A (en) * | 2021-05-27 | 2021-07-30 | 北京京东拓先科技有限公司 | Internet hospital system monitoring method, equipment, storage medium and program product |
| CN113254293A (en) * | 2021-05-28 | 2021-08-13 | 广州绿怡信息科技有限公司 | Fingerprint function detection method and device |
| CN113312321A (en) * | 2021-05-31 | 2021-08-27 | 中国民航信息网络股份有限公司 | Abnormal monitoring method for traffic and related equipment |
| CN113391990A (en) * | 2021-06-30 | 2021-09-14 | 未鲲(上海)科技服务有限公司 | System log monitoring method, device, equipment and storage medium |
| CN113485901B (en) * | 2021-07-06 | 2022-11-22 | 中国工商银行股份有限公司 | System evaluation method, device, equipment and medium based on log and index |
| CN113568967B (en) * | 2021-07-29 | 2024-10-11 | 掌阅科技股份有限公司 | Dynamic extraction method of time sequence index data, electronic equipment and storage medium |
| CN113590371B (en) * | 2021-08-09 | 2023-10-10 | 北京科银京成技术有限公司 | Event analyzer and event analysis method |
| CN113762765A (en) * | 2021-09-02 | 2021-12-07 | 南方电网数字电网研究院有限公司 | Index analysis method, index analysis system, electronic device and storage medium |
| CN115017127A (en) * | 2022-05-25 | 2022-09-06 | 厦门市美亚柏科信息股份有限公司 | Method and device for automatically inspecting acquisition equipment and storage medium |
| CN115357470B (en) * | 2022-10-21 | 2023-03-24 | 北京国电通网络技术有限公司 | Information generation method and device, electronic equipment and computer readable medium |
| CN115913896B (en) * | 2022-11-09 | 2024-10-01 | 中国联合网络通信集团有限公司 | Equipment detection method, server and medium |
| CN116542558B (en) * | 2023-04-27 | 2024-06-04 | 上海数禾信息科技有限公司 | Service index calculation method, device, computer equipment and storage medium |
| CN116471174B (en) * | 2023-05-05 | 2024-02-09 | 北京优特捷信息技术有限公司 | Log data monitoring system, method, device and storage medium |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103346912A (en) * | 2013-06-29 | 2013-10-09 | 华为技术有限公司 | Method, device and system for conducting warning correlation analysis |
| CN103856339A (en) * | 2012-12-04 | 2014-06-11 | 中国移动通信集团广西有限公司 | Method and device for compressing alarm information |
| CN105049247A (en) * | 2015-07-06 | 2015-11-11 | 中国科学院信息工程研究所 | Network safety log template extraction method and device |
| CN107491382A (en) * | 2017-07-21 | 2017-12-19 | 北京京东尚科信息技术有限公司 | Log-output method and device |
| CN107493275A (en) * | 2017-08-08 | 2017-12-19 | 北京盛华安信息技术有限公司 | The extracted in self-adaptive and analysis method and system of heterogeneous network security log information |
| CN108712294A (en) * | 2018-06-05 | 2018-10-26 | 陈艳 | A method of network equipment monitoring alarm is realized based on Syslog knowledge bases |
| CN108829558A (en) * | 2018-05-22 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of intelligent operation management method and system of data center's alarm |
| CN109189736A (en) * | 2018-08-01 | 2019-01-11 | 中国联合网络通信集团有限公司 | A kind of generation method and device of alarm association rule |
| CN109634818A (en) * | 2018-10-24 | 2019-04-16 | 中国平安人寿保险股份有限公司 | Log analysis method, system, terminal and computer readable storage medium |
| CN109714187A (en) * | 2018-08-17 | 2019-05-03 | 平安普惠企业管理有限公司 | Log analysis method, device, equipment and storage medium based on machine learning |
| CN109933633A (en) * | 2019-04-02 | 2019-06-25 | 北京睿至大数据有限公司 | A kind of O&M knowledge mapping construction method based on time and scene dimension |
| CN110166307A (en) * | 2019-07-02 | 2019-08-23 | 中国工商银行股份有限公司 | The method and apparatus that warning information is handled |
| CN110245045A (en) * | 2019-05-23 | 2019-09-17 | 平安科技（深圳）有限公司 | A kind of keyword alarm method and device based on log |
| CN110362453A (en) * | 2019-05-27 | 2019-10-22 | 中国平安人寿保险股份有限公司 | Log statistic alarm method and device, terminal and storage medium |
| CN110399278A (en) * | 2019-07-24 | 2019-11-01 | 江苏物联网研究发展中心 | Alert correlation system and method based on data center's abnormal monitoring |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070198312A1 (en) * | 2006-02-21 | 2007-08-23 | Sugato Bagchi | Data quality management using business process modeling |
| US20080005265A1 (en) * | 2006-06-30 | 2008-01-03 | Nokia Corporation | Method for automatic parsing of variable data fields from textual report data |
| US9928155B2 (en) * | 2015-11-18 | 2018-03-27 | Nec Corporation | Automated anomaly detection service on heterogeneous log streams |
| US10740170B2 (en) * | 2016-12-08 | 2020-08-11 | Nec Corporation | Structure-level anomaly detection for unstructured logs |
| US11210158B2 (en) * | 2017-11-29 | 2021-12-28 | Riverbed Technology, Inc. | Automated problem diagnosis on logs using anomalous telemetry analysis |
-
2019
- 2019-11-18 CN CN201911127824.4A patent/CN111221702B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103856339A (en) * | 2012-12-04 | 2014-06-11 | 中国移动通信集团广西有限公司 | Method and device for compressing alarm information |
| CN103346912A (en) * | 2013-06-29 | 2013-10-09 | 华为技术有限公司 | Method, device and system for conducting warning correlation analysis |
| CN105049247A (en) * | 2015-07-06 | 2015-11-11 | 中国科学院信息工程研究所 | Network safety log template extraction method and device |
| CN107491382A (en) * | 2017-07-21 | 2017-12-19 | 北京京东尚科信息技术有限公司 | Log-output method and device |
| CN107493275A (en) * | 2017-08-08 | 2017-12-19 | 北京盛华安信息技术有限公司 | The extracted in self-adaptive and analysis method and system of heterogeneous network security log information |
| CN108829558A (en) * | 2018-05-22 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of intelligent operation management method and system of data center's alarm |
| CN108712294A (en) * | 2018-06-05 | 2018-10-26 | 陈艳 | A method of network equipment monitoring alarm is realized based on Syslog knowledge bases |
| CN109189736A (en) * | 2018-08-01 | 2019-01-11 | 中国联合网络通信集团有限公司 | A kind of generation method and device of alarm association rule |
| CN109714187A (en) * | 2018-08-17 | 2019-05-03 | 平安普惠企业管理有限公司 | Log analysis method, device, equipment and storage medium based on machine learning |
| CN109634818A (en) * | 2018-10-24 | 2019-04-16 | 中国平安人寿保险股份有限公司 | Log analysis method, system, terminal and computer readable storage medium |
| CN109933633A (en) * | 2019-04-02 | 2019-06-25 | 北京睿至大数据有限公司 | A kind of O&M knowledge mapping construction method based on time and scene dimension |
| CN110245045A (en) * | 2019-05-23 | 2019-09-17 | 平安科技（深圳）有限公司 | A kind of keyword alarm method and device based on log |
| CN110362453A (en) * | 2019-05-27 | 2019-10-22 | 中国平安人寿保险股份有限公司 | Log statistic alarm method and device, terminal and storage medium |
| CN110166307A (en) * | 2019-07-02 | 2019-08-23 | 中国工商银行股份有限公司 | The method and apparatus that warning information is handled |
| CN110399278A (en) * | 2019-07-24 | 2019-11-01 | 江苏物联网研究发展中心 | Alert correlation system and method based on data center's abnormal monitoring |
Non-Patent Citations (1)
| Title |
|---|
| 胡沐创 ; .大数据日志分析平台应用探索与实践.金融科技时代.2018,(01),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111221702A (en) | 2020-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111221702B (en) | Log analysis-based exception handling method, system, terminal and medium | |
| CN110708204B (en) | Abnormity processing method, system, terminal and medium based on operation and maintenance knowledge base | |
| CN111190876A (en) | Log management system and operation method thereof | |
| CN108063699B (en) | Network performance monitoring method and device, electronic equipment and storage medium | |
| CN110928718A (en) | Exception handling method, system, terminal and medium based on correlation analysis | |
| US20060074621A1 (en) | Apparatus and method for prioritized grouping of data representing events | |
| CN110223146B (en) | System and method for monitoring whole process of electricity purchasing service of customer | |
| CN111694718A (en) | Method and device for identifying abnormal behavior of intranet user, computer equipment and readable storage medium | |
| CN111866016A (en) | Log analysis method and system | |
| Roschke et al. | A flexible and efficient alert correlation platform for distributed ids | |
| CN108390793A (en) | A kind of method and device of analysis system stability | |
| CN112988509B (en) | Alarm message filtering method and device, electronic equipment and storage medium | |
| US10073726B2 (en) | Detection of outage in cloud based service using usage data based error signals | |
| CN111339052A (en) | Unstructured log data processing method and device | |
| CN111782488B (en) | Message queue monitoring method, device, electronic equipment and medium | |
| US7617313B1 (en) | Metric transport and database load | |
| CN110677271B (en) | Big data alarm method, device, equipment and storage medium based on ELK | |
| CN105471938B (en) | Server load management method and device | |
| CN113282920B (en) | Log abnormality detection method, device, computer equipment and storage medium | |
| CN114785616A (en) | Data risk detection method and device, computer equipment and storage medium | |
| CN112256548B (en) | Abnormal data monitoring method and device, server and storage medium | |
| CN113778810A (en) | Log collection method, device and system | |
| CN113220551A (en) | Index trend prediction and early warning method and device, electronic equipment and storage medium | |
| CN115914064A (en) | Network system service performance evaluation method, device, computing equipment and storage medium | |
| CN113220530A (en) | Data quality monitoring method and platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |