CN111221702A - Exception handling method, system, terminal and medium based on log analysis - Google Patents
Exception handling method, system, terminal and medium based on log analysis Download PDFInfo
- Publication number
- CN111221702A CN111221702A CN201911127824.4A CN201911127824A CN111221702A CN 111221702 A CN111221702 A CN 111221702A CN 201911127824 A CN201911127824 A CN 201911127824A CN 111221702 A CN111221702 A CN 111221702A
- Authority
- CN
- China
- Prior art keywords
- log
- analysis
- index
- indexes
- class
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 57
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000012545 processing Methods 0.000 claims abstract description 11
- 230000002159 abnormal effect Effects 0.000 claims abstract description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 9
- 230000005856 abnormality Effects 0.000 claims description 5
- 238000012423 maintenance Methods 0.000 abstract description 14
- 238000012544 monitoring process Methods 0.000 abstract description 9
- 230000004044 response Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012417 linear regression Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
- G06F11/3082—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting the data filtering being achieved by aggregating or compressing the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
- G06F11/324—Display of status information
- G06F11/327—Alarm or error message display
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention provides an exception handling method based on log analysis, which comprises the following steps: reading a log stored in a database; analyzing the log according to a preset log template to obtain an analysis log; extracting corresponding indexes in the analysis log according to preset log indexes; and when the index is detected to be abnormal, alarming. The method can analyze the log, and realizes automatic abnormal data processing by combining the log according to the abnormal condition of the operation and maintenance of the log analysis system, thereby improving the instantaneity of the operation and maintenance monitoring of the system.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to an exception handling method, system, terminal and medium based on log analysis.
Background
The system operation and maintenance monitoring is mainly to collect various monitoring indexes through various technical tools, such as Tivoli, Zabbix, APM, network packet capturing, application and embedded point monitoring and the like, so as to monitor the system operation and maintenance. These indices are typically stored as time series data (i.e., including acquisition time and index values).
Because when daily system operation and maintenance, the mode of log can be adopted to record monitoring data usually, and the operating condition of system is known to the staff through looking over corresponding log, when the system appears unusually, in time handles the unusual condition, but this kind of exception handling method efficiency is lower, needs the staff to look over by hand, and the instantaneity is relatively poor.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides an exception handling method, system, terminal and medium based on log analysis, which can realize automatic exception data processing by combining logs and improve the instantaneity of system operation and maintenance monitoring.
In a first aspect, a log analysis-based exception handling method includes the following steps:
reading a log stored in a database;
analyzing the log according to a preset log template to obtain an analysis log;
extracting corresponding indexes in the analysis log according to preset log indexes;
and when the index is detected to be abnormal, alarming.
Preferably, the log comprises a system log and an application log.
Preferably, the setting method of the log template includes the following steps:
acquiring a log sample from an internal memory or a network;
performing multi-line merging according to a line head matching mode in the log sample to obtain a merging template;
extracting the log indexes in the merging module through a regular expression to obtain an extraction template;
and defining the extraction template as the log template.
Preferably, the category of the log index includes one or more combinations of:
an application transaction class, a middleware class, a database class, an operating system class, a network class, a server class, a security device class, and a storage device class.
Preferably, when the index abnormality is detected, the alarming specifically includes:
when the indexes extracted from the analysis log meet a preset alarm rule, alarming;
and acquiring the associated alarm associated with the alarm, and compressing the associated alarm.
Preferably, after the warning is performed when the index abnormality is detected, the method further includes:
establishing a prediction model according to a linear relation between indexes extracted from the analysis log and preset standard indexes;
and substituting the analysis log into the prediction model, and predicting the index utilization rate of each time node according to time, wherein the index utilization rate exceeds the standard.
In a second aspect, an exception handling system based on log analysis includes:
a collecting unit: the log reading device is used for reading the log stored in the database;
an analysis unit: the log analysis device is used for analyzing the log according to a preset log template so as to obtain an analysis log;
an extraction unit: extracting corresponding indexes in the analysis log according to preset log indexes;
a processing unit: and the alarm is given when the index is detected to be abnormal.
In a third aspect, a terminal comprises a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being connected to each other, wherein the memory is configured to store a computer program, the computer program comprising program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect.
In a fourth aspect, a computer-readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of the first aspect.
According to the technical scheme, the log analysis-based exception handling method, the log analysis-based exception handling system, the log analysis-based exception handling terminal and the log analysis-based exception handling medium can analyze the log, realize automatic exception data processing by combining the log according to the exception condition of operation and maintenance of the log analysis system, and improve the instantaneity of operation and maintenance monitoring of the system.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.
Fig. 1 is a flowchart of an exception handling method according to an embodiment of the present invention.
Fig. 2 is a block diagram of an exception handling system according to a second embodiment of the present invention.
Fig. 3 is a block diagram of a terminal according to a third embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby. It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
The first embodiment is as follows:
an exception handling method based on log analysis, referring to fig. 1, includes the following steps:
s1: reading a log stored in a database;
preferably, the log comprises a system log and an application log. In particular, the method may analyze system logs and application logs.
S2: analyzing the log according to a preset log template to obtain an analysis log;
preferably, the setting method of the log template includes the following steps:
s11: acquiring a log sample from an internal memory or a network;
s12: performing multi-line merging according to a line head matching mode in the log sample to obtain a merging template;
s13: extracting the log indexes in the merging module through a regular expression to obtain an extraction template;
s14: and defining the extraction template as the log template.
Specifically, the method needs to determine a head-of-line matching pattern in the log sample and determine whether multi-line merging is needed.
S3: extracting corresponding indexes in the analysis log according to preset log indexes;
preferably, the category of the log index includes one or more combinations of:
an application transaction class, a middleware class, a database class, an operating system class, a network class, a server class, a security device class, and a storage device class.
Specifically, the application transaction class mainly includes time of transaction-related log, transaction operation code, success-failure flag, transaction elapsed time, and the like. The middleware class mainly comprises time of logs such as Tomcat, log level, local IP, http sending data volume, remote IP, request protocol, request mode, request URL and the like. The database class mainly includes the log level, instance name, hostname, pid, etc. of the database log. The operating system class mainly comprises an operating system and related virtualized performance data, such as a CPU (central processing unit), a memory and the like; the network class mainly includes network-related data such as: source IP, destination IP, source port, destination port, protocol type, etc. The server class and the security device class mainly include performance data related to hardware such as hardware temperature, SSL hardware performance, and the like. The storage device class mainly includes storage related data, such as: storage space usage, etc.
S4: and when the index is detected to be abnormal, alarming.
Preferably, when the index abnormality is detected, the alarming specifically includes:
when the indexes extracted from the analysis log meet a preset alarm rule, alarming;
and acquiring the associated alarm associated with the alarm, and compressing the associated alarm.
Specifically, the alarm mainly focuses on the following points:
1) a rules engine;
the rule engine mainly performs alarm management on unstructured data such as logs and the like, supports the definition of alarm rules according to keyword search results, and determines whether to alarm and alarm level according to rule matching results.
2) Index management;
the index management is a precondition for setting a threshold, and the index management includes an index aggregated based on basic data, such as the number of false transactions in the last 5 minutes or the average transaction time in the last 5 minutes, in addition to the basic performance index. The definition of the aggregation indicator is generated using a query interface provided by a query engine. Any statistical aggregation statement supported by the query engine can be stored as an index.
3) Managing a threshold value;
the threshold value is defined depending on an index, the threshold value management supports setting different alarm values in different time periods, for example, the index of the CPU utilization rate is set to be 0.8 from 8 am to 8 pm, and the alarm values in other time periods are 0.9, so that a user can flexibly define the alarm threshold value according to the actual service condition.
4) A scheduling engine;
after the alarm is started, corresponding indexes or rules need to be calculated in real time according to a time period defined by the alarm, the indexes/rules of parallel real-time calculation which need to be supported by the operation and maintenance big data platform reach a million level, and the system needs to provide efficient scheduling and monitor the million-level indexes in real time in parallel.
The framework of the alarm engine is realized based on AKKA Cluster, each started alarm management item is an Actor which is a lightweight parallel model, the parallel model is lighter than a thread, the alarm rule, the index needing to be calculated, the threshold of the alarm, the calculation period and other information are determined to be stored in the Actor when the Actor is established, each Actor is monitored and managed in real time, and the Actor can be automatically restarted or reestablished when an exception occurs. A4 g virtual machine can easily create million levels of actors and support the improvement of the overall throughput of the AKKA Cluster by means of adding nodes.
5) Alarm compression;
the IT components which are actually operated have close association relationship, the abnormity of the bottom layer components can generate alarms of the upper layer components or services supported by the IT components besides the alarms generated by the abnormity of the bottom layer components, if the alarms are not compressed, an alarm storm can be generated, the system needs to automatically identify the alarms at the bottom layer according to the association relationship between the grasped and identified components, find out the associated alarms and compress the associated alarms, and the maintenance of the association relationship between the components is stored in a Neo4j database.
6) Alarm prediction;
the intelligent prediction method can provide intelligent prediction for the overall operation condition of the system by depending on the characteristic model provided by the underlying analysis engine, and can perform alarm prediction when the operation of the system does not conform to the model provided by the underlying analysis engine.
7) A keyword alarm;
after selection, the system enters an index alarm interface, and a fixed threshold configuration alarm of a certain index can be set, as shown in the following figure, a threshold alarm of the maximum Duration index is set, and a crisis alarm is set when the condition of exceeding 300 seconds continuously occurs for more than 3 times; the condition of exceeding 100 seconds continuously occurs more than 3 times, and is set as a general alarm; setting as information alarm and the like for 3 times exceeding 50 seconds.
The log analysis-based exception handling method can analyze the log, and realizes automatic exception data processing by combining the log according to the exception condition of the log analysis system operation and maintenance, thereby improving the instantaneity of the system operation and maintenance monitoring.
Preferably, after the warning is performed when the index abnormality is detected, the method further includes:
establishing a prediction model according to a linear relation between indexes extracted from the analysis log and preset standard indexes;
and substituting the analysis log into the prediction model, and predicting the index utilization rate of each time node according to time, wherein the index utilization rate exceeds the standard.
Specifically, capacity prediction builds a prediction model by analyzing the linear relationship between the indicators, using a multiple linear regression model. Capacity prediction distinguishes workload and traffic related indicators for users, transactions and systems by capturing performance related data in existing systems. Using capacity prediction, the following results can be obtained:
1) analyzing and predicting part of indexes according to time, and predicting the index utilization rate of a certain time node; when the index is predicted, when the time point is.
2) Analyzing the correlation between the transaction condition and the index of part of important indexes to obtain a mathematical formula between the transaction condition and the index, and estimating the utilization rate of the index according to the TPS and the daily transaction total amount; the maximum transaction condition that the server can load can be estimated according to the index utilization rate.
Example two:
an exception handling system based on log analysis, see fig. 2, comprising:
a collecting unit: the log reading device is used for reading the log stored in the database;
an analysis unit: the log analysis device is used for analyzing the log according to a preset log template so as to obtain an analysis log;
an extraction unit: extracting corresponding indexes in the analysis log according to preset log indexes;
a processing unit: and the alarm is given when the index is detected to be abnormal.
The log analysis-based exception handling system can analyze the log, and realizes automatic exception data processing by combining the log according to the exception condition of the operation and maintenance of the log analysis system, thereby improving the instantaneity of the operation and maintenance monitoring of the system.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed system may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
For the sake of brief description, the system provided by the embodiment of the present invention may refer to the corresponding content in the foregoing method embodiments.
Example three:
a terminal, see fig. 3, comprising a processor 801, an input device 802, an output device 803 and a memory 804, the processor 801, the input device 802, the output device 803 and the memory 804 being interconnected via a bus 805, wherein the memory 804 is adapted to store a computer program comprising program instructions, the processor 801 being configured to invoke the program instructions to perform the method described above.
It should be understood that in the present embodiment, the Processor 801 may be a Central Processing Unit (CPU), and the Processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 802 may include a touch pad, a fingerprint sensor (for collecting fingerprint information of a user and direction information of the fingerprint), a microphone, etc., and the output device 803 may include a display (LCD, etc.), a speaker, etc.
The memory 804 may include both read-only memory and random access memory, and provides instructions and data to the processor 801. A portion of the memory 804 may also include non-volatile random access memory. For example, the memory 804 may also store device type information.
For a brief description, the embodiment of the present invention may refer to the corresponding content in the foregoing method embodiments.
Example four:
a computer-readable storage medium, in which a computer program is stored, the computer program comprising program instructions which, when executed by a processor, cause the processor to carry out the above-mentioned method.
The computer readable storage medium may be an internal storage unit of the terminal according to any of the foregoing embodiments, for example, a hard disk or a memory of the terminal. The computer readable storage medium may also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the terminal. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the terminal. The computer-readable storage medium is used for storing the computer program and other programs and data required by the terminal. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.
For the sake of brief description, the media provided by the embodiments of the present invention, and the portions of the embodiments that are not mentioned, refer to the corresponding contents in the foregoing method embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (9)
1. An exception handling method based on log analysis is characterized by comprising the following steps:
reading a log stored in a database;
analyzing the log according to a preset log template to obtain an analysis log;
extracting corresponding indexes in the analysis log according to preset log indexes;
and when the index is detected to be abnormal, alarming.
2. The log analysis-based exception handling method according to claim 1,
the logs include system logs and application logs.
3. The log analysis-based exception handling method according to claim 1,
the setting method of the log template comprises the following steps:
acquiring a log sample from an internal memory or a network;
performing multi-line merging according to a line head matching mode in the log sample to obtain a merging template;
extracting the log indexes in the merging module through a regular expression to obtain an extraction template;
and defining the extraction template as the log template.
4. The log analysis-based exception handling method according to claim 1,
the category of the log index comprises one or more combinations of:
an application transaction class, a middleware class, a database class, an operating system class, a network class, a server class, a security device class, and a storage device class.
5. The log analysis-based exception handling method of claim 4,
when the index abnormality is detected, the alarming specifically includes:
when the indexes extracted from the analysis log meet a preset alarm rule, alarming;
and acquiring the associated alarm associated with the alarm, and compressing the associated alarm.
6. The log analysis-based exception handling method according to claim 1, further comprising, after said performing an alarm when said index is detected to be abnormal, the step of:
establishing a prediction model according to a linear relation between indexes extracted from the analysis log and preset standard indexes;
and substituting the analysis log into the prediction model, and predicting the index utilization rate of each time node according to time, wherein the index utilization rate exceeds the standard.
7. An exception handling system based on log analysis, comprising:
a collecting unit: the log reading device is used for reading the log stored in the database;
an analysis unit: the log analysis device is used for analyzing the log according to a preset log template so as to obtain an analysis log;
an extraction unit: extracting corresponding indexes in the analysis log according to preset log indexes;
a processing unit: and the alarm is given when the index is detected to be abnormal.
8. A terminal, comprising a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being interconnected, wherein the memory is configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method of any of claims 1-6.
9. A computer-readable storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method according to any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911127824.4A CN111221702B (en) | 2019-11-18 | 2019-11-18 | Log analysis-based exception handling method, system, terminal and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911127824.4A CN111221702B (en) | 2019-11-18 | 2019-11-18 | Log analysis-based exception handling method, system, terminal and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111221702A true CN111221702A (en) | 2020-06-02 |
| CN111221702B CN111221702B (en) | 2024-02-27 |
Family
ID=70810157
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911127824.4A Active CN111221702B (en) | 2019-11-18 | 2019-11-18 | Log analysis-based exception handling method, system, terminal and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111221702B (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111798237A (en) * | 2020-06-30 | 2020-10-20 | 中国工商银行股份有限公司 | Abnormal transaction diagnosis method and system based on application log |
| CN112015653A (en) * | 2020-08-31 | 2020-12-01 | 中移（杭州）信息技术有限公司 | Problem positioning method, server and storage medium |
| CN112100047A (en) * | 2020-09-22 | 2020-12-18 | 北京思特奇信息技术股份有限公司 | Service performance monitoring and analyzing method and device |
| CN112269730A (en) * | 2020-11-05 | 2021-01-26 | 北京小米松果电子有限公司 | Abnormal log detection method, abnormal log detection device, and storage medium |
| CN112463432A (en) * | 2020-12-08 | 2021-03-09 | 广州品唯软件有限公司 | Inspection method, device and system based on index data |
| CN112559300A (en) * | 2020-12-14 | 2021-03-26 | 中国工商银行股份有限公司 | Fault reason determining system, method and device |
| CN112667464A (en) * | 2020-12-22 | 2021-04-16 | 国网甘肃省电力公司 | Intelligent analysis method, system and equipment for information system state |
| CN112905429A (en) * | 2021-02-20 | 2021-06-04 | 北京物芯科技有限责任公司 | System simulation monitoring method and device |
| CN113190415A (en) * | 2021-05-27 | 2021-07-30 | 北京京东拓先科技有限公司 | Internet hospital system monitoring method, equipment, storage medium and program product |
| CN113220543A (en) * | 2021-04-15 | 2021-08-06 | 新浪网技术(中国)有限公司 | Automatic service alarm method and device |
| CN113254293A (en) * | 2021-05-28 | 2021-08-13 | 广州绿怡信息科技有限公司 | Fingerprint function detection method and device |
| CN113297046A (en) * | 2020-08-03 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Early warning method and device for memory fault |
| CN113312321A (en) * | 2021-05-31 | 2021-08-27 | 中国民航信息网络股份有限公司 | Abnormal monitoring method for traffic and related equipment |
| CN113391990A (en) * | 2021-06-30 | 2021-09-14 | 未鲲(上海)科技服务有限公司 | System log monitoring method, device, equipment and storage medium |
| CN113448798A (en) * | 2020-12-30 | 2021-09-28 | 北京新氧科技有限公司 | Log data monitoring method and related equipment |
| CN113485901A (en) * | 2021-07-06 | 2021-10-08 | 中国工商银行股份有限公司 | System evaluation method, device, equipment and medium based on log and index |
| CN113568967A (en) * | 2021-07-29 | 2021-10-29 | 掌阅科技股份有限公司 | Dynamic extraction method of time sequence index data, electronic equipment and storage medium |
| CN113590371A (en) * | 2021-08-09 | 2021-11-02 | 北京科银京成技术有限公司 | Event analyzer and event analysis method |
| CN113762765A (en) * | 2021-09-02 | 2021-12-07 | 南方电网数字电网研究院有限公司 | Index analysis method, index analysis system, electronic device and storage medium |
| CN113760645A (en) * | 2021-03-10 | 2021-12-07 | 京东科技控股股份有限公司 | System operation log monitoring method and device, electronic equipment and storage medium |
| CN114518983A (en) * | 2020-11-02 | 2022-05-20 | 电科云(北京)科技有限公司 | Abnormal log warning method and device |
| CN115017127A (en) * | 2022-05-25 | 2022-09-06 | 厦门市美亚柏科信息股份有限公司 | Method and device for automatically inspecting acquisition equipment and storage medium |
| CN115357470A (en) * | 2022-10-21 | 2022-11-18 | 北京国电通网络技术有限公司 | Information generation method and device, electronic equipment and computer readable medium |
| CN115913896A (en) * | 2022-11-09 | 2023-04-04 | 中国联合网络通信集团有限公司 | Device detection method, server and medium |
| CN116471174A (en) * | 2023-05-05 | 2023-07-21 | 北京优特捷信息技术有限公司 | Log data monitoring system, method, device and storage medium |
| CN116542558A (en) * | 2023-04-27 | 2023-08-04 | 上海数禾信息科技有限公司 | Service index calculation method, device, computer equipment and storage medium |
Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070198312A1 (en) * | 2006-02-21 | 2007-08-23 | Sugato Bagchi | Data quality management using business process modeling |
| US20080005265A1 (en) * | 2006-06-30 | 2008-01-03 | Nokia Corporation | Method for automatic parsing of variable data fields from textual report data |
| CN103346912A (en) * | 2013-06-29 | 2013-10-09 | 华为技术有限公司 | Method, device and system for conducting warning correlation analysis |
| CN103856339A (en) * | 2012-12-04 | 2014-06-11 | 中国移动通信集团广西有限公司 | Method and device for compressing alarm information |
| CN105049247A (en) * | 2015-07-06 | 2015-11-11 | 中国科学院信息工程研究所 | Network safety log template extraction method and device |
| US20170139806A1 (en) * | 2015-11-18 | 2017-05-18 | Nec Laboratories America, Inc. | Automated Anomaly Detection Service on Heterogeneous Log Streams |
| CN107491382A (en) * | 2017-07-21 | 2017-12-19 | 北京京东尚科信息技术有限公司 | Log-output method and device |
| CN107493275A (en) * | 2017-08-08 | 2017-12-19 | 北京盛华安信息技术有限公司 | The extracted in self-adaptive and analysis method and system of heterogeneous network security log information |
| US20180165147A1 (en) * | 2016-12-08 | 2018-06-14 | Nec Laboratories America, Inc. | Structure-level anomaly detection for unstructured logs |
| CN108712294A (en) * | 2018-06-05 | 2018-10-26 | 陈艳 | A method of network equipment monitoring alarm is realized based on Syslog knowledge bases |
| CN108829558A (en) * | 2018-05-22 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of intelligent operation management method and system of data center's alarm |
| CN109189736A (en) * | 2018-08-01 | 2019-01-11 | 中国联合网络通信集团有限公司 | A kind of generation method and device of alarm association rule |
| CN109634818A (en) * | 2018-10-24 | 2019-04-16 | 中国平安人寿保险股份有限公司 | Log analysis method, system, terminal and computer readable storage medium |
| CN109714187A (en) * | 2018-08-17 | 2019-05-03 | 平安普惠企业管理有限公司 | Log analysis method, device, equipment and storage medium based on machine learning |
| US20190163553A1 (en) * | 2017-11-29 | 2019-05-30 | Riverbed Technology, Inc. | Automated problem diagnosis on logs using anomalous telemetry analysis |
| CN109933633A (en) * | 2019-04-02 | 2019-06-25 | 北京睿至大数据有限公司 | A kind of O&M knowledge mapping construction method based on time and scene dimension |
| CN110166307A (en) * | 2019-07-02 | 2019-08-23 | 中国工商银行股份有限公司 | The method and apparatus that warning information is handled |
| CN110245045A (en) * | 2019-05-23 | 2019-09-17 | 平安科技（深圳）有限公司 | A kind of keyword alarm method and device based on log |
| CN110362453A (en) * | 2019-05-27 | 2019-10-22 | 中国平安人寿保险股份有限公司 | Log statistic alarm method and device, terminal and storage medium |
| CN110399278A (en) * | 2019-07-24 | 2019-11-01 | 江苏物联网研究发展中心 | Alert correlation system and method based on data center's abnormal monitoring |
-
2019
- 2019-11-18 CN CN201911127824.4A patent/CN111221702B/en active Active
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070198312A1 (en) * | 2006-02-21 | 2007-08-23 | Sugato Bagchi | Data quality management using business process modeling |
| US20080005265A1 (en) * | 2006-06-30 | 2008-01-03 | Nokia Corporation | Method for automatic parsing of variable data fields from textual report data |
| CN103856339A (en) * | 2012-12-04 | 2014-06-11 | 中国移动通信集团广西有限公司 | Method and device for compressing alarm information |
| CN103346912A (en) * | 2013-06-29 | 2013-10-09 | 华为技术有限公司 | Method, device and system for conducting warning correlation analysis |
| CN105049247A (en) * | 2015-07-06 | 2015-11-11 | 中国科学院信息工程研究所 | Network safety log template extraction method and device |
| US20170139806A1 (en) * | 2015-11-18 | 2017-05-18 | Nec Laboratories America, Inc. | Automated Anomaly Detection Service on Heterogeneous Log Streams |
| US20180165147A1 (en) * | 2016-12-08 | 2018-06-14 | Nec Laboratories America, Inc. | Structure-level anomaly detection for unstructured logs |
| CN107491382A (en) * | 2017-07-21 | 2017-12-19 | 北京京东尚科信息技术有限公司 | Log-output method and device |
| CN107493275A (en) * | 2017-08-08 | 2017-12-19 | 北京盛华安信息技术有限公司 | The extracted in self-adaptive and analysis method and system of heterogeneous network security log information |
| US20190163553A1 (en) * | 2017-11-29 | 2019-05-30 | Riverbed Technology, Inc. | Automated problem diagnosis on logs using anomalous telemetry analysis |
| CN108829558A (en) * | 2018-05-22 | 2018-11-16 | 郑州云海信息技术有限公司 | A kind of intelligent operation management method and system of data center's alarm |
| CN108712294A (en) * | 2018-06-05 | 2018-10-26 | 陈艳 | A method of network equipment monitoring alarm is realized based on Syslog knowledge bases |
| CN109189736A (en) * | 2018-08-01 | 2019-01-11 | 中国联合网络通信集团有限公司 | A kind of generation method and device of alarm association rule |
| CN109714187A (en) * | 2018-08-17 | 2019-05-03 | 平安普惠企业管理有限公司 | Log analysis method, device, equipment and storage medium based on machine learning |
| CN109634818A (en) * | 2018-10-24 | 2019-04-16 | 中国平安人寿保险股份有限公司 | Log analysis method, system, terminal and computer readable storage medium |
| CN109933633A (en) * | 2019-04-02 | 2019-06-25 | 北京睿至大数据有限公司 | A kind of O&M knowledge mapping construction method based on time and scene dimension |
| CN110245045A (en) * | 2019-05-23 | 2019-09-17 | 平安科技（深圳）有限公司 | A kind of keyword alarm method and device based on log |
| CN110362453A (en) * | 2019-05-27 | 2019-10-22 | 中国平安人寿保险股份有限公司 | Log statistic alarm method and device, terminal and storage medium |
| CN110166307A (en) * | 2019-07-02 | 2019-08-23 | 中国工商银行股份有限公司 | The method and apparatus that warning information is handled |
| CN110399278A (en) * | 2019-07-24 | 2019-11-01 | 江苏物联网研究发展中心 | Alert correlation system and method based on data center's abnormal monitoring |
Non-Patent Citations (1)
| Title |
|---|
| 胡沐创;: "大数据日志分析平台应用探索与实践", 金融科技时代, no. 01, 10 January 2018 (2018-01-10) * |
Cited By (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111798237B (en) * | 2020-06-30 | 2023-08-25 | 中国工商银行股份有限公司 | Abnormal transaction diagnosis method and system based on application log |
| CN111798237A (en) * | 2020-06-30 | 2020-10-20 | 中国工商银行股份有限公司 | Abnormal transaction diagnosis method and system based on application log |
| CN113297046A (en) * | 2020-08-03 | 2021-08-24 | 阿里巴巴集团控股有限公司 | Early warning method and device for memory fault |
| CN112015653A (en) * | 2020-08-31 | 2020-12-01 | 中移（杭州）信息技术有限公司 | Problem positioning method, server and storage medium |
| CN112100047A (en) * | 2020-09-22 | 2020-12-18 | 北京思特奇信息技术股份有限公司 | Service performance monitoring and analyzing method and device |
| CN114518983B (en) * | 2020-11-02 | 2024-09-24 | 电科云(北京)科技有限公司 | Abnormal log alarming method and device |
| CN114518983A (en) * | 2020-11-02 | 2022-05-20 | 电科云(北京)科技有限公司 | Abnormal log warning method and device |
| CN112269730A (en) * | 2020-11-05 | 2021-01-26 | 北京小米松果电子有限公司 | Abnormal log detection method, abnormal log detection device, and storage medium |
| CN112463432A (en) * | 2020-12-08 | 2021-03-09 | 广州品唯软件有限公司 | Inspection method, device and system based on index data |
| CN112559300A (en) * | 2020-12-14 | 2021-03-26 | 中国工商银行股份有限公司 | Fault reason determining system, method and device |
| CN112559300B (en) * | 2020-12-14 | 2024-03-01 | 中国工商银行股份有限公司 | Fault cause determining system, method and device |
| CN112667464A (en) * | 2020-12-22 | 2021-04-16 | 国网甘肃省电力公司 | Intelligent analysis method, system and equipment for information system state |
| CN113448798A (en) * | 2020-12-30 | 2021-09-28 | 北京新氧科技有限公司 | Log data monitoring method and related equipment |
| CN112905429A (en) * | 2021-02-20 | 2021-06-04 | 北京物芯科技有限责任公司 | System simulation monitoring method and device |
| CN113760645A (en) * | 2021-03-10 | 2021-12-07 | 京东科技控股股份有限公司 | System operation log monitoring method and device, electronic equipment and storage medium |
| CN113220543B (en) * | 2021-04-15 | 2024-02-23 | 新浪技术（中国）有限公司 | Service automatic alarm method and device |
| CN113220543A (en) * | 2021-04-15 | 2021-08-06 | 新浪网技术(中国)有限公司 | Automatic service alarm method and device |
| CN113190415A (en) * | 2021-05-27 | 2021-07-30 | 北京京东拓先科技有限公司 | Internet hospital system monitoring method, equipment, storage medium and program product |
| CN113254293A (en) * | 2021-05-28 | 2021-08-13 | 广州绿怡信息科技有限公司 | Fingerprint function detection method and device |
| CN113312321A (en) * | 2021-05-31 | 2021-08-27 | 中国民航信息网络股份有限公司 | Abnormal monitoring method for traffic and related equipment |
| CN113391990A (en) * | 2021-06-30 | 2021-09-14 | 未鲲(上海)科技服务有限公司 | System log monitoring method, device, equipment and storage medium |
| CN113485901A (en) * | 2021-07-06 | 2021-10-08 | 中国工商银行股份有限公司 | System evaluation method, device, equipment and medium based on log and index |
| CN113485901B (en) * | 2021-07-06 | 2022-11-22 | 中国工商银行股份有限公司 | System evaluation method, device, equipment and medium based on log and index |
| CN113568967A (en) * | 2021-07-29 | 2021-10-29 | 掌阅科技股份有限公司 | Dynamic extraction method of time sequence index data, electronic equipment and storage medium |
| CN113568967B (en) * | 2021-07-29 | 2024-10-11 | 掌阅科技股份有限公司 | Dynamic extraction method of time sequence index data, electronic equipment and storage medium |
| CN113590371B (en) * | 2021-08-09 | 2023-10-10 | 北京科银京成技术有限公司 | Event analyzer and event analysis method |
| CN113590371A (en) * | 2021-08-09 | 2021-11-02 | 北京科银京成技术有限公司 | Event analyzer and event analysis method |
| CN113762765A (en) * | 2021-09-02 | 2021-12-07 | 南方电网数字电网研究院有限公司 | Index analysis method, index analysis system, electronic device and storage medium |
| CN115017127A (en) * | 2022-05-25 | 2022-09-06 | 厦门市美亚柏科信息股份有限公司 | Method and device for automatically inspecting acquisition equipment and storage medium |
| CN115357470A (en) * | 2022-10-21 | 2022-11-18 | 北京国电通网络技术有限公司 | Information generation method and device, electronic equipment and computer readable medium |
| CN115913896A (en) * | 2022-11-09 | 2023-04-04 | 中国联合网络通信集团有限公司 | Device detection method, server and medium |
| CN116542558A (en) * | 2023-04-27 | 2023-08-04 | 上海数禾信息科技有限公司 | Service index calculation method, device, computer equipment and storage medium |
| CN116542558B (en) * | 2023-04-27 | 2024-06-04 | 上海数禾信息科技有限公司 | Service index calculation method, device, computer equipment and storage medium |
| CN116471174A (en) * | 2023-05-05 | 2023-07-21 | 北京优特捷信息技术有限公司 | Log data monitoring system, method, device and storage medium |
| CN116471174B (en) * | 2023-05-05 | 2024-02-09 | 北京优特捷信息技术有限公司 | Log data monitoring system, method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111221702B (en) | 2024-02-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111221702B (en) | Log analysis-based exception handling method, system, terminal and medium | |
| CN107992398B (en) | Monitoring method and monitoring system of service system | |
| CN110223146B (en) | System and method for monitoring whole process of electricity purchasing service of customer | |
| KR101388319B1 (en) | Method and device for managing security events | |
| CN111190876A (en) | Log management system and operation method thereof | |
| CN108063699B (en) | Network performance monitoring method and device, electronic equipment and storage medium | |
| US20060074621A1 (en) | Apparatus and method for prioritized grouping of data representing events | |
| CN111694718A (en) | Method and device for identifying abnormal behavior of intranet user, computer equipment and readable storage medium | |
| WO2015136624A1 (en) | Application performance monitoring method and device | |
| CN112988509B (en) | Alarm message filtering method and device, electronic equipment and storage medium | |
| CN113448812A (en) | Monitoring alarm method and device under micro-service scene | |
| CN108390793A (en) | A kind of method and device of analysis system stability | |
| CN113986595A (en) | Abnormity positioning method and device | |
| CN112001443A (en) | Network behavior data monitoring method and device, storage medium and electronic equipment | |
| CN112800061B (en) | Data storage method, device, server and storage medium | |
| CN107562601A (en) | A kind of alarm method and device | |
| CN111339052A (en) | Unstructured log data processing method and device | |
| US9116804B2 (en) | Transient detection for predictive health management of data processing systems | |
| CN117971606A (en) | Log management system and method based on elastic search | |
| CN111782488B (en) | Message queue monitoring method, device, electronic equipment and medium | |
| CN110874291A (en) | Real-time detection method for abnormal container | |
| CN110677271B (en) | Big data alarm method, device, equipment and storage medium based on ELK | |
| CN105471938B (en) | Server load management method and device | |
| CN112948223B (en) | Method and device for monitoring running condition | |
| CN114785616A (en) | Data risk detection method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |