CN111143304B

CN111143304B - Micro-service system exception log analysis method based on request link

Info

Publication number: CN111143304B
Application number: CN202010127054.XA
Authority: CN
Inventors: 武鑫; 吴小伟; 刘浩杨; 宋瑞国
Original assignee: Hangzhou Endpoint Network Technology Co ltd
Current assignee: Hangzhou Endpoint Network Technology Co ltd
Priority date: 2019-11-20
Filing date: 2020-02-28
Publication date: 2023-09-29
Anticipated expiration: 2040-02-28
Also published as: CN111143304A

Abstract

The invention discloses a micro-service system exception log analysis method based on a request link, which comprises the following specific steps: associating the log, the exception and the call link information in the Java Agent according to the request ID; in the data analysis module, generating an abnormal ID (identity) for the abnormal event data, carrying out abnormal aggregation according to the abnormal ID, and simultaneously storing the mapping relation between the abnormal event and the request ID; at the interface inquiry end, inquiring a corresponding call log and a call link according to the abnormal ID association; outputting an exception log in the service application; and calling an API interface triggering the exception, namely, the exception log can be collected and analyzed. The invention can search and analyze the abnormal log in the same platform, and the automatic association of the abnormal log and the calling link can quickly locate the abnormal business.

Description

Micro-service system exception log analysis method based on request link

Technical Field

The invention belongs to the technical field of PAAS and micro-service management systems, and particularly relates to a micro-service system exception log analysis method based on a request link.

Background

In the big data age, the production, sales, administration, management of enterprises are gradually transformed by technology driving into data driving, and the importance of data in this process is self-evident. In mass data accumulated in daily operation of enterprises, the log is one of the most available data with the widest coverage and the most valuable data.

The log refers to various data generated by users, servers, network devices, operating systems, databases, application software and the like, and the core concept of the log is a time sequence increasing event sequence. Based on two basic elements of time and event, we can trace back past records, record present states and search future trends in the log, which is the value of the log. And the exception log records exception information so as to facilitate the viewing and analysis of related personnel.

In a micro-service system, an exception log analysis generally needs to build a centralized log analysis platform, such as ELK. When the business is abnormal, the log analysis system can only inquire by taking the service as the dimension, and can not rapidly and effectively analyze the logs of all the services passing through the request link. Meanwhile, when the system scale is large, the cost for rapidly searching out the needed abnormal logs in the log analysis platform is increased, and besides the inaccurate analysis result, the large operation and maintenance cost of the log analysis platform is also brought. Thus, there is a need for a method that can quickly query logs.

Disclosure of Invention

In order to solve the problems, the invention provides a method for analyzing the abnormal log of a micro-service system based on a request link, which can rapidly locate the business abnormality in the micro-service system based on the abnormal log.

The technical scheme of the invention is as follows: a micro-service system exception log analysis method based on request link comprises the following specific steps:

s1, associating logs, exceptions and call link information in Java Agent according to a request ID;

s2, generating an abnormal ID (identity) for the abnormal event data in a data analysis module, carrying out abnormal aggregation according to the abnormal ID, and simultaneously storing the mapping relation between the abnormal event and the request ID;

s3, at the interface inquiry end, inquiring a corresponding call log and a call link according to the abnormal ID association;

s4, outputting an abnormal log in the service application;

s5, calling an API interface triggering the exception, namely, the exception log can be collected and analyzed.

Preferably, the generation rule of the anomaly ID of the present invention includes: item ID, service name, exception file, and exception line number.

Preferably, the method for realizing the association of the log, the exception and the call link information according to the request ID in the Java Agent comprises the following steps:

s101, intercepting a servlet to generate a unique request ID in each request, and generating calling link information at the same time;

s102, intercepting log frames such as log back and log4j, and unifying the output format of the log;

s103, intercepting an exception log, and converting the exception log into structured exception event data.

Preferably, the method for realizing the association of the log, the exception and the call link information according to the request ID in the Java Agent is performed by a byte code rewriting technique.

The present invention preferably uses a log framework to output an anomaly log in a business application.

Compared with the prior art, the invention has the beneficial effects that:

the invention can search and analyze the abnormal log in the same platform, and the automatic association of the abnormal log and the calling link can quickly locate the abnormal business.

Drawings

FIG. 1 is a schematic diagram of the present invention (associated request IDs in the log console).

Fig. 2 is a schematic diagram of the present invention (retrieving each service log passed in a request based on a request ID).

FIG. 3 is a schematic diagram of the present invention (an exception log aggregate query list).

Fig. 4 is a schematic diagram of the present invention (details of anomaly log).

FIG. 5 is a schematic diagram of the present invention (request link log associated with exception log).

FIG. 6 is a schematic diagram of the present invention (call links associated with exception logs).

Detailed Description

The invention is described in further detail below with reference to the attached drawings and detailed description:

example 1

The specific steps of this embodiment 1 are:

step 1, associating log, exception and call link information in Java Agent according to request ID, as shown in figure 1 and figure 2;

step 2, generating an abnormal ID (identity) for the abnormal event data in a data analysis module, and carrying out abnormal aggregation according to the abnormal ID as shown in fig. 3, and simultaneously storing the mapping relation between the abnormal event and the request ID;

step 3, at the interface inquiry end, inquiring the corresponding call log and call link according to the abnormal ID association;

step 4, using a log framework to output an abnormal log in the service application;

and step 5, calling an API interface triggering the exception, namely, collecting and analyzing an exception log, wherein details of the exception log are shown in fig. 4.

In this embodiment, the generation rule of the anomaly ID is: item ID + service name + exception file + exception line number.

The specific implementation mode for implementing the step 1 comprises the following steps:

step 101, intercepting a unique request ID generated by a servlet in each request through a byte code rewriting technology, and generating call link information at the same time, as shown in fig. 5 and 6;

102, intercepting a log frame such as log back and log4j through a byte code rewriting technology, and unifying the output format of the log;

and 103, intercepting the exception log through a byte code rewriting technology, and converting the exception log into structured exception event data.

Claims

1. A method for analyzing an exception log of a micro-service system based on a request link is characterized by comprising the following specific steps:

s4, outputting an abnormal log in the service application;

2. The method for analyzing the exception log of the micro service system based on the request link according to claim 1, wherein the generation rule of the exception ID comprises: item ID, service name, exception file, and exception line number.

3. The method for analyzing the exception log of the micro service system based on the request link according to claim 1, wherein the implementation method of the step S1 is as follows:

s102, intercepting a log frame and unifying the output format of a log;

4. A method for analyzing exception log of micro service system based on request link according to claim 1 or 3, wherein said step S1 is implemented by byte code rewriting technique.

5. The method of claim 1, wherein the exception log is output using a log framework.