CN114281280B - Log processing method and device, storage medium and electronic device - Google Patents
Log processing method and device, storage medium and electronic device Download PDFInfo
- Publication number
- CN114281280B CN114281280B CN202111672061.9A CN202111672061A CN114281280B CN 114281280 B CN114281280 B CN 114281280B CN 202111672061 A CN202111672061 A CN 202111672061A CN 114281280 B CN114281280 B CN 114281280B
- Authority
- CN
- China
- Prior art keywords
- log
- service
- starter
- traceid
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application discloses a log processing method and device, a storage medium and an electronic device. The method includes determining a log Starter configuration parameter; intercepting an inlet based on a controller interface method through the log Starter configuration parameters, acquiring parameter information of a request interface, and performing log printing according to a log preset standardized mode; integrating the log starters at each micro-service, locating the location of the log in the service link by generating a globally unique ID. The application solves the technical problem of poor effect of the log processing method. The application realizes the reduction of development tasks, unified management and operation of log files and standardization of log printing.
Description
Technical Field
The present application relates to the field of log processing, and more particularly, to a log processing method and apparatus, a storage medium, and an electronic apparatus.
Background
Log4j is an open source code item of Apache, and Log information delivery and output format of each Log can be controlled by using Log4 j.
It takes time to write the log and to have to pass the parameter check. Further, in a micro-service architecture, services are partitioned by traffic, however, inter-call between services results in log link tracking being difficult.
Aiming at the problem of poor effect of a log processing method in the related technology, no effective solution is proposed at present.
Disclosure of Invention
The application mainly aims to provide a log processing method and device, a storage medium and an electronic device, so as to solve the problem of poor effect of the log processing method.
In order to achieve the above object, according to one aspect of the present application, there is provided a log processing method.
The log processing method according to the present application comprises: determining a log Starter configuration parameter; intercepting an inlet based on a controller interface method through the log Starter configuration parameters, acquiring parameter information of a request interface, and performing log printing according to a log preset standardized mode; integrating the log starters at each micro-service, locating the location of the log in the service link by generating a globally unique ID.
Further, the obtaining the parameter information of the request interface performs log printing according to a log preset standardized mode, and the method further includes: confirming whether parameter verification is needed or not by acquiring configuration in parameter information of a request interface; and if so, checking and intercepting the target parameters.
Further, intercepting the entry based on the controller interface method through the log Starter configuration parameter includes: the log Starter configuration parameter includes at least one of: and requesting the type, the method name and the parameter information of the interface, and intercepting the interface by using the log Starter based on the @ Mapping of the controller interface method as a tangent point.
Further, integrating the log Starter at each micro-service, locating the location of the log in the service link by generating a globally unique ID, includes: integrating the log Starter at each micro-service to generate a traceID as the globally unique ID; acquiring the TraceID and corresponding transparent transmission parameters from a request head, and putting the TraceID and corresponding transparent transmission parameters into an MDC (MDC), wherein the MDC is used for storing log transparent transmission information by each thread through a storage space; the TraceID is synchronized into the context such that multithreading and message queue MQ transparent transfer is supported when locating the location of the log in the service link.
Further, the obtaining the parameter information of the request interface performs log printing according to a log preset standardized mode, including: the log preset standardized mode comprises date, service name, thread name, log level, log category name, traceID [ method name, line number ] and log information.
Further, after integrating the log Starter with each micro service and locating the location of the log in the service link by generating a globally unique ID, the method further includes: and integrating a log calling chain of the log Starter when log output is carried out on each micro service according to the print requests, wherein each print request has the globally unique ID, and each print request comprises printing in the service or printing through transmission service.
Further, the determining the log Starter configuration parameter includes: adding an applicator configuration according to a log back log component in the log Starter, and determining a root directory for generating a log file; and (3) carrying out log archiving interception through a log scrolling strategy label, and determining log archiving, a log file generation directory and names.
In order to achieve the above object, according to another aspect of the present application, there is provided a log processing apparatus.
The log processing device according to the present application includes: the determining module is used for determining a log Starter configuration parameter; the interception module is used for intercepting an inlet based on a controller interface method through the log Starter configuration parameters, acquiring parameter information of a request interface and performing log printing according to a log preset standardized mode; and the unique identification module is used for integrating the log Starter in each micro-service and positioning the position of the log in the service link by generating a global unique ID.
In order to achieve the above object, according to yet another aspect of the present application, there is provided a computer readable storage medium having a computer program stored therein, wherein the computer program is arranged to execute the method when run.
To achieve the above object, according to a further aspect of the present application, there is provided an electronic device comprising a memory, in which a computer program is stored, and a processor arranged to run the computer program to perform the method.
According to the log processing method, the device, the storage medium and the electronic device, a mode of determining the log Starter configuration parameters is adopted, the entry based on the controller interface method is intercepted through the log Starter configuration parameters, the parameter information of the request interface is obtained, and log printing is carried out according to a log preset standardized mode, so that the purposes of integrating the log Starter in each micro service and positioning the position of the log in a service link through generating a global unique ID are achieved, the technical effects of reducing development tasks, unified management operation and maintenance of the log files and standardization of the log printing are achieved, and the technical problem that the log processing method is poor in effect is solved. In addition, when intermodulation is used between micro services, the positioning log is more convenient and efficient to check.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application, are incorporated in and constitute a part of this specification. The drawings and their description are illustrative of the application and are not to be construed as unduly limiting the application. In the drawings:
FIG. 1 is a schematic diagram of a hardware configuration of a log processing method according to an embodiment of the present application;
FIG. 2 is a flow diagram of a log processing method according to an embodiment of the application;
FIG. 3 is a schematic diagram of a log processing device according to an embodiment of the present application;
fig. 4 is a flow chart of a log processing method according to a preferred embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to describe the embodiments of the application herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In the present application, the terms "upper", "lower", "left", "right", "front", "rear", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like indicate an azimuth or a positional relationship based on that shown in the drawings. These terms are only used to better describe the present application and its embodiments and are not intended to limit the scope of the indicated devices, elements or components to the particular orientations or to configure and operate in the particular orientations.
Also, some of the terms described above may be used to indicate other meanings in addition to orientation or positional relationships, for example, the term "upper" may also be used to indicate some sort of attachment or connection in some cases. The specific meaning of these terms in the present application will be understood by those of ordinary skill in the art according to the specific circumstances.
Furthermore, the terms "mounted," "configured," "provided," "connected," "coupled," and "sleeved" are to be construed broadly. For example, it may be a fixed connection, a removable connection, or a unitary construction; may be a mechanical connection, or an electrical connection; may be directly connected, or indirectly connected through intervening media, or may be in internal communication between two devices, elements, or components. The specific meaning of the above terms in the present application can be understood by those of ordinary skill in the art according to the specific circumstances.
Log4j is an open source item of Apache, and Log information delivery can be controlled by using Log4 j; the output format of each log can also be controlled; by defining the level of each piece of log information, the generation process of the log can be controlled more finely. These can be flexibly configured by one configuration file without modifying the code of the application.
The inventor finds that in practical development, a developer is forced to bear time pressure, and writing a log and checking necessary parameters are a very tedious matter and a part of redundancy.
For micro services, the micro service architecture is to divide services by traffic, using REST calls. An externally exposed interface may require a plurality of service cooperations to complete the interface function, which is very important for the transmission of important parameters to downstream services. With the continuous expansion of services, the inter-call between services becomes more and more complex, and the inter-call between services causes the log link tracking to be very difficult.
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
As shown in fig. 1, a hardware structure diagram of a log processing method according to an embodiment of the present application includes: micro service 200, server 100, log file data 300. The micro service 200 performs middle layer decomposition based on a micro service architecture to split the system into a plurality of small applications, and the micro service can be deployed on different servers 100 or on different containers of the same server 100. The log file data 300 is related to the actual business.
As shown in fig. 2, the method includes steps S201 to S203 as follows:
step S201, determining a log Starter configuration parameter;
step S202, intercepting an entry based on a controller interface method through the log Starter configuration parameter, acquiring parameter information of a request interface, and performing log printing according to a log preset standardized mode;
in step S203, the log starters are integrated in each micro service, and the location of the log is located in the service link by generating a globally unique ID.
From the above description, it can be seen that the following technical effects are achieved:
by adopting a mode of determining the configuration parameters of the log Starter, intercepting an entry based on a controller interface method through the configuration parameters of the log Starter, acquiring parameter information of a request interface, and performing log printing according to a log preset standardized mode, the purposes of integrating the log Starter in each micro-service and positioning the position of the log in a service link through generating a global unique ID are achieved, so that the technical effects of reducing development tasks, unified management and operation and maintenance of log files and log printing standardization are achieved, and the technical problem of poor log processing method effect is solved. In addition, when intermodulation is used between micro services, the positioning log is more convenient and efficient to check.
In step S201, a log Starter configuration parameter, for example, a unified log file specification position, is determined, and the archive log file is automatically compressed. Meanwhile, the maximum number of days of storing the log file is 30 days, and the maximum log file 300M is truncated once.
In a preferred embodiment, the determining the log Starter configuration parameter includes: adding an applicator configuration according to a log back log component in the log Starter, and determining a root directory for generating a log file; and (3) carrying out log archiving interception through a log scrolling strategy label, and determining log archiving, a log file generation directory and names.
Specifically, an app der configuration is added in the log Starter using a log back log component, and the root directory (/ home/logs) of the log file is generated by contract determination. The log archiving interception is carried out through a log scrolling policy tag, and the generation directory and the name of the log archiving log file are specified as follows: compressed files of $ { log.path }/$ { server.name }/info-%, { yyyy-MM-dd } -, i.gz.
Intercepting the entry based on the controller interface method through the log Starter configuration parameter in step S202 described above, for example, may use an AOP interception technique. The intercepting interface method based on the controller may be an interface with Mapping as a tangent point. And then acquiring parameter information of the request interface, and performing log printing according to a log preset standardized mode.
In a specific embodiment, the log presets standardized mode date, service name, thread name, log level, class name of log, traceID [ method name, line number ], log information: % d { yyy-MM-dd HH: MM: ss.SSS [ $ { server name } ] [% thread ]% -5level% log {100} [% tID ] [ method,% line ] -% msg% n.
Integrating the log starters at each micro service in the above step S203, locating the location of the log in the service link by generating a global unique ID, and adding the global unique ID: after the traceID, in the process of calling each other among the services, the check of the positioning log is more convenient and efficient, and the parameters of the important parameter service number, the service ID and the service type are transmitted to each service at the downstream.
In one embodiment, the log Starter is integrated at each microservice to generate a TraceID as the globally unique ID; acquiring the TraceID and corresponding transparent transmission parameters from a request head, and putting the TraceID and corresponding transparent transmission parameters into an MDC (MDC), wherein the MDC is used for storing log transparent transmission information by each thread through a storage space; the TraceID is synchronized into the context such that multithreading and message queue MQ transparent transfer is supported when locating the location of the log in the service link.
Specifically, the globally unique TraceID is generated by skywalking, the TraceID and the parameters of the transparent transmission are obtained from the request header and put into the MDC (the MDC realizes that each thread has an independent storage space to store log transparent transmission information, and can obtain the data of the MDC according to the unique ID, which is very independent), and then the method supports multithreading and mq transparent transmission in the skywalking context.
As a preferred embodiment of the present application, the obtaining parameter information of the request interface performs log printing according to a log preset standardized manner, and further includes: confirming whether parameter verification is needed or not by acquiring configuration in parameter information of a request interface; and if so, checking and intercepting the target parameters.
In the specific implementation, whether parameter verification is needed or not is confirmed by acquiring the configuration, and if the parameter verification interception is needed, a related technology is used for carrying out the parameter verification interception.
As a preferred embodiment, the intercepting the entry based on the controller interface method through the log Starter configuration parameter includes: the log Starter configuration parameter includes at least one of: and requesting the type, the method name and the parameter information of the interface, and intercepting the interface by using the log Starter based on the @ Mapping of the controller interface method as a tangent point.
In particular, AOP interception techniques are used. The intercepting interface method based on the controller may be an interface with Mapping as a tangent point. And then acquiring parameter information of the request interface, and performing log printing according to a log preset standardized mode.
As a preference in this embodiment, after integrating the log Starter with each micro service, locating the location of the log in the service link by generating a globally unique ID, further includes: and integrating a log calling chain of the log Starter when log output is carried out on each micro service according to the print requests, wherein each print request has the globally unique ID, and each print request comprises printing in the service or printing through transmission service.
In particular, in log output, a log call chain is printed, and each request has a unique identification TraceID. In looking up the problem, only all logs in this request need to be looked up. Preferably, it can be used with a link tracking system because log tracking across applications is implemented, filtering out irrelevant information.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
According to an embodiment of the present application, there is also provided a log processing apparatus for implementing the above method, as shown in fig. 3, the apparatus includes:
a determining module 301, configured to determine a log Starter configuration parameter;
the interception module 302 is configured to intercept an entry based on a controller interface method through the log Starter configuration parameter, obtain parameter information of a request interface, and perform log printing according to a log preset standardized mode;
a unique identification module 303, configured to integrate the log starters with each micro-service, and locate the location of the log in the service link by generating a globally unique ID.
In the embodiment of the present application, the determining module 301 determines a log Starter configuration parameter, for example, a unified and canonical location of a log file, and the archive log file is automatically compressed. Meanwhile, the maximum number of days of storing the log file is 30 days, and the maximum log file 300M is truncated once.
In a preferred embodiment, the determining the log Starter configuration parameter includes: adding an applicator configuration according to a log back log component in the log Starter, and determining a root directory for generating a log file; and (3) carrying out log archiving interception through a log scrolling strategy label, and determining log archiving, a log file generation directory and names.
Specifically, an app der configuration is added in the log Starter using a log back log component, and the root directory (/ home/logs) of the log file is generated by contract determination. The log archiving interception is carried out through a log scrolling policy tag, and the generation directory and the name of the log archiving log file are specified as follows: compressed files of $ { log.path }/$ { server.name }/info-%, { yyyy-MM-dd } -, i.gz.
In the intercepting module 302 according to the embodiment of the present application, the controller interface method based entry is intercepted by the log Starter configuration parameter, for example, an AOP intercepting technology may be used. The intercepting interface method based on the controller may be an interface with Mapping as a tangent point. And then acquiring parameter information of the request interface, and performing log printing according to a log preset standardized mode.
In a specific embodiment, the log presets standardized mode date, service name, thread name, log level, class name of log, traceID [ method name, line number ], log information: % d { yyy-MM-dd HH: MM: ss.SSS [ $ { server name } ] [% thread ]% -5level% log {100} [% tID ] [ method,% line ] -% msg% n.
The unique identification module 303 in the embodiment of the present application integrates the log Starter in each micro service, locates the position of the log in the service link by generating a global unique ID, and adds the global unique ID: after the traceID, in the process of calling each other among the services, the check of the positioning log is more convenient and efficient, and the parameters of the important parameter service number, the service ID and the service type are transmitted to each service at the downstream.
In one embodiment, the log Starter is integrated at each microservice to generate a TraceID as the globally unique ID; acquiring the TraceID and corresponding transparent transmission parameters from a request head, and putting the TraceID and corresponding transparent transmission parameters into an MDC (MDC), wherein the MDC is used for storing log transparent transmission information by each thread through a storage space; the TraceID is synchronized into the context such that multithreading and message queue MQ transparent transfer is supported when locating the location of the log in the service link.
Specifically, a globally unique TraceID is generated by Skywalking, the TraceID and parameters of the pass-through are obtained from the request header and put into the MDC, and then in the Skywalking context, multithreading and MQ pass-through are supported.
It will be apparent to those skilled in the art that the modules or steps of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, or they may alternatively be implemented in program code executable by computing devices, such that they may be stored in a memory device for execution by the computing devices, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps within them may be fabricated into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.
In order to better understand the flow of the log processing method, the following explanation is given with reference to the preferred embodiment, but the technical solution of the embodiment of the present application is not limited.
The log processing method in the embodiment of the application provides a standard log integrated starter for business side integration, and standard log printing can be realized without modifying codes and default configuration. After each micro-service integrates the log specification starter, development tasks are reduced. The log files are unified and standardized in position, the archive logs are automatically compressed, and old logs are cleared, so that operation and maintenance are convenient. For log transparent transmission, by adding a globally unique ID: after the traceID, in the process of calling each other among the services, the check of the positioning log is more convenient and efficient, and the parameters of the important parameter service number, the service ID and the service type are transmitted to each service at the downstream.
In the log processing method in the embodiment of the application, the section enhancement is performed by using AOP (section-oriented programming) in the Starter and using annotation request Mapping in the method as a tangent point interception request, and important information such as parameter entering, parameter exiting, time consuming and the like of the method is printed out. If the necessary parameter verification is started, the hibernate-valIDator technology is used for parameter verification, and verification information is printed out.
In the log processing method in the embodiment of the application, log transmission is carried out during inter-modulation between multiple services in the micro service, call information of the whole link of the downstream service is transmitted, and a global unique ID is generated based on skywalking: the traceID is used for the whole multi-service call link, and the unique traceID is used for the first time, so that the log can be quickly positioned in the multi-service link. And (3) using the MDC to carry out link burying, and storing important parameter information in the MDC as transparent transmission information of the current link. I.e. TraceID + service type + service ID + service code.
When the method is implemented, all the micro services are appointed to request parameters and response parameters by configuring a common section type printing method, and the time is consumed. Before the feign calls or sends MQ, the upper business service needs to call the business parameter transparent transmission method (BizTraceUtil. TransmissitParams), the parameters include business type (self-definition), business ID and business number; examples are as follows:
firstly, before the service parameter transparent transmission interface is called outwards (before the feign is called or before the MQ is sent), the method is required to call the transparent transmission service parameter
Biztraceutil. Transmissitparams (service type (e.g., limit), service ID, service number);
/(and/or)
Biztraceutil. Transmissitpaams (service type (e.g., limit), service ID, service number, service name (without setting this parameter, default current service name));
example//
BizTraceUtil.transmitParams("limit",100001L,"YX-ED-133220000000000")；
Secondly, printing the service parameters of the current service (without transmission, only printing in the service), calling a BizTraceUtil. CurrentBizParams method to write the current service parameters, and automatically outputting the set service parameters after log printing, wherein the current service parameters are in a log output format of [ C-TYPE: service TYPE C-ID: service ID C-CODE: service number ], and the example is as follows:
setting parameters of current service at the time of business entrance
BizTraceUtil.currentBizParams ("service type", "service ID", "service number (e.g., enterprise number)").
As shown in fig. 4, which is a flow chart of a log processing method in the embodiment of the present application, a specific implementation process includes the following steps:
step S401, adding an applicator configuration in a log Starter by using a log back log component;
specifically, the log Starter uses a log back log component to add an app der configuration, and the root directory (/ home/logs) of the log file is generated by contract determination. The log archiving interception is carried out through a log scrolling policy tag, and the generation directory and the name of the log archiving log file are specified as follows: compressed files of $ { log.path }/$ { server.name }/info-%d { yyyyy-MM-dd } -, i.gz, the log file has a maximum number of days of storage of 30 days and a maximum log file of 300M is truncated once.
In step S402, an entry requested by the Controller interface method is intercepted.
And intercepting the entry requested by the controller interface method by using an AOP interception technology. Interface interception is carried out by taking the interface request on the controller method as a tangent point, and information such as the type, the name and the parameters of the request interface is acquired through a proxy by the procedingJoinPoint to print. And confirming whether parameter verification is needed or not by acquiring configuration, and if so, using a hibernate-valIDator technology to intercept the parameter verification.
Step S403, after each service integration log Starter, starting the project to generate a service instance ID;
after each service integration log Starter, the start item first generates a service instance ID and then generates a TraceID for preparation. And for the parameter transparent transmission to the downstream service, intercepting the downstream service Feign request of the current service request, and putting the transparent transmission parameter information of the current service into a request head for release. The request is intercepted by a service tracking filter, a globally unique traceID is generated through skywalking, the traceID and the transparent parameters are acquired from the request header and are put into MDC (the MDC realizes that each thread has an independent storage space to store log transparent transmission information, and the data of the MDC can be acquired according to the unique ID so as to be very independent), and then the data are put into the skywalking context to support multithreading and mq transparent transmission. The property generation log printing specification of the custom log back is as follows:
the class name TraceID [ method name, line number ] log information of the date service name thread name log level log belongs to. % d { yyy-MM-dd HH: MM: ss.SSS [ $ { server name } ] [% thread ]% -5level% log {100} [% tID ] [ method,% line ] -% msg% n.
In step S404, in log output, a log call chain is printed, and each request has a unique id TraceID.
In log output, the TraceID is identified, a log call chain is printed, and each request has a unique identification TraceID, so that when a problem is checked, all logs in the request are checked. Typically used with link tracking systems because the latter can enable log tracking across applications, filtering out irrelevant information.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (10)
1. A log processing method, comprising:
determining a log Starter configuration parameter;
intercepting an inlet based on a controller interface method through the log Starter configuration parameters, acquiring parameter information of a request interface, and performing log printing according to a log preset standardized mode;
after each service integration log Starter, a starting item firstly generates a service instance ID, then generates a traceID for preparation, and for parameter transmission to downstream service, intercepts a downstream service Feign request of a current service request, and puts the transmitted parameter information of the current service into a request head for release;
the subsequent service tracking filter intercepts the request, generates globally unique traceID through skywalking, acquires the traceID and the transparent parameters from the request head and puts the traceID and the transparent parameters into MDC, and then supports multithreading and mq transparent transmission when the parameters are put into the skywalking context;
the property generation log printing specification of the custom log back is as follows: class name TraceID [ method name, line number ] log information of the date service name thread name log level log;
integrating the log starters at each micro-service, locating the location of the log in the service link by generating a globally unique ID.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the obtaining the parameter information of the request interface carries out log printing according to a log preset standardized mode, and the method further comprises the following steps:
confirming whether parameter verification is needed or not by acquiring configuration in parameter information of a request interface;
and if so, checking and intercepting the target parameters.
3. The method of claim 2, wherein the step of determining the position of the substrate comprises,
the intercepting, by the log Starter configuration parameter, the entry based on the controller interface method includes:
the log Starter configuration parameter includes at least one of: and requesting the type, the method name and the parameter information of the interface, and intercepting the interface by using the log Starter based on the @ Mapping of the controller interface method as a tangent point.
4. The method of claim 1, wherein the step of determining the position of the substrate comprises,
integrating the log Starter in each micro service, positioning the log position in the service link by generating a global unique ID, and comprising:
integrating the log Starter at each micro-service to generate a traceID as the globally unique ID;
acquiring the TraceID and corresponding transparent transmission parameters from a request head, and putting the TraceID and corresponding transparent transmission parameters into an MDC (MDC), wherein the MDC is used for storing log transparent transmission information by each thread through a storage space;
the TraceID is synchronized into the context such that multithreading and message queue MQ transparent transfer is supported when locating the location of the log in the service link.
5. The method of claim 4, wherein the step of determining the position of the first electrode is performed,
the obtaining the parameter information of the request interface to perform log printing according to a log preset standardized mode comprises the following steps:
the log preset standardized mode comprises date, service name, thread name, log level, log category name, traceID [ method name, line number ] and log information.
6. The method of claim 1, wherein the step of determining the position of the substrate comprises,
after each micro-service integrates the log starters, locating the location of the log in the service link by generating a globally unique ID, further comprising:
and integrating a log calling chain of the log Starter when log output is carried out on each micro service according to the print requests, wherein each print request has the globally unique ID, and each print request comprises printing in the service or printing through transmission service.
7. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the determining the log Starter configuration parameter includes:
adding an applicator configuration according to a log back log component in the log Starter, and determining a root directory for generating a log file;
and (3) carrying out log archiving interception through a log scrolling strategy label, and determining log archiving, a log file generation directory and names.
8. A log processing apparatus, comprising:
the determining module is used for determining a log Starter configuration parameter;
the interception module is used for intercepting an inlet based on a controller interface method through the log Starter configuration parameters, acquiring parameter information of a request interface and performing log printing according to a log preset standardized mode;
after each service integration log Starter, a starting item firstly generates a service instance ID, then generates a traceID for preparation, and for parameter transmission to downstream service, intercepts a downstream service Feign request of a current service request, and puts the transmitted parameter information of the current service into a request head for release;
the subsequent service tracking filter intercepts the request, generates globally unique traceID through skywalking, acquires the traceID and the transparent parameters from the request head and puts the traceID and the transparent parameters into MDC, and then supports multithreading and mq transparent transmission when the parameters are put into the skywalking context;
the property generation log printing specification of the custom log back is as follows: class name TraceID [ method name, line number ] log information of the date service name thread name log level log;
and the unique identification module is used for integrating the log Starter in each micro-service and positioning the position of the log in the service link by generating a global unique ID.
9. A computer-readable storage medium comprising,
the computer readable storage medium having stored therein a computer program, wherein the computer program is arranged to perform the method of any of the claims 1 to 7 when run.
10. An electronic device comprising a memory and a processor, characterized in that,
the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111672061.9A CN114281280B (en) | 2021-12-31 | 2021-12-31 | Log processing method and device, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111672061.9A CN114281280B (en) | 2021-12-31 | 2021-12-31 | Log processing method and device, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114281280A CN114281280A (en) | 2022-04-05 |
| CN114281280B true CN114281280B (en) | 2023-08-25 |
Family
ID=80879484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111672061.9A Active CN114281280B (en) | 2021-12-31 | 2021-12-31 | Log processing method and device, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114281280B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101635649A (en) * | 2002-04-30 | 2010-01-27 | 佳能株式会社 | Network device management system and control method of the same |
| CN110069354A (en) * | 2019-04-15 | 2019-07-30 | 必成汇(成都)科技有限公司 | The full link trace method of micro services and micro services framework |
| CN110750253A (en) * | 2019-10-28 | 2020-02-04 | 深圳前海环融联易信息科技服务有限公司 | Log standardization method and device, computer equipment and storage medium |
| CN110851471A (en) * | 2018-07-27 | 2020-02-28 | 北京京东尚科信息技术有限公司 | Distributed log data processing method, device and system |
| WO2020241014A1 (en) * | 2019-05-31 | 2020-12-03 | ブラザー工業株式会社 | Program, management device, and printing system |
| CN112118286A (en) * | 2020-08-04 | 2020-12-22 | 紫光云(南京)数字技术有限公司 | Multi-thread link tracking method and device based on MDC and computer readable storage medium |
| CN112286776A (en) * | 2020-11-04 | 2021-01-29 | 中国电力财务有限公司 | Method and system for tracking micro-service link |
| CN113342767A (en) * | 2021-04-28 | 2021-09-03 | 招银云创信息技术有限公司 | Log generation method, device, equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4906672B2 (en) * | 2007-10-22 | 2012-03-28 | 株式会社日立製作所 | Web application process recording method and process recording apparatus |
| US10140287B2 (en) * | 2015-09-09 | 2018-11-27 | International Business Machines Corporation | Scalable and accurate mining of control flow from execution logs across distributed systems |
-
2021
- 2021-12-31 CN CN202111672061.9A patent/CN114281280B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101635649A (en) * | 2002-04-30 | 2010-01-27 | 佳能株式会社 | Network device management system and control method of the same |
| CN110851471A (en) * | 2018-07-27 | 2020-02-28 | 北京京东尚科信息技术有限公司 | Distributed log data processing method, device and system |
| CN110069354A (en) * | 2019-04-15 | 2019-07-30 | 必成汇(成都)科技有限公司 | The full link trace method of micro services and micro services framework |
| WO2020241014A1 (en) * | 2019-05-31 | 2020-12-03 | ブラザー工業株式会社 | Program, management device, and printing system |
| CN110750253A (en) * | 2019-10-28 | 2020-02-04 | 深圳前海环融联易信息科技服务有限公司 | Log standardization method and device, computer equipment and storage medium |
| CN112118286A (en) * | 2020-08-04 | 2020-12-22 | 紫光云(南京)数字技术有限公司 | Multi-thread link tracking method and device based on MDC and computer readable storage medium |
| CN112286776A (en) * | 2020-11-04 | 2021-01-29 | 中国电力财务有限公司 | Method and system for tracking micro-service link |
| CN113342767A (en) * | 2021-04-28 | 2021-09-03 | 招银云创信息技术有限公司 | Log generation method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114281280A (en) | 2022-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9692717B2 (en) | System and method for supporting detachable administration commands in a cloud platform environment | |
| CN110196731B (en) | Operation and maintenance system, method and storage medium | |
| CN110543328B (en) | Cross-platform component management method, system, terminal and storage medium based on Ambari | |
| CN111865900B (en) | RPC protocol-based cross-network regional proxy access method and system | |
| CN107229520A (en) | Data center operating system | |
| CN113949702B (en) | Multi-layer network protocol processing method and device for service grid | |
| CN109714188A (en) | Configuration data management method, equipment and storage medium based on Zookeeper | |
| CA3138764A1 (en) | Data processing method, device, computer equipment and storage medium | |
| CN114531197B (en) | On-orbit distributed information resource application service system | |
| CN109684028B (en) | Method, device and equipment for separating operating system from user data | |
| CN113835836A (en) | System, method, computer device and medium for dynamically publishing container service | |
| CN113938321B (en) | Extensible operation and maintenance management system, method, electronic equipment and readable storage medium | |
| CN114168179A (en) | Micro-service management method, device, computer equipment and storage medium | |
| US10185613B2 (en) | Error determination from logs | |
| CN114281280B (en) | Log processing method and device, storage medium and electronic device | |
| CN113238849A (en) | Timed task processing method and device, storage medium and electronic equipment | |
| CN115134367A (en) | Cloud platform and service processing method | |
| US11582345B2 (en) | Context data management interface for contact center | |
| CN113434281A (en) | Equipment scheduling method and cloud platform | |
| CN113110917B (en) | Data discovery and security access method based on Kubernetes | |
| CN113839792B (en) | Network access configuration method and device, computer equipment and readable storage medium | |
| CN115334086A (en) | Unified monitoring and management method for CMP system logs based on Docker | |
| CN115905391A (en) | Configuration management data acquisition method, system, equipment and medium applied to Kubernets | |
| CN117056562A (en) | Synchronous storage method, system, terminal and storage medium for unstructured data | |
| CN117178539A (en) | Hybrid cloud event notification management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |