CN116932268A

CN116932268A - Alarm method, service system, electronic device and computer readable storage medium

Info

Publication number: CN116932268A
Application number: CN202310919574.8A
Authority: CN
Inventors: 肖如杏; 欧阳晔
Original assignee: Asiainfo Technologies China Inc
Current assignee: Asiainfo Technologies China Inc
Priority date: 2023-07-25
Filing date: 2023-07-25
Publication date: 2023-10-24

Abstract

The embodiment of the application provides an alarm method, a service system, electronic equipment and a computer readable storage medium, and relates to the technical field of computers. The method comprises the following steps: the business application container generates a log template of a log based on template content of the log template in the business code, determines a log level of the log and a template state of the log template, and if the log template is a newly added log template, outputs the newly added log template and the template state of the newly added log template to a log center; and for each read log, determining a target template matched with the log from all log templates stored in the log center by the alarm module, and if the template state of the target template is an abnormal state, initiating an alarm for the target template. According to the embodiment of the application, the log templates are determined in a simpler and more accurate manner without clustering the logs, the method is not limited by massive historical logs, and the calculation amount consumed by using an algorithm to identify the massive log templates is reduced.

Description

Alarm method, service system, electronic device and computer readable storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to the field of computer system log anomaly detection, and in particular, to an alarm method, a service system, an electronic device, and a computer readable storage medium.

Background

For computer systems, when a new/unknown software failure occurs, an operator or developer may first analyze and search the logs for exception logs, thereby classifying the exception logs and the context of the exception logs. However, the number of logs is large, noisy, and mostly unstructured, and it is difficult to quickly and accurately discover anomalies in the logs based on manual work.

The massive logs generated by the system are expanded from a few log templates, one log template can code one type of system event, the extraction of the log templates is a very important ring in log analysis, and the existing scheme generally extracts the log templates by mining all frequent item sets of massive historical logs or clustering the massive historical logs, and takes all the frequent item sets as all the log templates or takes common parts in all clusters as the log templates.

However, the frequent item sets obtained by the existing scheme based on the frequent item sets and the clustering are of various types and massive, the clustering result obtained during the clustering is unstable, the clustering result also comprises a large number of clustering clusters, the analysis is time-consuming and labor-consuming, and in addition, after each log template is obtained by the existing scheme, whether the log template is abnormal or not needs to be analyzed through an algorithm, so that a large amount of calculation resources are consumed.

Disclosure of Invention

The embodiment of the application provides an alarm method, a service system, electronic equipment, a computer readable storage medium and a computer program product, which are used for solving at least one technical problem in the background technology.

According to a first aspect of an embodiment of the present application, there is provided an alarm method, a service system including a service application container, a log center, and an alarm module; the business application container comprises at least one preset business code and a log printing library; the business code comprises template content of a log template corresponding to a log to be printed and names of log printing functions to be called, and the log printing library comprises the log printing functions; the log printing function comprises a log level of a log to be printed; the log center is used for storing logs and log templates;

The method comprises the following steps:

in each first period, the business application container operates at least one business code to obtain at least one log printed by at least one business code, each business code calls a corresponding log printing function from a log printing library based on the name of the log printing function to be called in the operation process, and the log printed by the business code is output to a log center through the log printing function;

for each business code, the business application container also generates a corresponding log template based on template content of the log template in the business code, determines a log level of the log from a log printing function corresponding to the business code, determines a template state of the log template based on the log level of the log, and if the log template is determined to be a newly added log template, outputs the newly added log template and the template state of the newly added log template to a log center; the template state includes an abnormal state;

the alarm module reads the log from the log center in real time; and for each read log, determining a target template matched with the log from all log templates stored in a log center, and if the template state of the target template is determined to be an abnormal state and the alarm for the target template is not initiated in the first period, initiating the alarm for the target template.

According to a second aspect of an embodiment of the present application, there is provided a service system including a service application container, a log center, and an alarm module; the business application container comprises at least one preset business code and a log printing library; the business code comprises template content of a log template corresponding to a log to be printed and names of log printing functions to be called, and the log printing library comprises the log printing functions; the log printing function comprises a log level of a log to be printed; the log center is used for storing logs and log templates;

in each first period, the service application container is used for running at least one service code to obtain at least one log printed by the at least one service code, each service code calls a corresponding log printing function from a log printing library based on the name of the log printing function to be called in the running process, and the log printed by the service code is output to a log center through the log printing function;

the business application container is used for generating a corresponding log template based on template content of the log template in the business code for each business code, determining a log level of the log from a log printing function corresponding to the business code, determining a template state of the log template based on the log level of the log, and outputting the newly added log template and the template state of the newly added log template to a log center if the log template is determined to be the newly added log template; the template state includes an abnormal state;

The alarm module is used for reading the log from the log center in real time; and for each read log, determining a target template matched with the log from all log templates stored in a log center, and if the template state of the target template is determined to be an abnormal state and the alarm for the target template is not initiated in the first period, initiating the alarm for the target template.

According to a third aspect of embodiments of the present application, there is provided an electronic device comprising a memory, a processor and a computer program stored on the memory, the processor implementing the steps of the method as provided in the first aspect when the program is executed.

According to a fourth aspect of embodiments of the present application, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method as provided by the first aspect.

According to a fifth aspect of embodiments of the present application, there is provided a computer program product comprising computer instructions stored in a computer readable storage medium, which when read from the computer readable storage medium by a processor of a computer device, the computer instructions are executed by the processor causing the computer device to perform the steps of the method as provided by the first aspect.

The technical scheme provided by the embodiment of the application has the beneficial effects that:

the embodiment of the application can determine the log template of the log from the service codes of the printed log in the log printing process, does not need to cluster the log, determines each log template according to the clustering result, has simple and accurate mode for determining the log template, and is not limited by massive historical logs. In addition, the embodiment of the application determines the template state of the log template corresponding to the log based on the log level of the log, can directly initiate the alarm of the log template aiming at the abnormal state based on the template state of the log template, does not need to carry out excessive analysis on the log template, and simultaneously reduces the calculation amount consumed by using an algorithm to carry out mass log template identification.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments of the present application will be briefly described below.

FIG. 1 is a flow chart of an alarm method provided by the scheme of the embodiment of the application;

FIG. 2 is a schematic flow chart of an alarm method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an alarm module according to an embodiment of the present application for matching log templates for each log in real time;

FIG. 4 is a flowchart of another alarm method according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a service system according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described below with reference to the drawings in the present application. It should be understood that the embodiments described below with reference to the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and the technical solutions of the embodiments of the present application are not limited.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and "comprising," when used in this specification, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof, all of which may be included in the present specification. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates that at least one of the items defined by the term, e.g., "a and/or B" may be implemented as "a", or as "B", or as "a and B".

For the purpose of making the objects, technical solutions and advantages of the present application more apparent, the embodiments of the present application will be described in further detail with reference to the accompanying drawings.

First, several terms related to the present application are described and explained:

log, network equipment, system, service program, etc., all generate a log record called log during operation; each row of the log records the description of the date, time, user, action, and other related operations.

The log template can represent a class of system events, and massive log data generated by the system are expanded by a few log templates. The log record is usually composed of constant and variable, which can also be called template words and parameter words, the job of extracting the log template is to extract the template words therein, each template word forms the log template, and the template words are usually sentences defined by print in the program.

A container is a technique for binding an application with all of its necessary files into one runtime environment. As a unit, the container can be easily moved and run on any operating system in any environment. The use of containers can isolate software so that it can run independently in different operating systems, hardware, networks, storage systems, and security policies. This allows for seamless transitions of container-based applications in development, testing, and production environments. Since the operating system is not packaged into containers, each container requires only minimal computing resources, occupies little space, and is easy to install.

As shown in fig. 1, which is a flowchart exemplarily showing an alert method provided by an existing scheme, the alert method is applied to an existing service system, the existing service system includes a service application container, a log sending container, a log center, and an alert module, including the steps of (1) the service application container running a service code, and (2) printing at least one log to a log file in a log print library; the log sending container reads the log file, (4) sends the read log to a REST API interface of a log center, the log center stores the log obtained by the REST API interface, the log center provides a log query API to the outside, and the alarm module can read the log of the log center by accessing the log query API; step (5) log template recognition is carried out to obtain each log template, specifically, history logs can be clustered to obtain each cluster, and common parts in each cluster are used as log templates; step (6) storing the log template in a log template library; step (7) reading real-time logs from a log center, reading log templates from a log template library, and carrying out real-time template matching on the logs for each real-time log; step (8), counting the number of logs corresponding to each log template; step (9), determining whether the number of logs corresponding to each log template is abnormal; and step (d) alarming log templates with abnormal template quantity.

The above existing algorithm has the following defects:

1) After the log is clustered, when the log is clustered into a plurality of templates, the quality of the templates is difficult to evaluate effectively, particularly, the number of the templates is large in the implementation process or the online process, the time for judging manually one by one is too long, and the operation and maintenance personnel may not have enough time to judge manually one by one;

2) The template itself has no abnormal label, after the log is matched with the template in real time, whether the current log is abnormal or not cannot be judged according to the template matching result, and the log quantity abnormality detection is needed to be further carried out based on the counting of the matching result, or the variable value is extracted to be carried out for variable abnormality detection;

3) The requirements of different application targets on the template are different, and the template should not be generalized when some type of log anomaly detection is possible, but another thing is possible, and whether the template needs to be generalized or not is a very subjective thing;

4) Lack of feedback-based template tuning capabilities makes it difficult to cope with personalized requirements such as "such templates should be split according to this variable", "this variable should be generalized";

5) The algorithms of the steps (1) to (3) are usually implemented by service code developers, and the steps (4) to (4) are implemented by algorithm personnel, and in addition, the operation and maintenance personnel need to check an abnormality detection result and analyze the abnormality of the service system according to the abnormality detection result. The communication between the operation and maintenance personnel and the algorithm personnel is difficult, the operation and maintenance expert and the algorithm team are separated by an implementation team, the feedback chain is long, and the feedback is not direct.

The application provides an alarm method, a service system, electronic equipment, a computer readable storage medium and a computer program product, which aim to solve the technical problems in the prior art.

The technical solutions of the embodiments of the present application and technical effects produced by the technical solutions of the present application are described below by describing several exemplary embodiments. It should be noted that the following embodiments may be referred to, or combined with each other, and the description will not be repeated for the same terms, similar features, similar implementation steps, and the like in different embodiments.

The embodiment of the application provides an alarm method which is applied to a service system, wherein the service system comprises a service application container, a log center and an alarm module; the business application container comprises at least one preset business code and a log printing library; the business code comprises template content of a log template corresponding to a log to be printed and names of log printing functions to be called, and the log printing library comprises the log printing functions; the log printing function comprises a log level of a log to be printed; the log center is used for storing logs and log templates;

As shown in fig. 2, the method includes:

in step S201, in each first period, the service application container is configured to run at least one service code to obtain at least one log printed by the at least one service code, where each service code calls a corresponding log printing function from the log printing library based on a name of the log printing function to be called in the running process, and outputs the log printed by the service code to the log center through the log printing function.

The service application container of the embodiment of the application comprises at least one preset service code and a log printing library, each service code can run in the service container, the service code can be any service code, such as the code of an application program, the code of an operating system and the like, and each service code can print a log every day.

The business code of the embodiment of the application comprises the template content of the log template of the log to be printed and the name of the log printing function which can be called, wherein the log to be printed is expanded on the basis of the existing log template, the log template comprises template words, the template content belongs to the template words, and parameter words can be added on the basis of the log template to obtain the log. The log printing function to be called is used for outputting a log to a log file and outputting a log template to the log template file.

The log printing functions are located in the log printing library, each log printing function is used for outputting a log, the log printing functions can limit the log level of the log to be printed, and the code format of the business code can be defined as follows: logger.func (format, var.),

the log is a log printing object of a program language, and the log printing object is log for a python language, wherein the log is a java language and a Nodejs language which are generally corresponding to the log; for Golang language, log print object is log;

the func is a log printing function, the log printing function itself comprises log levels, such as log, log. Debug, log. Info, log. Wave, log. Error, track, debug, info, warn and error belong to the log levels, and the template state of the log template corresponding to the log can be determined based on the log level of the log, and the template state comprises a normal state and an abnormal state.

Wherein, the log level is track to represent the log as tracking record; the log level is debug to represent that the log is a debugging record, and the debugging record plays an important role in debugging an application program and is mainly used for printing some running information in the development process; the log is characterized by the rank of info, is important information record, and is commonly used for locating and searching problems; the log level characterizes the log for the warning record of the program, and the warning record is not necessarily a record aiming at error information, but also a record of prompt information; the log level is error representing program exception information displayed by the log, and printing errors and exception information; of course, the log levels may be off and all, where off is used to close all logs, all is used to open all log records, and the priority of the log levels from high to low is as follows: off > error > wave > info > debug > track > all.

The format is characterized by a log template, and the log template is a template word; var is a variable in the log.

Specifically, a certain service code compiled by java language is: a log, debug ("Send request { } to service A", uri); the log is a log printing object, the debug is a log printing function, the log level is represented, the debug level is a "Send request { } to service A" is a log template, and uri is a variable in the log.

Some business codes compiled in the Python language are: logging, info ('sample log from% s', name), wherein logging is a log print object, info is a log print function, log level is characterized, and is also an info level, "sample log from% s" is a log template, and name is a variable in a log.

Some business codes as compiled in Golang language are: log info ("sample log from% v", name), where log is a log print object, info is a log print function, log level is characterized, and "sample log from% v" is a log template, name is a variable in the log.

Some service codes compiled by Nodejs language are as follows: logging, info ('sample log from% s', name), wherein logging is a log print object, info is a log print function, log level is characterized, and is also an info level, "sample log from% s" is a log template, and name is a variable in a log.

The business system of the embodiment of the application also comprises a log center, wherein the log center is used for storing logs and log templates of the logs, and in order to reduce the waste of storage space, the log center does not need the association relation between independent logs and the log templates corresponding to the logs, but stores the logs and the log templates, and the log templates stored in the log center are unique.

In the embodiment of the application, the alarm is sent periodically, and if a plurality of identical anomalies exist in a first period, an alarm is sent once aiming at the identical anomalies so as to reduce the alarm times.

In the embodiment of the application, in each first period, the service container operates at least one service code to obtain at least one log, and in the operation process of each service code, the log printing function with the corresponding name is called from the log printing library based on the name of the log printing function of the log to be called, and after the log printing function is called, the log printing library can output the log corresponding to the service code to the log center.

Step S202, a business application container is used for generating a corresponding log template of a log based on template content of the log template in the business code, determining a log level of the log from a log printing function corresponding to the business code, determining a template state of the log template of the log based on the log level of the log, and outputting the newly added log template and the template state of the newly added log template to a log center if the log template is determined to be the newly added log template; the template state includes an abnormal state.

When the service container runs the service code, the log template of the log can be generated based on the template content of the log template in the service code, the log level of the log is determined according to the log printing function, and specifically, if the log printing function is a track function, the log level is a track level; if the log printing function is a debug function, the log level is a debug level; if the log printing function is an info function, the log level is an info level; if the log printing function is an error function, the log level is an error level; if the log print function is a warn function, the log level is a warn level.

The embodiment of the application can determine the template state of the log template of the log based on the log level of the log, wherein the template state comprises a normal state and an abnormal state, the abnormal state comprises a warn state and an error state, and if the log level of the log is any one of a track level, a debug level and an info level, the template state of the log template of the log is determined to be the normal state; if the log level of the log is the warn level or the error level, determining that the template state of the log template of the log is an abnormal state, wherein if the log level of the log is the warn level, determining that the template state of the log template of the log is the warn state in the abnormal state, and if the log level of the log is the error level, determining that the template state of the log template of the log is the error state in the abnormal state.

When the log template is generated, the template state of the log template can be used as a part of the log template, and the template state of the log template can be determined by directly reading the log template.

As shown in table 1 below, which illustrates a log template of a log generated based on a service code:

"normal", "wave" and "error" in the log template characterize the template state.

Because the logs are all expanded on the basis of the log templates, a plurality of logs possibly have the same template, and the log center does not repeatedly store the log templates when storing the log templates, so that whether the log templates of the logs are newly added log templates needs to be judged, and if the log templates are newly added log templates, the template states of the newly added log templates are output to the log center.

The templates in the log center of the embodiment of the application are not generated based on massive historical logs, but are generated when the printing log is implemented, namely, the generation of the log templates is not limited and the massive historical logs are generated.

Step S203, an alarm module is used for reading logs from a log center in real time; and for each read log, determining a target template matched with the log from all log templates stored in a log center, and if the template state of the target template is determined to be an abnormal state and the alarm for the target template is not initiated in the first period, initiating the alarm for the target template.

According to the embodiment of the application, the alarm module can read the logs from the log query API of the log center in real time, and for each read log in real time, the alarm module can determine the target template matched with the log from all log templates stored in the log center, wherein the log template in the log center is the generated log template under the current service, so that the calculation amount of template matching can be effectively reduced.

If the template state of the target template is determined to be abnormal, judging whether to initiate the alarm for the target template in the first period, if not, initiating the alarm for the target template, and alarming for the target template instead of the log, so that the number of alarms can be reduced.

According to the embodiment of the application, the log template of the log can be determined from the service codes of the printed log in the log printing process, the log is not required to be clustered, each log template is determined according to the clustering result, and the log template is determined in a simple and accurate manner and is not limited by massive logs. In addition, the embodiment of the application determines the template state of the log template corresponding to the log based on the log level of the log, can directly initiate the alarm of the log template aiming at the abnormal state based on the template state of the log template, does not need to carry out excessive analysis on the log template, and simultaneously reduces the calculation amount consumed by using an algorithm to carry out mass log template identification.

The embodiment of the application provides a possible implementation manner, which determines a target template matched with the log from all log templates stored in a log center, and then comprises the following steps:

if the alarm module determines that the template state of the target template does not belong to an abnormal state, adding 1 to the number of logs corresponding to the target template in the first period;

at the end time of the second period, the second period comprises a plurality of first periods, and the total number of logs corresponding to the target template in the second period is determined; the total number is the sum of the log numbers of the log templates in a plurality of first periods;

and if the total number of the logs corresponding to the target template in the second period is greater than a preset threshold value, initiating an alarm aiming at the target template.

If the alarm module determines that the template state of the log template does not belong to an abnormal state, determining that the log template is in a normal state and is a normal log template, and adding 1 to the log number corresponding to the target template in the first period, namely counting the log number corresponding to the target template in the first period.

When the system or the micro-service example in the system is attacked to generate logs, a large number of logs are usually generated, each log may be disguised as a normal log, in this case, the total number of the logs corresponding to each target template in the normal state in the second period may be counted.

The second period in the embodiment of the present application includes a plurality of first periods, for example, the first period may be 2 minutes, the second period may be 3 hours, and if the total number of logs corresponding to the target template in the second period is counted at the end time of the second period, the total number is the sum of the log numbers of the log templates in the plurality of first periods. For example, if the preset threshold is 5000, and the total number of logs corresponding to the target template in the second period is 6000 and greater than the preset threshold 5000, it is indicated that the number of logs corresponding to the target template is abnormal, and the micro-service instance where the target template is located is attacked, and at this time, an alarm for the target template can be initiated.

Of course, if the total number of logs corresponding to the target template in the second period is not greater than the preset threshold, an alarm for the target template is not required to be initiated.

As shown in fig. 3, which schematically illustrates the alert module matching log templates for each log in real time, the alert module includes:

step S301, a log is read from a log center;

step S302, matching the log with each log template stored in a log center;

step S303, determining a target template matched with the log from the log templates;

Step S304, judging whether the template state of the target template is an abnormal state; if yes, go to step S305; if not, executing step S307;

step S305, judging whether to initiate an alarm aiming at the target template in a first period; if not, executing step S306; if yes, reading the next log (not shown in the figure);

step S306, an alarm aiming at a target template is initiated;

step S307, adding 1 to the number of logs corresponding to the target template in the first period;

step S308, at the end time of the second period, determining the total number of logs corresponding to the target template in the second period; the second period comprises a plurality of first periods;

step S309, judging whether the total number exceeds a preset threshold; if yes, go to step S310; if not, reading the next log (not shown);

step S310, an alarm for the target template is initiated.

The embodiment of the application provides a possible implementation mode, wherein the target template comprises an application instance name of an application instance of the printing target template and a micro-service instance name of a micro-service instance; the application instance comprises a plurality of micro service instances;

initiating an alarm for the target template, and then further comprising:

Determining a target application instance required for printing the target template based on the application instance name in the target template;

determining a target micro service instance from a plurality of micro service instances of the target application instance based on the micro service instance name in the target template;

and locating the code segment where the target micro service instance is located, and determining the fault code from the code segment.

In fact, the log template includes, in addition to the template Content, a template status level, a template id, a micro Service of the printing template, a Version of the micro Service, an instance name serviceInstance of the micro Service instance of the printing log template, an application instance name of the application instance of the printing log template, and other attributes serviceProperties of the template Service, and the log template of the log center is stored in the following format:

{

”level”:“error”,-

“content”:”Example log from{}"，

“id”:“xxxx”,

”service”:“xxxx”,

”version”:“xxxx”,

“serviceInstance”:“xxx”,

“application”:“xxx”,

“serviceProperties”:{}

}

after the alarm for the target template is initiated, the embodiment of the application can determine the target application instance required by printing the target template based on the application instance name in the target template; determining a target micro service instance from a plurality of micro service instances of the target application instance based on the micro service instance name in the target template; the fault code may be determined from the code segment by manually analyzing the code segment, or may be determined from the code segment based on other intelligent methods, which is not limited in this embodiment of the present application.

The embodiment of the application provides a possible implementation mode, wherein a log printing library comprises a template buffer area, and the template buffer area comprises log templates of all output logs, hash values of all log templates and template states of all templates;

determining that the log template is a newly added log template comprises:

generating a hash value of a log template of the log, and if the hash value of one log template does not exist in the template buffer area and the hash value of the log template of the log are the same, determining the log template of the log as a newly added log template;

determining that the log template is a newly added log template, and then further comprising:

storing the newly added log template, the hash value of the newly added log template and the template state of the newly added log template in a template buffer zone, and indicating a preset template synchronization process to read the template buffer zone so as to write the newly added log template, the hash value of the newly added log template and the template state of the newly added log template in the template buffer zone into a preset log template file.

The log printing library further comprises a template buffer zone, wherein the template buffer zone stores log templates of all output logs and hash values of all log templates, the hash value of each log template is generated based on a preset hash algorithm, and the preset hash algorithm is not limited.

In the embodiment of the application, for each log, after the log template of the log is obtained, the hash value of the log template of the log is generated, if the hash value of one log template is not determined to be the same as the hash value of the log template of the log in the template buffer, the log template of the log is determined to be the newly added log template, and in this case, the newly added log template and the hash value of the newly added log template can be stored in the template buffer.

For the newly added log template, the hash value of the newly added log template and the template state of the newly added log template are also required to be output to the log center, specifically, the newly added log template, the hash value of the newly added log template and the template state of the newly added log template can be firstly output to a preset log template file based on a preset template synchronization process, the log template file is read by a log sending container, and the newly added log template, the hash value of the newly added log template and the template state of the newly added log template are read and sent to the log center.

If the hash value of one log template in the template buffer area is identical to the hash value of the log template of the log, the log template of the log is stored in the template buffer area, and the log template of the log is not required to be stored.

The embodiment of the application provides a possible implementation mode, wherein the log level of each log is any one of tracking track, debugging debug, information info, alarm warning and error; the template state of each log template is a normal state and an abnormal state;

the template state of the log template is determined by:

if the log level of the log is determined to be any one of track, debug, info, determining that the template state of the log template of the log is a normal state;

if the log level of the log is determined to be warn or error, determining that the template state of the log template of the log is abnormal.

The foregoing embodiments have been described, and will not be described in detail herein.

The embodiment of the application provides a possible implementation manner, which determines a target template matched with a log from all log templates stored in a log center, and comprises the following steps:

converting the log into a first character string, and respectively converting each log template in the log center into a second character string;

for each second string, determining a similarity between the first string and the second string;

and taking the log template corresponding to the character string corresponding to the maximum similarity as a target template matched with the log.

The alarm module of the embodiment of the application can specifically determine the target template matched with the log from all the log templates stored in the log center by converting the log into the first character string, respectively converting all the log templates in the log center into the second character string, and determining the similarity between the first character string and the second character string based on a preset similarity algorithm for each second character string, wherein the preset similarity algorithm can be a pre-similarity algorithm, and the log template corresponding to the character string corresponding to the maximum similarity value can be used as the target template matched with the log.

The embodiment of the application provides a possible implementation manner, which outputs a log printed by a service code to a log center through a log printing function, and comprises the following steps:

the business application container outputs a log printed by the business code to a log file as a new log through a log printing function, and sends a first indication message to the log sending container, wherein the first indication message is used for indicating that the new log is generated;

the log sending container responds to the first indication message, reads a new log from the log file and sends the read new log to the log center;

Outputting the newly added log template and the template state of the newly added log template to a log center, comprising:

the business application container outputs the newly added log template and the template state of the newly added log template to the log template file, and sends a second indication message to the log sending container, wherein the second indication message is used for indicating that the newly added log template is generated;

the log sending container responds to the second indication message, reads the newly-added log template and the template state of the newly-added log template from the log template file, and sends the read newly-added log template and the template state of the newly-added log template to the log center.

The service system of the embodiment of the application further comprises a log sending container SideCar, wherein the log sending container can read a log generated by the service application container and send the read log to a log center, and concretely, after the log sending container obtains the log, the log sending container outputs the log to a log file through a corresponding log printing function and sends a first indication message to the log sending container, wherein the first indication message is used for indicating that a new log is generated; the log sending container reads a new log from the log file in response to the first indication message and sends the read new log to the log center.

In addition, the log sending container can also read the log template generated by the service application container and send the read log template to the log center, specifically, after the log sending container obtains the log template, the log sending container judges whether the log template is a newly added log template through a log printing function, if the log template is determined to be the newly added log template through the log printing function, the newly added log template and the template state of the newly added log template are output to the log template file through a preset template synchronization process, and a second indication message is sent to the log sending container, wherein the second indication message is used for indicating that the newly added log template is generated. The log sending container responds to the second indication message, reads the newly-added log template and the template state of the newly-added log template from the log template file, and sends the read newly-added log template and the template state of the newly-added log template to the log center.

FIG. 4 is a schematic flow chart of another alarm method according to an embodiment of the present application, which is applied to a service system, wherein the service system includes a service application container, a log sending container, a log center and an alarm module; the business application container comprises at least one preset business code and a log printing library; the business codes are used for printing logs; the business code comprises template content of a log template corresponding to a log to be printed and names of log printing functions to be called, and the log printing library comprises the log printing functions; the log printing function comprises a log level of a log to be printed; the log center is used for storing logs and log templates;

Specifically, the method comprises the following steps: in each first period, the business application container operates at least one business code to obtain at least one log; calling a corresponding log printing function from a log printing library based on the name of the log printing function to be called in the running process of each service code, and outputting a corresponding log to a log file through the log printing function;

the business application container generates a corresponding log template based on template content of the log template in the business code for each business code, determines a log level of the log from a log printing function corresponding to the business code, determines a template state of the log template based on the log level of the log, and outputs the newly added log template and the template state of the newly added log template to a preset log template file if the log template is determined to be the newly added log template;

the log sending container reads a new log from the log file and sends the read new log to a log center; the method comprises the steps that a log sending container reads a newly-added log template and a template state of the newly-added log template from a log template file, the read newly-added log template and the template state of the newly-added log template are sent to a REST API interface of a log center, the log center reads the REST API interface, and the new log, the newly-added log template and the template state of the newly-added log template are respectively stored;

The alarm module reads the log from the log inquiry API of the log center;

the alarm module in step (6) determines a target template matched with the log from all log templates stored in the log center for each read log;

step (7) judging whether the template state of the target template is an abnormal state;

if yes, the step (8) is that the alarm for the target template is not initiated in the first period, the alarm for the target template is initiated,

if not, adding 1 to the number of logs corresponding to the target template in the first period;

step (c), the alarm module is used for determining the total number of logs corresponding to the target template in the second period at the end time of the second period, wherein the second period comprises a plurality of first periods; the total number is the sum of the log numbers of the log templates in a plurality of first periods;

step (a)And if the total number of the logs corresponding to the target template in the second period is larger than a preset threshold value, determining that the total number of the logs corresponding to the target template is abnormal, and initiating an alarm for the target template by the alarm module.

The embodiment of the application provides a service system 50, as shown in fig. 5, the service system 50 comprises a service application container 510, a log center 520 and an alarm module 530; the service application container 510 includes at least one preset service code and a log print library; the business codes are used for printing logs; the business code comprises template content of a log template corresponding to a log to be printed and names of log printing functions to be called, and the log printing library comprises the log printing functions; the log printing function comprises a log level of a log to be printed; the log center 520 is used for storing logs and log templates;

In each first period, the service application container 510 is configured to run at least one service code to obtain at least one log, where each service code calls a corresponding log printing function from the log printing library based on a name of the log printing function to be called in the running process, and outputs a corresponding log to the log center 520 through the log printing function;

the service application container 510 is configured to generate, for each service code, a corresponding log template based on template content of the log template in the service code, determine a log level of the log from the log print function corresponding to the service code, determine a template state of the log template based on the log level of the log, and if the log template is determined to be a newly added log template, output the newly added log template and the template state of the newly added log template to the log center 520; the template state includes an abnormal state;

the alarm module 530 is used for reading the log from the log center 520 in real time; and for each read log, determining a target template matched with the log from all log templates stored in a log center, and if the template state of the target template is determined to be an abnormal state and the alarm for the target template is not initiated in the first period, initiating the alarm for the target template.

initiating an alarm for the target template, and then further comprising:

determining that the log template is a newly added log template comprises:

The template state of the log template is determined by:

The embodiment of the application provides a possible implementation mode, and the service system also comprises a log sending container;

outputting the log printed by the business code to a log center through a log printing function, comprising:

The device of the embodiment of the present application may perform the method provided by the embodiment of the present application, and its implementation principle is similar, and actions performed by each module in the device of the embodiment of the present application correspond to steps in the method of the embodiment of the present application, and detailed functional descriptions of each module of the device may be referred to the descriptions in the corresponding methods shown in the foregoing, which are not repeated herein.

The embodiment of the application provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to realize the steps of an alarm method, and compared with the related technology, the method can realize the steps of the alarm method: according to the embodiment of the application, the log template of the log can be determined from the service codes of the printed log in the log printing process, the log is not required to be clustered, each log template is determined according to the clustering result, and the log template is determined in a simple and accurate manner and is not limited by massive logs. In addition, the embodiment of the application determines the template state of the log template corresponding to the log based on the log level of the log, can directly initiate the alarm of the log template aiming at the abnormal state based on the template state of the log template, does not need to carry out excessive analysis on the log template, and reduces the calculation amount consumed by using an algorithm to identify the massive log templates.

In an alternative embodiment, an electronic device is provided, as shown in fig. 6, and an electronic device 6000 shown in fig. 6 includes: a processor 6001 and a memory 6003. In which a processor 6001 is coupled to a memory 6003, such as via a bus 6002. Optionally, the electronic device 6000 may also include a transceiver 6004, the transceiver 6004 may be used for data interactions between the electronic device and other electronic devices, such as transmission of data and/or reception of data and the like. It should be noted that, in practical applications, the transceiver 6004 is not limited to one, and the structure of the electronic device 6000 is not limited to the embodiment of the present application.

The processor 6001 may be a CPU (Central Processing Unit ), general purpose processor, DSP (Digital Signal Processor, data signal processor), ASIC (Application Specific Integrated Circuit ), FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic device, transistor logic device, hardware components, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. The processor 6001 may also be a combination that performs computing functions, e.g., including one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

Bus 6002 may include a path to transfer information between the aforementioned components. Bus 6002 may be a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus or EISA (Extended Industry Standard Architecture ) bus, or the like. The bus 6002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 6, but not only one bus or one type of bus.

The Memory 6003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory ) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory ), a CD-ROM (Compact Disc Read Only Memory, compact disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer, without limitation.

The memory 6003 is for storing a computer program for executing an embodiment of the present application, and is controlled to be executed by the processor 6001. The processor 6001 is configured to execute a computer program stored in the memory 6003 to implement the steps shown in the foregoing method embodiments.

Among them, the electronic device package may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is merely an example, and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.

Embodiments of the present application provide a computer readable storage medium having a computer program stored thereon, which when executed by a processor, implements the steps of the foregoing method embodiments and corresponding content. Compared with the prior art, can realize: according to the embodiment of the application, the log template of the log can be determined from the service codes of the printed log in the log printing process, the log is not required to be clustered, each log template is determined according to the clustering result, and the log template is determined in a simple and accurate manner and is not limited by massive logs. In addition, the embodiment of the application determines the template state of the log template corresponding to the log based on the log level of the log, can directly initiate the alarm of the log template aiming at the abnormal state based on the template state of the log template, does not need to carry out excessive analysis on the log template, and reduces the calculation amount consumed by using an algorithm to identify the massive log templates.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

The embodiment of the application also provides a computer program product, which comprises a computer program, wherein the computer program can realize the steps and corresponding contents of the embodiment of the method when being executed by a processor. Compared with the prior art, can realize: according to the embodiment of the application, the log template of the log can be determined from the service codes of the printed log in the log printing process, the log is not required to be clustered, each log template is determined according to the clustering result, and the log template is determined in a simple and accurate manner and is not limited by massive logs. In addition, the embodiment of the application determines the template state of the log template corresponding to the log based on the log level of the log, can directly initiate the alarm of the log template aiming at the abnormal state based on the template state of the log template, does not need to carry out excessive analysis on the log template, and reduces the calculation amount consumed by using an algorithm to identify the massive log templates.

The terms "first," "second," "third," "fourth," "1," "2," and the like in the description and in the claims and in the above figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate, such that the embodiments of the application described herein may be implemented in other sequences than those illustrated or otherwise described.

It should be understood that, although various operation steps are indicated by arrows in the flowcharts of the embodiments of the present application, the order in which these steps are implemented is not limited to the order indicated by the arrows. In some implementations of embodiments of the application, the implementation steps in the flowcharts may be performed in other orders as desired, unless explicitly stated herein. Furthermore, some or all of the steps in the flowcharts may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or phases may be performed at the same time, or each of these sub-steps or phases may be performed at different times, respectively. In the case of different execution time, the execution sequence of the sub-steps or stages can be flexibly configured according to the requirement, which is not limited by the embodiment of the present application.

The foregoing is only an optional implementation manner of some implementation scenarios of the present application, and it should be noted that, for those skilled in the art, other similar implementation manners based on the technical ideas of the present application are adopted without departing from the technical ideas of the scheme of the present application, which also belongs to the protection scope of the embodiments of the present application.

Claims

1. The alarm method is characterized by being applied to a service system, wherein the service system comprises a service application container, a log center and an alarm module; the business application container comprises at least one preset business code and a log printing library; the business code comprises template content of a log template corresponding to a log to be printed and names of log printing functions to be called, and the log printing library comprises the log printing functions; the log printing function comprises a log level of a log to be printed; the log center is used for storing logs and log templates;

the method comprises the following steps:

in each first period, the business application container operates the at least one business code to obtain at least one log printed by the at least one business code, and each business code calls a corresponding log printing function from the log printing library based on the name of the log printing function to be called in the operation process, and outputs the log printed by the business code to the log center through the log printing function;

the business application container also generates a corresponding log template of the log based on template content of the log template in the business code, determines a log level of the log from a log printing function corresponding to the business code, determines a template state of the log template of the log based on the log level of the log, and outputs the newly added log template and the template state of the newly added log template to the log center if the log template is determined to be the newly added log template; the template state includes an abnormal state;

The alarm module reads logs from the log center in real time; and for each read log, determining a target template matched with the log from all log templates stored in the log center, and if the template state of the target template is determined to be an abnormal state and the alarm for the target template is not initiated in the first period, initiating the alarm for the target template.

2. The method of claim 1, wherein the determining the target template to which the log matches from among the log templates stored in the log center further comprises:

if the alarm module determines that the template state of the target template does not belong to an abnormal state, adding 1 to the number of logs corresponding to the first period of the target template;

at the end time of a second period, the second period comprises a plurality of first periods, and the total number of logs corresponding to the target template in the second period is determined; the total number is the sum of the log numbers of the log templates in the first plurality of cycles;

and if the total number of the logs corresponding to the target template in the second period is larger than a preset threshold value, initiating an alarm aiming at the target template.

3. The method according to claim 1 or 2, wherein the target template comprises an application instance name of an application instance of the target template and a micro service instance name of a micro service instance; the application instance comprises a plurality of micro service instances;

the method comprises the steps of initiating an alarm for the target template, and further comprising:

determining a target micro-service instance from a plurality of micro-service instances of the target application instance based on the micro-service instance name in the target template;

and locating the code segment where the target micro service instance is located, and determining a fault code from the code segment.

4. The method according to claim 1, wherein the log print library comprises a template buffer zone, and the template buffer zone comprises log templates of each outputted log, hash values of each log template and template states of each template;

the determining that the log template is a newly added log template includes:

The determining that the log template is a newly added log template further comprises:

storing the newly added log template, the hash value of the newly added log template and the template state of the newly added log template in the template buffer, and indicating a preset template synchronization process to read the template buffer so as to write the newly added log template, the hash value of the newly added log template and the template state of the newly added log template in the template buffer into a preset log template file.

5. The method of claim 1, wherein the log level of each log is any one of trace, debug, info, alert warn, and error; the template state of each log template is a normal state and an abnormal state;

the template state of the log template is determined by:

and if the log level of the log is determined to be warn or error, determining that the template state of the log template of the log is an abnormal state.

6. The method of claim 1, wherein said determining a target template to which the log matches from among log templates stored in the log center comprises:

converting the log into a first character string, and respectively converting each log template in a log center into a second character string;

7. The method of claim 1, wherein the business system further comprises a log sending container;

the outputting, by the log printing function, the log printed by the service code to the log center includes:

the business application container outputs the log printed by the business code to a log file through the log printing function as a new log, and sends a first indication message to the log sending container, wherein the first indication message is used for indicating that the new log is generated;

the log sending container responds to the first indication message, reads the new log from the log file and sends the read new log to a log center;

The outputting the newly added log template and the template state of the newly added log template to the log center includes:

the business application container outputs the newly added log template and the template state of the newly added log template to a log template file, and sends a second indication message to the log sending container, wherein the second indication message is used for indicating that the newly added log template is generated;

the log sending container responds to the second indication message, reads the newly-added log template and the template state of the newly-added log template from the log template file, and sends the read newly-added log template and the template state of the newly-added log template to a log center.

8. The service system is characterized by comprising a service application container, a log center and an alarm module; the business application container comprises at least one preset business code and a log printing library; the business code comprises template content of a log template corresponding to a log to be printed and names of log printing functions to be called, and the log printing library comprises the log printing functions; the log printing function comprises a log level of a log to be printed; the log center is used for storing logs and log templates;

In each first period, the service application container is used for running the at least one service code to obtain at least one log printed by the at least one service code, and each service code calls a corresponding log printing function from the log printing library based on the name of the log printing function to be called in the running process, and the log printed by the service code is output to the log center through the log printing function;

the service application container is used for generating a corresponding log template based on template content of the log template in the service code for each service code, determining a log level of the log from a log printing function corresponding to the service code, determining a template state of the log template based on the log level of the log, and outputting the newly added log template and the template state of the newly added log template to the log center if the log template is determined to be the newly added log template; the template state includes an abnormal state;

the alarm module is used for reading logs from the log center in real time; and for each read log, determining a target template matched with the log from all log templates stored in the log center, and if the template state of the target template is determined to be an abnormal state and the alarm for the target template is not initiated in the first period, initiating the alarm for the target template.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to carry out the steps of the method according to any one of claims 1-7.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-7.