CN113360467B

CN113360467B - Log file analysis method, device, medium and computer equipment

Info

Publication number: CN113360467B
Application number: CN202010145599.3A
Authority: CN
Inventors: 张磊
Original assignee: Wuhan Douyu Network Technology Co Ltd
Current assignee: Wuhan Douyu Network Technology Co Ltd
Priority date: 2020-03-05
Filing date: 2020-03-05
Publication date: 2022-07-05
Anticipated expiration: 2040-03-05
Also published as: CN113360467A

Abstract

The invention provides a method, a device, a medium and computer equipment for analyzing log files, wherein the method comprises the following steps: polling the target log file, and reading the log content of the current row of logs; judging whether the value of the line variable is null, and if not, matching the starting line log content of the target log file; if the log content of the starting line is successfully matched, judging whether the starting content of the log content of the starting line is the log content meeting the preset requirement, and if the starting content of the log content of the starting line is the log content meeting the preset requirement, adding the log content of the starting line to the current line set; acquiring the content of the rest row logs one by one, and adding the rest row logs to the current row set; analyzing the log content in the current row set based on a preset log structure, and acquiring and printing an analysis result corresponding to a target log file; therefore, the target log file can be automatically analyzed to obtain the analysis result of the log content of each row, and the analysis efficiency of the log file is improved by the full-automatic analysis mode.

Description

Log file analysis method, device, medium and computer equipment

Technical Field

The invention belongs to the technical field of data analysis, and particularly relates to a method, a device, a medium and computer equipment for analyzing a log file.

Background

In the Android system, each application generates a large amount of log information, such as warning log information, fault log information, and the like, in the development and test processes. If crash or other failure problems occur, the BUG can be eliminated by searching corresponding log information to determine the corresponding cause of the failure.

In the prior art, various Android log information are usually confused, and are generally analyzed and processed by manually comparing log files, so that the log information cannot be efficiently and quickly analyzed, the manual labor intensity is increased, and the analysis efficiency cannot be ensured. Moreover, when manual analysis is performed, errors are inevitable, and the analysis precision cannot be guaranteed.

Disclosure of Invention

Aiming at the problems in the prior art, the embodiment of the invention provides a method, a device, a medium and computer equipment for analyzing a log file, which are used for solving the technical problems that in the prior art, when an android log is analyzed manually, the analysis efficiency is low and the analysis precision cannot be ensured.

The invention provides a method for analyzing a log file, which comprises the following steps:

acquiring a target log file based on the file name of the target log file;

polling the target log file, calling a reading function to read the log content of the current row of logs in the polling process aiming at the current row of logs, and assigning the log content of the current row of logs to a preset row variable; the current row of logs is any row of logs in the target log file;

judging whether the value of the row variable is empty or not, and if not, matching the starting row log content of the target log file;

if the starting line log content of the target log file is successfully matched, judging whether the starting content of the starting line log content is the log content meeting the preset requirement, if so, directly obtaining the starting line log content, and adding the starting line log content to the current line set;

based on a preset loop strategy, acquiring the content of the remaining row logs one by one, and adding the remaining row logs to the current row set;

analyzing the log content in the current row set by using a preset analysis function based on a preset log structure to obtain an analysis result corresponding to the target log file; the log structure includes: the method comprises the following steps of (1) log grade, log printing time, process number, thread number, log label TAG information, file name, file method name and line number, and log specific content;

printing an analysis result corresponding to the target log file; and the current row set is used for performing row-changing processing on a plurality of rows of log contents so as to display the log contents of each row in the target log file in a row-changing manner when the analysis result corresponding to the target log file is printed.

Optionally, if the initial content of the start line log content is a log content meeting a preset requirement, the method further includes:

calling a first time recording function to record the analysis starting time of the target log file;

and after the analysis of the log content of the starting line is finished, calling a line marking function to set the analysis mark of the log content of the starting line as a preset identification value.

Optionally, the analyzing the log content in the current row set by using a preset analysis function based on a preset log structure to obtain an analysis result corresponding to the target log file includes:

acquiring a target character string corresponding to a log content group in a current row set;

screening the target character string by using a preset screening function to obtain a first character string;

segmenting the first character string based on a preset first segmentation symbol to obtain a first array;

intercepting the first array based on a preset interception function to obtain a plurality of subdata, and reassigning the subdata to the first array;

judging whether the data length of the first array meets a preset log structure, if so, partitioning fourth bit data in the first array based on a preset second partition character to obtain a second array;

judging whether the data length of the second array meets the preset log structure, if so, taking out the zero-bit data, the first-bit data and the second-bit data in the second array, assigning the zero-bit data to a preset file name variable, assigning the first-bit data to a preset file method name variable, and assigning the second-bit data to a preset line number variable;

calling a preset interception function to intercept data between fifth bit data and last bit data from the first array, and splicing the intercepted data to form a second character string;

calling a time analysis function to analyze the second bit data in the first array to obtain log printing time;

packaging the zero-bit data in the first array, the log printing time, the second-bit data in the first array, the third-bit data in the first array, the preset file name variable, the preset file method name variable, the preset line number variable and the second character string into a preset packaging object by using a packaging function;

calling a result returning function to perform format conversion on the packaged object and returning a result, wherein the result is an analysis result corresponding to the target log file; the first-order data in the first array is a log level, and the second-order data in the first array is a process number and a thread number; and the third bit of data in the first array is log TAG information.

Optionally, if the initial content of the start line log content is a log content that does not meet a preset requirement, the method further includes:

judging whether the length of the log content in the current row set is greater than 0, if so, calling a copy function to copy the log content from the current row set in a deep copy mode;

assembling the copied log contents into a third character string by using a preset assembling function;

filtering the third character string by using a preset filtering function to obtain a fourth character string;

calling a clearing function to empty the log content in the current row set;

and analyzing the fourth character string by using the analysis function to obtain an analysis result corresponding to the target log file.

Optionally, the method further comprises:

and in the polling process, counting the number of the log lines to be analyzed of the target log file by using a statistical function.

Optionally, if the value of the row variable is null, the method further includes:

printing the analyzed log line number in the target log file;

calling a second time recording function to count the analysis ending time of the target log file;

printing the time overhead of the analysis process based on the analysis starting time and the analysis ending time of the target log file;

and finishing polling the target log file by using a loop termination function.

The invention also provides a device for analyzing the log file, which comprises:

an acquisition unit configured to acquire a target log file based on a file name of the target log file;

the reading unit is used for polling the target log file, calling a reading function to read the log content of the current row of logs in the polling process aiming at the current row of logs, and assigning the log content of the current row of logs to a preset row variable; the current row of logs is any row of logs in the target log file;

the adding unit is used for judging whether the value of the line variable is empty or not, and if the value of the line variable is not empty, matching the starting line log content of the target log file;

the analysis unit is used for analyzing the log content in the current row set by using a preset analysis function based on a preset log structure to obtain an analysis result corresponding to the target log file; the log structure includes: the method comprises the following steps of (1) log grade, log printing time, process number, thread number, log label TAG information, file name, file method name and line number, and log specific content;

the printing unit is used for printing an analysis result corresponding to the target log file; and the current row set is used for performing row-changing processing on a plurality of rows of log contents so as to display the log contents of each row in the target log file in a row-changing manner when the analysis result corresponding to the target log file is printed.

Optionally, the parsing unit is further configured to:

intercepting the first array based on a preset interception function to obtain a plurality of subdata, and re-assigning the subdata to the first array;

judging whether the data length of the first array meets a preset log structure or not, if so, segmenting fourth bit data in the first array based on a preset second segmentation symbol to obtain a second array;

The invention also provides a computer-readable storage medium, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the method of any of the above.

The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the program.

The embodiment of the invention provides a method, a device, a medium and computer equipment for analyzing log files, wherein the method comprises the following steps: acquiring a target log file based on the file name of the target log file; polling the target log file, calling a reading function to read the log content of the current row of logs in the polling process aiming at the current row of logs, and assigning the log content of the current row of logs to a preset row variable; the current row of logs is any row of logs in the target log file; judging whether the value of the line variable is null or not, and if not, matching the beginning line log content of the target log file; if the starting line log content of the target log file is successfully matched, judging whether the starting content of the starting line log content is the log content meeting the preset requirement, if so, directly obtaining the starting line log content, and adding the starting line log content to the current line set; based on a preset loop strategy, acquiring the content of the remaining row logs one by one, and adding the remaining row logs to the current row set; analyzing the log content in the current row set by using a preset analysis function based on a preset log structure to obtain an analysis result corresponding to the target log file; the log structure includes: log grade ] [ log printing time ] [ process number, thread number ] [ log label TAG information ] [ file name, file method name, line number ] [ log specific content ]; printing an analysis result corresponding to the target log file; and the current row set is used for performing row-changing processing on a plurality of rows of log contents so as to display the log contents of each row in the target log file in a row-changing manner when the analysis result corresponding to the target log file is printed. Therefore, after the target log file is obtained, the log contents of all rows of the target log file can be automatically analyzed according to the preset log structure, and the analysis result of all rows of the log contents is obtained, so that the full-automatic analysis mode not only can improve the analysis efficiency of the log file, but also can avoid errors caused by manual analysis, and improve the analysis precision; and when the analysis result is printed, the log contents of each line can be displayed in a line-changing manner, so that the staff can conveniently look up the log contents, and the needed log contents can be quickly found.

Drawings

Fig. 1 is a schematic flow chart of a method for parsing a log file according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of an apparatus for parsing a log file according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a computer device for parsing a log file according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a computer medium for parsing a log file according to an embodiment of the present application.

Detailed Description

The method aims to solve the technical problems that when android logs are analyzed manually, the analysis efficiency is low and the analysis precision cannot be ensured. The invention provides a method, a device, a medium and computer equipment for analyzing a log file.

The technical solution of the present invention is further described in detail by the accompanying drawings and the specific embodiments.

Example one

The embodiment provides a method for analyzing a log file, as shown in fig. 1, the method includes:

s110, acquiring the target log file based on the file name of the target log file;

in order to analyze the log file, firstly, the target log file needs to be acquired based on the file name of the target log file.

The method comprises the steps of packaging an integral analysis process by utilizing a pre-designed packaging function, wherein the packaging function is Def _ trace _ file (file), the packaging function defines an acquisition function through Def keywords, the name of the acquisition function is trace _ file, the acquisition function has a parameter file for entering parameters, and the file can represent a specific log file name.

Based on this, if the target log file needs to be acquired, the file name of the target log file is assigned to the entry parameter file, that is, the target log file can be acquired based on the file name of the target log file.

In order to better understand the technical solution of the present application, the following acquisition function is introduced.

In the parse _ log _ file function, predefined initial variables are included, and the initial variables are mainly used for subsequent analysis of the target log file. For example, the initial variables may include:

line _ count is 0, and the variable is mainly used to count the number of parsed log lines in the log file.

The variable is in a set form, mainly in order to enable a multi-line log to be processed in a line feed manner during analysis when the target log file has the multi-line log, and to enable the log content of each line of the target log file to be displayed in a line feed manner when an analysis result corresponding to the target log file is printed. Here, since the variable is a set, the log contents in multiple lines can be converted into one line of data to be stored in the set, so that the log file can be parsed according to the log structure preset in the present application. It can be understood that, in the set, each line of log has a corresponding start mark and end mark, so that even if the log is converted into a line of data, the analysis result of each line of log can be obtained during analysis, and line feed is realized during printing.

The first _ count is 0, and the variable is used to indicate whether the start content of the start line log content is the log content that meets the preset requirement. Here, because some description information may exist in the header of the log file, and the description information is useless, when the initial content of the start line log content is the description information, it is indicated that the initial content is useless content and is the log content that does not meet the preset requirement. Therefore, whether the initial content of the log content of the initial line is the log content meeting the preset requirement can be marked by the first _ count variable.

The start _ time is 0, and the variable is used for identifying the analysis starting time;

end _ time is 0, and the variable is used for identifying the analysis ending time; the analysis starting time and the analysis ending time are used for counting the whole time consumption of log file analysis, so that the analysis efficiency can be obtained.

And the result _ json _ list [ ] is a result set and is used for storing a result analysis result corresponding to each row of log content, and the analysis result is packaged in a json format, so that the system can be better butted with and used by other systems, wherein the analysis result corresponding to each row of log content is data in the json format.

Further, after the target log file is obtained, the target log file is opened by calling a file opening method with "r" and "encoding" utf-8') as f, and the target log file is marked as an f variable.

S111, polling the target log file, calling a reading function to read the log content of the current row of logs in the polling process, and assigning the log content of the current row of logs to a preset row variable;

after the target log file is opened, the process of analyzing the target log file line by line can be realized in a loop body. Specifically, an infinite polling body is realized by calling while True to poll the target log file.

In the polling process, for a current line log of a target log file, calling a read function line ═ f. And the current row of logs is any row of logs in the target log file.

And in the polling process, adding 1 to the number of line _ counts by using a statistical function line _ count +1, so as to count the number of log lines to be analyzed in the target log file.

S112, judging whether the value of the line variable is null or not, and if not, matching the starting line log content of the target log file;

and judging whether the value of the row variable is null None, if not, indicating that the row variable has contents, and at the moment, analyzing the target log file.

Specifically, during parsing, the log content of the start line of the target log file needs to be matched first. Match the log content of the start line by using a regular expression res ═ re. Wherein, the meaning of the regular expression' \\ S \ is that when there is a mark which takes the symbol "[" as the beginning and contains any capital letter and takes the symbol "]" as the end, the successful matching of the log content of the start line is determined.

S113, if the starting line log content of the target log file is successfully matched, judging whether the starting content of the starting line log content is the log content meeting the preset requirement, if so, directly obtaining the starting line log content, and adding the starting line log content to the current line set;

if the starting line log content of the target log file is successfully matched, judging whether the starting line log content is empty or not by using a judgment function if is not None, and if not, indicating that the starting content of the starting line log content is obtained. However, because the log file generally has description information in the header, and the description information is useless for us, when the initial content is the description information, it is not in accordance with the preset requirement, and therefore it is necessary to determine whether the initial content of the log content of the start line is a log (which can also be understood as a required content) in accordance with the preset requirement.

The program for judging whether the initial content of the log content of the initial line is the log content meeting the preset requirement is implemented as follows:

if the variable first _ count is True, it indicates that the start content of the start line log content is the log content meeting the preset requirement, the start line log content is directly obtained, and the start line log content is added to the current _ line set by calling an adding function current _ line.

And based on a preset loop strategy continue, obtaining the residual row log contents one by one, obtaining the residual row log contents, and adding the residual row log contents to the current row set. Thus, although the log contents of multiple lines are stored in the current line set, the log contents of multiple lines in the current line set are stored in one line of data, and a line feed mark is arranged between the log contents of each line.

As an optional embodiment, if the starting content of the log content in the starting row is the log content meeting the preset requirement, the method further includes:

calling a first time recording function start _ time ═ int (round (time. time ()) 1000) to record the analysis starting time of the target log file;

and after the analysis of the log content of the starting line is finished, calling a line marking function first _ count ═ False to set the analysis mark of the log content of the starting line as a preset identification value.

The preset identification value is False, so that the judgment condition that the starting content of the starting line log content is the log content meeting the preset requirement is avoided being executed for multiple times before the starting line log content is obtained.

S114, analyzing the log content in the current row set by using a preset analysis function based on a preset log structure, and acquiring an analysis result corresponding to the target log file;

and then analyzing the log content in the current row set by using a preset analysis function based on a preset log structure to obtain an analysis result corresponding to the target log file.

The log structure in this application is: [ Log level ] [ Log printing time ] [ Process number, thread number ] [ Log TAG information ] [ File name, File method name, line number ] [ specific content of Log ]. Here, because the log structure in the present application partitions the log content in a "[ ]" partitioning manner, compared with the prior art that the log file is parsed by using an xml tag format to describe the log information, a large number of description tags can be avoided, and thus the memory occupied by the log file is reduced.

Compared with the mode of describing the log information by using the json format in the prior art, the method can reduce the resolution complexity and improve the resolution efficiency. When the json format is used to describe the log information, a large number of { } exists, which results in complicated parsing and reduced parsing efficiency.

Based on this, the log information is described in the segmentation mode of [ ], the analysis process is convenient and fast, and balance between the analysis efficiency and the occupied resource overhead can be ensured.

Here, the specific process of parsing the log content (one line of data of multiple lines of log content conversion) in the current line set by using the parsing function includes:

calling a time analysis function to analyze the second bit data in the first array to obtain the log printing time;

calling a result returning function to perform format conversion on the packaged object and returning a result, wherein the result is an analysis result corresponding to the target log file; the first bit data in the first array is a log level, and the second bit data in the first array is a process number and a thread number; and the third bit of data in the first array is log TAG information.

Specifically, a function line _ json is called, self, part _ line _ str a (current _ line _ str) is used for parsing the target character string, part _ line _ str is a preset parsing function, and the implementation process of the part _ line _ str parsing function is as follows:

screening the target character string line _ str A by using a screening function line _ array ═ line _ str A.strip (). replace (' \ n ', '). replace (' ] ', ') and split ([ '), and removing all space exchange rows and ' ] ' in the line _ str A to obtain a first character string;

and then, segmenting the first character string according to a first segmentation character '[' in the segmentation function to obtain a first array line _ array.

Here, since the first data in the first array can be determined to be null data according to the pre-designed log structure, the null data needs to be removed. Specifically, the first array is truncated from the 1 st bit to the len (line _ array) th bit based on a preset truncation function line _ array [1: len (line _ array) ], so as to obtain a plurality of subdata, and the plurality of subdata are re-assigned to the first array. This corresponds to the removal of the 0 th bit data (i.e., the first dummy data).

And checking whether the length of the first ARRAY meets a preset log structure or not by using a check function if LEN (line _ ARRAY) > -filecheck. Line _ ARRAY _ LEN is a constant number, and the constant data is determined according to a log structure. The log structure in the application is as follows: the constant data is 6 because the constant data is [ log level ] [ log printing time ] [ process number, thread number ] [ log TAG information ] [ file name, file method name, line number ] [ log specific content ].

It is determined that the data length of the first array satisfies a preset log structure if the data length of the first array is greater than or equal to 6. If the data length of the first array does not satisfy the preset log structure, an end function return None is called to end the analysis process.

If the data length of the first array meets a preset log structure, based on a preset second partition character ',' the fourth bit data in the first array is partitioned by using a partition function code _ info _ array ═ line _ array [4]. split (','), so as to obtain a second array code _ info _ array.

Similarly, the data length of the second array code _ info _ array needs to be checked. Specifically, the check is performed by using a check function if LEN (code _ info _ ARRAY) > -filecheck, method _ ARRAY _ LEN, wherein the fourth bit of data in the log structure is [ file name, file method name, line number ], that is, the data length of the fourth bit of data is 3, and thus the filecheck, method _ ARRAY _ LEN is a constant number of 3.

If the data length of the second array meets the log structure, the data of the second array needs to be constructed, which specifically includes: taking out the zero-bit data, the first bit data and the second bit data in the second array code _ info _ array, assigning the zero-bit data to a preset file name variable code _ file, assigning the first bit data to a preset file method name variable code _ method, and assigning the second bit data to a preset line number variable code _ line, wherein the program implementation is as follows:

code_file＝code_info_array[0]

code_method＝code_info_array[1]

code_line＝code_info_array[2]

after construction is finished, intercepting data between fifth-bit data and last-bit data from a first array line _ array by calling an intercepting function log _ str ═ join (line _ array [5: len (line _ array) ]), splicing the intercepted data in a join mode to form a second character string, and assigning the second character string to a log _ str variable.

Here, the reason why the truncated data is spliced is that there may be a plurality of '[' dividers in the first array, which results in that the data length of the first array may be greater than 6, and the truncated data needs to be re-spliced together in order to ensure the integrity of the log content.

And after splicing the intercepted data, calling a time analysis function time _ info-self, part _ time (line _ array [1]) to analyze the first bit data in the first array, and obtaining specific log printing time information. The first-order data in the first array is a log level, and the second-order data in the first array is a process number and a thread number; and the third bit of data in the first array is log TAG information.

Calling a packaging function to package the zero-bit data in the first array, the log printing time, the second-bit data in the first array, the third-bit data in the first array, the preset file name variable, the preset file method name variable, the preset line number variable and the second character string into a preset ditt type packaging object log _ info; the procedure was implemented as follows:

then, format conversion is carried out on the packaged object by calling a result return function (log _ info, intensire _ ascii ═ False) and a result is returned, wherein the result is an analysis result corresponding to the target file log; here, json. copies (log _ info, intensire _ ascii) indicates that the content of log _ info is converted into json format. The analysis result of each row of log content is a json data line _ json, that is, the analysis result corresponding to the target file log at least comprises a json data.

Thus, the analysis flow of the whole log content is completed.

Further, as an optional embodiment, if the value of the line variable is null, it indicates that the parsing to the last line of the target log file is finished, and the method further includes:

printing the analyzed log line number in the target log file;

and finishing polling the target log file by using a loop termination function.

Specifically, the program is implemented as follows:

that is, if the line is None, then the not line satisfies the condition, at this time, the number of lines of the whole log parsing is printed by counting the function print ("parsing finishes line _ count:% d"% line _ count), the time information corresponding to the parsing finishes is counted by calling the second time recording function end _ time ═ int (round (time. time ()) 1000), then the time overhead of the whole parsing process is printed by calling the print ("parsing consumes:% d"% (end _ time-start _ time)), and finally the whole infinite polling process is terminated by means of break, at this time, the target log file parsing is finished.

Further, as an optional embodiment, if the initial content of the log content in the initial row is the log content that does not meet the preset requirement, the log content that does not meet the preset requirement needs to be filtered, and then the filtered log content is analyzed, which specifically includes:

judging whether the length of the log content in the current _ line of the current line set is greater than 0, if so, calling a copy function to copy the log content in the current set from the current line set in a deep copy mode;

calling a clearing function to empty the log content in the current row set;

For example, if the length of the log content in current _ line of the current line set is greater than 0, it indicates that the log content already exists in the current _ line, and then a copy function copy. The copying by using the deep copy mode can avoid the loss of the original information after the content in the current _ line is cleared.

The copied log content is assembled into a third character string line _ str B through a join function, the third character string line _ str B is screened through a screening function, spaces and lines in the third character string line _ str B are taken out, and a fourth character string is obtained.

Clearing () to clear the current log content in the current line set current _ line after copying; similarly, a function line _ json is called, self, part _ line _ str B (current _ line _ str) is used for analyzing the fourth character string, and an analysis result corresponding to the target log file is obtained.

Here, the specific process of parsing the fourth character string by the parsing function pars _ line _ str is completely the same as the parsing process of the target character string by the parsing function, and thus is not described again.

Further, as an optional embodiment, since an analysis result corresponding to each row of log content is a json data line _ json, if the line _ json is not empty, the method further includes:

and calling a storage function result _ json _ list.apend (line _ json) to store an analysis result corresponding to each row of log content in a preset result set result _ json _ list, and calling an adding function current _ line.apend (res.string) again to store the log content of the subsequent log file to be analyzed into the emptied current line set so as to continuously analyze the subsequent log file to be analyzed.

In the above description, if the initial content of the start line log content is the log content that does not meet the preset requirement, the program corresponding to the executed method is implemented as follows:

and S115, printing an analysis result corresponding to the target log file.

After the target log file is analyzed, the analysis result corresponding to each row of logs can be obtained, the analysis result corresponding to the content of each row of logs is printed, and the analysis result is displayed in the console. However, since the information amount of the log file is very large, if the analysis result of the target log file is printed to the console for display by the conventional scheme, the information of the content of the target log file may not be displayed completely. Therefore, in the application, the analysis result of each row of log content is output to a preset result file in sequence, and then the result file is printed to the console for displaying.

Before printing the analysis result corresponding to the current row of log content, whether the result file exists needs to be judged, if yes, the current log content in the result file is deleted (the current log content is the analysis result corresponding to the previous row of log content of the current row of log) so as to avoid conflict. The procedure was implemented as follows:

if os.path.exists(r'D:\tmp\result.txt'):

os.remove(r'D:\tmp\result.txt')

thus, the analysis result of the target log file can be printed to the console for displaying.

For example, taking a certain log file as an example, when the log file has only one row of log contents, the log information finally displayed on the console is:

[I] [2019-07-16+ 8.001: 34:15.152] [0,10786144064] [ Step-DouyuBatman ] [ DYAppDelegateBusinessConfigservice.m, application,69] [ thermal remediation start sync, app version (594), current patch version (0) ]

When the log file comprises a plurality of lines of log contents, the log information displayed on the console is as follows:

in addition, after the analysis result corresponding to each row of log content is stored in the preset result set result _ json _ list, an external interface is reserved so as to be capable of being connected with other systems and used.

It is worth noting that when log information is displayed on the console, if fault information occurs, the fault information can be marked, so that the fault information can be conveniently positioned by workers, and faults can be timely eliminated.

Based on the same inventive concept, the application also provides a log file analysis device, which is described in detail in embodiment two.

Example two

The present embodiment provides an apparatus for parsing a log file, as shown in fig. 2, the apparatus includes: an acquisition unit 21, a reading unit 22, a judgment unit 23, an addition unit 24, an analysis unit 25, and a printing unit 26; wherein the content of the first and second substances,

an acquisition unit 21 configured to acquire a target log file based on a file name of the target log file;

the reading unit 22 is used for polling the target log file, calling a reading function to read the log content of the current row of logs in the polling process aiming at the current row of logs, and assigning the log content of the current row of logs to a preset row variable; the current row of logs is any row of logs in the target log file;

a determining unit 23, configured to determine whether the value of the line variable is null, and if the value of the line variable is not null, match the starting line log content of the target log file;

an adding unit 24, configured to, if the start line log content of the target log file is successfully matched, determine whether the start content of the start line log content is a log content meeting a preset requirement, and if the start content of the start line log content is a log content meeting the preset requirement, directly obtain the start line log content and add the start line log content to a current line set;

the analysis unit 24 is configured to analyze the log content in the current row set by using a preset analysis function based on a preset log structure, and obtain an analysis result corresponding to the target log file;

and a printing unit 25, configured to print an analysis result corresponding to the target log file.

Specifically, in order to implement the analysis of the log file, the obtaining unit 21 needs to obtain the target log file based on the file name of the target log file.

Based on this, if the target log file needs to be acquired, the file name of the target log file is assigned to the entry parameter file, and the acquiring unit 21 may acquire the target log file based on the file name of the target log file.

The variable is in a set form, mainly in order to enable line feed processing to a multi-line log during analysis when the multi-line log exists in a target log file, and to enable line feed display of the log contents of each line of the target log file during printing of an analysis result corresponding to the target log file. Here, since the variable is a set, the log contents in multiple lines can be converted into one line of data to be stored in the set, so that the log file can be parsed according to the log structure preset in the present application. It can be understood that, in the set, each line of log has a corresponding start mark and end mark, so that even if the log is converted into a line of data, the analysis result of each line of log can be obtained during analysis, and line feed is realized during printing.

Further, after acquiring the target log file, the acquiring unit 21 opens the target log file by calling a file opening method with open (file, mode ═ r, encoding ═ utf-8') as f, and marks the target log file as an f variable.

After the target log file is opened, the process of analyzing the target log file line by line can be realized in a loop body. Specifically, the reading unit 22 implements an infinite polling body by calling while True to poll the target log file.

In the polling process, the number of line _ counts is added by 1 using a statistical function line _ count +1, so as to count the number of analyzed log lines of the target log file.

The determining unit 23 is configured to determine whether the value of the row variable is null, and if the value of the row variable is not null, it indicates that there is content in the row variable, and at this time, the target log file needs to be analyzed.

Specifically, the determining unit 23 determines whether the value of the row variable is null, and if the value of the row variable is not null, it indicates that there is content in the row variable, and at this time, the target log file needs to be analyzed.

In parsing, the determining unit 23 needs to match the log content of the start line of the target log file. Match the log content of the start line by using a regular expression res ═ re. Wherein, the meaning of the regular expression' \\ S \ is that when there is a mark which takes the symbol "[" as the beginning and contains any capital letter and takes the symbol "]" as the end, the successful matching of the log content of the start line is determined.

If the log content of the start line of the target log file is successfully matched, the adding unit 24 determines whether the log content of the start line is empty by using a determination function if is not None, and if not, it indicates that the start content of the log content of the start line is already obtained. However, because the log file generally has description information in the header, and the description information is useless for us, when the initial content is the description information, it is not in accordance with the preset requirement, and therefore it is necessary to determine whether the initial content of the log content of the start line is a log (which can also be understood as a required content) in accordance with the preset requirement.

And based on a preset loop strategy continue, obtaining the residual row log content one by one, obtaining the residual row log content, and adding the residual row log content to the current row set. Thus, although the log contents of multiple lines are stored in the current line set, the log contents of multiple lines in the current line set are stored in one line of data, and a line feed mark is arranged between the log contents of each line.

As an alternative embodiment, the apparatus further comprises: the parsing unit 25, if the initial content of the log content in the initial row is the log content meeting the preset requirement, the parsing unit 25 is further configured to:

Then, the parsing unit 25 parses the log content in the current row set by using a preset parsing function based on a preset log structure, and obtains a parsing result corresponding to the target log file.

The log structure in this application is: [ Log level ] [ Log printing time ] [ Process number, thread number ] [ Log TAG information ] [ File name, File method name, line number ] [ specific content of Log ]. Here, because the log structure in the present application is to divide the log content in a "[ ]" division manner, compared with the prior art that the log information is described in an xml tag form when the log file is parsed, a large number of description tags can be avoided, and the memory occupied by the log file is further reduced.

Here, the analysis unit 25 is specifically configured to:

packaging the zero-bit data in the first array, the log printing time, the second-bit data in the first array, the third-bit data in the first array, the preset file name variable, the preset file method name variable, the preset line number variable and the third character string into a preset packaging object by using a packaging function;

Specifically, the parsing unit 25 parses the target character string by calling a function line _ json ═ self, part _ line _ str a (current _ line _ str), where part _ line _ str is a preset parsing function, and the realization process of the part _ line _ str parsing function is as follows:

Here, since it can be determined that the first data in the first array is empty data according to the pre-designed log structure, it is necessary to remove the empty data. Specifically, the first array is truncated from the 1 st bit to the len (line _ array) th bit based on a preset truncation function line _ array [1: len (line _ array) ], so as to obtain a plurality of subdata, and the plurality of subdata are re-assigned to the first array. This corresponds to the removal of the 0 th bit data (i.e., the first dummy data).

code_file＝code_info_array[0]

code_method＝code_info_array[1]

code_line＝code_info_array[2]

Here, the reason for splicing the intercepted data is that there may be a plurality of '[' dividers in the first array, which results in that the data length of the first array may be greater than 6, and therefore, in order to ensure the integrity of the log content, the intercepted data needs to be spliced together again.

Thus, the analysis flow of the whole log content is completed.

Further, as an alternative embodiment, if the value of the line variable is null, it indicates that the last line of the target log file has been resolved, and the parsing unit 25 is configured to:

printing the analyzed log line number in the target log file;

and finishing polling the target log file by using a loop termination function.

Specifically, the program is implemented as follows:

that is, if the line is None, then the not line satisfies the condition, and at this time, the parsing unit 25 may print the number of lines of the whole log parsing by counting the function print ("parsing finishes line _ count:% d"% line _ count), count the time information corresponding to the parsing finishes by calling the second time recording function end _ time ═ int (time. time ()) to print the time overhead of the whole parsing process by calling the print ("parsing total time consumption:% d"% (end _ time-start _ time)), and finally terminate the whole infinite polling process by means of break, where the target log file parsing finishes.

Further, as an optional embodiment, if the initial content of the log content in the start line is the log content that does not meet the preset requirement, the log content that does not meet the preset requirement needs to be filtered, and then the filtered log content is analyzed, where the analyzing unit 25 is further configured to:

calling a clearing function to empty the log content in the current row set;

Further, as an optional embodiment, because an analysis result corresponding to each row of log content is a json data line _ json, if the line _ json is not empty, the adding unit 24 is further configured to:

calling a storage function result _ json _ list. ap pend (line _ json) to store an analysis result corresponding to each row of log content in a preset result set result _ json _ list;

and calling the adding function current _ line.applied (res.string) again to store the log content of the subsequent log file to be analyzed into the emptied current line set current _ line so as to continuously analyze the subsequent log file to be analyzed.

after the target log file is analyzed, the printing unit 26 may obtain an analysis result corresponding to each line of log, print the analysis result corresponding to the content of each line of log, and display the analysis result in the console. However, since the information amount of the log file is very large, if the analysis result of the target log file is printed to the console for display by the conventional scheme, the information of the content of the target log file may not be displayed completely. Therefore, in the application, the analysis result of each row of log content is output to a preset result file in sequence, and then the result file is printed to the console for displaying.

if os.path.exists(r'D:\tmp\result.txt'):

os.remove(r'D:\tmp\result.txt')

For example, taking a log file as an example, when the log file has only one row of log contents, the log information finally displayed on the console is:

[I] [2019-07-16+ 8.001: 34:15.152] [0,10786144064] [ Step-DouyuBatman ] [ DYAppDelegateBusinessConfigservice. m, application,69] [ Hot repair start sync, app version (594), Current Patch version (0) ]

in addition, after the analysis result corresponding to each row of log content is stored in a preset result set result _ json _ list, an external interface is reserved so as to be capable of being connected with other systems in a butt joint mode and used.

The method, the device, the medium and the computer equipment for analyzing the log file provided by the embodiment of the invention have the beneficial effects that at least:

the embodiment of the invention provides a method, a device, a medium and computer equipment for analyzing log files, wherein the method comprises the following steps: the embodiment of the invention provides a method, a device, a medium and computer equipment for analyzing log files, wherein the method comprises the following steps: acquiring a target log file based on the file name of the target log file; polling the target log file, calling a reading function to read the log content of the current row of logs in the polling process aiming at the current row of logs, and assigning the log content of the current row of logs to a preset row variable; the current row of logs is any row of logs in the target log file; judging whether the value of the line variable is null or not, and if not, matching the beginning line log content of the target log file; if the starting line log content of the target log file is successfully matched, judging whether the starting content of the starting line log content is the log content meeting the preset requirement, if so, directly obtaining the starting line log content, and adding the starting line log content to the current line set; based on a preset loop strategy, acquiring the content of the remaining row logs one by one, and adding the remaining row logs to the current row set; analyzing the log content in the current row set by using a preset analysis function based on a preset log structure to obtain an analysis result corresponding to the target log file; the log structure includes: the method comprises the following steps of (1) log grade, log printing time, process number, thread number, log label TAG information, file name, file method name and line number, and log specific content; printing an analysis result corresponding to the target log file; the current row set is used for performing row-changing processing on a plurality of rows of log contents so as to display the log contents of each row in the target log file in a row-changing manner when the analysis result corresponding to the target log file is printed; therefore, after the target log file is obtained, the log contents of all rows of the target log file can be automatically analyzed according to the preset log structure, and the analysis result of all rows of the log contents is obtained, so that the full-automatic analysis mode not only can improve the analysis efficiency of the log file, but also can avoid errors caused by manual analysis, and improve the analysis precision; and when the analysis result is printed, the log contents of each line can be displayed in a line-changing manner, so that the staff can conveniently look up the log contents, and the needed log contents can be quickly found.

EXAMPLE III

The present embodiment provides a computer apparatus, as shown in fig. 3, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320, where the processor 320 executes the computer program 311 to implement the following steps:

acquiring a target log file based on the file name of the target log file;

if the starting line log content of the target log file is successfully matched, judging whether the starting content of the starting line log content is the log content meeting the preset requirement, if so, directly obtaining the starting line log content, and adding the starting line log content into the current line set;

In particular, when the processor 320 executes the computer program 311, any one of the first embodiment can be implemented.

Since the computer device described in this embodiment is a device used for implementing the method for parsing a log file in the first embodiment of the present application, based on the method described in the first embodiment of the present application, a person skilled in the art can understand a specific implementation manner of the computer device in this embodiment and various variations thereof, so that a detailed description of how the server implements the method in the embodiment of the present application is not given here. The equipment used by those skilled in the art to implement the methods in the embodiments of the present application is within the scope of the present application.

Based on the same inventive concept, the application provides a storage medium corresponding to the fourth embodiment, which is described in detail in the fourth embodiment.

Example four

The present embodiment provides a computer-readable storage medium 400, as shown in fig. 4, on which a computer program 411 is stored, the computer program 411 implementing the following steps when executed by a processor:

acquiring a target log file based on the file name of the target log file;

judging whether the value of the line variable is null or not, and if not, matching the beginning line log content of the target log file;

In a specific implementation, when the computer program 411 is executed by a processor, any one of the first embodiment may be implemented.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

The above description is only exemplary of the present application and should not be taken as limiting the scope of the present application, as any modifications, equivalents, improvements, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

The above description is only exemplary of the present invention and should not be taken as limiting the scope of the present invention, and any modifications, equivalents, improvements, etc. that are within the spirit and principle of the present invention should be included in the present invention.

Claims

1. A method for parsing a log file, the method comprising:

acquiring a target log file based on the file name of the target log file;

printing an analysis result corresponding to the target log file; the current row set is used for performing row-changing processing on a plurality of rows of log contents so as to display the log contents of each row in the target log file in a row-changing manner when the analysis result corresponding to the target log file is printed; wherein the content of the first and second substances,

the analyzing the log content in the current row set by using a preset analyzing function based on a preset log structure to obtain an analyzing result corresponding to the target log file includes:

2. The method of claim 1, wherein if the starting content of the starting line log content is a log content meeting a preset requirement, the method further comprises:

3. The method of claim 1, wherein if the start content of the start line log content is a log content that does not meet a preset requirement, the method further comprises:

calling a clearing function to empty the log content in the current row set;

4. The method of claim 1, wherein the method further comprises:

5. The method of claim 1, wherein if the value of the row variable is null, the method further comprises:

printing the analyzed log line number in the target log file;

and finishing polling the target log file by using a loop termination function.

6. An apparatus for parsing a log file, the apparatus comprising:

the adding unit is used for judging whether the value of the line variable is null or not, and if the value of the line variable is not null, matching the starting line log content of the target log file;

the printing unit is used for printing an analysis result corresponding to the target log file; the current row set is used for performing row-changing processing on a plurality of rows of log contents so as to display the log contents of each row of the target log file in a row-changing manner when the analysis result corresponding to the target log file is printed; wherein the content of the first and second substances,

the parsing unit is further configured to:

7. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 5.

8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 5 when executing the program.