CN114840672A - Log analysis method, device, equipment and storage medium - Google Patents
Log analysis method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114840672A CN114840672A CN202210481364.0A CN202210481364A CN114840672A CN 114840672 A CN114840672 A CN 114840672A CN 202210481364 A CN202210481364 A CN 202210481364A CN 114840672 A CN114840672 A CN 114840672A
- Authority
- CN
- China
- Prior art keywords
- event
- log
- log analysis
- log file
- graph
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 130
- 238000000034 method Methods 0.000 claims abstract description 36
- 238000004590 computer program Methods 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 14
- 230000015654 memory Effects 0.000 claims description 14
- 238000007639 printing Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 13
- 238000011161 development Methods 0.000 abstract description 5
- 238000012423 maintenance Methods 0.000 abstract description 5
- 230000000007 visual effect Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 5
- 238000009877 rendering Methods 0.000 description 5
- 239000003086 colorant Substances 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000006698 induction Effects 0.000 description 2
- 238000007670 refining Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000007794 visualization technique Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/34—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Quality & Reliability (AREA)
- Library & Information Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application discloses a log analysis method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a first log file; acquiring running information of a first event from the first log file according to a keyword of the first event; drawing a graph identifier of the first event in a log analysis graph according to the running information of the first event, wherein the graph identifier of the first event is used for representing the running information of the first event; and displaying the drawn log analysis graph on a log analysis interface. Therefore, according to keywords of the first event concerned by the user, the running information of the first event is acquired from the first log file, the running information of the first event is converted into a graphic identifier and is drawn in a log analysis graph, a more visual log analysis result is provided for the user in a graphic display mode, the log analysis efficiency is improved, and further the development and maintenance efficiency of the chip is improved.
Description
Technical Field
The present application relates to computer technologies, and in particular, to a log analysis method, apparatus, device, and storage medium.
Background
The output log (log) of the system plays a very critical role during chip development and maintenance. Generally, developers can reserve various printing statements of key information in corresponding software in advance; in the running process of the chip, the printing statements are called by a system, and key information such as software and hardware running states, input and output parameters, abnormity and the like is printed in a log file, so that relevant engineers can analyze and verify the chip flow and trace problems.
The current popular log analysis methods are all based on searching keyword content in log files. As described above, the keyword is information written in the software print statement by the developer in advance, and is generally used to mark the state of the key process or display the key parameter information. Extracting a target text containing target keywords in the original log, refining and sorting the target text into a relatively simplified log file, and printing the log file in a specified window according to the sequence of the target text appearing in the original log. Therefore, the efficiency of combing the chip operation flow and verifying the key data by related workers can be greatly improved.
However, in the chip workflow combing and problem tracing process, although the log analysis method can sort and summarize the interested target text according to the sequence of the target text appearing in the original log, the obtained log file content is still too much, and further analysis and deduction are required by workers, and the log analysis efficiency is low, so that the development and maintenance efficiency of the chip is affected.
Disclosure of Invention
In order to solve the foregoing technical problems, embodiments of the present application are intended to provide a log analysis method, apparatus, device and storage medium.
The technical scheme of the application is realized as follows:
in a first aspect, a log analysis method is provided, including:
acquiring a first log file;
acquiring running information of a first event from the first log file according to a keyword of the first event;
drawing a graph identifier of the first event in a log analysis graph according to the running information of the first event, wherein the graph identifier of the first event is used for representing the running information of the first event;
and displaying the drawn log analysis graph on a log analysis interface.
In a second aspect, there is provided a log analysis apparatus, including:
a first acquisition unit configured to acquire a first log file;
a second obtaining unit, configured to obtain, according to a keyword of a first event, operation information of the first event from the first log file;
the processing unit is used for drawing a graph identifier of the first event in a log analysis graph according to the running information of the first event, wherein the graph identifier of the first event is used for representing the running information of the first event;
and the display unit is used for displaying the drawn log analysis chart on the log analysis interface.
In a third aspect, a log analysis device is provided, including: a processor and a memory configured to store a computer program capable of running on the processor,
wherein the processor is configured to perform the steps of the aforementioned method when running the computer program.
In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the aforementioned method.
The embodiment of the application provides a log analysis method, a log analysis device, log analysis equipment and a log analysis storage medium. Therefore, when the log file is analyzed, a more visual log analysis result is provided for a user in a graphical display mode, the log analysis efficiency is improved, and the development and maintenance efficiency of the chip is further improved.
Drawings
FIG. 1 is a first flowchart of a log analysis method according to an embodiment of the present application;
FIG. 2 is a first diagram of a log analysis graph in an embodiment of the present application;
FIG. 3 is a second diagram of a log analysis graph in an embodiment of the present application;
FIG. 4 is a third schematic diagram of a log analysis graph in an embodiment of the present application;
FIG. 5 is a fourth schematic diagram of a log analysis graph in an embodiment of the present application;
FIG. 6 is a second flowchart of a log analysis method according to an embodiment of the present application;
FIG. 7 is a diagram illustrating a structure of a journal text in an embodiment of the present application;
FIG. 8 is a schematic diagram illustrating a structure of a log analysis apparatus according to an embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of a log analysis device in an embodiment of the present application.
Detailed Description
So that the manner in which the features and elements of the present embodiments can be understood in detail, a more particular description of the embodiments, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another, and are not necessarily used to describe a particular order or sequence. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. Nor should such information be limited to such terms
For example, the first event is not specifically referred to as an event in the embodiment of the present application, and the analysis method of the first event may be understood as a log analysis method of any event or multiple events.
Fig. 1 is a schematic first flow chart of a log analysis method in an embodiment of the present application, and as shown in fig. 1, the method may specifically include:
step 101: acquiring a first log file;
here, the first log file may be an original log file output by the chip, or a partial log file obtained by intercepting the original log file.
Illustratively, when the first log file is a partial log file of the original log file. In some embodiments, the obtaining the first log file comprises: acquiring an original log file; acquiring at least one section of interesting text from the original log file according to preset selection configuration information, and generating a second log file; and acquiring a first event text containing the keyword of the first event from the second log file according to the keyword of the first event, and generating the first log file.
Further, the method further comprises: determining the event type of the first event according to preset event classification information; and saving the first event text to a position corresponding to the event type of the first event in the first log file. In other words, after the event texts of different event types are extracted, the event texts are classified and stored according to the event types, so that the drawing of a subsequent log analysis graph is facilitated.
In other embodiments, the obtaining the first log file includes: acquiring an original log file; and acquiring at least one section of interesting text from the original log file according to preset selection configuration information, generating a second log file, and taking the second log file as the first log file.
And selecting configuration information as related parameters of the interesting text, and cutting the original log file to extract the interesting text. The selected configuration information can be set by a user according to the log analysis requirement. For example, the start line and end line of the text of interest, or the start line and size of the text of interest, or the start print timestamp and end print timestamp of the text of interest, etc.
Illustratively, if multiple sections of interesting texts are obtained, a first separator is set between different interesting texts in the second log file. And if a plurality of first event texts are obtained, setting a second division house between different first event texts in the first log file. Illustratively, the first delimiter and the second delimiter are the same or different, the delimiter is used to divide the text, and the delimiter can be a carriage return, a comma, a period, a dash, a wavy line, or other special symbols not present in the log file.
Therefore, the original log file is segmented to obtain a part of text which is interested by the user, and the part of text is analyzed, so that the log analysis efficiency can be improved.
The log analysis tool automatically acquires an original log file output by the chip; according to the setting of a user, cutting the whole original log file, intercepting one or more sections of interesting texts which are interesting to the user, and storing the texts to a second log file (hereinafter referred to as file 2); according to the keyword information inputted by the user, the event text (using carriage return as a separator or period as a separator) containing the keyword in the file 2 is extracted by a search mode and stored in the first log file (file 1). And acquiring the running information of the first event from the first log file.
For example, for intercepting an interested part in an original log file, a start line and an end line which directly require a user to input the interested part can be selected, and the original log file is directly intercepted according to the start line and the end line to obtain a file 2; or searching based on special keywords input by the user, intercepting log information of the part where the special keywords appear to obtain a file 2; or the two schemes are combined, the special keywords are searched in the starting line and the ending line appointed by the user, the context with a certain length of the appearance point of the special keywords is intercepted, and the file 2 is obtained. The method is beneficial to acquiring the part which is really interested by the user from a longer log file, refining and sorting the part into a relatively simplified log file, and graphically displaying the simplified log file, so that the log analysis efficiency can be improved, and the analysis efficiency of the longer log file is improved more remarkably.
Step 102: acquiring running information of a first event from the first log file according to a keyword of the first event;
wherein the operation information is used for representing the operation condition of the first event. Illustratively, the operational information includes at least one of: the occurrence time of the event, the occurrence frequency of the event and the abnormal condition of the event. For example, the occurrence time of the event may be a print time stamp of an event text or an event time stamp, where the event time stamp represents the time inside the chip when the event corresponding to one row of the log occurs (or is triggered). Generally, the print timestamp in the log is printed at the beginning of each line log, and the event timestamp may appear anywhere in the corresponding line log except at the beginning (possibly at the end, and possibly in the line).
The occurrence frequency of the event may be the number of occurrences of the event within a preset time period, or the number of occurrences of the event per unit time. For example, the occurrence frequency of different types of events can be analyzed by a histogram.
Illustratively, when the running information includes the occurrence time of the event, in some embodiments, the obtaining the running information of the first event from the first log file according to the keyword of the first event includes: acquiring a printing time stamp of a first event text from the first log file according to a keyword of the first event; and taking the printing time stamp of the first event text as the occurrence time of the first event.
When the running information includes the occurrence time of an event, the obtaining the running information of the first event from the first log file according to the keyword of the first event includes: acquiring a printing time stamp of a first event text from the first log file according to a keyword of the first event; and obtaining the occurrence time of the first event according to the printing time stamp of the first event text and the reference time stamp of the first log file.
For example, the print timestamp may be selected as the time information of the event, and in a scenario with a high requirement for the event time information, the event timestamp is obtained by using the print timestamp and the reference timestamp of the first log file, that is, the event timestamp is selected as the time information.
For example, the reference timestamp may be a print timestamp of a start line of the first log file, the reference timestamp may be a print timestamp of a start line of the original file when the first log file is not the original log file, and the reference timestamp may be a timestamp set according to a log analysis requirement. Illustratively, the reference timestamp is 18:04:45.692399, the print timestamp of the text of the first event is 18:04:45.793668, and the difference between the print timestamp and the reference timestamp is the time of occurrence of the first event, i.e., 0.101269. Further, for convenience of expression, the actually obtained occurrence time of the first event may be converted into time information that is easy to graphically display, for example, the occurrence time is converted into a time between 0 and 60 s. Here, the first event text is used to represent the first event.
Illustratively, when the running information includes the occurrence frequency of the event, the obtaining the running information of the first event from the first log file according to the keyword of the first event includes: acquiring the occurrence frequency of a first event text from the first log file according to keywords of the first event; and determining the occurrence frequency of the first event according to the occurrence frequency of the first event text.
Illustratively, when the running information includes an abnormal condition of an event, the obtaining the running information of the first event from the first log file according to a keyword of the first event includes: acquiring the abnormal time of a first event from the first log file according to the abnormal keyword of the first event; and drawing an abnormal identifier of the first event in the log analysis graph according to the abnormal time of the first event.
That is, the keyword of the first event may be a keyword in which the first event normally operates and/or a keyword in which the first event abnormally operates. According to the log analysis requirement, the running information of the first event can be the running information of the first event in normal running and/or the running information of the first event in abnormal running. Different graphical indicia may also be used to distinguish between normal operation and abnormal operation.
Step 103: drawing a graph identifier of the first event in a log analysis graph according to the running information of the first event, wherein the graph identifier of the first event is used for representing the running information of the first event;
illustratively, the log analysis graph includes at least one of: scatter plot, bar chart, line chart, pie chart, bar chart, area chart.
Illustratively, the log analysis graph includes a first dimension; the drawing a graph identifier of the first event in a log analysis graph according to the running information of the first event comprises the following steps: determining a first drawing parameter of the first event in the first dimension according to the running information of the first event; and drawing the graphic identification of the first event in the log analysis graph according to the first drawing parameter.
Here, the first graphic drawing parameter may be a graphic position, a graphic shape, a graphic color, or the like. For example, when the log analysis map scatter diagram is obtained, the first dimension is a time dimension, and the position of the first dimension is determined according to the starting time of the first event, wherein the shape and the color of the graph can be preset. When the log analysis graph is a pie graph, the first dimension is a time dimension, and the proportion of the first event in the pie graph is determined according to the proportion of the occurrence time of the first event to the total occurrence time of the chip. In practical application, the matched log analysis graph can be selected according to the analysis requirement of the log file, and details are not repeated here.
For example, when the running information is the occurrence time, as shown in fig. 2, the keyword 1 represents the first event, t0 is the reference timestamp, and t1-t5 is the first to fifth occurrence times of the first event.
Illustratively, the running information may also occur from a start time to an end time, as shown in fig. 3, the vertical axis represents the first event by using a keyword 1, the horizontal axis represents time, t0 is a reference time stamp, t1-t2 is a first occurrence time period of the first event, t3-t4 is a second occurrence time period of the first event, and t5-t6 is a third occurrence time period of the first event.
Illustratively, the log analysis graph further comprises a second dimension;
the method further comprises the following steps: determining the event type of the first event according to preset event classification information; determining a first drawing parameter of the first event on the second dimension according to the event type of the first event; and drawing the graphic identifier of the first event in the log analysis graph according to the first drawing parameter and the second drawing parameter.
That is, different event types can be drawn in one log analysis graph, and the running information of different event types can be separately displayed, so that the transverse comparison between different event types can be realized.
Illustratively, in some embodiments, the log analysis graph is a two-dimensional graph, the first plotted parameter comprises a horizontal coordinate, and the second plotted parameter comprises a vertical coordinate; or, the first rendering parameter includes a vertical coordinate, and the second rendering parameter includes a horizontal coordinate. In other embodiments, the log analysis graph may also be a three-dimensional graph that is graphically displayed in three dimensions according to two types of operational information and event types.
For example, the event classification information may be classified according to event keywords, and different event keywords represent different event types. The event classification information may be classified according to module division of the chip, for example, the chip is divided into three functional modules, and events executed by one functional module are divided into the same event type. The event classification information may be classified according to event functions, for example, low power consumption events, high power consumption events, image processing events, audio processing events, and the like.
Illustratively, when the running information is occurrence time, the running information is classified according to event keywords, as shown in fig. 4, a vertical coordinate position corresponding to a keyword 1 represents a first event type, a vertical coordinate position corresponding to a keyword 2 represents a second event type, a vertical coordinate position corresponding to a keyword n represents an nth event type, t0 is a reference timestamp, t1-t5 represents first to fifth occurrence times of the first event type, and occurrence times (not shown in fig. 4) of the second event type and the nth event type refer to the first event type.
In practice, the keyword representing an event type may be one or more keywords, and the keywords 1, 2 and n are not used to limit the number of keywords. In practical applications, different event types can be distinguished by different graphical identifiers, such as different shapes and different colors.
For example, in some embodiments, when a first event includes multiple sub-events, the plotting a graphical representation of the first event in a log analysis graph according to the running information of the first event includes: determining a first drawing parameter of the first event in a first dimension according to the running information of a plurality of sub-events in the first event; and drawing the graphic identification of the first event in the log analysis graph according to the first drawing parameter.
That is, the graphical identifier of the first event is affected by a plurality of sub-events, the sub-events can be analyzed independently, and the sub-events can also be analyzed as a whole, that is, the operation condition of each sub-event affects the operation condition of the whole event. Illustratively, the first event includes a sub-event 1 and a sub-event 2, where the sub-event 1 and the sub-event 2 run normally at the same time, which indicates that the first event runs normally, and if there is an abnormal operation, which indicates that the first event runs abnormally.
For example, the log analysis graph may be a dot graph or a line graph, and for the dot graph, one point corresponds to one keyword event; for a line graph, one line contains a start point and an end point, corresponding to two keyword events. As shown in fig. 5, at time t1 to time t2 and at time t4 to time t5, a sub-event a and a sub-event B occur successively, indicating that the events run normally, the sub-event a occurs at time t3, and the sub-event B does not occur thereafter, an anomaly identifier is added to the log analysis graph to indicate that the first event is abnormal, and the anomaly identifier may be marked with a highlight color.
Step 104: and displaying the drawn log analysis graph on a log analysis interface.
By adopting the technical scheme, the operation information of the first event is acquired from the first log file according to the keyword of the first event concerned by the user, the operation information of the first event is converted into the graphic identifier, and the graphic identifier is drawn in the log analysis graph. Therefore, when the log file is analyzed, a more visual log analysis result is provided for a user in a graphical display mode, the log analysis efficiency is improved, and the development and maintenance efficiency of the chip is further improved.
Compared with the method that after keyword search is carried out, sorting and induction are carried out according to the line number sequence of the lines of the texts containing the keywords, or sorting and induction are carried out according to the printing time sequence of the texts containing the keywords, the method further needs analysis and deduction of workers, and the operation information of the events cannot be visually displayed to the workers. According to the method and the device, the extracted event texts are subjected to graphical analysis and display, and compared with the existing log analysis method, the log analysis efficiency can be improved.
To further illustrate the object of the present application based on the above embodiments of the present application, as shown in fig. 6, the method specifically includes:
step 601: acquiring an original log file;
step 602: obtaining an interesting text from an original log file, and storing the interesting text in a file 2;
here, the file 2 is a second log file in the above-described embodiment of the present application.
Step 603: acquiring a first event text matched with keywords of a first event, and storing the first event text in a file 1;
here, document 1 is a first log document in the embodiment of the present application. Specifically, after automatically acquiring an original log file, the log analysis tool selects an interested part according to user settings and stores the interested part in the file 2; acquiring keywords of an analysis event, separating the keywords by separators such as carriage returns or periods and the like from the file 2 based on a keyword searching method, acquiring an event text containing the keywords, and storing the event text as a file 1; matching the time information in the file 1 with the event text; and drawing by using the matching information to obtain a log analysis chart.
It should be noted that, in some embodiments, the file 1 is directly used as the first log file, or the original log file is directly used as the first log file in this embodiment, that is, the original log file is not required to be intercepted, and the running information of the first event is directly obtained therefrom.
Step 604: inquiring time information of the event in the file 1;
here, the time information may be a print time stamp of the event file, or an event time stamp.
The system output log file usually includes two times. One is a print time stamp, and the marked time can be regarded as the time when a line of log is printed, and the time in a log printing module; the other is an event timestamp, which represents the time data inside the chip when the event corresponding to one row of log occurs (or is triggered). Generally, the print timestamp in the log is printed at the beginning of each line log, and the event timestamp may appear anywhere in the corresponding line log except at the beginning (possibly at the end, and possibly in the line), as shown in fig. 7. In practical applications, the event timestamp may be understood as the time difference between the time when the chip starts to occur and the time when the event occurs. For matching convenience, the embodiment of the application can select the printing time stamp as the time information for use; in a scenario with a high requirement on event time, the event timestamp may be selected as the time information for matching. Generally, the print timestamp in the log is printed at the beginning of each line log, and the event timestamp may appear anywhere in the corresponding line log except at the beginning (possibly at the end, and possibly in the line).
Step 605: classifying the time information of different events according to keywords;
illustratively, different event types are labeled with different colors or shapes according to preset priorities or abnormality, so that the visualization degree can be improved.
Step 606: and drawing a log analysis graph by taking time as an X axis and taking an event type as a Y axis.
Here, the log analysis graph includes an X-axis and a Y-axis, where one is a first dimension and one is a second dimension. The X axis is a time dimension, the Y axis is an event type dimension, and the process is generally the following process for drawing the log analysis graph: and taking a certain time point as a reference time point, taking the difference between the time point matched with the event and the reference time point as the abscissa of the event, and distributing the ordinate of the event according to different keywords and user settings or system sequencing. In the image drawing, a dot diagram or a line diagram may exist, and for the dot diagram, one point corresponds to one keyword event; for a line graph, one line contains a start point and an end point, corresponding to two keyword events. For a line graph, one line comprises more than three points corresponding to more than three keyword events, for the line graph, two keyword events can be regarded as a whole, and if the two events occur successively, the events are regarded as normal; if one of the events does not occur, the event is considered to be abnormal.
That is to say, the core of the log analysis in the embodiment of the present application is: automatically acquiring an original log file output by a chip; according to the setting of a user, cutting the whole original log file, intercepting a certain section or a plurality of sections of interesting texts which are interesting to the user, and storing the texts to a second log file (hereinafter referred to as file 2); according to the keyword information inputted by the user, the event text (using carriage return as a separator or period as a separator) containing the keyword in the file 2 is extracted by a search mode and stored in the first log file (file 1). Classifying the texts in the file 1 according to different keywords, simultaneously searching occurrence time points of events corresponding to the keywords, and matching event information with time information one by one; classifying the event information and the corresponding time information according to different keywords; establishing a log analysis graph, selecting a certain time as a reference time point, representing the time by an X-axis, drawing the classified events on different horizontal lines parallel to the X-axis according to categories, and taking the coordinate of the events in the X direction as the difference value between the time corresponding to the events and a time base point; and classifying the events according to the priority order or whether the events are abnormal or not, and labeling various information by using different shapes or colors for distinguishing. The completed log analysis graph is shown in fig. 4.
Therefore, the information contained in the original log file of the chip is utilized, and the first log file to be processed is reduced and obtained by means of intercepting and searching keywords and the like; and matching the events with the time to enable the events containing the keywords to correspond to the corresponding time information, and performing visual processing to obtain a simple, effective and reliable log analysis sequence diagram.
It should be noted that the log analysis method provided in the embodiment of the present application may be applied to not only visualization processing of the chip output log file, but also analysis of output log files of other modules and systems. The core point is based on the screening and keyword search of the log files, and makes full use of the time information of the events in the log files, visualizes the obtained event information, and draws corresponding log analysis graphs.
For parameters (such as the selection configuration information of the region of interest, keywords of an event, and the like) to be set in the scheme, priori knowledge can be adopted for presetting, and a corresponding neural network model can be trained to determine the parameters, so that the intelligent degree of a log analysis tool is improved. Meanwhile, the log analysis chart mentioned in the embodiment of the application is only an exemplary illustration of a visualization method, and the obtained result can also be drawn by adopting other image display modes.
In order to implement the method of the embodiment of the present application, based on the same inventive concept, an embodiment of the present application further provides a log analysis apparatus, as shown in fig. 8, where the log analysis apparatus 80 includes:
a first acquisition unit 801 for acquiring a first log file;
a second obtaining unit 802, configured to obtain, according to a keyword of a first event, operation information of the first event from the first log file;
a processing unit 803, configured to draw, according to the running information of the first event, a graph identifier of the first event in a log analysis graph, where the graph identifier of the first event is used to represent the running information of the first event;
and the display unit 804 is used for displaying the drawn log analysis graph on the log analysis interface.
In some embodiments, the log analysis graph comprises a first dimension;
a processing unit 803, configured to determine, according to the running information of the first event, a first rendering parameter of the first event in the first dimension; and drawing the graphic identification of the first event in the log analysis graph according to the first drawing parameter. Here, the first graphic drawing parameter may be a graphic position, a graphic shape, a graphic color, or the like.
In some embodiments, the log analysis graph further comprises a second dimension;
the processing unit 803 is configured to determine an event type of the first event according to preset event classification information; determining a second drawing parameter of the first event on the second dimension according to the event type of the first event; and drawing the graphic identifier of the first event in the log analysis graph according to the first drawing parameter and the second drawing parameter.
In some embodiments, the log analysis graph is a two-dimensional graph, the first plotted parameter comprises a horizontal coordinate, and the second plotted parameter comprises a vertical coordinate; or, the first rendering parameter includes a vertical coordinate, and the second rendering parameter includes a horizontal coordinate.
In some embodiments, the operational information includes at least one of: the occurrence time of the event, the occurrence frequency of the event and the abnormal condition of the event.
In some embodiments, the operational information includes a time of occurrence of an event; a processing unit 803, configured to obtain, from the first log file, a print timestamp of a first event text according to a keyword of the first event; and obtaining the occurrence time of the first event according to the printing time stamp of the first event text and the reference time stamp of the first log file.
In some embodiments, when the running information includes the occurrence frequency of the event, the processing unit 803 is configured to obtain the occurrence frequency of the text of the first event from the first log file according to a keyword of the first event; and determining the occurrence frequency of the first event according to the occurrence frequency of the first event text. Here, the frequency of occurrence of the event may be the number of occurrences of the event within a preset time period, or the number of occurrences of the event per unit time.
In some embodiments, the first obtaining unit 801 is configured to obtain an original log file; acquiring at least one section of interesting text from the original log file according to preset selection configuration information, and generating a second log file; acquiring a first event text containing the keyword of the first event from the second log file according to the keyword of the first event, and generating the first log file; or, the second log file is used as the first log file. Therefore, the original log file is segmented to obtain a part of text which is interested by the user, and the part of text is analyzed, so that the log analysis efficiency can be improved.
Further, the first obtaining unit 801 is further configured to determine an event type of the first event according to preset event classification information; and saving the first event text to a position corresponding to the event type of the first event in the first log file.
Based on the hardware implementation of each unit in the log analysis apparatus, an embodiment of the present application further provides a log analysis device, as shown in fig. 9, where the log analysis device 90 includes: a processor 901 and a memory 902 configured to store a computer program capable of running on the processor;
wherein the processor 901 is configured to execute the method steps in the previous embodiments when running the computer program.
Of course, in actual practice, as shown in fig. 9, the various components of the log analysis device 90 are coupled together by a bus system 903. It is understood that the bus system 903 is used to enable communications among the components. The bus system 903 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, the various buses are designated in the figure as the bus system 903.
In practical applications, the processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, and a microprocessor. It is understood that the electronic devices for implementing the above processor functions may be other devices, and the embodiments of the present application are not limited in particular.
The Memory may be a volatile Memory (volatile Memory), such as a Random-Access Memory (RAM); or a non-volatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard Disk (HDD), or a Solid-State Drive (SSD); or a combination of the above types of memories and provides instructions and data to the processor.
In practical applications, the apparatus may be a log analysis device, or may be a chip applied to the log analysis device. In this application, the apparatus may implement the functions of multiple units through either software or hardware or a combination of software and hardware, so that the apparatus may execute the log analysis method provided in any of the above embodiments. And the technical effects of the technical schemes of the device can refer to the technical effects of the corresponding technical schemes in the log analysis method, which is not repeated in this application.
In an exemplary embodiment, the present application further provides a computer readable storage medium, such as a memory including a computer program, which is executable by a processor of a log analysis device to perform the steps of the foregoing method.
Embodiments of the present application also provide a computer program product comprising computer program instructions.
Optionally, the computer program product may be applied to the log analysis device in the embodiment of the present application, and the computer program instruction enables the computer to execute the corresponding process implemented by the log analysis device in each method in the embodiment of the present application, which is not described herein again for brevity.
The embodiment of the application also provides a computer program.
Optionally, the computer program may be applied to the log analysis device in the embodiment of the present application, and when the computer program runs on a computer, the computer is enabled to execute a corresponding process implemented by the log analysis device in each method in the embodiment of the present application, and details are not described herein for brevity.
It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The expressions "having", "may have", "include" and "contain", or "may include" and "may contain" in this application may be used to indicate the presence of corresponding features (e.g. elements such as values, functions, operations or components) but does not exclude the presence of additional features.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another, and are not necessarily used to describe a particular order or sequence. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention.
The technical solutions described in the embodiments of the present application can be arbitrarily combined without conflict.
In the several embodiments provided in the present application, it should be understood that the disclosed method, apparatus, and device may be implemented in other ways. The above-described embodiments are merely illustrative, and for example, the division of a unit is only one logical function division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application.
Claims (10)
1. A method of log analysis, the method comprising:
acquiring a first log file;
acquiring running information of a first event from the first log file according to a keyword of the first event;
drawing a graph identifier of the first event in a log analysis graph according to the running information of the first event, wherein the graph identifier of the first event is used for representing the running information of the first event;
and displaying the drawn log analysis graph on a log analysis interface.
2. The method of claim 1, wherein the log analysis graph comprises a first dimension;
the drawing a graph identifier of the first event in a log analysis graph according to the running information of the first event comprises the following steps:
determining a first drawing parameter of the first event in the first dimension according to the running information of the first event;
and drawing the graphic identification of the first event in the log analysis graph according to the first drawing parameter.
3. The method of claim 2, wherein the log analysis graph further comprises a second dimension;
the method further comprises the following steps:
determining the event type of the first event according to preset event classification information;
determining a second drawing parameter of the first event on the second dimension according to the event type of the first event;
and drawing the graphic identifier of the first event in the log analysis graph according to the first drawing parameter and the second drawing parameter.
4. The method according to any of claims 1-3, wherein the operational information comprises at least one of: the occurrence time of the event, the occurrence frequency of the event and the abnormal condition of the event.
5. The method of claim 1, wherein the operational information includes a time of occurrence of an event;
the acquiring the running information of the first event from the first log file according to the keyword of the first event comprises the following steps:
acquiring a printing time stamp of a first event text from the first log file according to the keyword of the first event;
and obtaining the occurrence time of the first event according to the printing time stamp of the first event text and the reference time stamp of the first log file.
6. The method of claim 1, wherein the obtaining the first log file comprises:
acquiring an original log file;
acquiring at least one section of interesting text from the original log file according to preset selection configuration information, and generating a second log file;
and acquiring a first event text containing the keyword of the first event from the second log file according to the keyword of the first event, and generating the first log file.
7. The method of claim 6, further comprising:
determining the event type of the first event according to preset event classification information;
and saving the first event text to a position corresponding to the event type of the first event in the first log file.
8. An apparatus for log analysis, the apparatus comprising:
a first acquisition unit configured to acquire a first log file;
a second obtaining unit, configured to obtain, according to a keyword of a first event, operation information of the first event from the first log file;
the processing unit is used for drawing a graph identifier of the first event in a log analysis graph according to the running information of the first event, wherein the graph identifier of the first event is used for representing the running information of the first event;
and the display unit is used for displaying the drawn log analysis chart on the log analysis interface.
9. A log analysis device, characterized in that the device comprises: a processor and a memory configured to store a computer program capable of running on the processor,
wherein the processor is configured to perform the steps of the method of any one of claims 1 to 7 when running the computer program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210481364.0A CN114840672A (en) | 2022-05-05 | 2022-05-05 | Log analysis method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210481364.0A CN114840672A (en) | 2022-05-05 | 2022-05-05 | Log analysis method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114840672A true CN114840672A (en) | 2022-08-02 |
Family
ID=82568586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210481364.0A Pending CN114840672A (en) | 2022-05-05 | 2022-05-05 | Log analysis method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114840672A (en) |
-
2022
- 2022-05-05 CN CN202210481364.0A patent/CN114840672A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP4060942A1 (en) | Configuration anomaly detection method, server and storage medium | |
| US9904517B2 (en) | System and method for automatic modeling of an application | |
| CN104516891A (en) | Layout analyzing method and system | |
| CN110334138B (en) | MATLAB-based data consistency analysis method | |
| CN110471945B (en) | Active data processing method, system, computer equipment and storage medium | |
| CN112632960A (en) | Log analysis method and system based on dynamic field template | |
| CN108768790A (en) | Distributed search cluster monitoring method and device, computing device, storage medium | |
| CN110209643A (en) | A kind of data processing method and device | |
| CN103258021B (en) | The character terminal characteristic extracting method that a kind of Behavior-based control is analyzed | |
| CN108038125B (en) | Method, device, equipment and storage medium for automatically comparing fund system test values | |
| CN115357689B (en) | Data processing method, device and medium of distributed log and computer equipment | |
| CN114840672A (en) | Log analysis method, device, equipment and storage medium | |
| CN109508244B (en) | Data processing method and computer readable medium | |
| CN111459796A (en) | Automatic testing method and device, computer equipment and storage medium | |
| CN111177311A (en) | Data analysis model and analysis method of event processing result | |
| CN115098679A (en) | Method, device, equipment and medium for detecting abnormality of text classification labeling sample | |
| CN114020717A (en) | Method, device, equipment and medium for acquiring performance data of distributed storage system | |
| CN107480038B (en) | Performance analysis method of real-time operating system | |
| CN116414610B (en) | Method, device, equipment and storage medium for acquiring abnormal log fragments | |
| WO2017081866A1 (en) | Log analysis system, method, and program | |
| CN109325166B (en) | Method and device for configuring analysis rules in crawler system | |
| CN111654410B (en) | Gateway request monitoring method, device, equipment and medium | |
| CN116798053B (en) | Icon generation method and device | |
| CN110298935B (en) | Method for acquiring user operation habit information, diagnosis equipment and server | |
| CN117472862A (en) | Log analysis method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |