CN115510005A - Display method and device of abnormal log information, electronic equipment and storage medium - Google Patents

Display method and device of abnormal log information, electronic equipment and storage medium Download PDF

Info

Publication number
CN115510005A
CN115510005A CN202211215597.2A CN202211215597A CN115510005A CN 115510005 A CN115510005 A CN 115510005A CN 202211215597 A CN202211215597 A CN 202211215597A CN 115510005 A CN115510005 A CN 115510005A
Authority
CN
China
Prior art keywords
log
abnormal
data
information
log information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211215597.2A
Other languages
Chinese (zh)
Inventor
付迎鑫
徐蕾
王健
徐锐
刘桥
姬照中
徐冬冬
槐正
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Telecom Corp Ltd
Original Assignee
China Telecom Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Telecom Corp Ltd filed Critical China Telecom Corp Ltd
Priority to CN202211215597.2A priority Critical patent/CN115510005A/en
Publication of CN115510005A publication Critical patent/CN115510005A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/1734Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/17Details of further file system functions
    • G06F16/172Caching, prefetching or hoarding of files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/18File system types
    • G06F16/1805Append-only file systems, e.g. using logs or journals to store data
    • G06F16/1815Journaling file systems

Abstract

The embodiment of the invention provides a method and a device for displaying abnormal log information, electronic equipment and a storage medium, wherein the method comprises the following steps: the method comprises the steps of obtaining service flow codes corresponding to all service systems and log files corresponding to the service flow codes, extracting text contents corresponding to preset log content options from the log files, taking the text contents as valid log data, obtaining log link tracking matrixes corresponding to the valid log data, positioning abnormal log information of the valid log data by using the log link tracking matrixes, classifying the abnormal log information by using the service flow codes, obtaining at least one abnormal log information group corresponding to the abnormal log information, and displaying the abnormal log information group, so that the tracking accuracy is guaranteed, the redundant storage of invalid log data is reduced, the positioning efficiency of the abnormal log information is improved, the abnormal log information group is displayed in a group unit, and a user can obtain information intuitively.

Description

Display method and device of abnormal log information, electronic equipment and storage medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method and an apparatus for displaying abnormal log information, an electronic device, and a computer-readable storage medium.
Background
The log files are file sets used for recording system operation events or application program operation, can be divided into event logs and message logs, and have the functions of processing historical data, tracking problems and understanding system activities.
However, as the number of applications accessing the log collection system and the number of users logging in the applications increase, a large number of invalid log files are easily stored in the log collection system, on one hand, the log collection system is prone to malfunction or failure due to too large system resource pressure, and on the other hand, when a system operation and maintenance worker and a system developer collect log data and trace problems, the amount of data of the log files needing to be browsed or processed is too large, which results in low efficiency of the trace problems.
Disclosure of Invention
The embodiment of the invention provides a method and a device for displaying abnormal log information, electronic equipment and a computer readable storage medium, which are used for solving or partially solving the problem of low tracking efficiency caused by excessive log files needing to be browsed or processed when the system resource pressure of a log collecting system is high and related technical personnel collect log data and track the problem.
The embodiment of the invention discloses a method for displaying abnormal log information, which comprises the following steps:
acquiring a service flow code corresponding to each service system and a log file corresponding to the service flow code, wherein the service flow code is an identifier for identifying the service system;
extracting text content corresponding to a preset log content option from the log file, and taking the text content as effective log data, wherein the preset log content option is an option generated according to attribute information of the log file;
acquiring a log link tracking matrix corresponding to the effective log data, and positioning to abnormal log information of the effective log data by adopting the log link tracking matrix;
and classifying the abnormal log information by adopting the service flow codes, obtaining at least one abnormal log information group corresponding to the abnormal log information, and displaying the abnormal log information group.
Optionally, the obtaining the service pipeline code corresponding to each service system includes:
respectively acquiring a father code and a service identifier corresponding to each service system and a terminal identifier of a terminal to which the service identifier belongs;
performing binary conversion processing on the father code, the service identifier and the terminal identifier to obtain binary data;
performing position division on the binary data to obtain position information of the binary data, and performing position deformation on the position information of the binary data to generate deformation displacement data;
and carrying out induction processing on the deformation shift data to obtain a service flow code corresponding to the induction processing.
Optionally, the service system includes a local cache bit, and the extracting a text content corresponding to a preset log content option from the log file, and using the text content as valid log data includes:
extracting log data corresponding to preset log content options from the log file;
according to the recording time sequence of the log data, grouping the log data by adopting the index identification of the service flow code to obtain a log data group corresponding to a service system;
directionally extracting text content corresponding to a key chain from the log data packet, and storing the text content to the local cache bit through global packet capture;
and taking the text content stored to the local cache bit as the effective log data.
Optionally, the log file includes invalid log data, the extracting a text content corresponding to a preset log content option from the log file, and taking the text content as valid log data further includes:
and taking the log data in the log file except the local cache bit as the invalid log data, and removing the invalid log data from the service system.
Optionally, the obtaining a log link tracking matrix corresponding to the valid log data, and locating abnormal log information of the valid log data by using the log link tracking matrix includes:
identifying the service flow code of the effective log data, tracking the service flow code, and generating a log link tracking matrix corresponding to the service flow code, wherein the log link tracking matrix is a matrix for positioning the abnormal log information;
and in response to the query operation aiming at the effective log data, adopting the log link tracking matrix to locate the abnormal log information of the effective log data.
Optionally, the classifying the abnormal log information by using the service pipeline code to obtain at least one abnormal log information group corresponding to the abnormal log information includes:
classifying the abnormal log information belonging to the same service flow code according to preset classification information to obtain at least one abnormal log information group corresponding to the abnormal log information;
the preset classification information at least comprises error frequency, abnormal time and service type aiming at the abnormal log information.
Optionally, the displaying the abnormal log information group includes:
acquiring an abnormal data report corresponding to the abnormal log information;
and displaying the abnormal data report, wherein the abnormal data report comprises the system state of the target business system corresponding to the abnormal log information.
The embodiment of the invention also discloses a device for displaying the abnormal log information, which comprises:
the service flow code acquisition module is used for acquiring service flow codes corresponding to all service systems and log files corresponding to the service flow codes, and the service flow codes are identifiers used for identifying the service systems;
the effective log data extraction module is used for extracting text contents corresponding to preset log content options from the log file, and taking the text contents as effective log data, wherein the preset log content options are options generated according to attribute information of the log file;
the abnormal log information positioning module is used for acquiring a log link tracking matrix corresponding to the effective log data and positioning the abnormal log information of the effective log data by adopting the log link tracking matrix;
and the abnormal log information display module is used for classifying the abnormal log information by adopting the service flow codes, acquiring at least one abnormal log information group corresponding to the abnormal log information and displaying the abnormal log information group.
Optionally, the service pipeline code obtaining module is specifically configured to:
respectively acquiring a father code and a service identifier corresponding to each service system and a terminal identifier of a terminal to which the service identifier belongs;
performing binary conversion processing on the father code, the service identifier and the terminal identifier to obtain binary data;
performing position division on the binary data to obtain position information of the binary data, and performing position deformation on the position information of the binary data to generate deformation displacement data;
and carrying out induction processing on the deformation shift data to obtain a service flow code corresponding to the induction processing.
Optionally, the service system includes a local cache bit, and the valid log data extraction module is specifically configured to:
extracting log data corresponding to preset log content options from the log file;
according to the recording time sequence of the log data, grouping the log data by adopting the index identification of the service flow code to obtain a log data group corresponding to a service system;
directionally extracting text contents corresponding to key chains from the log data packets, and storing the text contents to the local cache bit through global packet capturing;
and taking the text content stored to the local cache bit as the effective log data.
Optionally, the log file includes invalid log data, and the apparatus further includes:
and the invalid log data removing module is used for taking the log data in the log file except the local cache bit as the invalid log data and removing the invalid log data from the service system.
Optionally, the abnormal log information positioning module is specifically configured to:
identifying the service flow code of the effective log data, tracking the service flow code, and generating a log link tracking matrix corresponding to the service flow code, wherein the log link tracking matrix is a matrix for positioning the abnormal log information;
and in response to the query operation aiming at the effective log data, adopting the log link tracking matrix to locate the abnormal log information of the effective log data.
Optionally, the abnormal log information display module is specifically configured to:
classifying the abnormal log information belonging to the same service flow code according to preset classification information to obtain at least one abnormal log information group corresponding to the abnormal log information;
the preset classification information at least comprises error frequency, abnormal time and service type aiming at the abnormal log information.
Optionally, the abnormal log information display module is specifically configured to:
acquiring an abnormal data report corresponding to the abnormal log information;
and displaying the abnormal data report, wherein the abnormal data report comprises the system state of the target business system corresponding to the abnormal log information.
The embodiment of the invention also discloses electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory finish mutual communication through the communication bus;
the memory is used for storing a computer program;
the processor is configured to implement the method according to the embodiment of the present invention when executing the program stored in the memory.
Also disclosed is a computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform a method according to an embodiment of the invention.
The embodiment of the invention has the following advantages:
in the embodiment of the invention, the service flow codes corresponding to all service systems and the log files corresponding to the service flow codes are obtained, the text content corresponding to the preset log content option is extracted from the log files, the text content is used as effective log data, the log link tracking matrix corresponding to the effective log data is obtained, the log link tracking matrix is adopted to locate the abnormal log information of the effective log data, the abnormal log information is classified by the service flow codes, at least one abnormal log information group corresponding to the abnormal log information is obtained and displayed, the accuracy of tracking the abnormal service systems is ensured by obtaining the service flow codes aiming at all the service systems, the effective log data required by users can be extracted by adopting the preset log content option, the redundant storage of the ineffective log data is reduced, the processed log data amount is reduced, meanwhile, the abnormal log information is quickly located based on the log link tracking matrix, the locating efficiency of the abnormal log information is improved, the logs are grouped, and the abnormal log information groups are displayed in a group unit, which is beneficial to visually obtaining information by the users.
Drawings
FIG. 1 is a schematic diagram of an overall framework of a log management system provided in an embodiment of the invention;
fig. 2 is a flowchart illustrating steps of a method for displaying exception log information according to an embodiment of the present invention;
FIG. 3 is a flow chart of the steps provided in an embodiment of the present invention to generate a traffic pipeline code;
FIG. 4 is a flowchart of the steps provided in an embodiment of the present invention to clean up invalid log data;
FIG. 5 is a schematic diagram of data transmission between an abnormal data report, a log tracking system, and a display platform according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of locating anomalous log information via a log management system as provided in an embodiment of the present invention;
fig. 7 is a block diagram of a structure of an apparatus for presenting exception log information provided in the embodiment of the present invention;
fig. 8 is a block diagram of an electronic device provided in the embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention more comprehensible, the present invention is described in detail with reference to the accompanying drawings and the detailed description thereof.
With the update iteration of various open source projects and frames, the technology of the log collection system is continuously upgraded and perfected, and system operation and maintenance personnel and system developers can quickly locate the position with abnormal conditions through the log collection system, so that the high availability of the system is ensured.
In view of the above, one of the core invention points of the embodiments of the present invention is to obtain a service flow code corresponding to each service system and a log file corresponding to the service flow code, extract text content corresponding to a preset log content option from the log file, use the text content as valid log data, obtain a log link tracking matrix corresponding to the valid log data, locate abnormal log information to the valid log data by using the log link tracking matrix, classify the abnormal log information by using the service flow code, obtain at least one abnormal log information group corresponding to the abnormal log information, and display the abnormal log information group.
Referring to fig. 1, an overall framework schematic diagram of a log management system provided in an embodiment of the present invention is shown, where the log management system includes a service system, a running water code generation system, a log collection system, a log screening system, and a log tracking matrix system, where the running water code system may be configured to generate a unified running water code for different service systems according to a preset rule, so as to ensure uniqueness and queryable effect of the running water code, and the running water code system at least includes a rule center for storing the rule, a running water generation module for generating the running water code, and a running water management module for managing the running water codes of different services. The service system can be used for acquiring service flow codes generated by the flow code system and transmitting log information to the log collection system, and the log collection system can be used for collecting logs, participating in log screening system to process the logs, primarily performing compliance processing on the logs and removing the logs not carrying the service flow codes. The log screening system can be used for screening logs sent by the log collection system according to screening options set by related technicians to obtain invalid log contents and valid log contents, can eliminate the invalid log contents, stores the valid log contents, can extract problem flow unified rule logs through preset rules, and can perform early warning on abnormal logs through early warning rule contents set in advance. The log tracking matrix system can generate a tracking matrix according to a uniform flow code sent by the log collection system, and sends the tracking matrix to the log collection system, so that abnormal log information is classified and displayed through the uniform flow code, a series of contents are generated to generate a log chain tracking matrix from a data source to a problem, an abnormal data report is generated according to the contents of the log chain tracking matrix and displayed by a display platform, and therefore the log management system obtains a unique tracking stream through processes of flow code generation, data exchange, matrix generation and the like with other systems, all problem logs are connected in series through streams, and a cross-domain log group is used for finding a source of problem generation and solving the problem corresponding to a service system through flow tracking.
Referring to fig. 2, a flowchart illustrating steps of a method for displaying exception log information provided in the embodiment of the present invention is shown, which may specifically include the following steps:
step 201, acquiring a service flow code corresponding to each service system and a log file corresponding to the service flow code, where the service flow code is an identifier for identifying the service system;
in the embodiment of the present invention, the service system may be a system providing various functions to a user, for example, the service system may be an order system, a logistics system, a user system, and other different types of systems, the pipeline code may be a code commonly used in a production line, and machines of a general operating system may have the pipeline code themselves for distinguishing and defining product output quantity, identifying and checking products. The log file may be a recording file or a file set for recording system operation events, and is divided into an event log and a message log, which have the functions of processing historical data, tracing diagnosis problems, understanding system activities, and the like.
In the specific implementation, the pipeline code system may respectively obtain basic parameters such as a parent code, a service identifier, and a terminal identifier of a terminal belonging to the parent code, the service identifier, and the terminal identifier corresponding to each service system, perform binary conversion on the parent code, the service identifier, and the terminal identifier to obtain binary data, perform position division on the binary data to obtain position information of the binary data, perform position deformation on the position information of the binary data to generate deformation shift data, perform induction processing on the deformation shift data to obtain a service pipeline code corresponding to the induction processing.
In an example of the present invention, the service identifier may be a service code ID of a service system, the terminal identifier may be a machine ID, and fig. 3 shows a flowchart of steps for generating a service flow code according to an embodiment of the present invention, after the flow code system obtains a parent code, the service code ID, and the machine ID, and the like, the flow code system may call a unified flow code production rule set in a rule center according to a user requirement, and then perform induction processing on the parent code, the service code ID, and the machine ID, and the like by using the unified flow code production rule, so as to obtain induction information, package the induction information, perform uniqueness processing on the induction information by using a snowflake algorithm, and generate the service flow code for the service system.
Specifically, the deformation shift data is mainly realized by a snowflake algorithm, original data (a father code, a service code ID and a machine ID) are firstly input into the snowflake algorithm, then binary processing is carried out on the original data, the data type of the original data is converted into binary data, then the positions are distinguished through the binary data for positioning and dividing, the position content is divided through the whole binary data, the position information is specified, the preparation content is prepared for subsequent position deformation, and the position deformation of the position information of the binary data comprises the following substeps:
s21, beginning to deform I, shifting the timestamp left, and vacating related data bits;
s22, starting to displace the second time, and calculating a central value of the data machine;
s23, starting displacement III and calculating a generator ID value;
s24, deforming to four, calculating to generate a self-increment numerical value ID;
s25, collecting the deformation shift data values;
and S26, verifying the numerical value, and generating and returning the ID numerical value.
And finally, summarizing and summarizing the ID numerical value to obtain the service flow code, and returning the service flow code to the service system, thereby generating the service flow code aiming at the service system.
Step 202, extracting text content corresponding to a preset log content option from the log file, and taking the text content as effective log data, wherein the preset log content option is an option generated according to attribute information of the log file;
in this embodiment of the present invention, the preset log content option may be an option generated according to attribute information of the log file, for example, specific content of the log file, record or storage time of the log file, and the preset log content option may include a problem keyword, an assigned date, an assigned error level, and other options, and the present invention is not limited to this, and the valid log data may be a log matched with the preset log content option, for example, if the error level includes an error level i, an error level ii, and an error level iii, and the preset log content option specifies that the error level is an error level ii, the text content corresponding to the error level ii is used as valid log data, and if the preset log content option is a keyword: if the order is wrong, the text content corresponding to the order is wrong is taken as effective log data, and if the preset log content option is the specified date: and in 9 months 2022, the text content recorded by the business system in 9 months 2022 is taken as valid log data. In an embodiment of the present invention, the service system includes a local cache bit, the log screening system may extract log data corresponding to a preset log content option from a log file, then group the log data by using an index identifier of a service pipeline code according to a recording timing sequence of the log data, obtain a plurality of log data groups corresponding to the service system, directionally extract text content corresponding to a key chain from the plurality of log data groups by using the index identifier, store the text content to the local cache bit by performing a global packet capturing operation on the file content, and use the text content stored to the local cache bit as valid log data.
The local cache bit may be a position for storing the grouping after the log data is grouped, a Critical Chain (Critical Chain) is a project management method based on a constraint theory (Theoryof Constraints), the log data stored in the local cache bit is used as valid log data, and the log data except the local cache bit is used as invalid log data.
In another embodiment of the invention, the log file comprises invalid log data, the log data in the log file except the local cache bit in the time period is taken as invalid log data, the invalid log data is removed from the service system, and the invalid and redundant data is cleaned by removing the invalid log data, so that the system pressure of the log management system is greatly reduced, and the high utilization rate of space is ensured.
As an example, referring to fig. 4, which shows a flowchart of a step of clearing invalid log data according to an embodiment of the present invention, a log collection system processes log data according to a time dimension, extracts a set rule condition, such as a keyword and a time period, performs group-locating operation on a pipeline code corresponding to the log data, performs group-locating operation, then directionally extracts valid log data corresponding to a key chain from each log group, stores the valid log data in a local cache bit in a global packet capturing manner, and performs data clearing operation on invalid log data outside the local cache bit in the time period, so as to clear invalid and redundant log data, reduce resource pressure of a log management system, and improve processing efficiency.
Step 203, acquiring a log link tracking matrix corresponding to the effective log data, and positioning to abnormal log information of the effective log data by using the log link tracking matrix;
in the embodiment of the invention, the log link tracking matrix is used for tracking abnormal log information, and the log link tracking matrix corresponding to the effective log data is adopted, so that the abnormal log information of the effective log data can be quickly positioned, and the processing efficiency of the log management system is improved.
Optionally, the log tracking matrix system may obtain a service flow code corresponding to valid log data, identify and judge the service flow code, track a data stream link of the valid log data, generate a log link tracking matrix based on the data stream link, where the log link tracking matrix has a corresponding data processing rule, so that a relevant technician can obtain a complete data stream for the valid log data based on the log link tracking matrix, and when the relevant technician needs to perform problem tracking on a log with a problem, the log tracking matrix can be quickly located to a problem source.
In a specific implementation, after the log tracking matrix system generates the log link tracking matrix for the valid log data, the log management system may locate abnormal log information from the valid log data using the log link tracking matrix in response to a query operation of a user on the valid log data.
And 204, classifying the abnormal log information by adopting the service flow codes, acquiring at least one abnormal log information group corresponding to the abnormal log information, and displaying the abnormal log information group.
In the embodiment of the invention, the log management system can classify the abnormal log information by adopting the service flow codes, obtain at least one abnormal log information group corresponding to the abnormal log information and display the abnormal log information group, thereby grouping the abnormal log information and realizing the integral and dynamic statistical analysis of the log with problems.
Optionally, the abnormal log information belonging to the same service pipeline code is classified according to preset classification information to obtain at least one abnormal log information group corresponding to the abnormal log information, where the preset classification information may be classification information set by a relevant technician according to actual requirements, and the classification information at least includes an error frequency, an abnormal time, and a service type for the abnormal log information.
Specifically, if the preset classification information is an error frequency, the abnormal log information may be classified according to the error frequency and the service flow code, and an abnormal log information group with the "error frequency" as a classification standard is obtained, for example, the abnormal log information group is obtained as 5 times of the error frequency, 10 times of the error frequency, and 20 times of the error frequency, and an abnormal log information group (1) corresponding to the 5 times of the error frequency, an abnormal log information group (2) corresponding to the 10 times of the error frequency, and an abnormal log information group (3) corresponding to the 20 times of the error frequency are displayed; if the preset classification information is a service type, classifying the abnormal log information according to the service type and the service flow code, and taking the service type as an abnormal log information group of a classification standard, for example, obtaining an abnormal log information group (4) corresponding to an order service and an abnormal log information group (5) corresponding to an order rejection service, and displaying the abnormal log information group (4) and the abnormal log information group (5); if the preset classification information is abnormal time, the abnormal log information can be classified according to the abnormal time and the service flow codes, and an abnormal log information group taking an abnormal time point or an abnormal time period as a classification standard, for example, an abnormal log information group (6) with the abnormal time within 2 days is displayed.
As an example, referring to fig. 5, a schematic diagram of data transmission among an abnormal data report, a log tracking system, and a presentation platform provided in an embodiment of the present invention is shown, where the log tracking system generates a log chain tracking matrix based on a business pipeline code, and forms a data processing logic layer and a report presentation layer by surrounding a unique business pipeline code, and mainly includes the following sub-steps:
s41, identifying and judging the flow codes, and tracking the flow codes to form a complete log link tracking matrix so as to quickly position an abnormal source;
s42, forming a self-contained data processing rule according to the matrix content by the data synchronization logic layer, and synchronously processing and displaying data processing through a screening rule device;
s43, completing the data processing processes of the two layers, customizing responsibility items on a report display layer after data combing, and screening report information meeting the requirements of users according to the responsibility items.
The responsibility items refer to query contents input by related technicians in the log management system according to actual requirements, and classification information can be obtained according to the responsibility items selected by the user, so that abnormal data reports are displayed based on classification confidence, and the related technicians can further know the system state of the business system conveniently.
In one example of the invention, an abnormal data report corresponding to abnormal log information is obtained, and the abnormal data report is displayed, wherein the abnormal data report comprises a system state of a target service system corresponding to the abnormal log information, and the system state can comprise a normal working state and an abnormal working state of the service system, and the abnormal log information and a data stream corresponding to the abnormal log information are visually displayed by adopting the abnormal data report, so that the effects of improving the positioning accuracy of system problems, determining the cause of the problems across systems, breaking a communication problem barrier, reducing the occupied time of a system log disk and the like are achieved.
As an example, referring to fig. 6, a schematic diagram of positioning abnormal log information through a log management system according to an embodiment of the present invention is shown, if an operator finds that a service system has a fault or is abnormal, an error may be reported to a developer or the developer finds a problem by himself, the operation and maintenance staff may maintain the entire log management system to ensure that the systems can work normally, before the developer starts a log collection system, a unified stream code system may generate a unique stream code for each service system and send the stream code to a corresponding service system, the developer may set a rule and an early warning manner in advance in a log watching summary platform, when a problem log in the service system needs to be searched, the log collection system may collect log information and a service stream code of the service system, and then send the log information and the service stream code to the log watching summary platform, after the log information and the service stream code are received by the log watching summary platform, the log information may be processed, and the developer may trigger generation of a log tracking matrix based on the log chain, thereby quickly positioning to the abnormal log information, and obtain report data showing the report.
It should be noted that, the embodiments of the present invention include, but are not limited to, the above examples, and it is understood that, under the guidance of the idea of the embodiments of the present invention, a person skilled in the art may set the idea according to practical situations, and the present invention is not limited to this.
In the embodiment of the invention, the service flow codes corresponding to all service systems and the log files corresponding to the service flow codes are obtained, the text content corresponding to the preset log content option is extracted from the log files, the text content is used as effective log data, the log link tracking matrix corresponding to the effective log data is obtained, the log link tracking matrix is adopted to locate the abnormal log information of the effective log data, the abnormal log information is classified by the service flow codes, at least one abnormal log information group corresponding to the abnormal log information is obtained and displayed, the accuracy of tracking the abnormal service systems is ensured by obtaining the service flow codes aiming at all the service systems, the effective log data required by users can be extracted by adopting the preset log content option, the redundant storage of the ineffective log data is reduced, the processed log data amount is reduced, meanwhile, the abnormal log information is quickly located based on the log link tracking matrix, the locating efficiency of the abnormal log information is improved, the logs are grouped, and the abnormal log information groups are displayed in a group unit, which is beneficial to visually obtaining information by the users.
It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Referring to fig. 7, a block diagram of a structure of a display apparatus for exception log information provided in the embodiment of the present invention is shown, which specifically includes the following modules:
a service flow code obtaining module 701, configured to obtain a service flow code corresponding to each service system and a log file corresponding to the service flow code, where the service flow code is an identifier for identifying the service system;
an effective log data extraction module 702, configured to extract text content corresponding to a preset log content option from the log file, and use the text content as effective log data, where the preset log content option is an option generated according to attribute information of the log file;
an abnormal log information positioning module 703, configured to obtain a log link tracking matrix corresponding to the valid log data, and position the abnormal log information of the valid log data by using the log link tracking matrix;
an abnormal log information display module 704, configured to classify the abnormal log information by using the service pipeline code, obtain at least one abnormal log information group corresponding to the abnormal log information, and display the abnormal log information group.
In an optional embodiment, the service pipeline code obtaining module 701 is specifically configured to:
respectively acquiring a father code and a service identifier corresponding to each service system and a terminal identifier of a terminal to which the service identifier belongs;
performing binary conversion processing on the father code, the service identifier and the terminal identifier to obtain binary data;
performing position division on the binary data to obtain position information of the binary data, and performing position deformation on the position information of the binary data to generate deformation displacement data;
and carrying out induction processing on the deformation shift data to obtain a service flow code corresponding to the induction processing.
In an optional embodiment, the service system includes a local cache bit, and the valid log data extraction module 702 is specifically configured to:
extracting log data corresponding to preset log content options from the log file;
according to the recording time sequence of the log data, grouping the log data by adopting the index identification of the service flow code to obtain a log data group corresponding to a service system;
directionally extracting text contents corresponding to key chains from the log data packets, and storing the text contents to the local cache bit through global packet capturing;
and taking the text content stored to the local cache bit as the effective log data.
In an alternative embodiment, the log file includes invalid log data, the apparatus further comprising:
and the invalid log data removing module is used for taking the log data in the log file except the local cache bit as the invalid log data and removing the invalid log data from the service system.
In an optional embodiment, the abnormal log information positioning module 703 is specifically configured to:
identifying the service flow code of the effective log data, tracking the service flow code, and generating a log link tracking matrix corresponding to the service flow code, wherein the log link tracking matrix is a matrix for positioning the abnormal log information;
and in response to the query operation aiming at the effective log data, adopting the log link tracking matrix to position the abnormal log information of the effective log data.
In an optional embodiment, the exception log information presenting module 704 is specifically configured to:
classifying the abnormal log information belonging to the same service flow code according to preset classification information to obtain at least one abnormal log information group corresponding to the abnormal log information;
the preset classification information at least comprises error frequency, abnormal time and service type aiming at the abnormal log information.
In an optional embodiment, the exception log information presenting module 704 is specifically configured to:
acquiring an abnormal data report corresponding to the abnormal log information;
and displaying the abnormal data report, wherein the abnormal data report comprises the system state of the target business system corresponding to the abnormal log information.
For the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for relevant points.
In addition, an embodiment of the present invention further provides an electronic device, including: the processor, the memory, and the computer program stored in and executable on the memory, where the computer program is executed by the processor to implement the processes of the above-mentioned method for displaying abnormal log information, and can achieve the same technical effects, and are not described herein again to avoid repetition.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements each process of the above-mentioned method for displaying abnormal log information, and can achieve the same technical effect, and in order to avoid repetition, the details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
Fig. 8 is a schematic structural diagram of an electronic device implementing various embodiments of the present invention.
The electronic device 800 includes, but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, a processor 810, and a power supply 811. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 8 does not constitute a limitation of the electronic device, and that the electronic device may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present invention, the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer, and the like.
It should be understood that, in the embodiment of the present invention, the radio frequency unit 801 may be used for receiving and sending signals during a message sending and receiving process or a call process, and specifically, receives downlink data from a base station and then processes the received downlink data to the processor 810; in addition, the uplink data is transmitted to the base station. In general, radio frequency unit 801 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. Further, the radio frequency unit 801 can also communicate with a network and other devices through a wireless communication system.
The electronic device provides wireless broadband internet access to the user via the network module 802, such as to assist the user in sending and receiving e-mails, browsing web pages, and accessing streaming media.
The audio output unit 803 may convert audio data received by the radio frequency unit 801 or the network module 802 or stored in the memory 809 into an audio signal and output as sound. Also, the audio output unit 803 may also provide audio output related to a specific function performed by the electronic apparatus 800 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 803 includes a speaker, a buzzer, a receiver, and the like.
The input unit 804 is used for receiving an audio or video signal. The input Unit 804 may include a Graphics Processing Unit (GPU) 8041 and a microphone 8042, and the Graphics processor 8041 processes image data of still pictures or video obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 806. The image frames processed by the graphics processor 8041 may be stored in the memory 809 (or other storage medium) or transmitted via the radio frequency unit 801 or the network module 802. The microphone 8042 can receive sound, and can process such sound into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 801 in case of a phone call mode.
The electronic device 800 also includes at least one sensor 805, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor includes an ambient light sensor that can adjust the brightness of the display panel 8061 according to the brightness of ambient light and a proximity sensor that can turn off the display panel 8061 and/or the backlight when the electronic device 800 is moved to the ear. As one type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of an electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), and vibration identification related functions (such as pedometer, tapping); the sensors 805 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.
The display unit 806 is used to display information input by the user or information provided to the user. The Display unit 806 may include a Display panel 8061, and the Display panel 8061 may be configured by a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 807 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic apparatus. Specifically, the user input unit 807 includes a touch panel 8071 and other input devices 8072. The touch panel 8071, also referred to as a touch screen, may collect touch operations by a user on or near the touch panel 8071 (e.g., operations by a user on or near the touch panel 8071 using a finger, a stylus, or any other suitable object or accessory). The touch panel 8071 may include two portions of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 810, receives a command from the processor 810, and executes the command. In addition, the touch panel 8071 can be implemented by various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 8071, the user input unit 807 can include other input devices 8072. In particular, other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein.
Further, the touch panel 8071 can be overlaid on the display panel 8061, and when the touch panel 8081 detects a touch operation on or near the touch panel 8081, the touch operation can be transmitted to the processor 810 to determine the type of the touch event, and then the processor 810 can provide a corresponding visual output on the display panel 8061 according to the type of the touch event. Although in fig. 8, the touch panel 8071 and the display panel 8061 are shown as two separate components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 8071 and the display panel 8061 may be integrated to implement the input and output functions of the electronic device, which is not limited herein.
The interface unit 808 is an interface for connecting an external device to the electronic apparatus 800. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 808 may be used to receive input (e.g., data information, power, etc.) from external devices and transmit the received input to one or more elements within the electronic device 800 or may be used to transmit data between the electronic device 800 and external devices.
The memory 809 may be used to store software programs as well as various data. The memory 809 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 809 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
The processor 810 is a control center of the electronic device, connects various parts of the whole electronic device by using various interfaces and lines, performs various functions of the electronic device and processes data by running or executing software programs and/or modules stored in the memory 809 and calling data stored in the memory 809, thereby monitoring the whole electronic device. Processor 810 may include one or more processing units; preferably, the processor 810 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 810.
The electronic device 800 may also include a power supply 811 (e.g., a battery) for powering the various components, and preferably, the power supply 811 may be logically coupled to the processor 810 via a power management system to manage charging, discharging, and power consumption management functions via the power management system.
In addition, the electronic device 800 includes some functional modules that are not shown, and are not described in detail herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention or portions thereof contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the methods according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one type of logical functional division, and other divisions may be realized in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a U disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. A method for displaying abnormal log information is characterized by comprising the following steps:
acquiring a service flow code corresponding to each service system and a log file corresponding to the service flow code, wherein the service flow code is an identifier for identifying the service system;
extracting text content corresponding to a preset log content option from the log file, and taking the text content as effective log data, wherein the preset log content option is an option generated according to attribute information of the log file;
acquiring a log link tracking matrix corresponding to the effective log data, and positioning to abnormal log information of the effective log data by adopting the log link tracking matrix;
and classifying the abnormal log information by adopting the service flow codes, obtaining at least one abnormal log information group corresponding to the abnormal log information, and displaying the abnormal log information group.
2. The method of claim 1, wherein the obtaining the service pipeline code corresponding to each service system comprises:
respectively acquiring a father code and a service identifier corresponding to each service system and a terminal identifier of a terminal to which the service identifier belongs;
performing binary conversion processing on the father code, the service identifier and the terminal identifier to obtain binary data;
performing position division on the binary data to obtain position information of the binary data, and performing position deformation on the position information of the binary data to generate deformation displacement data;
and carrying out induction processing on the deformation shift data to obtain a service flow code corresponding to the induction processing.
3. The method of claim 1, wherein the service system comprises a local cache bit, wherein the service pipeline code comprises an index identifier, wherein extracting the text content corresponding to a preset log content option from the log file and using the text content as valid log data comprises:
extracting log data corresponding to preset log content options from the log file;
according to the recording time sequence of the log data, grouping the log data by adopting the index identification of the service flow code to obtain a log data group corresponding to a service system;
directionally extracting text content corresponding to a key chain from the log data packet, and storing the text content to the local cache bit through global packet capture;
and taking the text content stored to the local cache bit as the effective log data.
4. The method according to claim 3, wherein the log file includes invalid log data, the extracting text content corresponding to a preset log content option from the log file, and regarding the text content as valid log data, further comprising:
and taking the log data in the log file except the local cache bit as the invalid log data, and removing the invalid log data from the service system.
5. The method of claim 2, wherein the obtaining of the log link tracking matrix corresponding to the valid log data and using the log link tracking matrix to locate the abnormal log information of the valid log data comprises:
identifying the service flow code of the effective log data, tracking the service flow code, and generating a log link tracking matrix corresponding to the service flow code, wherein the log link tracking matrix is a matrix for positioning the abnormal log information;
and in response to the query operation aiming at the effective log data, adopting the log link tracking matrix to locate the abnormal log information of the effective log data.
6. The method of claim 1, wherein the classifying the abnormal log information using the traffic pipeline code to obtain at least one abnormal log information group corresponding to the abnormal log information comprises:
classifying the abnormal log information belonging to the same service flow code according to preset classification information to obtain at least one abnormal log information group corresponding to the abnormal log information;
the preset classification information at least comprises error frequency, abnormal time and service type aiming at the abnormal log information.
7. The method of claim 1, wherein said exposing the exception log information group comprises:
acquiring an abnormal data report corresponding to the abnormal log information;
and displaying the abnormal data report, wherein the abnormal data report comprises the system state of the target business system corresponding to the abnormal log information.
8. An apparatus for displaying abnormal log information, the apparatus comprising:
the service flow code acquisition module is used for acquiring service flow codes corresponding to all service systems and log files corresponding to the service flow codes, and the service flow codes are identifiers used for identifying the service systems;
the effective log data extraction module is used for extracting text contents corresponding to preset log content options from the log file and taking the text contents as effective log data, wherein the preset log content options are options generated according to the attribute information of the log file;
the abnormal log information positioning module is used for acquiring a log link tracking matrix corresponding to the effective log data and positioning the abnormal log information of the effective log data by adopting the log link tracking matrix;
and the abnormal log information display module is used for classifying the abnormal log information by adopting the service flow codes, acquiring at least one abnormal log information group corresponding to the abnormal log information and displaying the abnormal log information group.
9. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
the memory is used for storing a computer program;
the processor, when executing a program stored on the memory, implementing the method of any one of claims 1-7.
10. A computer-readable storage medium having stored thereon instructions, which when executed by one or more processors, cause the processors to perform the method of any one of claims 1-7.
CN202211215597.2A 2022-09-30 2022-09-30 Display method and device of abnormal log information, electronic equipment and storage medium Pending CN115510005A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211215597.2A CN115510005A (en) 2022-09-30 2022-09-30 Display method and device of abnormal log information, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211215597.2A CN115510005A (en) 2022-09-30 2022-09-30 Display method and device of abnormal log information, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115510005A true CN115510005A (en) 2022-12-23

Family

ID=84508824

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211215597.2A Pending CN115510005A (en) 2022-09-30 2022-09-30 Display method and device of abnormal log information, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115510005A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116661758A (en) * 2023-08-01 2023-08-29 青岛蓝图智库信息技术有限公司 Method, device, electronic equipment and medium for optimizing log framework configuration

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116661758A (en) * 2023-08-01 2023-08-29 青岛蓝图智库信息技术有限公司 Method, device, electronic equipment and medium for optimizing log framework configuration
CN116661758B (en) * 2023-08-01 2023-11-03 青岛蓝图智库信息技术有限公司 Method, device, electronic equipment and medium for optimizing log framework configuration

Similar Documents

Publication Publication Date Title
CN111368290A (en) Data anomaly detection method and device and terminal equipment
CN112422711B (en) Resource allocation method and device, electronic equipment and storage medium
CN108734133A (en) Vehicle trouble processing method and mobile terminal
CN108900386B (en) Method and device for generating alarm information and electronic equipment
CN114595124B (en) Time sequence abnormity detection model evaluation method, related device and storage medium
CN115510005A (en) Display method and device of abnormal log information, electronic equipment and storage medium
CN107103086B (en) Data acquisition auditing method and system, and computer readable storage medium
CN111753520B (en) Risk prediction method and device, electronic equipment and storage medium
CN109218082A (en) A kind of network configuration parameters processing method, device and equipment
CN109067979A (en) A kind of reminding method and mobile terminal
CN112256748A (en) Abnormity detection method and device, electronic equipment and storage medium
CN116976898A (en) Data acquisition method, data visualization method, device and related products
CN107832189A (en) A kind of monitoring method and mobile terminal of I/O systems
CN108600356B (en) Message pushing method and device
CN111080305A (en) Risk identification method and device and electronic equipment
CN109508425B (en) Setting item recommendation method and terminal equipment
CN112596980A (en) ios performance collection method and device, mobile terminal and computer readable storage medium
CN112363895A (en) System fault positioning method and device and electronic equipment
CN111818548A (en) Data processing method, device and equipment
CN109976854A (en) A kind of applied program processing method and terminal device
CN116225751A (en) Positioning prediction method and device for IT fault
CN115422311A (en) Travel track data generation method and device, electronic equipment and storage medium
CN109886324B (en) Icon identification method and device
CN116680156A (en) Log monitoring processing method, device, equipment and storage medium
CN107862013B (en) Schedule searching method and mobile terminal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination