CN113742192A - Log rule quality analysis method, system, electronic device and storage medium - Google Patents
Log rule quality analysis method, system, electronic device and storage medium Download PDFInfo
- Publication number
- CN113742192A CN113742192A CN202111069491.1A CN202111069491A CN113742192A CN 113742192 A CN113742192 A CN 113742192A CN 202111069491 A CN202111069491 A CN 202111069491A CN 113742192 A CN113742192 A CN 113742192A
- Authority
- CN
- China
- Prior art keywords
- log
- rule
- analysis
- result
- log analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application relates to a log rule quality analysis method, a system, an electronic device and a storage medium, wherein the log rule quality analysis method comprises the following steps: acquiring a target log analysis rule packet and a log sample; inputting the target log analysis rule packet and the log sample into a log analysis module to output a first log analysis result, and inputting the target log analysis rule packet and the log sample into a test framework to output a second log analysis result; the log analysis module is used for generating a first log analysis result, and the test framework is used for generating a second log analysis result; and obtaining a log analysis rule checking result according to the first log analysis result and the second log analysis result by using a preset checking rule. By the method and the device, the problems of low checking efficiency and high error rate of the analysis rule test result are solved, and automatic quality analysis of the log analysis result is realized.
Description
Technical Field
The present application relates to the field of log parsing rules, and in particular, to a log rule quality analysis method, system, electronic device, and storage medium.
Background
The log rule, i.e. the log analysis rule, is associated with the log field through the created log analysis rule, and when the field in the log meets the filtering condition configured by the analysis rule, the analysis rule is automatically matched to realize data analysis. The log rule is from the initial manual writing of regular expressions to the analysis method based on source codes, such as frequent pattern mining, hierarchical clustering, iterative partitioning and the like, and the automatic analysis of the log is realized. In the automatic analysis, the set analysis rule needs to be tested to judge the accuracy of the analysis.
In the prior art related to analysis rule testing, a log sample is mainly imported to a log audit platform and a result is output, and whether a log analysis result and an analysis field are correct is manually checked item by item according to a preset analysis regular expression and the log sample.
Aiming at the problems of low checking efficiency and high error rate of analysis rule test results in the related technology, no effective solution is provided at present.
Disclosure of Invention
The embodiment provides a log rule quality analysis method, a log rule quality analysis system, an electronic device and a storage medium, so as to solve the problems of low checking efficiency and high error rate in the related art.
In a first aspect, in this embodiment, a log rule quality analysis method is provided, including:
acquiring a target log analysis rule packet and a log sample;
inputting the target log analysis rule packet and the log sample into a log analysis module to output a first log analysis result, and inputting the target log analysis rule packet and the log sample into a test framework to output a second log analysis result; the log analysis module is used for generating a first log analysis result, and the test framework is used for generating a second log analysis result;
and obtaining a log analysis rule checking result according to the first log analysis result and the second log analysis result by using a preset checking rule.
In some of these embodiments, the obtaining the target log parsing rule package comprises:
acquiring an initial log analysis rule packet, acquiring field names defined in the initial log analysis rule packet and field values corresponding to the field names; wherein the field name comprises: the method comprises the following steps of (1) obtaining a rule name, an asset type corresponding to the rule, a regular expression and a prejudgment mode; the pre-judging mode is a pre-matching mode matched with an analysis rule defined in the log analysis rule packet in advance;
and carrying out format conversion on the acquired field name and the field value to obtain a conversion field name and a conversion field value, and obtaining the target log analysis rule packet according to the conversion field name and the conversion field value.
In some of these embodiments, obtaining the log sample comprises:
acquiring the log sample by using a preset log storage file; or, the log sample is obtained by using a network protocol specified by the analysis rule.
In some of these embodiments, the inputting the target log resolution rule package and the log sample to a log resolution module to output a first log resolution result includes:
inputting the target log analysis rule packet and the log sample into the log analysis module;
acquiring a log analysis rule corresponding to a pre-judging mode according to the pre-judging mode in the target log analysis rule packet by using the log analysis module, and analyzing according to a regular expression in the log analysis rule to obtain a corresponding field name and a corresponding field value;
and utilizing the log analysis module to perform first analysis operation on the log sample according to the field name and the field value, and outputting to obtain a first log analysis result.
In some embodiments, the performing, by the log parsing module, a first parsing operation on the log sample according to the field name and the field value, and outputting to obtain the first log parsing result includes:
obtaining an analysis operation check result of the log analysis module for judging whether a first analysis operation is normal or not, obtaining a log analysis module field check result of the log analysis module for judging whether a rule data field in the target log analysis rule packet is missing or not, and obtaining a log analysis module matching check result of the log analysis module for judging whether different log analysis rules are mismatched or not;
and utilizing the log analysis module to perform first analysis operation on the log sample according to the field name and the field sampling value to obtain an initial analysis result, verifying the initial analysis result according to the analysis operation verification result, the log analysis module field verification result and the log analysis module matching verification result, and finally outputting to obtain the first log analysis result.
In some of these embodiments, the inputting the target log resolution rule package and the log sample to a test framework to output a second log resolution result includes:
inputting the target log analysis rule packet and the log sample into the test framework;
acquiring a log analysis rule corresponding to a pre-judging mode according to the pre-judging mode in the target log analysis rule packet by using the test frame, and analyzing according to a regular expression in the log analysis rule to obtain a corresponding field name and a corresponding field value;
and carrying out second analysis operation on the log sample according to the field name and the field value by using the test framework, and outputting to obtain a second log analysis result.
In some embodiments, the obtaining, by using a preset check rule, a log parsing rule check result according to the first log parsing result and the second log parsing result includes:
comparing the field name and the field value of the first log analysis result obtained by analysis with the field name and the field value of the second log analysis result by using the check rule according to each field name and corresponding field value defined in the log analysis rule packet to obtain a comparison result; obtaining a log analysis rule checking result according to the comparison result;
sending the first log analysis result, the second log analysis result and the log analysis rule verification result to terminal equipment; and the terminal equipment is used for rendering an interface aiming at the log analysis rule verification result, generating a visual test report and displaying the visual test report.
In a second aspect, in this embodiment, a log rule quality analysis system is provided, including: a terminal device, a transmission device and a server device; the terminal equipment is connected with the server equipment through the transmission equipment;
the terminal equipment is used for receiving a target log analysis rule packet and a log sample;
the transmission equipment is used for sending the target log analysis rule packet and the log sample to the server equipment;
the server device is configured to execute the log rule quality analysis method according to the first aspect.
In a third aspect, in this embodiment, there is provided an electronic apparatus, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the log rule quality analysis method according to the first aspect.
In a fourth aspect, in the present embodiment, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements the log rule quality analysis method of the first aspect.
Compared with the related art, the log rule quality analysis method, the log rule quality analysis system, the electronic device and the storage medium provided by the embodiment of the invention solve the problems of low checking efficiency and high error rate of the analysis rule test result by comparing the analysis result of the log analysis module with the analysis result of the test frame, and realize automatic quality analysis of the log analysis result.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a diagram illustrating an exemplary implementation of a log rule quality analysis method;
FIG. 2 is a schematic flow chart diagram illustrating a method for log rule quality analysis in one embodiment;
FIG. 3 is a schematic flow chart diagram of a log rule quality analysis method in another embodiment;
FIG. 4 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
For a clearer understanding of the objects, aspects and advantages of the present application, reference is made to the following description and accompanying drawings.
Unless defined otherwise, technical or scientific terms used herein shall have the same general meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The use of the terms "a" and "an" and "the" and similar referents in the context of this application do not denote a limitation of quantity, either in the singular or the plural. The terms "comprises," "comprising," "has," "having," and any variations thereof, as referred to in this application, are intended to cover non-exclusive inclusions; for example, a process, method, and system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or modules, but may include other steps or modules (elements) not listed or inherent to such process, method, article, or apparatus. Reference throughout this application to "connected," "coupled," and the like is not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Reference to "a plurality" in this application means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. In general, the character "/" indicates a relationship in which the objects associated before and after are an "or". The terms "first," "second," "third," and the like in this application are used for distinguishing between similar items and not necessarily for describing a particular sequential or chronological order.
The log rule quality analysis method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal device 102 communicates with the server 104 over a network. The terminal device 102 receives an initial log analysis rule packet and a log sample; the server 104 acquires an initial log analysis rule packet, and performs format conversion on the acquired field name and field value to obtain a target log analysis rule packet; the server 104 inputs the target log analysis rule packet and the log sample into the log analysis module to output a first log analysis result, and inputs the target log analysis rule packet and the log sample into the test framework to output a second log analysis result; the server 104 obtains a log analysis rule verification result according to the first log analysis result and the second log analysis result by using a preset verification rule. The terminal device 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by an independent server or a server cluster formed by a plurality of servers.
In this embodiment, a log rule quality analysis method is provided, and fig. 2 is a flowchart of the log rule quality analysis method of this embodiment, as shown in fig. 2, the flowchart includes the following steps:
step S202, a target log analysis rule packet and a log sample are obtained. The log sample can be obtained by manually importing a log sample text, or can be received through a network protocol.
Step S204, inputting the target log analysis rule packet and the log sample into a log analysis module to output a first log analysis result, and inputting the target log analysis rule packet and the log sample into a test frame to output a second log analysis result; the log analysis module is used for generating a first log analysis result, and the test framework is used for generating a second log analysis result.
It should be noted that the log parsing module parses the log sample according to the log parsing rule packet, checks whether the parsing result is normal, whether the data field is missing, and whether a mismatch exists, and outputs a first log parsing result; the test framework extracts field names and field characteristic values in the log analysis rule package, and the test framework may be a Python test framework such as Pytest or UnitTest, or a test framework supporting other programming languages, which is not described herein again.
Step S206, obtaining a log parsing rule checking result according to the first log parsing result and the second log parsing result by using a preset checking rule. The preset check rule is to compare the first log analysis result and the second log analysis result one by one to obtain a check result of the log analysis rule.
Through the steps, the field names and the field characteristic values in the log analysis rule packet are extracted by using the test framework to output a second log analysis result, and the second log analysis result is compared with the first log analysis result output by the log analysis module to be detected automatically one by one, so that the problems of low checking efficiency and high error rate of the analysis rule test result are solved, and the automatic quality analysis of the log analysis result is realized.
In some embodiments, the obtaining the target log parsing rule package comprises:
acquiring an initial log analysis rule packet, acquiring field names defined in the initial log analysis rule packet and field values corresponding to the field names; wherein, the field name includes: the method comprises the following steps of (1) obtaining a rule name, an asset type corresponding to the rule, a regular expression and a prejudgment mode; the pre-judging mode is a pre-matching mode matched with the analysis rule defined in the log analysis rule packet in advance;
and carrying out format conversion on the acquired field name and the field value to obtain a conversion field name and a conversion field value, and obtaining the target log analysis rule packet according to the conversion field name and the conversion field value.
It should be noted that, the log parsing rule packet in the initial text form may be automatically obtained by obtaining the initial log parsing rule packet through a script program, and the script program may be embedded in the Pytest test frame and the log parsing module, or may be separately compiled and run outside the Pytest frame and the log parsing module; the asset type corresponding to the rule refers to the asset type to which the asset generating the log sample belongs, and may be software, equipment, a system and the like constituting a computer system, for example, Windows; the asset types corresponding to the rules distinguish different format log samples generated by different types, and the different log samples generated by different types of assets respectively correspond to different analysis rules; the regular expression is used for describing or matching character strings meeting a certain rule, and is used for matching log texts meeting a certain rule in log analysis; the step of performing format conversion on the obtained field name and the field value refers to converting the field name and the field value in the log parsing rule packet in the obtained text form into a format suitable for being executed by a computer programming language, for example, converting a text or a character string in a certain format into a certain data structure, where the format conversion may be parser conversion or other format conversion methods capable of performing data result conversion.
By the embodiment, the initial log analysis rule packet in the text form can be converted into the field name and the field value in the program language form suitable for the execution of a computer program, so that the automatic loading of the log analysis rule packet is realized, and a basis is provided for the automatic quality analysis of the analysis result of a subsequent log analysis module; meanwhile, by determining the asset type of the log sample, the asset type of the asset generating the log sample can be determined before verification so as to screen out the log sample of a corresponding type, reduce the sample size and improve the efficiency of analysis and verification.
In some of these embodiments, obtaining the log sample comprises:
acquiring the log sample by using a preset log storage file; alternatively, the log sample is obtained using a network protocol specified by the parsing rule. The log sample is obtained by using a preset log storage file, wherein the log sample can be selected from the preset log storage file and then manually imported; the network protocol may be a UDP protocol, a TCP protocol, a RELP protocol, a syslog protocol, or a Rsyslog protocol.
Through the embodiment, in the implementation process of the log rule quality analysis method, the log sample can be obtained through two modes of manual introduction and network protocol acquisition. The input of the log sample is required to be used as a data basis of quality analysis in the log rule quality analysis, the log rule quality analysis is controlled to be started by manually controlling the acquisition of the log sample in a manual importing mode, and a user can manually judge the result of the log rule quality analysis; or, the starting behavior is embedded into an automation program in a mode of acquiring the log sample through a network protocol, so that the analysis results of all log analysis modules can be verified; the multiple starting modes of the log rule quality analysis can cause two different starting execution modes of manual starting execution and automatic starting execution to adapt to different application scenes.
In some of these embodiments, inputting the target log resolution rule package and the log sample to a log resolution module to output a first log resolution result comprises:
inputting the target log analysis rule packet and the log sample into the log analysis module;
acquiring a log analysis rule corresponding to a pre-judging mode according to the pre-judging mode in the target log analysis rule packet by using the log analysis module, and analyzing according to a regular expression in the log analysis rule to obtain a corresponding field name and a corresponding field value;
and performing a first analysis operation on the log sample according to the field name and the field sampling value by using the log analysis module, and outputting to obtain a first log analysis result.
The predetermined pattern is a predetermined matching pattern that matches the analysis rule defined in the log analysis rule packet in advance. Furthermore, the head character strings of the log sample text have a certain format, the pre-judging mode in the log analysis rule packet comprises a plurality of log sample text head character strings with different formats, the log sample text head character strings with different formats correspond to different analysis rules, and in the process of acquiring the log analysis rule corresponding to the pre-judging mode in the pre-judging mode, the pre-judging mode preliminarily screens the log samples to match the log samples with the corresponding head character strings, and then finds the corresponding log analysis rule according to the pre-judging mode of the corresponding head character strings. Specifically, the acquired log sample starts from Nov 615: 49:39, the prejudgment mode acquires the start of the log sample and finds an analysis rule corresponding to the start of the log sample, and a regular expression corresponding to the start of the log sample in the analysis rule is as follows: < '>' S {1,3} \ S \ d {1,2 }.
Through the steps, the log analysis module serving as a log rule quality analysis object generates a first log analysis result, and a log analysis rule verification result, namely a log rule quality analysis result of the log analysis module, is obtained through comparison of the correctness of the first log analysis result in the subsequent log rule quality analysis process, so that the problems of low checking efficiency and high error rate of analysis rule test results are solved, and automatic quality analysis of log rules is realized.
In some embodiments, performing, by the log parsing module, a first parsing operation on the log sample according to the field name and the field value, and outputting a first log parsing result, further includes:
obtaining an analysis operation check result of whether the log analysis module is normal for the first analysis operation, obtaining a log analysis module field check result of whether the log analysis module is missing for a rule data field in the target log analysis rule packet, and obtaining a log analysis module matching check result of whether the log analysis module has mismatching for different log analysis rules;
and utilizing the log analysis module to perform a first analysis operation on the log sample according to the field name and the field sampling value to obtain an initial analysis result, verifying the initial analysis result according to the analysis operation verification result, the log analysis module field verification result and the log analysis module matching verification result, and finally outputting to obtain the first log analysis result.
Through the steps, the log analysis module is utilized to verify whether the first analysis operation is normal, whether the rule data field in the target log analysis rule packet is missing and whether different log analysis rules are mismatched, so that the efficiency and the accuracy of log analysis can be ensured, the output results of the three aspects are important contents of the first log analysis result, the analysis quality of the log analysis module can be reflected, the problems of low verification efficiency and high error rate of analysis rule test results are solved, and the automatic quality analysis of the log rules is realized.
In some of these embodiments, the inputting the target log resolution rule package and the log sample to a test framework to output a second log resolution result includes:
inputting the target log analysis rule packet and the log sample into the test framework;
acquiring a log analysis rule corresponding to a pre-judging mode according to the pre-judging mode in the target log analysis rule packet by using the test frame, and analyzing according to a regular expression in the log analysis rule to obtain a corresponding field name and a corresponding field value;
and performing second analysis operation on the log sample according to the field name and the field sampling value by using the test framework, and outputting to obtain a second log analysis result.
Through the steps, the field name and the field characteristic value in the log analysis rule packet are extracted by using the test framework to output a second log analysis result, and the second log analysis result is automatically compared with the first log analysis result output by the log analysis module to be detected one by one in the subsequent log rule quality analysis process, so that the problems of low checking efficiency and high error rate of the analysis rule test result are solved, and the automatic quality analysis of the log analysis result is realized.
In some embodiments, the obtaining, by using a preset check rule, a log parsing rule check result according to the first log parsing result and the second log parsing result includes:
utilizing the check rule to automatically compare the field name and the field value of the first log analysis result obtained by analysis with the field name and the field value of the second log analysis result one by one according to each field name and corresponding field value defined in the log analysis rule packet to obtain a comparison result; obtaining a log analysis rule checking result according to the comparison result;
sending the first log analysis result, the second log analysis result and the log analysis rule verification result to the terminal equipment; the terminal equipment is used for rendering an interface according to the log analysis rule verification result, generating a visual test report and displaying the visual test report.
Through the steps, the field name and the field value of the first log analysis result obtained through analysis and the field name and the field value of the second log analysis result are compared automatically one by one according to the field names and the corresponding field values defined in the log analysis rule packet to obtain a comparison result, and the log analysis rule verification result is obtained according to the comparison result, so that the problems of low verification efficiency and high error rate of analysis rule test results are solved, and automatic quality analysis of the log analysis results is realized.
It should be understood that, although the steps in the flowchart of fig. 2 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a portion of the steps in fig. 2 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
The present embodiment is described and illustrated below by means of preferred embodiments.
Fig. 3 is a flow diagram of a log rule quality analysis method of another embodiment. Firstly, acquiring a log analysis rule packet to be tested, carrying out Parser conversion of an analysis rule format, sending the converted analysis rule to a Pytest test frame and a log analysis module, and simultaneously sending a log sample to the Pytest test frame and the log analysis module; secondly, extracting field names and field values in the converted log analysis rule packet by a Pytest test framework to obtain a result corresponding to the log sample text and the analysis rule; meanwhile, the log analysis module analyzes the log sample according to the log analysis rule; extracting the analysis result of the Pytest test frame and the analysis result of the log analysis module, and verifying according to a verification rule; and finally, constructing a data verification task through the API, and outputting a data verification result to a UI display interface for visual display.
It should be noted that the steps illustrated in the above-described flow diagrams or in the flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order different than here. For example, the step of constructing a data verification task (API/UI) may be performed simultaneously with the step of sending the parsing rule/log sample, or may be performed simultaneously with the step of verifying the verification rule; the field name and field value taking steps in the Pytest extraction and analysis rule can be carried out simultaneously with the log analysis module execution steps or sequentially.
In this embodiment, a log rule quality analysis system is further provided, where the system includes: a terminal device, a transmission device and a server device; the terminal equipment is connected with the server equipment through the transmission equipment;
the terminal equipment is used for receiving a target log analysis rule packet and a log sample;
the transmission equipment is used for sending the target log analysis rule packet and the log sample to the server equipment;
the server device is adapted to perform the steps of any of the above method embodiments.
There is also provided in this embodiment an electronic device comprising a memory having a computer program stored therein and a processor arranged to run the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
and S2, acquiring the target log analysis rule packet and the log sample.
S4, inputting the target log analysis rule packet and the log sample into a log analysis module to output a first log analysis result, and inputting the target log analysis rule packet and the log sample into a test framework to output a second log analysis result; the log analysis module is used for generating the first log analysis result, and the test framework is used for generating the second log analysis result.
And S6, obtaining a log analysis rule verification result according to the first log analysis result and the second log analysis result by using a preset verification rule.
It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and optional implementations, and details are not described again in this embodiment.
In addition, in combination with the log rule quality analysis method provided in the foregoing embodiment, a storage medium may also be provided to implement this embodiment. The storage medium having stored thereon a computer program; the computer program, when executed by a processor, implements any of the log rule quality analysis methods in the above embodiments.
In one embodiment, a computer device is also provided, and the computer device may be a server, and the internal structure diagram thereof may be as shown in fig. 4. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a log rule quality analysis method.
Those skilled in the art will appreciate that the architecture shown in fig. 4 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to be limiting. All other embodiments, which can be derived by a person skilled in the art from the examples provided herein without any inventive step, shall fall within the scope of protection of the present application.
It is obvious that the drawings are only examples or embodiments of the present application, and it is obvious to those skilled in the art that the present application can be applied to other similar cases according to the drawings without creative efforts. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.
The term "embodiment" is used herein to mean that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly or implicitly understood by one of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the patent protection. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (10)
1. A log rule quality analysis method is characterized by comprising the following steps:
acquiring a target log analysis rule packet and a log sample;
inputting the target log analysis rule packet and the log sample into a log analysis module to output a first log analysis result, and inputting the target log analysis rule packet and the log sample into a test framework to output a second log analysis result; the log analysis module is used for generating a first log analysis result, and the test framework is used for generating a second log analysis result;
and obtaining a log analysis rule checking result according to the first log analysis result and the second log analysis result by using a preset checking rule.
2. The log rule quality analysis method of claim 1, wherein the obtaining a target log parsing rule packet comprises:
acquiring an initial log analysis rule packet, acquiring field names defined in the initial log analysis rule packet and field values corresponding to the field names; wherein the field name comprises: the method comprises the following steps of (1) obtaining a rule name, an asset type corresponding to the rule, a regular expression and a prejudgment mode; the pre-judging mode is a pre-matching mode matched with an analysis rule defined in the log analysis rule packet in advance;
and carrying out format conversion on the acquired field name and the field value to obtain a conversion field name and a conversion field value, and obtaining the target log analysis rule packet according to the conversion field name and the conversion field value.
3. The log rule quality analysis method of claim 1, wherein obtaining the log sample comprises:
acquiring the log sample by using a preset log storage file; or, the log sample is obtained by using a network protocol specified by the analysis rule.
4. The log rule quality analysis method of claim 1, wherein the inputting the target log parsing rule packet and the log sample to a log parsing module to output a first log parsing result comprises:
inputting the target log analysis rule packet and the log sample into the log analysis module;
acquiring a log analysis rule corresponding to a pre-judging mode according to the pre-judging mode in the target log analysis rule packet by using the log analysis module, and analyzing according to a regular expression in the log analysis rule to obtain a corresponding field name and a corresponding field value;
and utilizing the log analysis module to perform first analysis operation on the log sample according to the field name and the field value, and outputting to obtain a first log analysis result.
5. The log rule quality analysis method according to claim 4, wherein the performing, by the log parsing module, a first parsing operation on the log sample according to the field name and the field value and outputting a result of the first log parsing further comprises:
obtaining an analysis operation check result of the log analysis module for judging whether a first analysis operation is normal or not, obtaining a log analysis module field check result of the log analysis module for judging whether a rule data field in the target log analysis rule packet is missing or not, and obtaining a log analysis module matching check result of the log analysis module for judging whether different log analysis rules are mismatched or not;
and utilizing the log analysis module to perform first analysis operation on the log sample according to the field name and the field sampling value to obtain an initial analysis result, verifying the initial analysis result according to the analysis operation verification result, the log analysis module field verification result and the log analysis module matching verification result, and finally outputting to obtain the first log analysis result.
6. The log rule quality analysis method of claim 1, wherein the inputting the target log resolution rule package and the log sample to a test framework to output a second log resolution result comprises:
inputting the target log analysis rule packet and the log sample into the test framework;
acquiring a log analysis rule corresponding to a pre-judging mode according to the pre-judging mode in the target log analysis rule packet by using the test frame, and analyzing according to a regular expression in the log analysis rule to obtain a corresponding field name and a corresponding field value;
and carrying out second analysis operation on the log sample according to the field name and the field value by using the test framework, and outputting to obtain a second log analysis result.
7. The log rule quality analysis method according to any one of claims 1 to 6, wherein obtaining a log parsing rule check result according to the first log parsing result and the second log parsing result by using a preset check rule comprises:
comparing the field name and the field value of the first log analysis result obtained by analysis with the field name and the field value of the second log analysis result by using the check rule according to each field name and corresponding field value defined in the log analysis rule packet to obtain a comparison result; obtaining a log analysis rule checking result according to the comparison result;
sending the first log analysis result, the second log analysis result and the log analysis rule verification result to terminal equipment; and the terminal equipment is used for rendering an interface aiming at the log analysis rule verification result, generating a visual test report and displaying the visual test report.
8. A log rule quality analysis system, comprising: a terminal device, a transmission device and a server device; the terminal equipment is connected with the server equipment through the transmission equipment;
the terminal equipment is used for receiving a target log analysis rule packet and a log sample;
the transmission equipment is used for sending the target log analysis rule packet and the log sample to the server equipment;
the server device is configured to perform the log rule quality analysis method of any one of claims 1 to 7.
9. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and the processor is configured to execute the computer program to perform the log rule quality analysis method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the log rule quality analysis method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111069491.1A CN113742192A (en) | 2021-09-13 | 2021-09-13 | Log rule quality analysis method, system, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111069491.1A CN113742192A (en) | 2021-09-13 | 2021-09-13 | Log rule quality analysis method, system, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113742192A true CN113742192A (en) | 2021-12-03 |
Family
ID=78738433
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111069491.1A Pending CN113742192A (en) | 2021-09-13 | 2021-09-13 | Log rule quality analysis method, system, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113742192A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1917445A (en) * | 2006-09-07 | 2007-02-21 | 上海交通大学 | Method for auditing log event of fire wall, and teaching experimental system |
| US20090119307A1 (en) * | 2007-10-22 | 2009-05-07 | Check Point Software Technologies Ltd. | Syslog parser |
| CN106294673A (en) * | 2016-08-08 | 2017-01-04 | 杭州玳数科技有限公司 | A kind of method and system of User Defined rule real time parsing daily record data |
| CN110765148A (en) * | 2019-10-28 | 2020-02-07 | 支付宝(杭州)信息技术有限公司 | Service data processing method and device |
| CN111143312A (en) * | 2019-12-24 | 2020-05-12 | 广东电科院能源技术有限责任公司 | Format analysis method, device, equipment and storage medium for power logs |
-
2021
- 2021-09-13 CN CN202111069491.1A patent/CN113742192A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1917445A (en) * | 2006-09-07 | 2007-02-21 | 上海交通大学 | Method for auditing log event of fire wall, and teaching experimental system |
| US20090119307A1 (en) * | 2007-10-22 | 2009-05-07 | Check Point Software Technologies Ltd. | Syslog parser |
| CN106294673A (en) * | 2016-08-08 | 2017-01-04 | 杭州玳数科技有限公司 | A kind of method and system of User Defined rule real time parsing daily record data |
| CN110765148A (en) * | 2019-10-28 | 2020-02-07 | 支付宝(杭州)信息技术有限公司 | Service data processing method and device |
| CN111143312A (en) * | 2019-12-24 | 2020-05-12 | 广东电科院能源技术有限责任公司 | Format analysis method, device, equipment and storage medium for power logs |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10394686B2 (en) | Static feature extraction from structured files | |
| US10108535B2 (en) | Web application test script generation to test software functionality | |
| CN111061526B (en) | Automatic test method, device, computer equipment and storage medium | |
| US10572370B2 (en) | Test-assisted application programming interface (API) learning | |
| KR20190026641A (en) | Method of character recognition of claims document, apparatus, server and storage medium | |
| CN111813701B (en) | HTTP-based interface testing method and device, computer equipment and storage medium | |
| CN111176996A (en) | Test case generation method and device, computer equipment and storage medium | |
| CN108920359B (en) | Application program testing method and device, storage medium and electronic device | |
| CN110955600B (en) | Interface testing method and device | |
| CN110704304A (en) | Application program testing method and device, storage medium and server | |
| CN110955608B (en) | Test data processing method, device, computer equipment and storage medium | |
| CN111736811A (en) | Form data verification method, system, server and user terminal | |
| EP2722783A2 (en) | License verification method and apparatus | |
| CN110287700B (en) | iOS application security analysis method and device | |
| CN113434400A (en) | Test case execution method and device, computer equipment and storage medium | |
| CN112612706A (en) | Automated testing method, computer device and storage medium | |
| CN117312148A (en) | Automatic interface testing method and device, computer equipment and storage medium | |
| CN113742192A (en) | Log rule quality analysis method, system, electronic device and storage medium | |
| CN111078529A (en) | Client write-in module testing method and device and electronic equipment | |
| CN116069324A (en) | Dynamic form construction method and device based on Vue | |
| CN108446127B (en) | Update package processing method and device | |
| CN115904978A (en) | Redfish interface testing method, computing device and storage medium | |
| CN114417812A (en) | Text checking method, device, equipment and storage medium | |
| CN113868210A (en) | Validity verification method, system, equipment and storage medium for imported data | |
| US20230071959A1 (en) | Analysis device, analysis method, and analysis program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |