CN112905579A - Log optimization method and system - Google Patents
Log optimization method and system Download PDFInfo
- Publication number
- CN112905579A CN112905579A CN202110236184.1A CN202110236184A CN112905579A CN 112905579 A CN112905579 A CN 112905579A CN 202110236184 A CN202110236184 A CN 202110236184A CN 112905579 A CN112905579 A CN 112905579A
- Authority
- CN
- China
- Prior art keywords
- log data
- log
- data
- preset
- original
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/248—Presentation of query results
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a log optimization method and a log optimization system. Wherein, the method comprises the following steps: acquiring original log data; presetting the original log data to obtain first log data; generating second log data according to the first log data through a preset rule; and displaying the first log data and the second log data. The invention solves the technical problems that the prior art directly optimizes and applies the original log data and has lower accuracy.
Description
Technical Field
The invention relates to the field of log processing, in particular to a log optimization method and a log optimization system.
Background
With the continuous development of intelligent and information technology, people use intelligent devices in more and more lives, works and studies, for example, in a large amount of data, the devices need logs of different types to record data traces, and the change of data to be checked by adding, deleting and modifying needs to be realized through the logs.
At present, in the process of log processing, in order to obtain a more accurate log recording result, a method of directly optimizing and sorting original data, then recording the log, displaying and analyzing is generally adopted, but when the data volume is huge, the original data is directly optimized, so that the occurrence of data error rate is increased, namely certain error data are applied to log analysis, and the accuracy of the log application process is reduced.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a log optimization method and a log optimization system, which at least solve the technical problem that the prior art directly optimizes and applies original log data and has low accuracy.
According to an aspect of an embodiment of the present invention, there is provided a log optimization method, including: acquiring original log data; presetting the original log data to obtain first log data; generating second log data according to the first log data through a preset rule; and displaying the first log data and the second log data.
Optionally, the performing the preset processing on the original log data to obtain the first log data includes: splitting the original log data to obtain log data elements; and optimizing the log data elements one by one according to a preset processing mode to obtain the first log data.
Optionally, the generating, according to the first log data and by using a preset rule, second log data includes: comparing the first log data with a preset parameter to obtain a comparison result; and according to the comparison result, generating the second log data from the first log data through the preset rule, wherein the preset rule is used for further optimizing the first log data so that the first log data better accord with the configuration of the preset parameter.
Optionally, before the presenting the first log data and the second log data, the method further includes: merging the first log data and the second log data into a preset data set.
According to another aspect of the embodiments of the present invention, there is also provided a log optimization system, including: the acquisition module is used for acquiring original log data; the preprocessing module is used for carrying out preset processing on the original log data to obtain first log data; the generating module is used for generating second log data according to the first log data through a preset rule; and the display module is used for displaying the first log data and the second log data.
Optionally, the preprocessing module includes: the splitting unit is used for splitting the original log data to obtain log data elements; and the optimization unit is used for optimizing the log data elements one by one according to a preset processing mode to obtain the first log data.
Optionally, the generating module includes: the comparison unit is used for comparing the first log data with preset parameters to obtain a comparison result; and the generating unit is used for generating the second log data from the first log data through the preset rule according to the comparison result, wherein the preset rule is used for further optimizing the first log data so that the first log data better accord with the configuration of the preset parameter.
Optionally, the system further includes: and the merging module is used for merging the first log data and the second log data into a preset data set.
According to another aspect of embodiments of the present invention, there is also provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform a method of log optimization.
According to another aspect of the embodiments of the present invention, there is also provided a non-volatile storage medium, where the non-volatile storage medium includes a stored program, and the program controls a device in which the non-volatile storage medium is located to execute a log optimization method when running.
In the embodiment of the invention, the original log data is obtained; presetting the original log data to obtain first log data; generating second log data according to the first log data through a preset rule; the method for displaying the first log data and the second log data solves the technical problem that the accuracy is low when the original log data are directly optimized and applied in the prior art.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow chart of a method of log optimization according to an embodiment of the invention;
fig. 2 is a block diagram of a log optimization system according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In accordance with an embodiment of the present invention, there is provided a method embodiment of a log optimization method, it being noted that the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than that presented herein.
Example one
Fig. 1 is a flowchart of a log optimization method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, obtaining original log data.
Specifically, in order to analyze and verify the acquired log, the embodiment of the present invention needs to first acquire original log data, where the log data is a log obtained by acquiring and recording device-related information in real time, and the data relates to sensing data of a plurality of sensors or operation data of a server.
Step S104, performing preset processing on the original log data to obtain first log data.
Specifically, after the log data is obtained, in order to facilitate analysis and verification of the log data, optimization processing (preset processing) needs to be performed on the original log data, that is, the log data needs to be preprocessed, where the preprocessing includes: and (6) redundant elimination. Therefore, the preprocessed log data can be better executed by subsequent analysis and verification operations, and the efficiency of the whole log analysis and verification is increased.
It should be noted that the elimination of the redundant log may be by capturing a data packet; analyzing the data packet to obtain a log corresponding to the data packet; acquiring relevant characteristics of a log to be detected, and judging whether the log is a relevant log or not according to the relevant characteristics; if the log to be detected is an irrelevant log, judging whether the log is a repeated log according to the repeated characteristics of the log, and performing corresponding processing after judging whether the log is the relevant log or the repeated log.
Optionally, the performing the preset processing on the original log data to obtain the first log data includes: splitting the original log data to obtain log data elements; and optimizing the log data elements one by one according to a preset processing mode to obtain the first log data.
Specifically, according to the optimization processing process of the original log data in the embodiment of the present invention, after the log data is split into the log data elements, each of the split log data elements needs to be optimized and preset, and the optimized log data is used as the first log data for subsequent log optimization and analysis.
And step S106, generating second log data according to the first log data through a preset rule.
Optionally, the generating, according to the first log data and by using a preset rule, second log data includes: comparing the first log data with a preset parameter to obtain a comparison result; and according to the comparison result, generating the second log data from the first log data through the preset rule, wherein the preset rule is used for further optimizing the first log data so that the first log data better accord with the configuration of the preset parameter.
Specifically, after the first log data is obtained, in order to make the optimization effect of the first log data better, the first log data may be compared with a preset parameter to obtain a comparison result; and according to the comparison result, generating the second log data from the first log data through the preset rule, wherein the preset rule is used for further optimizing the first log data so that the first log data better accord with the configuration of the preset parameter. The data content of the second log data is the same as that of the first log data, and the second log data is derived from the original log data, but according to the requirement of the preset parameter, the part of the first log data which is not optimized is further optimized through a preset rule, and then the second log data is generated.
Step S108, displaying the first log data and the second log data.
Specifically, after the second log data is obtained, in order to enable a user to see the first log data and the second log data more clearly and intuitively, the first log data and the second log data can be displayed simultaneously, so that the significance of certain log optimization comparison data is obtained, the user experience is increased, and more use options are provided for the user.
Optionally, before the presenting the first log data and the second log data, the method further includes: merging the first log data and the second log data into a preset data set.
Specifically, when the log optimization data is displayed, the first log data and the second log data need to be displayed simultaneously, so that the first log data and the second log data need to be merged into a preset data set.
Through the embodiment, the technical problems that the original log data are directly optimized and applied and the accuracy is low in the prior art are solved.
Example two
Fig. 2 is a block diagram of a log optimization system according to an embodiment of the present invention, and as shown in fig. 2, the system includes:
and an obtaining module 20, configured to obtain the original log data.
Specifically, in order to analyze and verify the acquired log, the embodiment of the present invention needs to first acquire original log data, where the log data is a log obtained by acquiring and recording device-related information in real time, and the data relates to sensing data of a plurality of sensors or operation data of a server.
The preprocessing module 22 is configured to perform preset processing on the original log data to obtain first log data.
Specifically, after the log data is obtained, in order to facilitate analysis and verification of the log data, optimization processing (preset processing) needs to be performed on the original log data, that is, the log data needs to be preprocessed, where the preprocessing includes: and (6) redundant elimination. Therefore, the preprocessed log data can be better executed by subsequent analysis and verification operations, and the efficiency of the whole log analysis and verification is increased.
It should be noted that the elimination of the redundant log may be by capturing a data packet; analyzing the data packet to obtain a log corresponding to the data packet; acquiring relevant characteristics of a log to be detected, and judging whether the log is a relevant log or not according to the relevant characteristics; if the log to be detected is an irrelevant log, judging whether the log is a repeated log according to the repeated characteristics of the log, and performing corresponding processing after judging whether the log is the relevant log or the repeated log.
Optionally, the preprocessing module includes: the splitting unit is used for splitting the original log data to obtain log data elements; and the optimization unit is used for optimizing the log data elements one by one according to a preset processing mode to obtain the first log data.
Specifically, according to the optimization processing process of the original log data in the embodiment of the present invention, after the log data is split into the log data elements, each of the split log data elements needs to be optimized and preset, and the optimized log data is used as the first log data for subsequent log optimization and analysis.
And a generating module 24, configured to generate second log data according to the first log data by using a preset rule.
Optionally, the generating module includes: the comparison unit is used for comparing the first log data with preset parameters to obtain a comparison result; and the generating unit is used for generating the second log data from the first log data through the preset rule according to the comparison result, wherein the preset rule is used for further optimizing the first log data so that the first log data better accord with the configuration of the preset parameter.
Specifically, after the first log data is obtained, in order to make the optimization effect of the first log data better, the first log data may be compared with a preset parameter to obtain a comparison result; and according to the comparison result, generating the second log data from the first log data through the preset rule, wherein the preset rule is used for further optimizing the first log data so that the first log data better accord with the configuration of the preset parameter. The data content of the second log data is the same as that of the first log data, and the second log data is derived from the original log data, but according to the requirement of the preset parameter, the part of the first log data which is not optimized is further optimized through a preset rule, and then the second log data is generated.
A display module 26, configured to display the first log data and the second log data.
Specifically, after the second log data is obtained, in order to enable a user to see the first log data and the second log data more clearly and intuitively, the first log data and the second log data can be displayed simultaneously, so that the significance of certain log optimization comparison data is obtained, the user experience is increased, and more use options are provided for the user.
Optionally, the system further includes: and the merging module is used for merging the first log data and the second log data into a preset data set.
Specifically, when the log optimization data is displayed, the first log data and the second log data need to be displayed simultaneously, so that the first log data and the second log data need to be merged into a preset data set.
According to another aspect of embodiments of the present invention, there is also provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform a method of log optimization.
Specifically, the log optimization method includes: acquiring original log data; presetting the original log data to obtain first log data; generating second log data according to the first log data through a preset rule; and displaying the first log data and the second log data.
According to another aspect of the embodiments of the present invention, there is also provided a non-volatile storage medium, where the non-volatile storage medium includes a stored program, and the program controls a device in which the non-volatile storage medium is located to execute a log optimization method when running.
Specifically, the log optimization method includes: acquiring original log data; presetting the original log data to obtain first log data; generating second log data according to the first log data through a preset rule; and displaying the first log data and the second log data.
Through the embodiment, the technical problems that the original log data are directly optimized and applied and the accuracy is low in the prior art are solved.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method for log optimization, comprising:
acquiring original log data;
presetting the original log data to obtain first log data;
generating second log data according to the first log data through a preset rule;
and displaying the first log data and the second log data.
2. The method according to claim 1, wherein the pre-processing the original log data to obtain the first log data comprises:
splitting the original log data to obtain log data elements;
and optimizing the log data elements one by one according to a preset processing mode to obtain the first log data.
3. The method according to claim 1, wherein the generating second log data according to the first log data by a preset rule comprises:
comparing the first log data with a preset parameter to obtain a comparison result;
and according to the comparison result, generating the second log data from the first log data through the preset rule, wherein the preset rule is used for further optimizing the first log data so that the first log data better accord with the configuration of the preset parameter.
4. The method of claim 1, wherein prior to said exposing the first log data and the second log data, the method further comprises:
merging the first log data and the second log data into a preset data set.
5. A log optimization system, comprising:
the acquisition module is used for acquiring original log data;
the preprocessing module is used for carrying out preset processing on the original log data to obtain first log data;
the generating module is used for generating second log data according to the first log data through a preset rule;
and the display module is used for displaying the first log data and the second log data.
6. The system of claim 5, wherein the pre-processing module comprises:
the splitting unit is used for splitting the original log data to obtain log data elements;
and the optimization unit is used for optimizing the log data elements one by one according to a preset processing mode to obtain the first log data.
7. The system of claim 5, wherein the generating module comprises:
the comparison unit is used for comparing the first log data with preset parameters to obtain a comparison result;
and the generating unit is used for generating the second log data from the first log data through the preset rule according to the comparison result, wherein the preset rule is used for further optimizing the first log data so that the first log data better accord with the configuration of the preset parameter.
8. The system of claim 5, further comprising:
and the merging module is used for merging the first log data and the second log data into a preset data set.
9. A computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of any one of claims 1 to 4.
10. A non-volatile storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the non-volatile storage medium is located to perform the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110236184.1A CN112905579A (en) | 2021-03-03 | 2021-03-03 | Log optimization method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110236184.1A CN112905579A (en) | 2021-03-03 | 2021-03-03 | Log optimization method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112905579A true CN112905579A (en) | 2021-06-04 |
Family
ID=76107581
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110236184.1A Withdrawn CN112905579A (en) | 2021-03-03 | 2021-03-03 | Log optimization method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112905579A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113360723A (en) * | 2021-07-08 | 2021-09-07 | 北京智思迪科技有限公司 | Data acquisition method and device |
-
2021
- 2021-03-03 CN CN202110236184.1A patent/CN112905579A/en not_active Withdrawn
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113360723A (en) * | 2021-07-08 | 2021-09-07 | 北京智思迪科技有限公司 | Data acquisition method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106980573B (en) | Method, device and system for constructing test case request object | |
| CN107797894B (en) | APP user behavior analysis method and device | |
| CN111552633A (en) | Interface abnormal call testing method and device, computer equipment and storage medium | |
| CN110213238A (en) | Threat detection method and device, storage medium, the computer equipment of data | |
| CN110888805A (en) | RESTful API playback test method and system | |
| CN112905579A (en) | Log optimization method and system | |
| CN112860733A (en) | Log verification method and system | |
| CN107368500A (en) | Data pick-up method and system | |
| CN112836807A (en) | Data processing method and device based on neural network | |
| CN112866800A (en) | Video content similarity detection method, device, equipment and storage medium | |
| CN112579352A (en) | Quality monitoring result generation method, storage medium and quality monitoring system of service data processing link | |
| CN115774707B (en) | Object attribute-based data processing method and device, electronic equipment and storage medium | |
| CN116841846A (en) | Real-time log abnormality detection method, device, equipment and storage medium thereof | |
| CN113312902A (en) | Intelligent auditing and checking method and device for same text | |
| CN113672497B (en) | Method, device and equipment for generating non-buried point event and storage medium | |
| CN115756486A (en) | Data interface analysis method and device | |
| CN115827691A (en) | Batch processing result verification method and device, computer equipment and storage medium | |
| CN112256552B (en) | Pressure testing method and device for map engine | |
| CN115509637A (en) | Form-based intelligent filling method, system, equipment and medium | |
| CN112860956A (en) | Multivariate environmental data analysis method and system | |
| CN112860678A (en) | Log display method and system | |
| CN112860456A (en) | Log processing method and device | |
| CN112860527A (en) | Fault monitoring method and device of application server | |
| CN113313615A (en) | Method and device for quantitatively grading and grading enterprise judicial risks | |
| CN110298935B (en) | Method for acquiring user operation habit information, diagnosis equipment and server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20210604 |