CN111144086B - Log formatting method and device, electronic equipment and storage medium - Google Patents
Log formatting method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111144086B CN111144086B CN201911329806.4A CN201911329806A CN111144086B CN 111144086 B CN111144086 B CN 111144086B CN 201911329806 A CN201911329806 A CN 201911329806A CN 111144086 B CN111144086 B CN 111144086B
- Authority
- CN
- China
- Prior art keywords
- log
- format
- template
- templates
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The application provides a log formatting method and device, electronic equipment and a storage medium, wherein the method comprises the following steps: determining a first journal, the first journal comprising journal content of at least one parameter type; acquiring N log templates corresponding to a log server requiring formatting from pre-stored log templates; storing the format string and the parameter type list corresponding to each log template in the N log templates into an operating memory; determining N log content stacks corresponding to the N log templates one by one according to the log contents and storing the log content stacks in the running memory; and formatting each log content stack according to the format string in the log template corresponding to each log content stack. By the method, the log can be formatted according to format requirements of different log servers, and the format of the log is changed in the later period, so that only the log template is required to be modified, codes are not required to be modified, and resources are saved.
Description
Technical Field
The present application relates to the field of communications technologies, and in particular, to a log formatting method and apparatus, an electronic device, and a storage medium.
Background
With the development of networks, internet applications have penetrated every corner of social life and become indispensable tools for people to study, work and live. The openness, interactivity and extensibility of the internet provide great convenience for people to quickly acquire knowledge, communicate in real time and communicate across regions; meanwhile, the convenience and the virtualization of the internet also become a hotbed for breeding of various discordant behaviors, and the problems of network confusion, meat searching, website browsing irrelevant to work during working time and the like are solved, so that more and more serious challenges are brought to social harmony, enterprise efficiency and the like. Therefore, how to solve the negative problems brought by the internet and maintain a healthy and efficient network environment becomes a problem to be solved urgently.
In order to solve the above problems, the industry proposes that the user is prevented from accessing unhealthy or malicious content which is irrelevant to work by using the internet surfing behavior audit, so that the internet surfing behavior of the user is more transparent, and the method has great benefits for standardizing the network. However, the auditing of the internet behavior depends on logs generated by the user behavior, and in a network scene with a large number of users, the auditing logs need to be stored in different third-party servers, and the format requirements of the logs by different servers are different.
In order to send various internet access behavior audit logs to different log servers according to format requirements of different log servers, the logs are formatted in the prior art by adopting the following two ways:
1) A manner of hard coding;
and formatting the log according to the format requirement of the log server by adopting a code writing mode, and then sending the log to the log server. Although the processing performance of the technology is high, the code needs to be revised each time different log servers are connected, and the code needs to be revised if the format requirements of the log servers change at a later stage, so that the efficiency of log formatting is extremely low.
2) Adopting a script mode;
and formatting the log according to the format requirement of the log server by adopting a script mode, and then sending the log to the log server. Due to the adoption of the script mode, the method can be flexibly customized according to the requirements of the server, and only the script needs to be released again each time, the code does not need to be revised, and the human resources can be saved. However, the script belongs to an explanatory language, the running efficiency is low, and if the audit log quantity is huge, the scheme is not applicable.
In summary, how to efficiently format the log according to different format requirements of the log server is an urgent problem to be solved.
Disclosure of Invention
The embodiment of the application provides a log formatting method, which is used for solving the problems that in the prior art, the efficiency of formatting logs according to format requirements of a plurality of log servers is low, and resources are wasted.
The technical scheme of the application is as follows:
in a first aspect, an embodiment of the present application provides a log formatting method, including:
determining a first journal, the first journal comprising journal content of at least one parameter type;
acquiring N log templates from pre-stored log templates, wherein N is the total number of the log templates corresponding to the log server requiring formatting, and is more than or equal to 1;
storing a format string and a parameter type list corresponding to each log template in the N log templates into an operating memory, wherein the format string is used for formatting the log, and at least one parameter type in the parameter type list is arranged according to a first sequence;
determining N log content stacks corresponding to the N log templates one by one according to the log contents, and storing the N log content stacks into the running memory, wherein the log contents in each log content stack are arranged according to a second sequence according to the corresponding parameter type, and the second sequence is opposite to the first sequence;
and formatting each log content stack according to the format string in the log template corresponding to each log content stack.
Optionally, the determining, according to the log content, N log content stacks corresponding to the N log templates one to one includes:
establishing the N log content stacks, wherein each log content stack comprises at least one storage space with the same number of the parameter types of the corresponding log template, and the at least one storage space is arranged according to the second sequence according to the corresponding parameter types;
and storing the log content in the first log into a corresponding storage space in each log content stack according to the parameter type corresponding to each log template.
Optionally, the storing the format string and the parameter type list corresponding to each log template in the N log templates into an operating memory includes:
judging whether each log template meets a preset condition, wherein the preset condition is that the number of format symbols in a format string corresponding to each log template is equal to the number of parameter types of log content in the first log corresponding to the format string, the data types of the parameter types in a parameter type list corresponding to each log template correspond to the data types of format symbols in the format string corresponding to each log template one by one, and one format symbol is used for formatting the log content corresponding to one parameter type;
and if each log template meets the preset condition, storing the format string and the parameter type list corresponding to each log template in the N log templates into an operating memory.
Optionally, the method further includes:
if the log templates which do not accord with the preset conditions exist in the N log templates, upgrading the log templates which do not accord with the preset conditions according to a preset upgrading file.
Optionally, the formatting each log content stack according to the format string in the log template corresponding to each log content stack includes:
formatting the log content stacks according to format characters in format strings in the log templates corresponding to the log content stacks to obtain second logs, so that the data formats of the log contents in the second logs and the format characters in the format strings corresponding to the log templates are the same; the data format meets the data format requirement of the log server corresponding to each log template;
adding an operator in each log template, wherein the operator is used for performing a symbol operation on the log content of the second log, so that the symbol format of the second log after the symbol operation meets the symbol format requirement of the log server corresponding to each log template.
In a second aspect, an embodiment of the present application provides a log formatting device, including:
a processing module to determine a first journal, the first journal comprising journal content of at least one parameter type;
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring N log templates from pre-stored log templates, N is the total number of the log templates corresponding to a log server requiring formatting, and N is more than or equal to 1;
the processing module is further configured to store a format string and a parameter type list corresponding to each log template in the N log templates into an operating memory, where the format string is used to format the log, and at least one parameter type in the parameter type list is arranged according to a first order;
the processing module is further configured to determine, according to the log content, N log content stacks corresponding to the N log templates one to one and store the N log content stacks in the operating memory, where the log content in each log content stack is arranged according to a second sequence according to a corresponding parameter type, and the second sequence is opposite to the first sequence;
the processing module is further configured to format each log content stack according to a format string in the log template corresponding to each log content stack.
Optionally, the processing module is configured to determine, according to the log content, N log content stacks corresponding to the N log templates one to one, and specifically configured to:
establishing the N log content stacks, wherein each log content stack comprises at least one storage space with the same number of the parameter types of the corresponding log template, and the at least one storage space is arranged according to the second sequence according to the corresponding parameter types;
and storing the log content in the first log into a corresponding storage space in each log content stack according to the parameter type corresponding to each log template.
Optionally, when the processing module is configured to store the format string and the parameter type list corresponding to each log template in the N log templates in the operating memory, the processing module is specifically configured to:
judging whether each log template meets a preset condition, wherein the preset condition is that the number of format symbols in a format string corresponding to each log template is equal to the number of parameter types of log content in the first log corresponding to the format string, the data types of the parameter types in a parameter type list corresponding to each log template correspond to the data types of format symbols in the format string corresponding to each log template one by one, and one format symbol is used for formatting the log content corresponding to one parameter type;
and if each log template meets the preset condition, storing the format string and the parameter type list corresponding to each log template in the N log templates into an operating memory.
Optionally, the processing module is further configured to: and when the log templates which do not accord with the preset conditions exist in the N log templates, upgrading the log templates which do not accord with the preset conditions according to a preset upgrading file.
Optionally, the processing module is configured to format each log content stack according to a format string in the log template corresponding to each log content stack, and specifically configured to:
formatting the log content stacks according to the format symbols in the format strings in the log templates corresponding to the log content stacks to obtain second logs, so that the data formats of the log contents in the second logs and the format symbols in the format strings corresponding to the log templates are the same; the data format meets the data format requirement of the log server corresponding to each log template;
adding an operational character into each log template, wherein the operational character is used for performing symbol operation on the log content of the second log, so that the symbol format of the second log after the symbol operation meets the symbol format requirement of the log server corresponding to each log template.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method when executing the program.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-mentioned method.
The application provides a log formatting method, which comprises the following steps: determining a first journal, the first journal comprising journal content of at least one parameter type; acquiring N log templates from pre-stored log templates, wherein N is the total number of the log templates corresponding to the log server requiring formatting, and is more than or equal to 1; storing a format string and a parameter type list corresponding to each log template in the N log templates into an operating memory, wherein the format string is used for formatting the log, and at least one parameter type in the parameter type list is arranged according to a first sequence; determining N log content stacks corresponding to the N log templates one by one according to the log contents, and storing the N log content stacks into the running memory, wherein the log contents in each log content stack are arranged according to a second sequence according to the corresponding parameter type, and the second sequence is opposite to the first sequence; and formatting each log content stack according to the format string in the log template corresponding to each log content stack. By the method, the log can be formatted according to format requirements of different log servers, and the format of the log is changed in the later period, so that only the log template is required to be modified, codes are not required to be modified, and resources are saved.
Drawings
Fig. 1 is a schematic flowchart of a log formatting method according to an embodiment of the present application;
fig. 2 is a schematic diagram of a storage chain table obtained by analyzing and storing a plurality of log templates before parameter classification according to an embodiment of the present application;
fig. 3 is a schematic diagram illustrating a storage chain table obtained by analyzing and storing a plurality of log templates after parameter classification according to the embodiment of the present application;
FIG. 4 is a schematic structural diagram of a log formatting apparatus according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The background of the present application is briefly described below.
At present, in order to maintain a harmonious network environment, the industry proposes that the user is prevented from accessing unhealthy or malicious contents which are irrelevant to work by using internet surfing behavior audit, so that the internet surfing behavior of the user is more transparent, and the method has great benefits for standardizing the network. However, the auditing of the internet behavior depends on logs generated by the user behavior, and in a network scene with a large number of users, the auditing logs need to be stored in different third-party servers, and the format requirements of the logs by different servers are different.
In order to send various online behavior audit logs to different log servers according to format requirements of different log servers, in the prior art, logs are formatted in a hard coding mode and a script mode, however, in the hard coding mode, once the log format is revised at a later stage, codes need to be changed, and resources are wasted; in the script mode, the log formatting efficiency is low due to low operation efficiency.
Therefore, how to efficiently realize the rapid formatting of the log according to the format requirements of different log servers is an urgent problem to be solved.
In order to solve the above technical problem, the present application provides a log formatting method, including: determining a first journal, the first journal comprising journal content of at least one parameter type; acquiring N log templates from pre-stored log templates, wherein N is the total number of the log templates corresponding to the log server requiring formatting, and is more than or equal to 1; storing a format string and a parameter type list corresponding to each log template in the N log templates into an operating memory, wherein the format string is used for formatting logs, and at least one parameter type in the parameter type list is arranged according to a first sequence; determining N log content stacks corresponding to the N log templates one by one according to the log contents, and storing the N log content stacks into the running memory, wherein the log contents in each log content stack are arranged according to a second sequence according to the corresponding parameter type, and the second sequence is opposite to the first sequence; and formatting each log content stack according to the format string in the log template corresponding to each log content stack. By the method, the log can be rapidly formatted according to format requirements of different log servers, and the format of the log is changed in the later period, so that only the log template is required to be modified, codes are not required to be modified, and resources are saved.
The following describes a specific implementation process of the technical solution provided in the embodiment of the present application.
Referring to fig. 1, fig. 1 is a schematic flow chart of a log formatting method according to an embodiment of the present application, where the method includes:
101: a first log is determined, the first log including log content of at least one parameter type.
For example, taking a switch as an example, the switch stores a plurality of logs in advance, and the switch selects and formats a specific log a from the plurality of logs in the storage space of the switch, where the log a includes a network Protocol (IP) address, a Mac address, a url address, and a timestamp (timestamp).
102: and acquiring N log templates from the pre-stored log templates, wherein N is the total number of the log templates corresponding to the log server requiring formatting, and is more than or equal to 1.
It should be understood that "acquire" herein refers to the electronic device acquiring from its local storage space.
Illustratively, taking a switch as an example, 100 log templates corresponding to 50 log servers are pre-stored in the switch, and 20 log templates are selected from the 100 log templates according to the requirements of the log servers to format the log a.
103: and storing a format string and a parameter type list corresponding to each log template in the N log templates into an operating memory, wherein the format string is used for formatting the log, and at least one parameter type in the parameter type list is arranged according to a first sequence.
It should be understood that, after the format string and the parameter type list corresponding to each log template in the N log templates are stored in the operating memory, a storage linked list is obtained, please refer to fig. 2, fig. 2 is a schematic diagram of the storage linked list obtained by analyzing and storing a plurality of log templates before parameter classification provided by the embodiment of the present application, the storage linked list shown in fig. 2 is obtained after the N log templates are analyzed and stored in the operating memory space, and the storage form of the log templates of two log servers corresponding to the log is shown in the figure by taking http logs as an example.
In a possible implementation manner, assuming that an electronic device that executes the log formatting method provided in the embodiment of the present application is a switch, before storing a format string and a parameter type list corresponding to each log template in N log templates into an operating memory, an original parameter list exists in each log template, where the parameter list includes a plurality of parameters, and the parameters need to be mapped into corresponding parameter types to obtain a parameter type list, and specific mapping manners include the following two manners:
mode 1: and the switch maps the parameters in the parameter list one by one to obtain the corresponding parameter types. For example, the parameter lists [ http _ ip, http _ mac, http _ url, http _ timestamp ] are mapped to corresponding types [ http _ ip _ type, http _ mac _ type, http _ url _ type, http _ timestamp _ type ], respectively.
Mode 2: the switch classifies the parameters according to the data types of the parameters, and each data type corresponds to a type. For example, integers are classified into one class (stored in an integer array, the corresponding type is int), and string type variables are classified into one class (stored in a string array, the corresponding type is string). Referring to fig. 3, fig. 3 is a schematic diagram illustrating a storage linked list obtained by parsing and storing a plurality of log templates after parameter classification according to an embodiment of the present disclosure.
It should be understood that, in the method 2, the parameter types are fewer, and the number of times of traversing the lookup type by the electronic device is reduced in the process of stacking the log content by the subsequent switch according to the parameter types, that is, the stacking efficiency is improved.
In a possible implementation manner, the storing, by the electronic device, the format string and the parameter type list corresponding to each log template in the N log templates in an operating memory includes: the electronic equipment judges whether each log template meets a preset condition, wherein the preset condition is that the number of format symbols in a format string corresponding to each log template is equal to the number of parameters in a log content parameter list in the first log corresponding to the format string, the data types of the parameter types in the parameter type list corresponding to each log template correspond to the data types of format symbols in the format string corresponding to each log template one by one, and one format symbol is used for formatting the log content corresponding to one parameter type; and if each log template meets the preset condition, the electronic equipment stores the format string and the parameter type list corresponding to each log template in the N log templates into an operating memory. If the log templates which do not accord with the preset conditions exist in the N log templates, upgrading the log templates which do not accord with the preset conditions according to a preset upgrading file.
It should be understood that the preset upgrade file may be written in advance by a technician and stored in a storage space of the electronic device, or may be obtained online through a network, and the embodiment of the present application is not particularly limited.
For example, assume that there are 4 parameters in the original parameter list in the log template a and there are 4 format characters in the format string; the switch then checks whether the parameter type in the original parameter list is consistent with the format string format, for example, the format string of the http log preset in the log template a is { ip =% s, mac =% s, url =% s, time =% u }, where 1-3 format symbols are% s, and the corresponding 1-3 parameters are of a string type; the 4 th format is% u, and the corresponding 4 th parameter is a decimal unsigned integer, then the log template a meets the requirements.
104: and determining N log content stacks corresponding to the N log templates one by one according to the log contents, and storing the N log content stacks into the operating memory, wherein the log contents in each log content stack are arranged according to a second sequence according to the corresponding parameter type, and the second sequence is opposite to the first sequence.
It should be understood that in order to correspond the log contents to the parameter variables in the log template one to one, the log contents also need to be stored in the operating memory space in a classified manner according to the parameter types.
Illustratively, taking an http log as an example, the switch classifies the content of the http log according to a preset log content type, stores an ip address in the http log into an http _ ip _ type variable, stores a mac address into an http _ mac _ type variable, stores url into an http _ url _ type variable, and stores a timestamp into an http _ timestamp _ type variable.
In a possible implementation manner, the determining, by the electronic device, N log content stacks corresponding to the N log templates one to one according to the log content includes: establishing the N log content stacks, wherein each log content stack comprises at least one storage space with the same number of the parameter types of the corresponding log template, and the at least one storage space is arranged according to the second sequence according to the corresponding parameter types; and storing the log content in the first log into a corresponding storage space in each log content stack according to the parameter type corresponding to each log template.
For example, taking the http log above as an example, assuming that the arrangement order of the parameter types in the parameter type list of the log template a is as shown in table 1, and the order is from bottom to top, the switch sequentially pushes the log contents corresponding to the parameter types in table 1 in the order from top to bottom in table 1, so as to obtain the log content stack as shown in table 2.
| 4、http_timestamp_type |
| 3、http_url_type |
| 2、http_mac_type |
| 1、http_ip_type |
Table 1: arranging sequence of parameter types in parameter type list of log template A
| 1、http_ip_type：218.88.122.180 |
| 2、http_mac_type：00-11-22-33-44-55 |
| 3、http_url_type：https://www.baidu.com |
| 4、http_timestamp_type：2018/06/19_08:34:59_313317 |
Table 2 log content stack
105: and formatting each log content stack according to the format string in the log template corresponding to each log content stack.
It should be understood that the "format string" herein refers to a format string in the log template after storing the N log templates in the operating memory space, where the log may be formatted simultaneously according to the format strings of multiple log templates, or the log may be formatted by selecting a format string of one log template, and the embodiment of the present application is not limited specifically.
In a possible implementation manner, formatting, at the electronic device, each log content stack according to a format string in a log template corresponding to the log content stack includes: the electronic equipment formats the log content stacks according to the format symbols in the format strings in the log templates corresponding to the log content stacks to obtain second logs, so that the data formats of the log contents in the second logs and the format symbols in the format strings corresponding to the log templates are the same; the data format meets the data format requirement of the log server corresponding to each log template; adding an operator in each log template, wherein the operator is used for performing a symbol operation on the log content of the second log, so that the symbol format of the second log after the symbol operation meets the symbol format requirement of the log server corresponding to each log template.
It should be understood that the symbolic operation may be an operation of adding or deleting a connection symbol to or from a symbol in the log content, and the specific implementation may be that a function index is carried in the operator, and the electronic device calls a corresponding function according to the function index to perform symbol conversion on the log content, which is not limited in this embodiment of the application.
It should be understood that in the log formatting method provided by the present application, the operator may carry one or more function indexes, that is, the electronic device may implement formatting all contents of the log content stack once according to the operator, and may also perform format conversion on each content of the log content stack one by one according to the function index carried by the operator, and the embodiments of the present application are not limited in particular.
For example, in the process of formatting the log content stack shown in table 2 according to the format string, taking the mac address in the log content stack as an example, it is assumed that the mac address in the log after formatting is required by the log server a is http _ mac _ type: formal output of aabb.ccdd.eeff; the server can only obtain http _ mac _ type after formatting the http log according to the format string: 00-11-22-33-44-55; at this time, the index carried by the operator in the log template a is used for locating a corresponding format _ mac (http _ mac, n) function, and the http _ mac _ type is processed according to the format _ mac (http _ mac, n) function: 00-11-22-33-44-55 into http _ mac _ type:0011.2233.4455.
in a possible implementation manner, after the electronic device obtains the logs meeting the data format requirement and the symbol format requirement of the log server, the formatted logs need to be output to the corresponding log servers.
Illustratively, in an implementation manner similar to a printf function, the switch outputs the formatted log according to the format string in the log template corresponding to the current log server and the log content stack corresponding to the format string. Taking the http log as an example, a printf function is called as follows: printf ("ip =% s, mac =% s, url =% s, time =% u", http _ ip _ type, http _ mac _ type, http _ url _ type, http _ timestamp _ type); the formatted log output is as follows:
“
ip＝218.88.122.1803
mac＝00-11-22-33-44-55
url＝https://www.baidu.com
timestamp＝2018/06/19_08:34:59_313317
”
the following description explains the technical solution of the present application in detail with reference to specific embodiments.
Example 1
The electronic device takes a switch as an example, and the switch selects an http log from a plurality of logs in a storage space of the switch to format the http log so as to output the http log to different log servers. The http log includes an Internet Protocol (IP) address, a mac address, a url address, and a timestamp.
The inside of the switch also stores 100 log templates corresponding to 50 log servers in advance, and selects a log template A from the 100 log templates to format the http log. Firstly, the switch maps parameter lists [ http _ ip, http _ mac, http _ url, http _ timestamp ] of the log template A into corresponding types [ http _ ip _ type, http _ mac _ type, http _ url _ type, http _ timestamp _ type ], respectively ].
The switch judges the validity of the log template A: 4 parameters are in an original parameter list in the log template A, and 4 format characters are in a format string in the log template A, which indicates that the number of the parameters is equal to the number of the format characters in the format string; the switch maps the parameters of the log template A one by one to obtain a parameter type list, and further judges whether each parameter type in the parameter type list is consistent with the format character of the format string in the log template A corresponding to the parameter type list; specifically, format strings of the http log in the log template a are { ip =% s, mac =% s, url =% s, and time =% u }, where 1-3 format symbols are% s, 1-3 parameters in the corresponding parameter type list are character string types, 4-th format symbols are% u, and 4-th parameters in the corresponding parameter type list are decimal unsigned integers, so that the log template a meets the preset condition of the log server.
After the validity check, the switch maps the parameters in the original parameter list of the log template a one by one to obtain a parameter type list and stores the format strings in the log template a in the operating memory space to obtain the storage linked list shown in fig. 2.
And the exchanger stores the content of the http log into the running memory. Specifically, the content of the http log is classified according to the preset log content type, an ip address in the http log is stored into an http _ ip _ type variable, a mac address is stored into an http _ mac _ type variable, a url is stored into an http _ url _ type variable, and a timestamp is stored into an http _ timestamp _ type variable.
The switch determines a log content stack according to the parameter type list of the log template A and the content of the http log, the arrangement sequence of the parameter types in the parameter type list is as shown in Table 1, the log content corresponding to the parameter types in Table 1 is sequentially stacked from top to bottom in the sequence of Table 1, and the log content stack as shown in Table 2 is obtained.
In the process of formatting the log content stack shown in table 2 by the switch according to the format string in the log template a, an operator may also be added in the log template a. Taking the Mac address in the log content stack as an example, since the Mac address in the log after the log server a requires formatting is http _ Mac _ type: formal output of aabb.ccdd.eeff; the server can only obtain http _ mac _ type after formatting the http log according to the format string: 00-11-22-33-44-55; at this time, a function index carried by an operator in the log template a is used for positioning to a corresponding format _ mac (http _ mac, n) function, and the http _ mac _ type is converted according to the format _ mac (http _ mac, n) function: 00-11-22-33-44-55 into http _ mac _ type:0011.2233.4455. and after each content in the log content is formatted one by one in the same way, the log output formatted like printf is adopted.
Example 2
The electronic device takes a switch as an example, and the server acquires a http log from the log interceptor from a plurality of logs in a storage space of the switch to format the http log so as to output the http log to different log servers. The http log includes an Internet Protocol (IP) address, a mac address, a url address, and a timestamp.
100 log templates corresponding to 50 log servers are also stored in the switch in advance, and an example of formatting the http log by using the log template A is selected from the 100 log templates. The method comprises the following steps that a switch divides a parameter list [ http _ ip, http _ mac, http _ url and http _ timestamp ] of a log template A into a character type and an integer type according to the data type of the parameter in the parameter list; the switch classifies parameters of integer type into one class (stored in an integer array, the mapping type is int) and parameters of character string type into one class (stored in a character string array, the mapping type is string). Namely, the parameter type list obtained by mapping after the parameters in the log template A are classified is [ (string, 1), (string, 2), (string, 3), (int, 1) ].
The switch judges the validity of the log template A: the original parameter list in the log template A has 4 parameter numbers, and the format string in the log template A has 4 format characters; the switch then checks whether the parameter type of the log template A is consistent with the format character of the format string in the corresponding log template A; the format string of the http log in the log template a is { ip =% s, mac =% s, url =% s, time =% u }, wherein 1-3 formats are% s, the 1 st parameter to the 3 rd parameter in the corresponding log template a are of a character string type, the 4 th format is% u, and the 4 th parameter in the corresponding log template a is a decimal unsigned integer, so that the log template a meets the requirement.
After the validity check, the switch classifies the parameters in the log template a, and then maps the classified parameters to obtain a parameter type list of [ (string, 1), (string, 2), (string, 3), (int, 1) ] and a format string in the log template a to be stored in the running memory space, so as to obtain a storage linked list schematic diagram shown in fig. 3.
And the exchanger stores the content of the http log into the running memory. Specifically, the switch classifies the content of the http log according to the preset log content type, stores an ip address in the http log into an http _ ip _ type variable, stores a mac address into an http _ mac _ type variable, stores url into an http _ url _ type variable, and stores a timestamp into an http _ timestamp _ type variable.
| string1：218.88.122.180 |
| string2：00-11-22-33-44-55 |
| string3：https://www.baidu.com |
| int1：2018/06/19_08:34:59_313317 |
Table 3: the switch obtains a log content stack according to the parameter list
The switch sequentially pushes the log content corresponding to each parameter in the http log according to the parameter lists [ (string, 1), (string, 2), (string, 3) and (int, 1) ] to obtain a log content stack as shown in table 3.
In the process of formatting the log content stack shown in table 3 by the switch according to the format string in the log template a, an operator may also be added in the log template a. Taking the mac address in the log content stack as an example, the mac address in the log after the log server a requires formatting is represented by http _ mac _ type: formal output of aabb.ccdd.eeff; the server can only obtain http _ mac _ type after formatting the http log according to the format string: 00-11-22-33-44-55; at this time, the index carried by the operator in the log template a is used for positioning to a corresponding format _ mac (http _ mac, n) function, and the http _ mac _ type is converted into the http _ mac _ type according to the format _ mac (http _ mac, n) function: 00-11-22-33-44-55 into http _ mac _ type:0011.2233.4455. after each content in the log content is formatted one by one in the same way, the formatted log is output to the log server corresponding to the requirement in a printf-like way.
Based on the same inventive concept, the embodiment of the present application further provides a log formatting device, which may be, for example, the server in the foregoing embodiment, or may be a hardware structure, a software module, or a hardware structure plus a software module. Referring to fig. 4, fig. 4 is a schematic diagram of a path detection apparatus according to an embodiment of the present application, where the apparatus includes:
a processing module 401 configured to determine a first log, the first log comprising log content of at least one parameter type;
an obtaining module 402, configured to obtain N log templates from pre-stored log templates, where N is a total number of log templates corresponding to a log server requiring formatting, and N is greater than or equal to 1;
the processing module 401 is further configured to store a format string and a parameter type list corresponding to each log template in the N log templates into an operating memory, where the format string is used to format a log, and at least one parameter type in the parameter type list is arranged according to a first order;
the processing module 401 is further configured to determine, according to the log content, N log content stacks corresponding to the N log templates one to one, and store the N log content stacks in the operating memory, where the log content in each log content stack is arranged according to a second sequence according to a corresponding parameter type, and the second sequence is opposite to the first sequence;
the processing module 401 is further configured to format each log content stack according to a format string in the log template corresponding to each log content stack.
Optionally, the processing module 401 is configured to determine, according to the log content, N log content stacks corresponding to the N log templates one to one, and specifically configured to:
establishing the N log content stacks, wherein each log content stack comprises at least one storage space with the same number of the parameter types of the corresponding log template, and the at least one storage space is arranged according to the second sequence according to the corresponding parameter types;
and storing the log content in the first log into a corresponding storage space in each log content stack according to the parameter type corresponding to each log template.
Optionally, when the processing module 401 is configured to store the format string and the parameter type list corresponding to each log template in the N log templates in the running memory, specifically, the processing module is configured to:
judging whether each log template meets preset conditions or not, wherein the preset conditions are that the number of format symbols in a format string corresponding to each log template is equal to the number of parameter types of log contents in the first log corresponding to each log template, the data types of the parameter types in a parameter type list corresponding to each log template correspond to the data types of format symbols in the format string corresponding to each log template one by one, and one format symbol is used for formatting the log contents corresponding to one parameter type;
and if each log template meets the preset condition, storing the format string and the parameter type list corresponding to each log template in the N log templates into an operating memory.
Optionally, the processing module 401 is further configured to: and when the log templates which do not accord with the preset conditions exist in the N log templates, upgrading the log templates which do not accord with the preset conditions according to a preset upgrading file.
Optionally, the processing module 401 is configured to format each log content stack according to a format string in the log template corresponding to each log content stack, and specifically configured to:
formatting the log content stacks according to the format symbols in the format strings in the log templates corresponding to the log content stacks to obtain second logs, so that the data formats of the log contents in the second logs and the format symbols in the format strings corresponding to the log templates are the same; the data format meets the data format requirement of the log server corresponding to each log template;
adding an operator in each log template, wherein the operator is used for performing a symbol operation on the log content of the second log, so that the symbol format of the second log after the symbol operation meets the symbol format requirement of the log server corresponding to each log template.
Based on the above embodiments, refer to fig. 5, which is a schematic structural diagram of an electronic device in an embodiment of the present application.
Based on the same inventive concept, an electronic device (e.g., the switch described above) provided with a log formatting function is provided in an embodiment of the present application, please refer to fig. 5, where fig. 5 is a schematic structural diagram of the electronic device provided in the embodiment of the present application. The electronic device provided with the log formatting function includes at least one processor 502 and a memory 501 connected to the at least one processor, in this embodiment of the present application, a specific connection medium between the processor 502 and the memory 501 is not limited, fig. 5 illustrates that the processor 502 and the memory 501 are connected through a bus 500, the bus 500 is represented by a thick line in fig. 5, and connection manners between other components are only schematically illustrated and not limited thereto. The bus 500 may be divided into an address bus, a data bus, a control bus, etc., and is shown with only one thick line in fig. 5 for ease of illustration, but does not represent only one bus or one type of bus.
In the embodiment of the present application, the memory 501 stores instructions executable by the at least one processor 502, and the at least one processor 502 may perform the steps included in the aforementioned log formatting method by calling the instructions stored in the memory 501.
The processor 502 is a control center of the electronic device provided with the log formatting function, and can connect various parts of the whole electronic device provided with the log formatting function by using various interfaces and lines, and implement various functions of the electronic device provided with the log formatting function by executing instructions stored in the memory 501. Optionally, the processor 502 may include one or more processing units, and the processor 502 may integrate an application processor and a modem processor, wherein the application processor mainly handles operating systems, user interfaces, application programs, and the like, and the modem processor mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 502. In some embodiments, the processor 502 and the memory 501 may be implemented on the same chip, or in some embodiments, they may be implemented separately on separate chips.
The memory 501, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The Memory 501 may include at least one type of storage medium, which may include, for example, a flash Memory, a hard disk, a multimedia card, a card-type Memory, a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Programmable Read Only Memory (PROM), a Read Only Memory (ROM), a charged Erasable Programmable Read Only Memory (EEPROM), a magnetic Memory, a magnetic disk, an optical disk, and the like. The memory 501 is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to such. The memory 501 in the embodiments of the present application may also be a circuit or any other device capable of implementing a storage function for storing program instructions and/or data.
The processor 502 may be a general-purpose processor, such as a Central Processing Unit (CPU), digital signal processor, application specific integrated circuit, field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, that implements or performs the methods, steps, and logic blocks disclosed in embodiments of the present application. A general purpose processor may be a microprocessor or any conventional processor or the like. The steps of the log formatting method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.
By programming the processor 502, the code corresponding to the log formatting method described in the foregoing embodiment may be solidified into a chip, so that the chip can execute the steps of the log formatting method when running.
Based on the above embodiments, in the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program, when executed by a processor, implements the log formatting method in any of the above method embodiments.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (8)
1. A method for log formatting, the method comprising:
determining a first journal, the first journal comprising journal content of at least one parameter type;
acquiring N log templates from pre-stored log templates, wherein N is the total number of the log templates corresponding to the log server requiring formatting, and is more than or equal to 1;
storing a format string and a parameter type list corresponding to each log template in the N log templates into an operating memory, wherein the format string is used for formatting logs, and at least one parameter type in the parameter type list is arranged according to a first sequence;
determining N log content stacks corresponding to the N log templates one by one according to the log content and storing the log content stacks in the running memory, wherein the method comprises the following steps: establishing N log content stacks, wherein each log content stack comprises at least one storage space with the same number of parameter types of the corresponding log template, and the at least one storage space is arranged according to the corresponding parameter types according to a second sequence, wherein the second sequence is opposite to the first sequence; storing the log content in the first log into a corresponding storage space in each log content stack according to the parameter type corresponding to each log template;
formatting each log content stack according to a format string in a log template corresponding to each log content stack; the method comprises the following steps:
formatting the log content stacks according to the format symbols in the format strings in the log templates corresponding to the log content stacks to obtain second logs, so that the data formats of the log contents in the second logs and the format symbols in the format strings corresponding to the log templates are the same; the data format meets the data format requirement of the log server corresponding to each log template;
adding an operator in each log template, wherein the operator is used for performing a symbol operation on the log content of the second log, so that the symbol format of the second log after the symbol operation meets the symbol format requirement of the log server corresponding to each log template.
2. The method of claim 1, wherein storing the format string and the parameter type list corresponding to each of the N log templates in a run-time memory comprises:
judging whether each log template meets preset conditions or not, wherein the preset conditions are that the number of format symbols in a format string corresponding to each log template is equal to the number of parameter types of log contents in the first log corresponding to each log template, the data types of the parameter types in a parameter type list corresponding to each log template correspond to the data types of format symbols in the format string corresponding to each log template one by one, and one format symbol is used for formatting the log contents corresponding to one parameter type;
and if each log template meets the preset condition, storing the format string and the parameter type list corresponding to each log template in the N log templates into an operating memory.
3. The method of claim 2, wherein the method further comprises:
if the log templates which do not accord with the preset conditions exist in the N log templates, upgrading the log templates which do not accord with the preset conditions according to a preset upgrading file.
4. An apparatus for formatting logs, comprising:
a processing module to determine a first journal, the first journal comprising journal content of at least one parameter type;
the device comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring N log templates from pre-stored log templates, N is the total number of the log templates corresponding to a log server requiring formatting, and N is more than or equal to 1;
the processing module is further configured to store a format string and a parameter type list corresponding to each log template in the N log templates into an operating memory, where the format string is used to format the log, and at least one parameter type in the parameter type list is arranged according to a first order;
the processing module is further configured to determine, according to the log content, N log content stacks corresponding to the N log templates one to one and store the log content stacks in the operating memory; the processing module is configured to establish the N log content stacks, where each log content stack includes at least one storage space with the same number of parameter types as that of a corresponding log template, and the at least one storage space is arranged according to a second order according to the corresponding parameter types, where the second order is opposite to the first order; storing the log content in the first log into a corresponding storage space in each log content stack according to the parameter type corresponding to each log template;
the processing module is further configured to format each log content stack according to a format string in the log template corresponding to each log content stack; when formatting each log content stack, the method includes:
formatting the log content stacks according to the format symbols in the format strings in the log templates corresponding to the log content stacks to obtain second logs, so that the data formats of the log contents in the second logs and the format symbols in the format strings corresponding to the log templates are the same; the data format meets the data format requirement of the log server corresponding to each log template;
adding an operational character into each log template, wherein the operational character is used for performing symbol operation on the log content of the second log, so that the symbol format of the second log after the symbol operation meets the symbol format requirement of the log server corresponding to each log template.
5. The apparatus according to claim 4, wherein the processing module, when configured to store the format string and the parameter type list corresponding to each log template in the N log templates in the operating memory, is specifically configured to:
judging whether each log template meets a preset condition, wherein the preset condition is that the number of format symbols in a format string corresponding to each log template is equal to the number of parameter types of log content in the first log corresponding to the format string, the data types of the parameter types in a parameter type list corresponding to each log template correspond to the data types of format symbols in the format string corresponding to each log template one by one, and one format symbol is used for formatting the log content corresponding to one parameter type;
and if each log template meets the preset condition, storing the format string and the parameter type list corresponding to each log template in the N log templates into an operating memory.
6. The apparatus of claim 5, wherein the processing module is further to:
and when the log templates which do not accord with the preset conditions exist in the N log templates, upgrading the log templates which do not accord with the preset conditions according to a preset upgrading file.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method according to any of claims 1-3 are implemented when the processor executes the program.
8. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when being executed by a processor realizes the steps of the method of any one of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911329806.4A CN111144086B (en) | 2019-12-20 | 2019-12-20 | Log formatting method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911329806.4A CN111144086B (en) | 2019-12-20 | 2019-12-20 | Log formatting method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111144086A CN111144086A (en) | 2020-05-12 |
| CN111144086B true CN111144086B (en) | 2023-03-21 |
Family
ID=70519208
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911329806.4A Active CN111144086B (en) | 2019-12-20 | 2019-12-20 | Log formatting method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111144086B (en) |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8533193B2 (en) * | 2010-11-17 | 2013-09-10 | Hewlett-Packard Development Company, L.P. | Managing log entries |
| CN104144071B (en) * | 2013-05-10 | 2018-02-06 | 北京新媒传信科技有限公司 | The processing platform of the processing method and system daily record of system journal |
| CN105550378A (en) * | 2016-02-04 | 2016-05-04 | 贵州电网有限责任公司信息中心 | Extraction and analysis method for heterogeneous security log information under complex network system |
| CN107015948B (en) * | 2017-03-16 | 2021-05-28 | 武汉斗鱼网络科技有限公司 | Log information formatting method and system |
| CN109800223A (en) * | 2018-12-12 | 2019-05-24 | 平安科技（深圳）有限公司 | Log processing method, device, electronic equipment and storage medium |
-
2019
- 2019-12-20 CN CN201911329806.4A patent/CN111144086B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111144086A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111240688B (en) | excel file analysis method and device, computer equipment and storage medium | |
| CN110474900B (en) | Game protocol testing method and device | |
| CN110928802A (en) | Test method, device, equipment and storage medium based on automatic generation of case | |
| CN114238948A (en) | Application program detection method and device, electronic equipment and storage medium | |
| US20200134029A1 (en) | Compression of Log Data Using Pattern Recognition | |
| US10599755B2 (en) | System and method for automatically inserting correct escaping functions for field references in a multi-tenant computing environment | |
| CN112817935A (en) | Data processing method, device and equipment based on field type and storage medium | |
| CN112181430A (en) | Code change statistical method and device, electronic equipment and storage medium | |
| CN113360300B (en) | Interface call link generation method, device, equipment and readable storage medium | |
| CN111144086B (en) | Log formatting method and device, electronic equipment and storage medium | |
| CN110442353B (en) | Installation package management method and device and electronic equipment | |
| CN117008920A (en) | Engine system, request processing method and device, computer equipment and storage medium | |
| US10740218B2 (en) | Method and device for determining usage log | |
| CN116010499A (en) | Method and device for determining analysis rule and electronic equipment | |
| CN115757174A (en) | Database difference detection method and device | |
| US20190317877A1 (en) | Application state monitoring | |
| CN113342647A (en) | Test data generation method and device | |
| CN112287643B (en) | Message monitoring method, device, equipment and computer readable storage medium | |
| CN109783134B (en) | Front-end page configuration method and device and electronic equipment | |
| CN112380541A (en) | Method and device for detecting vulnerability of intelligent contract and computer equipment | |
| CN111580799A (en) | Domain specific language script assembling method and system | |
| CN115705294B (en) | Method, device, electronic equipment and medium for acquiring function call information | |
| CN116661758B (en) | Method, device, electronic equipment and medium for optimizing log framework configuration | |
| CN113419775B (en) | Firmware configuration method, system, equipment and medium | |
| KR101921123B1 (en) | Field-Indexing Method for Message |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |