CN111639059A - Log information storage and positioning method, electronic equipment and storage medium - Google Patents
Log information storage and positioning method, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111639059A CN111639059A CN202010469328.3A CN202010469328A CN111639059A CN 111639059 A CN111639059 A CN 111639059A CN 202010469328 A CN202010469328 A CN 202010469328A CN 111639059 A CN111639059 A CN 111639059A
- Authority
- CN
- China
- Prior art keywords
- log information
- log
- file
- capacity
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 79
- 238000012545 processing Methods 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000011218 segmentation Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 15
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000012216 screening Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 206010033799 Paralysis Diseases 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present application relates to the field of data processing technologies, and in particular, to a method for storing and positioning log information, an electronic device, and a storage medium. The storage method of the log information comprises the following steps: acquiring log information to be stored and a global thread number corresponding to the log information; acquiring the generation time of log information and the capacity of a log file for storing the log information; performing two-dimensional cutting on the log information according to the generation time and the capacity of the log file to obtain a plurality of log information segments; and sequentially storing the plurality of log information segments into corresponding log files, wherein the log files are marked with global thread numbers and generation time corresponding to the log information. The log information storage and positioning method provided by the application realizes ordered storage of the log information and improves the positioning efficiency of the log information.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method for storing and positioning log information, an electronic device, and a storage medium.
Background
The log records and records the running data and the intermediate data in the running process of the system, a large amount of log information can be generated in the running process of the system along with the acceleration of the informatization process, and if the generated log information is not stored, the data in the running process of the system cannot be traced back, or the abnormity in the running process of the system cannot be known in time.
In the prior art, log information can be stored in a log file divided according to time or a log file divided according to file capacity by using the generation time of the log information, and the storage modes can cause the following situations, such as: the data size of the log file corresponding to a certain time period is large, or the log information corresponding to a certain time period is stored in different log files, which is not beneficial to subsequent search of the log information, i.e. the log information is searched with low efficiency due to disorder of the storage mode of the log information.
Disclosure of Invention
The application provides a storage method of log information, electronic equipment and a storage medium, and mainly aims to store the log information in order and improve the positioning efficiency of the log information.
The embodiment of the application firstly provides a method for storing log information, which comprises the following steps:
acquiring log information to be stored and a global thread number corresponding to the log information;
acquiring the generation time of log information and the capacity of a log file for storing the log information;
performing two-dimensional cutting on the log information according to the generation time and the capacity of the log file to obtain a plurality of log information segments;
and sequentially storing the plurality of log information segments into corresponding log files, wherein the log files are marked with global thread numbers and generation time corresponding to the log information.
Optionally, the step of performing two-dimensional cutting on the log information according to the generation time and the capacity of the log file to obtain a plurality of log information segments includes:
acquiring a storage period of log information, cutting the log information according to the storage period, and acquiring a plurality of log information segments divided according to the storage period;
when the data volume of any log information segment exceeds the capacity of the log file, the log information segment is divided according to the capacity of the log file, and a plurality of log information segments which do not exceed the capacity and are divided according to the storage period are obtained.
Optionally, before performing two-dimensional cutting on the log information according to the generation time and the capacity of the log file, the method further includes:
newly establishing a subclass inheriting a FileAppender parent class, and defining the capacity and the storage period of a log file in a configuration file of the subclass;
and generating a log file for storing log information according to the configuration file.
Optionally, the configuration file further includes:
inheriting the file naming rule of the FileAppender parent class so as to determine the name of the log file according to the naming rule.
Optionally, obtaining a capacity of a log file for storing the log information includes:
acquiring the data volume of the log information to be stored;
and determining the capacity of each log file according to the data volume of the log information and the system processing performance.
Optionally, before the obtaining of the log information to be stored, the method further includes:
acquiring the preset number of the log files, and determining the storable data volume of the log files according to the capacity of the log files and the preset number;
and determining the log information to be stored according to the data volume.
Further, an embodiment of the present application further provides a method for positioning log information, including:
receiving a search request for a target event, and responding to the search request to acquire the occurrence time of the target event and a corresponding global thread number;
determining at least one log file using the occurrence time; the log information is stored in a corresponding log file in advance according to the storage method of the log information in any technical scheme;
and traversing the at least one log file by using the global thread number, and determining log information corresponding to the target event.
Optionally, after determining the log information corresponding to the target event, the method further includes:
obtaining the call links of the global thread number in a plurality of systems;
and sequencing the log information according to the calling link to form a log information chain.
Further, an embodiment of the present application also provides an electronic device, where the electronic device includes: the storage device comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the storage method of the log information or the positioning method of the log information.
In addition, to achieve the above object, the present application further provides a computer-readable storage medium, which includes a storage program of log information and a positioning program, and when the storage program of log information and the positioning program are executed by a processor, the steps in the storage method of log information or the steps in the positioning method of log information are implemented.
Compared with the prior art, the scheme provided by the application at least has the following advantages:
according to the storage method of the log information, the log information is cut into the log information segments according to the generation time of the log information, the capacity of the log file and the global thread number corresponding to the log information, the log information segments are stored in the corresponding log file in order, the filing efficiency of the log information is improved, and the generation time of the log information and the corresponding global thread number are marked on the log file, so that the target log information can be rapidly positioned through the generation time of the log information and the global thread number in the following process.
According to the method for positioning the log information, when a search request for a target event is received, the log file where the log information corresponding to the target event is located is positioned, initial screening is performed by using the occurrence time of the target event, a limited number of log files are obtained, after the initial screening, only the limited number of log files need to be loaded, the data loading amount is reduced, the log files are traversed according to the global thread number corresponding to the target event, the log information corresponding to the target event is quickly positioned or obtained, and the rapid search of abnormal data for high-concurrency operation can be achieved.
Drawings
Fig. 1 is a flowchart of a method for storing log information according to an embodiment of the present application;
fig. 2 is a flowchart illustrating two-dimensional cutting of the log information according to the generation time and the capacity of the log file to obtain a plurality of log information segments according to an embodiment of the present application;
fig. 3 is a flowchart of a method for locating log information according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a storage device for log information according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a device for locating log information according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustration purposes only and are not intended to limit the scope of the present application.
It should be understood that the various steps recited in the method embodiments of the present application may be performed in a different order and/or in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present application is not limited in this respect.
The term "include" and its variants, as used herein, are inclusive, i.e., "including but not limited to"; the term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.
It is noted that references to "a", "an", and "the" modifications in this application are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that reference to "one or more" unless the context clearly dictates otherwise.
The following describes the technical solutions of the present application and how to solve the above technical problems in detail with specific embodiments. The following embodiments may be combined, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
First, an embodiment of the present application provides a method for storing log information, and fig. 1 is a flowchart of the method for storing log information provided in an embodiment of the present application, where the method may be executed by a device, the device may be implemented by software and/or hardware, and the method may be executed at a server.
S110, acquiring log information to be stored and a global thread number corresponding to the log information;
s120, acquiring the generation time of the log information and the capacity of a log file for storing the log information;
s130, performing two-dimensional cutting on the log information according to the generation time and the capacity of the log file to obtain a plurality of log information sections;
and S140, sequentially storing the plurality of log information segments into corresponding log files, wherein the log files are marked with global thread numbers and generation time corresponding to the log information.
The log information to be stored may be log information generated at a preset time period before or after the current time. The global thread number corresponding to the log information is a thread number for a background of the log information record to respond to a certain event, the event can be a search request for a target session, an acquisition request for a certain interface and the like, and the global thread number has unique identification. The global thread number can be generated at a browser end, namely a front end, one global thread number can correspond to at least one section of log information, and the global thread numbers formed by different systems aiming at the calling of the same event can form a calling chain of the thread.
Specifically, a global thread number corresponding to the log information is obtained, the global thread number in the request parameter corresponding to the target event in the web interface can be captured through debugging the web interface, and the corresponding log information is determined at the back end by using the global thread number.
The capacity of the log file, that is, the data volume of the log information that the log file can contain, is set for the log file, that is, the storage upper limit is set, so that the problem that when a large amount of log information is stored in one log file, the log information in the log file is loaded or positioned, the data volume to be loaded is large, and the processing speed is low is avoided. The plurality of log files can be set to the same capacity to avoid large differences in the data amount of different log files.
And performing one-dimensional cutting according to the generation time of the log information, such as: the storage period is set as follows: the storage period is day, namely, the log information generated in one day is stored in a log file, and the storage time is a time period, such as: the log information generated every two hours is stored in a log file. And then, cutting another dimension according to the capacity of the log file, such as: and storing the log information into the log file according to the time sequence until the storage space of the log file is exhausted, and starting the next log file to store the log information according to the same method. If the log information is cut according to the generation time of the log information, the difference of the data amount stored in the log file is easily large, and if an emergency occurs in a certain day, for example: the log information amount stored in a certain log file is huge due to e-commerce activities, examination registration and the like, and once abnormity occurs, when abnormity positioning needs to be carried out through searching log information, the searched data amount is large, and the positioning efficiency is low. If the log information is cut according to the capacity of the log files, the log information of one storage period is easily stored in a plurality of log files, so that the problems of low searching efficiency of the log information, incomplete log information and the like are caused.
According to the log information processing method and device, log information is cut in a two-dimensional mode, the cut log information section does not exceed the capacity of a log file, storage can be conducted according to the generation time of the log information, ordered storage of a large number of log information is achieved, the log information filing efficiency is improved, and due to the fact that the capacity of the log file is limited, namely the data size of each log file is determined, the operation condition of the server in each operation period can be compared transversely through the number of the log files.
The log information segments after the two-dimensional cutting are stored in the corresponding log files in sequence, the global thread numbers and the generation time corresponding to the log information are marked on the log files, so that the log information stored in the log files can be acquired through the marking information on the log files, and the efficiency of subsequent log information searching, positioning and other operations can be improved.
Optionally, the generation time marked on the log file may be the generation time of the log information stored in the log file last, or the generation time of the log information stored in the log file first, and the generation time period of the log information stored in each log file can be determined according to the generation time of two adjacent log files, or the generation time period of the log information stored in the log file, so that the log file can be searched according to the generation time later.
Optionally, the generation time on the log file further includes time zone information where the generation time is located. The accurate generation time of the log information is determined according to the time zone information of the generation time, so that the situation that a plurality of log files with the same generation time appear in a scene of high-speed movement is avoided.
Optionally, the log file may further be marked with a storage data size of the log file, so as to clarify the size of the data size stored in the log file before the log file is opened, and also to clarify whether the log file has abnormal log information.
According to the storage method of the log information, the log information is cut into a plurality of log information segments according to the generation time of the log information, the capacity of the log file and the global thread number corresponding to the log information, and the log information is stored in the corresponding log file in order, so that the filing efficiency of the log information is improved. And the log file is marked with the generation time of the log information and the corresponding global thread number, so that the target log information can be quickly positioned through the time and the global thread number in the following process.
In addition, if one log file contains more log information, that is, the log information is not reasonably cut and stored, even if the global thread number corresponding to the log information is determined, the time consumed for opening and searching the log file is longer, so that the time for acquiring the log information is longer, and the system paralysis can be caused.
In order to make the storage scheme of the log information and the technical effect thereof more clear, the following examples are provided to describe specific embodiments thereof in detail.
In an embodiment, the two-dimensional cutting of the log information according to the generation time and the capacity of the log file provided in step S130 to obtain a plurality of log information segments can be implemented in the following manner, and a flowchart thereof is shown in fig. 2, and includes:
s210, acquiring a storage period of log information, cutting the log information according to the storage period, and acquiring a plurality of log information segments divided according to the storage period;
s220, when the data volume of any log information segment exceeds the capacity of the log file, the log information segment is divided according to the capacity of the log file, and a plurality of log information segments which do not exceed the capacity and are divided according to the storage period are obtained.
The storage period of the log information, such as day, hour, etc., stores the log information generated every day into a log file, and the log information segment is divided according to the storage period of the log information.
Determining whether the log information belongs to the same storage period according to the generation time of the log information, and cutting the log information according to the storage period, namely, taking the log information corresponding to the generation time belonging to the same storage period as a log information segment.
For any log information segment, the following operations are executed: s211, judging whether the data volume of the log information segment exceeds the capacity of the log file; if yes, executing S220, namely dividing the log information segment according to the capacity of the log file to obtain a plurality of log information segments which do not exceed the capacity and are divided according to the storage period; if not, the log information segment will not need to be split.
According to the scheme provided by the embodiment, a plurality of log information segments which do not exceed the capacity of the log file and are divided according to the storage period are obtained, so that each log information segment is stored in the corresponding log file, for example: and correspondingly storing the obtained log information segments into a log file, wherein the log information stored in each log file belongs to the same storage period, and the data volume does not exceed the capacity of the log file.
In an alternative embodiment, a way to obtain the capacity of a log file is provided, which includes:
a1, acquiring the data volume of the log information to be stored;
and A2, determining the capacity of each log file according to the data volume of the log information and the system processing performance.
The capacity of the log file can be adjusted through the data volume of log information to be stored and the system processing performance, different systems can correspond to the capacity of different log files and can acquire the system processing performance, if the system processing performance is stronger, the capacity of the log file can be set to be slightly larger so as to reduce the number of the log files, and if the system processing performance is poorer, the capacity of the log file can be set to be slightly lower so as to improve the data processing efficiency.
In an optional embodiment, before S110, the method for storing log information provided by the present application further includes:
b1, acquiring the preset number of the log files, and determining the data volume which can be accommodated by the log files according to the capacity of the log files and the preset number;
and B2, determining the log information to be stored according to the data volume.
If the capacity of the log files and the preset number of the log files are predetermined, the storable data volume of all the log files can be determined, and the log information to be stored is determined according to the storable data volume of all the log files, such as: determining the data volume of the log information to be stored, the initial storage time of the log information to be stored, the storage time period of the log information and the log information to be deleted in the current log file.
Another embodiment of the present application further provides another way to determine the capacity and storage period of the log file, that is, before performing two-dimensional cutting on the log information, the log file may be obtained by the following way, including:
c1, newly building a subclass inheriting the parent class of the FileAppender, and defining the capacity and the storage period of a log file in a configuration file of the subclass;
and C2, generating a log file for storing log information according to the configuration file.
log4j is a common log frame in Java programs, and can obtain configuration parameters of log files through configuration files of log4j, such as: the capacity of the log file, the storage period of the log information, the naming rule of the log file, and the like. One of the interfaces of the Log4j architecture, the appendix, is responsible for outputting Log information, and can output a Log to a file (fileappendix).
The FileAppender class inherits from a WriterAppender parent class, the WriterAppender is used for writing log information into a file, main log writing logic is already processed in the WriterAppender, and the main processing logic of the FileAppender mainly comprises the setting of a log output file name.
The newly created subclass may be named mydailyrollingfileprefix, the subclass inherits from the fileapplicator parent class, the log information can be cut according to the capacity and the generation time of the log file can be set in the configuration file of the subclass, and the capacity and the storage period of the log file can be flexibly set, the storage period is the output interval of each log file, for example: day, hour, minute, etc., to achieve two-dimensional cutting of log information.
Optionally, the configuration file further includes:
inheriting the file naming rule of the FileAppender parent class so as to determine the name of the log file according to the naming rule.
Because the newly-created child inherits from the FileAppender parent, the parent provides basic functions, such as: and the newly-built appendix class can inherit the logic of file format, file name naming rule and the like, so that the setting of basic function parameters in the configuration file of the subclass is reduced, and the code length is favorably reduced.
Further, an embodiment of the present application further provides a method for positioning log information, and a flowchart thereof refers to fig. 3, including:
s310, receiving a search request for a target event, responding to the search request, and acquiring the occurrence time of the target event and a corresponding global thread number;
s320, determining at least one log file by using the occurrence time; the log information is stored in the corresponding log file in advance according to the storage method of the log information;
s330, traversing the at least one log file by using the global thread number, and determining log information corresponding to the target event.
The log information generated in the system operation process is stored in the corresponding log file by using the storage method of the log information provided by the above embodiment in advance, that is, the global thread number and the generation time corresponding to the log information are identified on each log file.
When a search request for a target event, that is, a search request for log information corresponding to the target event, is received, acquiring occurrence time of the target event and a corresponding global thread number, where the occurrence time of the target event may be an initial time or an occurrence time period of occurrence of the target event, and corresponding to generation time of the log information, first, performing preliminary screening on log files according to the occurrence time of the target event, and acquiring at least one log file whose generation time of the log information matches the occurrence time of the target event, where the matching may include: if the generation time of the log information marked on the log file is a certain time, selecting that the generation time of the log information is the same as or similar to the occurrence time of the target event, and if the generation time of the log information marked on the log file is a time period, the time period is crossed with the occurrence time of the target event. The method comprises the steps of obtaining at least one log file to be checked, traversing the log files by utilizing the global thread number of a target event, carrying out secondary screening on log information through the global thread number as the global thread number of the event corresponding to the log information is marked on the log file, namely determining the target log file stored by the log information corresponding to the target event, namely determining the log information corresponding to the target event and the storage position of the log information.
If a plurality of global identification numbers are marked on one log file, the following are shown: the log file stores log information corresponding to a plurality of events, and the log information in the log file needs to be further traversed by using the global identification number to determine the log information corresponding to the target event stored in the log file.
If the log files are marked with the same global identification number, the following results are shown: the log information corresponding to the target event is stored in a plurality of log files, the log information of the log files marked with the global identification numbers of the target event is obtained, and the log information is spliced according to the generation time marked on the log files to form the complete log information corresponding to the target event.
According to the positioning method of the log information, when a search request for a target event is received, the occurrence time of the target event and the corresponding global thread number are obtained, at least one log file is determined by using the occurrence time, the at least one log file is traversed by using the global thread number, the log information corresponding to the target event is determined, and the positioning of the log information corresponding to the target event is achieved.
The positioning method provided by the embodiment can be applied to activities such as second killing, ticket robbing and the like, each session request in the activities is taken as a target event, and the realized process and effect are as follows: when the log information corresponding to the target event is located, the log file where the log information corresponding to the target event is located, the number of the log files obtained through primary screening is limited, only the log files with the limited number need to be loaded, the log files are traversed according to the global thread number corresponding to the target event, the log information corresponding to the target event can be determined rapidly, and the rapid searching of abnormal data for high-concurrency operation can be achieved.
On this basis, after determining the log information corresponding to the target event in step S330, the method further includes:
d1, obtaining the call links of the global thread number in a plurality of systems;
d2, sorting the log information according to the call link to form a log information chain.
If multiple systems call or respond to the target event, that is, the target event passes through multiple systems in the response processing process, the global thread number may appear in the multiple systems, the log information marked with the global thread number has multiple segments, the multiple segments of log information corresponding to the target event are determined according to the schemes provided in S310 to S330, the call links of the global thread number in the multiple systems are obtained, and the multiple segments of log information are sequenced according to the call links to form a log chain.
If the global thread number corresponding to the target event is the thread number of the father node, the father node comprises at least one child node, the log file identification information corresponding to the child node comprises identification information of the father node, multiple pieces of log information corresponding to the global thread number of the father node are determined according to the schemes provided in S310 to S330, the multiple pieces of log information are sequenced according to the link relation of the nodes, and a log information chain is formed.
The scheme for forming the log information chain is suitable for high concurrency, micro-service or a framework with a plurality of components or subsystems, and aiming at the condition of multi-component and multi-system calling of the micro-service, the log information corresponding to the whole calling link can be chained by the global thread number generated by the front end, so that the calling process among a plurality of systems is clear, the positioning of the log information can be realized, the positioning of the whole calling link can also be realized, and the global thread number generated by the front end can quickly position the log information of the whole calling link.
Accordingly, an embodiment of the present application further provides a storage apparatus 400 for log information, a schematic structural diagram of which is shown in fig. 4, where the storage apparatus 400 for log information includes: the first obtaining module 410, the second obtaining module 420, the cutting log information module 430, and the storage module 440 are specifically as follows:
a first obtaining module 410, configured to obtain log information to be stored and a global thread number corresponding to the log information;
a second obtaining module 420, configured to obtain a generation time of log information and a capacity of a log file for storing the log information;
a log information cutting module 430, configured to perform two-dimensional cutting on the log information according to the generation time and the capacity of the log file, so as to obtain multiple log information segments;
the storage module 440 is configured to store the plurality of log information segments into corresponding log files in sequence, where the log files are marked with global thread numbers and generation times corresponding to the log information.
With regard to the storage device of the log information in the above-mentioned embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be described in detail here.
Correspondingly, an embodiment of the present application further provides a positioning apparatus 500 for log information, a schematic structural diagram of which is shown in fig. 5, where the positioning apparatus for log information includes: the response request module 510, the log file determining module 520, and the log information determining module 530 are specifically as follows:
a response request module 510, configured to receive a search request for a target event, and in response to the search request, obtain occurrence time of the target event and a corresponding global thread number;
a determine log file module 520 for determining at least one log file using the occurrence time; the log information is stored in a corresponding log file in advance according to the storage method of the log information in any technical scheme;
a log information determining module 530, configured to traverse the at least one log file by using the global thread number, and determine log information corresponding to the target event.
With regard to the positioning apparatus of log information in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment related to the method, and will not be elaborated here.
The storage method of the log information or the positioning method of the log information provided by the above embodiment can be applied to an electronic device. The structure diagram is shown in fig. 6.
In this embodiment, the electronic device 600 may be a terminal device having an arithmetic function, such as a server, a smart phone, a tablet computer, a portable computer, or a desktop computer.
The electronic device includes: a memory and a processor, wherein the processor may be referred to as the processing device 601 hereinafter, and the memory may include at least one of a Read Only Memory (ROM)602, a Random Access Memory (RAM)603 and a storage device 608 hereinafter, which are specifically shown as follows:
as shown in fig. 6, electronic device 600 may include a processing means (e.g., central processing unit, graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 3 illustrates an electronic device 600 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.
It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.
In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText transfer protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform operations comprising:
acquiring log information to be stored and a global thread number corresponding to the log information; acquiring the generation time of log information and the capacity of a log file for storing the log information; performing two-dimensional cutting on the log information according to the generation time and the capacity of the log file to obtain a plurality of log information segments; and sequentially storing the plurality of log information segments into corresponding log files, wherein the log files are marked with global thread numbers and generation time corresponding to the log information.
Or performing the following operations: receiving a search request for a target event, and responding to the search request to acquire the occurrence time of the target event and a corresponding global thread number; determining at least one log file using the occurrence time; the log information is stored in a corresponding log file in advance according to the storage method of the log information in any technical scheme; and traversing the at least one log file by using the global thread number, and determining log information corresponding to the target event.
Furthermore, embodiments of the present application also provide a computer-readable storage medium, which may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable medium may be a machine readable signal medium or a machine readable storage medium. A computer readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a computer-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The computer readable storage medium includes a storage program and a positioning program of log information, and the storage program and the positioning program of the log information implement the steps of the storage method of the log information or the positioning method of the log information according to any one of the above technical solutions when executed by a processor.
The specific implementation of the computer-readable storage medium of the present application is substantially the same as the specific implementation of the above-mentioned log information storage and positioning method and electronic device, and will not be described herein again.
The above description is only a preferred embodiment of the present application, and not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application, or which are directly or indirectly applied to other related technical fields, are included in the scope of the present application.
Claims (10)
1. A method for storing log information, comprising:
acquiring log information to be stored and a global thread number corresponding to the log information;
acquiring the generation time of log information and the capacity of a log file for storing the log information;
performing two-dimensional cutting on the log information according to the generation time and the capacity of the log file to obtain a plurality of log information segments;
and sequentially storing the plurality of log information segments into corresponding log files, wherein the log files are marked with global thread numbers and generation time corresponding to the log information.
2. The method for storing log information according to claim 1, wherein the step of performing two-dimensional segmentation on the log information according to the generation time and the capacity of the log file to obtain a plurality of log information segments comprises:
acquiring a storage period of log information, cutting the log information according to the storage period, and acquiring a plurality of log information segments divided according to the storage period;
when the data volume of any log information segment exceeds the capacity of the log file, the log information segment is divided according to the capacity of the log file, and a plurality of log information segments which do not exceed the capacity and are divided according to the storage period are obtained.
3. The method for storing log information according to claim 1, wherein before the performing the two-dimensional cutting on the log information according to the generation time and the capacity of the log file, the method further comprises:
newly establishing a subclass inheriting a FileAppender parent class, and defining the capacity and the storage period of a log file in a configuration file of the subclass;
and generating a log file for storing log information according to the configuration file.
4. The method of claim 3, wherein the configuration file further comprises:
inheriting the file naming rule of the FileAppender parent class so as to determine the name of the log file according to the naming rule.
5. The method according to claim 1, wherein acquiring a capacity of a log file for storing the log information comprises:
acquiring the data volume of the log information to be stored;
and determining the capacity of each log file according to the data volume of the log information and the system processing performance.
6. The method according to claim 1, wherein before the obtaining the log information to be stored, the method further comprises:
acquiring the preset number of the log files, and determining the storable data volume of the log files according to the capacity of the log files and the preset number;
and determining the log information to be stored according to the data volume.
7. A method for positioning log information, comprising:
receiving a search request for a target event, and responding to the search request to acquire the occurrence time of the target event and a corresponding global thread number;
determining at least one log file using the occurrence time; wherein, the log information is stored in the corresponding log file in advance according to the log information storage method of any one of the above claims 1 to 6;
and traversing the at least one log file by using the global thread number, and determining log information corresponding to the target event.
8. The method for locating log information according to claim 7, wherein after determining the log information corresponding to the target event, the method further comprises:
obtaining the call links of the global thread number in a plurality of systems;
and sequencing the log information according to the calling link to form a log information chain.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method for storing log information according to any one of claims 1 to 6 or the method for locating log information according to claim 7 or 8 when executing the program.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium includes a storage program of log information, a positioning program, and the storage program of log information, the positioning program, when executed by a processor, implement the steps of the storage method of log information according to any one of claims 1 to 6, or the steps of the positioning method of log information according to claim 7 or 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010469328.3A CN111639059A (en) | 2020-05-28 | 2020-05-28 | Log information storage and positioning method, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010469328.3A CN111639059A (en) | 2020-05-28 | 2020-05-28 | Log information storage and positioning method, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111639059A true CN111639059A (en) | 2020-09-08 |
Family
ID=72333308
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010469328.3A Pending CN111639059A (en) | 2020-05-28 | 2020-05-28 | Log information storage and positioning method, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111639059A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112202608A (en) * | 2020-09-28 | 2021-01-08 | 亚信科技（中国）有限公司 | Log processing method and device, electronic equipment and computer readable storage medium |
| CN114385551A (en) * | 2021-12-20 | 2022-04-22 | 武汉物易云通网络科技有限公司 | Log time-sharing management method, device, equipment and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110276573A1 (en) * | 2010-05-06 | 2011-11-10 | Computer Associates Think, Inc. | Journal event consolidation |
| US8719462B1 (en) * | 2013-10-16 | 2014-05-06 | Google Inc. | Systems and methods for distributed log file processing |
| US20140317137A1 (en) * | 2012-03-12 | 2014-10-23 | Hitachi, Ltd. | Log management computer and log management method |
| CN106055703A (en) * | 2016-06-22 | 2016-10-26 | 北京科摩仕捷科技有限公司 | Real-time log analysis method and system |
| US20180225189A1 (en) * | 2016-12-23 | 2018-08-09 | Pusan National University Industry-University Cooperation Foundation | Large event log replay method and system |
| US10210162B1 (en) * | 2010-03-29 | 2019-02-19 | Carbonite, Inc. | Log file management |
| US20190188091A1 (en) * | 2017-12-15 | 2019-06-20 | Microsoft Technology Licensing, Llc | Write-ahead style logging in a persistent memory device |
| CN109960686A (en) * | 2019-03-26 | 2019-07-02 | 北京百度网讯科技有限公司 | The log processing method and device of database |
| CN109977089A (en) * | 2019-03-13 | 2019-07-05 | 深圳壹账通智能科技有限公司 | Blog management method, device, computer equipment and computer readable storage medium |
| CN110209643A (en) * | 2019-04-23 | 2019-09-06 | 深圳壹账通智能科技有限公司 | A kind of data processing method and device |
| CN110413478A (en) * | 2019-06-28 | 2019-11-05 | 苏州浪潮智能科技有限公司 | A kind of method, equipment and medium monitoring log processing |
-
2020
- 2020-05-28 CN CN202010469328.3A patent/CN111639059A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10210162B1 (en) * | 2010-03-29 | 2019-02-19 | Carbonite, Inc. | Log file management |
| US20110276573A1 (en) * | 2010-05-06 | 2011-11-10 | Computer Associates Think, Inc. | Journal event consolidation |
| US20140317137A1 (en) * | 2012-03-12 | 2014-10-23 | Hitachi, Ltd. | Log management computer and log management method |
| US8719462B1 (en) * | 2013-10-16 | 2014-05-06 | Google Inc. | Systems and methods for distributed log file processing |
| CN106055703A (en) * | 2016-06-22 | 2016-10-26 | 北京科摩仕捷科技有限公司 | Real-time log analysis method and system |
| US20180225189A1 (en) * | 2016-12-23 | 2018-08-09 | Pusan National University Industry-University Cooperation Foundation | Large event log replay method and system |
| US20190188091A1 (en) * | 2017-12-15 | 2019-06-20 | Microsoft Technology Licensing, Llc | Write-ahead style logging in a persistent memory device |
| CN109977089A (en) * | 2019-03-13 | 2019-07-05 | 深圳壹账通智能科技有限公司 | Blog management method, device, computer equipment and computer readable storage medium |
| CN109960686A (en) * | 2019-03-26 | 2019-07-02 | 北京百度网讯科技有限公司 | The log processing method and device of database |
| CN110209643A (en) * | 2019-04-23 | 2019-09-06 | 深圳壹账通智能科技有限公司 | A kind of data processing method and device |
| CN110413478A (en) * | 2019-06-28 | 2019-11-05 | 苏州浪潮智能科技有限公司 | A kind of method, equipment and medium monitoring log processing |
Non-Patent Citations (2)
| Title |
|---|
| 石凯;徐明;徐建;郑宁;: "基于NMEA 0183的GPS设备轨迹恢复方法", 计算机应用与软件, no. 12, pages 13 - 20 * |
| 蒋宗礼;张恒庆;: "多维度特征的用户查询意图自动分类", 电脑与信息技术, no. 01, pages 1 - 4 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112202608A (en) * | 2020-09-28 | 2021-01-08 | 亚信科技（中国）有限公司 | Log processing method and device, electronic equipment and computer readable storage medium |
| CN114385551A (en) * | 2021-12-20 | 2022-04-22 | 武汉物易云通网络科技有限公司 | Log time-sharing management method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111581291A (en) | Data processing method and device, electronic equipment and readable medium | |
| US9680719B2 (en) | Communication system, client terminal, and server | |
| CN110932918B (en) | Log data acquisition method and device and storage medium | |
| CN111309747A (en) | Data synchronization method, system and device | |
| CN111639059A (en) | Log information storage and positioning method, electronic equipment and storage medium | |
| CN111628938B (en) | Branch merging method, device, electronic equipment and computer storage medium | |
| CN110737655B (en) | Method and device for reporting data | |
| CN111161072A (en) | Block chain-based random number generation method, equipment and storage medium | |
| CN115996179A (en) | Service node testing method and device, readable medium and electronic equipment | |
| CN111309366B (en) | Method, device, medium and electronic equipment for managing registration core | |
| CN112433757A (en) | Method and device for determining interface calling relationship | |
| CN116743785A (en) | Cloud network data storage method, device, equipment and medium based on fog calculation | |
| CN116860286A (en) | Page dynamic update method, device, electronic equipment and computer readable medium | |
| CN113742426B (en) | Data synchronization method, device, equipment, server and storage medium | |
| WO2023273576A1 (en) | Abnormal request processing method and apparatus, electronic device and storage medium | |
| CN111367500A (en) | Data processing method and device | |
| CN115328506A (en) | Application package generation method, loading method, device, medium and equipment | |
| CN112507676B (en) | Method and device for generating energy report, electronic equipment and computer readable medium | |
| CN114115941A (en) | Resource sending method, page rendering method, device, electronic equipment and medium | |
| CN114035861A (en) | Cluster configuration method and device, electronic equipment and computer readable medium | |
| CN112311842A (en) | Method and device for information interaction | |
| CN111625692A (en) | Feature extraction method, device, electronic equipment and computer readable medium | |
| CN114968621B (en) | Data communication method, device, equipment, storage medium and system | |
| CN118035594B (en) | Method, apparatus, electronic device and computer readable medium for accessing production document | |
| US20240152504A1 (en) | Data interaction method, apparatus, and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20200908 |