CN114218160A

CN114218160A - Log processing method and device, electronic equipment and medium

Info

Publication number: CN114218160A
Application number: CN202111558552.0A
Authority: CN
Inventors: 闫江飞
Original assignee: CCB Finetech Co Ltd
Current assignee: CCB Finetech Co Ltd
Priority date: 2021-12-17
Filing date: 2021-12-17
Publication date: 2022-03-22

Abstract

The disclosure provides a log processing method which can be applied to the technical field of finance. The log processing method comprises the following steps: reading a log file; determining target information according to the log file; the target information comprises a name, a line number and timestamp information corresponding to the log file; and establishing an index corresponding to the log file according to the target information. The present disclosure also provides a log processing apparatus, a device, a storage medium, and a program product.

Description

Log processing method and device, electronic equipment and medium

Technical Field

The present disclosure relates to the field of computer technologies, and more particularly, to a method, an apparatus, a device, a medium, and a program product for processing a log.

Background

The existing log processing system needs to build a large data system in advance, the building process is complex, the technical requirement is high, and the flexibility is low. Because the existing log processing system is not flexible enough in log processing mode, the existing log processing system brings extremely poor operation experience to users.

Disclosure of Invention

In view of the above, the present disclosure provides a log processing method, apparatus, device, medium, and program product that improve log processing efficiency.

According to a first aspect of the present disclosure, there is provided a log processing method including: reading a log file; determining target information according to the log file; wherein the target information includes name, line number and timestamp information corresponding to the log file; and establishing an index corresponding to the log file according to the target information.

According to an embodiment of the present disclosure, the method further comprises: in response to receiving a query condition corresponding to the index, determining historical data of the log file; extracting key information according to the historical data of the log file; and transmitting the key information.

According to an embodiment of the present disclosure, extracting key information according to the history data of the log file includes: determining object information corresponding to the log file according to the historical data of the log file; and extracting an operation record of the object in a time period corresponding to the time stamp information based on the object information and the time stamp information.

According to an embodiment of the present disclosure, the method further comprises, prior to the reading the log file: determining whether the log file is stored in a predetermined storage database; and in the case that the log file is determined not to be stored in a predetermined storage database, downloading the log file and storing the downloaded log file in the predetermined storage database.

A second aspect of the present disclosure provides a log processing apparatus including: the reading module is used for reading the log file; the first determining module is used for determining target information according to the log file; wherein the target information includes name, line number and timestamp information corresponding to the log file; and the establishing module is used for establishing an index corresponding to the log file according to the target information.

A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the log processing method described above.

The fourth aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the above-described log processing method.

A fifth aspect of the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the above-described log processing method.

Drawings

The foregoing and other objects, features and advantages of the disclosure will be apparent from the following description of embodiments of the disclosure, which proceeds with reference to the accompanying drawings, in which:

fig. 1 schematically illustrates an application scenario diagram of a log processing method, apparatus, device, medium, and program product according to embodiments of the present disclosure;

FIG. 2 schematically shows a flow chart of a log processing method according to an embodiment of the present disclosure;

FIG. 3 schematically illustrates an execution diagram of responding to a query request according to an embodiment of the disclosure;

FIG. 4 schematically illustrates a flow diagram for determining key information according to an embodiment of the disclosure;

fig. 5 schematically shows a block diagram of a structure of a log processing apparatus according to an embodiment of the present disclosure; and

fig. 6 schematically shows a block diagram of an electronic device adapted to implement a log processing method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.

Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

The embodiment of the disclosure provides a log processing method and a log processing device, which are used for reading a log file; determining target information according to the log file; the target information comprises a name, a line number and timestamp information corresponding to the log file; and establishing an index corresponding to the log file according to the target information. The index corresponding to the log file is established according to the target information, so that the rapid index can be realized, a user can conveniently and rapidly determine the log file through the index in the inquiry process, and the problem of poor operation experience of the user is solved.

Fig. 1 schematically illustrates an application scenario diagram of a log processing method, apparatus, device, medium, and program product according to an embodiment of the present disclosure.

As shown in fig. 1, the application scenario 100 according to this embodiment may include

terminal devices

101, 102, 103, a network 104 and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The

terminal devices

101, 102, 103 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).

The

terminal devices

101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

The server 105 may be a server providing various services, such as a background management server (for example only) providing support for websites browsed by users using the

terminal devices

101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.

It should be noted that the log processing method provided by the embodiment of the present disclosure may be generally executed by the server 105. Accordingly, the log processing apparatus provided by the embodiment of the present disclosure may be generally disposed in the server 105. The log processing method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster different from the server 105 and capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105. Accordingly, the log processing apparatus provided by the embodiment of the present disclosure may also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the

terminal devices

101, 102, 103 and/or the server 105.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

The log processing method of the disclosed embodiment will be described in detail below with reference to fig. 2 based on the scenario described in fig. 1.

Fig. 2 schematically shows a flow chart of a log processing method according to an embodiment of the present disclosure.

As shown in fig. 2, the embodiment includes operations S210 to S230, and the log processing method may be performed by a server.

In operation S210, a log file is read.

The log file is a record file or a file set for recording system operation events and can be divided into an event log and a message log. The log file is read, and the end position of one reading can be recorded at each reading, and the next reading is started, and the reading is performed again from the end position.

In operation S220, determining target information according to the log file; wherein the target information includes name, line number and timestamp information corresponding to the log file.

The contents of the log file may include: line number, timestamp information, name information, object information, etc. And as the line number: 1, time stamp information: 2021-11-20-11-16-18, name information: XXXXXX, object information: three operators, object information: viewing the basic information; and as the line number: 2, time stamp information: 2021-11-20-11-16-36, name information: XXXYYY, object information: three operators, object information: changing the basic information; ...; and as the line number: 100, time stamp information: 2021-11-20-12-09-08, name information: yyyyy, object information: operator li, object information: and checking the basic information.

Based on the content of each log file, target information is determined. Such as the name, line number, and timestamp information corresponding to each file.

In operation S230, an index corresponding to the log file is established according to the target information.

For example, an index of the relational database is established according to the name of each log file, the line number of the text and the timestamp information of the log file, and the index is stored in the relational database.

In the log processing method provided by the embodiment, the log file is read; determining target information according to the log file; and establishing an index corresponding to the log file according to the target information, so that quick indexing can be realized, a user can conveniently and quickly determine the log file through the index in the inquiry process, and the problem of poor operation experience of the user is solved.

The log processing method further comprises the following steps: determining historical data of the log file in response to receiving a query condition corresponding to the index; extracting key information according to historical data of the log file; and sending the critical information.

And establishing a query condition corresponding to the index in advance, and when a user inputs the query condition corresponding to the index, accurately positioning the relevant log file through the index at a server. For example, for the index corresponding to the log file yyyyy, the corresponding query condition is created as 123123. The server, in response to receiving query 123123 corresponding to the index, quickly locates to the log file yyyyy, and obtains historical data for the log file YYYYYY.

Fig. 3 schematically shows an execution diagram of responding to a query request according to an embodiment of the present disclosure, see fig. 3. For example, a corresponding query condition is created as 123123 for the index corresponding to the log file yyyyy in advance. The user inputs the query condition 123123 through the client 310, the client 310 carries the query condition 123123 to send a query request to the server 320, and the server 320 sends the query condition 123123 to the log processing device 330 in response to the query request. The log processing means 330 first determines the log file yyyyy according to the query condition 123123. Since the log file has an important role in processing the history data, the history data of the log file can be acquired. Based on the history data recorded in the log file, relevant information, such as predetermined key information, can be further extracted. After determining the key information, the log processing device 330 sends the key information to the client, so that the user can acquire the key information.

In the embodiment, in response to receiving the query condition corresponding to the index, the historical data of the log file is quickly determined through the pre-created index, so that the key information is determined according to the historical data of the log file, the processing mode is simple and flexible, a user can accurately and quickly obtain the key information conveniently, the waiting time of the user is reduced, and the operation experience of the user is improved.

Extracting key information according to the historical data of the log file, wherein the key information comprises the following steps: determining object information corresponding to the log file according to the historical data of the log file; based on the object information and the time stamp information, an operation record of the object in a period corresponding to the time stamp information is extracted.

The object information may be an operator or an operation item.

For example, the indices are as follows: 2021-11-20-11-16-18 (date), zhang san (operator), c: //2021-11-20-11-1.log (file location), 1 (file line number); 2021-11-20-11-16-36 (date), zhang san (operator), c: //2021-11-20-11-1.log (file location), 2 (file line number); 2021-11-20-12-09-08 (date), lie four (operator), c: //2021-11-20-11-1.log (file location), 100 (file line number). The user inquires about the content of all operations of three pages in the day 2021-11-20, the log processing device firstly determines which row (row number) of which log file records the information of three pages in the database, then reads the information according to the screening result to determine that the information is recorded in the database, and the operation person of the day 2021-11-20 records two operations of three pages, namely in the file c: the log reads the information of the first line and the second line, Zhang III respectively checks the basic information and changes the basic information in the day of 2021-11-20, and the operation is performed twice.

According to the embodiment, the operation record of the object in the time period corresponding to the timestamp information can be conveniently, quickly and accurately acquired by the user based on the object information and the timestamp information.

The log processing method further comprises, before reading the log file: determining whether the log file is stored in a predetermined storage database; in case it is determined that the log file is not stored in the predetermined storage database, the log file is downloaded and the downloaded log file is stored in the predetermined storage database.

Fig. 4 schematically shows a flow diagram for determining key information according to an embodiment of the present disclosure. Referring to fig. 4, the embodiment includes operations S401 to S408.

In operation S401, it is determined whether the log file has a predetermined storage database, where the predetermined storage database may be a local database, a cloud database, or a database for centrally storing the log file.

If the log file does not have the predetermined storage database, operation S402 is performed, the log file is collected, the log file is downloaded to the predetermined storage database, and then operation S403 is performed.

If the log file already exists in the predetermined storage database, operation S403 is performed to read the log file.

In operation S404, target information, such as a name, a line number, and time stamp information of the log file, is determined.

In operation S405, an index corresponding to the log file is built according to target information, such as the name, line number, and timestamp information of the old log file.

When the query condition sent by the client is received, operation S406 is performed, and in response to receiving the query condition corresponding to the index, the history data of the log file is determined.

In operation S407, key information is extracted according to the history data of the log file, and the key information is determined.

In operation S408, key information is transmitted to the user.

In this embodiment, by storing the log file in a predetermined storage database, such as a cloud or a local database, it is beneficial to improve the security of the log file and share the storage data. When complex deployment is needed, for example, based on the object information and the timestamp information, the operation record of the object in the time period corresponding to the timestamp information is determined, so that the user can accurately and quickly obtain the operation record of the target object, and the user experience is improved.

Based on the log processing method, the disclosure also provides a log processing device. The apparatus will be described in detail below with reference to fig. 5.

Fig. 5 schematically shows a block diagram of a log processing apparatus according to an embodiment of the present disclosure.

As shown in fig. 5, the log processing apparatus 500 of this embodiment includes a reading module 510, a first determining module 520, and a creating module 530.

A reading module 510, configured to read a log file; a first determining module 520, configured to determine target information according to the log file; the target information comprises a name, a line number and timestamp information corresponding to the log file; and an establishing module 530, configured to establish an index corresponding to the log file according to the target information.

The log processing apparatus further includes: a second determining module, configured to determine, in response to receiving a query condition corresponding to the index, historical data of the log file; the extraction module is used for extracting key information according to the historical data of the log file; and a sending module for sending the key information.

An extraction module comprising: the third determining submodule is used for determining object information corresponding to the log file according to the historical data of the log file; and the extraction submodule is used for extracting the operation record of the object in the time period corresponding to the time stamp information based on the object information and the time stamp information.

The log processing apparatus further includes: a download module to, prior to reading the log file: determining whether the log file is stored in a predetermined storage database; in case it is determined that the log file is not stored in the predetermined storage database, the log file is downloaded and the downloaded log file is stored in the predetermined storage database.

According to an embodiment of the present disclosure, any plurality of the reading module 510, the first determining module 520, and the creating module 530 may be combined into one module to be implemented, or any one of them may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the reading module 510, the first determining module 520, and the creating module 530 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware. Alternatively, at least one of the reading module 510, the first determining module 520, and the creating module 530 may be at least partially implemented as a computer program module, which when executed, may perform a corresponding function.

As shown in fig. 6, an electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. Processor 601 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 601 may also include onboard memory for caching purposes. Processor 601 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the disclosure.

In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are stored. The processor 601, the ROM602, and the RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM602 and/or RAM 603. It is to be noted that the programs may also be stored in one or more memories other than the ROM602 and RAM 603. The processor 601 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.

Electronic device 600 may also include input/output (I/O) interface 605, input/output (I/O) interface 605 also connected to bus 604, according to an embodiment of the disclosure. The electronic device 600 may also include one or more of the following components connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM602 and/or RAM 603 described above and/or one or more memories other than the ROM602 and RAM 603.

Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flow chart. When the computer program product runs in a computer system, the program code is used for causing the computer system to realize the item recommendation method provided by the embodiment of the disclosure.

The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 601. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed in the form of a signal on a network medium, downloaded and installed through the communication section 609, and/or installed from the removable medium 611. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program, when executed by the processor 601, performs the above-described functions defined in the system of the embodiments of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.

In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims

1. A log processing method, comprising:

reading a log file;

determining target information according to the log file; wherein the target information includes name, line number and timestamp information corresponding to the log file; and

and establishing an index corresponding to the log file according to the target information.

2. The method of claim 1, further comprising:

in response to receiving a query condition corresponding to the index, determining historical data of the log file;

extracting key information according to the historical data of the log file; and

and sending the key information.

3. The method of claim 2, said extracting key information from historical data of said log file comprising:

determining object information corresponding to the log file according to the historical data of the log file;

and extracting an operation record of the object in a time period corresponding to the time stamp information based on the object information and the time stamp information.

4. The method of claim 1, further comprising, prior to said reading the log file:

determining whether the log file is stored in a predetermined storage database;

and in the case that the log file is determined not to be stored in a predetermined storage database, downloading the log file and storing the downloaded log file in the predetermined storage database.

5. A log processing apparatus comprising:

the reading module is used for reading the log file;

the first determining module is used for determining target information according to the log file; wherein the target information includes name, line number and timestamp information corresponding to the log file; and

and the establishing module is used for establishing an index corresponding to the log file according to the target information.

6. The apparatus of claim 5, further comprising:

a second determining module, configured to determine, in response to receiving a query condition corresponding to the index, historical data of the log file;

the extraction module is used for extracting key information according to the historical data of the log file; and

and the sending module is used for sending the key information.

7. The apparatus of claim 6, the extraction module, comprising:

the third determining submodule is used for determining object information corresponding to the log file according to the historical data of the log file;

and the extraction submodule is used for extracting the operation record of the object in a time period corresponding to the timestamp information based on the object information and the timestamp information.

8. An electronic device, comprising:

one or more processors;

a storage device for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-4.

9. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 4.

10. A computer program product comprising a computer program which, when executed by a processor, implements a method according to any one of claims 1 to 4.