CN113934428A

CN113934428A - Log analysis method and device

Info

Publication number: CN113934428A
Application number: CN202111197724.6A
Authority: CN
Inventors: 林泽江; 李建民
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2021-10-14
Filing date: 2021-10-14
Publication date: 2022-01-14

Abstract

The embodiment of the invention provides a log analysis method and a log analysis device, wherein the method comprises the following steps: running a log analysis program to obtain log information generated based on misoperation; determining a plug-in identifier of the log information, and acquiring corresponding plug-in script data from a preset database according to the plug-in identifier; the plug-in script data is the script data which is updated to a preset database when the structure of the log information is changed or newly added log information is accessed; and executing the plug-in script data to analyze the log information to obtain analysis content so as to realize a recovery process of the log related to the misoperation based on the analysis content. When the structure of the game log is changed or a newly added game log is accessed, the differentiated analysis of the game log can be realized only by updating the modified or newly added script data to the database, and the updating of the online application program can be realized without performing additional actions such as releasing, updating, restarting and the like, so that the effect of real-time hot update response is achieved.

Description

Log analysis method and device

Technical Field

The present invention relates to the field of computer technologies, and in particular, to a log parsing method and a log parsing apparatus.

Background

During the development and operation of the game, game logs related to the operation of the game can be recorded, and the game logs can be classified into the following types according to the types of the logs: the system log is used for describing the daily running state of the server, the user log is used for recording the behaviors and operations of the players, and the abnormal log is used for recording the abnormal occurrence of the program in the game server, so that the log of the running condition of the game can be known based on the recorded game log, and the emergency in the game can be found and processed according to the game log.

The emergency situation may include a misoperation situation of a game player in the game. In the game customer service business, it is usually necessary to process a recovery process for a large amount of misoperation data of game players, including a plurality of different games and/or a plurality of different scenarios under the same game, for example, the recovery of misinterpretation of equipment in the game, the recovery of malpurchasing of properties in a mall in an opponent game, and specifically, the recovery of the process needs to be realized by analyzing game logs, however, format differences between game logs of different games or different scenarios of the same game are large, and a subsequent recovery business process needs to be performed on the premise of performing differential analysis on different game logs. In the related differentiation analysis technology, a hard-coding method is usually adopted to solve the differentiation, however, a hard-coding method is performed on each flow, and the whole adjustment cycle is time-consuming for updating the flow itself, which may affect the use of the game function.

Disclosure of Invention

In view of the above problems, embodiments of the present invention are proposed to provide a log parsing method and a corresponding log parsing apparatus that overcome or at least partially solve the above problems.

The embodiment of the invention discloses a log analysis method, which comprises the following steps:

running a log analysis program to obtain log information generated based on misoperation;

determining a plug-in identifier of the log information, and acquiring corresponding plug-in script data from a preset database according to the plug-in identifier; the plug-in script data is script data which is updated to a preset database based on the change of the structure of the log information or the access of newly added log information;

and executing the plug-in script data to analyze the log information to obtain analysis content so as to realize a recovery process of the log related to the misoperation based on the analysis content.

Optionally, the log information has a recovery procedure identifier; the determining the plug-in identifier of the log information includes:

and acquiring a plug-in binding table for recording a recovery flow identifier and a plug-in identifier, and searching the plug-in identifier which has a binding relation with the recovery flow identifier in the plug-in binding table.

Optionally, the obtaining, according to the plug-in identifier, corresponding plug-in script data from a preset database includes:

acquiring a plug-in list for recording plug-in identification and plug-in script content from the preset database;

and determining plug-in information corresponding to the plug-in identifier in the plug-in list, and searching plug-in script data corresponding to the plug-in information.

Optionally, the log parser provides a plug-in central control service; the executing the plug-in script data to analyze the log information to obtain analysis content includes:

searching an analysis method of an analysis format corresponding to the plug-in identification through the plug-in central control service;

and executing the plug-in script data in a reflection mode by adopting the analysis method so that the plug-in script data analyzes the log information to obtain analysis content.

Optionally, the method further comprises:

and updating the plug-in script data in the preset database, and establishing the binding relationship between the updated plug-in script data and the log information.

Optionally, the updating the plug-in script data in the preset database, and establishing a binding relationship between the updated plug-in script data and the log information includes:

under the condition that the structure of the log information is changed, modifying script logic of the plug-in script data which is in binding relation with the log information, and coding the modified script logic to obtain modified plug-in script data;

and updating the modified plug-in script data to a plug-in list of the preset database so as to realize plug-in replacement.

Optionally, the log parser has a plug-in central control service; the updating the modified plug-in script data to the plug-in list of the preset database includes:

detecting the modification operation of the script logic of the plug-in script data, and issuing a plug-in change message; the plug-in change message carries the modified plug-in script data and the plug-in information for modification;

compiling the modified plug-in script data based on the plug-in change message through the plug-in central control service, and replacing the plug-in script data in the plug-in list corresponding to the modified plug-in information.

writing plug-in script data based on the analysis format of the newly added log information under the condition of accessing the newly added log information;

and updating the written plug-in script data to a plug-in list of the preset data table, and establishing a binding relationship between the written plug-in script data and the log information so as to realize the newly added plug-ins.

Optionally, the log parser has a plug-in central control service; the updating the written plug-in script data to the plug-in list of the preset data table comprises:

detecting the insertion operation of the written plug-in script data and issuing a plug-in newly-added message; the plug-in change message carries the modified plug-in script data;

compiling the compiled plug-in script data to obtain newly added plug-in information through the plug-in central control service based on the plug-in newly added message, and establishing a corresponding relation between the newly added plug-in information and the compiled plug-in script data in the plug-in list.

Optionally, the log parsing program has a plug-in central control service, and before the log parsing program is run, the method further includes:

constructing a database storage component for realizing that the plug-in binding table and the plug-in list have the capacity of increasing, deleting, modifying and checking, and constructing a message receiving component for having the function of publishing and subscribing;

injecting the database warehousing component and the message receiving component into the plug-in central control service;

and scanning the plug-in central control service to obtain a plug-in list, compiling plug-in script data in the plug-in list to obtain plug-in information, and injecting the plug-in information into a container of the log analysis program.

The embodiment of the invention also discloses a log analysis device, which comprises:

the log analysis program running module is used for running a log analysis program and acquiring log information generated based on misoperation;

the plug-in script data acquisition module is used for determining a plug-in identifier of the log information and acquiring corresponding plug-in script data from a preset database according to the plug-in identifier; the plug-in script data is script data which is updated to a preset database based on the change of the structure of the log information or the access of newly added log information;

and the log information analysis module is used for executing the plug-in script data to analyze the log information to obtain analysis content so as to realize a recovery process of the log related to the misoperation based on the analysis content.

Optionally, the log information has a recovery procedure identifier; the plug-in script data acquisition module comprises:

and the plug-in identification determining submodule is used for acquiring a plug-in binding table for recording the recovery process identification and the plug-in identification, and searching the plug-in identification which has a binding relation with the recovery process identification in the plug-in binding table.

Optionally, the plug-in script data obtaining module includes:

the plug-in list acquisition submodule is used for acquiring a plug-in list for recording plug-in identification and plug-in script contents from the preset database;

and the plug-in script data searching submodule is used for determining plug-in information corresponding to the plug-in identifier in the plug-in list and searching plug-in script data corresponding to the plug-in information.

Optionally, the log parser provides a plug-in central control service; the log information analysis module comprises:

the analysis method searching sub-module is used for searching an analysis method of an analysis format corresponding to the plug-in identification through the plug-in central control service;

and the log information analysis submodule is used for executing the plug-in script data in a reflection mode by adopting the analysis method so as to enable the plug-in script data to analyze the log information to obtain analysis content.

Optionally, the apparatus further comprises:

and the plug-in script data updating module is used for updating the plug-in script data in the preset database and establishing the binding relationship between the updated plug-in script data and the log information.

Optionally, the plug-in script data updating module includes:

the plug-in script data modification submodule is used for modifying the script logic of the plug-in script data which is in binding relation with the log information under the condition that the structure of the log information is changed, and coding the modified script logic to obtain the modified plug-in script data;

and the first plug-in script data updating submodule is used for updating the modified plug-in script data to a plug-in list of the preset database so as to realize the replacement of a plug-in.

Optionally, the log parser has a plug-in central control service; the first plug-in script data update submodule comprises:

the plug-in change message issuing unit is used for detecting the modification operation of the script logic of the plug-in script data and issuing a plug-in change message; the plug-in change message carries the modified plug-in script data and the plug-in information for modification;

and the plug-in script data replacing unit is used for compiling the modified plug-in script data based on the plug-in change message through the plug-in central control service and replacing the plug-in script data in the plug-in list corresponding to the modified plug-in information.

Optionally, the plug-in script data updating module includes:

the plug-in script data compiling submodule is used for compiling plug-in script data based on the analytic format of the newly added log information under the condition that the newly added log information is accessed;

and the second plug-in script data updating submodule is used for updating the written plug-in script data to a plug-in list of the preset data table and establishing a binding relationship between the written plug-in script data and the log information so as to realize the addition of a plug-in.

Optionally, the log parser has a plug-in central control service; the second plug-in script data update submodule comprises:

the plug-in newly-added message issuing unit is used for detecting the insertion operation of the written plug-in script data and issuing a plug-in newly-added message; the plug-in change message carries the modified plug-in script data;

and the plug-in script information adding unit is used for compiling the compiled plug-in script data to obtain added plug-in information through the plug-in central control service based on the plug-in adding message, and establishing the corresponding relation between the added plug-in information and the compiled plug-in script data in the plug-in list.

Optionally, the log parsing program has a plug-in central control service, and before the log parsing program is run, the apparatus further includes:

the component construction module is used for constructing a database storage component for realizing that the plug-in binding table and the plug-in list have the capacity of increasing, deleting, modifying and checking, and constructing a message receiving component for having the function of publishing and subscribing;

the component injection module is used for injecting the database warehousing component and the message receiving component into the plug-in central control service;

and the plug-in information compiling module is used for scanning through the plug-in central control service to obtain a plug-in list, compiling plug-in script data in the plug-in list to obtain plug-in information, and injecting the plug-in information into a container of the log analysis program.

The embodiment of the invention also discloses an electronic device, which comprises: a processor, a memory and a computer program stored on the memory and capable of running on the processor, the computer program when executed by the processor implementing the steps of any of the log parsing methods.

The embodiment of the invention also discloses a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when being executed by a processor, the computer program realizes the steps of any log analysis method.

The embodiment of the invention has the following advantages:

in the embodiment of the invention, after the log analysis program is run, the log information generated based on the misoperation of the user can be acquired, the plug-in identification of the log information is determined, and the corresponding plug-in script data can be acquired from the preset database according to the determined plug-in identification, wherein the plug-in script data in the preset database can be updated based on the change sent by the structure of the log information or the access of newly added log information, and at the moment, the acquired plug-in script data can be executed to analyze the log information to obtain the analysis content, so that the misoperation recovery program can subsequently perform the recovery process on the log related to the misoperation of the user based on the analysis content obtained by analysis. When the structure of the game log is changed or a newly added game log is accessed, the differentiated analysis of the game log can be realized only by updating the modified or newly added script data to the database, and the updating of the online application program can be realized without performing additional actions such as releasing, updating, restarting and the like, so that the effect of real-time hot update response is achieved, and the timeliness is high.

Drawings

FIG. 1 is a schematic diagram of an overall structure of a log parsing program according to an embodiment of the present invention;

FIG. 2 is a flow chart of the steps of one embodiment of a method of log parsing of the present invention;

FIG. 3 is a diagram illustrating the relationship between the misoperation recovery procedure and the log analysis procedure in the embodiment of the present invention;

FIG. 4 is a flow chart of steps in another embodiment of a method of log resolution of the present invention;

FIG. 5 is a diagram illustrating the execution of a fault recovery routine according to an embodiment of the present invention;

FIG. 6 is a diagram illustrating an implementation of a log parsing routine in an embodiment of the invention;

fig. 7 is a block diagram of a log parsing apparatus according to an embodiment of the present invention.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.

Aiming at the recovery process of misoperation data of a game player in a game, the format difference of game logs of different games or different scenes in the same game is large, and the subsequent recovery business process needs to be carried out on the premise of carrying out differential analysis on different game logs.

In the related technology of differential analysis, differentiation is usually solved by means of program hard coding, in which variables are fixed values, which cannot be modified in the game running process, and each flow has its own separate hard coding (including code structure and class structure), that is, for each error recovery flow and log, separate program coding development, program compiling, and installation package issuing are performed according to its format to a user update or update server.

Specifically, a log analysis mode may be determined according to the misoperation recovery flow and the game log, then the determined log analysis mode is developed through program coding, and a program is compiled to obtain an installation package after the misoperation performed by the game player in the game is recovered, and then the misoperation recovery flow is completed based on the installation package obtained through compilation. In the process of completing the misoperation recovery process based on the installation package, for a C/S (Client/Server) project, the installation package can be issued to a user for updating, and for a B/S (Browser/Server) project, the installation package can be used for updating and restarting a related Server, so that the misoperation data can be recovered.

However, the manner of re-hard-coding each flow individually, since hard-coding cannot be modified during the game play, may have the following disadvantages: (1) if the structure of the game log is changed or a newly added game log is to be accessed, the steps of determining the format, developing, compiling, releasing, updating, restarting and the like need to be performed, for updating of the process, the whole adjusting period takes longer time (the period can be more than one week), and real-time hot updating response cannot be performed; particularly, for the software of the C/S architecture, the situation that the user updates the program after a long time even can occur, which affects the use of the game function, thereby causing low timeliness of the misoperation recovery flow; (2) and under the condition that a newly-added game log is accessed every time, no matter whether a reusable development program exists or not, the steps of determining the format, developing, compiling, releasing, updating, restarting and the like need to be repeated, and when an online problem is met, the program cannot be rapidly traumatic, so that the development efficiency of the whole project is influenced, and the problem of low development efficiency of the game program is caused.

One of the core ideas of the embodiment of the invention is to provide a reusable differentiated log analysis program which supports rapid development and iteration and supports hot update, when a log is analyzed, the corresponding analysis format can be obtained through a plug-in identifier bound by the log to analyze to obtain analysis content, when the structure of the game log is changed or a newly added game log is accessed, the differentiated analysis of the game log can be realized only by updating modified or newly added script data to a database, and the online application program can be updated without performing additional actions of issuing, updating, restarting and the like, so that the effect of real-time hot update response is achieved, and the timeliness is high; when the structure of the game log is changed or a newly added game log is accessed, the whole development process only needs to finish the writing of a plug-in script, when the field problem is met, the rapid iteration of the program can be finished directly by modifying the script, the adjustment period of the process is shortened (only 1 day is needed probably), and the development efficiency of the whole project is improved; and under the condition that the plug-in with the same analysis logic as the game log exists, the multiplexing of the plug-in script logic can be directly completed in the plug-in binding table by adding the board top information, so that the reusability of the program is realized.

In the embodiment of the present invention, the format difference of the game logs of different games or different scenes in the same game is large, and at this time, the text file output in the game program, that is, the service log output by the program in the game process, for example, the device-related data output when the device is discarded, needs to be analyzed differentially, so as to process the game log by adopting different processing logics according to the game logs of different formats, and process the game log into a format capable of meeting the related service requirements, specifically, the format difference can be realized by the log analysis program.

In a specific implementation, the constructed log parsing program may be an application program that is based on a Spring application program (which is a lightweight open source framework in the programming field), and based on a MongoDB (which is an open source database system based on distributed file storage) and a Groovy script (which is a language that has characteristics of Java language and has dynamic capability, when the Groovy script is run, it is compiled into a JVM bytecode or a Java virtual machine bytecode, and then the JVM bytecode or the Java virtual machine bytecode is executed by a JNM virtual machine), and has a differential parsing function on game logs, referring to fig. 1, a schematic diagram of an overall structure of the log parsing program in an embodiment of the present invention is shown, and may be divided into the following 5 layers: a) an application layer, b) a service layer, c) a plug-in layer, d) a storage layer, and e) a data layer.

The application layer a) may refer to an misoperation recovery program, the recovery program may be a service module belonging to the same application program as the log analysis program, or may refer to access of other application programs, and the recovery program plays a role in receiving a processing result of the log analysis program, such as analysis content obtained by analysis; b) the service layer can be mainly divided into two modules, namely a log downloader and a plug-in central control service, wherein the log downloader can be used for being responsible for log downloading, such as log information generated by misoperation and log information recorded based on user purchasing behavior, and the plug-in central control service can be used for being responsible for scanning, compiling and injecting plug-ins and has the functions of receiving plug-in content change information and executing the plug-ins; c) the plug-in layer mainly comprises plug-ins injected into the Spring Ioc container, and can provide various plug-ins compiled and injected into the Spring Ioc container for the plug-in central control service of the upper layer, namely the service layer, so that the plug-in central control service can execute corresponding plug-in capacity; d) the storage layer can be divided into a log storage module and a MongoDB storage service, the log storage module can provide log acquisition capability for a log downloader positioned on the service layer, the MongoDB storage service can be used for building a MongoDB database, the MongoDB database can contain a plug-in list for storing the basic information of the plug-ins and the script contents of the plug-ins, and a plug-in binding table for maintaining the binding relationship between the process and the plug-ins, and the storage of the script contents of the plug-ins is realized through the two tables; e) the data layer can be divided into log data, Groovy script data and plug-in binding relationship data, and can provide related storage data for the data storage capacity of the upper layer, namely the log storage module located in the storage layer and the MongoDB storage service.

It should be noted that, in the process of constructing the overall structure of the log parsing program based on the Spring program, the log parsing program may be sequentially constructed according to the sequence from the bottom structure to the upper structure, for example, according to the structural sequence of the data layer-storage layer-plug-in layer-service layer-application layer, so as to provide corresponding functions for the upper module through the lower module, for example, provide the storage layer with related data to be stored via the data layer, provide the service layer with various plug-ins to be executed via the plug-in layer, provide the application layer with log downloading capability and plug-in central control service, and the like; in the process of construction, the structure may be sequentially constructed according to the sequence from the upper layer structure to the bottom layer structure, for example, according to the structural sequence of the application layer, the service layer, the plug-in layer, the storage layer, and the data layer, so as to clarify the composition relationship between different layers, for example, the application layer with log parsing capability needs to be implemented by the log downloading capability provided by the service layer and the plug-in central control service, that is, the application layer needs to be composed via the service layer, and the like.

Referring to fig. 2, a flowchart of steps of an embodiment of a log parsing method according to the present invention is shown, focusing on a construction process of a log parsing program, which is mainly expressed as injecting a plug-in the log parsing program, and specifically may include the following steps:

step 201, before running a log analysis program, constructing a database storage component for realizing that a plug-in binding table and a plug-in list have the capacity of increasing, deleting, modifying and checking, and constructing a message receiving component for having a publishing and subscribing function;

in an embodiment of the present invention, a game log of different games or different scenes in the same game needs to be analyzed differentially, and the differential analysis can be implemented by analyzing a plug-in an analysis format corresponding to the game log, and at this time, the plug-in needs to be injected in a log analysis program, so as to provide various plug-ins capable of being used for performing log differential analysis for the log analysis program.

The time for injecting the plug-in the log analysis program may be before the log analysis program is run, that is, when the start trigger of the log analysis program is detected, an operation of injecting the plug-in is required when the log analysis program is started each time, the injection operation is equivalent to loading a plug-in analysis function for the log analysis program, and the log analysis program can be started and completed after the plug-in is injected. It should be noted that, after the plug-in injection operation is performed, the injection success or failure of the plug-in does not affect the start of the log analysis program, and if a certain plug-in injection failure only indicates that the plug-in is not valid.

In practical application, the trigger mode for starting the log analysis program may be determined based on a relationship between the log analysis program and the misoperation recovery program, and may be specifically divided into the following two cases: when the log analysis program and the misoperation recovery program belong to an inclusion relationship, the log analysis program can be triggered and started through the starting operation of the misoperation recovery process; when the log analysis program and the misoperation recovery program belong to a parallel relation, the log analysis program can be directly started as an independent service.

Specifically, referring to fig. 3, a schematic diagram of a relationship between the misoperation recovery program and the log analysis program in the embodiment of the present invention is shown, in one case, the misoperation recovery program in the application layer refers to the log analysis program, that is, the log analysis program is used as a module of the application program, at this time, the misoperation recovery program and the log analysis program in the application layer may be regarded as a logic inclusion relationship, in another case, the misoperation recovery program and the log analysis program in the application layer are two independent server application programs respectively, wherein the log analysis program can integrate functions of log query and analysis, and can provide an interface of a HTTP (hypertext Transfer Protocol) network request for the misoperation recovery program, that is, the two may communicate with each other through an HTTP interface, and at this time, the two belong to a parallel relationship.

In one embodiment of the present invention, for the operation of injecting the plug-in, it is first necessary to construct the components related to the plug-in to be injected, specifically to construct the MongoDB database repository component and the message receiving component, so as to provide the related plug-in functions for the plug-in central control service module located in the service layer based on the constructed components.

Step 202, injecting a database warehousing component and a message receiving component into a plug-in central control service;

the constructed MongoDB warehousing component can realize the capacity of increasing, deleting, modifying and checking the plug-in binding table and the plug-in table based on spring-data-MongoDB, and the constructed message receiving component can subscribe the message of plug-in content modification based on the publish-subscribe function of Redis, and at the moment, the constructed component can be injected into the plug-in central control service so as to provide the relevant plug-in functions for the plug-in central control service module of the service layer.

In practical application, firstly, the constructed MongoDB warehousing component and the message receiving component can be respectively injected into a Spring Ioc container, then the plug-in central control service can be constructed by using a class construction method based on the components injected into the Spring Ioc container, and after the plug-in central control service is constructed, the constructed MongoDB warehousing component and the message receiving component can respectively depend on the plug-in central control service to complete the injection of the components related to the plug-in functions.

Step 203, obtaining a plug-in list through the plug-in central control service scanning, compiling plug-in script data in the plug-in list to obtain plug-in information, and injecting the plug-in information into a container of the log analysis program.

After the injection of the components related to the plug-in functions is completed, the plug-in central control service can scan the MongoDB plug-in table located in the storage layer through the MongoDB storage component, sequentially compile scripts for all the scanned plug-in lists to obtain plug-in information, and inject the plug-ins into the Spring Ioc container to realize the injection operation of the plug-ins so as to successfully start the log analysis program.

It should be noted that, the plugin central control Service module located in the Service layer is essentially a Service in a Spring framework, and this Service can not only implement injection of the MongoDB warehousing component and the message receiving component, but also implement the capability of compiling Groovy scripts, injecting plugins, and executing plugins.

Then, when compiling the script, the capability of compiling the Groovy script can be used, and the capability can be realized based on the Groovy package of org.codehaus.groovy, and the Groovy package can specifically provide a Groovy shell class, which can provide a parse parsing method to compile the contents of the Groovy script into the byte code of the JVM; and the capability of using the injection plug-in when injecting the plug-in can be used, and the implementation can be specifically realized by a DefaultListableBeanFactor based on a Spring Ioc container, wherein the DefaultListableBeanFactor is a default implementation of a Spring Bean factory, and the class can provide a Bean injection registerSingleton method, so that after the compiled Groovy script is assembled into the plug-in, the plug-in can be used as a Bean and injected into the Spring Ioc container by taking a plug-in ID as a name, and the plug-in can be used by an application program.

In the embodiment of the invention, when the log is analyzed, the corresponding analysis format can be obtained through the plug-in identification bound by the log to analyze to obtain the analysis content, when the structure of the game log is changed or a newly added game log is accessed, the game log can be analyzed in a differentiation mode only by updating the modified or newly added script data to the database, and the online application program can be updated without additional actions of issuing, updating, restarting and the like, so that the effect of real-time hot update response is achieved, and the timeliness is high.

Referring to fig. 4, a flowchart illustrating steps of another embodiment of the log parsing method of the present invention is shown, and the method focuses on a using process of the log parsing program, and specifically may include the following steps:

step 401, running a log analysis program to obtain log information generated based on misoperation;

in the embodiment of the invention, after the log analysis program is subjected to the operation of plug-in injection, the log analysis program can be successfully started and operated, and at the moment, the log information generated based on the misoperation of a user can be acquired so as to determine the log information to be analyzed by the log analysis program and realize the subsequent recovery process of the misoperation log.

In practical application, the log information can be acquired through a log downloader located at a service layer. The log downloader module may be essentially a component of an HTTP network request, and may be implemented by using frames such as HTTP live, Okhttp3, RestTemplate, and the like. The log obtaining capability of the log downloader module can be provided by a log storage module located in the storage layer, the log storage module can specifically provide a HTTP request interface to achieve the log obtaining capability, and the log data related to the log information to be downloaded can be stored in the data layer.

In a preferred embodiment, in order to implement the differentiated parsing of the log information, the differentiated parsing can be implemented by parsing a plug-in a parsing format corresponding to the game log, and at this time, a plug-in matching with the log information structure can be updated. It should be noted that the update timing may be before the program is executed or during the program execution, and the embodiment of the present invention is not limited thereto.

Specifically, when the structure of the log information is changed (i.e., there is a plug-in change) or new log information is accessed (i.e., there is a plug-in addition), the plug-in of the log analysis program may be updated, which may be specifically represented by updating the plug-in script data in the preset database, and establishing a binding relationship between the updated plug-in script data and the log information. The preset database can be a MongoDB database built by a MongoDB storage service, the MongoDB is a database based on distributed file storage, is between a relational database and a non-relational database, is a database which has the most abundant functions and is the most similar to the relational database in the non-relational database, and can store more complex data types and script contents of complex plug-ins because the supported data structure is very loose (similar to the BSON format of JSON and is a storage format in a binary system form), and the maximum capacity of a single MongoDB document can reach 16 MB.

Then, for the update of the plug-in, in the first case, that is, in the case that the structure of the log information is changed, the script logic of the plug-in script data having a binding relationship with the log information may be modified, and the modified script logic is encoded to obtain the modified plug-in script data, and then the modified plug-in script data is updated to the plug-in list of the preset database, so as to implement the replacement of the plug-in.

Specifically, when detecting a modification operation on the script logic of the plug-in script data, a plug-in change message may be issued, where the issued plug-in change message carries the modified plug-in script data and the modified plug-in information, and at this time, the modified plug-in script data may be compiled based on the plug-in change message through the plug-in central control service, and the plug-in script data in the plug-in list corresponding to the modified plug-in information may be replaced.

In practical application, firstly, logic of a plug-in Groovy script can be modified according to actual requirements, and after the Groovy script is subjected to Base64 encoding (which is the most common binary encoding method), the script encoded by the Base64 can be updated to a document corresponding to a MongoDB plug-in table; at this time, a Redis (which is a key-value storage system) can publish a plugin change message, and the plugin central control service can subscribe a message to the Redis based on a pre-injected message receiving component to receive the published message, so that when the plugin central control service receives the plugin change message, the MongoDB repository component can be used for inquiring the latest Script of the plugin and recompiling the Groovy Script, and then the Groovy Script of the corresponding plugin in the Spring Ioc container is replaced, so that the plugin is replaced.

In the second case, that is, in the case of accessing the newly added log information, the plug-in script data may be written based on the parsing format of the newly added log information, and then the written plug-in script data is updated to the plug-in list of the preset data table, and the binding relationship between the written plug-in script data and the log information is established, so as to implement the newly addition of the plug-in.

Specifically, when the insertion operation of the written plug-in script data is detected, a plug-in addition message can be issued, the issued plug-in change message carries the modified plug-in script data, at this time, the written plug-in script data can be compiled by the plug-in central control service based on the plug-in addition message to obtain the added plug-in information, and the corresponding relationship between the added plug-in information and the written plug-in script data is established in the plug-in list.

In practical application, firstly determining an analysis rule of a newly added flow log, judging whether a plug-in meeting the analysis rule exists, if various plug-ins of a Spring Ioc container do not have the same analysis logic as the newly added flow log, writing a new Groovy script, carrying out Base64 encoding on the Groovy script, and newly inserting the script encoded by Base64 into a MongoDB plug-in table; at this time, a newly added plug-in message can be issued based on the Redis, the plug-in central control service can subscribe the message to the Redis based on the pre-injected message receiving component to receive the message issued by the Redis, and then when the newly added plug-in message is received by the plug-in central control service, the script content of the new plug-in can be obtained through the MongoDB storage component, and the Groovy script content is compiled to construct a new plug-in, and then the newly constructed plug-in is injected into the Spring Ioc container, and the record of the plug-in binding table is added, so that the newly added plug-in is realized.

In an optional embodiment, if it is determined that there is an existing plugin meeting the parsing rule, that is, it is determined whether the new flow log has the same logic in each plugin of the Spring Ioc container, multiplexing may also be directly completed through newly adding binding information in the binding table.

In practical applications, the specific judgment manner may be realized by human understanding judgment of the program, rather than automatic judgment and determination of the application program.

Take the following three real logs as an example:

[1] log example 1

[2021-05-2718:28:21] [ DropItem ], { "camp _ id": "," item _ score ":0," fresh ": null," roll _ name ": test 14", "server":00, "app _ channel": "app _ store", "new _ schedule":0, "drill _ levels": 2,5, "" sys _ id ":1," left _ yuanbao ":0," old _ access _ count ": xxx", "count":1, "occuvision": 0, "roll _ level":1, "item _ attr _ str" { 'user _ time': 1, "ad _ r _ ad _ id":33, "xxx _ scan": 0, "xxx _ id": 0, "xxx _ id _ 1," xxx _ ad _ id "{" 0 ':0, "0' {" update _ id "{" 0 ': 0' { "update _ id", "acquisition _ level":0, "item _ id":35232, "item _ id": xxx "," camp _ age ":1," account _ id ": xxxxxxxx", "left _ free _ yuanbao":0, "cordinate": 47237.61181640625,674.183349609375, -1791.460205078125 }

[2] Log example 2

％Beast(2096,7)

i(iType 2096,iLife 8000,pCor 20,pMag 20,pStr 20,pRes 20,pDex 20,iDamage 33,iMagDam 27,iDefense 49,iMagDef 36,iSpeed 39,iHp 141,iHp_Max 141,iMp_Max 101,iMp 101,iFengDodge 1,iFengAttack 1,iPos 7,isSummon 1,iOwner 294213727,bbtype 2,qCor 4252,qDam 756,qMagDam 2016,qDef 1323,qSpe 630,iGrowUp 1000,iPoint 100,pCorBorn 20,pMagBorn 20,pStrBorn 20,pResBorn 20,pDexBorn 20,createTime 1617735103,iMark 1831,DAYNO_iIgnoreSkillConfm 2736,iLowDrugCellCntVer 1)

c (cName large sea turtle, cGBLKe qqqN 0000W9vu, cBornSkill 419|417|525|403|404)

O(SkillOrder({"419","417","525","403","404",})

mxBeastLowDrugCell([0:0,])

mxBeastSeniorDrugCell([0:0,])

mxBeastUniqueDrugCell([0:0,])

[3] Log example 3

[2021-06-2800:57:37][ObtainRecipe],{"camp_id":"W91HNyJg6y2RZmLg","recipe_id":1001 83,"role_name":"ss","server":600247,"app_channel":"app_store","new_schedule":2,"skill_levels":[105,105,104],"left_yuanbao":209,"old_accountid":"aebfxdtwfu4mz63y@ios.netease.win.163.com","occupation":12,"role_level":20,"role_id":"W9w9xr/glGBBTvh6","scene_id":127,"user_no":12242835,"os_name":"ios","occupation_level":9,"camp_age":8,"account_id":"aebfxdtwfu4mz63y@ios.netease.win.163.com","left_free_yuanbao":145}

The judgment rule based on the manual program understanding is mainly to judge according to the format of the log, and the log example [1] [2] [3] is combined, wherein the format of the log of the two logs [1] [3] both meets the format of the [ time ] [ log name ] and the log content JSON serialized character string, so that the two logs [1] [3] can be judged to have the same structure and can be multiplexed with the same log analysis plug-in, and the format of the log example [2] does not meet the format of the [ time ] [ log name ] and the log content JSON serialized character string', so that the log analysis plug-in which is the same as the log [1] [3] cannot be multiplexed.

Step 402, determining a plug-in identifier of the log information, and acquiring corresponding plug-in script data from a preset database according to the plug-in identifier;

in an embodiment of the present invention, after the log parsing program is run to obtain the log information and update the log parsing plug-in, the running log parsing program may be used to obtain plug-in script data that can be matched with the parsing format of the log information, so as to parse the log information.

The plug-in script data can be script data which is updated to a preset database based on the fact that the structure of the log information is changed or the newly added log information is accessed, so that when the structure of the game log is changed or the newly added game log is accessed, the game log can be analyzed in a differentiated mode only by updating the modified or newly added script data to the database.

When the log is analyzed, the corresponding analysis format of the log can be mainly obtained through the plug-in identifier bound to the log for analysis, and firstly, the plug-in for analyzing the log needs to be determined.

The plug-in mode can be determined through a plug-in binding table. The acquired log information has a recovery flow identifier, and the plug-in binding table in the storage layer mainly has two fields of a flow ID (i.e., the recovery flow identifier) and a plug-in ID (i.e., the plug-in identifier), which can be used to maintain the binding relationship between the flow (which refers to the specific game misoperation business flow in the misoperation recovery program) and the plug-in, and at this time, the plug-in binding table for recording the recovery flow identifier and the plug-in identifier can be acquired, and the plug-in identifier having the binding relationship with the recovery flow identifier is searched in the plug-in binding table.

In an optional embodiment, after determining a plug-in for log parsing, the log parsing needs to be implemented by running corresponding plug-in script data, and for an obtaining manner of the plug-in script data, the obtaining manner may be obtained through a plug-in list located in the storage layer.

Specifically, the plug-in list located in the storage tier may have four fields, namely, a plug-in ID, a plug-in name, a plug-in status, and a plug-in script content, where the plug-in ID field may be used as the name of a plug-in to be injected when the plug-in is injected into the Spring Ioc container, and is used for binding with the process ID of the log information in the plug-in binding table; the name of the plug-in can be used for conveniently managing the plug-in so as to know the specific use of the plug-in; the plug-in state is used for identifying the enabling state of the plug-in, and hot plug of the plug-in can be controlled through the field (namely enabling and closing of the plug-in are realized under the condition of not closing the running program); the plug-in script content can be used for storing the script after being coded by the Base64, the Base64 coding can enable the script to be more beneficial to network transmission and has non-readability, and the safety of script data can be improved compared with original text storage. Then, a plug-in list for recording the plug-in identifier and the plug-in script content may be obtained from the preset database, plug-in information corresponding to the plug-in identifier may be determined in the plug-in list, and plug-in script data corresponding to the plug-in information may be searched.

And 403, executing the plug-in script data to analyze the log information to obtain analysis content, so as to implement a recovery process of the log related to the misoperation based on the analysis content.

In an embodiment of the invention, an analysis method of an analysis format corresponding to the plug-in identifier can be searched through the plug-in central control service, then the analysis method is adopted to reflect and execute the plug-in script data, and the log information is analyzed to obtain analysis content when the plug-in script data is executed, so that the recovery process of the log related to the misoperation is realized through the misoperation recovery program.

In practical applications, the log parsing process can be implemented by using the capability of executing the plug-in, and is mainly implemented by using an ApplicationContext based on a Spring Ioc container, where the ApplicationContext refers to a sub-interface of a Spring Bean factory, also referred to as a Spring context, and the interface can provide a getteam method, and the method can obtain the plug-in the Spring Ioc container based on a plug-in ID, so that a unified method of the plug-in can be called after the plug-in is obtained, that is, the plug-in can be executed.

The method comprises the steps that various plugins injected into a Spring Ioc container can be generated through compiling of a plugin layer, so that a plugin central control service can obtain corresponding plugins when executing, a uniform template class can be defined, the template class provides a chase method, namely all plugins need to be generated by the template class (generated by a template class construction method), and the plugins can be executed by the uniform method chase after the plugin central control service is realized. The template class can also contain a Script field of the Groovy Script type, the field can be used for storing the Groovy Script compiled by the plug-in central control service, and in specific implementation, the parse method of the template class can use reflection technology to execute the content of the Script field.

In order to enable those skilled in the art to further understand the application process of the log parsing program proposed in the embodiment of the present invention, the following description is made with reference to an execution diagram and related application scenarios:

in the embodiment of the present invention, the log parsing program may be integrated into an error recovery application for operation.

Referring to fig. 5, which is a schematic diagram illustrating an execution of a misoperation recovery program in an embodiment of the present invention, a game developer may record a game log related to game operation during a game development process, so as to know an operation situation of the game based on the recorded game log, and find and process an emergency in the game according to the game log; if the player performs misoperation in the process of game running, for example, the player can initiate a misoperation recovery application to related game customer service personnel if the player performs misoperation, such as the faulty decomposition recovery of equipment in the game, the faulty purchase recovery of properties in a mall in a game, and the like, the player can analyze the log information generated after the misoperation through a faulty operation recovery program which comprises a log analysis program in a logic manner to obtain the analysis content (namely faulty operation data) of the faulty operation log information, and the faulty operation can be recovered based on the obtained faulty operation data.

When a game research and development worker actively updates a game, namely the format of the log information is changed, the log to be updated can be returned to the log analysis program and written into the newly-added log information, the game log can be artificially notified to be changed at the moment, the log analysis program can be thermally updated at the moment, whether a plug-in which has the same analysis logic as the newly-added log information and is used for analyzing the log exists can be specifically judged, and if the plug-in exists, the game research and development worker can modify the binding table to directly complete multiplexing through the newly-added binding information; if not, the corresponding plug-in is needed to change the script logic or write corresponding script logic data to realize the addition of the plug-in, and the binding relationship between the changed or added plug-in and the added log information is established.

Referring to fig. 6, which is a schematic diagram illustrating an execution of a log parsing program according to an embodiment of the present invention, the log parsing program may be integrated in an incorrect operation recovery application, and the log parsing program may be actively triggered by the incorrect operation recovery program, and after the log parsing program completes a relevant log parsing logic, may return an parsing content to the incorrect operation recovery program, so that the incorrect operation recovery program may continue a chinning data recovery process after receiving the parsing content.

Specifically, if a player performs a misoperation in the running process of the game, for example, the player performs a misoperation such as the misoperation decomposition recovery of equipment in the game, the misoperation recovery of a mall property in a game, and the like, the player can initiate a misoperation recovery application to a relevant game customer service staff, the player can specifically submit a work order or dial a customer service hotline in the game, the misoperation recovery program can search a log generated by the misoperation in the corresponding game according to information provided by the player through an HTTP network request or a content sharing mode, at this time, the log analysis program can obtain the stored log information from the storage layer and the data layer through a log downloader of the service layer and return the log information to the log analysis program, and the analysis content obtained by analysis can be returned to the misoperation recovery program after executing the logic of the relevant plug-in to analyze the log information, the misoperation recovery program can execute a subsequent misoperation recovery process, specifically, the data generated by the misoperation is deleted, the data lost due to the misoperation is recovered, and the recovery of the misoperation data is completed.

As an example, a player may have a mind mistakenly returning an mind, at which point the player may contact a customer service requesting recovery of the mind, in the misoperation recovery program, when a player submits an application/request for recovering spirit, the player can simultaneously submit the related information of the spirit false return, including account information, role information, spirit type information, misoperation time and the like, at the moment, the misoperation recovery program can trigger and start a log analysis program, the log analysis program can firstly search the game log of the player in a data layer through a log downloader according to the information provided by the player, under the condition that the game log of the related god is found and the log is not recovered, the god can be recovered and the player can be correspondingly recovered, such as "XX god has recovered, now in the no-return state", etc., to inform the player that the false return operation of the formula god has completed the recovery process.

In the process of recovering the formula spirit, firstly, under the condition that the searched log is judged not to be recovered, the log analysis program can analyze the searched log and return the analysis content to the misoperation recovery program, the misoperation recovery program can inquire the prop content generated when the player type spirit returns by mistake, judge whether enough props exist in the current backpack content of the player to be deducted according to the prop content generated when the formula spirit returns by mistake, if the prop content exists, directly deduct the prop data generated when the player type spirit returns by mistake, recover the formula spirit of the player type spirit returning by mistake, and reply the player after recovery is completed. It should be noted that, differences exist for different games and different misoperation recovery flows, and the embodiment of the present invention is not limited to this.

It should be noted that, for simplicity of description, the method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the illustrated order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments of the present invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Referring to fig. 7, a block diagram of a log parsing apparatus according to an embodiment of the present invention is shown, which may specifically include the following modules:

a log analysis program running module 701, configured to run a log analysis program to obtain log information generated based on an incorrect operation;

a plug-in script data obtaining module 702, configured to determine a plug-in identifier of the log information, and obtain corresponding plug-in script data from a preset database according to the plug-in identifier; the plug-in script data is script data which is updated to a preset database based on the change of the structure of the log information or the access of newly added log information;

and the log information analysis module 703 is configured to execute the plug-in script data to analyze the log information to obtain analysis content, so as to implement a recovery process for the log related to the misoperation based on the analysis content.

In an embodiment of the present invention, the log information has a recovery procedure identifier; the plug-in script data acquisition module 702 may include the following sub-modules:

In one embodiment of the present invention, the plug-in script data acquisition module 702 may include the following sub-modules:

In one embodiment of the invention, the log parser provides a plug-in central control service; the log information parsing module 703 may include the following sub-modules:

In an embodiment of the present invention, the apparatus may further include the following modules:

In one embodiment of the present invention, the plug-in script data update module may include the following sub-modules:

In one embodiment of the invention, the log parser has a plug-in central service; the first plug-in script data update submodule may include the following elements:

In one embodiment of the invention, the log parser has a plug-in central service; the second plug-in script data update submodule may include the following elements:

In an embodiment of the present invention, the log parsing program has a plug-in central service, and before the log parsing program is run, the apparatus may further include the following modules:

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

An embodiment of the present invention further provides an electronic device, including:

the log analysis method comprises a processor, a memory and a computer program which is stored on the memory and can run on the processor, wherein when the computer program is executed by the processor, each process of the log analysis method embodiment is realized, the same technical effect can be achieved, and the details are not repeated here to avoid repetition.

The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements each process of the log analysis method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.

The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.

Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or terminal that comprises the element.

The above detailed description is given to a log parsing method and a log parsing apparatus provided by the present invention, and a specific example is applied in the present document to explain the principle and the implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims

1. A method of log parsing, the method comprising:

2. The method of claim 1, wherein the log information has a recovery flow identification; the determining the plug-in identifier of the log information includes:

3. The method according to claim 1, wherein the obtaining corresponding plug-in script data from a preset database according to the plug-in identifier comprises:

4. The method of claim 1 or 3, wherein the log parser provides a plug-in central control service; the executing the plug-in script data to analyze the log information to obtain analysis content includes:

5. The method of claim 1, further comprising:

6. The method according to claim 5, wherein the updating the plug-in script data in the preset database and establishing a binding relationship between the updated plug-in script data and the log information comprises:

and updating the modified plug-in script data to a plug-in list of the preset database so as to realize the replacement of the plug-in.

7. The method of claim 6, wherein the log parser has a plug-in central service; the updating the modified plug-in script data to the plug-in list of the preset database includes:

8. The method according to claim 5, wherein the updating the plug-in script data in the preset database and establishing a binding relationship between the updated plug-in script data and the log information comprises:

and updating the written plug-in script data to a plug-in list of the preset data table, and establishing a binding relationship between the written plug-in script data and the log information so as to realize the addition of the plug-ins.

9. The method of claim 8, wherein the log parser has a plug-in central service; the updating the written plug-in script data to the plug-in list of the preset data table comprises:

10. The method of claim 1, wherein the log parser has a plug-in central service, and further comprising, prior to running the log parser:

11. An apparatus for log parsing, the apparatus comprising:

12. An electronic device, comprising: processor, memory and computer program stored on the memory and being executable on the processor, the computer program, when executed by the processor, implementing the steps of the log parsing method according to any of claims 1-11.

13. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the log parsing method according to any one of claims 1 to 11.