CN113468115A

CN113468115A - Log processing method, device, equipment and medium

Info

Publication number: CN113468115A
Application number: CN202110833789.9A
Authority: CN
Inventors: 赖文星
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-07-23
Filing date: 2021-07-23
Publication date: 2021-10-01

Abstract

The embodiment of the application provides a log processing method, a log processing device, log processing equipment and a log processing medium, wherein the method comprises the following steps: receiving a log query request; searching the registration information of the target log set based on the log set identification of the target log set; determining a target query condition according to the log query request and the access authority information of the target log set, and acquiring at least one candidate log from the target log set according to the target query condition; and determining a registration processing requirement according to the log query request and the registration processing information of the target log set, and processing at least one candidate log according to the registration processing requirement to obtain a log query result. By adopting the method and the device, the flexibility of managing the access authority information and registering the processing information can be improved in the log query process.

Description

Log processing method, device, equipment and medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a log processing method, apparatus, device, and medium.

Background

The platform often comprises a plurality of services (or service systems), and each service needs to log relevant operations executed on the service; therefore, it is important to query the log to display the log or to provide other services with query to generate alarm information.

In the prior art, a set of query service is developed for each service one to one based on information such as authentication requirements, query conditions and the like of each service; when a user needs to query the log of any service, the log query can be realized by adopting the query service developed for the service. Therefore, the query process of the log under each service is fixed, and the query services under different services are difficult to multiplex, so that the log query process is not flexible enough. How to improve the flexibility of the log query process becomes a hot topic for research.

Disclosure of Invention

Embodiments of the present application provide a log processing method, an apparatus, a device, and a medium, which can improve flexibility of managing access right information and registering processing information in a log query process.

In one aspect, an embodiment of the present application provides a log processing method, where the method includes:

receiving a log query request, wherein the log query request comprises a log set identifier of a target log set requested to be queried;

searching the registration information of the target log set based on the log set identifier of the target log set, wherein the registration information of the target log set comprises the access authority information and the registration processing information of the target log set;

determining a target query condition according to the log query request and the access authority information of the target log set, and acquiring at least one candidate log from the target log set according to the target query condition;

determining a log query request and the registration processing information of the target log set, and processing at least one candidate log according to the registration processing request to obtain a log query result

In another aspect, an embodiment of the present application provides a log processing apparatus, where the apparatus includes:

the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for receiving a log query request which contains a log set identifier of a target log set requested to be queried;

the processing unit is used for searching the registration information of the target log set based on the log set identifier of the target log set, and the registration information of the target log set comprises the access authority information and the registration processing information of the target log set;

the processing unit is further used for determining a target query condition according to the log query request and the access authority information of the target log set, and acquiring at least one candidate log from the target log set according to the target query condition;

and the processing unit is further used for determining a registration processing requirement according to the log query request and the registration processing information of the target log set, and processing at least one candidate log according to the registration processing requirement to obtain a log query result.

In one implementation, the log query request further includes a query processing requirement; a processing unit further to:

processing the log query result according to the query processing requirement to obtain a log query response result;

and returning the log query response result to the querying party.

In one implementation mode, the access right information comprises at least one access right, authorized user information corresponding to each access right, and query condition information under each access right; the log query request also comprises identity information of a query party and candidate query conditions; the processing unit is configured to, when determining the target query condition according to the log query request and the access right information of the target log set, specifically:

determining target access authority possessed by the inquirer according to the identity information of the inquirer and authorized user information corresponding to each access authority;

generating a registration query condition based on query condition information under the target access authority;

and determining the registration query condition and the candidate query condition as target query conditions.

In one implementation, the query condition information under the target access right defines a query rule; the processing unit is configured to, when generating the registration query condition based on the query condition information under the target access right, specifically:

and determining the query rule in the query condition information as the registration query condition.

In one implementation, the query condition information under the target access right includes a first code segment, and the first code segment defines a query function; the processing unit is configured to, when generating the registration query condition based on the query condition information under the target access right, specifically:

determining the log query request as input information of a query function;

and calling the first code segment to run a query function, and processing the log query request to obtain a registration query condition.

In one implementation, the query condition information under the target access authority includes a condition management interface provided by a registrant of the target log set; the processing unit is configured to, when generating the registration query condition based on the query condition information under the target access right, specifically:

calling a condition management interface to send a log query request to a registrant of a target log set;

and receiving a registration inquiry condition returned by the registrant through the condition management interface.

In one implementation mode, the registration processing information includes at least one registration processing authority, authorized user information corresponding to each registration processing authority and processing information under each registration processing authority; the log query request also comprises identity information of a query party; the processing unit is configured to, when determining a registration processing requirement according to the log query request and the registration processing information of the target log set, specifically:

determining target registration processing authority possessed by the inquirer according to the identity information of the inquirer and authorized user information corresponding to each registration processing authority;

a registration processing request is generated based on processing information under the target registration processing authority.

In one implementation, the processing information under the authority of target registration processing defines a registration processing rule; the processing unit is configured to, when generating a registration processing request based on processing information under a target registration processing authority, specifically:

the registration processing rule defined by the processing information is determined as a registration processing requirement.

In one implementation, the processing information under the authority of target registration processing includes a second code segment, and the second code segment defines a registration processing function; the processing unit is configured to, when generating a registration processing request based on processing information under a target registration processing authority, specifically:

determining the log query request as input information of a registration processing function;

and calling the second code segment to run a registration processing function, and processing the log query request to obtain a registration processing requirement.

In one implementation, the processing information under the authority of target registration processing includes a registration processing interface provided by a registrant of the target log set; the processing unit is configured to, when generating a registration processing request based on processing information under a target registration processing authority, specifically:

calling a registration processing interface to send a log query request to a registrant of a target log set;

and receiving a registration processing requirement returned by the registrant.

In one implementation, the processing unit is further configured to:

receiving a log editing request sent by a registrant of a target log set, wherein the log editing request is used for requesting to edit the registration information of the target log set;

responding to the log editing request to execute editing operation on the registration information of the target log set to obtain updated registration information of the target log set;

wherein the editing operation comprises: one or more of add operations, modify operations, and delete operations.

In one implementation, the registration information of the target log set is stored in a log set storage space, and the log set storage space stores the registration information of a plurality of log sets; each log set is used for recording logs generated in one service system; the processing unit is configured to, when receiving the log query request, specifically:

calling a log query service interface to receive a log query request;

the log query service interface is adapted to a plurality of service systems, and is used for receiving a query request of a log set corresponding to any one service system.

In another aspect, an embodiment of the present application provides a log processing apparatus, including:

a processor adapted to execute a computer program;

a computer readable storage medium, in which a computer program is stored, which, when executed by a processor, implements a log processing method as described above.

In another aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer program is suitable for being loaded by a processor and executing the log processing method as described above.

In another aspect, embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the log processing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the log processing device executes the log processing method.

In the embodiment of the application, a target query condition can be determined based on a log query request sent by a query party and registration information of a target log set registered by a registration party, and at least one candidate log can be queried by adopting the log query request; and determining a registration processing requirement based on the log query request and the registration processing information, and processing at least one candidate log according to the registration processing requirement to obtain a log query result. In the above scheme, the target query condition for querying the at least one candidate log is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing the at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the flexibility of managing access rights and processing requirements in the log query process is improved, the requirement of dynamically screening and processing the candidate logs according to different query requests can be met, and the log query efficiency is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

FIG. 1 is a block diagram illustrating an architecture of a log processing system according to an exemplary embodiment of the present application;

FIG. 2 is a flowchart illustrating a log processing method according to an exemplary embodiment of the present application;

FIG. 3 is a diagram illustrating a log query request provided by an exemplary embodiment of the present application;

FIG. 4 is a diagram illustrating registration information for a target log set, provided by an exemplary embodiment of the present application;

FIG. 5 is a flowchart illustrating a method for determining a target query condition based on a log query request and access right information of a target log set according to an exemplary embodiment of the present application;

FIG. 6 is a flowchart illustrating a query of at least one candidate log based on target query criteria according to an exemplary embodiment of the present application;

FIG. 7 is a flowchart illustrating a method for determining a requirement for registration processing according to a log query and registration processing information of a target log set according to an exemplary embodiment of the present application;

FIG. 8 is a flowchart illustrating processing of at least one candidate log according to a registration processing requirement to obtain a log query result according to an exemplary embodiment of the present application;

FIG. 9 is a flowchart illustrating a log processing method according to an exemplary embodiment of the present application;

FIG. 10 is a flow diagram illustrating a method for determining target query terms and target processing requirements according to an exemplary embodiment of the present application;

FIG. 11 illustrates a schematic diagram of an ordered display log provided by an exemplary embodiment of the present application;

fig. 12 is a schematic structural diagram illustrating a log processing apparatus according to an exemplary embodiment of the present application;

fig. 13 is a schematic structural diagram illustrating a log processing device according to an exemplary embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The embodiment of the application relates to a log (or called a log file), wherein the log is a text object formed by a plurality of unordered key-value pairs; for example, the log may include, but is not limited to: JSON objects, any data row in any data table in a database (such as a relational database), data generated by recording related operations (such as access operation, deletion operation, browsing operation and the like) generated in the running process of a platform (or an operating system, an application program, a network device and the like), and the like. Each key-value pair in the log is also referred to as a tag, where the key is referred to as a tag name and the value is referred to as a tag value; representations of tag values may include, but are not limited to: text, numbers, objects, arrays (e.g., arrays including text, numbers, or objects). It should be noted that the log does not necessarily display the tag name of each tag, that is, the tag names of the tags included in the log may be the same, and at this time, the tag name may be displayed at only one position of the log file; for example, a log of related operations of any one of the data tables in the relational database is recorded, and the tag name of each tag thereof can be implicitly specified by the header of the any one of the data tables.

The above mentioned platforms may include, but are not limited to: application platforms, network platforms, machine learning platforms, and the like; the Machine Learning platform is also called an intelligent Machine Learning platform (or an intelligent titanium platform), and is a Learning platform providing a full-process training capability for Machine Learning (ML). Machine learning is a multi-field cross subject and relates to a plurality of subjects such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like; the special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.

The platform often includes different services (or called service systems), each service may correspond to one or more log sets, and any log set includes logs related to the service; for example, when a target service (such as any service) accesses 5 data sources, if a registrar wishes to expose 3 data sources of the 5 data sources, 3 log sets can be created, that is, the target service corresponds to the 3 log sets; the number of log sets corresponding to the service is not limited in the embodiments of the present application, and is described here. For example, the machine learning platform includes a training management service (or referred to as a training service) of a network model (such as a neural network model), a management service (or referred to as an inference service) of model inference, a rights management service, a platform management service, and the like. Wherein, the log set corresponding to the training service may include: and recording a log of error information in the network model training process. The log set corresponding to the inference service may contain: and recording a log of interface calling information in the network model reasoning process. The log set corresponding to the platform management service may include: logs of user operations (e.g., create operations, delete operations, change operations, etc. on items) are recorded.

Currently, a query service, which is separately developed for each service by a developer, may be used to query logs under different services. For example: the platform comprises 3 services, and then 3 query services which are independent of each other need to be developed for the 3 services; when a querier wants to query logs under a target service (e.g., either service), the querier can invoke a query service developed for the target service to query. It is easy to find that, because the query authority requirements (such as query conditions, authentication requirements, etc.) and the processing logic (such as post-processing requirements, etc.) of the logs of each service are different and the implementation logic difference is large, the query services (or query modules) developed for each service are difficult to reuse, and the query logic provided for the query services developed for each service is fixed, so that the flexibility of log query is low and the efficiency of log query is reduced. Based on this, the embodiment of the present application provides a log processing scheme, which decouples log query, access right for setting log query, and processing requirement, to provide a set of total log query service for log query under different services, where the log query service supports query of logs under different services; moreover, registrars of different services only need to appoint or realize how to obtain the access authority and the processing requirement of the log, so that the registrars of different services can be prevented from repeatedly developing log query services only because the access authority and the processing requirement are different in logic; moreover, a set of total log query service is adopted to query logs under different services, so that the log query efficiency can be improved.

The log query process of the log processing scheme provided by the embodiment of the application roughly comprises the following steps: receiving a log query request sent by a query party, wherein the log query request comprises a log set identifier of a target log set which is requested to be queried, and the target log set corresponds to a service system (or service); according to the log set identification, the registration information of the target log set can be found, and the registration information is obtained by registering of a registrant of the target log set; then, target query conditions can be obtained according to the log query request and the registration information of the target log set, and at least one candidate log can be obtained by using the target query conditions for query; and determining a registration processing requirement based on the log query request and the registration processing information of the target log set, and processing at least one candidate log by adopting the registration processing requirement to obtain a log query result. In the above scheme, the target query condition for querying the candidate logs is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the flexibility of managing access rights and processing requirements in the log query process is improved, the requirement of dynamically screening and processing the candidate logs according to different query requests can be met, and the log query efficiency is improved.

In order to better understand the log processing scheme provided in the embodiment of the present application, a log query scenario is described in detail below with reference to the log processing system shown in fig. 1; as shown in fig. 1, the log processing system may include a query terminal 101, a computer device 102, and a data source 103, and the embodiment of the present application does not limit names and numbers of the devices (such as the query terminal 101, the computer device 102, and the data source device 103) included in the log processing system. All devices included in the log processing system can be in direct or indirect communication connection in a limited or wireless mode, and the communication mode among the devices is not limited in the embodiment of the application; for example: the communication means between devices may include, but is not limited to: the method adopts the modes of HTTP request, Remote Procedure Call (Remote Procedure Call), Socket (Socket), content sharing and the like. The following briefly introduces various devices included in the log processing system, wherein:

the query terminal 101 may be a terminal device used by the querying party to query the log, and the terminal device may include but is not limited to: smart phones (such as Android phones, iOS phones, and the like), tablet computers, portable personal computers, Mobile Internet Devices (MID), smart televisions, vehicle-mounted Devices, head-mounted Devices, and the like. Alternatively, if the log of the desired program is an application log, an application having a function of querying the application log is deployed in the query terminal 101, and when the application is opened and used, a log query request for requesting to query the log may be generated.

The computer device 102 is a device for executing log query, and specifically, a log query service is deployed in the computer device, and the log query service can be used to implement log query. Among other things, computer devices may include, but are not limited to: terminal devices such as personal computers, smart phones, tablets, laptops, desktops, edge computing devices, embedded devices, etc.; or a service device such as a data processing server, a web server, an application server, etc. The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, a cloud server providing basic cloud computing service, or a node server on a block chain.

Data source 103, which is a device for storing logs, data source 103 may include, but is not limited to: database management systems (e.g., relational, non-relational), full-text search engines, text files, cloud storage, cloud databases, blockchain-based data storage, and the like. The distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system which integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network through application software or application interfaces to cooperatively work through functions such as cluster application, grid technology, distributed storage file system and the like, and provides data storage and service access functions to the outside. A cloud Database (Database), which can be regarded as an electronic file cabinet, in short, a place for storing electronic files, and a user can add, query, update, delete, etc. data in the files. A "database" is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.

The following briefly introduces the log query process by taking the log processing scheme executed by the query terminal 101, the computer device 102, and the data source 103 as an example: first, an inquiring party with log query requirements can send a log query request from the querying device 101 to the computer device 102 by way of a network connection. Secondly, after acquiring the log query request, the computer device 102 may search the registration information of the target log set from the log set storage space based on the log query request, where the registration information is obtained by registering a registrant of the target log set, and the registration information includes access right information and registration processing information; the computer device 102 determines a target query condition based on the access right information of the target log set and the log query request, and sends the target query condition to the data source 103 so as to pull at least one candidate log matching the target log query condition from the data source 103. Secondly, the computer device 103 determines a registration processing requirement based on the registration processing information and the log query request, and then processes at least one candidate log according to the registration processing requirement to obtain a log query result. Of course, if the log query request includes the candidate processing requirement, the computer device may further process the log query result according to the candidate processing requirement to obtain the log query response result. Finally, the computer apparatus 102 returns the log query result (or log query response result) to the querying apparatus 101. Of course, if the log query request is initiated locally by the computer device 102, the computer device 102 displays the log query result (or log query response result) in the display screen.

It should be noted that (i) the log set storage space shown in fig. 1 is located in the computer device 102, that is, the registration information of the log set (e.g., the target log set) is stored in the computer device 102; it is understood that the log set registration information may also be stored in other storage space independent from the computer device 102, such as a cloud database, a block chain-based database, a database, etc. The number of the data sources 103 shown in fig. 1 is 1, that is, log sets under different services are stored in the data sources 103; however, it should be understood that the number of the data sources 103 may be multiple, each data source 103 may be configured to store a log set under one or more services, and the storage manner of the log set is not limited in the embodiment of the present application. The computer device 102 and the query device 101 may be the same device, and in this implementation, the log query request received by the computer device 102 is initiated locally; the computer device 102 and the data source 103 may also be the same device, and in this implementation, the computer device 102 may query the at least one candidate log by local access. The computer device 102 may also receive operation requests for registration, modification, deletion, and the like of the log set from local or through network access, so as to register and update the registration information of the log set.

Based on the above-described log processing scheme, a more detailed log processing method is provided in the embodiments of the present application, and the following describes in detail the log processing method provided in the embodiments of the present application with reference to the accompanying drawings.

FIG. 2 is a flowchart illustrating a log processing method according to an exemplary embodiment of the present application; the log processing method may be performed by the above mentioned computer device, and may include, but is not limited to, steps S201 to S204:

s201, receiving a log query request.

As described above, in the embodiment of the present application, a set of total log query services is provided to query logs under different services, and then an implementation manner of receiving a log query request may include: and calling a log query service interface to receive the log query request. The log query service interface is provided by a log query service and is an interface (such as an HTTP interface, a remote system call interface, a message queue, a shared memory, a cloud function, etc.) adapted to a plurality of service systems (or services); in other words, the log query service interface of the log query service can be used to receive a query request for a log set corresponding to any service system included in the platform. Compared with the method and the system for developing the log query service for each service system, the method and the system for developing the log query service support the unified log query service interface for the front-end page, the application program and the like with the log query function, further avoid redundant interface adaptation work and reduce the work load of interface adaptation.

The log query request may include a log set identifier of a target log set requesting a query, where the log set identifier of the target log set may be used to uniquely identify the target log set, and the log set identifier may include, but is not limited to: and the name of the service (or service system) corresponding to the target log set can be found according to the log set identifier. Of course, the log query request may include, in addition to the log set identifier of the target log set, the following: firstly, meta information is requested; if the meta information is requested, it may include: application category information, authentication key, identity information of the inquiring party (such as identity level information), authority information, and IP address, etc. The type information of the application program can refer to the type of the running environment of the application program which generates the log query request; for example, if the target application (e.g., IM (Instant Messaging) application, content interaction application, etc.) has a function of log query, the querying party may open and initiate a log query request using the target application, so that the category information of the target application may include: web page versions of the target application, cell phone versions of the target application, and so on. The application programs are classified, because the inquired content rights of the same application program are different when the same application program belongs to different types; for example, a cell phone version of a target application may support the return of logs recorded for social dynamic information (such as a circle of friends), while a web page version of the target application may not support the return of logs recorded for social dynamic information. The method for distinguishing the types of the application programs can improve the log query safety, ensure that the logs are not leaked and improve the log storage safety.

Candidate query conditions; the candidate query is a filter condition for filtering logs in the target set of logs. For example, the candidate query conditions are: the last 30 minutes indicates that the inquirer wants to inquire the log within a 30-minute time period before the time of initiating the log inquiry request. The query processing requirement is a processing requirement for processing the queried log (i.e. the candidate log mentioned above) or the information of the log. For example, the query processing requirements are: if 20 characters are reserved at most for the label value of the label name "detailed XX" in the log, after the log containing the label name "label XX" is inquired, the log needs to be processed, so that the characters contained in the label value corresponding to the label name of the log are deleted to 20 characters at most. The aggregation condition is used for aggregating the inquired logs; for example, calculating an average value of tag values under a certain tag in the queried log; the following steps are repeated: if the inquired logs all contain the label 'region', the inquired logs are aggregated, and the number of the logs with the label value of 'X region' in the label 'region' can be counted; and so on. A schematic diagram of the content contained in an exemplary log query request can be seen in fig. 3.

S202, searching the registration information of the target log set based on the log set identification of the target log set.

The registration information of the target log set comprises the access authority information and the registration processing information of the target log set; of course, the registration information of the target log set may also include meta information of the target log set (e.g., log set identifier of the target log set, etc.) and access information of the data source (e.g., data source address information, key, etc.) that is the storage space storing the logs of the target log set. An exemplary schematic diagram of the registration information of the target log set can be seen in fig. 4, and as shown in fig. 4, the registration information of the target log set includes access right information 401, registration processing information 402, meta information 403 of the target log set, and access information 404 of the data source.

The registration information of the target log set may be stored in a log set storage space, which may be a local storage space of the computer device or a storage space independent of the computer device for storing the registration information of the log set. The log set storage space stores the registration information of a plurality of log sets, and each log set is used for recording logs generated in one service system. In other words, the log set storage space stores the registration information of the log set corresponding to all or part of the service systems under the platform, so that the registration information of the log set corresponding to any service system under the platform can be pulled from the log set storage space through the unified log query service interface.

S203, determining a target query condition according to the log query request and the access authority information of the target log set, and acquiring at least one candidate log from the target log set according to the target query condition.

The access right information of the target log set is information for determining a filtering condition (or referred to as a query condition) for filtering the target log set to obtain the candidate log. The access right information comprises at least one access right, authorized user information corresponding to each access right information and query condition information under each access right. One exemplary access rights information may include: 1. if the log query request contains an effective authentication key issued by a target service (such as any service), all logs in a target log set can be queried; 2. the administrator for project X may view all logs with a tag name of "tag name X1" and a tag value of "tag value X2". Then it is determined that the access right information of the target log set includes 2 types of access rights: the access right 1 is '1', the log query request contains an effective authentication key issued by a target service (such as any service), all logs in the target log set can be queried, the authorized user information corresponding to the access right 1 is 'a querier with the effective authentication key issued by the target service (such as any service'), and the query condition information under the access right 1 is 'querying all logs in the target log set'. The administrator with the access right 2 being "2 and the item X can view all logs with the tag name" tag name X1 "and the tag value" tag value X2 ", the authorized user information corresponding to the access right 2 is" administrator of the item X ", and the query condition information under the access right 2 is" view all logs with the tag name "tag name X1" and the tag value "tag value X2".

The candidate query condition is a filtering condition set by the querier for filtering the logs in the target log set to query the candidate logs which the querier wants to query. The following briefly introduces an implementation process for determining a target query condition according to a log query request and access right information of a target log set with reference to fig. 5: firstly, determining the target access authority possessed by the inquiring party according to the identity information of the inquiring party and the authorized user information corresponding to each access authority. Based on step S201, it can be known that the target log query request includes identity information of the querying party, where the identity information may include but is not limited to: user nicknames, user rating information, authentication keys, etc. Continuing with the above example, assuming that the identity information in the log query request sent by the querying party indicates that the querying party is an administrator of the item X, based on the identity information of the querying party and the 2 kinds of access permissions corresponding to the target log set, it can be known that the target access permission possessed by the querying party is access permission 2. And generating registration inquiry conditions based on the inquiry condition information under the target access authority. The registration query condition is a filtering condition used for filtering logs in the target log set to obtain at least one candidate log; according to different contents contained in the query condition information, the manner of generating the registration query condition based on the query condition information is different. For example: if the query condition information includes the rule shown in the above example, the rule is directly determined as the registration query condition; the following steps are repeated: the query condition information comprises the code segments, the result obtained by inputting the log query request into the code segments can be used as the registration query condition, and the mode of defining the registration query condition by the code segments can simplify the operation of defining complex registration query conditions; for another example: the query condition information can also comprise an interface, and the result obtained by calling the interface to respond to the log query request can be used as a registration query condition, so that the log query request is sent to a registrant of the target log set through the interface, the registrant determines the registration query condition, and the registrant is endowed with the authority of customizing and complicating the setting of the query condition to a certain extent. Determining the registration query condition and the candidate query condition as a target query condition; based on step S201, it can be known that the target log query request includes the candidate query log. In other words, the target query condition for filtering the logs in the target log set may be a log query request sent by an inquirer and registration information of the target log set registered by a registrar, which to some extent realizes that the process of filtering the logs can be dynamically controlled according to identity information of the inquirer.

The process of generating the registration query condition is different according to different contents contained in the query condition information in the step two; the following are included in the query condition information, respectively: the rules (i.e. query rules), the code segments (i.e. first code segments) and the interfaces (i.e. condition management interfaces) are used as examples, and a specific implementation manner for generating the registration query condition based on the query condition information is described, where:

(1) the query condition information under the target access right defines a query rule. In this implementation, the implementation of generating the registration query condition based on the query condition information may include: and determining the query rule in the query condition information as the registration query condition. In other words, if the query condition information includes a series of query rules defined by the registrant of the target log set, the query rules defined by the registrant may be determined as the registration query condition. For example, it is assumed that the access right information of the target log set includes access right 1 and access right 2, the authorized user information corresponding to the access right 1 is "administrator of the platform", and the query condition information under the access right 1 is "all logs in the target log set can be queried"; the authorized user information corresponding to the access right 2 is "normal user", and the query condition information under the access right 2 is "log with queriable label name" XX "and label value" XX ". If the identity information contained in the log query request is ' ordinary user ', the identity information ' ordinary user ' of the querying party is the same as the authorized user information ' ordinary user ' of the access right 2, and it is determined that the log query request is matched with the access right 2, that is, the querying party can query the log indicated by the query condition information under the access right 2, that is, the log with the registration query condition of ' queryable label name ' XX ' and the label value ' XX '.

(2) The query condition information under the target access right includes a first code segment, and the first code segment may define a query function for the query log. In this implementation, the implementation of generating the registration query condition based on the query condition information may include: determining a target query request as input information of a query function; and calling the first code segment to run a query function, and processing the log query request to obtain a registration query condition. It is understood that the code segments have the advantages of easy understanding, simple logic writing and the like, so that the embodiment of the application supports the adoption of the code segments to define the query function, and the flexibility of defining the query function can be improved.

(3) The query condition information under the target access authority includes a condition management interface, specifically a condition management interface provided by a registrant of the target log set. In this implementation, the implementation of generating the registration query condition based on the query condition information may include: calling a condition management interface to send a log query request to a registrant of a target log set; and receiving a registration inquiry condition returned by the registrant through the condition management interface. The registration query condition is received through the condition management interface provided by the registrant, so that the complicated and customized authority control requirements of the registrant on the registration query condition can be met, and the acquisition mode of the registration query condition is enriched.

And obtaining the registration query condition for querying the candidate log based on the operation, and then executing query operation in the target log set by adopting the registration query condition and the candidate query condition contained in the log query request to obtain at least one candidate log. The following describes a process of querying at least one candidate log based on the target query condition with reference to fig. 6; as shown in fig. 6, it is assumed that the log query request includes: the identity information of the inquirer, namely 'common member of item X', the user name of 'Zhang III', the meta information 'without a key', and the candidate inquiry condition 'last 30 minutes'; the registration information of the target log set includes: the access authority 1 "log query request includes an authentication key within an expiration date, and can view all logs in a target log set", the access authority 2 "log with a tag name of" item number ", a tag value of" X "can be viewed by an administrator of item X", the access authority 3 "log with a tag name of" item number ", a tag value of" X "and a tag name of" creator ", and a tag value of" user name of the member "; based on the identity information of the inquiring party and the registration information of the target log set contained in the log query request, the target access authority possessed by the inquiring party is access authority 3, and the obtained registration query conditions are as follows: a log of the tag name "item number", the tag value "X", the tag name "creator", and the tag value "user name of the member"; further, the log query request includes a candidate query condition "last 30 minutes", and the target query condition obtained based on the log query request and the registration information of the target log set includes: a log of the tag name "item number", the tag value "X" and tag name "creator", the tag value "user name of the member" and tag name "timestamp", and the tag value "last 30 minutes".

S204, determining a registration processing requirement according to the log query request and the registration processing information of the target log set, and processing at least one candidate log according to the registration processing requirement to obtain a log query result.

The registration processing information is information for determining processing of a query result obtained based on the target query condition query. The registration processing information comprises at least one registration processing authority, authorized user information corresponding to each registration processing authority and processing information under each registration processing authority. An exemplary registration process information may include: 1. when the log query request is the external network access, removing the key value (namely the label value) of the label name 'XX'; 2. and when the log query request is local area network access, no processing is performed. It may be determined that the registration processing information includes two types of registration processing authorities, where the registration processing authority 1 is "1, and when the log query request is an extranet access, the key value (i.e., the tag value) of the tag name" XX "is removed, and the authorized user information corresponding to the registration processing authority 1 includes: the log query request is an external network access, and the processing information under the registration processing authority 1 is as follows: the key for the tag name "XX" is removed. The registration processing authority 2 is "2", when the log query request is local area network access, no processing is performed ", and the authorized user information corresponding to the registration processing authority 2 is: the log query request is local area network access, and the processing information under the registration processing authority 2 is as follows: no treatment was performed.

The following briefly introduces an implementation manner for determining a registration processing requirement according to a log query request and registration processing information of a target log set with reference to fig. 7: firstly, determining target registration processing authority possessed by an inquirer according to identity information of the inquirer and authorized user information corresponding to each registration processing authority; and then generating a registration processing requirement based on the processing information under the target registration processing authority. Similar to the access right information, the implementation manner of determining the registration processing requirement is different according to the different content included in the processing information under the target registration processing right. The following processing information under the authority of target registration processing includes: the rules (i.e., the registration processing rules), the code segments (i.e., the second code segments), and the interfaces (i.e., the registration processing interfaces) are used as examples, and an implementation manner for generating a registration processing requirement based on processing information under the authority of target registration processing is described, in which:

(1) the processing information under the authority of the target registration processing defines a registration processing rule. In this implementation, the implementation of generating the registration processing requirement based on the processing information under the authority of the target registration processing may include: and determining a registration processing requirement according to the registration processing rule defined by the processing information. In other words, if the process information under the authority of the target registration process includes a series of registration process rules defined by the registrant of the target log set, the registration process rules defined by the registrant may be determined as the registration process requirement.

(2) The processing information under the target registration processing authority comprises a second code segment, and the second code segment defines a registration processing function; namely, the registration processing requirement is obtained by means of the code segment. In this implementation, the implementation of generating the registration processing requirement based on the processing information under the authority of the target registration processing may include: and determining the log query request as the input information of the registration processing function, calling the second code segment to run the registration processing function, and processing the log query request to obtain the registration processing requirement. It should be noted that the input information of the registration processing function may include target information of at least one candidate log in addition to the log query request, so that the registration processing function may determine the registration processing requirement based on the target information of the at least one candidate log and the log query request as input information. Wherein, the target information of at least one candidate log may include but is not limited to: the total number of candidate logs, the duration of querying the candidate logs, the number of candidate logs that fail to return, and the like; the target information of the at least one candidate log is obtained in a process of querying the at least one candidate log from the data source based on the target query condition.

(3) The processing information under the authority of target registration processing comprises a registration processing interface provided by a registrant of the target log set. In this implementation, the implementation of generating the registration processing requirement based on the processing information under the authority of the target registration processing may include: and calling a registration processing interface to send the log query request to a registrant of the target log set and receive a registration requirement returned by the registrant. It should be noted that, in addition to invoking the registration processing interface to send the log query request to the registrant of the target log set, the target information of at least one candidate log may also be sent to the registrant of the target log set as an invocation parameter, so that the registrant may determine the registration processing requirement based on the target information of at least one candidate log and the log query request. The log query service for log query does not need to care about the specific implementation mode and the running condition of the interface, realizes the decoupling of the log query and the setting of the registration processing requirement, and improves the flexibility of setting the registration processing requirement.

Based on the above operation, a log query result may be obtained by processing at least one candidate log with the registration processing requirement, where the log query result includes at least one log obtained by processing the at least one candidate log. Referring to fig. 8, an exemplary description will be given of a process of processing at least one candidate log according to a registration processing requirement to obtain a log query result; as shown in fig. 8, it is assumed that the log query request includes: identity information of the inquiring party "the inquiring party accesses the computer device (or log query service) through the external network". The registration information of the target log set comprises: registration processing authority 1 "does not process when accessing local area network", registration processing authority 2 "removes label of label name" IP address "and hides sensitive information in log when accessing external network". Based on the log query request and the registration information of the target log set, it can be known that: the identity information of the inquirer, that the inquirer accesses the computer device (or log inquiry service) through the external network, is the same as the authorized user information, that is, the external network access, corresponding to the registration processing authority 2, if it is determined that the target registration processing authority is the registration processing authority 2, the registration processing requirement includes: the tag with the tag name "IP address" is removed and sensitive information in the log is hidden. Assuming that filtering the obtained target log (e.g. any one of the at least one candidate log) based on the target query condition includes: a tag with a tag name of "tag 1", a tag with a tag name of "tag name 2", and a tag with a tag name of "IP address"; then, after removing the tag with the tag name "IP address" in the target log according to the obtained registration processing request, an updated target log may be obtained, where the updated target log includes: a tag with a tag name of "tag 1" and a tag with a tag name of "tag 2". Secondly, the updated target log is processed by hiding sensitive information in the target log, and a log query result is obtained.

It will be appreciated that the sensitive information in the log may be dynamically changing, for example, the type and number of domestic sensitive words may be different for the current time period (e.g., the current time period during which the log query is being executed) and for the historical time period (e.g., the time period prior to the current time). In order to more accurately detect sensitive information in the log, the embodiment of the application supports that the registration processing requirements obtained based on the log query request and the registration processing information of the target log set are interfaces (such as an HTTP interface, a remote system call interface, a message queue, a shared memory, a cloud function, and the like). In this implementation, the processing for any log may include: and taking the log as a parameter to carry out interface calling, and taking the interface return result as a processed result (or log query result). The method for hiding the sensitive information of the log by calling the service special for processing the sensitive information through the interface ensures that registrars of log query service or a target log set do not need to care about the implementation process of hiding the sensitive information, and realizes the decoupling of sensitive information hiding, target log set maintenance and target log set query. Of course, in addition to the requirement for registering the interface type, which may be required for removing the sensitive information in the log, there are other situations that also require the requirement for registering the interface type, which are described above by taking the example of removing the sensitive information in the log as an example, and are not limited to the embodiments of the present application, and are described here.

In the embodiment of the application, a target query condition can be determined based on a log query request sent by a query party and registration information of a target log set registered by a registration party, and at least one candidate log can be queried by adopting the log query request; and determining a registration processing requirement based on the log query request and the registration processing information, and processing at least one candidate log according to the registration processing requirement to obtain a log query result. In the above scheme, the target query condition for querying the candidate logs is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the flexibility of managing access rights and processing requirements in the log query process is improved, the requirement of dynamically screening and processing the candidate logs according to different query requests can be met, and the log query efficiency is improved.

Referring to fig. 9, fig. 9 is a flowchart illustrating a log processing method according to an exemplary embodiment of the present application; the log processing method may be performed by the above-mentioned computer device, and may include, but is not limited to, steps S901 to S910:

s901, receiving a registration request aiming at a target log set.

S902, responding to the registration request, and generating the registration information of the target log set.

In steps S901-S902, the registration request is initiated by the registrar for registering the target log set, and is used to implement customization of the registration information of the target log set, so as to improve the flexibility of management of the access right information and the registration processing information of the target log set. When registering the registration information of the target log set, the registration party mainly registers the access authority information and the registration processing information of the target log set. The registration of the registrant to the registration processing information of the target log set is taken as an example for introduction: assuming that the registrant hopes that only the administrator can access all logs of the target log set, and no processing operation is needed during the access in the local area network, the label with the label name "XXX" in the log is removed first during the access in the external network, and all sensitive information in the log needs to be hidden. The registrant may set the registration processing information to: if the source of the log query request is a local area network, the registration processing information is null (i.e. no processing is performed); otherwise, the registration processing information includes: registration processing information P1 and registration processing information P2, wherein the registration processing information P1 is a registration processing rule: removing the label with the label name "XXX" in the log; the registration processing information P2 is a registration processing interface (such as an HTTP call given address). The registration processing interface is provided by a registration party and has the function of hiding sensitive information.

The embodiment of the application supports the adoption of a log Boolean function to define the access authority information of a target log set, and particularly adopts the log Boolean function to define the query condition information under each access authority; and adopting a log processing function to define the registration processing information of the target log set, in particular adopting the log processing function to define the processing information under each registration processing authority. For the convenience of the following description, a common relational expression between the log and the labels contained in the log is given below, wherein: in the embodiment of the present application, a log is represented by a character R, and "a label name of R is a label value of a label of a character K" is equal to that of a character V "R [ K ] ═ V". Similarly, the embodiments of the present application support the use of the symbols ">, <, ≠ gtoreq, ≦" to signify that the tag value is greater than, less than, not equal to, greater than, or less than, respectively, a certain value. Similarly, in the embodiment of the present application, note that "the label value of the label with the label name of R as the character K satisfies the condition C" is that "R [ K ] satisfies the condition C"; the notation "R contains/does not contain the label with the label name of character K" is "R [ K ] exists/does not exist"; the label value of the label with the label name of R as a character K conforms to a regular expression, and the label value of the label with the label name of R as a character K conforms to the regular expression; and so on. The following describes the related concepts and terms of the log boolean function and the log processing function, wherein:

(1) the log boolean function may refer to: the input is a log R and the output is a function of boolean values, i.e. correct (true)/error (false). For example: is described as "input is denoted as R, output R [ K ]]The log Boolean function is the log Boolean function which can be used for filtering the logs to obtain candidate logs meeting the query condition; that is, the log boolean function corresponds to the query condition (e.g., the aforementioned registered query condition, the candidate query condition), and the log boolean function and the query condition are not distinguished in the embodiments of the present application and are described herein. If the output value of any log Boolean function is affected by at most one tag value in its input, then the any log Boolean function can be called an atomic log Boolean function; for example: "input is R, output is R [ item number]>1 "is an atomic log Boolean function (temporarily denoted F)_a) "the input is marked as R, and whether the label number of the output R is more than 10" is also an atomic log Boolean function (temporarily marked as F)_b). The atomic log boolean function may be a triplet-tag name or attribute name (e.g., "item number," "tag number"), relationship (e.g., "greater than," "contains"), and value (e.g., "2," "error," "null"). For example, "input is R, output R [ ID]Whether or not there is "this atomic log boolean function" can be described by a triplet (ID, exists, null).

The Boolean functions of the atomic log are one log Boolean function formed by operators; wherein the content of the first and second substances,operators may include, but are not limited to: logical operators (or V-shaped, and A, non)

) The "at least/at most" operator; for example, "F_a∨F_b"is a log boolean function. The data structure of the log boolean function may be a tree structure (e.g., a tree), the leaf nodes are atomic log boolean functions, and the non-leaf nodes are operators; for example: the Boolean function of the log is' F_a∨F_b"the root node in the tree is" or operator ", the two leaf nodes are (item number,>the sum of (1) and (number of tags,>,10). When a log R is given as the input of the log Boolean function, depth-first traversal can be sequentially performed on a tree defining the log Boolean function, each non-leaf node is judged according to the Boolean values of the child nodes of the non-leaf node, the leaf nodes are judged according to the triples defining the atomic log Boolean function and the log R, and finally the result of the log Boolean function is output according to the Boolean values of the root nodes.

(2) The log processing function may refer to: the input is a log R, and the output is a function of 0 log, 1 log or a plurality of logs. For example, it is described that "the input is recorded as R, if the length of R [ detailed information ] exceeds 20, the first 20 characters of R [ detailed information ] are retained and output" is a log processing function, so that at least one candidate log queried can be processed by using the log processing function; that is, the log processing function corresponds to a post-processing requirement (such as the registration processing requirement described above), and the log processing function and the registration processing requirement (or referred to as the post-processing requirement) are not distinguished in the embodiments of the present application and are described herein.

It should be noted that, in the embodiment of the present application, the registrar may define the obtaining manner of obtaining the log boolean function or the log processing function by using one or more of the following three manners, including but not limited to: firstly, a rule for acquiring a log Boolean function or a log processing function is defined; specifically, the method comprises the following steps: the registrant obtains a log Boolean function or a log processing function by customizing some rules. Obtaining a code segment of a log Boolean function or a log processing function by definition; specifically, the method comprises the following steps: the registrant defines a function (such as a query function) through the code segments to obtain a log Boolean function or a log processing function; for example, if a log query request is used as input information of a query function, the output result of the function can be used as a log boolean function. Obtaining the interface of the Boolean function or the log processing function by definition; specifically, the method comprises the following steps: the registrant provides an interface to acquire a log Boolean function or a log processing function; for example, if the registrant provides a condition management interface, and takes the log query request as a call parameter, the result returned by the condition management interface can be taken as a log boolean function.

Based on the above-described three ways of defining the obtaining of the log boolean function or the log processing function, a more specific implementation way of obtaining the log boolean function or the log processing function by using three ways is given below, where:

(1) and acquiring a log Boolean function. The access authority information contained in the registration information can be used for filtering logs in the target log set, and the logs meeting the query condition are obtained through screening; when the process of screening logs is described by using a log boolean function, the process of filtering the logs in the target log set to obtain the process meeting the query condition can be simply abstracted as follows: giving a log query request Q, and acquiring a log Boolean function F; then for each log in the target set of logs, if the log boolean function F returns correct (true) for the log, the log or logs may be accessed, i.e., determined to be candidate logs.

The access right information (or the query condition information under the access right contained in the access right information) of the target log set can be defined by a log boolean function: f ═ F₁∨F₂∨F₃(ii) a Wherein, F₁Is an atomic log Boolean function, F, obtained by the registrant defining the query rules₂Is an atomic log Boolean function, F, obtained by calling a code fragment provided by a registrant₃Is an atomic log boolean function that is obtained by calling a conditional management interface provided by the registrar. Of course, if registeredThe method does not define the atomic log Boolean function F₁Atomic Log Boolean function F₂Or atomic log Boolean function F₃Then the default defined log boolean function is an atomic log boolean function that always returns false. Below for F respectively₁、F₂And F₃The acquisition mode of (2) is briefly introduced:

F₁the acquisition mode of (1): the registrant defines a series of query rules-when a log query request for a target log set meets a certain target query condition, a certain log Boolean function is returned. For example, a log query request for a target log set may include: the user group of the user of the log query request, the IP address of the log query request, and other information, the registrar may specify two query rules: query rule 1: if the user group of the log query requesting user is 'platform administrator', a log Boolean function (marked as F)_A) The log Boolean function F_AAlways return true (i.e., any log in the target set of logs causes the log to be a Boolean function F_AReturn true, i.e., "platform administrator can access all logs under target set of logs"). Query rule 2: if the prefix of the IP address of the log query request is '192.168' and the USER name of the USER of the log query request is not null (the USER name is marked as USER), returning a log Boolean function (marked as F)_B) The journal Boolean function is described as "input is R and output is R [ creator]Whether it is USER "(i.e." the USER in this IP segment can access the log that the creator is his username "). If the log query request meets the two query rules, the obtained log boolean functions (or atomic log boolean functions) are respectively recorded as: f_A、F_BIf the log query request is obtained through the query rule, the log Boolean function F is obtained₁＝F_A∨F_B. Of course, the registrant defines two query rules as an example, and the number of the query rules may also be three, four, … …, etc., without limiting the embodiments of the present application.

F₂The acquisition mode of (1): the registrant defines a code segment (e.g., a first code segment) by code segmentAnd the input of the query function is a log query request to the target log set, and the output of the query function is a log Boolean function. In a specific implementation, when a log query service (or computer device) receives a log query request for a target log set, a query function defined by the code segment is called, the log query request is used as input information, and the output of the query function is used as a log Boolean function F₂。

F₃The acquisition mode of (1): calling a condition management interface provided by a registrant, taking a log query request as a calling parameter, and taking a log Boolean function returned by calling as F₃. This acquisition approach may allow the target log set to have complex, customized access rights logic. For example, if the registrar wants to verify the authentication key issued by itself, the log boolean function may be obtained using this obtaining method. In specific implementation, when a registrant of a target log set receives a log query request, the log query request can be used for calling a condition management interface provided by the registrant so that the registrant verifies an authentication key and returns a log Boolean function F according to an authentication result₃。

In summary, the registrant of the target log set defines the access right information of the target log set by defining the query rule for obtaining the log boolean function, obtaining the first code segment of the log boolean function, and obtaining the condition management interface of the log boolean function.

(2) A log processing function is obtained. The registration processing requirements of a target log set may be defined as a series of log processing functions P₁、P₂、P₃、……、P_nN is a positive integer, that is, at least one candidate log obtained by log Boolean function query is first selected from P₁Processing is carried out, and the obtained processing result is further P₂Processing is carried out by analogy, and P is added_nAnd the processed result is used as a log query result. Of course, the requirement of registration processing of the target log set may be null, i.e., at least one candidate log obtained by the query is not processed. The log processing function is obtained in a manner similar to the log boolean functionAlso included are three, wherein:

acquiring a log processing function through a registration processing rule defined by a registration party. The acquisition mode is mainly suitable for some simpler log processing functions, and the log processing functions can be defined in the following mode: and the input is recorded as R, if R enables the log Boolean function F to return true, the operation O is carried out on R to obtain a log query result, and if the operation O fails, P (R) is returned as the log query result. Namely, only the definition of F, O, P needs to be given, so that the log processing function can be defined; wherein F may be as F₁Is obtained, O is some operation on the log (e.g., remove R [ item number ]]) (ii) a P is a fixed log processing function, for example, P is "input is recorded as R, return is empty" (remove the log), "input is recorded as R, R [ item number]Set to default value 1 "(set default value).

For example, assume that the processing logic for the registrar wishing to process the candidate log is: when the IP of the log query request source is from a network segment of 192.168, if the candidate log contains a label 'timestamp', converting a label value of the label name 'timestamp' into a character string in an RFC3339 format, and if the conversion fails, removing the candidate log; otherwise, no processing is performed. If the IP of the log query request source is from a 192.168. times, returning to a log Boolean function F' and giving an input R, and returning whether the R contains an R [ time stamp ]; otherwise, a log Boolean function with a false output is returned. It is possible to define O as "input is R, R [ time stamp ] is converted into character string in RFC3339 format and output". The log handling function P may be defined as "input is noted R, return null".

And acquiring the log processing function through the code segment (such as a second code segment) provided by the registrant. The registrant defines a registration processing function through the code segments, the input of the registration processing function is a log query request aiming at the target log set and the target information of at least one candidate log, and the output of the registration processing function is a log processing function. In a specific implementation, when a log query service (or a computer device) receives a log query request for a target log set, a registration processing function defined by the code segment is called, the log query request is used as input information, and an output of the registration processing function is used as the log processing function.

And acquiring a log processing function through a registration processing interface provided by a registration party. And calling a registration processing interface provided by a registration party, taking the log query request and the target information of at least one candidate log as calling parameters, and taking the result returned by calling as a log processing function. The acquisition mode can allow the target log set to have complex and customized processing logic, can completely realize that the processing logic is provided for each log query request and each candidate log in a targeted manner, and even the registrant can acquire some other information by adopting a mode of providing a registration processing interface, such as whether each query for the target log set is successful, the time consumption of log query and the like.

It is particularly noted that the obtained log processing function may be an interface (such as an HTTP interface, a remote system call interface, a message queue, a shared memory, a cloud function, and the like); in this implementation, the processing of any log by the log processing function in the form of an interface may include: and taking the log R as a parameter to carry out interface calling, and taking a result returned by an interface as a result of processing the log R by adopting a log processing function. The log is processed by adopting a log processing function in an interface form, so that a registrant can dynamically change a processing result (such as a log query result) according to the state of the registrant, and the requirement of the registrant on the customized processing logic is met.

For example, suppose that the registrant has two service systems, namely "domestic sensitive word replacement" and "overseas sensitive word replacement", the access modes of the two are dynamically specified and the sensitive words contained in the log are dynamically updated over time; if the registrant hopes that the log query request contains the authentication key, no post-processing is carried out, otherwise, an access interface of the sensitive word replacement service is returned as a log processing function according to the source of the log query request, the registrant of the target log set defines the acquisition mode of the log processing function as a call registration processing interface, and the registration processing interface respectively returns the log processing function which does not carry out any processing and the log processing function of the registration processing interface which calls the corresponding service according to whether the log query request contains the legal authentication key and the source of the log query request.

In summary, the registrant of the target log set defines the above-mentioned series of log processing functions P₁、P₂、P₃、……、P_nEtc., the definition of the processing logic for the log can be achieved.

In addition, the embodiment of the application supports the registrant to edit the registered information of the registered target log set. In a specific implementation, a log editing request sent by a registrant of a target log set can be received, wherein the log editing request is used for requesting to edit the registration information of the target log set; responding to the log editing request to execute editing operation on the registration information of the target log set to obtain updated registration information of the target log set; wherein the editing operation may include: a combination of one or more of an add operation (e.g., adding a log boolean function or a log processing function to the registration information), a modify operation (e.g., modifying a log boolean function or a log processing function to the registration information), and a delete operation (e.g., deleting a log boolean function or a log processing function to the registration information). Of course, for the computer device (or the log query service), the computer device may receive the log editing request from the local, or may receive the registration editing request sent by another device through the network, and the embodiment of the present application does not limit the manner in which the computer device obtains the registration editing request, and is described herein.

And S903, receiving a log query request.

And S904, searching the registration information of the target log set based on the log set identification of the target log set.

S905, determining target query conditions according to the target query request and the access authority information of the target log set.

S906, at least one candidate log is obtained from the target log set according to the target query condition.

And S907, determining a registration processing requirement according to the log query request and the registration processing information of the target log set.

S908, processing at least one candidate log according to the registration processing requirement to obtain a log query result.

It should be noted that, for the specific implementation shown in steps S903 to S908, reference may be made to the related description of the specific implementation shown in steps S201 to S204 in the foregoing embodiment shown in fig. 2, which is not described herein again.

And S909, processing the log query result based on the query processing requirement to obtain a log query response result.

S910, returning the log query response result to the query party.

In steps S909 to S910, the log query request mentioned in the embodiment of the present application may further include a query processing requirement, where the query processing requirement is defined by the querying party for processing the log query result. In this implementation, referring to fig. 10, a process of determining a target query condition and a target processing requirement (e.g., including a registration processing requirement and a query processing requirement) based on a log query request and registration information of a target log set, where fig. 10 illustrates a schematic diagram of determining a target query condition and a target processing requirement according to an exemplary embodiment of the present application. As shown in fig. 10, the target query condition may be obtained based on the log query request and the access right information included in the registration information of the target log set, and the target query condition may include: registering one or both of the query condition and the candidate query condition, so that at least one candidate log and target information (such as query time consumption information, number information of the candidate logs and the like) of the at least one candidate log can be obtained based on the target query condition; similarly, the target processing requirement may be obtained based on the log query request and the registration processing information included in the registration information of the target log set, and the target processing requirement may include: one or both of a registration processing requirement and a query processing requirement; specifically, the log query result obtained by processing at least one queried candidate log by using the registration processing requirement can be processed, and then the log query result is processed by using the query processing requirement to obtain a log query response result; and returning the log query response result to the query party to realize the query of the logs in the target log set. Wherein the log query response result comprises at least one log.

In addition, the embodiment of the application also supports the aggregation processing of the log query response result and outputs the aggregated result after the aggregation processing. Among these, the polymerization process may include, but is not limited to: the number of logs including a target tag (such as any tag) in the logs included in the statistical log query result (or log query response result), the average value of the tag values of the tags with the tag names "XXXX" in the logs included in the log query response result, the number of logs including the tags with the tag names "XXX" in the logs included in the statistical log query response result, … …, and the like. In specific implementation, an aggregation request sent by a query party can be received, wherein the aggregation request carries aggregation conditions; and responding to the aggregation request, performing aggregation processing on at least one log contained in the log query response result by adopting an aggregation condition to obtain an aggregation result, and returning the aggregation result to the query party. The method for requesting the computer device (or the storage engine) to aggregate the log query response results directly by sending the aggregation request can reduce the calculation overhead. Of course, the aggregation condition for performing aggregation processing on the log query response request may also be carried in the log query request, and the manner of obtaining the aggregation request is not limited in the embodiment of the present application, which is described herein.

In addition, as can be seen from the schematic diagram of the registration information of the target log set shown in fig. 3, the registration information of the target log set also carries information such as paging conditions and sorting conditions, so that the embodiment of the present application supports that after a log query response result (or a log query result) is obtained, one or two of the paging conditions and the sorting conditions are adopted to perform paging processing, sorting processing, or paging processing and sorting processing on a plurality of logs included in the log query response result; therefore, the logs output to the inquiring party can be displayed in a paging mode or in a sequencing mode according to requirements, and the readability of the logs is improved. Taking the example of paging displaying a plurality of logs (for example, 200 logs) included in the log query response result according to the paging condition, assuming that 50 logs are displayed on each page according to the paging condition, 200 logs can be respectively displayed on 5 pages, which improves the readability of the logs and facilitates the query party to view and retrieve the logs. Taking the example of sorting and displaying the plurality of logs included in the log query response result according to the sorting condition, assuming that the sorting condition is that the logs are sorted in the order from near to far according to the log generation time, if the generation time of the log 1 is 12:00 at 8 days of 5 months, 16:00 at 8 days of 5 months, and 00 at 9 days of 5 months, the sorting order of the log 1, the log 2, and the log 3 can be determined as follows: log 1- > log 2- > log 3. An exemplary schematic diagram of sorting and displaying a plurality of logs according to log generation time can be seen in fig. 11, as shown in fig. 11, tag names of N logs are all implicitly guided by a header of a data table, N is a positive integer, and the N logs are sorted and displayed from near to far according to the generation time. The mode of displaying a plurality of logs in an ordering manner is beneficial for the inquirer to check and inquire the logs, and the inquiry experience of the inquirer is improved. It should be noted that, the display of the multiple logs is described in two manners of paging and sorting, but in an actual application scenario, the multiple logs may also be displayed based on other display conditions, and the display manner of the multiple logs is not limited in the embodiment of the present application, and is described here.

In summary, the embodiment of the present application decouples log query from the setting of the registration information of the target log set, that is, decouples log query from the setting of access permission information and registration processing information, so that a set of log query services can be implemented for querying logs under different service systems, and further, repeated development of the log query services is avoided. Moreover, the access authority information and the registration processing information are set by the registrant, so that the flexibility of the access authority and the processing logic can be improved, and further, the acquisition mode of the access authority information and the registration processing information can comprise code segments and interfaces, so that the flexibility of the access authority and the processing logic is improved, and the requirement that the registrant updates the registration information at any time is met. In addition, the target query condition for querying the candidate logs is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the requirements of dynamically screening and processing the candidate logs according to different query requests can be met, the log query flexibility is improved, and the log query efficiency is improved.

While the method of the embodiments of the present application has been described in detail above, to facilitate better implementation of the above-described aspects of the embodiments of the present application, the apparatus of the embodiments of the present application is provided below accordingly.

Fig. 12 is a schematic structural diagram illustrating a log processing apparatus according to an exemplary embodiment of the present application; the log processing means may be for use as a computer program (comprising program code) running on a computer device; the log processing means may be adapted to perform some or all of the steps in the method embodiments shown in fig. 2 and 9. Referring to fig. 12, the log processing apparatus includes the following units:

an obtaining unit 1201, configured to receive a log query request, where the log query request includes a log set identifier of a target log set requested to be queried;

a processing unit 1202, configured to search for registration information of a target log set based on a log set identifier of the target log set, where the registration information of the target log set includes access right information and registration processing information of the target log set;

the processing unit 1202 is further configured to determine a target query condition according to the log query request and the access right information of the target log set, and obtain at least one candidate log from the target log set according to the target query condition;

the processing unit 1202 is further configured to determine a registration processing requirement according to the log query request and the registration processing information of the target log set, and process at least one candidate log according to the registration processing requirement to obtain a log query result.

In one implementation, the log query request further includes a query processing requirement; a processing unit 1202, further configured to:

processing the log query result according to the query processing requirement to obtain a log query response result; and returning the log query response result to the querying party.

In one implementation mode, the access right information comprises at least one access right, authorized user information corresponding to each access right, and query condition information under each access right; the log query request also comprises identity information of a query party and candidate query conditions; the processing unit 1202 is configured to, when determining a target query condition according to the log query request and the access right information of the target log set, specifically:

In one implementation, the query condition information under the target access right defines a query rule; the processing unit 1202, when generating the registration query condition based on the query condition information under the target access right, is specifically configured to:

In one implementation, the query condition information under the target access right includes a first code segment, and the first code segment defines a query function; the processing unit 1202, when generating the registration query condition based on the query condition information under the target access right, is specifically configured to:

determining the log query request as input information of a query function;

In one implementation, the query condition information under the target access authority includes a condition management interface provided by a registrant of the target log set; the processing unit 1202, when generating the registration query condition based on the query condition information under the target access right, is specifically configured to:

In one implementation, the registration processing information includes at least one registration processing authority, authorized user information corresponding to each registration processing authority, and processing information under each registration processing authority; the log query request also comprises identity information of a query party; the processing unit 1202 is configured to, when determining a registration processing requirement according to the log query request and the registration processing information of the target log set, specifically:

In one implementation, the processing information under the authority of target registration processing defines a registration processing rule; the processing unit 1202 is configured to, when generating a registration processing request based on processing information under the target registration processing authority, specifically:

In one implementation, the processing information under the authority of target registration processing includes a second code segment, and the second code segment defines a registration processing function; the processing unit 1202 is configured to, when generating the registration processing requirement based on the processing information under the target registration processing authority, specifically:

In one implementation, the processing information under the authority of target registration processing includes a registration processing interface provided by a registrant of the target log set; the processing unit 1202 is configured to, when generating a registration processing request based on processing information under the target registration processing authority, specifically:

and receiving a registration processing requirement returned by the registrant.

In one implementation, the processing unit 1202 is further configured to:

In one implementation, the registration information of the target log set is stored in a log set storage space, and the log set storage space stores the registration information of a plurality of log sets; each log set is used for recording logs generated in one service system; the obtaining unit 1201 is configured to, when receiving a log query request, specifically:

calling a log query service interface to receive a log query request;

According to an embodiment of the present application, the units in the log processing apparatus shown in fig. 12 may be respectively or entirely combined into one or several other units to form the log processing apparatus, or some unit(s) thereof may be further split into multiple functionally smaller units to form the log processing apparatus, which may implement the same operation without affecting implementation of technical effects of embodiments of the present application. The units are divided based on logic functions, and in practical application, the functions of one unit can be realized by a plurality of units, or the functions of a plurality of units can be realized by one unit. In other embodiments of the present application, the log processing apparatus may also include other units, and in practical applications, these functions may also be implemented by being assisted by other units, and may be implemented by cooperation of multiple units. According to another embodiment of the present application, the log processing apparatus shown in fig. 12 may be configured by running a computer program (including program codes) capable of executing the steps involved in the respective methods shown in fig. 2 and 9 on a general-purpose computing device such as a computer including a processing element such as a Central Processing Unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), and a storage element, and the log processing method of the embodiment of the present application may be implemented. The computer program may be recorded on a computer-readable recording medium, for example, and loaded and executed in the above-described computing apparatus via the computer-readable recording medium.

In this embodiment of the present application, the processing unit 1202 can determine a target query condition based on a log query request sent by a querying party and registration information of a target log set registered by a registration party, and query at least one candidate log by using the log query request; and determining a registration processing requirement based on the log query request and the registration processing information, and processing at least one candidate log according to the registration processing requirement to obtain a log query result. In the above scheme, the target query condition for querying the at least one candidate log is dynamically determined based on the log query request and the registration information of the target log set, and the registration processing requirement for processing the at least one candidate log is also dynamically determined based on the log query request and the registration information of the target log set, so that the flexibility of managing access rights and processing requirements in the log query process is improved, the requirement of dynamically screening and processing the candidate logs according to different query requests can be met, and the log query efficiency is improved.

Fig. 13 is a schematic structural diagram illustrating a log processing device according to an exemplary embodiment of the present application. Referring to fig. 13, the log processing apparatus includes a processor 1201, a communication interface 1202, and a computer-readable storage medium 1203. The processor 1201, the communication interface 1202, and the computer readable storage medium 1203 may be connected by a bus or other means. The communication interface 1302 is used for receiving and transmitting data, among other things. A computer readable storage medium 1303 may be stored in the memory of the log processing device, the computer readable storage medium 1303 is used for storing a computer program, the computer program includes program instructions, and the processor 1301 is used for executing the program instructions stored by the computer readable storage medium 1303. The processor 1301 (or CPU) is a computing core and a control core of the log Processing apparatus, and is adapted to implement one or more instructions, and specifically, adapted to load and execute the one or more instructions so as to implement a corresponding method flow or a corresponding function.

An embodiment of the present application further provides a computer-readable storage medium (Memory), which is a Memory device in a log processing device and is used for storing programs and data. It is understood that the computer readable storage medium herein may include a built-in storage medium in the log processing device, and may also include an extended storage medium supported by the log processing device. The computer readable storage medium provides a storage space storing a processing system of the log processing apparatus. Also stored in the memory space are one or more instructions, which may be one or more computer programs (including program code), suitable for loading and execution by processor 1301. It should be noted that the computer-readable storage medium may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), such as at least one disk memory; optionally, at least one computer readable storage medium located remotely from the aforementioned processor is also possible.

In one embodiment, the log processing device may be the computer device mentioned in the previous embodiment; the computer-readable storage medium has one or more instructions stored therein; one or more instructions stored in the computer-readable storage medium are loaded and executed by the processor 1301 to implement the corresponding steps in the above-described log processing method embodiment; in particular implementations, one or more instructions in the computer-readable storage medium are loaded and executed by processor 1301 to perform the steps of:

and determining a registration processing requirement according to the log query request and the registration processing information of the target log set, and processing at least one candidate log according to the registration processing requirement to obtain a log query result.

In one implementation, the log query request further includes a query processing requirement; one or more instructions in the computer-readable storage medium are loaded by processor 1301 and further perform the steps of:

and returning the log query response result to the querying party.

In one implementation mode, when the access right information comprises at least one access right, authorized user information corresponding to each access right, and query condition information under each access right; the log query request also comprises identity information of a query party and candidate query conditions; one or more instructions in the computer-readable storage medium are loaded by the processor 1301, and when determining a target query condition according to the log query request and the access right information of the target log set, the instructions are specifically configured to perform the following steps:

In one implementation, the query condition information under the target access right defines a query rule; one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and when executing the process of generating the registration query condition based on the query condition information under the target access right, the process is specifically configured to perform the following steps:

In one implementation, the query condition information under the target access right includes a first code segment, and the first code segment defines a query function; one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and when executing the process of generating the registration query condition based on the query condition information under the target access right, the process is specifically configured to perform the following steps:

determining the log query request as input information of a query function;

In one implementation, the query condition information under the target access authority includes a condition management interface provided by a registrant of the target log set; one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and when executing the process of generating the registration query condition based on the query condition information under the target access right, the process is specifically configured to perform the following steps:

In one implementation, the registration processing information includes at least one registration processing authority, authorized user information corresponding to each registration processing authority, and processing information under each registration processing authority; the log query request also comprises identity information of a query party; one or more instructions in the computer-readable storage medium are loaded by the processor 1301, and when determining a registration processing requirement according to the log query request and the registration processing information of the target log set, the instructions are specifically configured to perform the following steps:

In one implementation, the processing information under the authority of target registration processing defines a registration processing rule; one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and when executing the registration processing requirement generated based on the processing information under the target registration processing authority, are specifically configured to perform the following steps:

In one implementation, the processing information under the authority of target registration processing includes a second code segment, and the second code segment defines a registration processing function; one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and, when executing the registration processing requirement generated based on the processing information under the target registration processing authority, are specifically configured to perform the following steps:

In one implementation, the processing information under the authority of target registration processing includes a registration processing interface provided by a registrant of the target log set; one or more instructions in the computer-readable storage medium are loaded by the processor 1301 and when executing the registration processing requirement generated based on the processing information under the target registration processing authority, are specifically configured to perform the following steps:

and receiving a registration processing requirement returned by the registrant.

In one implementation, the loading by the processor 1301 of one or more instructions in the computer readable storage medium further performs the steps of:

In one implementation, the registration information of the target log set is stored in a log set storage space, and the log set storage space stores the registration information of a plurality of log sets; each log set is used for recording logs generated in one service system; one or more instructions in the computer-readable storage medium are loaded by processor 1301 and when executing a receive log query request, are specifically configured to perform the following steps:

calling a log query service interface to receive a log query request;

Embodiments of the present application also provide a computer program product or a computer program comprising computer instructions stored in a computer-readable storage medium. The processor of the log processing device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the log processing device to execute the log processing method.

Those of ordinary skill in the art would appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions described in accordance with the embodiments of the invention are all or partially effected when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on or transmitted over a computer-readable storage medium. The computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., Solid State Disks (SSDs)), among others.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present disclosure, and all the changes or substitutions should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A log processing method, comprising:

searching the registration information of the target log set based on the log set identification of the target log set, wherein the registration information of the target log set comprises the access authority information and the registration processing information of the target log set;

and determining a registration processing requirement according to the log query request and the registration processing information of the target log set, and processing the at least one candidate log according to the registration processing requirement to obtain a log query result.

2. The method of claim 1, wherein the log query request further includes a query processing requirement; the method further comprises the following steps:

and returning the log query response result to the querying party.

3. The method of claim 1, wherein the access right information includes at least one access right, authorized user information corresponding to each access right, and query condition information under each access right; the log query request also comprises identity information of a query party and candidate query conditions; determining a target query condition according to the log query request and the access authority information of the target log set, wherein the target query condition comprises the following steps:

determining the target access authority possessed by the inquirer according to the identity information of the inquirer and the authorized user information corresponding to each access authority;

generating a registration query condition based on the query condition information under the target access authority;

and determining the registration query condition and the candidate query condition as the target query condition.

4. The method of claim 3, wherein the query condition information under the target access right defines a query rule; the generating of the registration query condition based on the query condition information under the target access right comprises:

5. The method of claim 3, wherein the query condition information under the target access right includes a first code segment, the first code segment defining a query function; the generating of the registration query condition based on the query condition information under the target access right comprises:

determining the log query request as input information of the query function;

and calling the first code segment to run the query function, and processing the log query request to obtain the registration query condition.

6. The method of claim 3, wherein the query condition information under the target access right includes a condition management interface provided by a registrant of the target log set; the generating of the registration query condition based on the query condition information under the target access right comprises:

calling the condition management interface to send the log query request to a registrant of the target log set;

7. The method of claim 1, wherein the registration processing information includes at least one type of registration processing authority, authorized user information corresponding to each type of registration processing authority, and processing information under each type of registration processing authority; the log query request also comprises identity information of a query party; the determining a registration processing requirement according to the log query request and the registration processing information of the target log set includes:

determining target registration processing authority possessed by the inquirer according to the identity information of the inquirer and the authorized user information corresponding to each registration processing authority;

and generating the registration processing requirement based on the processing information under the target registration processing authority.

8. The method of claim 7, wherein the processing information under the authority of the target registration process defines a registration process rule; the generating the registration processing requirement based on the processing information under the target registration processing authority comprises:

and determining the registration processing rule defined by the processing information as the registration processing requirement.

9. The method of claim 7, wherein the processing information under the authority of the target registration process includes a second code segment, the second code segment defining a registration process function; the generating the registration processing requirement based on the processing information under the target registration processing authority comprises:

determining the log query request as input information of the registration processing function;

and calling the second code segment to run the registration processing function, and processing the log query request to obtain the registration processing requirement.

10. The method of claim 1, wherein the processing information under the authority of the target registration process comprises a registration process interface provided by a registrant of the target log set; the generating the registration processing requirement based on the processing information under the target registration processing authority comprises:

calling the registration processing interface to send the log query request to a registrant of the target log set;

and receiving a registration processing requirement returned by the registrant.

11. The method of claim 1, wherein the method further comprises:

receiving a log editing request sent by a registrant of the target log set, wherein the log editing request is used for requesting to edit the registration information of the target log set;

12. The method of claim 1, wherein the registration information of the target logset is stored in a logset storage space in which registration information of a plurality of logsets is stored; each log set is used for recording logs generated in one service system; the receiving a log query request includes:

calling a log query service interface to receive the log query request;

the log query service interface is adapted to a plurality of the service systems, and is used for receiving a query request for a log set corresponding to any one of the service systems.

13. A log processing apparatus, comprising:

the device comprises an acquisition unit, a query unit and a query unit, wherein the acquisition unit is used for receiving a log query request which contains a log set identifier of a target log set requested to be queried;

the processing unit is used for searching the registration information of the target log set based on the log set identifier of the target log set, wherein the registration information of the target log set comprises the access authority information and the registration processing information of the target log set;

the processing unit is further configured to determine a target query condition according to the log query request and the access right information of the target log set, and obtain at least one candidate log from the target log set according to the target query condition;

the processing unit is further configured to determine a registration processing requirement according to the log query request and the registration processing information of the target log set, and process the at least one candidate log according to the registration processing requirement to obtain a log query result.

14. A log processing apparatus characterized by comprising:

a processor adapted to execute a computer program;

a computer-readable storage medium, in which a computer program is stored, which, when executed by the processor, implements the log processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program adapted to be loaded by a processor and to perform the log processing method according to any one of claims 1 to 12.