CN114297160A - Log processing method, system, electronic device and storage medium - Google Patents

Abstract

The present disclosure provides a log processing method, a log processing system, an electronic device and a storage medium, which relate to the technical field of computers, and in particular to the fields of log management, big data and the like. The specific implementation scheme is as follows: receiving a log processing request from a client, wherein the log processing request comprises a log type; acquiring a log to be processed which accords with the log type; and performing step-by-step funnel aggregation statistics on the logs to be processed to obtain a processing result. The embodiment of the disclosure can automatically complete the collection and processing of the logs, and improve the log processing efficiency.

Description

Log processing method, system, electronic device and storage medium

Technical Field

The present disclosure relates to the field of computer technology, and in particular, to the fields of log management, big data, and the like.

Background

At present, various application programs can be installed in mobile phones, flat panels and the like. The log analysis process in the application program depends on repeated communication confirmation between developers and operators, products and the like, the process is complex, and both time cost and labor cost are high.

Disclosure of Invention

The present disclosure provides a log processing method, system, electronic device, storage medium, and computer program product, which can automatically complete log collection and processing without requiring developers to participate in operations and products in multiple ways, and can reduce time cost and labor cost by repeatedly communicating and confirming.

According to a first aspect of the present disclosure, there is provided a log processing method, including: receiving a log processing request from a client, wherein the log processing request comprises a log type; acquiring a log to be processed which accords with the log type; and performing step-by-step funnel aggregation statistics on the logs to be processed to obtain a processing result.

According to a second aspect of the present disclosure, there is provided a log processing method including: sending a log processing request to a server, wherein the log processing request comprises a log type; and receiving a funnel analysis processing result obtained by carrying out step-by-step funnel aggregation statistics on the to-be-processed logs of the server, wherein the to-be-processed logs are logs conforming to the log types.

According to a third aspect of the present disclosure, there is provided a log processing apparatus including: the system comprises a receiving request module, a log processing module and a log processing module, wherein the receiving request module is used for receiving a log processing request from a client, and the log processing request comprises a log type; the log obtaining module is used for obtaining the logs to be processed which accord with the log types; and the log processing module is used for carrying out step-by-step funnel aggregation statistics on the logs to be processed to obtain a processing result.

According to a fourth aspect of the present disclosure, there is provided a log processing apparatus including: the sending request module is used for sending a log processing request to the server, wherein the log processing request comprises a log type; and the receiving result module is used for receiving a processing result obtained by performing step-by-step funnel aggregation statistics on the to-be-processed logs of the server, wherein the to-be-processed logs are logs conforming to the log types.

According to a fifth aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described above.

According to a sixth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the aforementioned method.

According to a seventh aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the aforementioned method.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

The scheme provided by the embodiment can automatically complete the collection and processing of the logs, and improve the log processing efficiency.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a first schematic diagram of a log processing method according to an embodiment of the present disclosure;

FIG. 2 is a second schematic diagram of a log processing method according to one embodiment of the present disclosure;

FIG. 3 is a third schematic diagram of a log processing method according to one embodiment of the present disclosure;

FIG. 4 is a fourth schematic diagram of a log processing method according to one embodiment of the present disclosure;

FIG. 5 is a fifth diagram of a log processing method according to one embodiment of the present disclosure;

FIG. 6 is a sixth schematic diagram of a log processing method according to one embodiment of the present disclosure;

FIG. 7 is a seventh schematic diagram of a log processing method according to one embodiment of the present disclosure;

FIG. 8 is a schematic flow chart of a log management analysis method according to the related art;

FIG. 9 is a schematic diagram of a log processing flow in accordance with an embodiment of the present disclosure;

FIG. 10 is an architectural schematic of a log processing system according to an embodiment of the disclosure;

FIG. 11a is a schematic diagram of a data warehouse model in an embodiment in accordance with the present disclosure;

FIG. 11b is an exemplary diagram of a processing result in accordance with an embodiment of the present disclosure;

FIG. 12 is a first schematic diagram of a log processing apparatus according to an embodiment of the present disclosure;

FIG. 13 is a second schematic diagram of a log processing apparatus according to one embodiment of the present disclosure;

fig. 14 is a block diagram of an electronic device for implementing a log processing method of an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The present disclosure provides a log processing method, which can be applied to a server. As shown in fig. 1, the log processing method includes:

s101, receiving a log processing request from a client, wherein the log processing request comprises a log type;

s102, acquiring a log to be processed according with the log type;

s103, performing step-by-step funnel aggregation statistics on the log to be processed to obtain a processing result.

In the embodiment of the disclosure, the log processing method can be realized through interaction between the client and the server. The client may include an application installed in an electronic device, such as a cell phone, tablet, etc. For example, the user may check the feature log of interest at the client, thereby obtaining the log type of the feature log. And then generating a log processing request according to the log type of the characteristic log, and sending the request to a server, thereby automatically generating a processing result.

In the embodiment of the disclosure, the collection and processing of the logs can be automatically completed, and the log processing efficiency is improved. For example, developers do not need to participate in operation and multiple parts of products, and repeated communication and confirmation are achieved, so that the whole process can realize self-help of users, the collection and processing work of logs can be automatically completed, and the log processing efficiency is improved. Furthermore, time cost and labor cost can be reduced, so that products and operators can be helped to make better operation decisions, developers can be helped to release repeated development labor, and the overall research and development efficiency is improved.

The disclosure also provides a log processing method, which can be applied to a server. The method of this embodiment may include one or more features of the method of the above-described embodiment. As shown in fig. 2, in some embodiments, performing progressive funnel aggregation statistics on the to-be-processed log includes:

s201, according to the log attributes and the step-by-step using sequence of the log attributes, performing step-by-step funnel aggregation statistics on the logs to be processed. The progressive usage order of the log attributes may include the log attributes that each stage needs to use. The log attributes used by the primary level may include one kind or a plurality of kinds. Funnel aggregation statistics is carried out through log attributes and the step-by-step using sequence of the log attributes, funnel analysis results comprising multiple stages can be obtained, and the funnel aggregation statistics method is suitable for various analysis scenes.

In some implementations, the log attributes can include at least one of a device type, an application type, an action type, a log type, and a log generation time.

For example, the action type may be data content in JSON (JSON Object Notation) format. The action type may record the user's specific behavior, such as clicking, viewing, commenting, praising, forwarding, purchasing, refunding, etc.

For example, the log type can be a specific type meaning of various log rules of the configuration, such as a chapter type, a short video type, a live type, and the like.

For example, the device type may include a device used by a client used by a user and an operating system setting type, such as a mobile phone-android version, a mobile phone-IOS version, a computer-PC version, and the like.

For example, the application type may include a specific application used on the client. The attribute can conveniently support subsequent application expansion.

In the embodiment of the disclosure, the user can perform configuration, selection, combination and the like in various log attributes. For example, the number of users watching the live type column (based on the log type), and the number of users using the application a through the mobile phone side (based on the device type and the application type).

In the embodiment of the disclosure, the multi-level funnel aggregation statistics can be performed through various log attributes, which is beneficial to being applicable to various analysis scenes.

In addition, the user can configure the progressive execution order of the log attributes. For example, a first level uses a device type, a second level uses an application type, and a third level uses an action type. As another example, the first level uses the application type, the second level uses the device type, and the third level uses the log type. As another example, the first level uses the application type, the second level uses the log type, the third level uses the log generation time, and the fourth level uses the action type. The specific number of levels and the log attributes used by each level may be flexibly selected according to actual requirements, and are not limited herein.

In some embodiments, performing step-by-step funnel aggregation statistics on the to-be-processed log according to the log attributes and the step-by-step usage order of the log attributes, further includes: and counting the number of the (N + 1) th level objects from the nth level objects in the log to be processed according to the nth level objects and the nth log attributes, wherein N is a positive integer.

In the embodiment of the disclosure, the number of the (N + 1) th-level objects is determined from the nth-level objects, so that the number of the objects can be reduced step by step, and a funnel analysis result more meeting various scene requirements is obtained.

In some embodiments, the number of first level objects is counted from the pending log according to at least one log attribute.

In the embodiment of the present disclosure, the number of the first-level objects may be obtained statistically from the to-be-processed log according to one or more log attributes; the log to be processed can also be directly used as the first-level object. Specifically, according to a step-by-step use sequence configured by a user, the number of the (N + 1) th-level objects is obtained through statistics from the Nth-level objects according to the Nth-type log attributes. In the embodiment of the disclosure, the number of the first-level objects can be flexibly determined, and then the number of the objects can be reduced step by step, so that the funnel analysis result can be used as a basis.

For example, the number of the first-level objects is obtained from the logs to be processed by statistics according to the device type and the application type. Then, the number of second level objects from the first level objects is determined according to a certain action type. By analogy, the number of N +1 level objects from the nth level object can be determined according to the nth action type.

For another example, all the logs to be processed are taken as the first-level objects. And counting the number of the second-level objects from the logs to be processed according to the type of the equipment. And counting the number of the third-level objects from the second-level objects according to the application type. And counting the number of the fourth-level objects from the third-level objects according to the log type.

In some embodiments, as shown in fig. 3, the method further comprises:

s301, receiving a login request from the client, wherein the login request comprises an application program identifier and a first user identifier requesting to login;

s302, acquiring an operation log corresponding to the first user identifier on the application program according to the first user identifier and the application program identifier;

s303, storing the operation log, the first user identifier and the application program identifier in an associated manner.

In the embodiment of the present disclosure, after the client sends the login request and successfully logs in, the dotting log generated in a certain application program is uploaded to the background in real time for storage, and in this embodiment, the dotting log may be stored in a storage module, such as an ElasticSearch, and each log corresponds to the client ID of the user. The user identifier requesting login may include identification information such as an ID account number and a registered mobile phone number of the user in the application program. The ElasticSearch is a distributed storage engine, and can support full-text retrieval and quickly respond to a query request.

In the embodiment of the disclosure, the log can be stored in association with the user identifier, the application program identifier and the like, so that the stored log can be conveniently searched and used subsequently. Furthermore, the log can be uploaded in real time, data is not lost, and log data can be managed more conveniently.

In some embodiments, as shown in fig. 4, the method further comprises:

s401, verifying the operation log according to a preset log format in a dotting log specification;

s402, marking the operation logs which do not conform to the preset log format.

For example, the preset format of the log in the dotting log specification set by the user may be a JSON parsing format. And screening out the dotting logs which do not accord with the specification according to the JSON analysis format, and labeling highlight labels on the non-specification logs, wherein the process can be an illegal log rendering process. The user may select from the non-highlighted logs when selecting the log type. By marking the operation logs which do not conform to the preset format, illegal and legal logs can be distinguished, and the subsequent selection of a proper log type is facilitated.

In the embodiment of the disclosure, the log can be sorted in a descending order according to the time stamps generated by the log, so that the latest operation log of a user can be displayed on a page firstly, and dynamic verification can be performed preferentially.

In some embodiments, the operation log is analyzed according to a log service mapping relationship to obtain an attribute of the operation log. For example, a user-configured log service mapping relationship may include a log service meaning mapping rule, which may be recorded into a log dictionary in mysql (a relational database management system) that primarily stores log type identifications and corresponding service meanings. And calling dictionary meanings to render the logs, and analyzing the logs and marking service meanings of each log. The log attributes can be conveniently obtained through the log service mapping relation, and further, dynamic verification can be carried out.

In some embodiments, the method further comprises: calling a first log corresponding to the first user identification; and sending the first log to the client. Therefore, log query can be automatically realized based on the user identification, the query efficiency is high, and the query result is accurate.

For example, after the client logs in, the page can automatically jump to a log display page, and the native log content generated by the user operation is queried in real time according to the end ID reported by the user, and the query process automatically filters the end ID of the user as a condition, so that only the behavior log generated by the current user operation is queried. By adopting the scheme, various information of the log of the current ID is automatically displayed to the user, the unified management of the user on various log service meanings, log categories and the like can be realized, and the follow-up maintenance and expansion are facilitated.

In some embodiments, obtaining a pending log that conforms to the log type includes:

calling a second log corresponding to a log type, wherein the log type is a log type selected from unmarked logs in the first log;

acquiring a second user identifier corresponding to the second log;

and calling the log to be processed corresponding to the second user identification.

For example, if the types of the target logs are browsing commodity a and purchasing commodity a, all logs of the same type can be called according to the types to form a user group. And analyzing the operation logs of the users in the user group as logs to be analyzed, namely performing funnel analysis on all user logs conforming to the type according to the log type defined by the users. More extensive log data can be obtained through the logs in which the user is interested, so that richer analysis results can be obtained. In addition, the behavior habits of wider users can be further known, and further more reasonable operation decisions can be made.

The present disclosure provides a log processing method, which can be applied to a client. As shown in fig. 5, the log processing method includes:

s501, sending a log processing request to a server, wherein the log processing request comprises a log type;

s502, receiving a funnel analysis processing result obtained by performing step-by-step funnel aggregation statistics on the to-be-processed logs of the server, wherein the to-be-processed logs are logs conforming to the log types.

In the embodiment of the disclosure, the collection and processing work of the log can be automatically completed, and the log processing efficiency is improved. Further, time and labor costs may be reduced.

In some embodiments, as shown in fig. 6, the method further comprises:

s601, sending a login request by using a code scanning function of an application program, wherein the login request comprises an application program identifier and a first user identifier which requests to login on the application program;

s602, receiving and displaying a first log corresponding to the first user identifier.

For example, a user (product or operation, or a user member in another role) accesses the system, and uploads a unique identification code (terminal ID) on a mobile phone terminal to the system by scanning a two-dimensional code through the mobile phone. The user can conveniently log in the system and check information of each link in the log management.

In some embodiments, as shown in fig. 7, the method further comprises:

s701, selecting a log type from unmarked logs in the first log;

s702, generating the log processing request according to the log type.

In the embodiment of the disclosure, automation of the whole process and self-help of the user can be realized, and the user can select the type of the processing log according to the requirement of the user to obtain the personalized processing result. Furthermore, the system can help professionals to make better operation decisions, can help developers to release repeated development manpower, and improves the overall research and development efficiency.

As shown in fig. 8, log verification often requires log dotting design according to specific requirements of product and operation descriptions, then manual testing and network packet capturing are performed, and then log extraction is performed on captured data. And extracting results of the extracted logs according to a specific standard, processing the extracted logs by technical means such as script tasks and the like to generate detailed results, and communicating and confirming the detailed results with personnel such as products, operators and the like. The whole process has huge communication cost and extremely high requirement on development manpower, and much unnecessary communication cost and manpower cost are wasted in the past. At present, logs caught by various tools mainly comprise original logs, and corresponding meta information is not extracted. Such as the responsible person of the log, the time and the service meaning reported by the log, the access amount brought by the unit independent visitor of the log, and whether to forward the recommendation strategy. Therefore, operators are also required to go to the log management system to look at the details. In addition, whether the log is reported repeatedly or not in the same event cannot be directly analyzed from the debugging log at present.

For personnel such as products and operations, the specific detail disassembly of the user behavior funnel result is often required to be known, so that the behavior habits of the user can be further known, and a more reasonable decision can be made.

The embodiment of the disclosure can realize automation of the whole log processing flow, support self-help acquisition of various result details of products and operators, and improve the whole research and development efficiency.

When a professional checks an online problem or designs a dotting point, the professional needs to use the responsible APP to check the details of the reported behavior log. In the related art, a Debug packet of an App needs to be installed, a mobile phone and a Mac agent are set, log details are checked by a packet capturing tool, or the log details are checked in a test environment of a log management system (the Debug packet also needs to be installed), and the method is high in cost and low in efficiency.

As shown in fig. 9, in an embodiment of the present disclosure, a log processing system is provided. The system can realize a whole set of platform services through a big data technology means, and automatically reform links of reporting, checking, funnel analysis, log meta-information hosting and the like of the logs on the terminal, so that the labor cost is saved, and the research and development efficiency is improved. The log processing flow of the system can be based on the dynamic code scanning end log real-time verification and processing, and specifically comprises the following steps:

1. a user, such as a user member of a role such as product (PM, i.e., project management) or operation, or a user member of another role, accesses a platform through a terminal device such as a mobile phone terminal, and uploads a unique identification code (e.g., terminal ID) on the mobile phone terminal to a system through scanning a two-dimensional code by the mobile phone terminal.

2. Automatic log streaming. And uploading the logs generated by the user operating the application program of the mobile phone end to the system background for storage in real time.

The front end of the log processing system inquires the log content generated by user operation in real time through the end ID reported by the user code scanning, simultaneously supports the service meaning of the analysis log to be displayed, and gives a highlight prompt to the log which does not meet the standard.

3. One-key behavior analysis. When a user such as a professional in operation, PM and the like views the result through the application program, the user can check the interested feature log and click an analysis button such as a 'one-click funnel analysis' button to perform behavior analysis, so that an analysis report such as a funnel analysis report is automatically generated. The characteristic log can support funnel analysis of logs of the same category of all end users. For example, the user browses the purchase behavior log of the commodity, and the following funnel analysis can be performed on the user group: which users only browse the commodity, which users buy after browsing, which users initiate refunds after buying, etc.

Referring to fig. 10, the log processing flow can be described in detail in conjunction with the architecture of the present system.

First, the user side can perform log specification configuration through a page entry (e.g., an entry link) of the interaction layer of the platform front end. Configuration may involve log traffic mapping, dotting log specification settings, etc. The log meaning mapping rule is recorded into a log dictionary in the mysql, and the log type identifier and the corresponding service meaning can be stored. The dotting log specification may store an agreed JSON structure for verifying whether the actually generated log meets expectations. The log specification settings may utilize the functionality of the rule generator of the service response layer. The rule generator can also support functions of query condition splicing, timestamp sorting, illegal log rendering, native log query, dictionary meaning rendering, log specification checking and the like.

For example, the log query page is skipped, the query session is established according to the end ID, and the log query result is displayed in real time by using the query condition splicing function in the rule generator.

Secondly, the user accesses the system code scanning inlet, and after the two-dimensional code is scanned by the mobile phone, the test mode is started through the platform button. User operation behaviors, for example, operations of a mobile phone end application such as a hundred-degree APP, generated dotting logs (e.g., behavior logs) can be uploaded to a storage engine of a background data storage layer, for example, an ElasticSearch storage in real time, and each log may correspond to an end ID of a user. The ElasticSearch is a distributed storage engine, can well support full-text retrieval and quickly respond to query requests.

In addition, after the user scans the code, the platform page automatically jumps to the log display page (i.e. log query page jump), and displays the log in real time. At this time, the platform establishes an inquiry session according to the end ID reported by the code scanning of the user, and inquires the native log content generated by the user operation in real time. The inquiry process can automatically filter the end ID of the user as a condition, and can also utilize the function of splicing the inquiry conditions, thereby only inquiring the behavior log generated by the current user operation.

Then, the system renders the original log according to the log specification set by the user and the corresponding dictionary meaning (dictionary meaning rendering function), and labels the business meaning of each log. And moreover, dotting logs which do not conform to the specification (log specification verification function) can be screened according to the JSON analysis format, and highlight label labeling (illegal log rendering function) can be carried out on the non-specification logs. And finally, performing descending sorting (a function of sorting the timestamps) according to the timestamps generated by the logs, so that the latest operation logs of the user can be displayed on the page at first, and dynamic verification is facilitated.

Finally, the user can check the interested service meaning log according to the displayed log content. After the characteristic log is checked, an analysis button such as a 'one-click funnel analysis' button can be clicked to analyze the operation behavior of the user, and the system can give a detailed funnel analysis report. And the service response layer supports the result viewing at the front-end interaction layer through result pushing.

For example, as shown in fig. 11a, in the construction of the underlying data warehouse model, the historical inventory behavior detail data of the user on the client may be recorded by using the user behavior log detail table in the data warehouse, for example, the Hive data table. For example, the attribute detail data in the detail table may include core attributes of log type, log time, device type, application type, behavior content, and the like of the log. Hive is a distributed off-line analytical data warehouse, and can flexibly support the storage and analysis of large-scale data sets by relying on a distributed file system. This data can be periodically batch synchronized into the list from a search server, such as an ElasticSearch, that stores log data. When the user selects the corresponding type log and clicks on the funnel analysis button, e.g., "one-click funnel analysis", the system starts a calculation engine, e.g., Spark calculation analysis task. Spark is a distributed computing engine based on a memory, has good transverse expansion capability, can support rapid off-line computation and analysis of large-scale data, and can well support interaction with bottom data warehouses such as Hive and the like.

An example of the processing logic of this analysis task is as follows: and acquiring all log detail data conforming to the changed types from the user stock behavior log detail table of Hive according to the log types transmitted by the users. The device type, application type, behavior content, etc. of the combined log are subjected to progressive funnel aggregation statistics through Spark SQL, for example. Spark SQL is a Spark-based data analysis language, can support flexible processing and analysis of large-scale data, and can convert the calculation task of the data into the calculation of RDD through the form of SQL.

Examples of the process of funnel aggregation statistics may include:

(1) the number of users using the application program B through the mobile phone end D1;

(2) the number of users D2 who viewed the live category among the D1 number of users;

(3) the number of users D3 who clicked the hitched items (obtained by analyzing the behavior content) in the live column among the users D2;

(4) the number of users D4 who finally purchased hooked merchandise in the live column among the users D3;

(5) the number of users who initiated a refund among the population of users who finally purchased D5.

Finally, the analysis results may be pushed to the front-end page. The front end may form a behavior funnel analysis graph as shown in fig. 11b after receiving the data result through rendering.

The above is an example display performed by taking a mobile phone terminal as an example, in addition, logs of other devices such as a computer terminal can also be processed, and the like, so that users of products or operations and the like can more intuitively see which application types can more effectively promote sales and the like of a certain commodity, and thus, more rapid and more accurate marketing activities or product strategies and the like can be developed.

In the embodiment of the disclosure, a user can configure log rules through a platform page, the reported logs can be verified in real time by scanning codes, manual intervention is not required, and the efficiency of PM design dotting and the efficiency of terminal RD verification dotting correctness can be greatly improved. The efficiency of the problem on the daily troubleshooting line is also promoted by a wide margin. The user can directly select interested logs on a page, funnel analysis is carried out by one key in a self-help mode, an analysis report can be automatically generated by the system, and research and development efficiency can be remarkably improved. The platform manages log meta-information in a unified way, including log service meanings, dotting mapping rules and the like, so that the subsequent use is facilitated, and the subsequent expansion is supported. The code scanning can access the testing environment to carry out a series of verification and analysis processes, so that complicated and tedious setting and interaction processes are avoided, and unnecessary communication cost is saved.

The embodiment of the present disclosure further provides a log processing apparatus, as shown in fig. 12, the apparatus may be disposed in a server, and includes a request receiving module 1201, a log obtaining module 1202, and a log processing module 1203; the receiving request module 1201 is configured to receive a log processing request from a client, where the log processing request includes a log type;

an obtaining log module 1202, configured to obtain a to-be-processed log that conforms to the log type;

and the log processing module 1203 is configured to perform step-by-step funnel aggregation statistics on the logs to be processed to obtain a processing result.

Therefore, the device of the embodiment can be used for realizing self-help of the whole log processing flow without repeated communication confirmation of developers and operation and products, namely, the collection and processing of the logs can be automatically completed, the time cost and the labor cost can be reduced, and the products and the operators can be helped to make better operation decisions.

In some embodiments, the log processing module is further configured to perform step-by-step funnel aggregation statistics on the logs to be processed according to log attributes and a step-by-step use sequence of the log attributes.

In some embodiments, the log processing module is further configured to obtain, from the log to be processed, the number of N +1 th-level objects statistically from the nth-level objects according to the nth-level object and the nth log attribute, where N is a positive integer.

In some embodiments, the log processing module is further configured to count the number of first-level objects from the pending log according to at least one log attribute.

In some embodiments, the apparatus further comprises a log storage module configured to receive a login request from the client, the login request including an application identifier and a first user identifier requesting login; acquiring an operation log corresponding to the first user identifier on the application program according to the first user identifier and the application program identifier; and storing the operation log, the first user identification and the application program identification in an associated manner.

In some embodiments, the apparatus further includes a log checking module, configured to check the operation log according to a preset log format in a dotting log specification; and marking the operation logs which do not accord with the preset log format.

In some embodiments, the apparatus further includes a log analysis module, configured to analyze the operation log according to a log service mapping relationship, so as to obtain an attribute of the operation log.

In some embodiments, the apparatus further includes a log sending module, configured to retrieve a first log corresponding to the first subscriber identity; and sending the first log to the client.

In some embodiments, the log obtaining module is further configured to call a second log corresponding to a log type, where the log type is a log type selected from among the unmarked logs in the first log; acquiring a second user identifier corresponding to the second log; and calling the log to be processed corresponding to the second user identification.

The embodiment of the present disclosure further provides a log processing apparatus, as shown in fig. 13, the apparatus may be disposed in a client. The apparatus may include a request sending module 1301 and a result receiving module 1302. The sending and requesting module 1301 is configured to send a log processing request to a server, where the log processing request includes a log type; a result receiving module 1302, configured to receive a processing result obtained by performing progressive funnel aggregation statistics on the to-be-processed log of the server, where the to-be-processed log is a log conforming to the log type. In the embodiment of the disclosure, the user can finish the log acquisition and processing work by himself, and the time and labor cost are reduced.

In some embodiments, the apparatus further comprises a log display module, configured to send a login request using a code scanning function of an application, where the login request includes an application identifier and a first user identifier requesting to login on the application; and receiving and displaying a first log corresponding to the first user identifier. The user can conveniently log in the system and check information of each link in the log management.

In some embodiments, the apparatus further comprises a request generation module to select a log type in the first log that is not marked; and generating the log processing request according to the log type.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 14 shows a schematic block diagram of an example electronic device 1400 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 14, the electronic device 1400 includes a computing unit 1401 that can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)1402 or a computer program loaded from a storage unit 1408 into a Random Access Memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the electronic device 1400 can also be stored. The calculation unit 1401, the ROM 1402, and the RAM 1403 are connected to each other via a bus 1404. An input/output (I/O) interface 1405 is also connected to bus 1404.

A number of components in the electronic device 1400 are connected to the I/O interface 1405, including: an input unit 1406 such as a keyboard, a mouse, or the like; an output unit 1407 such as various types of displays, speakers, and the like; a storage unit 1408 such as a magnetic disk, optical disk, or the like; and a communication unit 1409 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1409 allows the electronic device 1400 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 1401 may be a variety of general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The computing unit 1401 performs the respective methods and processes described above. For example, in some embodiments, the various methods described above may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1408. In some embodiments, part or all of the computer program can be loaded and/or installed onto the electronic device 1400 via the ROM 1402 and/or the communication unit 1409. When loaded into RAM 1403 and executed by computing unit 1401, may perform one or more of the steps of the respective methods described above. Alternatively, in other embodiments, the computing unit 1401 may be configured by any other suitable means (e.g. by means of firmware) to perform the respective methods described above.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims (29)

1. A log processing method, comprising:

receiving a log processing request from a client, wherein the log processing request comprises a log type;

acquiring a log to be processed which accords with the log type;

and performing step-by-step funnel aggregation statistics on the logs to be processed to obtain a processing result.

2. The method of claim 1, wherein performing progressive funnel aggregation statistics on the to-be-processed logs comprises:

and performing step-by-step funnel aggregation statistics on the logs to be processed according to the log attributes and the step-by-step using sequence of the log attributes.

3. The method of claim 2, wherein the log attributes include at least one of a device type, an application type, an action type, a log type, and a log generation time.

4. The method of claim 3, wherein performing progressive funnel aggregation statistics on the to-be-processed logs according to log attributes and a progressive usage order of the log attributes, further comprises:

and counting the number of the (N + 1) th level objects from the nth level objects in the log to be processed according to the nth level objects and the nth log attributes, wherein N is a positive integer.

5. The method of claim 3, wherein the number of first level objects is counted from the pending log according to at least one log attribute.

6. The method of any of claims 1 to 5, further comprising:

receiving a login request from the client, wherein the login request comprises an application program identifier and a first user identifier requesting login;

acquiring an operation log corresponding to the first user identifier on the application program according to the first user identifier and the application program identifier;

and storing the operation log, the first user identification and the application program identification in an associated manner.

7. The method of claim 6, further comprising:

verifying the operation log according to a log preset format in a dotting log specification;

and marking the operation logs which do not accord with the preset log format.

8. The method of claim 6 or 7, further comprising:

and analyzing the operation log according to the log service mapping relation to obtain the attribute of the operation log.

9. The method of any of claims 6 to 8, further comprising:

calling a first log corresponding to the first user identification;

and sending the first log to the client.

10. The method of claim 9, wherein obtaining a pending log that conforms to the log type comprises:

calling a second log corresponding to a log type, wherein the log type is a log type selected from unmarked logs in the first log;

acquiring a second user identifier corresponding to the second log;

and calling the log to be processed corresponding to the second user identification.

11. A log processing method, comprising:

sending a log processing request to a server, wherein the log processing request comprises a log type;

and receiving a processing result obtained by performing step-by-step funnel aggregation statistics on the to-be-processed logs of the server, wherein the to-be-processed logs are logs conforming to the log types.

12. The method of claim 11, further comprising:

sending a login request by using a code scanning function of an application program, wherein the login request comprises an application program identifier and a first user identifier which requests to login on the application program;

and receiving and displaying a first log corresponding to the first user identifier.

13. The method of claim 12, further comprising:

selecting a log type from the unmarked logs in the first log;

and generating the log processing request according to the log type.

14. A log processing apparatus comprising:

the system comprises a receiving request module, a log processing module and a log processing module, wherein the receiving request module is used for receiving a log processing request from a client, and the log processing request comprises a log type;

the log obtaining module is used for obtaining the logs to be processed which accord with the log types;

and the log processing module is used for carrying out step-by-step funnel aggregation statistics on the logs to be processed to obtain a processing result.

15. The apparatus of claim 14, wherein the log processing module is further configured to perform progressive funnel aggregation statistics on the log to be processed according to log attributes and a progressive usage order of the log attributes.

16. The apparatus of claim 15, wherein the log attributes comprise at least one of a device type, an application type, an action type, a log type, and a log generation time.

17. The apparatus according to claim 16, wherein the log processing module is further configured to statistically obtain the number of N + 1-th level objects from the nth level objects in the log to be processed according to the nth level objects and nth log attributes, where N is a positive integer.

18. The apparatus of claim 16, wherein the log processing module is further configured to count a number of first level objects from the pending log based on at least one log attribute.

19. The apparatus according to any one of claims 14 to 18, further comprising a log storage module for receiving a login request from the client, the login request comprising an application identity and a first user identity requesting login; acquiring an operation log corresponding to the first user identifier on the application program according to the first user identifier and the application program identifier; and storing the operation log, the first user identification and the application program identification in an associated manner.

20. The device of claim 19, further comprising a log verification module configured to verify the operation log according to a preset log format in a dotting log specification; and marking the operation logs which do not accord with the preset log format.

21. The apparatus according to claim 19 or 20, further comprising a log parsing module, configured to parse the operation log according to a log service mapping relationship, so as to obtain an attribute of the operation log.

22. The apparatus according to any one of claims 19 to 21, further comprising a log sending module configured to retrieve a first log corresponding to the first subscriber identity; and sending the first log to the client.

23. The apparatus of claim 22, wherein the get log module is further configured to retrieve a second log corresponding to a log type, the log type being a log type selected from among unmarked logs in the first log; acquiring a second user identifier corresponding to the second log; and calling the log to be processed corresponding to the second user identification.

24. A log processing apparatus comprising:

the sending request module is used for sending a log processing request to the server, wherein the log processing request comprises a log type;

and the receiving result module is used for receiving a processing result obtained by performing step-by-step funnel aggregation statistics on the to-be-processed logs of the server, wherein the to-be-processed logs are logs conforming to the log types.

25. The apparatus of claim 24, further comprising a log display module for sending a login request using a code scanning function of an application, the login request including an application identification and a first user identification requesting to login on the application; and receiving and displaying a first log corresponding to the first user identifier.

26. The apparatus of claim 25, further comprising a request generation module to select a log type in a log that is not marked in the first log; and generating the log processing request according to the log type.

27. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10 or 11-13.

28. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any of claims 1-10 or 11-13.

29. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-10 or 11-13.

Images