CN117687981A - Log management method and device, storage medium and electronic equipment - Google Patents

Log management method and device, storage medium and electronic equipment Download PDF

Info

Publication number
CN117687981A
CN117687981A CN202211073100.8A CN202211073100A CN117687981A CN 117687981 A CN117687981 A CN 117687981A CN 202211073100 A CN202211073100 A CN 202211073100A CN 117687981 A CN117687981 A CN 117687981A
Authority
CN
China
Prior art keywords
instruction
address
log
target
characteristic value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211073100.8A
Other languages
Chinese (zh)
Inventor
林欣达
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Beijing Co Ltd
Original Assignee
Tencent Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Beijing Co Ltd filed Critical Tencent Technology Beijing Co Ltd
Priority to CN202211073100.8A priority Critical patent/CN117687981A/en
Publication of CN117687981A publication Critical patent/CN117687981A/en
Pending legal-status Critical Current

Links

Landscapes

  • Debugging And Monitoring (AREA)

Abstract

The embodiment of the application discloses a log management method, a log management device, a storage medium and electronic equipment. The method comprises the following steps: extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack; analyzing the first instruction address to obtain a target original address, wherein the target original address refers to an original instruction address of the first instruction in a symbol mirror image; sequentially extracting the second instruction addresses; determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction; and generating a log according to the target original address, the first characteristic value and the second characteristic value corresponding to each second instruction address. The method and the device can reduce the log data volume.

Description

Log management method and device, storage medium and electronic equipment
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a log management method, a log management device, a storage medium and electronic equipment.
Background
The log data is widely applied to the computer technology, is commonly used in the fields of operation breakdown, katon detection, experience feedback and the like, and mainly refers to a log formed by information related to instructions generated in the operation process of an application program. Because the data size of such logs may be large, the related art may collect the logs at the client and transmit the logs to the server for management, but the collection efficiency, transmission efficiency and management efficiency of such logs are still to be further improved.
Disclosure of Invention
In order to solve at least one technical problem described above, embodiments of the present application provide a log management method, a log management device, a storage medium, and an electronic device.
In one aspect, an embodiment of the present application provides a log management method, applied to a client, where the method includes:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address points to an original instruction address of the first instruction in a symbol mirror image;
extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
Determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
and generating a log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
In another aspect, an embodiment of the present application provides a log management device, applied to a server, where the device includes:
obtaining a first target log through interaction with a client;
the first target log is obtained by the client through the following method:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address points to an original instruction address of the first instruction in a symbol mirror image;
extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
Determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
and generating the first target log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
In another aspect, embodiments of the present application provide a computer readable storage medium having at least one instruction or at least one program stored therein, where the at least one instruction or at least one program is loaded and executed by a processor to implement a log management method as described above.
In another aspect, an embodiment of the present application provides an electronic device including at least one processor, and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor, and the at least one processor implements a log management method by executing the instructions stored by the memory.
In another aspect, embodiments of the present application provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement a log management method as described above.
The embodiment of the application provides a log management method, a device, a storage medium and electronic equipment, wherein in the link of log acquisition, the log management method does not need to perform symbol restoration on an instruction, but only needs to acquire an original instruction address corresponding to a first instruction, and only needs to acquire corresponding characteristic value information of other increment instructions, and the log can be generated by the two parts of information. The aggregation deduplication can be completed only based on the characteristic value information in the log at the server side, so that the management efficiency of the server is improved, and the speed of aggregation deduplication is improved.
Drawings
In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or of the related art, the following description will briefly explain the drawings required to be used in the embodiments or the related art, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort to those skilled in the art.
FIG. 1 is a schematic diagram of a possible implementation framework of a log management method according to an embodiment of the present application;
fig. 2 is a flow chart of a log management method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of an instruction address mapping relationship according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of the relationship between instruction analysis tools used in embodiments of the present application;
FIG. 5 is a schematic diagram of data associated with a tag generator used in embodiments of the present application;
FIG. 6 is a log identification schematic provided in an embodiment of the present application;
FIG. 7 is a schematic diagram of log storage relationships provided in an embodiment of the present application;
fig. 8 is a schematic diagram of instruction information in the first target log or the second target log provided in the embodiment of the present application;
FIG. 9 is a schematic diagram showing the comparison of the logs obtained in the related art provided in the embodiment of the present application and the embodiment of the present application;
FIG. 10 is a diagram showing the comparison of the data amounts of logs obtained in the related art provided in the embodiment of the present application and the embodiment of the present application;
FIG. 11 is a flow chart of a client and server dual-end interaction scheme provided by an embodiment of the present application;
FIG. 12 is a block diagram of a log management device provided by an embodiment of the present application;
FIG. 13 is a block diagram of another log management device provided by an embodiment of the present application;
Fig. 14 is a schematic hardware structure of an apparatus for implementing the method provided in the embodiment of the present application.
Detailed Description
The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the embodiments of the present application, are within the scope of the embodiments of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the embodiments of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In order to make the objects, technical solutions and advantages disclosed in the embodiments of the present application more apparent, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present application embodiments and are not intended to limit the present application embodiments.
The terms "first" and "second" are used below for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more. In order to facilitate understanding of the technical solutions and the technical effects produced by the embodiments of the present application, the embodiments of the present application first explain related terms:
cloud technology (Cloud technology): the hosting technology is used for integrating hardware, application programs, networks and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud computing business model application-based network technology, information technology, integration technology, management platform technology, application technology and the like can be collectively called to form a resource pool, and the resource pool is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a large amount of computing, storage resources, such as video websites, picture-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.
The intelligent transportation system (Intelligent Traffic System, ITS), also called intelligent transportation system (Intelligent Transportation System), is a comprehensive transportation system which uses advanced scientific technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operation study, artificial intelligence, etc.) effectively and comprehensively for transportation, service control and vehicle manufacturing, and enhances the connection among vehicles, roads and users, thereby forming a comprehensive transportation system for guaranteeing safety, improving efficiency, improving environment and saving energy.
The intelligent vehicle-road cooperative system (Intelligent Vehicle Infrastructure Cooperative Systems, IVICS), which is simply called a vehicle-road cooperative system, is one development direction of an Intelligent Transportation System (ITS). The vehicle-road cooperative system adopts advanced wireless communication, new generation internet and other technologies, carries out vehicle-road dynamic real-time information interaction in an omnibearing manner, develops vehicle active safety control and road cooperative management on the basis of full-time idle dynamic traffic information acquisition and fusion, fully realizes effective cooperation of human-vehicle roads, ensures traffic safety, improves traffic efficiency, and forms a safe, efficient and environment-friendly road traffic system.
CPU generally refers to a central processing unit. A central processing unit (Central Processing Unit, abbreviated as CPU) is used as an operation and control core of the computer system, and is a final execution unit for information processing and program running.
Instruction (Instruction): instructions are the single CPU operations defined by the CPU architecture in computer technology, such as data transfer, arithmetic operations, flow control, etc., are the smallest unit of execution for the execution of computer software/applications.
Instruction address (Instruction Address): instruction addresses are used to describe the specific location of computer instructions in executable data of a software/application package, in computer technology, used for functional debugging, defect localization, etc., in the form of numbers.
Instruction address difference (Instruction Address Difference): the instruction addresses are represented in the executable data of the software/application package by consecutive numbers without readability, and the instruction address difference is represented by the length of the data segments between the two instruction addresses in the executable data of the software/application package.
Instruction number difference (Instruction Count Difference): the instruction number difference indicates the offset of the symbol mirror image where the nth instruction address is located and the symbol mirror image of the first instruction address in the consecutive instruction addresses in the log data.
Symbol (Symbol): symbols are represented in computer technology as a string of readable characters, including high-level programming language statements used by the developer in programming, assembly instruction statements of the CPU architecture set.
Symbol Images (symbol Images): symbol mirroring is a binary data block of a software/application package, storing important data including continuous executable instructions, consolidated symbol information, etc., in particular, in the embodiments of the present application, a data block storing continuous executable instructions.
Symbol Reduction (Symbol Reduction): symbol reduction is a fundamental operation in computer technology that represents a computational process of converting a nonsensical string of digital information into a meaningful and readable string of descriptive executable code.
Characteristic value: the characteristic value is one of the most common data in data matching, is a result obtained by carrying out a series of data calculation and change on stored data, and is mainly used for quickly searching needed data in the later data query process.
Log Data (Log Data): the log data is a data file that is sent by the client to the server through the network and covers various different types of data information, and in the embodiment of the present application, the term log data may refer to a log type that is sent by the client and that is added with address information of an execution instruction in the execution process of a software/application package.
Labeling (Labeling): in the computer technology, data are stored in various file formats such as text, software/application, images, videos and the like, and tagging is to endow different abstract description information of files to data content based on the files, so that the identifiability of the files is improved, and the requirements of quick classification and high-efficiency searching of the files are met.
Number difference (Number Difference): the number difference represents the difference between two numbers, and in this embodiment may refer to the number difference of the instruction address and the number difference of the instruction address.
Data Retrieval (Data Retrieval): the data retrieval is a common data searching mode in the computer technology, and the efficiency of data searching is improved by modifying the data storage mode, the data tagging mode and the like.
Call Stack (Call Stack): the call stack is a special storage structure in the computer technology, and instructions in the software/application execution process are stored in the stack next to each other and are stored in the order of first and last.
Calculation force (Calculated Performance): the computational effort represents the hardware computational performance in computer technology, the more instructions the hardware is able to execute per unit time, the higher the computational effort.
Polymerization de-duplication (Polymerization & Removal Duplicate): one of the basic operations in computer technology, for two pieces of data a and B, if data a is a subset of data B is true, a is B's duplicate data, removing a retains B.
Bugly is one of the services opened for mobile developers, and provides professional crash analysis, crash analysis and other quality services for the mobile developers. Bugly can help mobile internet developer find out in time that controls the unusual, understand more comprehensively and fix the unusual, repair the unusual of solution more high-efficient. The mobile developer can quickly find out the breakdown and the jamming of the user in the using process, and quickly locate and solve the problem according to the reported information.
Fabric is one of the blockchain frameworks. Has been adopted by a number of industries such as education, healthcare, the internet of things, logistics, supply chains, and the like. Recently, fabric-based proposals for air traffic management blockchains for security, authentication and privacy have been published.
The alliance trillion-level collection platform has billion-level data processing capability and million-level/second data service capability.
The log data is widely applied in the computer technology, is commonly used in the fields of operation breakdown, katon detection, experience feedback and the like, and mainly refers to a log formed by information related to instructions generated in the operation process of an application program, and is widely used in the scenes of program analysis, reverse engineering and the like. In the related art, log management services for such logs can be provided for clients through various open services such as big, fabric, alliance statistics and the like, and the services need to rely on the clients to perform log collection, and the general log collection operation and the log management services of the server side comprise the following steps:
Step 1, acquiring instruction address information of a call stack in an operation state, and acquiring hexadecimal format digital character string information with data of 0x000000 xxxx.
And 2, based on the instruction address information acquired in the step 1, searching corresponding symbol mirror image data in the executable instruction data of the client.
And step 3, carrying out symbol reduction on the instruction pointed by the instruction address information based on the symbol mirror image data acquired in the step 2 to obtain symbol character information. This step may fail.
And 4, generating a log based on the data in the steps 1, 2 and 3, and transmitting the log to a server through network transmission, wherein the server can provide log management service for the client.
And step 5, if the execution of the step 3 fails, the server receives the log and then re-performs symbol reduction on the instruction failed in the step 3 to obtain symbol character information.
And 6, if the step 3 fails to be executed, calculating the characteristic value of the symbol character information based on the result of the step 5, and labeling the log based on the characteristic value, and if the step 3 succeeds to be executed, calculating the characteristic value of the symbol character information based on the result of the step 3, and labeling the log based on the characteristic value.
And 7, searching whether the logs with the same characteristic value exist in the database by the server, and if so, merging and deduplicating.
The above related art has some problems, and the embodiments of the present application are briefly analyzed as follows:
(1) The performance is poor, and in the case that the general computing power of the client is lower than that of the server, the related art needs to perform symbol recovery calculation on a large number of instruction addresses at the client, as in the previous step 3.
(2) The transmission data volume is large, and most symbol restoration work is carried out on the client, so that the uploaded log can carry data such as symbol mirror image information, preliminarily restored symbol character information, overlong instruction addresses and the like.
(3) The duplicate checking efficiency is low, and when the server aggregates and de-duplicates the logs, the server needs to calculate the characteristic values of all symbol information of the related logs and then compares the characteristic values to de-duplicate the logs.
According to the foregoing, three links of client-side log collection, log transmission and server-side log management are involved in the log management process, and the client-side log collection needs to perform symbol restoration, so that the calculation amount of the client-side is large, and the generated log data amount is also large, which directly causes the burden of log transmission to be increased. In addition, in the log management link at the server end, because resources consumed by redundant logs need to be reduced, aggregation and deduplication are generally needed for the logs, and because the logs themselves comprise information of symbol restoration, the log data size is large, and the aggregation and deduplication process needs to consider the information of symbol restoration, so that the aggregation and deduplication efficiency is low. In view of this, the embodiment of the application discloses a log management method, which does not need to perform symbol restoration on an instruction in a link of log acquisition, but only needs to acquire an original instruction address corresponding to a first instruction, and only needs to acquire corresponding characteristic value information of other increment instructions, so that a log can be generated by the two parts of information, and the log generation process does not need to rely on symbol restoration, which obviously reduces the data volume of the log, improves the acquisition speed of the log, and reduces the resources consumed by log generation, thereby reducing the time cost and the bandwidth cost consumed by log transmission. The aggregation deduplication can be completed only based on the characteristic value information in the log at the server side, so that the management efficiency of the server is improved, and the speed of aggregation deduplication is improved.
The embodiment of the application may be applied to public cloud, private cloud or hybrid cloud scenarios, for example, the log generated in the application may be configured in the public cloud, private cloud or hybrid cloud, or the server for log management may be configured in the public cloud, private cloud or hybrid cloud. A Private Cloud (Private Cloud) is a Cloud infrastructure created with software and hardware resources within a firewall for organizations or departments within an enterprise to share resources within a data center. Public clouds (Public clouds) generally refer to clouds that third party providers provide to users that can use, and are generally available over the Internet, and may be free or low cost, with the core attribute of the Public clouds being shared resource services. There are many examples of such clouds that can provide services throughout the open public network today. HybrID Cloud (HybrID Cloud) fuses Public Cloud (Public Cloud) and Private Cloud (Private Cloud), which are the main modes and development directions of Cloud computing in recent years. Private clouds are mainly for enterprise users, and for safety reasons, enterprises prefer to store data in the private clouds, but at the same time wish to obtain computing resources of public clouds, in which case hybrid clouds are increasingly adopted, and mix and match the public clouds and the private clouds to obtain the best effect.
Referring to fig. 1, fig. 1 is a schematic diagram of one possible implementation framework of a log management method according to an embodiment of the present disclosure, and as shown in fig. 1, the implementation framework may at least include a client 10 and a log management server 20, where the client 10 and the log management server 20 communicate through a network 30. The client 10 is configured to collect and generate a log, and send the log to the log management server 20, and the log management server 20 may receive the log by communicating with the client 10, thereby managing the log.
Specifically, the client 10 may perform the following operations:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address refers to an original instruction address of the first instruction in a symbol mirror image;
extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
And generating a log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
In particular, the log management server 20 may store, aggregate, deduplicate, or provide a query service based on the stored logs, or other visualization service, the logs obtained from the client 10.
The log management server 20 may be located in a cloud environment, which may be any node of the cloud environment, where any node may provide the log management service described above.
The above-described framework of embodiments of the present invention may provide log management capabilities required for applications in various scenarios, including, but not limited to, cloud technology, cloud gaming, cloud rendering, artificial intelligence, intelligent transportation, assisted driving, video media, intelligent communities, instant messaging, and the like. The components in the framework may be terminal devices or servers. The terminal equipment comprises, but is not limited to, mobile phones, computers, intelligent voice interaction equipment, intelligent household appliances, vehicle-mounted terminals and the like.
Referring now to the log management method according to the embodiments of the present application, fig. 2 is a schematic flow chart of a log management method according to the embodiments of the present application, where the log management method may be executed on the client 10. Embodiments of the present application provide method operational steps as described above in the embodiments or flowcharts, but may include more or fewer operational steps based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one way of performing the order of steps and does not represent a unique order of execution. When implemented in a system, terminal device or server product in practice, the methods illustrated in the embodiments or figures may be performed sequentially or in parallel (e.g., in a parallel processor or multithreaded environment), and may include:
S101, extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack.
The preset operation is not limited to this embodiment, and may be a sequence formed by a series of instructions generated when the user triggers the application program to execute, for example, the user triggers preset operation 1, where the preset operation 1 is implemented by sequentially executing instructions A, B, C, D, E, and then triggering of the preset operation 1 may result in the call stack being respectively corresponding to the addresses of the loaded instructions A, B, C, D, E. Alternatively, the preset operation may be a sequence formed by a series of instructions generated when the application program spontaneously executes, for example, the preset operation 2 is automatically executed, and the preset operation 2 is implemented by sequentially executing the instructions A, B, D, E, so that triggering of the preset operation 2 may result in the addresses corresponding to the loaded instructions A, B, D, E in the call stack respectively.
Taking the preset operation 1 as an example, the first instruction of the foregoing is the instruction a, and the address corresponding to the instruction a in the call stack is the first instruction address.
S102, analyzing the first instruction address to obtain a target original address, wherein the target original address refers to an original instruction address of the first instruction in a symbol mirror image.
Please refer to fig. 3, which illustrates an instruction address mapping relationship. The association between instruction addresses in the call stack and the actual instruction addresses in the symbol mirror can be shown in fig. 3. The instruction address in the call stack and the instruction address in the symbol mirror may be considered to exist in the user address space, and thus the purpose of step S102 is to analyze its corresponding address in the symbol mirror based on the instruction address in the call stack.
In fact, the instruction addresses located in the call stack are derived based on the principle of process address space layout randomization. In short, a random base address is randomly allocated in the process of executing the application program, and based on the random base address and the instruction address of the instruction in the binary address space (symbol mirror image) corresponding to the application program, the instruction address of the instruction in the call stack can be uniquely determined, and step S101 is to extract the instruction address in the call stack from the call stack, which is an instruction address in a running state, and only if the address is analyzed, the original instruction address of the corresponding instruction in the symbol mirror image can be obtained.
As in fig. 3, there is also a random load address and an address of the dynamic link library in the user address space, that is, if the original instruction address of the instruction in the symbol image is determined, both the random load address and the address of the dynamic link library may ultimately affect the instruction address of the running state of the instruction in the call stack. The purpose of step S102 is to analyze the factor affecting the first instruction address of the first instruction in the call stack, so as to reversely analyze the original instruction address of the first instruction in the sign mirror based on the first instruction address.
In one embodiment, the executable binary instruction segment information associated with the preset operation can be obtained by a binary enumeration mode; and performing address retrieval based on the executable binary instruction segment information to obtain a random base address associated with the first instruction address, and obtaining the target original address based on the random base address and the executable binary instruction segment information.
In particular, the binary enumeration operation described above may be performed using a binary enumerator that may map the binary address space of an application into the form of a set of several sign-mirrored segments (executable binary instruction segment information). The address retriever may be used to perform the above address retrieval, that is, on the basis of the above-mentioned number of symbol mirror segments, a random base address associated with the first instruction address and a target original address may be analyzed, where the target original address may be characterized by the identity of the symbol mirror segment in which it is located and an offset of the target original address relative to the symbol mirror segment in which it is located. The embodiments of the present application are not limited to the sources of the binary enumerator and the address retriever, and may refer to related technologies, or may be designed by themselves. The address retriever not only can analyze the target original address based on the random base address, but also can consider the sharing instruction information of dynamic links in the analysis process to ensure the accuracy of analysis results.
S103, extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack.
S104, determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction number difference value of the corresponding instruction relative to the first instruction.
For a sequence formed by instructions associated with a preset operation, only the first instruction needs to be analyzed for a target original address, and the subsequent instructions only need to record some increment information of the instructions relative to the first instruction, so that the data volume of a log can be remarkably reduced by the operation. Steps S103 to S104 are steps of sequentially extracting delta information of other instructions located after the first instruction. Taking the preset operation 1 as an example by sequentially executing the instructions A, B, C, D, E, the steps S103-S104 are steps of sequentially extracting the first characteristic value and the second characteristic value corresponding to the instruction B, C, D, E, respectively. Taking the first eigenvalue of instruction C as an example, the first eigenvalue characterizes the address distance difference between instruction C and instruction a in binary space, the second eigenvalue characterizes the instruction number difference between instruction C and instruction a, for example, for the sequence formed by instruction A, B, C, D, E, instruction C is the third instruction, instruction a is the first instruction, and the instruction number difference is 3-1=2.
In one embodiment, the determining the first feature value and the second feature value for each second instruction address includes:
s201, address retrieval is carried out based on the executable binary instruction segment information, and a random base address associated with the second instruction address is obtained.
S202, obtaining the original instruction address of the instruction corresponding to the second instruction address in the symbol mirror image according to the random base address associated with the second instruction address and the executable binary instruction segment information.
The execution of steps S201-S202 is based on the same inventive concept as step S102, that is, the original instruction address of the instruction corresponding to the second instruction address in the symbol mirror image may be obtained based on the binary enumerator and the address retriever.
S203, determining a distance difference value between an original instruction address of an instruction corresponding to the second instruction address in the symbol mirror image and the target original address as the first characteristic value.
S204, determining the instruction quantity difference value between the instruction corresponding to the second instruction address and the first instruction as the second characteristic value.
The execution of steps S203-S204 may be performed based on the tag generator. Referring to FIG. 4, a schematic diagram of the relationship between instruction analysis tools used in embodiments of the present application is shown. The instruction address generator can be understood as the total software tool required to call in the log generation process, and comprises a binary enumerator, an address retriever and a label generator, wherein the first two instruction address generators can be used for analyzing the original instruction address of a certain instruction in a binary space, and the second instruction address generators are used for generating two characteristic values. Referring to fig. 5, a schematic diagram of data related to a tag generator used in an embodiment of the present application is shown. The label generator marks the first characteristic value and the second characteristic value through the original instruction address of the instruction corresponding to the second instruction address in the symbol mirror image, wherein the original instruction address of the instruction corresponding to the second instruction address in the symbol mirror image is characterized by a random base address ASLR output by an address retriever, an identification image index of a symbol mirror image section where the random base address ASLR is located, and an offset of the random base address ASLR relative to the symbol mirror image section where the random base address ASLR is located.
S105, generating a log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
Specifically, according to the sequence of the instruction corresponding to the preset operation, the data organization is performed on the target original address, the first characteristic value corresponding to each second instruction address, and the second characteristic value corresponding to each second instruction address, so as to obtain the log.
For the first instruction, the embodiment of the application records the target original address, and for the incremental instruction, the embodiment of the application only records the incremental information, namely the first characteristic value and the second characteristic value, that is, the original address corresponding to the incremental instruction in the binary space is not required to be recorded, and no sign reduction is required for the first instruction or the incremental instruction, so that the data volume of the log is obviously reduced.
The characteristic values in the embodiment of the application comprise the address difference information and the instruction quantity difference information of the instructions, so that the positions of the increment instructions in the instruction sequence and the content of the increment instructions can be comprehensively described, and compared with the related technology, the information cannot be lost, but the two characteristic values can be obtained quickly through simple difference, the log generation speed is improved, and the retrieval speed and the aggregate deduplication speed of the logs can be improved in a log management link.
In one embodiment, the method further comprises: compressing the log based on serialization processing to obtain a log sequence; and transmitting the log sequence to a server. To further reduce transmission consumption, serialization may be performed prior to transmission of the log, with the purpose of compressing its data size. For example, in one embodiment, protocol Buffers may be used to accomplish this serialization processing based compression. The Protocol Buffers are a data serialization format, and have various advantages of supporting multiple programming languages, small serialized data volume, high anti-serialization speed, automatic generation of serialized and anti-serialized codes, and the like. Compared with an object numbered musical notation, the method further reduces the data volume and improves the reverse serialization speed.
In one embodiment, there is also provided a log management method, which is applied to a server, the method including: obtaining a first target log through interaction with a client; the generation of the first target log is performed at the client, and the generation mode is based on the same inventive concept as the above, and is not described in detail. Briefly, a client may perform the following operations:
S301, extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack.
S302, analyzing the first instruction address to obtain a target original address, wherein the target original address refers to an original instruction address of the first instruction in a symbol mirror image.
S303, extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack.
S304, determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction number difference value of the corresponding instruction relative to the first instruction.
S305, generating the first target log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
In one embodiment, the first target log may be obtained by the client through compression based on serialization and transmitted to the server, and the server may obtain the first target log by obtaining a log sequence from the client and performing deserialization on the log sequence.
The server needs to store the first target log after obtaining the first target log, in fact, the server may store the log associated with the first target log, even the log same as the first target log, that is, redundancy is generated, in order to reduce redundancy, the server may perform aggregate deduplication on each log sent to the server to reduce log redundancy, and the key of aggregate deduplication is to identify the log with the association, for example, identify the log with complete independence, identify the log with complete identity, and identify the log with the subordinate relationship.
In one embodiment, please refer to fig. 6, which illustrates a log identification schematic. The log may be identified by:
s401, determining a second target log, wherein the second target log and the first target log have the same target original address.
In the embodiment of the present application, the executable instruction data of the application programs of the clients with the same version and the same hardware can be considered to be consistent, so that log management can be performed according to the executable instruction data. Specifically, the root node may be determined according to the application version information and the hardware model information; and storing the first target log or a subsequent log aggregation result obtained based on the first target log in a storage space corresponding to the root node. If the root node originally exists, the root node is directly stored, and if the root node originally does not exist, the root node is newly built.
Any one of the logs in the storage space corresponding to the root node, which is different from the first target log, can be determined as the second target log as long as the second target log has the same target original address as the first target log. Of course, if any one of the storage spaces corresponding to the root node is different from the other logs of the first target log, the other logs of the storage space corresponding to the root node are not necessarily irrelevant to the first target log if the other logs of the storage space corresponding to the root node are not identical to the first target log in the original target address, and log identification is not required. As shown in fig. 7, which shows a log storage relationship diagram. For a root node, a plurality of logs may be stored in a corresponding storage space, each log has a corresponding target original address, and the logs having the same target original address may also be a plurality of logs.
S402, acquiring a first target value group associated with a first target instruction in the first target log, wherein the first target instruction is an Nth instruction in the first target log, N is an integer which is greater than 1 and smaller than or equal to M, M is the minimum value of the total number of instructions contained in the first target log and the total number of instructions contained in the second target log, and the first target value group comprises a first characteristic value corresponding to the first target instruction and a second characteristic value corresponding to the first target instruction.
S403, acquiring a second target value group associated with a second target instruction in the second target log, wherein the second target instruction is an N-th instruction in the second target log, and the second target value group comprises a first characteristic value corresponding to the second target instruction and a second characteristic value corresponding to the second target instruction.
Taking the first target log as the log formed by the instruction sequence A, B, C, D and the second target log as the log formed by the instruction sequence A, B, C, E, F as an example, the total number of instructions of the first target log is 4, the total number of instructions of the second target log is 5, then M is equal to 4, and n is an integer greater than 1 and less than or equal to 4.
Taking N as 2 as an example, the first target instruction is the instruction B, and the first target value group includes a first characteristic value of the instruction B and a second characteristic value of the instruction B obtained based on the first target log. Correspondingly, the second target instruction is also the instruction B, and the second target value group comprises a first characteristic value of the instruction B and a second characteristic value of the instruction B obtained based on the second target log.
Taking N as 4 as an example, the first target instruction is an instruction D, and the first target value group includes a first characteristic value of the instruction D and a second characteristic value of the instruction D obtained based on the first target log. Correspondingly, the second target instruction is also instruction E, and the second target value group comprises a first characteristic value of the instruction E and a second characteristic value of the instruction E obtained based on the second target log.
S404, determining the first target log and the second target log as suspected associated logs when the first target value group and the second target value group are identical.
Obviously, if N is 2, the first target log and the second target log are determined to be suspected associated logs.
S405, determining the first target log and the second target log as irrelevant logs when the first target value group and the second target value group are not identical.
Obviously, if N is 4, the first target log and the second target log are determined as irrelevant logs.
Thus, steps S401-S405 are only one preliminary identification aimed at quickly eliminating irrelevant logs and quickly locking suspected relevant logs, and finally aimed at speeding up aggregate deduplication.
Further, in the case that the suspected association log is determined, the method further includes:
s501, acquiring a first number and a second number, wherein the first number is the total number of instructions in the first target log, and the second number is the total number of instructions in the second target log.
S502, determining the suspected association log as a suspected identical log when the first number is identical to the second number.
S503, determining the suspected associated logs as suspected subordinate logs when the first number and the second number are different, wherein the subordinate logs represent the first target logs as subsets of the second target logs or the second target logs as subsets of the first target logs.
In particular, if the first number is less than the second number, the first target log may be a subset of the second target log, and if the second number is less than the first number, the second target log may be a subset of the first target log.
In a scenario where the suspected identical log or the suspected subordinate log is determined, the information related to each instruction in the first target log and the second target log needs to be determined, please refer to fig. 8, which illustrates a schematic diagram of instruction information in the first target log or the second target log, and for each instruction, the instruction includes two feature values, a first feature value address differences and a second feature value count differences. The specific judgment process is described below.
In the case that the suspected identical log is determined, the method further includes: when the target value group corresponding to each instruction in the first target log is completely the same as the target value group corresponding to the associated instruction in the second target log, determining the suspected identical log as the identical log, wherein the associated instruction is the instruction corresponding to each instruction in the second target log; otherwise, the first target log and the second target log are determined to be irrelevant logs.
In the case where the first number is less than the second number and the suspected slave log is determined, the method further comprises: when the target value group corresponding to each instruction in the first target log is identical to the target value group corresponding to the associated instruction in the second target log, determining the suspected slave log as the slave log, wherein the associated instruction is the instruction corresponding to each instruction in the second target log; otherwise, the first target log and the second target log are determined to be irrelevant logs.
And if the same log or the subordinate log is determined, aggregating and deduplicating the first target log and the second target log to obtain a log aggregation result.
Specifically, if the first target log and the second target log are the same log, the first target log may be deleted directly, and if the first target log and the second target log have a relationship, the logs belonging to a subset may be deleted.
In some embodiments, symbol restoration may be further performed on the first target log or a log aggregation result obtained based on the first target log to obtain symbol restoration information. Specifically, the server may obtain a file with debug information, such as a dsym file published based on the iOS platform. iOS is a mobile operating system developed by apple corporation and dsym files can be used to record debug information for applications developed based on iOS. When the dsym file is obtained, the server may extract the target original address of the log related to the dsym file and the first characteristic value corresponding to each incremental instruction, so as to restore the specific interface name of each instruction in the log in the application program related to the dsym file, where the symbol restore information includes the specific interface name. Of course, in some embodiments, symbol recovery may be performed on both the first target log and the second target log, if the symbol recovery information is the same, the first target log and the second target log are determined to be the same log, and if the symbol recovery information of the first target log belongs to the symbol recovery information of the second target log, the first target log and the second target log are determined to be subordinate logs.
Please refer to fig. 9, which shows a comparison schematic diagram of the logs obtained by the related art and the embodiment of the present application, it is obvious that the left graph is the log obtained by the related art, which includes specific symbol restoration information, the data volume is huge, and the right graph is the log obtained by the embodiment of the present application, which includes only the target original address of the first instruction and the increment information of the increment instruction, and the data volume is very small. Obviously, the method obviously improves the log generation speed and reduces the resources consumed by each link of log generation, transmission and storage. Referring to fig. 10, a diagram illustrating comparison of data amounts of logs obtained in the related art and the embodiments of the present application is shown. From a combination of fig. 9 and 10, it can be seen that the effect of the embodiments of the present application on reducing the data volume of the log is very significant.
Referring to fig. 11, a flowchart of a client and server dual-end interaction scheme in an embodiment of the present application is shown. Specifically, for the first instruction, the client may call the address retriever through the instruction address generator to obtain the target original address of the first instruction, and for the incremental instruction, call the tag generator through the instruction address generator to obtain the first characteristic value and the second characteristic value, and generate a log according to the target original address, the first characteristic value and the second characteristic value, where the log may be transmitted to the server, and the server may perform type identification on the obtained log, and aggregate and deduplicate the same log and the subordinate log to obtain the optimized log record. Symbol restoration can be performed on the log records to obtain symbol restoration information.
The log in the embodiment of the application has the double characteristic values, and compared with the related art, the double characteristic value retrieval speed is higher, and the aggregation duplicate removal efficiency is improved. The data volume of the log is smaller, compared with the disk space used by the related technology, the hardware space used by the computer for storing single English characters (such as character a) can store 256 numbers at most, and the space utilization rate is far higher than that of the traditional storage scheme. The embodiment of the application regulations can be used for various scenes needing log acquisition, log management and log analysis, taking games as examples, and can provide support for the log analysis of the games under the support of remote control debugging.
Referring to fig. 12, a block diagram of a log management device in the present embodiment is shown, and the log management device is applied to a client, where the device includes:
a first address extraction module 101, configured to extract a first instruction address in a call stack, where the first instruction address is an address corresponding to a first instruction that is preset to operate in the call stack;
the analysis module 102 is configured to analyze the first instruction address to obtain a target original address, where the target original address refers to an original instruction address of the first instruction in a symbol mirror image;
A second address extracting module 103, configured to extract a second instruction address from the call stack, where the second instruction address is an address corresponding to another instruction of the preset operation on the call stack;
a feature value determining module 104, configured to determine, for each second instruction address, a first feature value and a second feature value, where the first feature value is a distance difference between an original instruction address of a corresponding instruction in the symbol mirror image and the target original address, and the second feature value is an instruction number difference between the corresponding instruction and the first instruction;
the log generating module 105 is configured to generate a log according to the target original address, the first feature value corresponding to each of the second instruction addresses, and the second feature value corresponding to each of the second instruction addresses.
In one embodiment, the analysis module 102 is configured to perform the following operations:
obtaining executable binary instruction segment information associated with the preset operation in a binary enumeration mode;
and performing address retrieval based on the executable binary instruction segment information to obtain a random base address associated with the first instruction address, and obtaining the target original address based on the random base address and the executable binary instruction segment information.
In one embodiment, the characteristic value determining module 104 is configured to perform the following operations:
address retrieval is carried out based on the executable binary instruction segment information to obtain a random base address associated with the second instruction address;
obtaining an original instruction address of an instruction corresponding to the second instruction address in the symbol mirror image according to the random base address related to the second instruction address and the executable binary instruction segment information;
determining a distance difference value between an original instruction address of an instruction corresponding to the second instruction address in the symbol mirror image and the target original address as the first characteristic value;
and determining the instruction quantity difference value between the instruction corresponding to the second instruction address and the first instruction as the second characteristic value.
In one embodiment, the apparatus further comprises a transmission module, where the transmission module is configured to perform the following operations:
compressing the log based on serialization processing to obtain a log sequence;
and transmitting the log sequence to a server.
Referring to fig. 13, a block diagram of a log management device in the present embodiment is shown, and the log management device is applied to a server, where the device includes:
A log obtaining module 201, configured to obtain a first target log by interacting with a client;
the first target log is obtained by the client through the following method:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address refers to an original instruction address of the first instruction in a symbol mirror image;
extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
and generating the first target log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
In one embodiment, the apparatus further includes an aggregate deduplication module 202, where the aggregate deduplication module 202 is configured to perform the following operations:
determining a second target log, wherein the second target log and the first target log have the same target original address;
acquiring a first target value group associated with a first target instruction in the first target log, wherein the first target instruction is an nth instruction in the first target log, N is an integer greater than 1 and less than or equal to M, M is a minimum value of the total number of instructions contained in the first target log and the total number of instructions contained in the second target log, and the first target value group comprises a first characteristic value corresponding to the first target instruction and a second characteristic value corresponding to the first target instruction;
acquiring a second target value group associated with a second target instruction in the second target log, wherein the second target instruction is an Nth instruction in the second target log, and the second target value group comprises a first characteristic value corresponding to the second target instruction and a second characteristic value corresponding to the second target instruction;
when the first target value group and the second target value group are identical, determining the first target log and the second target log as suspected associated logs;
And determining the first target log and the second target log as irrelevant logs when the first target value group and the second target value group are not identical.
In one embodiment, the aggregate deduplication module 202 is configured to perform the following operations:
acquiring a first number and a second number when the suspected association log is determined, wherein the first number is the total number of instructions in the first target log, and the second number is the total number of instructions in the second target log;
determining the suspected associated log as a suspected identical log when the first number is the same as the second number;
and determining the suspected associated log as a suspected subordinate log, wherein the subordinate log characterizes the first target log as a subset of the second target log or the second target log as a subset of the first target log, when the first number and the second number are different.
In one embodiment, the aggregate deduplication module 202 is configured to perform the following operations:
when the suspected identical log is determined, determining the suspected identical log as an identical log when the target value group corresponding to each instruction in the first target log is identical to the target value group corresponding to the associated instruction in the second target log, wherein the associated instruction is an instruction corresponding to each instruction in the second target log;
Otherwise, the first target log and the second target log are determined to be irrelevant logs.
In one embodiment, the aggregate deduplication module 202 is configured to perform the following operations:
when the first number is smaller than the second number and the suspected slave log is determined, determining the suspected slave log as a slave log when the target value group corresponding to each instruction in the first target log is identical to the target value group corresponding to the associated instruction in the second target log, wherein the associated instruction is an instruction corresponding to each instruction in the second target log;
otherwise, the first target log and the second target log are determined to be irrelevant logs.
In one embodiment, the aggregate deduplication module 202 is configured to perform the following operations:
and under the condition that the same log or the subordinate log is determined, aggregating and deduplicating the first target log and the second target log to obtain a log aggregation result.
In one embodiment, the log obtaining module 201 is configured to perform the following operations:
the first target log comprises application program version information and hardware model information, and a root node is determined according to the application program version information and the hardware model information;
And storing the first target log or a log aggregation result obtained based on the first target log in a storage space corresponding to the root node.
In one embodiment, the log obtaining module 201 is configured to perform the following operations:
and extracting the second target log from a storage space corresponding to a root node, wherein the root node is determined based on the application version information and the hardware model information.
In one embodiment, the log obtaining module 201 is configured to perform the following operations:
acquiring a log sequence from a client, wherein the log sequence is obtained by compressing the first target log based on serialization by the client;
and performing deserialization processing on the log sequence to obtain the first target log.
In one embodiment, the log obtaining module 201 is configured to perform the following operations:
and performing symbol restoration on the first target log or a log aggregation result obtained based on the first target log to obtain symbol restoration information.
The device part and the method embodiment in the embodiment of the present application are based on the same inventive concept, and are not described herein in detail.
Further, fig. 14 shows a schematic diagram of a hardware structure of an apparatus for implementing the method provided by the embodiment of the present application, where the apparatus may participate in forming or including the device or the system provided by the embodiment of the present application. As shown in fig. 14, the apparatus 10 may include one or more processors 102 (shown as 102a, 102b, … …,102 n) which may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, the method may further include: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power supply, and/or a camera. It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 14 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, the device 10 may also include more or fewer components than shown in fig. 14, or have a different configuration than shown in fig. 14.
It should be noted that the one or more processors 102 and/or other data processing circuits described above may be referred to generally herein as "data processing circuits. The data processing circuit may be embodied in whole or in part as an application, hardware, firmware, or any combination thereof. Further, the data processing circuitry may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the device 10 (or mobile device). As referred to in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of the variable resistor termination path to interface with).
The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the methods described in the embodiments of the present application, and the processor 102 executes the software programs and modules stored in the memory 104 to perform various functional applications and data processing, i.e., implement a log management method described above. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory located remotely from processor 102, which may be connected to device 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 106 is arranged to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communications provider of device 10. In one example, the transmission device 106 includes a network adapter (NetworkInterfaceController, NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a radio frequency (RadioFrequency, RF) module for communicating wirelessly with the internet.
The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the device 10 (or mobile device).
It should be noted that: the foregoing sequence of the embodiments of the present application is only for describing, and does not represent the advantages and disadvantages of the embodiments. And the foregoing description has been directed to specific embodiments of this application. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
All embodiments in the embodiments of the present application are described in a progressive manner, and identical and similar parts of all embodiments are mutually referred to, and each embodiment focuses on the differences from other embodiments. In particular, for the device and server embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference is made to the description of portions of the method embodiments where relevant.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program for instructing relevant hardware, where the program may be stored in a computer readable storage medium, and the storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The instructions in the storage medium may implement a log management method, applied to a client, where the method includes:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address refers to an original instruction address of the first instruction in a symbol mirror image;
Extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
and generating a log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
In one embodiment, the analyzing the first instruction address to obtain the target original address includes:
obtaining executable binary instruction segment information associated with the preset operation in a binary enumeration mode;
and performing address retrieval based on the executable binary instruction segment information to obtain a random base address associated with the first instruction address, and obtaining the target original address based on the random base address and the executable binary instruction segment information.
In one embodiment, the determining the first feature value and the second feature value for each second instruction address includes:
address retrieval is carried out based on the executable binary instruction segment information to obtain a random base address associated with the second instruction address;
obtaining an original instruction address of an instruction corresponding to the second instruction address in the symbol mirror image according to the random base address related to the second instruction address and the executable binary instruction segment information;
determining a distance difference value between an original instruction address of an instruction corresponding to the second instruction address in the symbol mirror image and the target original address as the first characteristic value;
and determining the instruction quantity difference value between the instruction corresponding to the second instruction address and the first instruction as the second characteristic value.
In one embodiment, the method further comprises:
compressing the log based on serialization processing to obtain a log sequence;
and transmitting the log sequence to a server.
The instructions in the storage medium may further implement a log management method, applied to a server, where the method includes:
obtaining a first target log through interaction with a client;
The first target log is obtained by the client through the following method:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address refers to an original instruction address of the first instruction in a symbol mirror image;
extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
and generating the first target log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
In one embodiment, the method further comprises:
Determining a second target log, wherein the second target log and the first target log have the same target original address;
acquiring a first target value group associated with a first target instruction in the first target log, wherein the first target instruction is an nth instruction in the first target log, N is an integer greater than 1 and less than or equal to M, M is a minimum value of the total number of instructions contained in the first target log and the total number of instructions contained in the second target log, and the first target value group comprises a first characteristic value corresponding to the first target instruction and a second characteristic value corresponding to the first target instruction;
acquiring a second target value group associated with a second target instruction in the second target log, wherein the second target instruction is an Nth instruction in the second target log, and the second target value group comprises a first characteristic value corresponding to the second target instruction and a second characteristic value corresponding to the second target instruction;
when the first target value group and the second target value group are identical, determining the first target log and the second target log as suspected associated logs;
And determining the first target log and the second target log as irrelevant logs when the first target value group and the second target value group are not identical.
In one embodiment, in a case where the suspected association log is determined, the method further includes:
acquiring a first number and a second number, wherein the first number is the total number of instructions in the first target log, and the second number is the total number of instructions in the second target log;
determining the suspected associated log as a suspected identical log when the first number is the same as the second number;
and determining the suspected associated log as a suspected subordinate log, wherein the subordinate log characterizes the first target log as a subset of the second target log or the second target log as a subset of the first target log, when the first number and the second number are different.
In one embodiment, in a case where the suspected identical log is determined, the method further includes:
when the target value group corresponding to each instruction in the first target log is completely the same as the target value group corresponding to the associated instruction in the second target log, determining the suspected identical log as the identical log, wherein the associated instruction is the instruction corresponding to each instruction in the second target log;
Otherwise, the first target log and the second target log are determined to be irrelevant logs.
In one embodiment, in a case where the first number is smaller than the second number and the suspected slave log is determined, the method further includes:
when the target value group corresponding to each instruction in the first target log is identical to the target value group corresponding to the associated instruction in the second target log, determining the suspected slave log as the slave log, wherein the associated instruction is the instruction corresponding to each instruction in the second target log;
otherwise, the first target log and the second target log are determined to be irrelevant logs.
In one embodiment, the method further comprises:
and under the condition that the same log or the subordinate log is determined, aggregating and deduplicating the first target log and the second target log to obtain a log aggregation result.
In one embodiment, the first target log includes application version information and hardware model information, and the method further includes:
determining a root node according to the application version information and the hardware model information;
And storing the first target log or a log aggregation result obtained based on the first target log in a storage space corresponding to the root node.
In one embodiment, the first target log includes application version information and hardware model information, the determining a second target log, the second target log having a same target original address as the first target log includes:
and extracting the second target log from a storage space corresponding to a root node, wherein the root node is determined based on the application version information and the hardware model information.
In one embodiment, the obtaining the first target log through interaction with the client includes:
acquiring a log sequence from a client, wherein the log sequence is obtained by compressing the first target log based on serialization by the client;
and performing deserialization processing on the log sequence to obtain the first target log.
In one embodiment, the method further comprises:
and performing symbol restoration on the first target log or a log aggregation result obtained based on the first target log to obtain symbol restoration information.
The foregoing description of the preferred embodiments is merely exemplary in nature and is not intended to limit the embodiments of the present application, but is intended to cover any modifications, equivalents, alternatives, and improvements falling within the spirit and principles of the embodiments of the present application.

Claims (14)

1. A method for log management, applied to a client, the method comprising:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address points to an original instruction address of the first instruction in a symbol mirror image;
extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
And generating a log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
2. The method of claim 1, wherein said analyzing said first instruction address to obtain a target original address comprises:
obtaining executable binary instruction segment information associated with the preset operation in a binary enumeration mode;
and performing address retrieval based on the executable binary instruction segment information to obtain a random base address associated with the first instruction address, and obtaining the target original address based on the random base address and the executable binary instruction segment information.
3. The method of claim 2, wherein determining the first and second eigenvalues for each second instruction address comprises:
address retrieval is carried out based on the executable binary instruction segment information, and a random base address associated with the second instruction address is obtained;
obtaining an original instruction address of an instruction corresponding to the second instruction address in the symbol mirror image according to the random base address associated with the second instruction address and the executable binary instruction segment information;
Determining a distance difference value between an original instruction address of an instruction corresponding to the second instruction address in the symbol mirror image and the target original address as the first characteristic value;
and determining an instruction quantity difference value between the instruction corresponding to the second instruction address and the first instruction as the second characteristic value.
4. A method according to any one of claims 1 to 3, further comprising:
compressing the log based on serialization processing to obtain a log sequence;
and transmitting the log sequence to a server.
5. A log management method, applied to a server, the method comprising:
obtaining a first target log through interaction with a client;
the first target log is obtained by the client through the following method:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address points to an original instruction address of the first instruction in a symbol mirror image;
extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
Determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
and generating the first target log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
6. The method of claim 5, wherein the method further comprises:
determining a second target log, wherein the second target log and the first target log have the same target original address;
acquiring a first target value group associated with a first target instruction in the first target log, wherein the first target instruction is an N-th instruction in the first target log, and the first target value group comprises a first characteristic value corresponding to the first target instruction and a second characteristic value corresponding to the first target instruction;
acquiring a second target value group associated with a second target instruction in the second target log, wherein the second target instruction is an Nth instruction in the second target log, and the second target value group comprises a first characteristic value corresponding to the second target instruction and a second characteristic value corresponding to the second target instruction;
Determining the first target log and the second target log as suspected association logs when the first target value group and the second target value group are identical;
determining the first target log and the second target log as uncorrelated logs if the first target set and the second target set are not identical;
wherein N is an integer greater than 1 and less than or equal to M, M being the minimum of the total number of instructions contained in the first target log and the total number of instructions contained in the second target log.
7. The method of claim 6, wherein in the event that the suspected association log is determined, the method further comprises:
obtaining a first number and a second number, wherein the first number is the total number of instructions in the first target log, and the second number is the total number of instructions in the second target log;
determining the suspected associated log as a suspected identical log if the first number and the second number are identical;
and if the first number and the second number are different, determining the suspected associated log as a suspected subordinate log, wherein the subordinate log characterizes that the first target log is a subset of the second target log or the second target log is a subset of the first target log.
8. The method of claim 7, wherein in the event that the suspected identical log is determined, the method further comprises:
when the target value group corresponding to each instruction in the first target log is completely the same as the target value group corresponding to the associated instruction in the second target log, determining the suspected identical log as the identical log, wherein the associated instruction is the instruction corresponding to each instruction in the second target log;
otherwise, the first target log and the second target log are determined to be irrelevant logs.
9. The method according to claim 7 or 8, wherein in case the first number is smaller than the second number and the suspected slave log is determined, the method further comprises:
when the target value group corresponding to each instruction in the first target log is completely the same as the target value group corresponding to the associated instruction in the second target log, determining the suspected slave log as the slave log, wherein the associated instruction is the instruction corresponding to each instruction in the second target log;
Otherwise, the first target log and the second target log are determined to be irrelevant logs.
10. The method according to claim 8 or 9, characterized in that the method further comprises:
and under the condition that the same log or the subordinate log is determined, aggregating and deduplicating the first target log and the second target log to obtain a log aggregation result.
11. The method of claim 5, wherein the first target log includes application version information and hardware model information, the method further comprising:
determining a root node according to the application version information and the hardware model information;
and storing the first target log or a log aggregation result obtained based on the first target log in a storage space corresponding to the root node.
12. The method of claim 6, wherein the first target log includes application version information and hardware model information, and wherein the determining the second target log includes:
and extracting the second target log from a storage space corresponding to a root node, wherein the root node is determined based on the application version information and the hardware model information.
13. A log management device for use with a client, the device comprising:
the first address extraction module is used for extracting a first instruction address in a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
the analysis module is used for analyzing the first instruction address to obtain a target original address, wherein the target original address points to an original instruction address of the first instruction in the symbol mirror image;
the second address extraction module is used for extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
the characteristic value determining module is used for determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
and the log generation module is used for generating a log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
14. A log management device for use with a server, the device comprising:
the log acquisition module is used for obtaining a first target log through interaction with the client;
the first target log is obtained by the client through the following method:
extracting a first instruction address from a call stack, wherein the first instruction address is an address corresponding to a first instruction which is preset to operate in the call stack;
analyzing the first instruction address to obtain a target original address, wherein the target original address points to an original instruction address of the first instruction in a symbol mirror image;
extracting a second instruction address from the call stack, wherein the second instruction address is an address corresponding to other instructions of the preset operation on the call stack;
determining a first characteristic value and a second characteristic value for each second instruction address, wherein the first characteristic value is a distance difference value of an original instruction address of a corresponding instruction in the symbol mirror image relative to the target original address, and the second characteristic value is an instruction quantity difference value of the corresponding instruction relative to the first instruction;
and generating the first target log according to the target original address, the first characteristic value corresponding to each second instruction address and the second characteristic value corresponding to each second instruction address.
CN202211073100.8A 2022-09-02 2022-09-02 Log management method and device, storage medium and electronic equipment Pending CN117687981A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211073100.8A CN117687981A (en) 2022-09-02 2022-09-02 Log management method and device, storage medium and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211073100.8A CN117687981A (en) 2022-09-02 2022-09-02 Log management method and device, storage medium and electronic equipment

Publications (1)

Publication Number Publication Date
CN117687981A true CN117687981A (en) 2024-03-12

Family

ID=90127032

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211073100.8A Pending CN117687981A (en) 2022-09-02 2022-09-02 Log management method and device, storage medium and electronic equipment

Country Status (1)

Country Link
CN (1) CN117687981A (en)

Similar Documents

Publication Publication Date Title
CN109034993B (en) Account checking method, account checking equipment, account checking system and computer readable storage medium
CN107220142B (en) Method and device for executing data recovery operation
CN107506451B (en) Abnormal information monitoring method and device for data interaction
CN110855473A (en) Monitoring method, device, server and storage medium
CN109672722B (en) Data deployment method and device, computer storage medium and electronic equipment
CN112732647B (en) Log searching method, device, equipment and storage medium
CN112732982A (en) Fault code storage method and device, terminal equipment and readable storage medium
US10089167B2 (en) Log file reduction according to problem-space network topology
CN113254767A (en) Big data searching method and device, computer equipment and storage medium
CN111400393A (en) Data processing method and device based on multi-application platform and storage medium
CN111181769B (en) Network topological graph drawing method, system, device and computer readable storage medium
CN109408309A (en) The test method and device of multiple terminals
WO2015139565A1 (en) Heterogeneous logging system management configuration
CN113656369A (en) Log distributed streaming acquisition and calculation method in big data scene
CN113010542A (en) Service data processing method and device, computer equipment and storage medium
CN110505289B (en) File downloading method and device, computer readable medium and wireless communication equipment
CN115002011B (en) Flow bidirectional test method and device, electronic equipment and storage medium
CN112579406A (en) Log call chain generation method and device
CN117687981A (en) Log management method and device, storage medium and electronic equipment
CN114756301A (en) Log processing method, device and system
CN110727532B (en) Data restoration method, electronic equipment and storage medium
CN116069744A (en) Data processing method, device, equipment, storage medium and product
CN113761293A (en) Graph data strong-connectivity component mining method, device, equipment and storage medium
CN112800130A (en) Data uplink method, system, device, equipment and storage medium
CN112181929A (en) Cloud management platform log processing method and device, electronic device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination