CN111522746B - Data processing method, device, equipment and computer readable storage medium - Google Patents
Data processing method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111522746B CN111522746B CN202010327853.1A CN202010327853A CN111522746B CN 111522746 B CN111522746 B CN 111522746B CN 202010327853 A CN202010327853 A CN 202010327853A CN 111522746 B CN111522746 B CN 111522746B
- Authority
- CN
- China
- Prior art keywords
- function
- probability
- determining
- context
- call link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application provides a data processing method, a device, equipment and a computer readable storage medium, wherein the method comprises the following steps: acquiring log data; determining a function dynamic call link according to the log data; and determining the context exception of the function dynamic call link according to the function dynamic call link. According to the method, the function dynamic call link of the service program is determined in the running process of the service program according to the log data, and the context abnormity of the function dynamic call link is analyzed, so that the context abnormity of the function dynamic call link is informed to a user, and the abnormity analysis accuracy in the service program is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for data processing.
Background
The log for recording the information of the service program during operation is widely applied to the abnormal detection in the service program, and the log is a plain text consisting of a fixed part and a variable part. As the service iteration and the requirement increase, the function call link of the service program gradually becomes more hierarchical and longer, and the abnormal condition of the service program in the execution process may not be found only by the log. In the prior art, a user often checks logs manually by using keyword search and rule matching, a fixed part and a variable part are extracted from logs of a business program, the fixed part is used as the key point of the logs, the abnormality in the business program is analyzed according to the fixed part, the accuracy of analyzing the abnormality in the business program depends on the quality of the logs of the business program, the fixed part and the variable part are changed due to the modification of the contents of the logs, the quality of the logs is reduced, and the accuracy of analyzing the abnormality in the business program is further reduced.
Disclosure of Invention
The present application provides a method, an apparatus, an electronic device, and a computer-readable storage medium for data processing, which are used to solve the problem of how to improve the accuracy of analyzing an exception in a service program.
In a first aspect, the present application provides a data processing method, including:
acquiring log data;
determining a function dynamic call link according to the log data;
and determining the context exception of the function dynamic call link according to the function dynamic call link.
Optionally, before acquiring the log data, the method further includes:
and recording the function call data into a log by the section-oriented programming, and determining log data.
Optionally, determining a function dynamic call link according to log data includes:
determining a function dynamic call link according to a scene chromosome identifier included in log data through a preset call link generation module, wherein the function dynamic call link comprises a plurality of hierarchies, a first order relation exists among the plurality of hierarchies, a second order relation exists among functions included in a sub-call link corresponding to the same hierarchy in the plurality of hierarchies, and the scene chromosome identifier comprises at least one of transaction hash and block height.
Optionally, determining that the context of the function dynamic call link is abnormal according to the function dynamic call link includes:
predicting a first probability of calling each function for the first time through a preset first long-short term memory recurrent neural network, and determining the context abnormality of a function dynamic calling link according to the first probability;
or according to the second probability of calling each function for the Nth time, predicting the third probability of calling each function for the (N + 1) th time through a preset second long-short term memory recurrent neural network, and according to the third probability, determining the context abnormality of the function dynamic calling link, wherein N is a positive integer.
Optionally, determining a contextual anomaly of the function dynamic call link according to the first probability comprises:
and when the first probability is smaller than a preset first threshold, determining that the function called for the first time corresponding to the first probability is in a context abnormal state.
Optionally, determining the context exception of the function dynamic call link according to the third probability includes:
and when the third probability is smaller than a preset second threshold, determining that the function called for the (N + 1) th time corresponding to the third probability is in a context abnormal state.
Optionally, determining a function dynamic call link according to a scene chromosome identifier included in the log data includes:
and determining a function dynamic call link corresponding to an application scene according to the scene chromosome identifier included in the log data, wherein the application scene comprises at least one of inserting common transactions, inserting intelligent contracts and inserting multiple signature transactions.
In a second aspect, the present application provides an apparatus for data processing, comprising:
the first processing module is used for acquiring log data;
the second processing module is used for determining a function dynamic call link according to the log data;
and the third processing module is used for determining the context exception of the function dynamic call link according to the function dynamic call link.
Optionally, the first processing module is further specifically configured to record function call data into a log through the facet-oriented programming, and determine log data.
Optionally, the second processing module is specifically configured to determine, by a preset call link generation module, a function dynamic call link according to a scenario chromosome identifier included in the log data, where the function dynamic call link includes multiple hierarchies, a first order relationship exists among the multiple hierarchies, a second order relationship exists among functions included in a sub-call link corresponding to a same hierarchy in the multiple hierarchies, and the scenario chromosome identifier includes at least one of a transaction hash and a block height.
Optionally, the third processing module is specifically configured to predict a first probability of calling each function for the first time through a preset first long-short term memory recurrent neural network, and determine that a context of a function dynamic call link is abnormal according to the first probability; or according to the second probability of calling each function for the Nth time, predicting the third probability of calling each function for the (N + 1) th time through a preset second long-short term memory recurrent neural network, and according to the third probability, determining the context abnormality of the function dynamic calling link, wherein N is a positive integer.
Optionally, the third processing module is specifically configured to determine that the function called for the first time corresponding to the first probability is in a context abnormal state when the first probability is smaller than a preset first threshold.
Optionally, the third processing module is specifically configured to determine that the function called at the (N + 1) th time corresponding to the third probability is in a context abnormal state when the third probability is smaller than a preset second threshold.
Optionally, the second processing module is specifically configured to determine, according to a scenario chromosome identifier included in the log data, a function dynamic call link corresponding to an application scenario, where the application scenario includes at least one of inserting a common transaction, inserting an intelligent contract, and inserting a multiple signature transaction.
In a third aspect, the present application provides an electronic device, comprising: a processor, a memory, and a bus;
a bus for connecting the processor and the memory;
a memory for storing operating instructions;
and the processor is used for executing the data processing method of the first aspect of the application by calling the operation instruction.
In a fourth aspect, the present application provides a computer readable storage medium storing a computer program for performing the method of data processing of the first aspect of the present application.
The technical scheme provided by the embodiment of the application at least has the following beneficial effects:
acquiring log data; determining a function dynamic call link according to the log data; and determining the context exception of the function dynamic call link according to the function dynamic call link. Therefore, according to the log data, the function dynamic call link of the service program is determined in the operation process of the service program, and the context abnormity of the function dynamic call link is analyzed, so that the context abnormity of the function dynamic call link is informed to a user, and the abnormity analysis accuracy in the service program is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
Fig. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application;
FIG. 2 is a schematic diagram of data processing provided by an embodiment of the present application;
FIG. 3 is a schematic diagram of data processing provided by an embodiment of the present application;
FIG. 4 is a schematic diagram of data processing provided by an embodiment of the present application;
FIG. 5 is a schematic diagram of data processing provided by an embodiment of the present application;
FIG. 6 is a schematic diagram of data processing provided by an embodiment of the present application;
FIG. 7 is a schematic diagram of data processing provided by an embodiment of the present application;
FIG. 8 is a schematic flow chart diagram illustrating another method for data processing according to an embodiment of the present disclosure;
fig. 9 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more apparent and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.
Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.
The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning or deep learning and the like.
Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formal education learning.
The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism and an encryption algorithm. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.
The block chain underlying platform can comprise processing modules such as user management, basic service, intelligent contract and operation monitoring. The user management module is responsible for identity information management of all blockchain participants, and comprises public and private key generation maintenance (account management), key management, user real identity and blockchain address corresponding relation maintenance (authority management) and the like, and under the authorization condition, the user management module supervises and audits the transaction condition of certain real identities and provides rule configuration (wind control audit) of risk control; the basic service module is deployed on all block chain node equipment and used for verifying the validity of the service request, recording the service request to storage after consensus on the valid request is completed, for a new service request, the basic service firstly performs interface adaptation analysis and authentication processing (interface adaptation), then encrypts service information (consensus management) through a consensus algorithm, transmits the service information to a shared account (network communication) completely and consistently after encryption, and performs recording and storage; the intelligent contract module is responsible for registering and issuing contracts, triggering the contracts and executing the contracts, developers can define contract logics through a certain programming language, issue the contract logics to a block chain (contract registration), call keys or other event triggering and executing according to the logics of contract clauses, complete the contract logics and simultaneously provide the function of upgrading and canceling the contracts; the operation monitoring module is mainly responsible for deployment, configuration modification, contract setting, cloud adaptation in the product release process and visual output of real-time states in product operation, such as: alarm, monitoring network conditions, monitoring node equipment health status, and the like.
The platform product service layer provides basic capability and an implementation framework of typical application, and developers can complete block chain implementation of business logic based on the basic capability and the characteristics of the superposed business. The application service layer provides the application service based on the block chain scheme for the business participants to use.
For better understanding and description of the embodiments of the present application, some technical terms used in the embodiments of the present application will be briefly described below.
LSTM: the LSTM (Long Short-Term Memory, recurrent neural network of Long and Short Term Memory) is a special recurrent neural network, and mainly aims to solve the problems of gradient extinction and gradient explosion in the Long sequence training process.
AOP: AOP (Aspect organized Programming) extracts a section in the business processing process, and dynamically cuts in codes to a specified method and position.
AspectJ: AspectJ is a technology for realizing AOP (Aspect organized Programming), is a framework facing to a tangent plane, and extends Java language; AspectJ defines the AOP syntax, which has a specialized compiler to generate a Class file that complies with the Java byte coding specification.
Instrumentation: instrumentation refers generally to techniques for obtaining data on the state of computer software or hardware; a common implementation is to inject a piece of code into the target program.
ASM: the ASM is a Java byte code manipulation framework; ASM can be used to dynamically generate classes or enhance the functionality of existing classes; the ASM can directly generate a binary class file, and can also dynamically change the class behavior before the class is loaded into the Java virtual machine; an application scenario of the ASM is AOP (Aspect organized Programming).
Block height: the block height is the number of blocks linked to the main chain, i.e. the number of blocks linked to the block chain.
Transaction hashing: the transaction hash is a character segment required for marking transfer, namely personal transfer certificates; the transaction corresponding to each hash is a unique transaction, and the privacy and the safety of the transaction can be ensured.
The technical solution provided by the embodiment of the present application relates to machine learning of artificial intelligence, and the following detailed description is provided for the technical solution of the present application and how to solve the above technical problems with the technical solution of the present application. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
An embodiment of the present application provides a data processing method, a flowchart of the method is shown in fig. 1, and the method includes:
s101, obtaining log data.
Optionally, before acquiring the log data, the method further includes:
and recording the function call data into a log by the AOP facing the section programming, and determining log data.
Optionally, as shown in fig. 2, by performing enhanced injection on the analyzed program service through AOP, function call data collection may be performed at compile time or at runtime, where the function call data includes a function F1 call, a function F2 call, a function F3 call, and the like, isolation between the injection code corresponding to the function call data and the source code of the analyzed program service is achieved, the source code is not contaminated, and the data collection injector may record the function call data in a log.
Alternatively, code injection in the compile-time may be implemented by code enhancement techniques, such as AspectJ; runtime code injection, such as Instrumentation in conjunction with the ASM scheme, may be implemented.
And S102, determining a function dynamic call link according to the log data.
Optionally, determining a function dynamic call link according to log data includes:
determining a function dynamic call link according to a scene chromosome identifier included in log data through a preset call link generation module, wherein the function dynamic call link comprises a plurality of hierarchies, a first order relation exists among the plurality of hierarchies, a second order relation exists among functions included in a sub-call link corresponding to the same hierarchy in the plurality of hierarchies, and the scene chromosome identifier comprises at least one of transaction hash and block height.
Optionally, determining a function dynamic call link according to a scene chromosome identifier included in the log data includes:
and determining a function dynamic call link corresponding to an application scene according to the scene chromosome identifier included in the log data, wherein the application scene comprises at least one of inserting common transactions, inserting intelligent contracts and inserting multiple signature transactions.
Optionally, as shown in fig. 2, the log collection agent uploads the log data to a call link analysis console, and the call link in the call link analysis console generates a part of function dynamic call links corresponding to the application scenario extracted according to the scenario chromosome identifier included in the log data, and generates all function dynamic call links corresponding to the application scenario.
Optionally, as shown in fig. 3, the function dynamic call link includes a plurality of levels, the plurality of levels including a level 0, a level 1.1, a level 2.1, a level 2.2, a level 3.1, a level 4.1, and a level 4.2; the 0 layer represents an entry function of an application scene, the 0 layer has a function F1, the 1.1 layer has a function F2 and a function F16, the 2.1 layer has a function F3, a function F12, a function F13, a function F14, and a function F15, the 2.2 layer has a function F17, the 3.1 layer has a function F4 and a function F11, the 4.1 layer has a function F5, and the 4.2 layer has a function F6, a function F7, a function F8, a function F9, and a function F10. The arrowed lines between the levels represent the sub-level calls, i.e., the first order relationship; the arrow lines inside each level indicate the precedence of function calls inside the function, i.e., the second order relationship. For example, the arrowed line between function F1 in layer 0 and function F2 in layer 1.1 represents a sub-level call between levels, layer 1.1 is a sub-call link inside function F1, function F1 calls function F2; the arrow line between the function F2 and the function F16 in the 1.1 layer indicates the calling sequence of the function F2 and the function F16, and the function F2 is called first, and then the function F16 is called.
S103, determining the context abnormality of the function dynamic call link according to the function dynamic call link.
Optionally, determining that the context of the function dynamic call link is abnormal according to the function dynamic call link includes:
predicting a first probability of calling each function for the first time through a preset first long-short term memory recurrent neural network, and determining the context abnormality of a function dynamic calling link according to the first probability;
or according to the second probability of calling each function for the Nth time, predicting the third probability of calling each function for the (N + 1) th time through a preset second long-short term memory recurrent neural network, and according to the third probability, determining the context abnormality of the function dynamic calling link, wherein N is a positive integer.
Optionally, determining a contextual anomaly of the function dynamic call link according to the first probability comprises:
and when the first probability is smaller than a preset first threshold, determining that the function called for the first time corresponding to the first probability is in a context abnormal state.
Optionally, determining the context exception of the function dynamic call link according to the third probability includes:
and when the third probability is smaller than a preset second threshold, determining that the function called for the (N + 1) th time corresponding to the third probability is in a context abnormal state.
Alternatively, as shown in fig. 4, there may be different situations for the first-called function of the sub-level call, for example, the first-called function of the sub-level call of the function F2 may be F3, the first-called function of the sub-level call of the function F2 may be F12, and the first-called function of the sub-level call of the function F2 may be F20. The first-to-call functions of sub-level calls for different cases are evaluated by an LSTM model, called HC-LSTM model, the recurrent neural network of the first long-short term memory. Through function call link learning of different application scenarios in the call link analysis console as shown in fig. 2, the HC-LSTM for the function F2 learns the probability model as shown in fig. 4, that is, the first probability of the first call function F3 of the sub-level call of the function F2 is 0.8, the first probability of the first call function F12 of the sub-level call of the function F2 is 0.15, and the first probability of the first call function F20 of the sub-level call of the function F2 is 0.05.
Optionally, as shown in fig. 5, for the precedence order of function calls inside the function, the probability of the next occurrence of the function call based on the probabilities of the first N functions occurring is evaluated by an LSTM module, which is called FC-LSTM model, i.e. the recurrent neural network of the second long-short term memory, where N is a positive integer. For example, according to the second probability of each of the 3 functions that the FC-LSTM appears for the first time, i.e., the second probability of the function F13, the second probability of the function F14, and the second probability of the function F15, where N is 3, the probability of the function call appearing next time, i.e., the third probability of the function F16 appearing next time is assumed to be 0.7, the third probability of the function F18 appearing next time is assumed to be 0.22, and the third probability of the function F21 appearing next time is assumed to be 0.08.
Alternatively, two LSTM models, HC-LSTM and FC-LSTM, respectively, are formed by invoking link learning. When the first probability is smaller than a preset first threshold, determining that a first-called function corresponding to the first probability is in a context abnormal state, wherein the first threshold is 0.1; and when the third probability is smaller than a preset second threshold, determining that the called function corresponding to the third probability is in a context abnormal state, wherein the second threshold is 0.1. As shown in fig. 6, the first probability that the function F2 calls F20 is 0, and since the first probability 0 is less than the first threshold 0.1, the first call of the function F2 is in a context abnormal state, that is, the function F20 called the function F2 for the first time is in a context abnormal state; the first probability that the function F2 calls F27 is 0, and since the first probability 0 is less than the first threshold 0.1, the first call of the function F2 is in a context abnormal state, that is, the function F27 called the function F2 for the first time is in a context abnormal state. As shown in fig. 7, according to the second probabilities of the respective 3 functions that occurred before FC-LSTM, that is, the second probability of the function F13, the second probability of the function F14, and the second probability of the function F15, the probability of the next occurrence of the function call is estimated, that is, the third probability of the next occurrence of the function F16 is estimated to be 0.7, the third probability of the occurrence of the function F18 is estimated to be 0.22, and the third probability of the occurrence of the function F21 is estimated to be 0.08; since the third probability 0.08 that the called function F21 occurs is less than the second threshold 0.1, the called function F21 is in a context abnormal state.
Alternatively, as shown in fig. 7, the function F13, the function F14, and the function F15 are one context, and there is a context correlation relationship between the function F13, the function F14, and the function F15.
In the embodiment of the application, log data are obtained; determining a function dynamic call link according to the log data; and determining the context exception of the function dynamic call link according to the function dynamic call link. Therefore, according to the log data, the function dynamic call link of the service program is determined in the operation process of the service program, and the context abnormity of the function dynamic call link is analyzed, so that the context abnormity of the function dynamic call link is informed to a user, and the abnormity analysis accuracy in the service program is improved.
Optionally, the function dynamic call link is determined by a message reporting mode.
Optionally, the number of pre-occurrence functions of the LSTM model is not limited, and may be adjusted according to actual conditions.
Another data processing method is provided in the embodiment of the present application, a flowchart of the method is shown in fig. 8, and the method includes:
s201, recording the function call data into a log through the AOP facing the section programming, and determining log data.
Optionally, the function call data is logged by at least one of AspectJ, Instrumentation in combination with ASM.
S202, the log collection agent collects log data and uploads the log data to a call link analysis middle desk.
Optionally, invoking the link analysis middling stage comprises invoking link generation and invoking link learning, and the invoking link analysis middling stage is used for data collection, data processing, data algorithms and analysis.
And S203, determining a function dynamic call link through the call link generation in the link analysis intermediate station.
Optionally, a call link in the call link analysis console is generated, and according to a scene chromosome identifier included in the log data, the call link generates and extracts a part of function dynamic call links corresponding to the application scene, and generates all function dynamic call links corresponding to the application scene.
S204, predicting the first probability through the HC-LSTM model.
Optionally, the first calling function of sub-level calling of the function dynamic calling link is evaluated through the HC-LSTM model, and the first probability of calling each function for the first time is predicted.
And S205, predicting the third probability through the FC-LSTM model.
Optionally, evaluating, by the FC-LSTM model, a second probability of occurrence of each of M functions occurring at the nth time of the function dynamic call link, and predicting a third probability of each of H functions after the N +1 th time of calling the M functions, where N, M, and H are positive integers.
S206, determining the context exception of the function dynamic call link according to the first probability.
Optionally, when the first probability is smaller than a preset first threshold, it is determined that the function called for the first time corresponding to the first probability is in a context abnormal state.
And S207, determining the context abnormity of the function dynamic call link according to the third probability.
Optionally, when the third probability is smaller than a preset second threshold, it is determined that the function called for the (N + 1) th time corresponding to the third probability is in a context abnormal state.
In the embodiment of the application, the business program is injected through the AOP, the function dynamic call link is recorded in the running process, and the context abnormal scene of the function dynamic call link is analyzed by using the HC-LSTM model and the FC-LSTM model, so that developers or testers are informed to deeply analyze problems, and whether potential problems exist is checked. By AOP injection, the service is ensured not to be influenced without changing the source code, and the influenced performance is ensured to be within an acceptable range; acquiring a function dynamic call link generated by function execution data according to the AOP; and analyzing a function dynamic call link through the HC-LSTM model and the FC-LSTM model, and analyzing a scene possibly with context exception.
In order to better understand the method provided by the embodiment of the present application, the following further describes the scheme of the embodiment of the present application with reference to an example of a specific application scenario.
The method provided by the embodiment of the application is applied to the context exception analysis of the dynamic function call link of the program service. As shown in fig. 2, by performing enhanced injection on the program service to be analyzed through the AOP, function call data collection can be performed during a compile time or a runtime, where the function call data includes a function F1 call, a function F2 call, a function F3 call, and the like, so as to implement isolation between an injection code corresponding to the function call data and a source code of the program service to be analyzed, and not to pollute the source code, the data collection injector records the function call data into a log, the log collection agent synchronizes the log data to a call link analysis console, and the call link analysis console performs context anomaly analysis on a function dynamic call link of the program service.
In the embodiment of the application, the function dynamic call links of different actually-operated scenes are collected and combined, so that the context abnormal scene of the function dynamic call link is analyzed, and the developer or the tester is informed of the context abnormal scene, so that the developer or the tester can conveniently deeply analyze the problem and investigate whether the potential problem exists.
Based on the same inventive concept, the embodiment of the present application further provides a data processing apparatus, a schematic structural diagram of the apparatus is shown in fig. 9, and the data processing apparatus 30 includes a first processing module 301, a second processing module 302, and a third processing module 303.
A first processing module 301, configured to obtain log data;
the second processing module 302 is configured to determine a function dynamic call link according to the log data;
the third processing module 303 is configured to determine that the context of the function dynamic call link is abnormal according to the function dynamic call link.
Optionally, the first processing module 301 is further specifically configured to record function call data into a log through the facet-oriented programming, and determine log data.
Optionally, the second processing module 302 is specifically configured to determine, by a preset call link generation module, a function dynamic call link according to a scenario chromosome identifier included in the log data, where the function dynamic call link includes multiple hierarchies, a first order relationship exists among the multiple hierarchies, a second order relationship exists among functions included in a sub-call link corresponding to the same hierarchy in the multiple hierarchies, and the scenario chromosome identifier includes at least one of a transaction hash and a block height.
Optionally, the third processing module 303 is specifically configured to predict a first probability of calling each function for the first time through a preset first long-short term memory recurrent neural network, and determine that a context of a function dynamic call link is abnormal according to the first probability; or according to the second probability of calling each function for the Nth time, predicting the third probability of calling each function for the (N + 1) th time through a preset second long-short term memory recurrent neural network, and according to the third probability, determining the context abnormality of the function dynamic calling link, wherein N is a positive integer.
Optionally, the third processing module 303 is specifically configured to determine that the function called for the first time corresponding to the first probability is in a context abnormal state when the first probability is smaller than a preset first threshold.
Optionally, the third processing module 303 is specifically configured to determine that the function called at the (N + 1) th time corresponding to the third probability is in a context abnormal state when the third probability is smaller than a preset second threshold.
Optionally, the second processing module 302 is specifically configured to determine, according to the scenario chromosome identifier included in the log data, a function dynamic call link corresponding to an application scenario, where the application scenario includes at least one of inserting a common transaction, inserting an intelligent contract, and inserting a multiple signature transaction.
For the content that is not described in detail in the data processing apparatus provided in the embodiment of the present application, reference may be made to the data processing method provided in the foregoing embodiment, and the beneficial effects that the data processing apparatus provided in the embodiment of the present application can achieve are the same as the data processing method provided in the foregoing embodiment, and are not described herein again.
The application of the embodiment of the application has at least the following beneficial effects:
acquiring log data; determining a function dynamic call link according to the log data; and determining the context exception of the function dynamic call link according to the function dynamic call link. Therefore, according to the log data, the function dynamic call link of the service program is determined in the operation process of the service program, and the context abnormity of the function dynamic call link is analyzed, so that the context abnormity of the function dynamic call link is informed to a user, and the abnormity analysis accuracy in the service program is improved.
Based on the same inventive concept, an embodiment of the present application further provides an electronic device, a schematic structural diagram of which is shown in fig. 10, where the electronic device 6000 includes at least one processor 6001, a memory 6002, and a bus 6003, and each of the at least one processor 6001 is electrically connected to the memory 6002; the memory 6002 is configured to store at least one computer-executable instruction that the processor 6001 is configured to execute in order to perform the steps of any of the methods of data processing as provided in any one of the embodiments or any one of the alternative embodiments of the present application.
Further, the processor 6001 may be an FPGA (Field-Programmable Gate Array) or other device with logic processing capability, such as an MCU (micro controller Unit) or a CPU (Central processing Unit).
The application of the embodiment of the application has at least the following beneficial effects:
acquiring log data; determining a function dynamic call link according to the log data; and determining the context exception of the function dynamic call link according to the function dynamic call link. Therefore, according to the log data, the function dynamic call link of the service program is determined in the operation process of the service program, and the context abnormity of the function dynamic call link is analyzed, so that the context abnormity of the function dynamic call link is informed to a user, and the abnormity analysis accuracy in the service program is improved.
Based on the same inventive concept, the present application provides another computer-readable storage medium, which stores a computer program, and the computer program is used for implementing, when being executed by a processor, the steps of any one of the data processing methods provided in any one of the embodiments or any one of the alternative embodiments of the present application.
The computer-readable storage medium provided by the embodiments of the present application includes, but is not limited to, any type of disk including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks, ROMs (Read-Only memories), RAMs (Random Access memories), EPROMs (Erasable Programmable Read-Only memories), EEPROMs (Electrically Erasable Programmable Read-Only memories), flash memories, magnetic cards, or optical cards. That is, a readable storage medium includes any medium that stores or transmits information in a form readable by a device (e.g., a computer).
The application of the embodiment of the application has at least the following beneficial effects:
acquiring log data; determining a function dynamic call link according to the log data; and determining the context exception of the function dynamic call link according to the function dynamic call link. Therefore, according to the log data, the function dynamic call link of the service program is determined in the operation process of the service program, and the context abnormity of the function dynamic call link is analyzed, so that the context abnormity of the function dynamic call link is informed to a user, and the abnormity analysis accuracy in the service program is improved.
It will be understood by those within the art that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions. Those skilled in the art will appreciate that the computer program instructions may be implemented by a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, implement the aspects specified in the block or blocks of the block diagrams and/or flowchart illustrations disclosed herein.
Those of skill in the art will appreciate that the various operations, methods, steps in the processes, acts, or solutions discussed in this application can be interchanged, modified, combined, or eliminated. Further, other steps, measures, or schemes in various operations, methods, or flows that have been discussed in this application can be alternated, altered, rearranged, broken down, combined, or deleted. Further, steps, measures, schemes in the prior art having various operations, methods, procedures disclosed in the present application may also be alternated, modified, rearranged, decomposed, combined, or deleted.
The foregoing is only a partial embodiment of the present application, and it should be noted that, for those skilled in the art, several modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations should also be regarded as the protection scope of the present application.
Claims (8)
1. A method of data processing, comprising:
acquiring log data;
determining function dynamic call links of different application scenarios according to scenario chromosome identifiers included in the log data, wherein the scenario chromosome identifiers include at least one of transaction hash and block height, and the application scenarios include at least one of common transaction insertion, intelligent contract insertion and multiple signature transaction insertion;
determining the context abnormality of the function dynamic call link according to the function dynamic call link;
before the acquiring the log data, further comprising:
and recording the function call data collected in real time into a log by the section-oriented programming, and determining log data.
2. The method according to claim 1, wherein the determining the function dynamic call links of different application scenarios according to the scenario chromosome identifiers included in the log data comprises:
determining, by a preset call link generation module, function dynamic call links of different application scenarios according to scenario chromosome identifiers included in the log data, where the function dynamic call links include multiple hierarchies, a first order relationship exists among the multiple hierarchies, and a second order relationship exists among functions included in sub-call links corresponding to the same hierarchy in the multiple hierarchies.
3. The method of claim 2, wherein determining the context exception of the dynamic call link from the dynamic call link comprises:
predicting a first probability of calling each function for the first time through a preset first long-short term memory recurrent neural network, and determining that the context of the function dynamic calling link is abnormal according to the first probability;
or according to the second probability of calling each function for the Nth time, predicting the third probability of calling each function for the (N + 1) th time through a preset second long-short term memory recurrent neural network, and according to the third probability, determining that the context of the function dynamic calling link is abnormal, wherein N is a positive integer.
4. The method of claim 3, wherein determining the context anomaly of the function dynamic call link according to the first probability comprises:
and when the first probability is smaller than a preset first threshold, determining that the function called for the first time corresponding to the first probability is in a context abnormal state.
5. The method of claim 3, wherein determining the context anomaly of the function dynamic call link according to the third probability comprises:
and when the third probability is smaller than a preset second threshold, determining that the function called for the (N + 1) th time corresponding to the third probability is in a context abnormal state.
6. An apparatus for data processing, comprising:
the first processing module is used for acquiring log data;
the second processing module is used for determining function dynamic calling links of different application scenes according to scene chromosome identifiers included in the log data, wherein the scene chromosome identifiers include at least one of transaction hash and block height, and the application scenes include at least one of common transaction insertion, intelligent contract insertion and multiple signature transaction insertion;
the third processing module is used for determining the context exception of the function dynamic call link according to the function dynamic call link;
the first processing module is further used for recording the function call data collected in real time into a log through the section-oriented programming, and determining log data.
7. An electronic device, comprising: a processor, a memory;
the memory for storing a computer program;
the processor for executing the method of data processing according to any of claims 1-5 by invoking the computer program.
8. A computer-readable medium, in which a computer program is stored which, when being executed by a processor, is adapted to carry out the method of data processing according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010327853.1A CN111522746B (en) | 2020-04-23 | 2020-04-23 | Data processing method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010327853.1A CN111522746B (en) | 2020-04-23 | 2020-04-23 | Data processing method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111522746A CN111522746A (en) | 2020-08-11 |
| CN111522746B true CN111522746B (en) | 2021-11-02 |
Family
ID=71903481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010327853.1A Active CN111522746B (en) | 2020-04-23 | 2020-04-23 | Data processing method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111522746B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112328335B (en) * | 2020-11-30 | 2023-03-21 | 深圳前海微众银行股份有限公司 | Method, device, equipment and storage medium for diagnosing timeout of concurrent requests |
| CN115757319A (en) * | 2021-06-21 | 2023-03-07 | 蚂蚁胜信(上海)信息技术有限公司 | Link generation method and device |
| CN114706733B (en) * | 2022-05-30 | 2022-09-20 | 支付宝(杭州)信息技术有限公司 | Section program abnormity monitoring method and device |
| CN116112379B (en) * | 2022-12-09 | 2024-02-02 | 国网湖北省电力有限公司信息通信公司 | Dynamic prediction method for directed link of multidimensional service sharing equipment of data center |
| CN116955090B (en) * | 2023-09-19 | 2023-11-28 | 四川集鲜数智供应链科技有限公司 | Calling chain monitoring system and method for chain service |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170295565A1 (en) * | 2013-01-17 | 2017-10-12 | Sun Patent Trust | Dynamic tdd uplink/downlink configuration using dci |
| CN110007957A (en) * | 2018-12-17 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Call link construction method, device and equipment |
| CN110865927A (en) * | 2019-11-20 | 2020-03-06 | 腾讯科技（深圳）有限公司 | Block chain call link abnormity detection method and device and computer equipment |
-
2020
- 2020-04-23 CN CN202010327853.1A patent/CN111522746B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170295565A1 (en) * | 2013-01-17 | 2017-10-12 | Sun Patent Trust | Dynamic tdd uplink/downlink configuration using dci |
| CN110007957A (en) * | 2018-12-17 | 2019-07-12 | 阿里巴巴集团控股有限公司 | Call link construction method, device and equipment |
| CN110865927A (en) * | 2019-11-20 | 2020-03-06 | 腾讯科技（深圳）有限公司 | Block chain call link abnormity detection method and device and computer equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111522746A (en) | 2020-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111522746B (en) | Data processing method, device, equipment and computer readable storage medium | |
| Bryans et al. | Opacity generalised to transition systems | |
| Nayrolles et al. | Clever: Combining code metrics with clone detection for just-in-time fault prevention and resolution in large industrial projects | |
| CN110263538B (en) | Malicious code detection method based on system behavior sequence | |
| CN107392025A (en) | Malice Android application program detection method based on deep learning | |
| Kotenko et al. | The CAPEC based generator of attack scenarios for network security evaluation | |
| Chandy et al. | Cyberattack detection using deep generative models with variational inference | |
| CN113723623A (en) | Federal learning auditing device, system and method | |
| Qin et al. | Association analysis-based cybersecurity risk assessment for industrial control systems | |
| CN114422224A (en) | Attack tracing-oriented threat information intelligent analysis method and system | |
| CN111092775A (en) | Network protocol security test evaluation method based on model learning | |
| CN110011990A (en) | Intranet security threatens intelligent analysis method | |
| Zhou et al. | Colefunda: Explainable silent vulnerability fix identification | |
| Luo et al. | Code confusion in white box crowdsourced software testing | |
| KR100990688B1 (en) | Method and Apparatus for Analyzing Security of Software | |
| CN117034263A (en) | Application method of intelligent contract in unmanned aerial vehicle cluster | |
| Jharko et al. | Extending functionality of early fault diagnostic system for online security assessment of nuclear power plant | |
| CN116661739A (en) | Processing method, device, equipment and storage medium of business rule | |
| Bernardi et al. | Improving Design Patterns Finder Precision Using a Model Checking Approach. | |
| CN107808097A (en) | JavaScript path enumerations | |
| CN113469377A (en) | Federal learning auditing method and device | |
| CN116070191A (en) | Information processing method and device, storage medium, and program product | |
| US10121008B1 (en) | Method and process for automatic discovery of zero-day vulnerabilities and expoits without source code access | |
| Khan | Representing security specifications in UML state machine diagrams | |
| Zulkernine et al. | Towards automatic monitoring of component-based software systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |