CN115480946A - Fault detection model modeling method, protection implementation method and related equipment - Google Patents
Fault detection model modeling method, protection implementation method and related equipment Download PDFInfo
- Publication number
- CN115480946A CN115480946A CN202211242342.5A CN202211242342A CN115480946A CN 115480946 A CN115480946 A CN 115480946A CN 202211242342 A CN202211242342 A CN 202211242342A CN 115480946 A CN115480946 A CN 115480946A
- Authority
- CN
- China
- Prior art keywords
- log data
- fault detection
- detection model
- log
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 127
- 238000000034 method Methods 0.000 title claims abstract description 65
- 239000011159 matrix material Substances 0.000 claims abstract description 76
- 238000013528 artificial neural network Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 24
- 238000003860 storage Methods 0.000 claims abstract description 20
- 238000004590 computer program Methods 0.000 claims description 23
- 230000015654 memory Effects 0.000 claims description 20
- 239000013598 vector Substances 0.000 claims description 20
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 230000002159 abnormal effect Effects 0.000 claims description 8
- 238000002372 labelling Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 18
- 238000005516 engineering process Methods 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000003058 natural language processing Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000000306 recurrent effect Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/079—Root cause analysis, i.e. error or fault diagnosis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0706—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation the processing taking place on a specific hardware platform or in a specific software environment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/0703—Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
- G06F11/0766—Error or fault reporting or storing
- G06F11/0787—Storage of error reports, e.g. persistent data storage, storage using memory protection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The embodiment of the disclosure provides a fault detection model modeling method, an application program protection device, computer equipment, a readable storage medium and a program, and relates to the technical field of computers. The method comprises the following steps: obtaining log data; grouping the log data, wherein each group of log data belongs to the same program; marking the log data by taking a group as a unit; converting the marked log data set into a word embedding matrix set; and training the neural network by taking the word embedded matrix group as input to complete the modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data. The fault detection model established by the scheme provided by the embodiment of the disclosure improves the detection accuracy of fault detection.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for modeling a fault detection model, a method and an apparatus for protecting an application program, a computer device, a readable storage medium, and a program.
Background
At present, the development of new technologies such as cloud computing, block chaining, sixth-generation mobile communication and the like drives new demands of massive servers, and the servers also evolve towards intensive integrated management. Virtualized server resources bring more efficient resource utilization, but also bring great difficulty for fault detection, and the too complex internal structure makes it difficult for developers to debug and test quickly. Therefore, how to rapidly judge the time and the type of the fault occurrence through an intelligent technology is a significant research direction in the future.
Disclosure of Invention
The embodiment of the disclosure provides a fault detection model modeling method, an application program protection device, computer equipment, a readable storage medium and a program, relates to the technical field of computers, and improves the detection accuracy of fault detection by the established fault detection model.
The embodiment of the disclosure provides a modeling method of a fault detection model, which comprises the following steps: acquiring log data; grouping the log data, wherein each group of log data belongs to the same program; marking the log data by taking a group as a unit; converting the marked log data set into a word embedding matrix set; and training the neural network by taking the word embedded matrix group as input to complete the modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data.
In one embodiment, the tagging of log data in units of groups comprises: when the log data is a fault log, marking the log data as the fault log data by taking a group as a unit; and when the log data is a non-fault log, marking the log data as the non-fault log data by taking a group as a unit.
In one embodiment, tagging log data in units of groups comprises: calculating the variance of each group of log data in the log data group with the non-fault or the fault; and determining the weight when the semantic vector is acquired according to the log data group according to the variance.
In one embodiment, converting the annotated log data set to a word embedding matrix set comprises: acquiring the position code of the first log according to the position information of the first log in the marked log data group; constructing a generating matrix according to the category information of the first log; multiplying the position code of the first log with the generating matrix to obtain the category code of the first log; combining the position code of the first log with the category code of the first log to obtain a word embedding matrix of the first log; and acquiring a word embedding matrix group according to the word embedding matrixes of all logs in the labeled log data group.
In one embodiment, training a neural network with words embedded in a matrix set as input to complete modeling of a fault detection model comprises: and training the sequence-to-sequence neural network by taking the word embedding matrix group as input so as to complete the modeling of the fault detection model.
The embodiment of the disclosure provides a protection method for an application program, which includes: acquiring access log data of a user program; converting the access log data into a word embedding matrix; the words of the access log data are embedded into the matrix and input to a trained fault detection model for detection; when the fault detection model detects that the log is normally accessed, allowing the user program to access; when the fault detection model detects that the log is accessed abnormally, the user program is refused to access; wherein the fault detection model is a fault detection model established by the method of any one of the above embodiments.
The embodiment of the present disclosure provides a modeling apparatus for a fault detection model, including: the first acquisition module is used for acquiring log data; the grouping module is used for grouping the log data, wherein each group of log data belongs to the same program; the marking module is used for marking the log data by taking the group as a unit; the first conversion module is used for converting the marked log data set into a word embedding matrix set; and the training module is used for training the neural network by taking the word embedded matrix group as input so as to complete the modeling of the fault detection model, so that the fault detection model has the capacity of identifying faults according to log data.
The embodiment of the present disclosure provides a protection device for an application program, including: the second acquisition module is used for acquiring access log data of the user program; the second conversion module is used for converting the access log data into a word embedding matrix; the input module is used for inputting the words of the access log data into the trained fault detection model for detection; the determining module is used for allowing the user program to access when the fault detection model is detected to be a normal access log; the determining module is also used for refusing the access of the user program when the fault detection model is detected to be an abnormal access log; wherein the fault detection model is a fault detection model established by the method of any one of the above embodiments.
The embodiment of the disclosure provides a computer device, which comprises a processor, a memory and an input/output interface; the processor is connected with the memory and the input/output interface respectively, wherein the input/output interface is used for receiving data and outputting the data, the memory is used for storing a computer program, and the processor is used for calling the computer program so as to enable the computer device to execute the method in any one of the above embodiments.
The disclosed embodiments provide a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any of the above embodiments.
The disclosed embodiments provide a computer program product comprising a computer program that when executed by a processor implements the method of any of the above embodiments.
According to the modeling method of the fault detection model, log data are obtained; grouping the log data, wherein each group of log data belongs to the same program; marking the log data by taking a group as a unit; converting the marked log data set into a word embedding matrix set; the word embedded matrix group is used as input to train the neural network to complete the modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data, the fault detection model for detecting the faults according to the log data can be established, and the detection accuracy of fault detection is improved.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 shows a schematic diagram of an exemplary system architecture to which the modeling method of the fault detection model of the disclosed embodiments may be applied;
FIG. 2 is a flow chart of a method for modeling a fault detection model provided by an embodiment of the present disclosure;
FIG. 3 illustrates a schematic diagram of the formation of a word embedding matrix set for one embodiment;
FIG. 4 illustrates a diagram of a neural network's training or prediction structure, according to one embodiment;
FIG. 5 is a flowchart of a method for protecting an application according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a modeling apparatus of a fault detection model according to an embodiment of the present disclosure;
FIG. 7 is a schematic structural diagram of a protection device for an application according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the embodiments described are only some embodiments of the present disclosure, rather than all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
In the embodiment of the present disclosure, log data may be acquired based on a sequence-to-sequence (seq 2 seq) technique; grouping the log data, wherein each group of log data belongs to the same program; marking the log data by taking a group as a unit; converting the marked log data set into a word embedding matrix set; and training the neural network by taking the word embedded matrix group as input to complete the modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data, and thus, the fault detection model capable of detecting the faults according to the log data is established.
Some terms of the present disclosure are first described below:
sequence-to-sequence seq2seq is an architectural approach represented by encoding (Encode) and decoding (Decode), and seq2seq models generate an output sequence Y from an input sequence X, and are widely used in translation, text auto-summarization, robotic auto-question answering, and some regression prediction tasks. The seq2seq model is represented by encode, which means to convert an input sequence into a vector of fixed length, and decode, which means to decode an input vector of fixed length into an output sequence. The encoding and decoding methods may be RNN (Recurrent Neural Network), CNN (convolutional Neural Network), LSTM (Long Short Term Memory), GRU (Gate Recurrent Unit), and the like.
Word embedding, the collective term for language models and characterization learning techniques in Natural Language Processing (NLP). Conceptually, it refers to embedding a high-dimensional space with dimensions of the number of all words into a continuous vector space with much lower dimensions, each word or phrase being mapped as a vector on the real number domain. The word embedding method comprises an artificial neural network, dimension reduction of a word co-occurrence matrix, a probability model, explicit representation of the context in which the word is positioned and the like. In the bottom layer input, the method of using word embedding to express word groups greatly improves the effects of a grammar analyzer, text emotion analysis and the like in NLP.
Linear block code, an [ n, k ] linear block code, is a code word of [ n, k ] linear block code, which divides information into k symbols as one segment (called information block), and changes the length of the segment into a group of n symbols by an encoder. If the value of each bit code element is q (q is prime power, q system), the k-th power code word of q is shared.
Hamming distance, which is a concept that represents the number of different characters at corresponding positions of two (same length) character strings, represents the hamming distance between two words x, y with d (x, y). And carrying out exclusive OR operation on the two character strings, and counting the number of 1, wherein the number is the Hamming distance.
The scheme provided by the embodiment of the disclosure relates to the technology of seq2seq, NLP, linear grouping and the like.
Fig. 1 shows a schematic diagram of an exemplary system architecture 100 to which the modeling method of the fault detection model of the disclosed embodiments may be applied.
As shown in fig. 1, the system architecture 100 may include one or more of terminals 101, 102, 103, a network 104, and a server 105. The network 104 is a medium to provide communication links between the terminals 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
It should be understood that the number of terminals, networks, and servers in fig. 1 are merely illustrative. There may be any number of terminals, networks, and servers, as desired for an implementation. For example, server 105 may be a server cluster comprised of multiple servers, or the like.
The terminals 101, 102, 103 interact with a server 105 via a network 104, may receive or send messages, etc. The terminals 101, 102, 103 may be various electronic devices having a display screen, including but not limited to smart phones, tablet computers, portable computers, desktop computers, and the like.
The server 105 may be a server that provides various services. For example, the terminal 103 (or the terminal 101 or 102) sends an instruction for modeling the fault detection model to the server 105, and the server 105 can acquire log data; grouping the log data, wherein each group of log data belongs to the same program; marking the log data by taking a group as a unit; converting the marked log data set into a word embedding matrix set; and training the neural network by taking the word embedded matrix group as an input to complete the modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data.
The system architecture in fig. 1 may also apply the protection method of the application program of the present application. After modeling is completed, the fault detection model can be arranged in the server 105, when a user sends a call request to the server 105 through the terminal 103 (or the terminal 101 or 102), the fault detection model at the server 105 end can detect access log data of a user program, and when the fault detection model detects that the access log data is a normal access log, the user is allowed to access the fault detection model; and when the fault detection model detects that the log is abnormally accessed, the user is denied access.
The terminal may be a mobile phone (e.g., the terminal 101) or a tablet computer (e.g., the terminal 102), or may be a desktop computer (e.g., the terminal 101), and the like, which is not limited herein. Among them, the terminal may display an application program, which may be a modeling application program of the fault detection model, or the like. The terminal in fig. 1 is only an example of a part of the devices, and the terminal in the present disclosure is not limited to the devices illustrated in fig. 1.
It is understood that the terminal mentioned in the embodiments of the present disclosure may be a user equipment, and the server in the embodiments of the present disclosure includes, but is not limited to, a server or a cluster of servers. The above-mentioned terminal may be an electronic device, including but not limited to a mobile phone, a tablet computer, an intelligent voice interaction device, an intelligent appliance, a vehicle-mounted terminal, a desktop computer, a notebook computer, a palm computer, a vehicle-mounted device, an Augmented Reality/Virtual Reality (AR/VR) device, a helmet display, an intelligent television, a wearable device, an intelligent speaker, a digital camera, a camera, and other Mobile Internet Devices (MID) with network access capability, or a terminal device in a scene such as a train, a ship, or a flight.
The above-mentioned server may be a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, vehicle-road coordination, a Content Delivery Network (CDN), a big data and artificial intelligence platform, or may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers.
Optionally, the data related to the embodiments of the present disclosure may be stored in a cloud platform, or the data may be stored based on a cloud storage technology and a block chain technology, which is not limited herein.
The log data is text data generated by a printout code embedded in a program. The log data records the variables and execution state of the program during operation. By log data, abnormal requests can be located, program execution logic can be tracked, and finer-grained fault diagnosis can be performed. The log data serving as text data can provide information for feature engineering of machine learning and belongs to natural language processing widely applied to deep learning.
Fig. 2 is a flowchart of a modeling method of a fault detection model according to an embodiment of the present disclosure. The method provided by the embodiment of the present disclosure may be executed by the terminal or the server in the embodiment of fig. 1, or executed by the terminal and the server interactively.
As shown in fig. 2, the method provided by the embodiment of the present disclosure may include the following steps.
In step S210, log data is acquired.
In this step, the terminal or the server acquires log data. Wherein the log data in this step can be log data for training and testing the model,
in step S20, the log data are grouped, wherein each group of log data belongs to the same program.
In this step, the terminal or the server groups log data, where each group of log data belongs to the same program. For example, after a program accesses a server, a plurality of pieces of log data are formed, and the plurality of pieces of log data are grouped, for example, 30 pieces of log data are used as a group, and less than 30 pieces of log data are filled with zeros, so that each group of log data are ensured to belong to the same program.
In step S230, the log data is labeled in units of groups.
In this step, the terminal or the server marks the log data in units of groups. In one embodiment, when the log data is a fault log, the log data is marked as fault log data in units of groups; and when the log data is a non-fault log, marking the log data as the non-fault log data by taking a group as a unit. In one embodiment, a variance of each set of log data in the log data set with a non-fault or a resulting fault is calculated; and determining the weight when the semantic vector is acquired according to the log data group according to the variance. The semantic vector is an intermediate variable of a neural network such as seq2 seq.
In step S240, the annotated log data set is converted into a word embedding matrix set.
In this step, the terminal or the server converts the labeled log data set into a word embedding matrix set. In one embodiment, the position code of the first log is obtained according to the position information of the first log in the marked log data group; constructing a generating matrix according to the category information of the first log; multiplying the position code of the first log with the generating matrix to obtain a category code of the first log; combining the position code of the first log with the category code of the first log to obtain a word embedding matrix of the first log; and acquiring a word embedding matrix group according to the word embedding matrixes of all logs in the labeled log data group. Wherein the first log is any one log in the log data group.
In step S250, the word embedding matrix set is used as an input to train the neural network to complete modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data.
In this step, the terminal or the server trains the neural network with the word embedded matrix group as an input to complete the modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data. In one embodiment, the sequence-to-sequence seq2seq neural network is trained with the word embedding matrix set as input to complete the modeling of the fault detection model.
The modeling method of the fault detection model of fig. 2, by obtaining log data; grouping the log data, wherein each group of log data belongs to the same program; marking the log data by taking a group as a unit; converting the marked log data set into a word embedding matrix set; and training the neural network by taking the word embedded matrix group as input to complete the modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data, and can establish the fault detection model for detecting the faults according to the log data.
The following description is made with reference to specific examples of a method for modeling a fault detection model.
FIG. 3 illustrates a schematic diagram of the formation of a word embedding matrix set for one embodiment.
As shown in fig. 3, the position information of the log data is formed into a position code; constructing a generating matrix according to the category information of the log data, and multiplying the position code of the log data by the generating matrix to obtain the category code of the log data; combining the position code of the log data with the category code of the log data to obtain a word embedding matrix of the log data; a plurality of logs (e.g., 30) form a word embedding matrix set. The log data conversion in fig. 3 may be converting the labeled log data set into a word embedding matrix set.
The conversion method of word embedding matrix group in fig. 3 increases hamming distance between log data vectors by adding category coding on the basis of position coding (text content coding) of log data, wherein the hamming distance is small if the log data belong to the same category, and the hamming distance is large if the log data belong to different categories; and new distinguishing features can be added to the training data, so that the model training is facilitated.
The formation of the word embedding matrix set of fig. 3 shows a method flow of converting log data into a word embedding matrix set, by adding category coding on the basis of position coding (text content) of the log data, hamming distance between the log data is increased.
FIG. 4 illustrates a diagram of a neural network training or prediction structure, according to one embodiment.
As shown in fig. 4, the neural network is, for example, seq2seq; the sentence vector may be a word embedding matrix set into which the annotated log data set is converted, and each sentence vector may include a word embedding matrix set of 30 logs, for example; sentence vector 1-sentence vector 4 as input to the encoder; the weights of the different sentences in obtaining the corresponding semantic vector (e.g., C1) are determined from the variance of the fault detection for the different sentences. The sentence vector 1 output may be, for example, what the sentence vector 1 is a failure.
The output result of Seq2Seq at each time instant will be constrained by the overall input sequence. It can be essentially considered as a conditional language model (equation 1):
p (Y-X) = P (Y _ 1-X) P (Y _ 2-Y _1, X) \ 8230 (equation 1)
Where P (Y | X) represents the probability of Y occurring when an X event occurs, and the other same principles.
The encoder of Seq2Seq adopts a recurrent neural network to calculate the hidden state of the input sequence (usually, the last hidden state is preserved), the hidden state is transmitted to the decoder through a full connection layer, the input of the encoder is the vector obtained by the convolution and pooling of the word vector matrix, and the input is sent to the encoder in a fixed length (one group of 30 log information).
The decoder also adopts a cyclic neural network, receives the hidden state provided by the encoder and the output vector of the convolutional layer as input during training, does not have a target sequence during prediction, receives the output of the decoder at the previous moment and the hidden state provided by the encoder as input, and outputs the probability distribution predicted at the next moment.
The training method of the neural network shown in fig. 4 determines the weight of different sentences when acquiring the corresponding semantic vector (for example, C1) according to the variance of the different sentences for fault detection, so as to improve the accuracy of the trained model for identifying faults.
Fig. 5 is a flowchart of a method for protecting an application according to an embodiment of the present disclosure. The method provided by the embodiment of the disclosure can be executed by the server in the embodiment of fig. 1.
After the modeling is completed, the fault detection model may be set at the server 105 side of the system architecture in fig. 1, and when a user sends a call request to the server 105 through the terminal 103 (or the terminal 101 or 102), corresponding access log data may be generated, and the fault detection model at the server 105 side may detect the access log data of the user program through the method in fig. 5.
As shown in fig. 5, a method provided by an embodiment of the present disclosure may include the following steps:
in step S510, access log data of the user program is acquired;
in step S520, converting the access log data into a word embedding matrix;
in step S530, inputting a word embedding matrix for accessing log data into a trained fault detection model for detection;
in step S540, when the fault detection model detects that the log is normally accessed, allowing the user program to access;
in step S550, when the fault detection model detects an abnormal access log, the user program is denied access.
According to the protection method of the application program, when a call request of the user program is received, access log data of the user program are obtained; converting the access log data into a word embedding matrix; the words of the access log data are embedded into the matrix and input to a trained fault detection model for detection; when the fault detection model detects that the log is normally accessed, allowing the user program to access; when the fault detection model is detected to be an abnormal access log, the access of the user program is refused, and the access of the user program can be detected.
Fig. 6 is a schematic structural diagram of a modeling apparatus for a fault detection model according to an embodiment of the present disclosure.
As shown in fig. 6, a modeling apparatus 600 of a fault detection model provided by an embodiment of the present disclosure may include:
a first obtaining module 610, configured to obtain log data;
a grouping module 620, configured to group log data, where each group of log data belongs to the same program;
a labeling module 630, configured to label the log data in units of groups;
a first conversion module 640, configured to convert the labeled log data set into a word embedding matrix set;
and the training module 650 is configured to train the neural network by using the word embedded matrix group as an input to complete modeling of the fault detection model, so that the fault detection model has the capability of identifying a fault according to log data.
The modeling device of the fault detection model is used for acquiring log data through the first acquisition module; the grouping module is used for grouping the log data, wherein each group of log data belongs to the same program; the marking module is used for marking the log data by taking the group as a unit; the first conversion module is used for converting the marked log data set into a word embedding matrix set; the training module is used for training the neural network by taking the word embedded matrix group as input so as to complete modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data, and the fault detection model capable of detecting the faults according to the log data is established.
Fig. 7 is a schematic structural diagram of a protection device for an application according to an embodiment of the present disclosure.
As shown in fig. 7, a guard 700 for an application provided by an embodiment of the present disclosure may include:
a second obtaining module 710, configured to obtain access log data of the user program;
a second conversion module 720, configured to convert the access log data into a word embedding matrix;
the input module 730 is used for inputting the word embedding matrix of the access log data into the trained fault detection model for detection;
a determining module 740, configured to allow the user program to access when the fault detection model detects that the log is a normal access log;
the determining module 740 is further configured to deny the user program access when the fault detection model detects that the fault detection model is an abnormal access log;
wherein the fault detection model is a fault detection model established as in any one of the above embodiments of the method of modeling a fault detection model.
The protection device of the application program is used for acquiring the access log data of the user program through the second acquisition module; the second conversion module is used for converting the access log data into a word embedding matrix; the input module is used for inputting the words of the access log data into the trained fault detection model for detection; the determining module is used for allowing the user program to access when the fault detection model is detected to be a normal access log; detection of access to a user program can be achieved.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a computer device 800 according to an embodiment of the present disclosure. As shown in fig. 8, the computer device in the embodiment of the present disclosure may include: one or more processors 801, a memory 802, and an input-output interface 803. The processor 801, the memory 802, and the input/output interface 803 are connected by a bus 804. The memory 802 is used for storing a computer program, which includes program instructions, and the input/output interface 803 is used for receiving data and outputting data, for example, for data interaction between a host and a computer device or data interaction between virtual machines in the host; the processor 801 is used to execute program instructions stored by the memory 802.
The processor 801 may perform the following operations:
acquiring log data; grouping the log data, wherein each group of log data belongs to the same program; marking the log data by taking a group as a unit; converting the marked log data set into a word embedding matrix set; and training the neural network by taking the word embedded matrix group as input to complete the modeling of the fault detection model, so that the fault detection model has the capability of identifying faults according to log data.
Alternatively, the processor 801 may perform the following operations:
acquiring access log data of a user program; converting the access log data into a word embedding matrix; the words of the access log data are embedded into the matrix and input to a trained fault detection model for detection; when the fault detection model detects that the log is normally accessed, allowing the user program to access; and when the fault detection model detects that the log is abnormal access, the access of the user program is refused.
In some possible implementations, the processor 801 may be a Central Processing Unit (CPU), and the processor may be other general purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 802 may include both read-only memory and random-access memory and provides instructions and data to the processor 801 and the input output interface 803. A portion of the memory 802 may also include non-volatile random access memory. For example, the memory 802 may also store device type information.
In a specific implementation, the computer device may execute, through each built-in functional module, an implementation manner provided in each step in the foregoing embodiments, which may be referred to specifically for the implementation manner provided in each step in the foregoing embodiments, and details are not described herein again.
The disclosed embodiments provide a computer device, including: the processor, the input/output interface and the memory, the computer program in the memory is obtained by the processor, and the steps of the method shown in the above embodiments are executed to perform the transmission operation.
The embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored, where the computer program is suitable for being loaded by the processor and executing the method provided in each step in the foregoing embodiments, and specific reference may be made to implementation manners provided in each step in the foregoing embodiments, which are not described herein again. In addition, the beneficial effects of the same method are not described in detail. For technical details not disclosed in embodiments of the computer-readable storage medium to which the present disclosure relates, refer to the description of embodiments of the method of the present disclosure. By way of example, a computer program can be deployed to be executed on one computer device or on multiple computer devices at one site or distributed across multiple sites and interconnected by a communication network.
The computer readable storage medium may be the apparatus provided in any of the foregoing embodiments or an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure Digital (SD) card, a flash memory card (flash card), and the like provided on the computer device. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the computer device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the computer device. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.
Embodiments of the present disclosure also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in the various alternatives in the embodiments described above.
The terms "first," "second," and the like in the description and in the claims and the drawings of the embodiments of the present disclosure are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "comprises" and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or apparatus that comprises a list of steps or elements is not limited to the listed steps or modules, but may alternatively include other steps or modules not listed or inherent to such process, method, apparatus, product, or apparatus.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the specification for the purpose of clearly illustrating the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
The method and the related apparatus provided by the embodiments of the present disclosure are described with reference to the flowchart and/or the structural diagram of the method provided by the embodiments of the present disclosure, and specifically, each flow and/or block of the flowchart and/or the structural diagram of the method, and the combination of the flows and/or blocks in the flowchart and/or the block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable transmission device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable transmission device, create means for implementing the functions specified in the flowchart flow or flows and/or block or blocks of the block diagram. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable transmission device to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable transmission device to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The disclosure of the present invention is not intended to be limited to the particular embodiments disclosed, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (11)
1. A method for modeling a fault detection model, comprising:
acquiring log data;
grouping the log data, wherein each group of log data belongs to the same program;
labeling the log data by taking a group as a unit;
converting the marked log data set into a word embedding matrix set;
and training a neural network by taking the word embedded matrix group as input to complete the modeling of a fault detection model, so that the fault detection model has the capability of identifying faults according to the log data.
2. The method of claim 1, wherein tagging the log data in units of groups comprises:
when the log data are fault logs, marking the log data as fault log data by taking a group as a unit;
and when the log data is a non-fault log, marking the log data as non-fault log data by taking a group as a unit.
3. The method of claim 1, wherein labeling the log data in units of groups comprises:
calculating the variance of each group of log data in the log data group with the non-fault or the fault;
and determining the weight when the semantic vector is obtained according to the log data group according to the variance.
4. The method of claim 1, wherein converting the annotated log data set to a word embedding matrix set comprises:
acquiring a position code of a first log according to position information of the first log in a log data set subjected to marking;
constructing a generating matrix according to the category information of the first log;
multiplying the position code of the first log with the generating matrix to obtain a category code of the first log;
combining the position code of the first log with the category code of the first log to obtain a word embedding matrix of the first log;
and acquiring the word embedding matrix group according to the word embedding matrixes of all the logs in the labeled log data group.
5. The method of claim 1, wherein training a neural network using the word embedding matrix set as an input to complete modeling of a fault detection model comprises:
and embedding the words into a matrix group as input to train a sequence-to-sequence neural network so as to complete the modeling of the fault detection model.
6. A method for protecting an application program, comprising:
acquiring access log data of a user program;
converting the access log data into a word embedding matrix;
inputting the word embedding matrix of the access log data into a trained fault detection model for detection;
when the fault detection model detects that the log is normally accessed, allowing the user program to access;
when the fault detection model detects that the fault detection model is an abnormal access log, the user program is refused to access;
wherein the fault detection model is a fault detection model established by the method of any one of claims 1-5.
7. A modeling apparatus for a fault detection model, comprising:
the first acquisition module is used for acquiring log data;
the grouping module is used for grouping the log data, wherein each group of log data belongs to the same program;
the marking module is used for marking the log data by taking a group as a unit;
the first conversion module is used for converting the marked log data set into a word embedding matrix set;
and the training module is used for training the neural network by taking the word embedded matrix group as input so as to complete the modeling of the fault detection model, so that the fault detection model has the capacity of identifying faults according to the log data.
8. An application guard, comprising:
the second acquisition module is used for acquiring access log data of the user program;
the second conversion module is used for converting the access log data into a word embedding matrix;
the input module is used for inputting the words of the access log data into the trained fault detection model for detection;
the determining module is used for allowing the user program to access when the fault detection model is detected to be a normal access log;
the determining module is further configured to deny the user program access when the fault detection model detects that the fault detection model is an abnormal access log;
wherein the fault detection model is a fault detection model established by the method of any one of claims 1-5.
9. A computer device comprising a processor, a memory, an input output interface;
the processor is connected to the memory and the input/output interface respectively, wherein the input/output interface is used for receiving data and outputting data, the memory is used for storing a computer program, and the processor is used for calling the computer program to enable the computer device to execute the method of any one of claims 1-5 or 6.
10. A computer-readable storage medium, characterized in that it stores a computer program adapted to be loaded and executed by a processor to cause a computer device having said processor to perform the method of any of claims 1-5 or 6.
11. A computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the method of any one of claims 1-5 or 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211242342.5A CN115480946A (en) | 2022-10-11 | 2022-10-11 | Fault detection model modeling method, protection implementation method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211242342.5A CN115480946A (en) | 2022-10-11 | 2022-10-11 | Fault detection model modeling method, protection implementation method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115480946A true CN115480946A (en) | 2022-12-16 |
Family
ID=84394674
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211242342.5A Pending CN115480946A (en) | 2022-10-11 | 2022-10-11 | Fault detection model modeling method, protection implementation method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115480946A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111198817A (en) * | 2019-12-30 | 2020-05-26 | 武汉大学 | SaaS software fault diagnosis method and device based on convolutional neural network |
| CN113094200A (en) * | 2021-06-07 | 2021-07-09 | 腾讯科技（深圳）有限公司 | Application program fault prediction method and device |
| EP3910571A1 (en) * | 2020-05-13 | 2021-11-17 | MasterCard International Incorporated | Methods and systems for server failure prediction using server logs |
| CN114201379A (en) * | 2021-12-15 | 2022-03-18 | 中国电信股份有限公司 | Fault type vectorization method, device, equipment and storage medium |
-
2022
- 2022-10-11 CN CN202211242342.5A patent/CN115480946A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111198817A (en) * | 2019-12-30 | 2020-05-26 | 武汉大学 | SaaS software fault diagnosis method and device based on convolutional neural network |
| EP3910571A1 (en) * | 2020-05-13 | 2021-11-17 | MasterCard International Incorporated | Methods and systems for server failure prediction using server logs |
| CN113094200A (en) * | 2021-06-07 | 2021-07-09 | 腾讯科技（深圳）有限公司 | Application program fault prediction method and device |
| CN114201379A (en) * | 2021-12-15 | 2022-03-18 | 中国电信股份有限公司 | Fault type vectorization method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113094200B (en) | Application program fault prediction method and device | |
| CN109743311B (en) | WebShell detection method, device and storage medium | |
| CN110234018B (en) | Multimedia content description generation method, training method, device, equipment and medium | |
| CN111931517B (en) | Text translation method, device, electronic equipment and storage medium | |
| CN108737243A (en) | Conversation message quality detecting method and device | |
| CN112468658B (en) | Voice quality detection method and device, computer equipment and storage medium | |
| CN112257471A (en) | Model training method and device, computer equipment and storage medium | |
| CN114422271B (en) | Data processing method, device, equipment and readable storage medium | |
| CN115237802A (en) | Artificial intelligence based simulation test method and related equipment | |
| CN115544560A (en) | Desensitization method and device for sensitive information, computer equipment and storage medium | |
| CN110674370A (en) | Domain name identification method and device, storage medium and electronic equipment | |
| CN116633804A (en) | Modeling method, protection method and related equipment of network flow detection model | |
| CN111506313B (en) | Program control flow confusion method and system based on neural network | |
| CN115480946A (en) | Fault detection model modeling method, protection implementation method and related equipment | |
| CN112818688B (en) | Text processing method, device, equipment and storage medium | |
| CN115328753A (en) | Fault prediction method and device, electronic equipment and storage medium | |
| CN114338129A (en) | Message anomaly detection method, device, equipment and medium | |
| CN112364649B (en) | Named entity identification method and device, computer equipment and storage medium | |
| CN116599718A (en) | Modeling method, protection method and related equipment of log detection model | |
| CN113688232A (en) | Method and device for classifying bidding texts, storage medium and terminal | |
| CN115146737B (en) | Modeling method of matching model, protection implementation method and related equipment | |
| CN115550014B (en) | Application program protection method and related equipment | |
| CN113254635B (en) | Data processing method, device and storage medium | |
| CN115905598B (en) | Social event abstract generation method, device, terminal equipment and medium | |
| CN117708331A (en) | Attribute word emotion classification method and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |