CN114863178A - Image data input detection method and system for neural network vision system - Google Patents
Image data input detection method and system for neural network vision system Download PDFInfo
- Publication number
- CN114863178A CN114863178A CN202210522508.2A CN202210522508A CN114863178A CN 114863178 A CN114863178 A CN 114863178A CN 202210522508 A CN202210522508 A CN 202210522508A CN 114863178 A CN114863178 A CN 114863178A
- Authority
- CN
- China
- Prior art keywords
- image data
- neural network
- model
- layer
- probability
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 66
- 238000001514 detection method Methods 0.000 title claims abstract description 28
- 238000003062 neural network model Methods 0.000 claims abstract description 51
- 238000012549 training Methods 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 44
- 230000007704 transition Effects 0.000 claims abstract description 31
- 239000000203 mixture Substances 0.000 claims abstract description 27
- 238000004364 calculation method Methods 0.000 claims abstract description 10
- 239000011229 interlayer Substances 0.000 claims abstract description 8
- 239000010410 layer Substances 0.000 claims description 125
- 210000002569 neuron Anatomy 0.000 claims description 17
- 230000008569 process Effects 0.000 claims description 16
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 230000002159 abnormal effect Effects 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 5
- 239000000463 material Substances 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 2
- 238000010801 machine learning Methods 0.000 claims description 2
- 230000009467 reduction Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
- G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Probability & Statistics with Applications (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image data input detection method and system facing a neural network vision system.A neural network model and a training image data set thereof are given, the training image data set is transmitted to the neural network model, and an intermediate result is collected to obtain implicit characteristics of the neural network; fitting the intermediate result by using a Gaussian mixture model to obtain model parameters, and collecting training image data set path frequency calculation probability; inputting image data to be detected into a neural network model, and collecting an intermediate result according to the method in the step one; and calculating the generation probability and the interlayer transition probability of the intermediate result by using the Gaussian mixture model in the step two, performing rapid probability estimation by using a joint probability estimation model, and verifying whether the input image data to be detected is effective or not.
Description
Technical Field
The invention relates to an image data input detection method and system for a neural network vision system, and belongs to the technical fields of deep neural network model input verification, intelligent software quality assurance, image data processing and the like.
Background
In recent years, software systems based on deep neural networks are widely applied to various fields of production and life, and great convenience is brought to life of people. An important field of application of deep learning models is machine vision, which in turn includes image classification, target detection, semantic segmentation, and other subtasks. Although software systems based on deep neural network technology perform well in these tasks, neural network models often do not achieve 100% accuracy. Moreover, the prediction process of the deep neural network model is difficult to interpret, so whether the prediction result of the visual model is correct or not cannot be judged by a simple method.
The neural network model-based vision system is already applied to the fields with high safety requirements such as face recognition and video monitoring, and the wrong prediction result may bring serious consequences. In order to ensure the safety of a machine vision system based on a deep neural network, some image data-oriented abnormal input detection technologies gradually begin to emerge. The method can estimate the probability of causing the neural network vision system to predict the abnormity aiming at a given image data sample, and rejects the sample and gives an alarm if the estimated probability of predicting the abnormity is larger than a preset threshold value, thereby ensuring the reliable operation of the machine vision intelligent system.
The currently existing image input detection technology still has some limitations in applicability and reliability. Some methods model the input image detection problem as a two-class problem of normal data and abnormal data based on a discriminant model. The construction process of the methods needs the participation of some abnormal input images, and the acquisition and labeling of picture samples increase the application cost. Furthermore, due to the involvement of anomalous data, the discriminant approach may produce an overfitting to a particular data distribution, making it difficult to guarantee the same generalization performance for image data from different distributions. In addition, some methods have the characteristic of model independence, and the methods use a specific strategy to retrain the deep neural network model, so that the cost and the difficulty of the application detection technology are increased, and the retrained model often causes the performance reduction, which is unacceptable in many application scenes.
Disclosure of Invention
The purpose of the invention is as follows: aiming at the problems and the defects in the prior art, the invention provides the image data input detection method and the image data input detection system based on the interlayer joint modeling for the neural network vision system.
The invention has the advantages that: firstly, the use scene is not limited by the complexity of the neural network model and the strict limit of the use scene, and is especially suitable for some scenes with abnormal image data which are difficult to collect. The method has higher accuracy for the validity detection of the input image data, and can effectively distinguish valid and invalid input images, thereby avoiding meaningless prediction results. The method is low in time cost for detecting the input image, can meet the requirement of verifying correctness in real time, and can be deployed in a machine vision model in operation for input detection.
The technical scheme is as follows: an image data input detection method for a neural network vision system comprises the following steps:
the method comprises the following steps: and (4) extracting implicit characteristics of the neural network. And giving a neural network model and a training image data set thereof, transmitting the training image data set to the neural network model and collecting intermediate results to obtain implicit characteristics of the neural network.
Firstly, rewriting the forward propagation subprocess of a given neural network model to be tested to lead the neural network model to derive an intermediate result in the reasoning process; a given training image dataset is then used, which is input into the neural network model and the intermediate layer implicit features are collected.
Step two: spatial joint probabilistic modeling is represented. And fitting the intermediate result by using a Gaussian mixture model to obtain model parameters, and collecting the path frequency calculation probability of the training image data set.
Two steps are needed for constructing the joint probability model, and firstly, a generative model based on a probability graph is established by using the intermediate layer implicit characteristics generated in the step one. Then mapping the characteristics of the middle layer into a discrete space to obtain the transition probability of the image data sample in the adjacent middle layer;
step three: a joint probability estimation model. Inputting the image data to be measured into the neural network model, and collecting the intermediate result according to the method as the step one. And calculating the generation probability and the interlayer transition probability of the intermediate result by using the Gaussian mixture model in the step two, performing rapid probability estimation by using a joint probability estimation model, and verifying whether the input image data to be detected is effective or not.
The neural network is a deep neural network, is a machine learning model which is formed by connecting neurons in a hierarchical manner and is used for extracting and predicting image data features, and comprises an input layer, a hidden layer and an output layer. The layers contain a large number of neurons, the neurons are connected among the layers, image data are transmitted from the input layer to the output layer, and prediction results, such as the category of the image data, are output; the intermediate layer result is implicit characteristic data output by neurons of an implicit layer of the neural network between the input layer and the output layer; the neuron is a structure that performs an operation on input data by using a built-in function or the like for neuron input and outputs an operation result; the input image data refers to a single or a batch of image data samples conforming to the input format of the deep neural network model.
In the first step, the neural network model to be tested is a pre-training model which contains model complete information (such as model architecture, neuron parameters and the like) and has complete operation authority (such as collecting model intermediate results, modifying model parameters and the like).
In the first step, the introduction of the training image data set into the neural network model and the collection of the intermediate result refer to covering the forward propagation process of the neural network to enable the neural network to retain the implicit characteristics output by the neurons in the intermediate layer, and collecting the implicit characteristics in the intermediate layer of the training image data set. Because the neurons in the neural network model are arranged in layers, intermediate results can also be divided into multiple layers. The implicit features of the intermediate layer can theoretically be fully collected and utilized, but due to practical hardware limitations, and the results obtained by sampling collection are already quite accurate, and all intermediate results are not necessarily used. In actual use, a uniform extraction strategy or a model structure extraction strategy can be adopted to sample the intermediate result. In practical application, if the characteristic data volume of the middle layer is still large, redundant information can be removed through dimension reduction. If global pooling is used to achieve good results, the PCA dimension reduction algorithm can also be used with the present invention.
In the second step, the Gaussian mixture model is a multi-dimensional Gaussian mixture clustering algorithm and has a fitting effect on multi-dimensional complex data distribution. The approximate solution of the gaussian mixture model can be solved by using a common expectation-maximization (EM) algorithm, so as to obtain several gaussian components and corresponding weights, and further obtain the sample probability by a generative method. These model parameters are the basis material for the framework for subsequent evaluation of the new sample probabilities.
In the second step, the joint probability modeling of the representation space refers to joint probability modeling of implicit features in the representation space of the deep neural network, and the purpose of the joint probability modeling is to model the inference process of the neural network. The expression space refers to a data space with hidden characteristics of the middle layer and contains the middle expression characteristics of the model input data.
In the second step, the fitting of the implicit features of the intermediate layer is to establish a probability distribution model of the implicit features of the intermediate layer by using a Gaussian Mixture Model (GMM). Using expectation-maximization (EM) algorithm to build a Gaussian mixture model on the ith intermediate layer output to obtain K i Parameters theta of the respective Gaussian components i And the weight of each Gaussian component i . Parameter phi in probabilistic graphical model i And Θ i Is a basic material for evaluating the abnormal probability of the image data.
In the second step, the probability is calculated by collecting the path frequency of the training image data set, and the probability calculation method is as follows: training setAll samples generate path data { (z) i ,z i-1 ) m |m∈Z m Where m is the training set size. Count out z i-1 To z i Probability of transition p (z) i |z i-1 ) And providing basic parameters for subsequent evaluation of sample probability.
In the second step, the transition probability of the adjacent middle layer refers to the middle layer characteristic x on the training image data set i Corresponding discrete component z i Transition probability between adjacent layers; the discrete component z i Means that according to the clustering result of the GMM of the ith layer, the input image data has the characteristic x of the middle layer of the ith layer i The cluster is z i . Further, z is calculated on the training image dataset i-1 To z i Probability of transition P (z) i |z i-1 ) I.e., transition probability from layer i-1 to layer i. z is a radical of i Possible values are K i The transition probability from layer i-1 to layer i is of size K i-1 ×K i Of the matrix of (a).
In the third step, the joint probability estimation model refers to estimating the middle layer output sequence { x ] through a probability map model 1 ,x 2 ,…,x m The joint probability of. Direct calculation of joint probability P (x) 1 ,x 2 ,…,x m ) The time complexity of (c) increases exponentially with respect to (m), and therefore the present invention uses a fast forward algorithm based on dynamic programming. Order to
α i (z i )≡P(z i ,x 1 ,x 2 ,…,x i )，
Then alpha i (. cndot.) can be generated by the following recursive process:
wherein K i-1 Indicates the number of Gaussian components of the i-1 st layer, P (x) i |z i ) Given by the GMM of layer i, P (z) i |z i-1 ) Is the transition probability. Therefore, we obtain the joint summary of the intermediate layer output with linear time complexityAnd (4) rate.
In the third step, the verifying whether the input image data to be tested is valid refers to judging whether the joint probability of the image data is greater than a preset threshold value; the threshold is a numerical value between 0 and 1, the closer to 0, the more inclined the precision rate of the method, and the closer to 1, the more inclined the recall rate of the method; one strategy that is practical is to set a threshold that normalizes most (e.g., 90%) of the image data based on the joint probabilities of the training image data sets.
An image data input detection system for a neural network vision system, comprising:
the neural network implicit feature extractor: and giving a neural network model and a training image data set thereof, transmitting the training image data set to the neural network model and collecting intermediate results to obtain implicit characteristics of the neural network.
Firstly, rewriting the forward propagation subprocess of a given neural network model to be tested to lead the neural network model to derive an intermediate result in the reasoning process; a given training image dataset is then used, which is input into the neural network model and the intermediate layer implicit features are collected.
Representing a spatial joint probability modeling tool: and fitting the intermediate result by using a Gaussian mixture model to obtain model parameters, and collecting the path frequency calculation probability of the training image data set.
Two steps are needed for constructing the joint probability model, and firstly, a generative model based on a probability graph is established by using intermediate layer hidden features generated by a neural network hidden feature extractor. Then mapping the characteristics of the middle layer into a discrete space to obtain the transition probability of the image data sample in the adjacent middle layer;
a joint probability estimation model; and inputting the image data to be detected into a neural network model, and collecting an intermediate result by using a neural network implicit feature extractor. And calculating the generation probability and the interlayer transition probability of the intermediate result by using a Gaussian mixture model in the space joint probability modeling tool, performing rapid probability estimation by using a joint probability estimation model, and verifying whether the input image data to be detected is effective or not.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method for detecting image data input for a neural network vision system as described above when executing the computer program.
A computer-readable storage medium storing a computer program for executing the neural network vision system-oriented image data input detection method as described above.
Has the advantages that: the method can make up the defect that the common neural network vision model is difficult to identify abnormal image data, thereby ensuring the safety of an intelligent system. Compared with the existing input data verification technology of the neural network model, the method does not need to use abnormal data for training, and is not easy to generate overfitting phenomenon on specific data. The invention can efficiently detect and evaluate the effectiveness of the input image data, and utilizes the evaluation effectiveness, thereby realizing the real-time screening of the test image and improving the actual deployment effect of the neural vision system.
Drawings
FIG. 1 is a method schematic of an embodiment of the invention;
FIG. 2 is a schematic diagram of an implicit feature extractor of a neural network according to an embodiment of the present invention;
FIG. 3 is a flow chart representing a spatial joint probability modeling tool according to an embodiment of the present invention;
FIG. 4 is a functional diagram of a joint probability estimation model according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a probability map model according to an embodiment of the present invention.
Detailed Description
The present invention is further illustrated by the following examples, which are intended to be purely exemplary and are not intended to limit the scope of the invention, as various equivalent modifications of the invention will occur to those skilled in the art upon reading the present disclosure and fall within the scope of the appended claims.
As shown in FIG. 1, the image data input detection method for the neural network vision system is composed of two stages of training and deployment. In the training stage, firstly, a hidden feature extractor is utilized to input training image data into a neural network model to generate hidden features generated by each intermediate layer. Thereafter, a Gaussian mixture model is trained on the intermediate layer features using a representation space joint modeling tool, and the transition probabilities between adjacent layers are calculated using the clustering results of the Gaussian mixture model. In the deployment stage, for any image data to be detected, the image data is firstly input into a neural network to obtain intermediate layer characteristics, and then a joint probability estimation model is used for calculating the joint probability of the input intermediate layer characteristics. And finally, comparing the joint probability value with a preset threshold value, and deciding to accept or reject the image data to be detected. The whole frame comprises three modules corresponding to three steps: the neural network implicit feature extractor represents a spatial joint probability modeling tool and a joint probability estimation model.
The image data input detection system facing the neural network vision system comprises:
the method comprises the following steps: and giving a neural network model and a training image data set thereof, transmitting the training image data set to the neural network model and collecting intermediate results to obtain implicit characteristics of the neural network.
As shown in FIG. 2, the forward propagation process of the neural network visual model is covered to retain the implicit characteristics of the neuron output in the middle layer, the training image data set is input into the neural network, and the implicit characteristics { x ] generated by each middle layer of the model are extracted 1 ,x 2 ,…,x m Where m is the number of interlayers. These intermediate features of the image data can theoretically be fully acquired and utilized, but due to practical hardware limitations, and the results obtained from the sample acquisition are already of a relatively high accuracy, and it is not necessary to use all the intermediate results. In actual use, a uniform extraction strategy or a model structure extraction strategy can be adopted for sampling. In practical application, if the characteristic data quantity of the middle layer is still large, redundant information output by the middle layer of the image data can be removed through dimension reduction. If global pooling is used to achieve good results, the PCA dimension reduction algorithm can also be used with the present invention. And storing the processed intermediate layer result for subsequent training.
In this step, the neural network vision system to be tested is required to include model complete information (such as model architecture, neuron parameters, etc.) and have complete operation authority (such as collecting model intermediate results, modifying model parameters, etc.). It is not applicable to the black box model that provides only API services.
Step two: and (3) modeling the inference process of the neural network visual model to the image data of the training set by utilizing the representation space joint probability modeling.
As shown in fig. 3, the gaussian mixture model is trained layer by layer on the intermediate layer data obtained in step one. The Gaussian mixture model of each layer can reasonably select parameters such as the number of Gaussian components according to the image data scale and the equipment calculation force. Reserving Gaussian mixture models on data of each intermediate layer, and clustering the image data of the training set to obtain an intermediate layer output clustering result z of the image data i And calculating the transition probability P (z) between adjacent middle-layer clusters according to the result i |z i-1 ) And storing the image data for the online to-be-detected image data.
Representation space joint probability modeling refers to joint probability modeling of implicit features in the representation space of a deep neural network, which aims at modeling the inference process of the neural visual network on image data. The representation space refers to a data space with hidden characteristics of the middle layer and contains the intermediate representation characteristics of the model input data. The method uses a probability graph model shown in figure 5 to represent the joint probability of the middle layers, the middle layers of the neural network and the layers in the probability graph model are in a corresponding relation, wherein x i Is the output of the ith intermediate layer of the neural network model, z i Is the hidden state of the ith layer of the probabilistic graphical model, phi i And Θ i Is a parameter of the method.
The step of respectively fitting the hidden features of the middle layer is to establish a probability distribution model of the hidden features of the middle layer by using a Gaussian Mixture Model (GMM). Using expectation-maximization (EM) algorithm to build a Gaussian mixture model on the ith intermediate layer output to obtain K i Parameters theta of the respective Gaussian components i And the weight Φ of each gaussian component i . Parameter phi in probabilistic graphical model i And Θ i Is to comment onAnd (3) estimating a base material of the abnormal probability of the image data.
The transition probabilities of adjacent intermediate layers refer to the intermediate layer features x on the training image dataset i Corresponding discrete component z i Transition probability between adjacent layers; discrete component z i The characteristic x of input data in the intermediate layer of the ith layer according to the clustering result of the GMM of the ith layer i The cluster is z i . Computing z on a training data set i-1 To z i Probability of transition P (z) i |z i-1 ) I.e., transition probability from layer i-1 to layer i. z is a radical of i Possible values are K i The transition probability from layer i-1 to layer i is of size K i-1 ×K i Of the matrix of (a).
Step three: and analyzing the effectiveness of the image data to be detected and reporting the effectiveness by using a joint probability estimation model.
As shown in fig. 4, one or a batch of image data to be measured is input into the neural network model, the inference process is executed, the intermediate result is extracted, and the dimension reduction processing is performed on the intermediate result and is stored. Note that the middle layer of the model selected in this step should be consistent with the operations of step one, as should the dimensionality reduction operations. Inputting the output data of the middle layer into the Gaussian mixture model corresponding to the middle layer to obtain the probability P (x) of the result of the middle layer on each Gaussian component i |z i ) Wherein z is i Representing the gaussian component of the image data at the ith layer. Estimating mid-layer output sequence x by a probabilistic graph model 1 ,x 2 ,…,x m The joint probability of. Direct calculation of joint probability P (x) 1 ,x 2 ,…,x m ) The time complexity of (c) increases exponentially with respect to (m), and a fast forward algorithm based on dynamic programming is used. Order to
α i (z i )≡P(z i ,x 1 ,x 2 ,…,x i )，
Then alpha i (. cndot.) can be generated by a recursive process as follows:
finally obtaining the characteristic sequence { x ] of the intermediate layer of the image data 1 ,x 2 ,…,x m The joint probability of. Judging whether the joint probability of the image data to be detected is greater than a preset threshold value or not; the threshold is a numerical value between 0 and 1, the closer to 0, the more inclined the precision rate of the method, and the closer to 1, the more inclined the recall rate of the method; one strategy that is feasible in practice is to set a threshold that normalizes most (e.g., 90%) of the image samples based on the joint probabilities of the training image data sets.
An image data input detection system for a neural network vision system, comprising:
the neural network implicit feature extractor: and (3) giving a neural network model and a training image data set thereof, transmitting the training image data set to the neural network model, and collecting an intermediate result to obtain the implicit characteristics of the neural network.
Firstly, rewriting the forward propagation subprocess of a given neural network model to be tested to lead the neural network model to derive an intermediate result in the reasoning process; the image dataset is then input into a neural network model and the intermediate layer implicit features are collected using a given training image dataset.
Representing a spatial joint probability modeling tool: and fitting the intermediate result by using a Gaussian mixture model to obtain model parameters, and collecting the path frequency calculation probability of the training image data set.
Two steps are needed for constructing a joint probability model, and firstly, a generation type model based on a probability chart is established by utilizing intermediate layer hidden features generated by a neural network hidden feature extractor. Then mapping the characteristics of the middle layer into a discrete space to obtain the transition probability of the image data sample in the adjacent middle layer;
a joint probability estimation model; and inputting the image data to be detected into a neural network model, and collecting an intermediate result by using a neural network implicit feature extractor. And calculating the generation probability and the interlayer transition probability of the intermediate result by using a Gaussian mixture model in the space joint probability modeling tool, performing rapid probability estimation by using a joint probability estimation model, and verifying whether the input image data to be detected is effective or not.
It is obvious to those skilled in the art that the steps of the image data input detection method for the neural network vision system or the modules of the image data input detection system for the neural network vision system of the above-described embodiments of the present invention can be implemented by a general-purpose computing device, they can be centralized on a single computing device or distributed on a network formed by a plurality of computing devices, alternatively, they can be implemented by program codes executable by the computing devices, so that they can be stored in a storage device and executed by the computing devices, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be respectively fabricated into various integrated circuit modules, or multiple modules or steps in them can be fabricated into a single integrated circuit module. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
Claims (10)
1. An image data input detection method for a neural network vision system is characterized by comprising the following steps:
the method comprises the following steps: extracting implicit characteristics of a neural network; giving a neural network model and a training image data set thereof, transmitting the training image data set to the neural network model and collecting an intermediate result to obtain implicit characteristics of the neural network;
step two: representing spatial joint probability modeling; fitting the intermediate result by using a Gaussian mixture model to obtain model parameters, and collecting training image data set path frequency calculation probability;
step three: a joint probability estimation model; inputting image data to be detected into a neural network model, and collecting an intermediate result according to the method in the step one; and calculating the generation probability and the interlayer transition probability of the intermediate result by using the Gaussian mixture model in the step two, performing rapid probability estimation by using a joint probability estimation model, and verifying whether the input image data to be detected is effective or not.
2. The image data input detection method for the neural network visual system as claimed in claim 1, wherein the neural network is a deep neural network, which is a machine learning model for image data feature extraction and prediction formed by hierarchical connection of neurons, and comprises an input layer, a hidden layer and an output layer; the layers comprise neurons, the neurons are connected among the layers, image data are transmitted from the input layer to the output layer, and a prediction result is output; the intermediate layer result is implicit characteristic data output by neurons of an implicit layer of the neural network between the input layer and the output layer; the neuron is a structure that performs an operation on input data by using a built-in function or the like for neuron input and outputs an operation result; the input image data refers to a single or a batch of image data samples conforming to the input format of the deep neural network model.
3. The method for detecting image data input facing to the neural network vision system as claimed in claim 1, wherein in the first step, according to the given neural network model to be detected, the forward propagation subprocess of the neural network model is rewritten so that the neural network model can derive the intermediate result in the inference process; a given training image dataset is then used, which is input into the neural network model and the intermediate layer implicit features are collected.
4. The neural network vision system-oriented image data input detection method as claimed in claim 1, wherein the intermediate results are sampled by using a uniform extraction strategy or a model structure extraction strategy.
5. The method for detecting image data input facing a neural network vision system as claimed in claim 1, wherein in the second step, two steps are required for constructing the joint probability model, and firstly, a generative model based on a probability map is established by using the intermediate layer implicit characteristics generated in the first step; then mapping the characteristics of the middle layer into a discrete space to obtain the transition probability of the image data sample in the adjacent middle layer;
the representation space refers to a data space with hidden characteristics of the middle layer and comprises middle representation characteristics of model input data;
the step of respectively fitting the intermediate layer hidden features is to establish a probability distribution model of the intermediate layer hidden features by using a Gaussian mixture model; using expectation-maximization algorithm to build a Gaussian mixture model on the ith intermediate layer output to obtain K i Parameters theta of the respective Gaussian components i And the weight of each Gaussian component i (ii) a Parameter phi in probabilistic graphical model i And Θ i Is a basic material for evaluating the abnormal probability of the image data.
6. The neural-network visual system-oriented image data input detection method according to claim 1, wherein in the second step, the transition probabilities of adjacent intermediate layers refer to the intermediate layer feature x on the training image data set i Corresponding discrete component z i Transition probability between adjacent layers; the discrete component z i Means that according to the clustering result of the GMM of the ith layer, the input image data has the characteristic x of the middle layer of the ith layer i The cluster is z i (ii) a Computing z on a training image dataset i-1 To z i Probability of transition P (z) i |z i-1 ) I.e., transition probability from layer i-1 to layer i; z is a radical of i Possible values are K i The transition probability from layer i-1 to layer i is of size K i-1 ×K i Of the matrix of (a).
7. The method for detecting image data input facing neural network vision system as claimed in claim 1, wherein in step three, said joint probability estimation model refers to estimating middle layer output sequence { x } through probability map model 1 ,x 2 ,…,x m J, directly calculating the joint probability P (x) 1 ,x 2 ,…,x m ) The time complexity of (a) increases exponentially with respect to (m), and therefore a fast forward algorithm based on dynamic programming is used; order to
α i (z i )≡P(z i ,x 1 ,x 2 ,…,x i )，
Then alpha i (. h) is generated by the following recursive process:
wherein K i-1 Indicates the number of Gaussian components of the i-1 st layer, P (x) i |z i ) Given by the GMM of layer i, P (z) i |z i-1 ) Is the transition probability.
8. An image data input detection system for a neural network vision system, comprising:
the neural network implicit feature extractor: giving a neural network model and a training image data set thereof, transmitting the training image data set to the neural network model and collecting an intermediate result to obtain implicit characteristics of the neural network;
firstly, rewriting a forward propagation subprocess of a given neural network model to be tested according to the neural network model to be tested so that an intermediate result is derived in a reasoning process; then, inputting the image data set into a neural network model by using a given training image data set and collecting implicit characteristics of the middle layer;
representing a spatial joint probability modeling tool: fitting the intermediate result by using a Gaussian mixture model to obtain model parameters, and collecting training image data set path frequency calculation probability;
two steps are needed for constructing the joint probability model, and firstly, a generative model based on a probability graph is established by using intermediate layer hidden features generated by a neural network hidden feature extractor. Then mapping the characteristics of the middle layer into a discrete space to obtain the transition probability of the image data in the adjacent middle layer;
a joint probability estimation model; inputting image data to be detected into a neural network model, and collecting intermediate results according to a neural network implicit feature extractor; and calculating the generation probability and the interlayer transition probability of the intermediate result by using a Gaussian mixture model in the space joint probability modeling tool, performing rapid probability estimation by using a joint probability estimation model, and verifying whether the input image data to be detected is effective or not.
9. A computer device, characterized by: the computer device comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the image data input detection method for the neural network vision system as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium characterized by: the computer-readable storage medium stores a computer program for executing the image data input detection method for a neural network vision system according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210522508.2A CN114863178A (en) | 2022-05-13 | 2022-05-13 | Image data input detection method and system for neural network vision system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210522508.2A CN114863178A (en) | 2022-05-13 | 2022-05-13 | Image data input detection method and system for neural network vision system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114863178A true CN114863178A (en) | 2022-08-05 |
Family
ID=82638079
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210522508.2A Pending CN114863178A (en) | 2022-05-13 | 2022-05-13 | Image data input detection method and system for neural network vision system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114863178A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122809A (en) * | 2017-04-24 | 2017-09-01 | 北京工业大学 | Neural network characteristics learning method based on image own coding |
| CN110633788A (en) * | 2019-08-14 | 2019-12-31 | 南京大学 | Input instance verification method based on interlayer analysis and oriented to neural network model |
| WO2020107687A1 (en) * | 2018-11-27 | 2020-06-04 | 邦鼓思电子科技（上海）有限公司 | Vision-based working area boundary detection system and method, and machine equipment |
-
2022
- 2022-05-13 CN CN202210522508.2A patent/CN114863178A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107122809A (en) * | 2017-04-24 | 2017-09-01 | 北京工业大学 | Neural network characteristics learning method based on image own coding |
| WO2020107687A1 (en) * | 2018-11-27 | 2020-06-04 | 邦鼓思电子科技（上海）有限公司 | Vision-based working area boundary detection system and method, and machine equipment |
| CN110633788A (en) * | 2019-08-14 | 2019-12-31 | 南京大学 | Input instance verification method based on interlayer analysis and oriented to neural network model |
| WO2021027052A1 (en) * | 2019-08-14 | 2021-02-18 | 南京大学 | Interlayer parsing-based input instance verfication method for neural network model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106897738B (en) | A kind of pedestrian detection method based on semi-supervised learning | |
| CN105738109A (en) | Bearing fault classification diagnosis method based on sparse representation and ensemble learning | |
| CN113610747B (en) | System and method for inspecting semiconductor samples | |
| CN114048468A (en) | Intrusion detection method, intrusion detection model training method, device and medium | |
| CN108846411A (en) | Main reducing gear method for diagnosing faults based on pairing sparse Bayesian extreme learning machine | |
| CN112504682A (en) | Chassis engine fault diagnosis method and system based on particle swarm optimization algorithm | |
| CN113255591A (en) | Bearing fault diagnosis method based on random forest and fusion characteristics | |
| CN117056834A (en) | Big data analysis method based on decision tree | |
| Du et al. | Convolutional neural network-based data anomaly detection considering class imbalance with limited data | |
| CN117676099B (en) | Security early warning method and system based on Internet of things | |
| CN111586728A (en) | Small sample characteristic-oriented heterogeneous wireless network fault detection and diagnosis method | |
| CN118427681A (en) | Cross-working condition open-set fault diagnosis method and equipment based on self-supervision contrast learning enhancement | |
| CN114357372A (en) | Aircraft fault diagnosis model generation method based on multi-sensor data driving | |
| CN113935413A (en) | Distribution network wave recording file waveform identification method based on convolutional neural network | |
| CN117909881A (en) | Fault diagnosis method and device for multi-source data fusion pumping unit | |
| CN117973511A (en) | Elevator fault diagnosis method integrating knowledge graph and neural network | |
| CN116188445A (en) | Product surface defect detection and positioning method and device and terminal equipment | |
| CN111031042A (en) | Network anomaly detection method based on improved D-S evidence theory | |
| CN114863178A (en) | Image data input detection method and system for neural network vision system | |
| CN114882289A (en) | SAR target open set identification method based on self-adaptive determination rejection criterion | |
| CN114022698A (en) | Multi-tag behavior identification method and device based on binary tree structure | |
| CN110555338A (en) | object identification method and device and neural network generation method and device | |
| CN114898164B (en) | Neural network image classifier confidence calibration method and system | |
| CN113032776B (en) | Feature-oriented embedded poisoning attack detection method, device and system | |
| CN118657784B (en) | Desoldering detection method based on unmanned aerial vehicle acquisition of photovoltaic module image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |