CN112434790B - Self-interpretation method for distinguishing part of black box problem of convolutional neural network - Google Patents
Self-interpretation method for distinguishing part of black box problem of convolutional neural network Download PDFInfo
- Publication number
- CN112434790B CN112434790B CN202011249200.2A CN202011249200A CN112434790B CN 112434790 B CN112434790 B CN 112434790B CN 202011249200 A CN202011249200 A CN 202011249200A CN 112434790 B CN112434790 B CN 112434790B
- Authority
- CN
- China
- Prior art keywords
- dclm
- layer
- cnn
- function
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 103
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000012549 training Methods 0.000 claims abstract description 25
- 238000005259 measurement Methods 0.000 claims abstract description 7
- 238000011176 pooling Methods 0.000 claims abstract description 7
- 238000002372 labelling Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 71
- 210000002569 neuron Anatomy 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 13
- 238000000605 extraction Methods 0.000 claims description 11
- 230000000593 degrading effect Effects 0.000 claims description 5
- 238000005315 distribution function Methods 0.000 claims description 3
- 238000000691 measurement method Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 210000004205 output neuron Anatomy 0.000 claims description 3
- 238000005192 partition Methods 0.000 claims description 3
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 230000001351 cycling effect Effects 0.000 claims 1
- 238000002474 experimental method Methods 0.000 description 11
- 238000013528 artificial neural network Methods 0.000 description 9
- 102100033592 Calponin-3 Human genes 0.000 description 6
- 101000945410 Homo sapiens Calponin-3 Proteins 0.000 description 6
- 238000013459 approach Methods 0.000 description 6
- 238000012886 linear function Methods 0.000 description 5
- 230000001364 causal effect Effects 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 238000003066 decision tree Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
- G06N5/042—Backward inferencing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a self-interpretation method for distinguishing part of black box problems of a convolutional neural network, which is implemented according to the following steps: step 1, providing an interpretable distance as an interpretability index of a measurement model; step 2, forming a feature extractor by a convolution layer and a pooling layer of the CNN network; step 3, taking the output of the feature extractor obtained in the step 2, namely the feature map, as the input of the CNN network full-connection layer; step 4, classifying and labeling input image samples in the CNN network by using the feature map in the step 3 in the full connection layer; step 5, using the full connection layer to form a judging part of the CNN network; step 6, constructing a DCLM model of the three-layer node; and 7, providing a novel game method to perform game training of the DCLM model and the CNN constructed in the step 6 so as to improve the interpretability of the judging part of the CNN. The invention solves the problem of large error of the interpretation method in the prior art.
Description
Technical Field
The invention belongs to the technical field of deep learning in the computer field, and particularly relates to a self-interpretation method for distinguishing part of black box problems of a convolutional neural network.
Background
In recent years, convolutional neural networks (hereinafter CNN) have significantly exceeded the ability of humans as representative algorithms for deep learning, on certain tasks such as computer vision, computer games, and the like. However, the manner in which the neural network handles the problem is difficult to understand and interpret, and this ubiquitous difficulty in interpretation, in turn, leads to many problems in terms of fairness, privacy, robustness, and credibility in the application; many researchers currently attempt to explain the black box problem around CNNs in different ways.
The so-called black box problem of CNN has major difficulties in the unexplained problem of its discrimination part, and there are two main methods for dealing with such problems currently: pre-and post-hoc methods (Holzinger et al, 2019); since the prior art approach is generally a transparent modeling approach (architaa et al 2020), it is difficult to obtain an explanation about the discriminant part, and thus the common technique is mainly focused on the second post-process approach.
Early post hoc approaches have mainly obtained global interpretations of neural networks by extracting a predictable model. (Craven & Shavlik,1999; krishnan et al, 1999; boz, 2002; johansson & Niklasson, 2009) proposes a method to find a decision tree for interpreting neural networks by maximizing gain ratio and estimating fidelity of the current model;
(Craven & Shavlik,1994; johansson & Niklasson, 2003; augasta & Kathovalavakumar, 2012; sebastin et al, 2015; zilke et al, 2016) a specific rule extraction method for searching for optimal interpretable rules from a neural network has been proposed;
in recent years, some feature association methods have also been generated, for example, a network classification decision decomposition method based on deep taylor decomposition proposed by montason et al (montason et al, 2017); a deep lift algorithm for calculating the importance scores of a multi-layer neural network proposed by Shrikumar et al (Shrikumar et al 2016) interprets the differences in output of certain reference outputs by interpreting the differences between their inputs and their reference inputs;
in addition, che et al have proposed a simple rectification method using gradient enhancement trees to extract an interpretable simple model called interpretable simulation learning, thiagarajan et al build a tree-like view representation of a complex model through hierarchical division of feature space to improve interpretability; still other scholars have proposed a rectification method for converting knowledge from a set of models to a single model, wu et al have proposed a tree regularization method based on knowledge rectification to represent neural network output feature space based on multi-layer perception;
however, these approaches only address the unexplained problem that exists with trained neural networks or trained deep neural networks with explicit input features; moreover, because data deviation and noise data exist in the training data set, the traditional machine learning method is difficult to ensure the interpretation performance and generalization performance of the identification part to be consistent and convergent; in addition, the interpretable feature is a feature that represents the distance between the discriminating portion and its optimal interpretable model, and when the discriminating portion has better generalization, it tends to deviate from its optimal interpretable model, thereby producing an erroneous interpretable model.
Disclosure of Invention
The invention aims to provide a self-interpretation method for the black box problem of a convolutional neural network discrimination part, which solves the problem of overlarge interpretation method error in the prior art.
The technical scheme adopted by the invention is that the self-interpretation method for judging the black box problem of the part of the convolutional neural network is implemented according to the following steps:
step 1, providing an interpretable distance as an interpretability index of a measurement model;
step 2, forming a feature extractor by a convolution layer and a pooling layer of the CNN network;
step 3, taking the output of the feature extractor obtained in the step 2, namely the feature map, as the input of the CNN network full-connection layer;
step 4, classifying and labeling input image samples in the CNN network by using the feature map in the step 3 in the full connection layer;
step 5, using the full connection layer to form a judging part of the CNN network;
step 6, constructing a DCLM model of the three-layer node;
and 7, providing a novel game method to perform game training of the DCLM model and the CNN constructed in the step 6 so as to improve the interpretability of the judging part of the CNN.
The present invention is also characterized in that,
in step 1, it is assumed that under the same input data set, the similarity is measured by the variance of the difference between the outputs of the original CNN network model and the constructed DCLM model, and this measurement is primarily named as the interpretation distance, the smaller the interpretation distance, the closer the shape of the discriminating portion of the CNN network is to the shape of the optimal interpretable model, so that the better the interpretable performance of the interpretation portion is;
assuming that Q is a compact metric space, Z is the sample set, v is the Borel metric, such as the Lebesgue metric or edge metric, in the metric space Q, in the square integrable function space over the metric space QIn the determination section f (x), the interpretation distance phi between the best interpreted models P (x) d (P x, f) is expressed as:
wherein the method comprises the steps of
The step 6 is specifically as follows:
the DCLM three-layer network structure is respectively: the feature predicate layer is formed by feature predicate nodes as a first layer, wherein each feature predicate node is provided with a feature predicate function Z (Γ) used for representing whether a neuron of a judging part of a first full-connection layer in the CNN network has the capability of capturing a feature, and the feature predicate function Z (Γ) is expressed as follows:
where i e {1,2,., k }, τ i Is the i-th feature map in Γ, wi is its corresponding weight vector in the first fully connected layer, called feature capture layer, where x represents convolution.
In the step 6, the decision result predicate layer is the bottommost layer, and each result predicate node has a result predicate function which indicates whether the output neuron of the judging part is larger than 0; the corresponding function is expressed as follows:
in the step 6, all characteristic predicate layer nodes and result predicate layer nodes are connected with extraction nodes of one or more middle layers, and the layers are called extraction layers and represent true or false truth conditions; each disjunct node represents disjunct relations of all characteristic predicate-layer nodes and result predicate-layer nodes, and the disjunct relations are represented by a disjunct paradigm; if a predicate layer node is connected with a disjunctive layer node by a true or false boundary, its predicate function follows the non-operation in a disjunctive operation, and a potential function of a disjunctive normal form is obtained by the Lukasiewicz method as follows:
φ c (y)＝min(1，T(Γ，y)) (3)
wherein:
wherein N is the node number of the characteristic predicate layer; Γ is a feature map, Z j () Is a characteristic predicate layer function, D () is a result layer function; if the logic network parameter a j If the predicate node and the disjunctive node are false boundaries, if the logical network parameter a is =1 j The value of 0 indicates that this is a true boundary.
Step 6, setting a DCLM model to include a disjunctive relation:
equal toWherein Z is j () Is a characteristic predicate layer function, j is E (1, 4), y is the neuron output of the CNN network; the potential function of this disjunctive paradigm is:
wherein a is 1 ＝a 4 =1 and a 2 ＝a 3 ＝0；
The ground trunk conditional probability distribution of DCLM is:
where G is the number of all group functions, and the partition function, xi, in the formula is expressed as follows:
output value y of neuron output layer of CNN network dclm The method comprises the following steps:
y dclm ＝(y dclm，1 ，y dclm，2 ，...，y dclm ，G) (7)
wherein lambda is i The optimal output value in the DCLM model can be obtained by maximizing the likelihood function for the weight of the i-th group function: y is dclm Logical network parameter a j And lambda is i The method comprises the steps of carrying out a first treatment on the surface of the The process of maximizing the likelihood function is shown in the following formula:
in phi ci As a potential function, a i Is a logical network parameter;
for DCLM, firstly, constructing a logic network, and then adopting a maximum a posterior MAP algorithm to extract a model, wherein the method comprises the following steps:
inputting a feature map, a feature capture matrix, a label corresponding to input and the number of inputs;
initializing an authenticity edge set and an initial value of y-dclm, and setting the number of initialization iterations as 1; entering the outer loop body performs the following operations on all feature maps:
calculating and storing the similarity of cosine by using the current feature map and the weight matrix corresponding to the current feature map, calculating and storing the product of the models of the current feature map and the weight matrix;
traversing each dimension of the output vector, solving the latest edge value in the extraction corresponding to the current dimension by using the similarity result obtained by the upper part of the dimension, the product of the modes and the corresponding output y-dclm, and adding 1 to the iteration count by using the existing edge value set and the current edge value set;
and when the iteration times are equal to the initial input number, namely the algorithm ending condition is met.
The iterative optimization thought of the game in the step 7 is as follows:
input: inputting a data set and outputting a target;
initializing: the logic network is named LN and the CNN network is named CN;
first,: putting a sample in a CN to obtain the output of a feature map FM and the CN, using the feature map FM in a logic network model LN to obtain the output of a logic network, using the output of the logic network, a label and the output of the CN to obtain a Loss function Loss, and updating the CN;
substituting a sample into the updated CN to obtain a new feature map, a weight matrix corresponding to the full-connection layer and the feature map and the output of the CN, and continuing to update and construct a logic network LN;
forming a logic network to continuously correct the CN, and constructing the logic network by the CN; the loop is thus performed until the end condition is satisfied.
The step 7 is specifically as follows:
as known from the measurement method of interpretation distance, when the distinguishing part of CNN is sufficiently similar to the shape of its optimal interpretable model, the distinguishing part has good interpretable performance, but its generalization performance tends to decrease, which is mainly due to the fact that two conditions for consistent convergence of performance are difficult to be ensured, as follows:
φ d (P x, f) =0, f (x) is the CNN discrimination part, and P (x) is the best interpretation model;
since this sufficient requirement is difficult to meet, the equalization problem is always present and its optimal interpretable model P and its optimal discriminating portion are unknown, it may be a viable solution to automatically extract one interpretable model from the discriminating portion during training and then iteratively reduce the interpretation distance between the two models when considering the equalization problem, as follows:
to avoid degrading the generalization performance, the highest probability of p (wX, y) t ) Wherein X is a training sample, w is a parameter set of CNN, y t Target direction of XThe amount can be obtained by:
wherein,
when DCLM is known, y dclm Is an optimal solution thereof, and
p(y*d clm |w，X)＝1 (14)
based on this, it is obtained:
likewise, the parameter set w, y of the training samples X, CNN of known input values t As the target vector of X, fnn is the optimal solution of the CNN network, which can be obtained:
p(y t |w，X)＝p(y t |f nn ，w，X)p(f nn |y* dclm ，w，X) (16)
if the parameter set w of CNN and training sample X are given, and the loss function is
The conditional probability distribution function is:
at the same time:
wherein, the xi 1, the xi 2 is the partitioning function; by maximizing the likelihood function of p (w|x, yt), the optimal parameter set w of CNN can be obtained, assuming that w obeys gaussian distribution, it is possible to obtain:
where α is a meta-parameter determined by the variance of the selected gaussian distribution, and converting this equation to a minimization problem yields:
the self-interpretation method for the black box problem of the convolutional neural network discrimination part has the beneficial effects that on the premise of not damaging the generalization performance of the convolutional neural network, the causal relationship of the CNN without the prior structure is actively extracted, the interpretation is finally obtained through the causal relationship, and meanwhile, the interpretable performance of the discrimination part of the CNN can be improved. In the process, the invention provides an interpretable model, namely a depth perception learning model (DCLM), for expressing the causal relationship of the judging part, and provides a greedy algorithm for automatically extracting the DCLM from the judging part by solving the maximum satisfiability problem of the judging part. A new game method is provided, the distance between two models is reduced through iteration, the two models are corrected, and the interpretable performance of the judging part is improved on the premise that the generalization performance of the judging part is not greatly reduced. Meanwhile, the generalization performance of the DCLM is improved. An interpretable distance is proposed for evaluating and measuring the distance between the discriminating portion and the interpretable model on the unexplained problem.
Drawings
Fig. 1 is a specific structural representation of a three-layer network structure of a deep perception learning model DCLM;
FIG. 2 is a representation of a disjunctive relationship in a deep-aware learning model DCLM;
FIG. 3 is the accuracy of DCLMs and CNN-DCLMs for each epoch in the gaming process;
FIG. 4 is an explanatory distance of DCLMs from CNNs and CNN-DCLMs;
FIG. 5 is the information entropy of DCLM;
fig. 6 is the precision of CNN and CNN-DCLM.
Detailed Description
The invention will be described in detail below with reference to the drawings and the detailed description.
The invention discloses a self-interpretation method for distinguishing part of black box problems of a convolutional neural network, which is implemented according to the following steps:
step 1, providing an interpretable distance as an interpretability index of a measurement model;
in step 1, it is assumed that under the same input data set, the similarity is measured by the variance of the difference between the outputs of the original CNN network model and the constructed DCLM model, and this measurement is primarily named as the interpretation distance, the smaller the interpretation distance, the closer the shape of the discriminating portion of the CNN network is to the shape of the optimal interpretable model, so that the better the interpretable performance of the interpretation portion is;
assuming that Q is a compact metric space, Z is the sample set, v is the Borel metric, such as the Lebesgue metric or edge metric, in the metric space Q, in the square integrable function space over the metric space QIn the determination section f (x), the interpretation distance phi between the best interpreted models P (x) d (P x, f) is expressed as:
wherein the method comprises the steps of
The step 6 is specifically as follows:
as shown in fig. 1, when constructing the DCLM model, since the relationship between the feature captured by the weight matrix of the first layer and the output vector thereof is an internal relationship of the CNN network structure, it is not related to the input data, but not to the abstract degree of the feature, so that the distinguishing portion can be considered to be better explained by a logical relationship, and the representation of the logical relationship is represented by the DCLM three-layer network structure, which is respectively: the feature predicate layer is formed by feature predicate nodes as a first layer, wherein each feature predicate node is provided with a feature predicate function Z (Γ) used for representing whether a neuron of a judging part of a first full-connection layer in the CNN network has the capability of capturing a feature, and the feature predicate function Z (Γ) is expressed as follows:
where i e {1,2,., k }, τ i Is the ith feature map in Γ, w i Is its corresponding weight vector in the first fully connected layer, called the feature capture layer, where x represents the convolution.
Step 2, forming a feature extractor by a convolution layer and a pooling layer of the CNN network;
step 3, taking the output of the feature extractor obtained in the step 2, namely the feature map, as the input of the CNN network full-connection layer;
step 4, classifying and labeling input image samples in the CNN network by using the feature map in the step 3 in the full connection layer;
step 5, using the full connection layer to form a judging part of the CNN network;
step 6, constructing a DCLM model of the three-layer node;
in the step 6, the decision result predicate layer is the bottommost layer, and each result predicate node has a result predicate function which indicates whether the output neuron of the judging part is larger than 0; the corresponding function is expressed as follows:
in the step 6, all characteristic predicate layer nodes and result predicate layer nodes are connected with extraction nodes of one or more middle layers, and the layers are called extraction layers and represent true or false truth conditions; each disjunct node represents disjunct relations of all characteristic predicate-layer nodes and result predicate-layer nodes, and the disjunct relations are represented by a disjunct paradigm; if a predicate layer node is connected with a disjunctive layer node by a true or false boundary, its predicate function follows the non-operation in a disjunctive operation, and a potential function of a disjunctive normal form is obtained by the Lukasiewicz method as follows:
φ c (y)＝min(1，T(Γ，y)) (3)
wherein:
wherein N is the node number of the characteristic predicate layer; Γ is a feature map, Z j () Is a characteristic predicate layer function, D () is a result layer function; if the logic network parameter a j If the predicate node and the disjunctive node are false boundaries, if the logical network parameter a is =1 j The value of 0 indicates that this is a true boundary.
As shown in fig. 2, the DCLM model set in step 6 includes a disjunctive relation:
equal toWherein Z is j () Is a characteristic predicate layer function, j is E (1, 4), y is the neuron output of the CNN network; the potential function of this disjunctive paradigm is:
wherein a is 1 ＝a 4 =1 and a 2 ＝a 3 ＝0；
The ground trunk conditional probability distribution of DCLM is:
where G is the number of all group functions, and the partition function, xi, in the formula is expressed as follows:
output value y of neuron output layer of CNN network dclm The method comprises the following steps:
y dclm ＝(y dclm，1 ，y dclm，2 ，...，y dclm ，G) (7)
wherein lambda is i The optimal output value in the DCLM model can be obtained by maximizing the likelihood function for the weight of the i-th group function: y is dclm Logical network parameter a j And lambda is i The method comprises the steps of carrying out a first treatment on the surface of the The process of maximizing the likelihood function is shown in the following formula:
in phi ci As a potential function, a i Is a logical network parameter;
for DCLM, firstly, constructing a logic network, and then adopting a maximum a posterior MAP algorithm to extract a model, wherein the method comprises the following steps:
inputting a feature map, a feature capture matrix, a label corresponding to input and the number of inputs;
initializing an authenticity edge set and an initial value of y-dclm, and setting the number of initialization iterations as 1; entering the outer loop body performs the following operations on all feature maps:
calculating and storing the similarity of cosine by using the current feature map and the weight matrix corresponding to the current feature map, calculating and storing the product of the models of the current feature map and the weight matrix;
traversing each dimension of the output vector, solving the latest edge value in the extraction corresponding to the current dimension by using the similarity result obtained by the upper part of the dimension, the product of the modes and the corresponding output y-dclm, and adding 1 to the iteration count by using the existing edge value set and the current edge value set;
and when the iteration times are equal to the initial input number, namely the algorithm ending condition is met.
The iterative optimization thought of the game in the step 7 is as follows:
input: inputting a data set and outputting a target;
initializing: the logic network is named LN and the CNN network is named CN;
first,: putting a sample in a CN to obtain the output of a feature map FM and the CN, using the feature map FM in a logic network model LN to obtain the output of a logic network, using the output of the logic network, a label and the output of the CN to obtain a Loss function Loss, and updating the CN;
substituting a sample into the updated CN to obtain a new feature map, a weight matrix corresponding to the full-connection layer and the feature map and the output of the CN, and continuing to update and construct a logic network LN;
forming a logic network to continuously correct the CN, and constructing the logic network by the CN; the loop is thus performed until the end condition is satisfied.
And 7, providing a novel game method to perform game training of the DCLM model and the CNN constructed in the step 6 so as to improve the interpretability of the judging part of the CNN.
The step 7 is specifically as follows:
as known from the measurement method of interpretation distance, when the distinguishing part of CNN is sufficiently similar to the shape of its optimal interpretable model, the distinguishing part has good interpretable performance, but its generalization performance tends to decrease, which is mainly due to the fact that two conditions for consistent convergence of performance are difficult to be ensured, as follows:
φ d (P x, f) =0, f (x) is the CNN discrimination part, and P (x) is the best interpretation model;
due to thisTo prove that this conclusion, one can first focus on only one neuron of the CNN network, because each input channel f (x) of the neuron can be considered as a kernel function K (x, w), where w is the weight vector containing the offset in the neuron, it can be Zhang Chenghe Hilbert space, Q is a compact metric space, Z is the sample set, v is the Borel metric, such as Lebesgue metric or edge metric, in the square integrable function space on metric space QIn (a):
H K is one arranged atThe linear function above, however, due to data bias and noise data in the training dataset, its optimal interpretable model is +.>The above is often a nonlinear function because the continuous linear function is collected +.>Is not dense. Therefore, the adequate requirements are difficult to satisfy, and the trade-off problem always exists, which also exists in the CNN discrimination section; however, its optimal interpretable model P x and its optimal discriminant part are unknown, so when considering the trade-off problem, it may be a viable solution to extract one interpretable model from the discriminant part during training and then iteratively reduce the interpretation distance between the two models, as follows: if the minimized loss function of CNN can guarantee the interpretabilityThe learning algorithm of CNN can improve its performance interpretability with the lowest performance. If not guaranteed, the trade-off between generalization performance and interpretability performance will not exist. To prove the existence of a problem, only one neuron of the CNN may be of interest; if an input channel f (x) of a neuron is considered a kernel function K (x, w) (w is a weight vector containing an offset of the neuron), it will open up a kernel hilbert space for the neuron. />
H K Can be regarded as one inThe upper set of linear functions is the solution space of the neurons. In the following theory->The conditions for consistent convergence of generalization performance and interpretable performance are discussed:
the lemma 1, the continuous linear set of functional on the separable Hilbert space is not dense in the square integrality function space;
the lemma 2, a set of consecutive nonlinear functions of separable hilbert space is dense everywhere in the L2 theorem.
The discussion is as follows:
at H K Wherein, when the optimal input channel f (x) approximates a linear function, the optimal interpretable model P (x) is at H K In the absence of any linear function, according to the argument 1, the conventional training procedure cannot guarantee that f (x) approximates P (x). From the quotients 2, it can be found that convergence cannot be achieved until P (x) approximates f (x). Here, if the approximation is defined as the similarity of f (x) and P (x) function curve shapes, the conditions for consistent convergence of the two properties are d (p×f=0). For the judgment part of CNN, the sufficiency requirement is still satisfied. However, in most engineering applications, this condition is difficult to preserve in most cases due to data bias and noise data in the training datasetSyndrome/pattern. There is always a trade-off between the two properties of the discriminating part.
From the above conclusion, to completely solve the trade-off problem, φ should be reduced d (p×f) =0 (f (x) is the CNN discrimination section, and p×x is the best interpretation model). But P and f are unknown, so that during training, an interpretable model P (x) is extracted from the discriminant part f (x) and then iteratively reduced by phi d (P, f) is a more convenient method.
To avoid degrading the generalization performance, the highest probability of p (wX, y) t ) Wherein X is a training sample, w is a parameter set of CNN, y t As the target vector of X, it is possible to obtain:
wherein,
when DCLM is known, y dclm Is an optimal solution thereof, and
p(y* dclm |w，X)＝1 (14)
based on this, it is obtained:
likewise, the parameter set w, y of the training samples X, CNN of known input values t As the target vector of X, fnn is the optimal solution of the CNN network, which can be obtained:
p(y t |w，X)＝p(y t |f nn ，w，X)p(f nn |y* dclm ，w，X) (16)
if the parameter set w of CNN and training sample X are given, and the loss function is
The conditional probability distribution function is:
at the same time:
wherein, the xi 1, the xi 2 is the partitioning function; by maximizing the likelihood function of p (w|x, yt), the optimal parameter set w of CNN can be obtained, assuming that w obeys gaussian distribution, it is possible to obtain:
where α is a meta-parameter determined by the variance of the selected gaussian distribution, and converting this equation to a minimization problem yields:
and (3) experimental verification:
experiments were designed in the present invention to verify the effectiveness of the invention. The first experiment verifies whether the self-explanatory method can improve the interpretable performance of CNN without degrading the generalization performance of CNN. A second experiment verifies whether the method can tend to stabilize and converge during the gaming process.
The structures of CNN3 (including 3 convolutional layers, 3 max pooling layers, 3 Fully Connected Layers (FCLs), and 1 output layer), CNN5 (including 5 convolutional layers, 5 max pooling layers, 3 fully connected layers, and 1 output layer), and CNN8 (including 8 convolutional layers, 8 max pooling layers, 3 fully connected layers, and 1 output layer) were used in the experiments. The final structures obtained after training these CNNs by this method are named CNN3-DCLM, CNN5-DCLM and CNN8-DCLM, respectively. All experiments were performed using Mnist (Lecunet al, 1998), fashion Mnist (Zalando, 2017) and Emnist (cohenetal, 2017) baseline data sets. All algorithms were implemented in Python using Pytorch libraries (paszkey et al 2019), and experiments were run on a server equipped with Intel Xeon 4110 (2.1 GHz) processor, 20GBRAM and nvidia telsat 4.
Table 1 classification accuracy and interpretation distance for three gaming methods on DCLM-CNN
Experiment 1 Performance verification of the methods herein on CNN
Experiments also compared Soft Decision Trees (SDT). (Hinton, 2017) is based on trained CNN3, CNN5 and CNN8, i.e. CNN3-sdt, CNN5-sdt and CNN8-sdt. The baseline methods also included SDT (frost & (hiton, 2017) based on trained CNN3-dclm, CNN5-dclm and CNN8-dclm were CNN3 x-SDT, CNN5 x-SDT and CNN8 x-SDT, respectively.
As can be seen from table 1, the accuracy of CNN-DCLMs is higher than both the interpretable models SDT and DCLM and about 1.4 percent lower than that of CNN on all baseline data sets. It is noted that, except for CNN3-SDT on the Emnist dataset, the interpretation distance of all CNN-DCLMs on most datasets is about 5 percent lower than the interpretation distance of most SDT from their CNNs, which may prove that the self-interpretation method can improve the interpretable performance of the resolved part of CNN without significantly degrading the generalization performance of CNN;
on the Mnist dataset, except for CNN3-SDT, on CNN5-SDT and CNN8-SDT, the accuracy of DCLMs was only 0.7 percent lower than that of SDT. From the Emnist dataset and the FashionMnist dataset, DCLMs can be found to be 2 percent less accurate than SDT CNN 3-SDT; it was also found that the accuracy of DCLMs for CNN5-SDT and CNN8-SDT was about 4.3 percent higher than for SDTs on both datasets. This is mainly because the feature graphs output by the feature extraction sections of CNN5 and CNN8 are more abstract than CNN3 for the Emnist dataset and the FashionMnist dataset. The abstract feature map can easily represent the undistorted nature of the DCLM of the predicate so it does not interfere with or affect the generation of the DCLM;
experiment 2 Convergence test of the proposed method
The convergence of the method is verified through experiments: CNN3, CNN5, CNN8 were compared with CNN3-dclm, CNN5-dclm, CNN8-dclm, respectively. 25 epochs are required for each training and all results are measured at each epoch as shown in figures 3, 4, 6, 5. Each graph includes 9 secondary graphs. The three subgraphs in the left column are experiments of the Mnist dataset. The three subgraphs in the middle column are the FashionMnist dataset and the three subgraphs in the right column are the Emnist dataset. Figure 3 shows the accuracy of the DCLMs and CNN-DCLMs for each epoch in the gaming process. It is evident from these data that the accuracy of dclm and that of these cnn-dclm are steadily increasing in the early stages. In the next stage, their accuracy tends to stabilize. This illustrates that the gaming method does not affect the improvement in dclm and cnn generalization performance;
FIG. 4 shows the interpretation distance of DCLMs from CNNs and CNN-DCLMs. It can be seen from these subgraphs that most of the time, CNN-DCLMs that are not involved in gaming are interpreted more than CNN-DCLMs, especially at the end of training. The result shows that the game method can effectively improve the interpretable performance of CNN-DCLMs.
As can be seen from fig. 5, dclm of CNN3, CNN5, CNN8 all have a stable entropy value at the end of the game. On the Mnist dataset, the entropy eventually converges between 120 and 145. On the fashionmist dataset, the entropy value eventually converges between 125 and 136. On the Emnist dataset, the entropy eventually converges between 100 and 120. The result shows that the game algorithm can ensure that the DCLM converges to a stable state
FIG. 6 shows the accuracy of the epochs CNN and CNN-DCLMs in the gaming process; from these sub-graphs, it can be seen that the accuracy of CNNs and CNN-DCLMs steadily increased in the early stages, while the accuracy of CNN-DCLMs was lower than CNNs. But in the last stage its accuracy tends to be stable and consistent. The main reason is that in the early stage, the trade-off problem between generalization performance and interpretive performance CNN-DCLMs must reduce the performance of CNN-DCLMs whose generalization performance increases the performance of its interpretive performance. The proposed gaming method can effectively reduce the gap between the two performances. This suggests that this approach is effective against trade-off problems
From the sub-graphs of fig. 3 and 4, it can also be seen that one DCLM model is available on each epoch. After 15 epochs, their interpretation distance tends to converge. This phenomenon suggests that the gaming process can be reduced in distance, while the training process of the network neural network identification section can be interpreted in real time with the dclm model after 15 epochs.
Claims (1)
1. A self-interpretation method for distinguishing part of black box problems of a convolutional neural network is characterized by comprising the following steps:
step 1, providing an interpretable distance as an interpretability index of a measurement model;
in the step 1, it is assumed that the similarity is measured by the variance of the difference between the outputs of the original CNN network model and the constructed DCLM model under the same input data set, and the measurement is primarily named as an interpretation distance, and the smaller the interpretation distance, the closer the shape of the discriminating portion of the CNN network is to the shape of the optimal interpretable model, so that the better the interpretable performance of the interpretation portion is;
assuming that Q is a compact metric space, Z is the sample set, v is the Borel metric, such as the Lebesgue metric or edge metric, in the metric space Q, in the square integrable function space over the metric space QIn the determination section f (x), the interpretation distance phi between the best interpreted models P (x) d (P x, f) is expressed as:
φ d (P*，f)＝∫ z (f(x)-P*(x)-μ P* (f)) 2 dv (9)
wherein mu P* (f)＝∫ z (f(x)-P * (x))dv (10)；
Step 2, forming a feature extractor by a convolution layer and a pooling layer of the CNN network;
step 3, taking the output of the feature extractor obtained in the step 2, namely the feature map, as the input of the CNN network full-connection layer;
step 4, classifying and labeling input image samples in the CNN network by using the feature map in the step 3 in the full connection layer;
step 5, using the full connection layer to form a judging part of the CNN network;
step 6, constructing a DCLM model of the three-layer node;
the step 6 specifically comprises the following steps:
the DCLM three-layer network structure is respectively: the feature predicate layer is formed by feature predicate nodes as a first layer, wherein each feature predicate node is provided with a feature predicate function Z (Γ) used for representing whether a neuron of a judging part of a first full-connection layer in the CNN network has the capability of capturing a feature, and the feature predicate function Z (Γ) is expressed as follows:
where i e {1,2,., k }, τ i Is the ith feature map in Γ, w i Is the corresponding weight vector in the first full connection layer, called the feature capture layer, where x represents convolution;
in the step 6, the decision result predicate layer is the bottommost layer, and each result predicate node has a result predicate function which indicates whether the output neuron of the judging part is larger than 0; the corresponding function is expressed as follows:
all characteristic predicate layer and result predicate layer nodes in the step 6 are connected with extraction nodes of one or more intermediate layers, and the layers are called extraction layers and represent true or false true value conditions; each disjunct node represents disjunct relations of all characteristic predicate-layer nodes and result predicate-layer nodes, and the disjunct relations are represented by a disjunct paradigm; if a predicate layer node is connected with a disjunctive layer node by a true or false boundary, its predicate function follows the non-operation in a disjunctive operation, and a potential function of a disjunctive normal form is obtained by the Lukasiewicz method as follows:
φ c (y)＝min(1，T(Γ，y)) (3)
wherein:
wherein N is the node number of the characteristic predicate layer; Γ is a feature map, Z j () Is a characteristic predicate layer function, D () is a result layer function; if the logic network parameter a j If the predicate node and the disjunctive node are false boundaries, if the logical network parameter a is =1 j A value of 0 indicates that this is a true boundary;
in the step 6, setting a DCLM model includes a disjunctive relation:
equal toWherein Z is j () Is a characteristic predicate layer function, j is E (1, 4), y is the neuron output of the CNN network; the potential function of this disjunctive paradigm is:
wherein a is 1 ＝a 4 =1 and a 2 ＝a 3 ＝0；
The ground trunk conditional probability distribution of DCLM is:
where G is the number of all group functions, and the partition function, xi, in the formula is expressed as follows:
output value y of neuron output layer of CNN network dclm The method comprises the following steps:
y dclm ＝(y dclm，1 ，y dclm，2 ，...，y dclm ，G) (7)
wherein lambda is i The optimal output value in the DCLM model can be obtained by maximizing the likelihood function for the weight of the i-th group function: y is dclm Logical network parameter a j And lambda is i The method comprises the steps of carrying out a first treatment on the surface of the The process of maximizing the likelihood function is shown in the following formula:
in phi ci As a potential function, a i Is a logical network parameter;
for DCLM, firstly, constructing a logic network, and then adopting a maximum a posterior MAP algorithm to extract a model, wherein the method comprises the following steps:
inputting a feature map, a feature capture matrix, a label corresponding to input and the number of inputs;
initializing an authenticity edge set and an initial value of y-dclm, and setting the number of initialization iterations as 1; entering the outer loop body performs the following operations on all feature maps:
calculating and storing the similarity of cosine by using the current feature map and the weight matrix corresponding to the current feature map, calculating and storing the product of the models of the current feature map and the weight matrix;
traversing each dimension of the output vector, solving the latest edge value in the extraction corresponding to the current dimension by using the similarity result obtained by the upper part of the dimension, the product of the modes and the corresponding output y-dclm, and adding 1 to the iteration count by using the existing edge value set and the current edge value set;
when the iteration times are equal to the initial input number, namely the algorithm ending condition is met;
step 7, providing a novel game method to perform game training of the DCLM model and the CNN constructed in the step 6 so as to improve the interpretability of the judging part of the CNN;
the step 7 game iterative optimization thinking is as follows:
input: inputting a data set and outputting a target;
initializing: the logic network is named LN and the CNN network is named CN;
first,: putting a sample in a CN to obtain the output of a feature map FM and the CN, using the feature map FM in a logic network model LN to obtain the output of a logic network, using the output of the logic network, a label and the output of the CN to obtain a Loss function Loss, and updating the CN;
substituting a sample into the updated CN to obtain a new feature map, a weight matrix corresponding to the full-connection layer and the feature map and the output of the CN, and continuing to update and construct a logic network LN;
forming a logic network to continuously correct the CN, and constructing the logic network by the CN; cycling is performed in this way until the end condition is met;
the step 7 specifically comprises the following steps:
as known from the measurement method of interpretation distance, when the distinguishing part of CNN is sufficiently similar to the shape of its optimal interpretable model, the distinguishing part has good interpretable performance, but its generalization performance tends to decrease, which is mainly due to the fact that two conditions for consistent convergence of performance are difficult to be ensured, as follows:
φ d (P x, f) =0, f (x) is the CNN discrimination part, and P (x) is the best interpretation model;
since this sufficient requirement is difficult to meet, the trade-off problem is always present and its optimal interpretable model P and its optimal discriminating portion are unknown, it may be a viable solution to extract one interpretable model from the discriminating portion and then iteratively reduce the interpretation distance between the two models during the training process when considering the trade-off problem, as follows:
to avoid degrading the generalization performance, the highest probability of p (wX, y) t ) Wherein X is a training sample, w is a parameter set of CNN, y t As the target vector of X, it is possible to obtain:
wherein,
p(y t |w，X)=∫p(y t |f，w，X)∫p(f|y dclm ，w，X)p(y dclm |w，X)dy dclm df (13)
when DCLM is known, y dclm Is an optimal solution thereof, and
p(y* dclm |w，X)=1 (14)
based on this, it is obtained:
∫p(f|y dclm ，w，X)p(y dclm |w，X)dy dclm =p(f|y* dclm ，w，X) (15)
likewise, the parameter set w, y of the training samples X, CNN of known input values t As the target vector of X, fnn is the optimal solution of the CNN network, which can be obtained:
p(y t |w，X)=p(y t |f nn ，w，X)p(f nn |y* dclm ，w，X) (16)
if the parameter set w of CNN and training sample X are given, and the loss function is
The conditional probability distribution function is:
at the same time:
wherein, the xi 1, the xi 2 is the partitioning function; by maximizing the likelihood function of p (w|x, yt), the optimal parameter set w of CNN can be obtained, assuming that w obeys gaussian distribution, it is possible to obtain:
where α (alpha) is a meta-parameter determined by the variance of a selected gaussian distribution, and after converting this equation to a minimization problem, it is possible to obtain:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011249200.2A CN112434790B (en) | 2020-11-10 | 2020-11-10 | Self-interpretation method for distinguishing part of black box problem of convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011249200.2A CN112434790B (en) | 2020-11-10 | 2020-11-10 | Self-interpretation method for distinguishing part of black box problem of convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112434790A CN112434790A (en) | 2021-03-02 |
| CN112434790B true CN112434790B (en) | 2024-03-29 |
Family
ID=74701211
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011249200.2A Active CN112434790B (en) | 2020-11-10 | 2020-11-10 | Self-interpretation method for distinguishing part of black box problem of convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112434790B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113673627B (en) * | 2021-09-02 | 2024-02-13 | 哈尔滨工程大学 | Automatic commodity classification method and system with interpretation |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609634A (en) * | 2017-08-21 | 2018-01-19 | 哈尔滨工程大学 | A kind of convolutional neural networks training method based on the very fast study of enhancing |
| CN108073917A (en) * | 2018-01-24 | 2018-05-25 | 燕山大学 | A kind of face identification method based on convolutional neural networks |
| CN110084215A (en) * | 2019-05-05 | 2019-08-02 | 上海海事大学 | A kind of pedestrian of the twin network model of binaryzation triple recognition methods and system again |
| WO2019180310A1 (en) * | 2018-03-21 | 2019-09-26 | Nokia Technologies Oy | A method, an apparatus and a computer program product for an interpretable neural network representation |
| CN111339935A (en) * | 2020-02-25 | 2020-06-26 | 西安电子科技大学 | Optical remote sensing picture classification method based on interpretable CNN image classification model |
| WO2020140386A1 (en) * | 2019-01-02 | 2020-07-09 | 平安科技（深圳）有限公司 | Textcnn-based knowledge extraction method and apparatus, and computer device and storage medium |
| WO2020172838A1 (en) * | 2019-02-26 | 2020-09-03 | 长沙理工大学 | Image classification method for improvement of auxiliary classifier gan |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108932480B (en) * | 2018-06-08 | 2022-03-15 | 电子科技大学 | Distributed optical fiber sensing signal feature learning and classifying method based on 1D-CNN |
-
2020
- 2020-11-10 CN CN202011249200.2A patent/CN112434790B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609634A (en) * | 2017-08-21 | 2018-01-19 | 哈尔滨工程大学 | A kind of convolutional neural networks training method based on the very fast study of enhancing |
| CN108073917A (en) * | 2018-01-24 | 2018-05-25 | 燕山大学 | A kind of face identification method based on convolutional neural networks |
| WO2019180310A1 (en) * | 2018-03-21 | 2019-09-26 | Nokia Technologies Oy | A method, an apparatus and a computer program product for an interpretable neural network representation |
| WO2020140386A1 (en) * | 2019-01-02 | 2020-07-09 | 平安科技（深圳）有限公司 | Textcnn-based knowledge extraction method and apparatus, and computer device and storage medium |
| WO2020172838A1 (en) * | 2019-02-26 | 2020-09-03 | 长沙理工大学 | Image classification method for improvement of auxiliary classifier gan |
| CN110084215A (en) * | 2019-05-05 | 2019-08-02 | 上海海事大学 | A kind of pedestrian of the twin network model of binaryzation triple recognition methods and system again |
| CN111339935A (en) * | 2020-02-25 | 2020-06-26 | 西安电子科技大学 | Optical remote sensing picture classification method based on interpretable CNN image classification model |
Non-Patent Citations (4)
| Title |
|---|
| 参数池化卷积神经网络图像分类方法;江泽涛;秦嘉奇;张少钦;;电子学报(第09期);全文 * |
| 基于CNN和SVM融合的交通标志识别;王新美;丁爱玲;雷梦宁;康盟;;计算机技术与发展(第06期);全文 * |
| 基于卷积神经网络多层特征提取的目标识别;江彤彤;成金勇;鹿文鹏;;计算机系统应用(第12期);全文 * |
| 基于深度学习算法的图像集识别;张玉冰;于威威;;现代计算机(专业版)(第21期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112434790A (en) | 2021-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hechtlinger et al. | A generalization of convolutional neural networks to graph-structured data | |
| CN107122809B (en) | Neural network feature learning method based on image self-coding | |
| Chen et al. | Unsupervised multi-manifold clustering by learning deep representation | |
| Liang et al. | Clustseg: Clustering for universal segmentation | |
| Wang et al. | Encoder-X: solving unknown coefficients automatically in polynomial fitting by using an autoencoder | |
| Xie et al. | Point clouds learning with attention-based graph convolution networks | |
| Bi et al. | A survey on evolutionary computation for computer vision and image analysis: Past, present, and future trends | |
| CN109325993B (en) | Saliency feature enhanced sampling method based on class octree index | |
| US20190087964A1 (en) | Method and apparatus for parsing and processing three-dimensional cad model | |
| CN107133496B (en) | Gene feature extraction method based on manifold learning and closed-loop deep convolution double-network model | |
| Sun et al. | Recognition of SAR target based on multilayer auto-encoder and SNN | |
| CN110263855B (en) | Method for classifying images by utilizing common-basis capsule projection | |
| Zeng et al. | Convolutional Neural Network Based Multi-feature Fusion for Non-rigid 3D Model Retrieval. | |
| CN112434790B (en) | Self-interpretation method for distinguishing part of black box problem of convolutional neural network | |
| Kamada et al. | An object detection by using adaptive structural learning of deep belief network | |
| Tsai et al. | Machine learning based common radiologist-level pneumonia detection on chest X-rays | |
| Alamgir et al. | An artificial intelligence driven facial emotion recognition system using hybrid deep belief rain optimization | |
| Chauhan et al. | Empirical study on convergence of capsule networks with various hyperparameters | |
| Li et al. | FVGNN: A novel GNN to finger vein recognition from limited training data | |
| Bai et al. | A unified deep learning model for protein structure prediction | |
| Yu et al. | Prototypical network based on Manhattan distance | |
| Lu et al. | Image classification and recognition of rice diseases: a hybrid DBN and particle swarm optimization algorithm | |
| Jiu et al. | Deep context networks for image annotation | |
| An et al. | Logic Rule Guided Attribution with Dynamic Ablation | |
| Yang | Image feature extraction algorithm based on random deep neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |