CN112434790A - Self-interpretation method for convolutional neural network to judge partial black box problem - Google Patents

Self-interpretation method for convolutional neural network to judge partial black box problem Download PDF

Info

Publication number
CN112434790A
CN112434790A CN202011249200.2A CN202011249200A CN112434790A CN 112434790 A CN112434790 A CN 112434790A CN 202011249200 A CN202011249200 A CN 202011249200A CN 112434790 A CN112434790 A CN 112434790A
Authority
CN
China
Prior art keywords
dclm
layer
cnn
network
predicate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011249200.2A
Other languages
Chinese (zh)
Other versions
CN112434790B (en
Inventor
赵金伟
王启舟
邱万力
黑新宏
答龙超
王伟
谢国
胡潇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian University of Technology
Original Assignee
Xian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Technology filed Critical Xian University of Technology
Priority to CN202011249200.2A priority Critical patent/CN112434790B/en
Publication of CN112434790A publication Critical patent/CN112434790A/en
Application granted granted Critical
Publication of CN112434790B publication Critical patent/CN112434790B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G06N5/042—Backward inferencing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a self-interpretation method for judging partial black box problems of a convolutional neural network, which is implemented according to the following steps: step 1, providing an interpretable distance as an interpretable index of a measurement model; step 2, forming a feature extractor by the convolution layer and the pooling layer of the CNN network; step 3, taking the output of the feature extractor obtained in the step 2, namely the feature graph, as the input of the CNN network full connection layer; step 4, in the full connection layer, classifying and labeling the input image samples in the CNN network by using the characteristic diagram in the step 3; step 5, using the full connection layer to form the discrimination part of the CNN network; step 6, constructing a DCLM model of three layers of nodes; and 7, providing a novel game method to carry out game training of the DCLM model and the CNN network constructed in the step 6 so as to improve the interpretability of the CNN network discrimination part. The invention solves the problem of large error of the interpretation method in the prior art.

Description

Self-interpretation method for convolutional neural network to judge partial black box problem
Technical Field
The invention belongs to the technical field of deep learning in the field of computers, and particularly relates to a self-interpretation method for judging partial black box problems of a convolutional neural network.
Background
In recent years, convolutional neural networks (hereinafter CNN), as a representative algorithm for deep learning, have been shown to significantly surpass the capability of human beings in some specific tasks such as computer vision, computer games, and the like. However, the way that neural networks deal with problems is difficult to understand and interpret by people, and due to the ubiquitous difficulty in interpretation, many problems in fairness, privacy, robustness and credibility are caused in application; many researchers are currently trying to explain in different ways around the black box problem of CNN.
The so-called black box problem of CNN is an unexplainable problem whose main difficulty is in its discrimination part, and there are two types of main methods for dealing with such problems: a pre-method and a post-method (Holzinger et al, 2019); since the prior method is generally a transparent modeling method (arietaa et al, 2020), which is difficult to obtain an explanation about a discriminant part, the common technique mainly focuses on the second posterior method.
Early post-hoc approaches obtained a global interpretation of the neural network, mainly by extracting a predictable model. (Craven & Shavlik, 1999; Krishnan et al, 1999; Botz, 2002; Johansson & Niklason, 2009) proposed a method to find a decision tree for interpreting neural networks by maximizing the gain ratio and estimating the fidelity of the current model;
(Craven & Shavlik, 1994; Johansson & Niklasson, 2003; Augasta & Kathirvakumar, 2012; Sebastian et al, 2015; Zilke et al, 2016) proposed a specific rule extraction method to search for the best interpretable rule from a neural network;
in recent years, some feature association methods have been produced, such as a network classification decision decomposition method based on deep taylor decomposition proposed by Montavon et al (Montavon et al, 2017); shrikumar et al (Shrikumar et al, 2016) propose a DeepLIFT algorithm for calculating a multi-layer neural network importance score by interpreting the differences in the outputs of certain reference outputs by their difference in their reference inputs;
in addition, Che et al propose a simple rectification method called interpretable simulation learning, which extracts interpretable simple models by using gradient enhanced trees, and Thiagarajan et al establish tree-like view representations of complex models by hierarchical division of feature spaces to improve interpretability; still other scholars have proposed a rectification method for transforming knowledge from a model set to a single model, Wu et al have proposed a tree regularization method based on knowledge rectification to represent a neural network output feature space based on multi-layer perception;
however, these methods can only solve the unexplained problems of the trained neural network or the trained deep neural network with explicit input features; moreover, due to the existence of data deviation and noise data in the training data set, the traditional machine learning method is difficult to ensure the essential condition that the interpretable performance and the generalization performance of the identification part are consistent and converged; besides, the interpretable performance is a distance between the discriminant part and its optimal interpretable model, and when the discriminant part has better generalization performance, it tends to deviate from its optimal interpretable model, thereby generating an incorrect interpretable model.
Disclosure of Invention
The invention aims to provide a self-interpretation method for the problem of judging partial black boxes of a convolutional neural network, and the self-interpretation method solves the problem of overlarge error of an interpretation method in the prior art.
The invention adopts the technical scheme that a self-explanation method for judging partial black box problems of a convolutional neural network is implemented according to the following steps:
step 1, providing an interpretable distance as an interpretable index of a measurement model;
step 2, forming a feature extractor by the convolution layer and the pooling layer of the CNN network;
step 3, taking the output of the feature extractor obtained in the step 2, namely the feature graph, as the input of the CNN network full connection layer;
step 4, in the full connection layer, classifying and labeling the input image samples in the CNN network by using the characteristic diagram in the step 3;
step 5, using the full connection layer to form the discrimination part of the CNN network;
step 6, constructing a DCLM model of three layers of nodes;
and 7, providing a novel game method to carry out game training of the DCLM model and the CNN network constructed in the step 6 so as to improve the interpretability of the CNN network discrimination part.
The present invention is also characterized in that,
in the step 1, under the same input data set, the similarity is measured by the variance of the difference between the output of the original CNN network model and the output of the constructed DCLM model, and the measurement result is initially named as an interpretation distance, and the shape of the judging part of the CNN network is closer to the shape of the optimal interpretable model when the interpretation distance is smaller, so that the interpretable performance of the interpreting part is better;
assuming that Q is a compact metric space, Z is a sample set, v is a Borel metric in the metric space Q, such as a Lebesgue metric or an edge metric, in a squared integrable function space to the metric space Q
Figure BDA0002771041850000031
In (f), the interpretation distance phi between the discriminating part f (x) and the model P (x) of the best interpretation thereofd(P x, f) is expressed as:
Figure BDA0002771041850000032
wherein
Figure BDA0002771041850000033
The step 6 is as follows:
the three-layer network structure of the DCLM is as follows: the characteristic predicate layer as a first layer is composed of characteristic predicate nodes, wherein each characteristic predicate node has a characteristic predicate function Z (Γ) used for representing whether a neuron of a discriminant part of a first fully-connected layer in the CNN network has the capability of capturing characteristics, and the characteristic predicate function Z (Γ) is represented as follows:
Figure BDA0002771041850000041
where i ∈ {1, 2,..., k }, τiIs the ith feature map in Γ, and wi is its corresponding weight vector in the first fully-connected layer, called the feature capture layer, where x represents the convolution.
In the step 6, the decision result predicate layer is the bottommost layer, and each result predicate node has a result predicate function which represents whether the output neuron of the judgment part is greater than 0; the corresponding function is expressed as follows:
Figure BDA0002771041850000042
all the nodes of the characteristic predicate layer and the result predicate layer in the step 6 are connected with one or more disjunct nodes of the middle layer, wherein the disjunct layers are called disjunct layers and represent true or false true value conditions; each disjunct represents a disjunct relationship of all the characteristic predicate layer nodes and the result predicate layer nodes, and the disjunct relationship is characterized by a disjunct normal form; if a predicate layer node is connected with a disjunct layer node by a boundary of true or false, and the predicate function of the predicate layer node is followed by a non-operation in the disjunct operation, a potential function of a disjunct normal form is obtained by a Lukasiewicz method as follows:
φc(y)＝min(1，T(Γ，y)) (3)
in the formula:
Figure BDA0002771041850000051
wherein N is the number of nodes of the feature predicate layer; gamma is characteristic diagram, Zj() Is a feature predicate layer function, and D () is a result layer function; if logical network parameter ajIf the logic network parameter a is equal to 1, the boundary between the predicate node and the disjunct node is falsejA value of 0 indicates a true boundary.
In step 6, the DCLM model is set to include a disjunctive relationship:
Figure BDA0002771041850000052
is equal to
Figure BDA0002771041850000053
Wherein Zj() Is a feature predicate layer function, and j is an element (1, 4), and y is the neuron output of the CNN network; the potential function of this disjunctive normal form is:
Figure BDA0002771041850000054
wherein, a1＝a41 and a2＝a3＝0；
The group truth conditional probability distribution of the DCLM is as follows:
Figure BDA0002771041850000055
where G is the number of all ground functions, the scoring function xi in the formula denotes as follows:
Figure BDA0002771041850000056
output value y of neuron output layer of CNN networkdclmComprises the following steps:
ydclm＝(ydclm，1，ydclm，2，...，ydclm，G) (7)
wherein λ isiFor the weight of the ith group function, by maximizing the likelihood function, the optimal output value in the DCLM model can be obtained: y isdclmLogical network parameter ajAnd λi(ii) a The process of maximizing the likelihood function is shown as follows:
Figure BDA0002771041850000061
in the formula, phiciAs a function of potential, aiIs a logical network parameter;
for the DCLM, firstly, a logic network is constructed, and then a maximum a posteriori MAP algorithm is adopted for model extraction, which specifically comprises the following steps:
inputting a feature map, a feature capture matrix, a label corresponding to input and an input number;
initializing initial values of a true pseudo edge set and y-dclm, and setting the number of initialization iterations to be 1; entering an outer loop body to perform the following operations on all characteristic graphs:
solving cosine similarity by using the current characteristic diagram and a weight matrix corresponding to the current characteristic diagram, storing the cosine similarity, and calculating and storing a product of a modulus of the current characteristic diagram and the weight matrix;
traversing each dimension of the output vector, solving the latest edge value in the analytic expression corresponding to the current dimension by using three parameters of a similarity result obtained from the upper part of the output vector, a product of models and corresponding output y-dclm, solving a union set by using an existing edge value set and the currently solved edge value, and increasing the iteration count and adding 1;
and when the iteration times are equal to the initial input number, the algorithm ending condition is met.
Step 7, the game iteration optimization idea is as follows:
inputting: inputting a data set and a target output;
initialization: the logic network is named as LN and CNN network is named as CN;
firstly: putting the sample in CN to obtain a characteristic diagram FM and the output of CN, obtaining the output of the logic network in a logic network model LN by using the characteristic diagram FM, calculating a Loss function Loss by using the output of the logic network, the output of the label and the output of CN, and updating CN;
secondly, substituting the updated CN into the sample to obtain a new characteristic diagram, a weight matrix corresponding to the full connection layer and the characteristic diagram and the output of the CN, and continuously updating and constructing a logic network LN;
forming a process that the CN is continuously corrected and the CN constructs the logic network; and circulating in such a way until the end condition is met and ending.
Step 7 is specifically as follows:
from the measurement of interpretation distance, when the discriminating portion of CNN is sufficiently similar to the shape of its optimal interpretable model, the discriminating portion has good interpretable performance, but its generalization performance tends to be degraded, which is mainly due to the fact that two sufficient conditions for consistent convergence of performance are difficult to guarantee, as shown in the following formula:
φd(P, f) ═ 0, f (x) is the CNN discriminant, P (x) is the best-explained model;
since this sufficient requirement is difficult to satisfy, the equalization problem always exists, and its optimal interpretable model P and its optimized discriminative part are unknown, it may be a feasible solution to automatically extract one interpretable model from the discriminative part and then iteratively reduce the interpretation distance between the two models in the training process when considering the equalization problem, as follows:
to avoid degrading generalization performance, the maximum probability of p (w | X, y) should be guaranteedt) Where X is a training sample, w is a parameter set for CNN, ytA target vector of X, one obtains:
Figure BDA0002771041850000081
wherein the content of the first and second substances,
Figure BDA0002771041850000082
when DCLM is known, ydclmIs an optimal solution thereof, and
p(y*dclm|w，X)＝1 (14)
based on this, we obtain:
Figure BDA0002771041850000083
likewise, the parameter set w, y of the training sample X, CNN is known as the input valuetFor the target vector of X, fnn is the optimal solution for CNN net, which can be:
p(yt|w，X)＝p(yt|fnn，w，X)p(fnn|y*dclm，w，X) (16)
if the parameter set w and the training sample X of CNN are given, and the loss function is
Figure BDA0002771041850000084
The conditional probability distribution function is then:
Figure BDA0002771041850000085
simultaneously:
Figure BDA0002771041850000086
wherein xi 1, xi 2 are scoring functions; by maximizing the likelihood function of p (w | X, yt), the optimal set w of parameters for CNN can be obtained, assuming w obeys a gaussian distribution:
Figure BDA0002771041850000087
where α is an element parameter determined by the variance of the selected gaussian distribution, this equation is transformed into a minimization problem, which is obtained:
Figure BDA0002771041850000091
the self-explanation method for the black box problem of the judging part of the convolutional neural network has the advantages that on the premise of not damaging the generalization performance of the convolutional neural network, the causal relationship of the CNN without the prior structure is actively extracted, the explanation is finally obtained through the causal relationship, and meanwhile, the interpretable performance of the judging part of the CNN can be improved. In the process, the invention provides an interpretable model, namely a depth perception learning model (DCLM), to express the causal relationship of the judging part, and provides a greedy algorithm to automatically extract the DCLM from the judging part by solving the problem of the maximum satisfiability of the judging part. A new game method is provided, the distance between the two models is reduced through iteration, the two models are corrected, and the interpretable performance of the judgment part is improved on the premise of not greatly reducing the generalization performance of the judgment part. Meanwhile, the generalization performance of the DCLM is also improved. An interpretable distance is presented for evaluating and measuring the distance between the discriminant and the interpretable model on the unexplainable problem.
Drawings
FIG. 1 is a concrete structural representation of a three-layer network structure of a deep perceptual learning model DCLM;
FIG. 2 is a disjunctive relational representation in the deep perceptual learning model DCLM;
FIG. 3 is a graph of the accuracy of DCLMs and CNN-DCLMs for epochs during a game;
FIG. 4 is an explanatory distance of DCLMs from CNNs and CNN-DCLMs;
FIG. 5 is the information entropy of DCLM;
FIG. 6 shows the accuracy of CNN and CNN-DCLM.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The invention relates to a self-explanation method for judging partial black box problems of a convolutional neural network, which is implemented according to the following steps:
step 1, providing an interpretable distance as an interpretable index of a measurement model;
in the step 1, under the same input data set, the similarity is measured by the variance of the difference between the output of the original CNN network model and the output of the constructed DCLM model, and the measurement result is initially named as an interpretation distance, and the shape of the judging part of the CNN network is closer to the shape of the optimal interpretable model when the interpretation distance is smaller, so that the interpretable performance of the interpreting part is better;
assuming that Q is a compact metric space, Z is a sample set, v is a Borel metric in the metric space Q, such as a Lebesgue metric or an edge metric, in a squared integrable function space to the metric space Q
Figure BDA0002771041850000101
In (f), the interpretation distance phi between the discriminating part f (x) and the model P (x) of the best interpretation thereofd(P x, f) is expressed as:
Figure BDA0002771041850000102
wherein
Figure BDA0002771041850000103
The step 6 is as follows:
as shown in fig. 1, when constructing the DCLM model, since the relationship between the features captured by the weight matrix of the first layer and the output vectors thereof is an internal relationship of the CNN network structure, which is neither related to the input data nor to the abstraction degree of the features, it can be considered that the distinguishing part can be better interpreted by a logical relationship, and the representation of the logical relationship is represented by a DCLM three-layer network structure, which is: the characteristic predicate layer as a first layer is composed of characteristic predicate nodes, wherein each characteristic predicate node has a characteristic predicate function Z (Γ) used for representing whether a neuron of a discriminant part of a first fully-connected layer in the CNN network has the capability of capturing characteristics, and the characteristic predicate function Z (Γ) is represented as follows:
Figure BDA0002771041850000111
where i ∈ {1, 2,..., k }, τiIs the ith characteristic diagram of Γ, wiIs its corresponding weight vector in the first fully-connected layer, called the feature capture layer, where x represents the convolution.
Step 2, forming a feature extractor by the convolution layer and the pooling layer of the CNN network;
step 3, taking the output of the feature extractor obtained in the step 2, namely the feature graph, as the input of the CNN network full connection layer;
step 4, in the full connection layer, classifying and labeling the input image samples in the CNN network by using the characteristic diagram in the step 3;
step 5, using the full connection layer to form the discrimination part of the CNN network;
step 6, constructing a DCLM model of three layers of nodes;
in the step 6, the decision result predicate layer is the bottommost layer, and each result predicate node has a result predicate function which represents whether the output neuron of the judgment part is greater than 0; the corresponding function is expressed as follows:
Figure BDA0002771041850000112
all the nodes of the characteristic predicate layer and the result predicate layer in the step 6 are connected with one or more disjunct nodes of the middle layer, wherein the disjunct layers are called disjunct layers and represent true or false true value conditions; each disjunct represents a disjunct relationship of all the characteristic predicate layer nodes and the result predicate layer nodes, and the disjunct relationship is characterized by a disjunct normal form; if a predicate layer node is connected with a disjunct layer node by a boundary of true or false, and the predicate function of the predicate layer node is followed by a non-operation in the disjunct operation, a potential function of a disjunct normal form is obtained by a Lukasiewicz method as follows:
φc(y)＝min(1，T(Γ，y)) (3)
in the formula:
Figure BDA0002771041850000121
wherein N is the number of nodes of the feature predicate layer; gamma is characteristic diagram, Zj() Is a feature predicate layer function, and D () is a result layer function; if logical network parameter ajIf the logic network parameter a is equal to 1, the boundary between the predicate node and the disjunct node is falsejA value of 0 indicates a true boundary.
As shown in fig. 2, the DCLM model set in step 6 includes an disjunctive relationship:
Figure BDA0002771041850000122
is equal to
Figure BDA0002771041850000123
Wherein Zj() Is a feature predicate layer function, and j is an element (1, 4), and y is the neuron output of the CNN network; the potential function of this disjunctive normal form is:
Figure BDA0002771041850000124
wherein, a1＝a41 and a2＝a3＝0；
The group truth conditional probability distribution of the DCLM is as follows:
Figure BDA0002771041850000125
where G is the number of all ground functions, the scoring function xi in the formula denotes as follows:
Figure BDA0002771041850000131
output value y of neuron output layer of CNN networkdclmComprises the following steps:
ydclm＝(ydclm，1，ydclm，2，...，ydclm，G) (7)
wherein λ isiFor the weight of the ith group function, by maximizing the likelihood function, the optimal output value in the DCLM model can be obtained: y isdclmLogical network parameter ajAnd λi(ii) a The process of maximizing the likelihood function is shown as follows:
Figure BDA0002771041850000132
in the formula, phiciAs a function of potential, aiIs a logical network parameter;
for the DCLM, firstly, a logic network is constructed, and then a maximum a posteriori MAP algorithm is adopted for model extraction, which specifically comprises the following steps:
inputting a feature map, a feature capture matrix, a label corresponding to input and an input number;
initializing initial values of a true pseudo edge set and y-dclm, and setting the number of initialization iterations to be 1; entering an outer loop body to perform the following operations on all characteristic graphs:
solving cosine similarity by using the current characteristic diagram and a weight matrix corresponding to the current characteristic diagram, storing the cosine similarity, and calculating and storing a product of a modulus of the current characteristic diagram and the weight matrix;
traversing each dimension of the output vector, solving the latest edge value in the analytic expression corresponding to the current dimension by using three parameters of a similarity result obtained from the upper part of the output vector, a product of models and corresponding output y-dclm, solving a union set by using an existing edge value set and the currently solved edge value, and increasing the iteration count and adding 1;
and when the iteration times are equal to the initial input number, the algorithm ending condition is met.
Step 7, the game iteration optimization idea is as follows:
inputting: inputting a data set and a target output;
initialization: the logic network is named as LN and CNN network is named as CN;
firstly: putting the sample in CN to obtain a characteristic diagram FM and the output of CN, obtaining the output of the logic network in a logic network model LN by using the characteristic diagram FM, calculating a Loss function Loss by using the output of the logic network, the output of the label and the output of CN, and updating CN;
secondly, substituting the updated CN into the sample to obtain a new characteristic diagram, a weight matrix corresponding to the full connection layer and the characteristic diagram and the output of the CN, and continuously updating and constructing a logic network LN;
forming a process that the CN is continuously corrected and the CN constructs the logic network; and circulating in such a way until the end condition is met and ending.
And 7, providing a novel game method to carry out game training of the DCLM model and the CNN network constructed in the step 6 so as to improve the interpretability of the CNN network discrimination part.
Step 7 is specifically as follows:
from the measurement of interpretation distance, when the discriminating portion of CNN is sufficiently similar to the shape of its optimal interpretable model, the discriminating portion has good interpretable performance, but its generalization performance tends to be degraded, which is mainly due to the fact that two sufficient conditions for consistent convergence of performance are difficult to guarantee, as shown in the following formula:
φd(P, f) ═ 0, f (x) is the CNN discriminant, P (x) is the best-explained model;
since this sufficient requirement is difficult to satisfy, the trade-off problem always exists, and its optimum interpretable model P and its decision of optimizationWhile not known, to prove this conclusion, one can first focus on only one neuron of the CNN network, because each input channel f (x) of the neuron can be regarded as a kernel function K (x, w), where w is a weight vector containing an offset in the neuron, which can span the Hilbert space, Q is a compact metric space, Z is a set of samples, and v is a Borel metric in the metric space Q, such as the Lebesgue metric or the edge metric, in the squared integrable function space to the metric space Q
Figure BDA0002771041850000151
The method comprises the following steps:
Figure BDA0002771041850000152
HKis one is arranged at
Figure BDA0002771041850000153
Linear function of (c), however, due to data bias and noisy data in the training dataset, its optimal interpretable model is in most engineering applications
Figure BDA0002771041850000154
The above is often a non-linear function because the continuous linear functional is integrated in
Figure BDA0002771041850000155
Is not dense. Therefore, the sufficient requirements are difficult to satisfy, and the trade-off problem always exists, which also exists in the discriminating part of CNN; however, its optimal interpretable model P and its optimized discriminative part are unknown, so, when considering the trade-off problem, extracting one interpretable model from the discriminative part and then iteratively reducing the interpretation distance between the two models during the training process may be a feasible solution, as follows: the learning algorithm for CNN can improve its performance interpretability if the minimum loss function for CNN can guarantee minimum interpretability performance. If not, generalization performance and interpretability performanceThe trade-off problem between (a) and (b) will no longer exist. To demonstrate the existence of a problem, one may focus on only one neuron of CNN; if one input channel f (x) of a neuron is considered as a kernel function K (x, w) (w is a weight vector containing a bias of the neuron), it will open up a kernel hilbert space for the neuron.
Figure BDA0002771041850000156
HKCan be regarded as one in
Figure BDA0002771041850000157
The linear function set above is the solution space of the neuron. Based on the following introduction
Figure BDA0002771041850000161
The essential conditions for consistent convergence of generalization performance and interpretable performance are discussed in the publication:
lemma 1, a continuous linear functional set on a separable hilbert space is not dense on a square integrable function space;
lemma 2, a set of continuous non-linear functionals of separable hilbert space is dense everywhere in the L2 theorem.
The discussion is as follows:
at HKWhen the optimal input channel f is approximate to a linear function, the optimal interpretable model P is in HKWithout any approximation of a linear function or in the absence, according to theorem 1, the conventional training process cannot guarantee that f (x) approximates P (x). From lemma 2, it can be found that convergence cannot be achieved until P x (x) approximates f x (x). Here, if the approximation is defined as the similarity of the curve shapes of the f (x) and P (x) functions, then the essential condition for the consistent convergence of the two performances is that φ (x) is thed(P, f) ═ 0. The sufficient requirements are still true for the discriminating part of CNN. However, in most engineering applications, this condition is difficult to guarantee in most cases due to the data bias and noise data in the training data set. There is always a trade-off between the two properties of the discriminating part.
From the above conclusions, to fully solve the trade-offsProblem, should reduce phid(P, f) ═ 0(f (x) is the CNN discriminant, and P (x) is the best-explained model. However, P and f are unknown, and therefore, in the training process, an interpretable model P (x) is extracted from the discriminative portion f (x), and then phi is iteratively reducedd(P, f) is a more convenient method.
To avoid degrading generalization performance, the maximum probability of p (w | X, y) should be guaranteedt) Where X is a training sample, w is a parameter set for CNN, ytA target vector of X, one obtains:
Figure BDA0002771041850000162
wherein the content of the first and second substances,
Figure BDA0002771041850000171
when DCLM is known, ydclmIs an optimal solution thereof, and
p(y*dclm|w，X)＝1 (14)
based on this, we obtain:
Figure BDA0002771041850000172
likewise, the parameter set w, y of the training sample X, CNN is known as the input valuetFor the target vector of X, fnn is the optimal solution for CNN net, which can be:
p(yt|w，X)＝p(yt|fnn，w，X)p(fnn|y*dclm，w，X) (16)
if the parameter set w and the training sample X of CNN are given, and the loss function is
Figure BDA0002771041850000173
The conditional probability distribution function is then:
Figure BDA0002771041850000174
simultaneously:
Figure BDA0002771041850000175
wherein xi 1, xi 2 are scoring functions; by maximizing the likelihood function of p (w | X, yt), the optimal set w of parameters for CNN can be obtained, assuming w obeys a gaussian distribution:
Figure BDA0002771041850000176
where α is an element parameter determined by the variance of the selected gaussian distribution, this equation is transformed into a minimization problem, which is obtained:
Figure BDA0002771041850000181
and (3) experimental verification:
experiments were designed in the present invention to verify the effectiveness of the invention. The first experiment verifies whether the self-explanatory method can improve the interpretable performance of CNN without reducing the generalization performance of CNN. The second experiment verifies whether the method can tend to stabilize and converge in the game process.
The structures of CNN3 (including 3 convolutional layers, 3 max pooling layers, 3 full-link layers (FCLs), and 1 output layer), CNN5 (including 5 convolutional layers, 5 max pooling layers, 3 full-link layers, and 1 output layer), and CNN8 (including 8 convolutional layers, 8 max pooling layers, 3 full-link layers, and 1 output layer) were used in the experiment. The final structures obtained by training the CNNs by the method are named as CNN3-DCLM, CNN5-DCLM and CNN8-DCLM respectively. All experiments were performed using the mnst (leconet al., 1998), fashinon mnst (Zalando,2017) and emist (cohenetal., 2017) reference data sets. All algorithms were implemented in Python using a pytorech library (paszkey et al, 2019), and the experiments were run on a server equipped with an Intel Xeon 4110(2.1GHz) processor, 20GBRAM, and nvidia telsa t 4.
TABLE 1 Classification precision and interpretation distance of three game methods on DCLM-CNN
Figure BDA0002771041850000191
Experiment 1 Performance validation of the methods herein on CNN
Experiments also compared Soft Decision Trees (SDT). (Hinton,2017) is based on trained CNN3, CNN5 and CNN8, namely CNN3-sdt, CNN5-sdt and CNN 8-sdt. These baseline methods also included SDT (frost & (Hinton,2017) based on trained CNN3-dclm, CNN5-dclm and CNN8-dclm, CNN3-SDT, CNN5-SDT and CNN8-SDT, respectively, all algorithms had these indicators on all test datasets as shown in table 1.
As can be seen from table 1, the accuracy of CNN-DCLMs was higher than both the SDT and DCLM interpretable models and about 1.4 percentage points lower than that of CNN on all baseline datasets. It is noteworthy, however, that with the exception of CNN3-SDT on the Emnist dataset, the interpretation distance of all CNN-DCLMs on most datasets was about 5 percentage points lower than that of most SDT from their CNNs, which may prove that the self-interpretation method can improve the interpretability of the resolved portion of CNN without significantly reducing the generalization performance of CNN;
on the Mnist dataset, the accuracy of DCLMs was only 0.7 percentage points lower than that of SDT on CNN5-SDT and CNN8-SDT, except CNN 3-SDT. Through the Emnist data set and the FashinonMnist data set, the accuracy of DCLMs can be found to be 2 percent lower than that of SDT CNN 3-SDT; it can also be found that the accuracy of DCLMs of CNN5-SDT and CNN8-SDT is about 4.3 percent higher than that of SDTs in the two data sets. This is mainly because the feature maps output by the feature extraction parts of CNN5 and CNN8 are more abstract than CNN3 for the Emnist dataset and the fast on mnsist dataset. The abstract feature graph can easily represent the property of the predicate that the DCLM is distortion-free, so it does not hinder or affect the generation of the DCLM;
experiment 2 Convergence test of the proposed method
The convergence of the method is verified through experiments: CNN3, CNN5, CNN8 were compared with CNN3-dclm, CNN5-dclm, CNN8-dclm, respectively. 25 epochs are required for each training and all results are measured at each epoch, as shown in fig. 3, 4, 6, 5. Each graph includes 9 secondary graphs. The three subgraphs in the left column are experiments with the mnst dataset. The three subgraphs in the middle column are the FashinnMnist dataset and the three subgraphs in the right column are the Emnist dataset. FIG. 3 shows the accuracy of DCLMs and CNN-DCLMs for each epoch during the gaming process. From these data it is evident that the accuracy of dclm and of these cnn-dclm are steadily increasing at an early stage. In the next stage, their accuracy tends to be stable. This indicates that the gaming method does not affect the enhancement of dclm and cnn generalization performance;
FIG. 4 shows the interpreted distances of DCLMs from CNNs and CNN-DCLMs. As can be seen from these figures, most of the time, the interpretation distance of CNN-DCLMs not participating in the game is greater than CNN-DCLMs, especially at the end of training. The result shows that the game method can effectively improve the interpretable performance of the CNN-DCLMs.
It can be seen from fig. 5 that the dclm of CNN3, CNN5, CNN8 all have stable entropy values at the end of the game. On the Mnist data set, the entropy converges to 120-145 finally. The entropy values eventually converge between 125 and 136 on the FashinonMnist dataset. On the Emnist dataset, the entropy eventually converges between 100 and 120. The result shows that the game algorithm can ensure that the DCLM converges to a stable state
FIG. 6 shows the accuracy of epochs CNN and CNN-DCLMs during a game; from these plots, it can be seen that the accuracy of CNNs and CNN-DCLMs steadily increased in the early stages, while the accuracy of CNN-DCLMs was lower than CNNs. But in the last stage its accuracy tends to be stable and consistent. The main reason is that at an early stage, CNN-DCLMs must reduce their generalization performance and increase their interpretability performance in terms of the tradeoff between generalization performance and interpretability performance. The proposed gaming method can effectively narrow the gap between the two performances. This indicates that the method is effective for the trade-off problem
From the subgraphs of fig. 3 and 4, it can also be seen that one DCLM model is available on each epoch. After 15 epochs, their interpretation distance tends to converge. This phenomenon shows that the gaming process can reduce the distance, and the training process of the recognition part of the network neural network can be interpreted in real time by a dclm model after 15 epochs thereof.

Claims (8)

1. A self-interpretation method for judging partial black box problems of a convolutional neural network is characterized by comprising the following steps:
step 1, providing an interpretable distance as an interpretable index of a measurement model;
step 2, forming a feature extractor by the convolution layer and the pooling layer of the CNN network;
step 3, taking the output of the feature extractor obtained in the step 2, namely the feature graph, as the input of the CNN network full connection layer;
step 4, in the full connection layer, classifying and labeling the input image samples in the CNN network by using the characteristic diagram in the step 3;
step 5, using the full connection layer to form the discrimination part of the CNN network;
step 6, constructing a DCLM model of three layers of nodes;
and 7, providing a novel game method to carry out game training of the DCLM model and the CNN network constructed in the step 6 so as to improve the interpretability of the CNN network discrimination part.
2. The self-interpretation method for black box problem of the discriminant part of the convolutional neural network as claimed in claim 1, wherein in the step 1, it is assumed that under the same input data set, the similarity is measured by the variance of the difference between the outputs of the original CNN network model and the constructed DCLM model, and the measurement result is primarily named as interpretation distance, the smaller the interpretation distance, the closer the shape of the discriminant part of the CNN network is to the shape of the optimal interpretable model, so the better the interpretable performance of the interpretation part is;
assuming that Q is a compact metric space, Z is a sample set, v is a Borel metric in the metric space Q, such as a Lebesgue metric or an edge metric, in a squared integrable function space to the metric space Q
Figure FDA0002771041840000011
In (f), the interpretation distance phi between the discriminating part f (x) and the model P (x) of the best interpretation thereofd(P x, f) is expressed as:
φd(P*，f)＝∫z(f(x)-P*(x)-μP*(f))2dv (9)
wherein muP*(f)＝∫z(f(x)-P*(x))dv (10)。
3. The self-interpretation method for the convolutional neural network to discriminate the partial black box problem as claimed in claim 2, wherein the step 6 is as follows:
the three-layer network structure of the DCLM is as follows: the characteristic predicate layer as a first layer is composed of characteristic predicate nodes, wherein each characteristic predicate node has a characteristic predicate function Z (Γ) used for representing whether a neuron of a discriminant part of a first fully-connected layer in the CNN network has the capability of capturing characteristics, and the characteristic predicate function Z (Γ) is represented as follows:
Figure FDA0002771041840000021
where i ∈ {1, 2,..., k }, τiIs the ith characteristic diagram of Γ, wiIs its corresponding weight vector in the first fully-connected layer, called the feature capture layer, where x represents the convolution.
4. The self-interpretation method for the black box problem of the discriminant part of the convolutional neural network as claimed in claim 3, wherein the decision result predicate layer in step 6 is the bottom layer, and each result predicate node has a result predicate function which indicates whether the output neuron of the discriminant part is greater than 0; the corresponding function is expressed as follows:
Figure FDA0002771041840000031
5. the self-interpretation method for the convolutional neural network to distinguish the partial black box problem as claimed in claim 4, wherein all the nodes of the feature predicate layer and the result predicate layer in the step 6 are connected with one or more extraction nodes of the middle layer, and the layer is called an extraction layer and represents the true or false true value condition; each disjunct represents a disjunct relationship of all the characteristic predicate layer nodes and the result predicate layer nodes, and the disjunct relationship is characterized by a disjunct normal form; if a predicate layer node is connected with a disjunct layer node by a boundary of true or false, and the predicate function of the predicate layer node is followed by a non-operation in the disjunct operation, a potential function of a disjunct normal form is obtained by a Lukasiewicz method as follows:
φc(y)＝min(1，T(Γ，y)) (3)
in the formula:
Figure FDA0002771041840000032
wherein N is the number of nodes of the feature predicate layer; gamma is characteristic diagram, Zj() Is a feature predicate layer function, and D () is a result layer function; if logical network parameter ajIf the logic network parameter a is equal to 1, the boundary between the predicate node and the disjunct node is falsejA value of 0 indicates a true boundary.
6. The method as claimed in claim 5, wherein the step 6 of setting the DCLM model comprises an extraction relationship:
Figure FDA0002771041840000033
is equal to
Figure FDA0002771041840000034
Wherein Zj() Is a feature predicate layer function, and j is an element (1, 4), and y is the neuron output of the CNN network; the potential function of this disjunctive normal form is:
Figure FDA0002771041840000041
wherein, a1＝a41 and a2＝a3＝0；
The group truth conditional probability distribution of the DCLM is as follows:
Figure FDA0002771041840000042
where G is the number of all ground functions, the scoring function xi in the formula denotes as follows:
Figure FDA0002771041840000043
output value y of neuron output layer of CNN networkdclmComprises the following steps:
ydclm＝(ydclm，1，ydclm，2，...，ydclm，G) (7)
wherein λ isiFor the weight of the ith group function, by maximizing the likelihood function, the optimal output value in the DCLM model can be obtained: y isdclmLogical network parameter ajAnd λi(ii) a The process of maximizing the likelihood function is shown as follows:
Figure FDA0002771041840000044
in the formula, phiciAs a function of potential, aiIs a logical network parameter;
for the DCLM, firstly, a logic network is constructed, and then a maximum a posteriori MAP algorithm is adopted for model extraction, which specifically comprises the following steps:
inputting a feature map, a feature capture matrix, a label corresponding to input and an input number;
initializing initial values of a true pseudo edge set and y-dclm, and setting the number of initialization iterations to be 1; entering an outer loop body to perform the following operations on all characteristic graphs:
solving cosine similarity by using the current characteristic diagram and a weight matrix corresponding to the current characteristic diagram, storing the cosine similarity, and calculating and storing a product of a modulus of the current characteristic diagram and the weight matrix;
traversing each dimension of the output vector, solving the latest edge value in the analytic expression corresponding to the current dimension by using three parameters of a similarity result obtained from the upper part of the output vector, a product of models and corresponding output y-dclm, solving a union set by using an existing edge value set and the currently solved edge value, and increasing the iteration count and adding 1;
and when the iteration times are equal to the initial input number, the algorithm ending condition is met.
7. The self-interpretation method for the convolutional neural network discriminant partial black box problem as claimed in claim 6, wherein the step 7 game iterative optimization idea is as follows:
inputting: inputting a data set and a target output;
initialization: the logic network is named as LN and CNN network is named as CN;
firstly: putting the sample in CN to obtain a characteristic diagram FM and the output of CN, obtaining the output of the logic network in a logic network model LN by using the characteristic diagram FM, calculating a Loss function Loss by using the output of the logic network, the output of the label and the output of CN, and updating CN;
secondly, substituting the updated CN into the sample to obtain a new characteristic diagram, a weight matrix corresponding to the full connection layer and the characteristic diagram and the output of the CN, and continuously updating and constructing a logic network LN;
forming a process that the CN is continuously corrected and the CN constructs the logic network; and circulating in such a way until the end condition is met and ending.
8. The self-interpretation method for the convolutional neural network to discriminate the partial black box problem as claimed in claim 7, wherein the step 7 is as follows:
from the measurement of interpretation distance, when the discriminating portion of CNN is sufficiently similar to the shape of its optimal interpretable model, the discriminating portion has good interpretable performance, but its generalization performance tends to be degraded, which is mainly due to the fact that two sufficient conditions for consistent convergence of performance are difficult to guarantee, as shown in the following formula:
φd(P, f) ═ 0, f (x) is the CNN discriminant, P (x) is the best-explained model;
since this sufficient requirement is difficult to satisfy, the trade-off problem always exists, and its optimal interpretable model P and its optimized discriminant are unknown, therefore, when considering the trade-off problem, extracting one interpretable model from the discriminant in the training process, and then iteratively reducing the interpretation distance between the two models may be a feasible solution, as follows:
to avoid degrading generalization performance, the maximum probability of p (w | X, y) should be guaranteedt) Where X is a training sample, w is a parameter set for CNN, ytA target vector of X, one obtains:
Figure FDA0002771041840000061
wherein the content of the first and second substances,
p(yt|w，X)＝∫p(yt|f，w，X)∫p(f|ydclm，w，X)p(ydclm|w，X)dydclmdf (13)
when the DCLM is known as such,y*dclmis an optimal solution thereof, and
p(y*dclm|w，X)＝1 (14)
based on this, we obtain:
∫p(f|ydclm，w，X)p(ydclm|w，X)dydclm＝p(f|y*dclm，w，X) (15)
likewise, the parameter set w, y of the training sample X, CNN is known as the input valuetFor the target vector of X, fnn is the optimal solution for CNN net, which can be:
p(yt|w，X)＝p(yt|fnn，w，X)p(fnn|y*dclm，w，X) (16)
if the parameter set w and the training sample X of CNN are given, and the loss function is
Figure FDA0002771041840000071
The conditional probability distribution function is then:
Figure FDA0002771041840000072
simultaneously:
Figure FDA0002771041840000073
wherein xi 1, xi 2 are scoring functions; by maximizing the likelihood function of p (w | X, yt), the optimal set w of parameters for CNN can be obtained, assuming w obeys a gaussian distribution:
Figure FDA0002771041840000074
where α (alpha) is an element parameter determined by the variance of a selected gaussian distribution, this equation can be converted to a minimization problem:
Figure FDA0002771041840000075
CN202011249200.2A 2020-11-10 2020-11-10 Self-interpretation method for distinguishing part of black box problem of convolutional neural network Active CN112434790B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011249200.2A CN112434790B (en) 2020-11-10 2020-11-10 Self-interpretation method for distinguishing part of black box problem of convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011249200.2A CN112434790B (en) 2020-11-10 2020-11-10 Self-interpretation method for distinguishing part of black box problem of convolutional neural network

Publications (2)

Publication Number Publication Date
CN112434790A true CN112434790A (en) 2021-03-02
CN112434790B CN112434790B (en) 2024-03-29

Family

ID=74701211

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011249200.2A Active CN112434790B (en) 2020-11-10 2020-11-10 Self-interpretation method for distinguishing part of black box problem of convolutional neural network

Country Status (1)

Country Link
CN (1) CN112434790B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113673627A (en) * 2021-09-02 2021-11-19 哈尔滨工程大学 Interpretive automatic commodity classification method and system

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107609634A (en) * 2017-08-21 2018-01-19 哈尔滨工程大学 A kind of convolutional neural networks training method based on the very fast study of enhancing
CN108073917A (en) * 2018-01-24 2018-05-25 燕山大学 A kind of face identification method based on convolutional neural networks
US20180357542A1 (en) * 2018-06-08 2018-12-13 University Of Electronic Science And Technology Of China 1D-CNN-Based Distributed Optical Fiber Sensing Signal Feature Learning and Classification Method
CN110084215A (en) * 2019-05-05 2019-08-02 上海海事大学 A kind of pedestrian of the twin network model of binaryzation triple recognition methods and system again
WO2019180310A1 (en) * 2018-03-21 2019-09-26 Nokia Technologies Oy A method, an apparatus and a computer program product for an interpretable neural network representation
CN111339935A (en) * 2020-02-25 2020-06-26 西安电子科技大学 Optical remote sensing picture classification method based on interpretable CNN image classification model
WO2020140386A1 (en) * 2019-01-02 2020-07-09 平安科技（深圳）有限公司 Textcnn-based knowledge extraction method and apparatus, and computer device and storage medium
WO2020172838A1 (en) * 2019-02-26 2020-09-03 长沙理工大学 Image classification method for improvement of auxiliary classifier gan

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107609634A (en) * 2017-08-21 2018-01-19 哈尔滨工程大学 A kind of convolutional neural networks training method based on the very fast study of enhancing
CN108073917A (en) * 2018-01-24 2018-05-25 燕山大学 A kind of face identification method based on convolutional neural networks
WO2019180310A1 (en) * 2018-03-21 2019-09-26 Nokia Technologies Oy A method, an apparatus and a computer program product for an interpretable neural network representation
US20180357542A1 (en) * 2018-06-08 2018-12-13 University Of Electronic Science And Technology Of China 1D-CNN-Based Distributed Optical Fiber Sensing Signal Feature Learning and Classification Method
WO2020140386A1 (en) * 2019-01-02 2020-07-09 平安科技（深圳）有限公司 Textcnn-based knowledge extraction method and apparatus, and computer device and storage medium
WO2020172838A1 (en) * 2019-02-26 2020-09-03 长沙理工大学 Image classification method for improvement of auxiliary classifier gan
CN110084215A (en) * 2019-05-05 2019-08-02 上海海事大学 A kind of pedestrian of the twin network model of binaryzation triple recognition methods and system again
CN111339935A (en) * 2020-02-25 2020-06-26 西安电子科技大学 Optical remote sensing picture classification method based on interpretable CNN image classification model

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
张玉冰;于威威;: "基于深度学习算法的图像集识别", 现代计算机(专业版), no. 21 *
江彤彤;成金勇;鹿文鹏;: "基于卷积神经网络多层特征提取的目标识别", 计算机系统应用, no. 12 *
江泽涛;秦嘉奇;张少钦;: "参数池化卷积神经网络图像分类方法", 电子学报, no. 09 *
王新美;丁爱玲;雷梦宁;康盟;: "基于CNN和SVM融合的交通标志识别", 计算机技术与发展, no. 06 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113673627A (en) * 2021-09-02 2021-11-19 哈尔滨工程大学 Interpretive automatic commodity classification method and system
CN113673627B (en) * 2021-09-02 2024-02-13 哈尔滨工程大学 Automatic commodity classification method and system with interpretation

Also Published As

Publication number Publication date
CN112434790B (en) 2024-03-29

Similar Documents

Publication Publication Date Title
CN107122809B (en) Neural network feature learning method based on image self-coding
Zhao et al. High-resolution image classification integrating spectral-spatial-location cues by conditional random fields
US10489914B2 (en) Method and apparatus for parsing and processing three-dimensional CAD model
Xie et al. Point clouds learning with attention-based graph convolution networks
Wang et al. Encoder-X: solving unknown coefficients automatically in polynomial fitting by using an autoencoder
WO2018010434A1 (en) Image classification method and device
Bi et al. A survey on evolutionary computation for computer vision and image analysis: Past, present, and future trends
WO2014205231A1 (en) Deep learning framework for generic object detection
Sun et al. Recognition of SAR target based on multilayer auto-encoder and SNN
CN113989890A (en) Face expression recognition method based on multi-channel fusion and lightweight neural network
Chen et al. Multi-SVM based Dempster–Shafer theory for gesture intention understanding using sparse coding feature
CN115080764A (en) Medical similar entity classification method and system based on knowledge graph and clustering algorithm
Bian et al. A survey of the methods on fingerprint orientation field estimation
Shao et al. Deep multi-center learning for face alignment
Jia et al. Nonlocal regularized CNN for image segmentation
Wang et al. Robust local metric learning via least square regression regularization for scene recognition
Liu et al. Bilaterally normalized scale-consistent sinkhorn distance for few-shot image classification
Sharif et al. Segmentation of Images Using Deep Learning: A Survey
Hariri et al. Recognition of 3D emotional facial expression based on handcrafted and deep feature combination
Cai et al. Pattern recognition using Markov random field models
CN112434790A (en) Self-interpretation method for convolutional neural network to judge partial black box problem
Yang et al. Fine-grained lip image segmentation using fuzzy logic and graph reasoning
Cao et al. Wpe: Weighted prototype estimation for few-shot learning
Yang et al. Bag of feature with discriminative module for non-rigid shape retrieval
Dai et al. Cross modal adaptive few-shot learning based on task dependence

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant