CN110020674B - Cross-domain self-adaptive image classification method for improving local category discrimination - Google Patents
Cross-domain self-adaptive image classification method for improving local category discrimination Download PDFInfo
- Publication number
- CN110020674B CN110020674B CN201910190041.4A CN201910190041A CN110020674B CN 110020674 B CN110020674 B CN 110020674B CN 201910190041 A CN201910190041 A CN 201910190041A CN 110020674 B CN110020674 B CN 110020674B
- Authority
- CN
- China
- Prior art keywords
- image
- matrix
- training set
- target domain
- content
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012549 training Methods 0.000 claims abstract description 73
- 238000012360 testing method Methods 0.000 claims abstract description 66
- 238000009826 distribution Methods 0.000 claims abstract description 28
- 238000012545 processing Methods 0.000 claims abstract description 12
- 239000011159 matrix material Substances 0.000 claims description 136
- 239000013598 vector Substances 0.000 claims description 99
- 238000005457 optimization Methods 0.000 claims description 37
- 239000006185 dispersion Substances 0.000 claims description 18
- 238000013507 mapping Methods 0.000 claims description 18
- 239000000203 mixture Substances 0.000 claims description 12
- 239000004576 sand Substances 0.000 claims description 6
- 238000002372 labelling Methods 0.000 claims description 3
- 230000017105 transposition Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 abstract description 2
- 230000002776 aggregation Effects 0.000 abstract description 2
- 238000013145 classification model Methods 0.000 abstract description 2
- 238000004458 analytical method Methods 0.000 description 3
- 241000764238 Isis Species 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a cross-domain self-adaptive image classification method for improving local category discrimination, and belongs to the technical field of image processing. According to the cross-domain self-adaptive image classification method, when an image content classification model is trained, images of a training set and a testing set are different in tone, angle, definition and the like, and data representing image information obeys different probability distribution. The method has the advantages that the distributed and shared characteristics of the two images are learned, common points among the images with the same content and differences among the images with different contents are mined in other images with similar styles of each image, the aggregation of the images with the same content is stronger, the mutual interference among the images with different contents is reduced, the local category distinguishing degree of the image data set is improved, and the image classification accuracy is improved.
Description
Technical Field
The invention relates to a cross-domain self-adaptive image classification method for improving local category discrimination, and belongs to the technical field of image processing.
Background
In a general image classification problem, training set and test set images generally exhibit the same style. These images may be captured or drawn collectively in the same environment by the same equipment in a short time, or collected and collated by the collector intentionally according to some criteria. After the images are subjected to the data processing, the data obey the same probability distribution. In this case, the classifier is trained by using known image content classification information (i.e., labels) in the training set, so that the image content in the test set can be accurately identified and classified. However, when solving the actual problem, the image set involved in the problem, i.e., the target domain data set, often has insufficient content tags that can be used directly because of problems such as tag production cost. To cope with this, another set of labeled image sets with a style not identical to the target image style may be used as the training set, i.e., the source domain data set. The images of the source domain and the target domain are subjected to different probability distributions after being digitized. In this case, the classifier of the training set (source domain) cannot be directly used for classification of the test set (target domain) data. The core of the cross-domain image classification problem is to solve the problem that the image styles of the training set and the test set are inconsistent.
In recent years, researchers have proposed a variety of models and algorithms that enable cross-domain image classification. Since each image is composed of a large number of pixel points, the dimension of the image itself is often as high as 1e6, a commonly used method is to find a low-dimensional subspace, so that the images of the source domain and the target domain are subjected to the same or similar distribution after being subjected to dimension reduction mapping to the subspace, wherein the migration component analysis method proposed by Pan et al in 2011 and the joint distribution adaptation algorithm proposed by Long et al in 2013 are representative. The core idea is to search an optimal low-dimensional mapping to minimize the sample probability distribution difference in the low-dimensional space. The distance measurement method of probability distribution used in these two methods is the Maximum Mean Distance (MMD). The MMD is defined as the distance maximum of two expected values distributed under a certain function class mapping. And minimizing the MMD in the low-dimensional subspace to obtain the optimal low-dimensional invariant feature representation of the image, and eliminating the difference between the source domain image and the target domain image after dimension reduction.
Minimizing MMD, while having good results in terms of distribution alignment, can only numerically translate two different sets of images into the same distribution, according to the mathematical definition of MMD, without retaining sufficient information of the images themselves. This causes part of information that is useful for image classification to be lost in the low-dimensional subspace, resulting in a reduction in classification accuracy. Therefore, how to perfect an objective function in an algorithm based on MMD optimization is an important problem to protect key content information of an image while ensuring the cross-domain characteristics of the obtained features.
In the related algorithm of the classification problem, there are a lot of methods for improving the image classification degree of data distribution. Typical methods such as Linear Discriminant Analysis (LDA) simultaneously maximize the dispersion between data classes and minimize the dispersion within data classes in the form of rayleigh quotient, thereby realizing the aggregation of homogeneous data and the separation of heterogeneous data. Similarly, Yan et al, 2007, proposed an edge Fisher analysis (MFA) method that improves the class clustering operation of LDA on the global data distribution to be performed in the local neighborhood of each data point, i.e., maximizes the dispersion between each data point and the heterogeneous points in its neighborhood, and minimizes the dispersion between each data point and the homogeneous points in its neighborhood. Compared with LDA, MFA does not need the hypothesis that various data distributions obey Gaussian distribution, does not need prior information of the distribution, has better generalization capability, and can effectively solve the problem of data multimodal distribution. However, this method has not been applied to the problem of cross-domain adaptation of images.
Disclosure of Invention
The invention aims to provide a cross-domain self-adaptive image classification method for improving local category discrimination, which optimizes the local dispersion characteristics of a plurality of image samples and enhances the content discrimination of an image in a local adjacent range so as to be beneficial to the prediction of image content classification labels.
The invention provides a cross-domain self-adaptive image classification method for improving local category discrimination, which comprises the following steps:
(1) the method comprises the steps of scanning a plurality of images line by line, sequentially arranging pixels obtained by line scanning into column vectors according to a scanning sequence, and dividing the column vectors by Euclidean norms of the column vectors to obtain a plurality of image column vectors with Euclidean norms of 1;
(2) dividing a plurality of image column vectors obtained in the step (1) into a source domain training set { ZS,YSAnd target domain test set ZT},Wherein Z isSIs a set of a plurality of image column vectors in a source domain training set, YSIs a set of content classification labels for a plurality of images in a source domain training set, nSThe number of image column vectors in the training set for the source domain,is ZSThe ith image column vector, i.e., the ith image sample in the source domain training set,is a content classification label for the ith image, i.e.Representing an object described by an image, with dimension 1;wherein Z isTIs an objectSet of multiple image column vectors in a domain test set, nTThe number of image column vectors in the test set for the target domain,is ZTA jth image column vector, namely a jth image sample in the target domain test set;
(3) respectively calculating a plurality of source domain training set samples of the step (2)First column vector of WhereinK (·,) is a kernel function selected arbitrarily among a Gaussian kernel function, a hyperbolic tangent kernel function or a linear kernel function, and superscript T represents matrix transposition; using first column vectors respectivelyRepresenting a plurality of images in a source domain training set and comparing the plurality of imagesSequentially arranged in rows to obtain a source domain training set matrix XS(ii) a Respectively calculating a plurality of target domain samples of the step (2)Second column vector of WhereinUsing second column vectors respectivelyRepresenting a plurality of images in a target domain test set and combining the plurality of imagesSequentially arranging the target domain test set matrixes according to rows to obtain a target domain test set matrix XT(ii) a Training set matrix X according to source domainSAnd a target domain test set matrix XTObtaining a whole-body data set matrix X, X ═ XS,XT];
(4) Setting a projection matrix ATUsing projection matrix ATPerforming linear mapping on the plurality of image column vectors obtained in the step (3), namely performing linear mapping on the plurality of image column vectorsAndrespectively linear mapping to obtain projection column vectorAnd
(5) taking the projection column vector obtained after the linear mapping in the step (4) as an image data point sample, and establishing an optimization model of cross-domain self-adaptive image classification features, wherein an objective function of the optimization model comprises the following steps:
a. the square MMD of the maximum mean distance sample estimation value between the probability distribution of the image samples in the source domain training set and the probability distribution of the image samples in the target domain testing set2(S, T) is minimum:
wherein, Tr represents the trace of the matrix, i.e. the sum of diagonal elements of the matrix, M is the maximum mean distance matrix:
wherein 1 represents an all-1 matrix;
b. according to the types of the content classification labels in the step (2), enabling the square sum of the maximum mean distance sample estimation values between the sample probability distribution of each type of image samples in the source domain training set and the sample probability distribution of each type of image samples in the target domain testing setTo a minimum:
wherein C represents the number of image sample classes,indicating that the data point is temporarily assigned at the current stepThe prediction content classification label of (1) dimension,representing the number of image samples in the source domain training set with a content classification label c,representing the number of image samples with a current predicted content classification label of c in the target domain test set, McIs the maximum mean distance matrix of the image samples with content classification label c:
wherein e isScIs of length nSIs a column vector composed of 0 and 1, e is a column vector composed of e when the content classification label of the corresponding image in the source domain training set is cScThe value of the element in (e) is 1, and when the content classification label of the corresponding image in the source domain training set is not c, eScThe value of the element in (A) is 0; e.g. of the typeTcIs of length nTIs composed of 0 and 1, eTcThe element value of (1) represents that the corresponding image in the target domain test set has the current prediction content classification label of c, eTcThe value of the element in the target domain is 0, which indicates that the classification label of the corresponding image in the target domain test set in the current prediction content is not c;
c. minimizing the weighted square sum of the Euclidean distance between the projection column vectors of every two image samples with the same content label in the source domain training set and the Euclidean distance between the projection column vectors of every two image samples with the same content label in the target domain testing set:
in the first row of the above-mentioned formula,andthe weighting coefficients for the distance between every two image samples with the same content label inside the source domain training set and the target domain test set respectively,indicating if the image sample isIs a sample of an imageLike k-neighbors of, then get ηijIf the image sample is 1Not of image samplesLike k-neighbors of, then get ηijWhen the value of k is 0, the value of k is determined according to the precision of image processing; alpha is alphacIs a positive coefficient, alpha, associated with a class in the source domain training setcIs determined according to the accuracy of the image processing, in one embodiment of the invention, αcIs 0.01;if the image sampleIs a sample of an imageLike k-neighbors of, then get ηklIf the image sample is 1Not of image samplesLike k-neighbors of, then get ηkl=0;βcIs a positive coefficient, β, associated with the class in the target domain test setcThe value of (a) is determined according to the precision of image processing;
in the first term of the second line of the above formula, WSIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix with diagonal elements in the diagonal matrix ofRSIs an intra-class dispersion matrix of a source domain training set,in the second term of the second line of the above formula, WTIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix with diagonal elements in the diagonal matrix ofRTIs an intra-class dispersion matrix of the target domain,
the definition of the matrix R in the third row of the above equation is:
wherein 0 represents a matrix with elements all 0;
d. maximizing the weighted square sum of the Euclidean distance between the projection column vectors of every two image samples with different content labels in the source domain training set and the Euclidean distance between the projection column vectors of every two image samples with different content labels in the target domain testing set:
in the first row of the above-mentioned formula,andweighting coefficients for the distance between the image samples with different content labels inside the source domain training set and the target domain testing set respectively,if the image sampleIs a sample of an imageOf different classes of neighbors, thenIf the image sampleNot of image samplesOf different classes of neighbors, then Indicating pointsIs a pointThe different kinds of neighboring points of (a),indicating pointsIs not a pointDifferent classes of neighbors of (1);
in the first term of the second line of the above formula, USIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix having diagonal elements ofPSIs an inter-class dispersion matrix of the source domain training set,in the second term of the second row, UTIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix having diagonal elements ofPTIs the inter-class dispersion matrix of the target domain test set,
the definition of matrix P in the third row of the above equation is:
e. making the projection matrix A in step (4)TThe regularization term of (d) is minimum:
wherein,is the sum of the squares of all elements in the matrix a, λ is a positive coefficient, and the value of λ is taken according to the image classification accuracy, and in one embodiment of the method, the value is 1;
according to the objective function, an optimization model of cross-domain self-adaptive image classification features is obtained as follows:
(6) and solving the optimization model of the cross-domain self-adaptive image classification characteristics, and initializing to obtain the following optimization model in the first iteration of solving the optimization model:
wherein I is an identity matrix, and the optimization model is solved by the following formula to obtain an intermediate optimal solution A*:
Where Θ is the diagonal matrix and the diagonal elements of the diagonal matrix are matricesRelative to the matrixSolving the matrix for the generalized eigenvalues ofRelative to the matrixN of (A) to (B)S+nTA sum of generalized eigenvalues and nS+nTN corresponding to each generalized eigenvalueS+nTA generalized characteristic column vector from nS+nTM minimum generalized eigenvalues are selected from the generalized eigenvalues and are arranged in the order from small to large, m generalized eigenvalue column vectors respectively corresponding to the m generalized eigenvalues obtained by selection are sequentially arranged in rows in the same order as the m generalized eigenvalues, and a matrix A is obtained*,A*Is a first intermediate optimal solution of the above-mentioned optimization model;
(7) according to the first intermediate optimal solution A obtained in the step (6)*For the original image column vectorAndlinear mapping is carried out to obtain the column vector of the image sampleAndusing column vectorsMaking training set, and aligning column vector by using nearest neighbor methodPredicting image content labels to obtain predicted content labels of a group of target domain test set samples
(8) Substituting the predicted content label obtained in the step (7) into the complete optimization model in the step (5), solving the optimization model, and solving the intermediate optimal solution A of the optimization model by using the following formula*:
Where Θ is the diagonal matrix and the diagonal elements of the diagonal matrix are matricesRelative to matrix XPXTThe generalized eigenvalues of (a); solving the matrixRelative to matrix XPXTN of (A) to (B)S+nTA generalized eigenvalue and nS+nTN corresponding to each generalized eigenvalueS+nTA generalized characteristic column vector from nS+nTM minimum generalized eigenvalues are selected from the generalized eigenvalues and are arranged in the order from small to large, m generalized eigenvalue column vectors respectively corresponding to the m generalized eigenvalues obtained by selection are sequentially arranged in rows in the same order as the m generalized eigenvalues, and a projection matrix A is obtained*,A*Is a second intermediate optimal solution of the above-mentioned optimization model;
(9) replacing the first intermediate optimal solution in the step (7) with the second intermediate optimal solution obtained in the step (8), repeating the steps (7) and (8), and predicting the content label of the step (7) obtained by N times of circulationMaking a judgment if the predicted content label obtained in N cyclesIf the two are identical, ending the iteration and labeling the predicted content obtained in the step (7) in the last iterationAs a prediction result, namely an image classification result, the cross-domain self-adaptive image classification for improving the local category discrimination is realized; if obtained in N cyclesContent measurement labelAnd (3) if the two solutions are not identical, returning to the step (7), replacing the first intermediate optimal solution in the step (7) with the second intermediate optimal solution obtained in the step (8), repeating the steps (7) and (8), determining the value of N according to the image classification precision, and in one embodiment of the method, setting the value of N to be 10.
The cross-domain self-adaptive image classification method for improving the local category discrimination, provided by the invention, has the characteristics and advantages that:
according to the cross-domain self-adaptive image classification method for improving the local category discrimination, when an image content classification model is trained, images of a training set and a testing set are different in aspects of tone, angle, definition and the like, and data representing image information obeys different probability distribution. The method has the advantages that the distributed and shared characteristics of the two images are learned, common points among the images with the same content and differences among the images with different contents are mined in other images with similar styles of each image, the images with the same content are enabled to be more highly aggregated, mutual interference among the images with different contents is reduced, and therefore the local category distinguishing degree of the image data set is improved, and the classifying accuracy is further improved.
In conclusion, the image classification method can optimize the local dispersion characteristics of a plurality of image samples, enhance the content discrimination of the image in the local neighbor range and facilitate the prediction of the image content classification label; and the predicted image content label is updated iteratively, so that the accuracy and the reliability of image content classification are improved.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The invention provides a cross-domain self-adaptive image classification method for improving local category discrimination, a flow chart of which is shown in figure 1, and the method comprises the following steps:
(1) the method comprises the steps of scanning a plurality of images line by line, sequentially arranging pixels obtained by line scanning into column vectors according to a scanning sequence, and dividing the column vectors by Euclidean norms of the column vectors to obtain a plurality of image column vectors with Euclidean norms of 1;
(2) dividing a plurality of image column vectors obtained in the step (1) into a source domain training set { ZS,YSAnd target domain test set ZT},Wherein Z isSIs a set of a plurality of image column vectors in a source domain training set, YSIs a set of content classification labels for a plurality of images in a source domain training set, nSThe number of image column vectors in the training set for the source domain,is ZSThe ith image column vector, i.e., the ith image sample in the source domain training set,is a content classification label for the ith image, i.e.Representing an object described by an image, with dimension 1;wherein Z isTIs a set of a plurality of image column vectors in a target domain test set, nTThe number of image column vectors in the test set for the target domain,is ZTThe jth image column vector is the jth image sample in the target domain test set, and the content classification label in the target domain test set is unknown;
(3) respectively calculating a plurality of source domain training set samples of the step (2)First column vector of WhereinK (·,) is a kernel function selected arbitrarily among a Gaussian kernel function, a hyperbolic tangent kernel function or a linear kernel function, and superscript T represents matrix transposition; using first column vectors respectivelyRepresenting a plurality of images in a source domain training set and comparing the plurality of imagesSequentially arranged in rows to obtain a source domain training set matrix XS(ii) a Respectively calculating a plurality of target domain samples of the step (2)Second column vector of WhereinUsing second column vectors respectivelyRepresenting a plurality of images in a target domain test set and combining the plurality of imagesSequentially arranging the target domain test set matrixes according to rows to obtain a target domain test set matrix XT(ii) a Training set matrix X according to source domainSAnd a target domain test set matrix XTObtaining a whole-body data set matrix X, X ═ XS,XT];
(4) Setting a projection matrix ATUsing projection matrix ATPerforming linear mapping on the plurality of image column vectors obtained in the step (3), namely performing linear mapping on the plurality of image column vectorsAndrespectively linear mapping to obtain projection column vectorAndmatrix ATThe value is undetermined;
(5) taking the projection column vector obtained after the linear mapping in the step (4) as an image data point sample, and establishing an optimization model of cross-domain self-adaptive image classification features, wherein an objective function of the optimization model comprises the following steps:
a. the square MMD of the maximum mean distance sample estimation value between the probability distribution of the image samples in the source domain training set and the probability distribution of the image samples in the target domain testing set2(S, T) is minimum:
wherein, Tr represents the trace of the matrix, i.e. the sum of diagonal elements of the matrix, M is the maximum mean distance matrix:
wherein 1 represents an all-1 matrix;
b. according to the types of the content classification labels in the step (2), the maximum average between the sample probability distribution of each type of image sample in the source domain training set and the sample probability distribution of each type of image sample in the target domain testing set is enabled to beSum of squares of the estimated values of the value distance samplesTo a minimum:
wherein C represents the number of image sample classes,indicating that the data point is temporarily assigned at the current stepThe prediction content classification label of (1) dimension,representing the number of image samples in the source domain training set with a content classification label c,representing the number of image samples with a current predicted content classification label of c in the target domain test set, McIs the maximum mean distance matrix of the image samples with content classification label c:
wherein e isScIs of length nSIs a column vector composed of 0 and 1, e is a column vector composed of e when the content classification label of the corresponding image in the source domain training set is cScThe value of the element in (e) is 1, and when the content classification label of the corresponding image in the source domain training set is not c, eScThe value of the element in (A) is 0; e.g. of the typeTcIs of length nTIs composed of 0 and 1, eTcThe element value of (1) represents that the corresponding image in the target domain test set has the current prediction content classification label of c, eTcThe value of the element in (1) is 0, which represents the target domain measurementThe classification label of the corresponding image in the trial set in the current prediction content is not c;
c. minimizing the weighted square sum of the Euclidean distance between the projection column vectors of every two image samples with the same content label in the source domain training set and the Euclidean distance between the projection column vectors of every two image samples with the same content label in the target domain testing set:
in the first row of the above-mentioned formula,andthe weighting coefficients for the distance between every two image samples with the same content label inside the source domain training set and the target domain test set respectively,indicating if the image sample isIs a sample of an imageLike k-neighbors of, then get ηijIf the image sample is 1Not of image samplesLike k-neighbors of, then get ηijWhen the value of k is 0, the value of k is determined according to the precision of image processing; alpha is alphacIs a positive coefficient, alpha, associated with a class in the source domain training setcIs determined according to the accuracy of the image processing, in one embodiment of the invention, αcIs 0.01;if the image sampleIs a sample of an imageLike k-neighbors of, then get ηklIf the image sample is 1Not of image samplesLike k-neighbors of, then get ηkl=0;βcIs a positive coefficient, β, associated with the class in the target domain test setcIs determined according to the accuracy of the image processing, in one embodiment of the invention, βcIs 0.01;
in the first term of the second line of the above formula, WSIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix with diagonal elements in the diagonal matrix ofRSIs an intra-class dispersion matrix of a source domain training set,in the second term of the second line of the above formula, WTIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix with diagonal elements in the diagonal matrix ofRTIs an intra-class dispersion matrix of the target domain,
the definition of the matrix R in the third row of the above equation is:
wherein 0 represents a matrix with elements all 0;
d. maximizing the weighted square sum of the Euclidean distance between the projection column vectors of every two image samples with different content labels in the source domain training set and the Euclidean distance between the projection column vectors of every two image samples with different content labels in the target domain testing set:
in the first row of the above-mentioned formula,andweighting coefficients for the distance between the image samples with different content labels inside the source domain training set and the target domain testing set respectively,if the image sampleIs a sample of an imageOf different classes of neighbors, thenIf the image sampleNot of image samplesOf different classes of neighbors, then Indicating pointsIs a pointThe different kinds of neighboring points of (a),indicating pointsIs not a pointDifferent classes of neighbors of (1);
in the first term of the second line of the above formula, USIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix having diagonal elements ofPSIs an inter-class dispersion matrix of the source domain training set,in the second term of the second row, UTIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix having diagonal elements ofPTIs the inter-class dispersion matrix of the target domain test set,
the definition of matrix P in the third row of the above equation is:
e. making the projection matrix A in step (4)TThe regularization term of (d) is minimum:
wherein,is the sum of the squares of all elements in the matrix a, λ is a positive coefficient, and the value of λ is taken according to the image classification accuracy, and in one embodiment of the method, the value is 1;
according to the objective function, an optimization model of cross-domain self-adaptive image classification features is obtained as follows:
(6) and solving the optimization model of the cross-domain self-adaptive image classification characteristics, and initializing to obtain the following optimization model in the first iteration of solving the optimization model:
wherein I is an identity matrix, and the optimization model is solved by the following formula to obtain an intermediate optimal solution A*:
Where Θ is the diagonal matrix and the diagonal elements of the diagonal matrix are matricesRelative to the matrixSolving the matrix for the generalized eigenvalues ofRelative to the matrixN of (A) to (B)S+nTA sum of generalized eigenvalues and nS+nTN corresponding to each generalized eigenvalueS+nTA generalized characteristic column vector from nS+nTM minimum generalized eigenvalues are selected from the generalized eigenvalues and are arranged in the order from small to large, m generalized eigenvalue column vectors respectively corresponding to the m generalized eigenvalues obtained by selection are sequentially arranged in rows in the same order as the m generalized eigenvalues, and a matrix A is obtained*,A*Is a first intermediate optimal solution of the above-mentioned optimization model;
(7) according to the first intermediate optimal solution A obtained in the step (6)*For the original image column vectorAndlinear mapping is carried out to obtain the column vector of the image sampleAndusing column vectorsMaking training set, and aligning column vector by using nearest neighbor methodPredicting image content labels to obtain predicted content labels of a group of target domain test set samples
(8) Substituting the predicted content label obtained in the step (7) into the complete optimization model in the step (5), solving the optimization model, and solving the intermediate optimal solution A of the optimization model by using the following formula*:
Where Θ is the diagonal matrix and the diagonal elements of the diagonal matrix are matricesRelative to matrix XPXTThe generalized eigenvalues of (a); solving the matrixRelative to matrix XPXTN of (A) to (B)S+nTA generalized eigenvalue and nS+nTN corresponding to each generalized eigenvalueS+nTA generalized characteristic column vector from nS+nTM minimum generalized eigenvalues are selected from the generalized eigenvalues and are arranged in the order from small to large, m generalized eigenvalue column vectors respectively corresponding to the m generalized eigenvalues obtained by selection are sequentially arranged in rows in the same order as the m generalized eigenvalues, and a projection matrix A is obtained*,A*Is a second intermediate optimal solution of the above-mentioned optimization model;
(9) replacing the first intermediate optimal solution in the step (7) with the second intermediate optimal solution obtained in the step (8), repeating the steps (7) and (8), and predicting the content label of the step (7) obtained by N times of circulationMaking a judgment if the predicted content label obtained in N cyclesIf the two are identical, ending the iteration and labeling the predicted content obtained in the step (7) in the last iterationAs a prediction result, namely an image classification result, the cross-domain self-adaptive image classification for improving the local category discrimination is realized; if predicted content label obtained in N cyclesAnd (3) if the two solutions are not identical, returning to the step (7), replacing the first intermediate optimal solution in the step (7) with the second intermediate optimal solution obtained in the step (8), repeating the steps (7) and (8), determining the value of N according to the image classification precision, and in one embodiment of the method, setting the value of N to be 10.
Claims (1)
1. A cross-domain self-adaptive image classification method for improving local category discrimination is characterized by comprising the following steps:
(1) the method comprises the steps of scanning a plurality of images line by line, sequentially arranging pixels obtained by line scanning into column vectors according to a scanning sequence, and dividing the column vectors by Euclidean norms of the column vectors to obtain a plurality of image column vectors with Euclidean norms of 1;
(2) dividing a plurality of image column vectors obtained in the step (1) into a source domain training set { ZS,YSAnd target domain test set ZT},Wherein Z isSIs a set of a plurality of image column vectors in a source domain training set, YSIs a set of content classification labels for a plurality of images in a source domain training set, nSThe number of image column vectors in the training set for the source domain,is ZSThe ith image column vector, i.e., the ith image sample in the source domain training set,is a content classification label for the ith image, i.e.Representing an object described by an image, with dimension 1;wherein Z isTIs a set of a plurality of image column vectors in a target domain test set, nTThe number of image column vectors in the test set for the target domain,is ZTA jth image column vector, namely a jth image sample in the target domain test set;
(3) respectively calculating a plurality of source domain training set samples of the step (2)First column vector of WhereinK (·,) is a kernel function selected arbitrarily among a Gaussian kernel function, a hyperbolic tangent kernel function or a linear kernel function, and superscript T represents matrix transposition; using first column vectors respectivelyRepresenting a plurality of images in a source domain training set and comparing the plurality of imagesSequentially arranged in rows to obtain a source domain training set matrix XS(ii) a Respectively calculating a plurality of target domain samples of the step (2)Second column vector of WhereinUsing second column vectors respectivelyTo representThe target domain test collects a plurality of images and combines the plurality of imagesSequentially arranging the target domain test set matrixes according to rows to obtain a target domain test set matrix XT(ii) a Training set matrix X according to source domainSAnd a target domain test set matrix XTObtaining a whole-body data set matrix X, X ═ XS,XT];
(4) Setting a projection matrix ATUsing projection matrix ATPerforming linear mapping on the plurality of image column vectors obtained in the step (3), namely performing linear mapping on the plurality of image column vectorsAndrespectively linear mapping to obtain projection column vectorAnd
(5) taking the projection column vector obtained after the linear mapping in the step (4) as an image data point sample, and establishing an optimization model of cross-domain self-adaptive image classification features, wherein an objective function of the optimization model comprises the following steps:
a. the square MMD of the maximum mean distance sample estimation value between the probability distribution of the image samples in the source domain training set and the probability distribution of the image samples in the target domain testing set2(S, T) is minimum:
wherein, Tr represents the trace of the matrix, i.e. the sum of diagonal elements of the matrix, M is the maximum mean distance matrix:
wherein 1 represents an all-1 matrix;
b. according to the types of the content classification labels in the step (2), enabling the square sum of the maximum mean distance sample estimation values between the sample probability distribution of each type of image samples in the source domain training set and the sample probability distribution of each type of image samples in the target domain testing setTo a minimum:
wherein C represents the number of image sample classes,indicating that the data point is temporarily assigned at the current stepThe prediction content classification label of (1) dimension,representing the number of image samples in the source domain training set with a content classification label c,representing the number of image samples with a current predicted content classification label of c in the target domain test set, McIs the maximum mean distance matrix of the image samples with content classification label c:
wherein e isScIs of length nSIs a column vector composed of 0 and 1, e is a column vector composed of e when the content classification label of the corresponding image in the source domain training set is cScThe value of the element in (e) is 1, and when the content classification label of the corresponding image in the source domain training set is not c, eScThe value of the element in (A) is 0; e.g. of the typeTcIs of length nTIs composed of 0 and 1, eTcThe element value of (1) represents that the corresponding image in the target domain test set has the current prediction content classification label of c, eTcThe value of the element in the target domain is 0, which indicates that the classification label of the corresponding image in the target domain test set in the current prediction content is not c;
c. minimizing the weighted square sum of the Euclidean distance between the projection column vectors of every two image samples with the same content label in the source domain training set and the Euclidean distance between the projection column vectors of every two image samples with the same content label in the target domain testing set:
in the first row of the above-mentioned formula,andthe weight coefficients of the distance between every two image samples with the same content label in the source domain training set and the target domain testing set respectively,ηijif an image sample is represented by 1Is a sample of an imageThe same class of K-neighbors of (1), then eta is obtainedijIf the image sample is 1Not of image samplesThe same class of K-neighbors of (1), then eta is obtainedijWhen the value of K is 0, the value of K is determined according to the precision of image processing; alpha is alphacIs a positive coefficient, alpha, associated with a class in the source domain training setcThe value of (a) is determined according to the precision of image processing;if the image sampleIs a sample of an imageThe same class of K-neighbors of (1), then eta is obtainedklIf the image sample is 1Not of image samplesThe same class of K-neighbors of (1), then eta is obtainedkl=0;βcIs a positive coefficient, β, associated with the class in the target domain test setcThe value of (a) is determined according to the precision of image processing;
in the first term of the second line of the above formula, WSIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix with diagonal elements in the diagonal matrix ofRSIs an intra-class dispersion matrix of a source domain training set,in the second term of the second line of the above formula, WTIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix with diagonal elements in the diagonal matrix ofRTIs an intra-class dispersion matrix of the target domain,
the definition of the matrix R in the third row of the above equation is:
wherein 0 represents a matrix with elements all 0;
d. maximizing the weighted square sum of the Euclidean distance between the projection column vectors of every two image samples with different content labels in the source domain training set and the Euclidean distance between the projection column vectors of every two image samples with different content labels in the target domain testing set:
in the first row of the above-mentioned formula,andthe weight coefficients of the distance between every two image samples with different content labels in the source domain training set and the target domain testing set respectively,if the image sampleIs a sample of an imageOf different classes of neighbors, thenIf the image sampleNot of image samplesOf different classes of neighbors, then Indicating pointsIs a pointThe different kinds of neighboring points of (a),indicating pointsIs not a pointDifferent classes of neighbors of (1);
in the first term of the second line of the above formula, USIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix having diagonal elements ofPSIs an inter-class dispersion matrix of the source domain training set,in the second term of the second row, UTIs formed by weight coefficientsThe weight matrix of the composition is formed,is a diagonal matrix having diagonal elements ofPTClass being target domain test setThe matrix of the inter-dispersion is,
the definition of matrix P in the third row of the above equation is:
e. making the projection matrix A in step (4)TThe regularization term of (d) is minimum:
wherein,is the sum of squares of all elements in the matrix A, lambda is a positive coefficient, and the value of lambda is 1 according to the image classification precision;
according to the objective function, an optimization model of cross-domain self-adaptive image classification features is obtained as follows:
(6) and solving the optimization model of the cross-domain self-adaptive image classification characteristics, and initializing to obtain the following optimization model in the first iteration of solving the optimization model:
wherein I is an identity matrix, and the optimization model is solved by the following formula to obtain an intermediate optimal solution A*:
Where Θ is the diagonal matrix and the diagonal elements of the diagonal matrix are matricesRelative to the matrixSolving the matrix for the generalized eigenvalues ofRelative to the matrixN of (A) to (B)S+nTA sum of generalized eigenvalues and nS+nTN corresponding to each generalized eigenvalueS+nTA generalized characteristic column vector from nS+nTM minimum generalized eigenvalues are selected from the generalized eigenvalues and are arranged in the order from small to large, m generalized eigenvalue column vectors respectively corresponding to the m generalized eigenvalues obtained by selection are sequentially arranged in rows in the same order as the m generalized eigenvalues, and a matrix A is obtained*,A*Is a first intermediate optimal solution of the above-mentioned optimization model;
(7) according to the intermediate optimal solution A obtained in the step (6)*For the original image column vectorAndlinear mapping is carried out to obtain the column vector of the image sampleAndusing column vectorsMaking training set, and aligning column vector by using nearest neighbor methodPredicting image content labels to obtain predicted content labels of a group of target domain test set samples
(8) Substituting the predicted content label obtained in the step (7) into the complete optimization model in the step (5), solving the optimization model, and solving the intermediate optimal solution A of the optimization model by using the following formula*:
Where Θ is the diagonal matrix and the diagonal elements of the diagonal matrix are matricesRelative to matrix XPXTThe generalized eigenvalues of (a); solving the matrixRelative to matrix XPXTN of (A) to (B)S+nTA generalized eigenvalue and nS+nTN corresponding to each generalized eigenvalueS+nTA generalized characteristic column vector from nS+nTM minimum generalized eigenvalues are selected from the generalized eigenvalues and are arranged in the order from small to large, m generalized eigenvalue column vectors respectively corresponding to the m generalized eigenvalues obtained by selection are sequentially arranged in rows in the same order as the m generalized eigenvalues, and a projection matrix A is obtained*,A*Is a second intermediate optimal solution of the above-mentioned optimization model;
(9) replacing the first intermediate optimal solution in the step (7) with the second intermediate optimal solution obtained in the step (8), repeating the steps (7) and (8), and predicting the content label of the step (7) obtained by N times of circulationMaking a judgment if the predicted content label obtained in the last N cyclesIf the two are identical, ending the iteration and labeling the predicted content obtained in the step (7) in the last iterationAs a prediction result, namely an image classification result, the cross-domain self-adaptive image classification for improving the local category discrimination is realized; if predicted content label obtained in N cyclesAnd (4) if the image classification accuracy is not the same, returning to the step (7), replacing the first intermediate optimal solution in the step (7) with the second intermediate optimal solution obtained in the step (8), repeating the steps (7) and (8), and determining the value of N according to the image classification accuracy.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910190041.4A CN110020674B (en) | 2019-03-13 | 2019-03-13 | Cross-domain self-adaptive image classification method for improving local category discrimination |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910190041.4A CN110020674B (en) | 2019-03-13 | 2019-03-13 | Cross-domain self-adaptive image classification method for improving local category discrimination |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110020674A CN110020674A (en) | 2019-07-16 |
CN110020674B true CN110020674B (en) | 2021-01-29 |
Family
ID=67189510
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910190041.4A Active CN110020674B (en) | 2019-03-13 | 2019-03-13 | Cross-domain self-adaptive image classification method for improving local category discrimination |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110020674B (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112199572B (en) * | 2020-11-09 | 2023-06-06 | 广西职业技术学院 | Beijing pattern collecting and arranging system |
CN112426161B (en) * | 2020-11-17 | 2021-09-07 | 浙江大学 | Time-varying electroencephalogram feature extraction method based on domain self-adaptation |
CN112861929B (en) * | 2021-01-20 | 2022-11-08 | 河南科技大学 | Image classification method based on semi-supervised weighted migration discriminant analysis |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015125662A (en) * | 2013-12-27 | 2015-07-06 | 株式会社リコー | Object identification program and device |
CN104102917B (en) * | 2014-07-03 | 2017-05-10 | 中国石油大学(北京) | Construction method of domain self-adaptive classifier, construction device for domain self-adaptive classifier, data classification method and data classification device |
CN105224949B (en) * | 2015-09-23 | 2018-11-13 | 电子科技大学 | SAR image terrain classification method based on cross-cutting transfer learning |
US10289909B2 (en) * | 2017-03-06 | 2019-05-14 | Xerox Corporation | Conditional adaptation network for image classification |
CN107273927B (en) * | 2017-06-13 | 2020-09-22 | 西北工业大学 | Unsupervised field adaptive classification method based on inter-class matching |
CN108845974A (en) * | 2018-04-24 | 2018-11-20 | 清华大学 | Linear dimension reduction method is supervised using the having for separation probability of minimax probability machine |
-
2019
- 2019-03-13 CN CN201910190041.4A patent/CN110020674B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN110020674A (en) | 2019-07-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111814584B (en) | Vehicle re-identification method based on multi-center measurement loss under multi-view environment | |
CN111860670B (en) | Domain adaptive model training method, image detection method, device, equipment and medium | |
CN110689485B (en) | SIFT image splicing method applied to infrared nondestructive testing of large pressure container | |
CN107609541B (en) | Human body posture estimation method based on deformable convolution neural network | |
CN108520226B (en) | Pedestrian re-identification method based on body decomposition and significance detection | |
CN111126482B (en) | Remote sensing image automatic classification method based on multi-classifier cascade model | |
CN110020674B (en) | Cross-domain self-adaptive image classification method for improving local category discrimination | |
US6993193B2 (en) | Method and system of object classification employing dimension reduction | |
CN109977994B (en) | Representative image selection method based on multi-example active learning | |
CN112200121B (en) | Hyperspectral unknown target detection method based on EVM and deep learning | |
CN112633382A (en) | Mutual-neighbor-based few-sample image classification method and system | |
CN115410088B (en) | Hyperspectral image field self-adaption method based on virtual classifier | |
CN111563544B (en) | Maximum signal-to-noise ratio hyperspectral data dimension reduction method for multi-scale superpixel segmentation | |
CN108805061A (en) | Hyperspectral image classification method based on local auto-adaptive discriminant analysis | |
CN109829494A (en) | A kind of clustering ensemble method based on weighting similarity measurement | |
CN111027636A (en) | Unsupervised feature selection method and system based on multi-label learning | |
CN111539910B (en) | Rust area detection method and terminal equipment | |
Liu et al. | Ground-based cloud classification using weighted local binary patterns | |
CN115189942A (en) | Multi-view common-identification-picture semi-supervised network intrusion detection system under guidance of pseudo labels | |
CN115205310A (en) | Image segmentation method and system | |
Wang et al. | Local defect detection and print quality assessment | |
CN105160666B (en) | SAR image change detection based on Non-Stationary Analysis and condition random field | |
CN104050489B (en) | SAR ATR method based on multicore optimization | |
CN116206208B (en) | Forestry plant diseases and insect pests rapid analysis system based on artificial intelligence | |
CN112465016A (en) | Partial multi-mark learning method based on optimal distance between two adjacent marks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |