CN106485286B - Matrix classification model based on local sensitivity discrimination - Google Patents
Matrix classification model based on local sensitivity discrimination Download PDFInfo
- Publication number
- CN106485286B CN106485286B CN201610923346.8A CN201610923346A CN106485286B CN 106485286 B CN106485286 B CN 106485286B CN 201610923346 A CN201610923346 A CN 201610923346A CN 106485286 B CN106485286 B CN 106485286B
- Authority
- CN
- China
- Prior art keywords
- matrix
- local
- class
- mode
- lsdmatmhks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000011159 matrix material Substances 0.000 title claims abstract description 56
- 238000013145 classification model Methods 0.000 title claims abstract description 10
- 230000035945 sensitivity Effects 0.000 title claims abstract description 9
- 238000000034 method Methods 0.000 claims abstract description 37
- 238000012549 training Methods 0.000 claims abstract description 15
- 238000012360 testing method Methods 0.000 claims abstract description 12
- 238000011478 gradient descent method Methods 0.000 claims abstract description 5
- 238000004422 calculation algorithm Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 2
- 230000003190 augmentative effect Effects 0.000 claims 1
- 230000006870 function Effects 0.000 abstract description 8
- 230000008901 benefit Effects 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000013461 design Methods 0.000 description 19
- 239000013598 vector Substances 0.000 description 12
- 238000004458 analytical method Methods 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 5
- 238000006467 substitution reaction Methods 0.000 description 2
- 241000287196 Asthenes Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a matrix classification model based on local sensitivity discrimination, and the first isFirstly, collecting a data set, and converting collected samples into a matrix mode; second, using training set to construct sub-graph in and between local classesAnduse ofAndconstructing regularization termsThen regularizing the termsIntroducing a matrix-mode-oriented classifier MatMHKS to generate a new matrix-mode-oriented classification model LSDMatMHKS, training the LSDMatMHKS by using a training set, and solving the optimal solution of the model LADMatMHKS by using a gradient descent method; then, testing an optimal solution by using the test set, and obtaining an optimal decision function; and finally, calculating the input unknown matrix mode by using the obtained optimal decision function, and classifying the unknown matrix mode according to the output result. Compared with the traditional classification technology, the method and the device have the advantages that local sensitive discrimination information is introduced, so that local patterns of the same type are close to each other as much as possible, and local patterns of different types are far away from each other as much as possible, and therefore the classification stability and the model learning capacity are improved.
Description
Technical Field
The invention relates to the field of pattern recognition, in particular to a method for a matrix learning machine model based on local sensitivity discrimination.
Background
Different from the traditional vectorization-oriented classifier design method, the matrix-mode-oriented classifier design method is a method capable of directly classifying the matrixed samples. In an actual experiment, the matrix pattern-oriented classifier design method can effectively improve the performance of the vectorization-oriented classifier design method. The reason is mainly shown in three aspects: firstly, the design method of the matrix-oriented pattern classifier can capture more structural information in a single sample, and the matrixing method requires relatively less storage space on the storage space of one pattern; secondly, the design method of the matrix-oriented pattern classifier avoids the operation of converting a single sample into a vector-form sample, and avoids high computational complexity caused by overhigh dimension to a certain extent; third, experiments and theories show that the design method of the vectorization-oriented classifier can be regarded as a special case of the design method of the matrix-mode-oriented classifier, namely, the method for solving the design method of the matrix-mode-oriented classifier is also suitable for the vectorization method.
The Matrix pattern Classifier design method only considers minimizing empirical errors and reducing generalized errors of the models and ignores locally sensitive discrimination information between the models, wherein a more typical linear algorithm is MatMHKS (Matrix-pattern-oriented Ho-Kashyap Classifier). At present, no one has mentioned a relevant method in a matrix pattern classifier design method to solve the defect. To address this deficiency, we propose a new regularization termAnd adding local sensitive discrimination information for the design of the matrix pattern classifier. The design method of the new regularization term comes from local Sensitive Discriminant Analysis (Locality Sensitive Discriminant Analysis). Local sensitive discriminant analysis can capture local discriminant information and local geometric structures. We exploit this advantage of locality sensitive discriminant analysis to design a new regularization termAnd local sensitive discrimination information is introduced for matrixing classification design, so that a new model for the classifier design of a matrix mode is generated. According to the idea of local sensitive discriminant analysis, two non-overlapping weighted neighbor subgraphs are constructedAnd. WhereinIs from the local connection relationship between matrix patterns of the same type,is to reflect the local connection relationship between different types of matrix patterns. We define class subgraphs in matrix modeIs composed ofAnd subgraph between classesIs composed ofTo characterize intra-class patterns as close as possible and inter-class patterns as far apart.
HandleIntroduced into the double-edge matrix type classifier MatMHKS, thereby generating a new classification algorithm lsdmathmks. The LSDMathMHKS can well capture local sensitive discrimination information, so that the classification accuracy of MatMHKS is improved, and the learning stability of the MatMHKS is also improved.
Disclosure of Invention
Aiming at the problem that the existing design method for the matrix pattern-oriented classifier does not consider local sensitive discrimination information between matrix patterns, the solution of the invention is to design a new regularization item on the frame of the original design method for the matrix pattern-oriented classifier to consider the local sensitive discrimination information, and the regularization item is modified into a regularization item suitable for the matrix patterns based on the advantages of local sensitive discrimination analysis, so as to generate a local sensitive discrimination matrix learning model. We applied this framework to our previous work MatMHKS and named LSDMatMHKS. And the LSDMathMHKS is solved by a gradient descent method, and the algorithm can effectively introduce local sensitive discrimination information and improve the accuracy of mode classification and the learning stability. As the model adopts the two-classification technology, the data set with the number of classes N can be converted into N (N-1)/2 two-classification problems, and it can be known that N (N-1)/2 models need to be trained.
The technical scheme adopted by the invention for solving the technical problems is as follows: firstly, a data set is collected, collected samples are converted into a matrix mode so that a later algorithm can process the data set, wherein the data set which is not numerical type is digitized, and the picture data set also needs dimension reduction processing by a traditional dimension reduction algorithm so as to remove noise. Second, using training set to construct sub-graph in and between local classesAnduse ofAndconstructing regularization terms. Then regularizing termIntroducing a matrix-oriented mode classifier MatMHKS to generate a new matrix-oriented mode classification model LSDMatMHKS, training the LSDMatMHKS by using a training set, and solving the model LADMatMHKS to obtain the optimal solution by using a gradient descent method. And then testing the optimal solution by using the test set, and obtaining an optimal decision function. Finally, the obtained optimal decision function is used for calculating the input unknown matrix mode, and the unknown matrix mode is divided according to the output resultAnd (4) class.
The technical scheme adopted by the invention can be further perfected. Within and between said constructed local classesAndthe method of (1) is to construct a local sensitivity weight matrixAndthen is reusedAndto construct a local matrix pattern discriminant relationship, whereinAndis determined by a neighbor method. Constructing regularization terms in said training moduleThe method for making the local intra-class as close as possible and the local inter-class as far as possible from the expression form is that the distance between the patterns in the local intra-class is minimized and the distance between the patterns in the local inter-class is maximized. This method is a classification model of matrix patterns, i.e. the method for vector patterns can be handled as well.
The invention has the beneficial effects that: finding out the local sensitive discrimination relation between the matrix modes by defining a weight matrix of the local sensitive discrimination relation of the matrix modes; by introducing the local sensitive discrimination information, the local patterns of the same type are close to each other as much as possible, and the local sensitive information is introduced into the traditional matrix-mode-oriented classification model, so that the classification accuracy is improved; meanwhile, the overfitting problem of the small sample is improved to a certain extent, and the classification stability and the learning capacity of the model are improved; and the method can directly process whether the image data set or the vector data set.
Drawings
FIG. 1 is a system framework of a local sensitivity discriminant matrix learning machine model according to the present invention.
Detailed Description
The invention will be further described with reference to the following figures and examples: the method of the invention is divided into four steps.
The first step is as follows: and (5) data set acquisition transformation.
Firstly, processing an acquired data set, digitizing the data set if the data set is not digitized, and reducing the dimension of the image data set by using a classical future algorithm after digitizing the image data set so as to facilitate subsequent processing; second converting the acquired data set into a matrix pattern, e.g.Converting it into matrix modeWherein。
The second step is that: and (5) training a model.
Assume a two-class matrix pattern of. The local sensitivity weight matrix is defined by using the training set as follows:
whereinTo controlAndthe relationship between them can be knownAndincluding locally sensitive discrimination information.
2) The conventional matrixing method has a common minimum structural risk frame, and the minimum structural branch frame is as follows:
whereinIs a risk term for the experience that,is a regularization term that aims to control the smoothness and computational complexity of the entire framework. Regularization parameterIs to balanceAndthe relationship (2) of (c). We term regularizationA new matrixing method framework can be obtained by introducing into (6), as shown in formula (7):
3) After introducing the new framework into matrix-oriented classifier MatMHKS, we can obtain LSDMatMHKS, and the objective function is shown in formula (8):
it is composed of,,,,In order to be offset in the amount of the offset,in the form of a matrix pattern, the matrix pattern,for each matrix pattern class index, N is the total number of patterns.
4) Firstly, matrixing the formula (8), and then solving the optimal weight vector of the LSDMatMHKS model by using a gradient descent methodAndrespectively to the pair of formula (8)Andderivation of the deviation, i.e.Andthen respectively orderAndto find a weight vectorAnd. The weight vector obtainedAndas shown in formulas (9) and (10):
And the calculation formula of the iteration termination condition is as follows:
Third, model testing
And after the model is optimized in the second step to obtain the weight vector, testing the obtained weight vector by using the test set to select an optimal decision function.
The fourth step, prediction
And selecting an optimal decision function through the third step to identify the unknown mode. Assume a pattern of unknown classes asThen the decision function is as follows:
Hereinbefore, specific embodiments of the present invention are described with reference to the drawings. It will be understood by those skilled in the art that various changes and substitutions may be made to the specific embodiments of the invention without departing from the spirit and scope of the invention. Such modifications and substitutions are intended to be included within the scope of the present invention as defined by the appended claims.
Results of the experiment
To verify the effectiveness and feasibility of our proposed method, we collected 4 vector data sets from UCI, KEEL, and also collected image data sets Coil-20 and Yale, for a total of 6 data sets to verify our algorithm. The selected data set is shown in table 1, and the dimensions, the number of categories, the scale of the training set and the scale of the test set of the data set are given in table 1. The data sets are divided into a training set and a test set, wherein the proportion of the training set to the test set in each data set is 0.5, and 5 rounds of Monte Carlo cross validation are adopted to obtain the classification accuracy. The setting of the model parameters is set by experiment and manual experience, wherein locally sensitive parameter valuesRegularization term parameter valuesAndinitializing boundary vectorsWeight vectorMaximum number of iterationsMinimum stopping errorIteration step size。
TABLE 1 data set
The parameter settings of the comparison algorithm are as follows:
the original algorithm MatMHKS and the modified algorithm LSDMatMHKS involved use the same parameter settings to facilitate the comparison. The parameters for MatMHKS were set as: regularization term parameter valuesInitializing boundary vectorsWeight vectorMaximum number of iterationsMinimum stopping errorIteration step size。
The results of the experiment are shown in table 2. From the experimental results, the accuracy and the standard deviation of the accuracy of the LSDMatMHKS are better than that of MatMHKS. This verifies the effectiveness and feasibility of our proposed method.
Table 2 data set accuracy (%)
Remarking: the above experimental data are all from the environments of Inter Xeon CPU E5-24072.20 GHZ, 16G RAMDDR3, Windows Server 2012 and Matlab.
Claims (2)
1. A matrix classification model based on local sensitivity discrimination is characterized in that: the method comprises the following specific steps:
1) firstly, collecting a data set: converting the collected samples into a matrix pattern so that the subsequent algorithm can process the samples, wherein the data set which is not numerical type is digitized, and the picture data set also needs to be subjected to dimension reduction processing by using a traditional dimension reduction algorithm so as to remove noise;
2) secondly, constructing partial intra-class and partial inter-class subgraphs A (f) and B (f) by using the training set, and constructing a regularization term R by using A (f) and B (f)LSDEta a (f) - (1-eta) b (f), wherein eta e [0,1]To control the relationship between A (f) and B (f);
3) then the regularization term RLSDIntroducing matrix-mode-oriented classifier MatMHKS to generate a matrix-mode-oriented classification model LSDMatMHKS considering the intra-class relation and the inter-class relation simultaneously, training the LSDMatMHKS by using a training set, and solving the model LSDMatMHKS to obtain the optimal solution by using a gradient descent method, wherein the target function of the LSDMatMHKS is
Wherein,v0b is not less than 0 and is offset, i, j is the number of the basic model, biIs the offset of the ith base model, AiFor the augmented matrix mode, c, η are hyper-parameters,in the form of a matrix pattern, the matrix pattern,for each matrix pattern class index, N is the total number of patterns, Ww,ijAnd Wb,ijRespectively a local intra-class matrix and a local inter-class matrix,
4) then, testing an optimal solution by using the test set, and obtaining an optimal decision function;
5) and finally, calculating the input unknown matrix mode by using the obtained optimal decision function, and classifying the unknown matrix mode according to the output result.
2. The matrix classification model based on locality sensitive discriminant as claimed in claim 1, wherein: the partial intra-class and partial inter-class subgraphs A (f) and B (f) are obtained by passing through a partial sensitivity weight matrix WwAnd WbIs shown in (a) wherein W iswAnd WbRespectively refer to the intra-local-class weight matrix and the inter-local-class weight matrix, WwAnd WbThe calculation of (2) is carried out by using a nearest neighbor method to obtain a current matrix modeNearest neighbor, namely, calculating the Euclidean distance between each sample and other samples, selecting the point with the nearest distance as the nearest neighbor point, and if the mode in the neighbor and the current mode belong to the same class, making the weight matrix W at the momentwThe element value of the corresponding position of (1) is 1, otherwise, it is 0; if the modes in the neighborhood and the current mode belong to different classes, the weight matrix W at the moment is orderedbThe element value of the corresponding position of (1) is 1, otherwise it is 0.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610923346.8A CN106485286B (en) | 2016-10-29 | 2016-10-29 | Matrix classification model based on local sensitivity discrimination |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201610923346.8A CN106485286B (en) | 2016-10-29 | 2016-10-29 | Matrix classification model based on local sensitivity discrimination |
Publications (2)
Publication Number | Publication Date |
---|---|
CN106485286A CN106485286A (en) | 2017-03-08 |
CN106485286B true CN106485286B (en) | 2020-10-20 |
Family
ID=58270426
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201610923346.8A Active CN106485286B (en) | 2016-10-29 | 2016-10-29 | Matrix classification model based on local sensitivity discrimination |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN106485286B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108363717B (en) * | 2017-12-29 | 2021-03-12 | 天津南大通用数据技术股份有限公司 | Data security level identification and detection method and device |
CN111931747B (en) * | 2020-10-10 | 2021-01-29 | 中南大学 | Remote sensing image classification method suitable for multitask iterative learning and memory |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130064901A1 (en) * | 2011-04-18 | 2013-03-14 | Agency For Science, Technology And Research | Gene expression profiling for classifying and treating gastric cancer |
CN102541050A (en) * | 2012-01-05 | 2012-07-04 | 浙江大学 | Chemical process fault diagnosis method based on improved support vector machine |
CN103941244B (en) * | 2014-04-23 | 2016-12-07 | 电子科技大学 | A kind of radar target-range image local optimum subspace identification method |
CN105469034B (en) * | 2015-11-17 | 2018-08-21 | 西安电子科技大学 | Face identification method based on Weighting type distinctive sparse constraint Non-negative Matrix Factorization |
CN105447520A (en) * | 2015-11-23 | 2016-03-30 | 盐城工学院 | Sample classification method based on weighted PTSVM (projection twin support vector machine) |
-
2016
- 2016-10-29 CN CN201610923346.8A patent/CN106485286B/en active Active
Non-Patent Citations (2)
Title |
---|
Credit systems for learning and skills:Current developments;Tony Tait;《LSDA Reports》;20031231;第1-20页 * |
LSDA: Large Scale Detection through Adaptation;Judy Hoffman 等;《MSEE & SMISC》;20141231;第1-9页 * |
Also Published As
Publication number | Publication date |
---|---|
CN106485286A (en) | 2017-03-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110070141B (en) | Network intrusion detection method | |
Xia et al. | Extended $ t $ t: Learning with mixed closed-set and open-set noisy labels | |
Zhu et al. | Iou-uniform r-cnn: Breaking through the limitations of rpn | |
Yin et al. | Joint multi-leaf segmentation, alignment, and tracking for fluorescence plant videos | |
CN112232413B (en) | High-dimensional data feature selection method based on graph neural network and spectral clustering | |
CN111461164B (en) | Sample data set capacity expansion method and model training method | |
CN109886284B (en) | Fraud detection method and system based on hierarchical clustering | |
CN112418387A (en) | Quantum data processing method and apparatus | |
Gao et al. | BLNN: Multiscale Feature Fusion‐Based Bilinear Fine‐Grained Convolutional Neural Network for Image Classification of Wood Knot Defects | |
CN111582315B (en) | Sample data processing method and device and electronic equipment | |
CN104573699A (en) | Trypetid identification method based on medium field intensity magnetic resonance dissection imaging | |
CN111507297B (en) | Radar signal identification method and system based on measurement information matrix | |
CN104616022A (en) | Classification method of near infrared spectrum | |
CN106485286B (en) | Matrix classification model based on local sensitivity discrimination | |
CN109657693B (en) | Classification method based on correlation entropy and transfer learning | |
You et al. | ShiftAddNAS: Hardware-inspired search for more accurate and efficient neural networks | |
Liu et al. | An orthogonal predictive model-based dynamic multi-objective optimization algorithm | |
Jiao et al. | An attention-based feature pyramid network for single-stage small object detection | |
Liu et al. | A weight-incorporated similarity-based clustering ensemble method | |
CN116958809A (en) | Remote sensing small sample target detection method for feature library migration | |
AU2020103440A4 (en) | A method for optimizing the convergence performance of data learning with minimal computational steps | |
CN115511262A (en) | Transformer quality detection method and device | |
Rodrigues et al. | A complex networks approach for data clustering | |
Jun et al. | Two-view correspondence learning via complex information extraction | |
Li et al. | A novel K-means classification method with genetic algorithm |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |