CN103077405A - Bayes classification method based on Fisher discriminant analysis - Google Patents

Bayes classification method based on Fisher discriminant analysis Download PDF

Info

Publication number
CN103077405A
CN103077405A CN2013100179553A CN201310017955A CN103077405A CN 103077405 A CN103077405 A CN 103077405A CN 2013100179553 A CN2013100179553 A CN 2013100179553A CN 201310017955 A CN201310017955 A CN 201310017955A CN 103077405 A CN103077405 A CN 103077405A
Authority
CN
China
Prior art keywords
classification
formula
space
sample
class
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN2013100179553A
Other languages
Chinese (zh)
Inventor
曹玲玲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Electronic Information Industry Co Ltd
Original Assignee
Inspur Electronic Information Industry Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Electronic Information Industry Co Ltd filed Critical Inspur Electronic Information Industry Co Ltd
Priority to CN2013100179553A priority Critical patent/CN103077405A/en
Publication of CN103077405A publication Critical patent/CN103077405A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a bayes classification method based on Fisher discriminant analysis, and the method is characterized in that a transformation matrix is used for transforming an original training sample to be projected into a novel sample space, a classifier conducts the learning classification in the novel sample space after the projection, any two attributes in an original sample attribute set have a given dependency relationship, the attributes of novel samples in the novel sample space after the projection are assumed to be independent from one another, a mode indicated in a measurement space with high dimensionality can be transformed to a mode indicated in a characteristic space with low dimensionality, so that the classification and recognition can be effectively realized, the characteristics capable of reflecting the classification nature can be obtained, classification effects are analyzed and compared through experiments, parametric approximate expression of the distribution of each type training sample set can be obtained.

Description

A kind of bayes classification method based on the Fisher discriminatory analysis
Technical field
The present invention relates to pattern-recognition and machine intelligence technical field, specifically a kind of bayes classification method based on the Fisher discriminatory analysis.
Background technology
Classification is machine learning, the problem of the association area broad research such as pattern-recognition and artificial intelligence.In recent years, along with continuing to bring out of new technology in the association area, sorting technique has also obtained new development.For different classification problems, sorting technique is varied, such as decision tree classification, support vector machine classification, neural network classification.In numerous sorting techniques, Bayes classifier has been subject to greatly paying attention to.Bayes classifier is based on maximum posteriori criterion, namely utilizes the prior probability of certain object to calculate its posterior probability, and selects to have the class of maximum a posteriori probability as the class under this object.In Bayesian model, model is simulated respectively the class condition joint probability distribution of each class, then uses Bayes' theorem and makes up the posterior probability sorter.But Bayes classifier has stronger restriction, require between the attribute be condition independently, and sorter itself also lacks taking full advantage of the training sample set data message.In the sorter building process and the information of utilizing between class and the class not yet in effect, and that this information is classified just is needed.
This paper in serious analysis on the basis of Bayesian model design feature and structural classification device method, in conjunction with the Fisher linear discriminant analysis, provided a kind of Bayes classifier based on the Fisher linear discriminant analysis.
) classical Bayes classifier
In continuous situation, establish observed value
Figure 352810DEST_PATH_IMAGE001
Be
Figure 334541DEST_PATH_IMAGE002
Dimensional feature vector
Figure 966511DEST_PATH_IMAGE003
, wherein
Figure 742706DEST_PATH_IMAGE004
It is one-dimension random variable.Measurement space
Figure 972043DEST_PATH_IMAGE005
By
Figure 304936DEST_PATH_IMAGE006
Individual state of nature forms:
Figure 40679DEST_PATH_IMAGE007
,
Figure 66404DEST_PATH_IMAGE008
Be
Figure 520388DEST_PATH_IMAGE009
The prior probability of class state,
Figure 391392DEST_PATH_IMAGE010
Be Class-conditionaldensity function, Expression is accepted
Figure 443848DEST_PATH_IMAGE001
Belong to Class
Figure 62752DEST_PATH_IMAGE012
Conditional probability, be also referred to as posterior probability.In the classification based on posterior probability.Problem can be described as:
If
Figure 835405DEST_PATH_IMAGE013
, then
Figure 186621DEST_PATH_IMAGE014
（1）
Its physical significance is: under the condition that the proper vector that observation obtains occurs, and the class of the maximum for belonging in all conditions probability of classification, it is minimum to do like this error rate that can make recognition decision, and this criterion is called maximum posteriori criterion.Utilize
Famous Bayesian formula
Figure 549951DEST_PATH_IMAGE015
, notice denominator
Figure 28337DEST_PATH_IMAGE016
Be a constant in comparison expression, through a series of derivation, can be expressed as decision-making formula (1):
If
Figure 593179DEST_PATH_IMAGE017
, then
Figure 131608DEST_PATH_IMAGE014
（2）
This has just consisted of classical Bayes classifier.
For the data set of many reality, normality assumption is normally a kind of more approximate.The probability density function of Multivariate Normal function is
Figure 781901DEST_PATH_IMAGE018
For processing conveniently, first it is carried out log-transformation, then can obtain such as the lower linear decision function
Figure 250928DEST_PATH_IMAGE019
Figure 686589DEST_PATH_IMAGE020
　　 （3）
Wherein
Figure 382537DEST_PATH_IMAGE021
If make
Figure 536438DEST_PATH_IMAGE022
, to all Set up, then will
Figure 896061DEST_PATH_IMAGE001
Be attributed to
Figure 759980DEST_PATH_IMAGE012
Class.Bayess classification in this case utilizes formula (3) to calculate exactly
Figure 135598DEST_PATH_IMAGE006
Individual discriminant function
Figure 8745DEST_PATH_IMAGE019
, therefrom selecting corresponding to discriminant function is that peaked class is as the result of decision again;
2) Fisher linear discriminant analysis
Fisher linear discriminant analysis (Fisher Linear Discriminant Analysis, FLDA) method is a kind of effective feature extracting method in the pattern-recognition.The Fisher linear discriminant analysis tries hard to find the projecting direction of one group of the best, on these projecting directions, and the sample that belongs to a different category in the differentiation training set that can be best.
Be provided with one group
Figure 153418DEST_PATH_IMAGE002
The training sample of dimension
Figure 450889DEST_PATH_IMAGE024
...,
Figure 313803DEST_PATH_IMAGE025
(
Figure 725061DEST_PATH_IMAGE026
For
Figure 989821DEST_PATH_IMAGE002
Dimension row vector,
Figure 398805DEST_PATH_IMAGE027
Figure 732704DEST_PATH_IMAGE002
), they belong to respectively
Figure 698385DEST_PATH_IMAGE006
Individual different classification, namely wherein size is
Figure 69848DEST_PATH_IMAGE028
Sample set
Figure 462784DEST_PATH_IMAGE029
Belong to classification
Figure 549557DEST_PATH_IMAGE030
. Fisher linear discriminant analysis basic problem to be solved is sought one group of optimum linear conversion exactly
Figure 53351DEST_PATH_IMAGE031
, raw data is projected to new sample space after by linear transformation, raw data is divided better in new space.For determining best projecting direction, need definition following matrix and vector:
The class mean vector:
Figure 276391DEST_PATH_IMAGE032
(4)
The population mean vector:
Figure 840227DEST_PATH_IMAGE033
(5)
Total population scatter matrix is:
Figure 148718DEST_PATH_IMAGE034
=
Figure 456202DEST_PATH_IMAGE035
（6）
Scatter matrix in the class:
Figure 612377DEST_PATH_IMAGE036
Figure 616890DEST_PATH_IMAGE037
(7)
Scatter matrix between class:
Figure 163409DEST_PATH_IMAGE038
=
Figure 195956DEST_PATH_IMAGE039
(8)
Obviously, what scatter matrix was expressed in the class is that sample is to the distance at the interior center of class in the same class, and the size of its value represents the intensity of similar sample.Its value is less, illustrates that similar sample is relatively more concentrated; Scatter matrix is the tolerance of inhomogeneous centre distance between class, and its value is larger, illustrates that the separability of foreign peoples's sample is better.If can be so that the space after projection, in the class in the sample set, sample separation between class can reach our purpose.
The projection process from from higher dimensional space to lower dimensional space, scatter matrix has experienced some conversion between the interior scatter matrix of class and class. and our target is to seek a projecting direction transformation matrix
Figure 393588DEST_PATH_IMAGE040
, can be in some sense so that between the class after the projection in scatter matrix and the class ratio of scatter matrix maximum. the criterion function that for this reason is defined as follows:
Figure 299227DEST_PATH_IMAGE041
Figure 582310DEST_PATH_IMAGE042
（9）
Make criterion function for finding the solution
Figure 293914DEST_PATH_IMAGE043
Projective transformation matrix when getting maximum value
Figure 362364DEST_PATH_IMAGE040
, can find the solution with the Lagrange multiplier method, obtain
Figure 691102DEST_PATH_IMAGE044
(10)
(10) formula of solution is for asking general matrix Eigenvalue problem.Through a series of derivation, can draw criterion function
Figure 648880DEST_PATH_IMAGE043
Solution when getting maximum value
Figure 571836DEST_PATH_IMAGE046
(11)
Can find out that more than the optimization problem that criterion is corresponding is equivalent to the generalized eigenvalue of finding the solution a complexity and the problem of proper vector, this is the core that realizes this classifier algorithm.In case determine transformation matrix
Figure 68545DEST_PATH_IMAGE040
, just can be according to projection equation
Figure 811374DEST_PATH_IMAGE047
(12)
With former sample set to
Figure 864780DEST_PATH_IMAGE040
Projection obtains new sample set
Figure 829194DEST_PATH_IMAGE048
Summary of the invention
Technical assignment of the present invention is to solve the deficiencies in the prior art, and a kind of bayes classification method based on the Fisher discriminatory analysis is provided.
Technical scheme of the present invention realizes in the following manner, utilize transformation matrix, original training sample is carried out conversion, project to new sample space, the new sample space of sorter after projection carries out learning classification, in the former sample property set, may there be certain dependence between any two attributes, after the projection in the new samples space, the attribute of new samples is assumed to separate, can become the pattern that in the lower feature space of dimension, represents to the pattern that represents in the higher measurement space of dimension by conversion, can effectively realize Classification and Identification like this, by experiment classifying quality has been carried out analysis and comparison, thereby obtained reflecting the feature of classification essence, concrete steps are as follows:
1) raw data is carried out normalized;
2) according to formula (4)
Figure 493875DEST_PATH_IMAGE032
The compute classes mean vector;
3) according to formula (5) The calculated population mean vector;
4) according to formula 97):
Figure 768047DEST_PATH_IMAGE036
Scatter matrix in the compute classes;
5) according to formula (8)
Figure 504108DEST_PATH_IMAGE038
=
Figure 221528DEST_PATH_IMAGE039
Scatter matrix between compute classes;
6) according to formula (11)
Figure 803688DEST_PATH_IMAGE046
The computational transformation matrix;
7) utilize formula (12) With former sample set to
Figure 378206DEST_PATH_IMAGE040
Projection obtains new sample set
Figure 835120DEST_PATH_IMAGE048
8) to the new samples collection according to formula (3)
Figure 971703DEST_PATH_IMAGE049
Carry out learning classification
Wherein
Figure 827532DEST_PATH_IMAGE050
If make
Figure 945530DEST_PATH_IMAGE051
, to all
Figure 637543DEST_PATH_IMAGE052
Set up, then will
Figure 827084DEST_PATH_IMAGE001
Be attributed to
Figure 288153DEST_PATH_IMAGE012
Class, Bayess classification in this case utilizes formula exactly
Figure 636439DEST_PATH_IMAGE049
Calculate
Figure 815747DEST_PATH_IMAGE006
Individual discriminant function , therefrom selecting corresponding to discriminant function is that peaked class is as the result of decision again.
Outstanding beneficial effect of the present invention: be from another angle, be devoted to remedy the problem that classical Bayes classifier can not extract information between class, seek the projector space that makes class and class maximum separation by using the Fisher discriminant analysis method, and then with former sample to the separable space projection of maximum, obtain new samples, take the differentiation amount as new attribute, in new samples, carry out learning classification with classical Bayesian Classification Arithmetic again.Experiment shows classical Bayes classifier and Fisher linear discriminant analysis method is combined, and can obtain better classifying quality.
Description of drawings
The improved bayesian algorithm process flow diagram that Fig. 1 is;
The Bayes classifier classification situation map of Fig. 2 classics;
The improved Bayes classifier classification of Fig. 3 situation map.
Embodiment
Below in conjunction with accompanying drawing the bayes classification method based on the Fisher discriminatory analysis of the present invention is described in further detail.
The improvement of Bayes classifier
Although formula
Figure 858976DEST_PATH_IMAGE054
, then
Figure 459721DEST_PATH_IMAGE055
The Bayes classifier of definition is simple and effectively, but its attribute independent hypothesis makes it can't represent dependence between the real world attribute.And from the process of Bayes classifier study, can't effectively use information between class, in order to improve this problem, this paper is in conjunction with the Fisher linear discriminant analysis, proposed a kind of improvement algorithm of the Bayes classifier based on the Fisher linear discriminant analysis.
The main thought of this algorithm is to utilize transformation matrix, and original training sample is carried out conversion, projects to new sample space, and the new sample space of sorter after projection carries out learning classification.In the former sample property set, may have certain dependence between any two attributes, in the new samples space, the attribute of new samples is assumed to separate after the projection.Can become the pattern that in the lower feature space of dimension, represents to the pattern that represents in the higher measurement space of dimension by conversion.Can effectively realize Classification and Identification like this, thereby obtain reflecting the feature of classification essence.The process flow diagram that provides this algorithm according to above-mentioned analysis is as shown in Figure 1:
Experimental result and analysis
The data of this experiment are selected from the CORK_STOPPERS.XLS data set, and concrete data declaration is as shown in the table
This kind division has reached the requirement that strictly separates, and the accuracy of the sorter of estimating out relatively approaches actual accuracy, but can be subject to the impact of deviation and the deviation that test set finite sample number produces of the generation of training set finite sample number.
In addition, this experiment has all been carried out normalized pre-service by following formula to all training samples and test sample book, y=(x-min)/(max-min).Wherein, x is a sample, and y is normalized data, and max and min are respectively maximal value and the minimum value of all training samples being obtained each feature.
With classical Bayes classifier and improved Bayes classifier 25 test sample books of a class and 20 test sample books of b class are classified respectively.Fig. 2 and Fig. 3 and table 2 are seen in result's demonstration.
Test findings is as shown in table 2
Figure 392091DEST_PATH_IMAGE058
Table 2
By relatively can finding out of above test findings, improved Bayes classifier is under the equal test sample book situation identical with starting condition, and the erroneous judgement sample number is less, and classifying quality is better, and accuracy is higher.
Conclusion
This paper article the principle of classification of classical Bayes classifier and improved Bayes classifier, and by experiment classifying quality has been carried out analysis and comparison.Can find out from experimental result, although classical Bayes classifier also can compare effective classification to sample, the more efficient classification performance of improved Bayess classification implement body.Although it is a kind of simply and effectively sorting algorithm that reason is classical Bayes classifier, but its independence assumption makes the dependence relation that exists between attribute in its real data that is beyond expression, namely do not use information between class, acquisition only be a kind of parameterized approximate expression that each classification training sample set is distributed.
Except the disclosed technical characterictic of instructions of the present invention, be the public office technology of those skilled in the art.

Claims (1)

1. bayes classification method based on the Fisher discriminatory analysis, it is characterized in that utilizing transformation matrix, original training sample is carried out conversion, project to new sample space, the new sample space of sorter after projection carries out learning classification, in the former sample property set, may there be certain dependence between any two attributes, after the projection in the new samples space, the attribute of new samples is assumed to separate, can become the pattern that in the lower feature space of dimension, represents to the pattern that represents in the higher measurement space of dimension by conversion, can effectively realize Classification and Identification like this, thereby obtain reflecting the feature of classification essence, by experiment classifying quality carried out analysis and comparison, a kind of parameterized approximate expression that acquisition distributes to each classification training sample set, concrete classifying step is as follows:
1) raw data is carried out normalized;
2) according to formula (4)
Figure 2013100179553100001DEST_PATH_IMAGE001
The compute classes mean vector;
3) according to formula (5)
Figure 33108DEST_PATH_IMAGE002
The calculated population mean vector;
4) according to formula 97):
Figure 2013100179553100001DEST_PATH_IMAGE003
Figure 826621DEST_PATH_IMAGE004
Scatter matrix in the compute classes;
5) according to formula (8)
Figure DEST_PATH_IMAGE005
= Scatter matrix between compute classes;
6) according to formula (11)
Figure DEST_PATH_IMAGE007
The computational transformation matrix;
7) utilize formula (12)
Figure 764194DEST_PATH_IMAGE008
With former sample set to
Figure DEST_PATH_IMAGE009
Projection obtains new sample set
Figure 891419DEST_PATH_IMAGE010
8) to the new samples collection according to formula (3)
Figure DEST_PATH_IMAGE011
Carry out learning classification
Wherein
Figure 539438DEST_PATH_IMAGE012
If make
Figure DEST_PATH_IMAGE013
, to all
Figure 156233DEST_PATH_IMAGE014
Set up, then will
Figure DEST_PATH_IMAGE015
Be attributed to
Figure 457246DEST_PATH_IMAGE016
Class, Bayess classification in this case utilizes formula exactly
Figure 794686DEST_PATH_IMAGE011
Calculate
Figure DEST_PATH_IMAGE017
Individual discriminant function
Figure 562791DEST_PATH_IMAGE018
, therefrom selecting corresponding to discriminant function is that peaked class is as the result of decision again.
CN2013100179553A 2013-01-18 2013-01-18 Bayes classification method based on Fisher discriminant analysis Pending CN103077405A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2013100179553A CN103077405A (en) 2013-01-18 2013-01-18 Bayes classification method based on Fisher discriminant analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2013100179553A CN103077405A (en) 2013-01-18 2013-01-18 Bayes classification method based on Fisher discriminant analysis

Publications (1)

Publication Number Publication Date
CN103077405A true CN103077405A (en) 2013-05-01

Family

ID=48153929

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2013100179553A Pending CN103077405A (en) 2013-01-18 2013-01-18 Bayes classification method based on Fisher discriminant analysis

Country Status (1)

Country Link
CN (1) CN103077405A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103500342A (en) * 2013-09-18 2014-01-08 华南理工大学 Human behavior recognition method based on accelerometer
CN105162413A (en) * 2015-09-08 2015-12-16 河海大学常州校区 Method for evaluating performances of photovoltaic system in real time based on working condition identification
CN106066493A (en) * 2016-05-24 2016-11-02 中国石油大学(北京) Bayes's petrofacies method of discrimination and device
CN108872819A (en) * 2018-07-29 2018-11-23 湖南湖大华龙电气与信息技术有限公司 Isolator detecting unmanned plane and method based on infrared thermal imagery and visible light
CN109784356A (en) * 2018-07-18 2019-05-21 北京工业大学 Matrix variables based on Fisher discriminant analysis are limited Boltzmann machine image classification method
CN113255212A (en) * 2021-05-17 2021-08-13 中国南方电网有限责任公司超高压输电公司昆明局 Model selection method for converter valve cooling system based on PCA and Bayesian classifier

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216879A (en) * 2007-12-28 2008-07-09 西安电子科技大学 Face identification method based on Fisher-supported vector machine
CN101650944A (en) * 2009-09-17 2010-02-17 浙江工业大学 Method for distinguishing speakers based on protective kernel Fisher distinguishing method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216879A (en) * 2007-12-28 2008-07-09 西安电子科技大学 Face identification method based on Fisher-supported vector machine
CN101650944A (en) * 2009-09-17 2010-02-17 浙江工业大学 Method for distinguishing speakers based on protective kernel Fisher distinguishing method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
曹玲玲等: "基于Fisher判别分析的贝叶斯分类器", 《计算机工程》 *
李旭升等: "基于多重判别分析的朴素贝叶斯分类器", 《信息与控制》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103500342A (en) * 2013-09-18 2014-01-08 华南理工大学 Human behavior recognition method based on accelerometer
CN103500342B (en) * 2013-09-18 2017-01-04 华南理工大学 A kind of Human bodys' response method based on accelerometer
CN105162413A (en) * 2015-09-08 2015-12-16 河海大学常州校区 Method for evaluating performances of photovoltaic system in real time based on working condition identification
CN106066493A (en) * 2016-05-24 2016-11-02 中国石油大学(北京) Bayes's petrofacies method of discrimination and device
CN109784356A (en) * 2018-07-18 2019-05-21 北京工业大学 Matrix variables based on Fisher discriminant analysis are limited Boltzmann machine image classification method
CN109784356B (en) * 2018-07-18 2021-01-05 北京工业大学 Matrix variable limited Boltzmann machine image classification method based on Fisher discriminant analysis
CN108872819A (en) * 2018-07-29 2018-11-23 湖南湖大华龙电气与信息技术有限公司 Isolator detecting unmanned plane and method based on infrared thermal imagery and visible light
CN113255212A (en) * 2021-05-17 2021-08-13 中国南方电网有限责任公司超高压输电公司昆明局 Model selection method for converter valve cooling system based on PCA and Bayesian classifier

Similar Documents

Publication Publication Date Title
CN108564129B (en) Trajectory data classification method based on generation countermeasure network
Gong et al. Twin auxilary classifiers gan
He et al. Triplet-center loss for multi-view 3d object retrieval
CN107766850B (en) Face recognition method based on combination of face attribute information
Zhang et al. Pedestrian detection method based on Faster R-CNN
Peng et al. A new approach for imbalanced data classification based on data gravitation
CN105740842B (en) Unsupervised face identification method based on fast density clustering algorithm
CN103077405A (en) Bayes classification method based on Fisher discriminant analysis
Zeng et al. Fine-grained image retrieval via piecewise cross entropy loss
CN110378366A (en) A kind of cross-domain image classification method based on coupling knowledge migration
CN110751027B (en) Pedestrian re-identification method based on deep multi-instance learning
CN105354593B (en) A kind of threedimensional model sorting technique based on NMF
CN109784405A (en) Cross-module state search method and system based on pseudo label study and semantic consistency
CN104732248B (en) Human body target detection method based on Omega shape facilities
CN104156945A (en) Method for segmenting gray scale image based on multi-objective particle swarm optimization algorithm
Xu et al. Enhancing 2D representation via adjacent views for 3D shape retrieval
Lu et al. Clustering by Sorting Potential Values (CSPV): A novel potential-based clustering method
CN103927554A (en) Image sparse representation facial expression feature extraction system and method based on topological structure
Cao et al. Combining re-sampling with twin support vector machine for imbalanced data classification
CN103345621A (en) Face classification method based on sparse concentration index
Lang et al. Study of face detection algorithm for real-time face detection system
Rosa et al. On the training of artificial neural networks with radial basis function using optimum-path forest clustering
Xia et al. Granular-ball computing: an efficient, robust, and interpretable adaptive multi-granularity representation and computation method
Li et al. A novel semantic approach for multi-ethnic face recognition
Leng et al. A powerful 3D model classification mechanism based on fusing multi-graph

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20130501