CN103077405A - Bayes classification method based on Fisher discriminant analysis - Google Patents
Bayes classification method based on Fisher discriminant analysis Download PDFInfo
- Publication number
- CN103077405A CN103077405A CN2013100179553A CN201310017955A CN103077405A CN 103077405 A CN103077405 A CN 103077405A CN 2013100179553 A CN2013100179553 A CN 2013100179553A CN 201310017955 A CN201310017955 A CN 201310017955A CN 103077405 A CN103077405 A CN 103077405A
- Authority
- CN
- China
- Prior art keywords
- classification
- formula
- space
- sample
- class
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a bayes classification method based on Fisher discriminant analysis, and the method is characterized in that a transformation matrix is used for transforming an original training sample to be projected into a novel sample space, a classifier conducts the learning classification in the novel sample space after the projection, any two attributes in an original sample attribute set have a given dependency relationship, the attributes of novel samples in the novel sample space after the projection are assumed to be independent from one another, a mode indicated in a measurement space with high dimensionality can be transformed to a mode indicated in a characteristic space with low dimensionality, so that the classification and recognition can be effectively realized, the characteristics capable of reflecting the classification nature can be obtained, classification effects are analyzed and compared through experiments, parametric approximate expression of the distribution of each type training sample set can be obtained.
Description
Technical field
The present invention relates to pattern-recognition and machine intelligence technical field, specifically a kind of bayes classification method based on the Fisher discriminatory analysis.
Background technology
Classification is machine learning, the problem of the association area broad research such as pattern-recognition and artificial intelligence.In recent years, along with continuing to bring out of new technology in the association area, sorting technique has also obtained new development.For different classification problems, sorting technique is varied, such as decision tree classification, support vector machine classification, neural network classification.In numerous sorting techniques, Bayes classifier has been subject to greatly paying attention to.Bayes classifier is based on maximum posteriori criterion, namely utilizes the prior probability of certain object to calculate its posterior probability, and selects to have the class of maximum a posteriori probability as the class under this object.In Bayesian model, model is simulated respectively the class condition joint probability distribution of each class, then uses Bayes' theorem and makes up the posterior probability sorter.But Bayes classifier has stronger restriction, require between the attribute be condition independently, and sorter itself also lacks taking full advantage of the training sample set data message.In the sorter building process and the information of utilizing between class and the class not yet in effect, and that this information is classified just is needed.
This paper in serious analysis on the basis of Bayesian model design feature and structural classification device method, in conjunction with the Fisher linear discriminant analysis, provided a kind of Bayes classifier based on the Fisher linear discriminant analysis.
) classical Bayes classifier
In continuous situation, establish observed value
Be
Dimensional feature vector
, wherein
It is one-dimension random variable.Measurement space
By
Individual state of nature forms:
,
Be
The prior probability of class state,
Be Class-conditionaldensity function,
Expression is accepted
Belong to
Class
Conditional probability, be also referred to as posterior probability.In the classification based on posterior probability.Problem can be described as:
Its physical significance is: under the condition that the proper vector that observation obtains occurs, and the class of the maximum for belonging in all conditions probability of classification, it is minimum to do like this error rate that can make recognition decision, and this criterion is called maximum posteriori criterion.Utilize
Famous Bayesian formula
, notice denominator
Be a constant in comparison expression, through a series of derivation, can be expressed as decision-making formula (1):
This has just consisted of classical Bayes classifier.
For the data set of many reality, normality assumption is normally a kind of more approximate.The probability density function of Multivariate Normal function is
For processing conveniently, first it is carried out log-transformation, then can obtain such as the lower linear decision function
If make
, to all
Set up, then will
Be attributed to
Class.Bayess classification in this case utilizes formula (3) to calculate exactly
Individual discriminant function
, therefrom selecting corresponding to discriminant function is that peaked class is as the result of decision again;
2) Fisher linear discriminant analysis
Fisher linear discriminant analysis (Fisher Linear Discriminant Analysis, FLDA) method is a kind of effective feature extracting method in the pattern-recognition.The Fisher linear discriminant analysis tries hard to find the projecting direction of one group of the best, on these projecting directions, and the sample that belongs to a different category in the differentiation training set that can be best.
Be provided with one group
The training sample of dimension
...,
(
For
Dimension row vector,
), they belong to respectively
Individual different classification, namely wherein size is
Sample set
Belong to classification
. Fisher linear discriminant analysis basic problem to be solved is sought one group of optimum linear conversion exactly
, raw data is projected to new sample space after by linear transformation, raw data is divided better in new space.For determining best projecting direction, need definition following matrix and vector:
Total population scatter matrix is:
Obviously, what scatter matrix was expressed in the class is that sample is to the distance at the interior center of class in the same class, and the size of its value represents the intensity of similar sample.Its value is less, illustrates that similar sample is relatively more concentrated; Scatter matrix is the tolerance of inhomogeneous centre distance between class, and its value is larger, illustrates that the separability of foreign peoples's sample is better.If can be so that the space after projection, in the class in the sample set, sample separation between class can reach our purpose.
The projection process from from higher dimensional space to lower dimensional space, scatter matrix has experienced some conversion between the interior scatter matrix of class and class. and our target is to seek a projecting direction transformation matrix
, can be in some sense so that between the class after the projection in scatter matrix and the class ratio of scatter matrix maximum. the criterion function that for this reason is defined as follows:
Make criterion function for finding the solution
Projective transformation matrix when getting maximum value
, can find the solution with the Lagrange multiplier method, obtain
(10)
(10) formula of solution is for asking general matrix
Eigenvalue problem.Through a series of derivation, can draw criterion function
Solution when getting maximum value
(11)
Can find out that more than the optimization problem that criterion is corresponding is equivalent to the generalized eigenvalue of finding the solution a complexity and the problem of proper vector, this is the core that realizes this classifier algorithm.In case determine transformation matrix
, just can be according to projection equation
(12)
Summary of the invention
Technical assignment of the present invention is to solve the deficiencies in the prior art, and a kind of bayes classification method based on the Fisher discriminatory analysis is provided.
Technical scheme of the present invention realizes in the following manner, utilize transformation matrix, original training sample is carried out conversion, project to new sample space, the new sample space of sorter after projection carries out learning classification, in the former sample property set, may there be certain dependence between any two attributes, after the projection in the new samples space, the attribute of new samples is assumed to separate, can become the pattern that in the lower feature space of dimension, represents to the pattern that represents in the higher measurement space of dimension by conversion, can effectively realize Classification and Identification like this, by experiment classifying quality has been carried out analysis and comparison, thereby obtained reflecting the feature of classification essence, concrete steps are as follows:
1) raw data is carried out normalized;
3) according to formula (5)
The calculated population mean vector;
If make
, to all
Set up, then will
Be attributed to
Class, Bayess classification in this case utilizes formula exactly
Calculate
Individual discriminant function
, therefrom selecting corresponding to discriminant function is that peaked class is as the result of decision again.
Outstanding beneficial effect of the present invention: be from another angle, be devoted to remedy the problem that classical Bayes classifier can not extract information between class, seek the projector space that makes class and class maximum separation by using the Fisher discriminant analysis method, and then with former sample to the separable space projection of maximum, obtain new samples, take the differentiation amount as new attribute, in new samples, carry out learning classification with classical Bayesian Classification Arithmetic again.Experiment shows classical Bayes classifier and Fisher linear discriminant analysis method is combined, and can obtain better classifying quality.
Description of drawings
The improved bayesian algorithm process flow diagram that Fig. 1 is;
The Bayes classifier classification situation map of Fig. 2 classics;
The improved Bayes classifier classification of Fig. 3 situation map.
Embodiment
Below in conjunction with accompanying drawing the bayes classification method based on the Fisher discriminatory analysis of the present invention is described in further detail.
The improvement of Bayes classifier
Although formula
, then
The Bayes classifier of definition is simple and effectively, but its attribute independent hypothesis makes it can't represent dependence between the real world attribute.And from the process of Bayes classifier study, can't effectively use information between class, in order to improve this problem, this paper is in conjunction with the Fisher linear discriminant analysis, proposed a kind of improvement algorithm of the Bayes classifier based on the Fisher linear discriminant analysis.
The main thought of this algorithm is to utilize transformation matrix, and original training sample is carried out conversion, projects to new sample space, and the new sample space of sorter after projection carries out learning classification.In the former sample property set, may have certain dependence between any two attributes, in the new samples space, the attribute of new samples is assumed to separate after the projection.Can become the pattern that in the lower feature space of dimension, represents to the pattern that represents in the higher measurement space of dimension by conversion.Can effectively realize Classification and Identification like this, thereby obtain reflecting the feature of classification essence.The process flow diagram that provides this algorithm according to above-mentioned analysis is as shown in Figure 1:
Experimental result and analysis
The data of this experiment are selected from the CORK_STOPPERS.XLS data set, and concrete data declaration is as shown in the table
This kind division has reached the requirement that strictly separates, and the accuracy of the sorter of estimating out relatively approaches actual accuracy, but can be subject to the impact of deviation and the deviation that test set finite sample number produces of the generation of training set finite sample number.
In addition, this experiment has all been carried out normalized pre-service by following formula to all training samples and test sample book, y=(x-min)/(max-min).Wherein, x is a sample, and y is normalized data, and max and min are respectively maximal value and the minimum value of all training samples being obtained each feature.
With classical Bayes classifier and improved Bayes classifier 25 test sample books of a class and 20 test sample books of b class are classified respectively.Fig. 2 and Fig. 3 and table 2 are seen in result's demonstration.
Test findings is as shown in table 2
Table 2
By relatively can finding out of above test findings, improved Bayes classifier is under the equal test sample book situation identical with starting condition, and the erroneous judgement sample number is less, and classifying quality is better, and accuracy is higher.
Conclusion
This paper article the principle of classification of classical Bayes classifier and improved Bayes classifier, and by experiment classifying quality has been carried out analysis and comparison.Can find out from experimental result, although classical Bayes classifier also can compare effective classification to sample, the more efficient classification performance of improved Bayess classification implement body.Although it is a kind of simply and effectively sorting algorithm that reason is classical Bayes classifier, but its independence assumption makes the dependence relation that exists between attribute in its real data that is beyond expression, namely do not use information between class, acquisition only be a kind of parameterized approximate expression that each classification training sample set is distributed.
Except the disclosed technical characterictic of instructions of the present invention, be the public office technology of those skilled in the art.
Claims (1)
1. bayes classification method based on the Fisher discriminatory analysis, it is characterized in that utilizing transformation matrix, original training sample is carried out conversion, project to new sample space, the new sample space of sorter after projection carries out learning classification, in the former sample property set, may there be certain dependence between any two attributes, after the projection in the new samples space, the attribute of new samples is assumed to separate, can become the pattern that in the lower feature space of dimension, represents to the pattern that represents in the higher measurement space of dimension by conversion, can effectively realize Classification and Identification like this, thereby obtain reflecting the feature of classification essence, by experiment classifying quality carried out analysis and comparison, a kind of parameterized approximate expression that acquisition distributes to each classification training sample set, concrete classifying step is as follows:
1) raw data is carried out normalized;
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2013100179553A CN103077405A (en) | 2013-01-18 | 2013-01-18 | Bayes classification method based on Fisher discriminant analysis |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2013100179553A CN103077405A (en) | 2013-01-18 | 2013-01-18 | Bayes classification method based on Fisher discriminant analysis |
Publications (1)
Publication Number | Publication Date |
---|---|
CN103077405A true CN103077405A (en) | 2013-05-01 |
Family
ID=48153929
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2013100179553A Pending CN103077405A (en) | 2013-01-18 | 2013-01-18 | Bayes classification method based on Fisher discriminant analysis |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN103077405A (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103500342A (en) * | 2013-09-18 | 2014-01-08 | 华南理工大学 | Human behavior recognition method based on accelerometer |
CN105162413A (en) * | 2015-09-08 | 2015-12-16 | 河海大学常州校区 | Method for evaluating performances of photovoltaic system in real time based on working condition identification |
CN106066493A (en) * | 2016-05-24 | 2016-11-02 | 中国石油大学(北京) | Bayes's petrofacies method of discrimination and device |
CN108872819A (en) * | 2018-07-29 | 2018-11-23 | 湖南湖大华龙电气与信息技术有限公司 | Isolator detecting unmanned plane and method based on infrared thermal imagery and visible light |
CN109784356A (en) * | 2018-07-18 | 2019-05-21 | 北京工业大学 | Matrix variables based on Fisher discriminant analysis are limited Boltzmann machine image classification method |
CN113255212A (en) * | 2021-05-17 | 2021-08-13 | 中国南方电网有限责任公司超高压输电公司昆明局 | Model selection method for converter valve cooling system based on PCA and Bayesian classifier |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101216879A (en) * | 2007-12-28 | 2008-07-09 | 西安电子科技大学 | Face identification method based on Fisher-supported vector machine |
CN101650944A (en) * | 2009-09-17 | 2010-02-17 | 浙江工业大学 | Method for distinguishing speakers based on protective kernel Fisher distinguishing method |
-
2013
- 2013-01-18 CN CN2013100179553A patent/CN103077405A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101216879A (en) * | 2007-12-28 | 2008-07-09 | 西安电子科技大学 | Face identification method based on Fisher-supported vector machine |
CN101650944A (en) * | 2009-09-17 | 2010-02-17 | 浙江工业大学 | Method for distinguishing speakers based on protective kernel Fisher distinguishing method |
Non-Patent Citations (2)
Title |
---|
曹玲玲等: "基于Fisher判别分析的贝叶斯分类器", 《计算机工程》 * |
李旭升等: "基于多重判别分析的朴素贝叶斯分类器", 《信息与控制》 * |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103500342A (en) * | 2013-09-18 | 2014-01-08 | 华南理工大学 | Human behavior recognition method based on accelerometer |
CN103500342B (en) * | 2013-09-18 | 2017-01-04 | 华南理工大学 | A kind of Human bodys' response method based on accelerometer |
CN105162413A (en) * | 2015-09-08 | 2015-12-16 | 河海大学常州校区 | Method for evaluating performances of photovoltaic system in real time based on working condition identification |
CN106066493A (en) * | 2016-05-24 | 2016-11-02 | 中国石油大学(北京) | Bayes's petrofacies method of discrimination and device |
CN109784356A (en) * | 2018-07-18 | 2019-05-21 | 北京工业大学 | Matrix variables based on Fisher discriminant analysis are limited Boltzmann machine image classification method |
CN109784356B (en) * | 2018-07-18 | 2021-01-05 | 北京工业大学 | Matrix variable limited Boltzmann machine image classification method based on Fisher discriminant analysis |
CN108872819A (en) * | 2018-07-29 | 2018-11-23 | 湖南湖大华龙电气与信息技术有限公司 | Isolator detecting unmanned plane and method based on infrared thermal imagery and visible light |
CN113255212A (en) * | 2021-05-17 | 2021-08-13 | 中国南方电网有限责任公司超高压输电公司昆明局 | Model selection method for converter valve cooling system based on PCA and Bayesian classifier |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108564129B (en) | Trajectory data classification method based on generation countermeasure network | |
Gong et al. | Twin auxilary classifiers gan | |
He et al. | Triplet-center loss for multi-view 3d object retrieval | |
CN107766850B (en) | Face recognition method based on combination of face attribute information | |
Zhang et al. | Pedestrian detection method based on Faster R-CNN | |
Peng et al. | A new approach for imbalanced data classification based on data gravitation | |
CN105740842B (en) | Unsupervised face identification method based on fast density clustering algorithm | |
CN103077405A (en) | Bayes classification method based on Fisher discriminant analysis | |
Zeng et al. | Fine-grained image retrieval via piecewise cross entropy loss | |
CN110378366A (en) | A kind of cross-domain image classification method based on coupling knowledge migration | |
CN110751027B (en) | Pedestrian re-identification method based on deep multi-instance learning | |
CN105354593B (en) | A kind of threedimensional model sorting technique based on NMF | |
CN109784405A (en) | Cross-module state search method and system based on pseudo label study and semantic consistency | |
CN104732248B (en) | Human body target detection method based on Omega shape facilities | |
CN104156945A (en) | Method for segmenting gray scale image based on multi-objective particle swarm optimization algorithm | |
Xu et al. | Enhancing 2D representation via adjacent views for 3D shape retrieval | |
Lu et al. | Clustering by Sorting Potential Values (CSPV): A novel potential-based clustering method | |
CN103927554A (en) | Image sparse representation facial expression feature extraction system and method based on topological structure | |
Cao et al. | Combining re-sampling with twin support vector machine for imbalanced data classification | |
CN103345621A (en) | Face classification method based on sparse concentration index | |
Lang et al. | Study of face detection algorithm for real-time face detection system | |
Rosa et al. | On the training of artificial neural networks with radial basis function using optimum-path forest clustering | |
Xia et al. | Granular-ball computing: an efficient, robust, and interpretable adaptive multi-granularity representation and computation method | |
Li et al. | A novel semantic approach for multi-ethnic face recognition | |
Leng et al. | A powerful 3D model classification mechanism based on fusing multi-graph |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
WD01 | Invention patent application deemed withdrawn after publication | ||
WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20130501 |