CN107357761A - A kind of minimal error entropy computational methods of quantization - Google Patents
A kind of minimal error entropy computational methods of quantization Download PDFInfo
- Publication number
- CN107357761A CN107357761A CN201710509553.3A CN201710509553A CN107357761A CN 107357761 A CN107357761 A CN 107357761A CN 201710509553 A CN201710509553 A CN 201710509553A CN 107357761 A CN107357761 A CN 107357761A
- Authority
- CN
- China
- Prior art keywords
- quantization
- coefficient
- algorithm
- computational methods
- minimal error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/15—Correlation function computation including computation of convolution operations
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Complex Calculations (AREA)
Abstract
The present invention discloses a kind of minimal error entropy computational methods of quantization, it is applicable not only under the hypothesis of Gaussian noise, contain non-gaussian in data, in the case of extraordinary noise even multimodal noise, relatively good performance can be reached, but the computation complexity of the algorithm is higher, with data volume (N) increase, its complexity can be into N2Increase, well solve MEE algorithm complexes it is higher the shortcomings that, while MEE algorithm complexes are significantly reduced, it is also ensured that the precision of algorithm, be highly suitable for applying in practical problem.
Description
【Technical field】
The invention belongs to machine learning field, it is related to a kind of minimal error entropy computational methods of quantization.
【Background technology】
Machine learning (Machine Learning, ML) is a multi-field cross discipline, be related to probability theory, statistics,
The multi-door subjects such as Approximation Theory, convextiry analysis, algorithm complex theory.Specialize in the study that the mankind were simulated or realized to computer how
Behavior, to obtain new knowledge or skills, reorganize the existing structure of knowledge and be allowed to constantly improve the performance of itself.
It is required for a cost function to carry out the noise model of fitting data for different machine learning algorithms, wherein most normal
The cost function seen is least mean-square error (MSE) criterion, its advantage be calculate it is simple, but be only applicable to Gaussian noise it is assumed that
Gaussian noise is comprised only in data.In practical problem, the hypothesis of Gaussian noise is difficult to meet, therefore MSE criterions are in reality
Effect in problem is not usually highly desirable.
Except most classical MSE criterions, also many modified hydrothermal process, such as information theory is learnt into (ITL) and is applied to generation
In the study of valency function, wherein more commonly used is maximum cross-correlation entropy (MCC) criterion and minimal error entropy (MEE), MCC's is excellent
Point is that computation complexity is relatively low, fine for the exceptional value robust performance that contains in noise.The advantages of MEE is that its cost function is
Learnt according to initial data, not only robustness is good, and assumes that even multimodal noise is assumed for other non-Gaussian noises
Effect it is all fine, be that algorithm complex is too high the shortcomings that MEE, with the increase of data volume, the used time of the algorithm can increase greatly
Add.
【The content of the invention】
It is an object of the invention to provide a kind of minimal error entropy computational methods of quantization, have good universality, fit
Close and be applied under the hypothesis of noise present in the practical applications such as non-Gaussian noise, while there is the characteristics of low algorithm complex.
To reach above-mentioned purpose, present invention employs following technical scheme：
A kind of minimal error entropy computational methods of quantization, iterative process are as follows：It is defeated for a linear system, legacy data
It is X to enter, and data volume size is N, and the output of data is Y, and the coefficient of combination of the linear system is W*；Its relation is Y=W* TX；
The step of minimal error entropy computational methods of quantization learn the system be：
Coefficient of combination W is randomly derived first, then systematic error is E=Y-W* TX, error vector E dimension are N, Ran Houliang
Change error vector E and obtain point of quantification β, the dimension of quantization threshold ε, β vector is M, in wherein point of quantification β corresponding to each system
Point of quantification number is α；Gaussian kernel function is applied at point of quantification, i.e.,：The coefficient sets of each kernel function
Into vector be α, be superimposed to obtain corresponding cost function according to the gaussian kernel function at point of quantification, you can obtain iteration once
Coefficient of combination W afterwards, then the optimal coefficient of combination of the algorithm is obtained by the method for loop iteration.
Further, the size of the core width cs of the gaussian kernel function is according to the requirement unrestricted choice of user.
Further, quantization threshold ε=λ σ, its coefficient lambda are the parameters of artificial selection, and the size of parameter controls quantization deutomerite
The number M of point.
Beneficial effects of the present invention are embodied in：
A kind of minimal error entropy computational methods of quantization of the present invention, the algorithm have good universality, are adapted to application
Under the noise present in the practical applications such as non-Gaussian noise is assumed, while there is the characteristics of low algorithm complex, solve most
The characteristics of under small mean-square error criteria to the not robust and high minimal error entropy criterion algorithm complex of non-Gaussian noise, have compared with
For important Research Significance and extensive engineering application value.Present invention incorporates maximum cross-correlation entropy criterion and minimal error entropy
The advantages of criterion, in each iterative process, after obtained error vector is quantified, it can greatly reduce computation complexity,
The method quantified simultaneously can also keep the performance of original MEE criterions.
【Brief description of the drawings】
Fig. 1 is the circle of equal altitudes of algorithms of different performance surface under two-dimensional linear system；
(a) circle of equal altitudes of algorithm performance surface under two-dimensional linear system
(b) circle of equal altitudes of MCC algorithms performance surface under two-dimensional linear system
(c) circle of equal altitudes of MEE algorithms performance surface under two-dimensional linear system
(d) circle of equal altitudes of QMEE algorithms performance surface under two-dimensional linear system
Fig. 2 is the inventive method and traditional minimal error entropy algorithm calculates the change of time when with data volume increase
Curved surface.
【Embodiment】
The present invention will be further described below in conjunction with the accompanying drawings.
In the present invention, carried algorithm and some tradition are illustrated using the example of simple linear regression in machine learning
The comparison of algorithm.The model of linear regression isInput data is X ∈ Rd×N, the wherein dimension of input data is d,
The number of sample is N, and output data is y ∈ R1×N.The purpose of linear regression is sought to from input data and output data learning
Obtain the coefficient w of linear system*∈Rd×1.Some noises can be mixed with general output data, this just needs different algorithms.Such as
MSE, MCC and MEE etc..
Minimal error entropy (QMEE) algorithm of quantization
For a two-dimentional linear system, it is assumed that the coefficient of its system is w*=[a, b]T, input data is X ∈ Rd×N,
So data dimension d=2, it is assumed that sample size N=200.Then preferable output data isWherein T is transposition
Operator, v are measurement noises.
For this particular problem, for a kind of minimal error entropy computational methods of quantization, algorithm is realized as follows：
1. it is w=[0,0] to initialize coefficient of combinationT；
2. obtain error e=w of systemTX-y, error vector e dimension are N；
3. the error vector after pair error vector is quantified is β, the data point number after quantization is α, amount
Change coefficient ε=λ σ.α and β dimension is all M；
4. trying to achieve cost function is
5. the coefficient of combination w for the cost function renewal learning tried to achieve according to step 4；
6. return to step 2.
The method wherein quantified is mainly reflected in step 3 and 4.Algorithm obtains optimal solution w by multiple study.The calculation
Method can save the time of calculating compared with traditional minimal error entropy algorithm, and computation complexity is accurate between maximum cross-correlation entropy
Then between (MCC) and minimal error entropy algorithm (MEE).The performance of algorithm is substantially protected with original minimal error entropy algorithm simultaneously
Hold consistent.
In place of showing the advantage of the present invention, some traditional algorithms (MSE, MCC, MEE) under simulated environment are given
With algorithm effect comparison diagram proposed by the invention, 1, Fig. 1,2 are shown in Table.
Table 1
Table 1 illustrates traditional least-mean-square error algorithm (MSE), maximum cross-correlation entropy algorithm (MCC), minimal error entropy
Algorithm (MEE) and a kind of minimal error entropy computational methods (QMEE) of quantization proposed by the invention are under different noises hypothesis
Performance and the comparison for calculating the time.Case1, case2, case3 and case4 in table represent multimodal symmetrical noise respectively, more
Peak asymmetrical noise, binomial noise and unimodal Gaussian noise, and all situations are all mixed with extraordinary noise.The data from table
It can be seen that performance is all poor in all cases for MSE algorithms, the noise of complexity can not be widely suitable for by having reacted MSE algorithms
Under assuming that；The performance of QMEE algorithms substantially maintains unanimously with MEE, does not almost lose, and in some cases, performance can be with
More than MEE, its performance is substantially between MCC and MEE；The QMEE calculating time is far below traditional MEE algorithms, it was demonstrated that uses
The method of quantization is greatly reduced the calculating time of original algorithm.
Fig. 1 illustrates the contour map of performance surface of the above-mentioned algorithms of different under case1 noises, and circle represents in figure
The true coefficient of linear system, cross represent the optimal solution that algorithms of different is tried to achieve, therefrom it can be seen that traditional MSE and
MCC methods are all Biased estimator under the influence of noise, and MEE and QMEE proposed by the present invention are nearly all unbiased esti-mators, are also said
Bright QMEE is maintain original algorithm the advantages of during quantization.
Fig. 2 illustrates original MEE and QMEE proposed by the present invention under case1 noises, during with sample size increase, two
The time of kind algorithm single step iteration, wherein lower curve represent QMEE, and top curve represents MEE.Therefrom it can be seen that with
The increase of data sample, the calculating time of original MEE algorithms sharply increases, and liftings of the QMEE to primal algorithm complexity is more
Substantially.
Above content is to combine specific preferred embodiment further description made for the present invention, it is impossible to is assert
The embodiment of the present invention is only limitted to this, for general technical staff of the technical field of the invention, is not taking off
On the premise of from present inventive concept, some simple deduction or replace can also be made, should all be considered as belonging to the present invention by institute
Claims of submission determine scope of patent protection.
Claims (3)
1. the minimal error entropy computational methods of a kind of quantization, it is characterised in that iterative process is as follows：It is former for a linear system
It is X to have data input, and data volume size is N, and the output of data is Y, and the coefficient of combination of the linear system is W*；Its relation is
Y=W* TX；
The step of minimal error entropy computational methods of quantization learn the system be：
Coefficient of combination W is randomly derived first, then systematic error is E=Y-W* TX, error vector E dimension are N, then quantify to miss
Difference vector E obtains point of quantification β, and the dimension of quantization threshold ε, β vector is M, is quantified in wherein point of quantification β corresponding to each system
Point number is α；Gaussian kernel function is applied at point of quantification, i.e.,：The coefficient composition of each kernel function
Vector is α, is superimposed to obtain corresponding cost function according to the gaussian kernel function at point of quantification, you can after obtaining iteration once
Coefficient of combination W, then the optimal coefficient of combination of the algorithm is obtained by the method for loop iteration.
A kind of 2. minimal error entropy computational methods of quantization according to claim 1, it is characterised in that：The gaussian kernel function
Core width cs size according to the requirement unrestricted choice of user.
A kind of 3. minimal error entropy computational methods of quantization according to claim 1, it is characterised in that：Quantization threshold ε=λ σ,
Its coefficient lambda is the parameter of artificial selection, and the size of parameter controls the number M for quantifying posterior nodal point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710509553.3A CN107357761A (en) | 2017-06-28 | 2017-06-28 | A kind of minimal error entropy computational methods of quantization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710509553.3A CN107357761A (en) | 2017-06-28 | 2017-06-28 | A kind of minimal error entropy computational methods of quantization |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107357761A true CN107357761A (en) | 2017-11-17 |
Family
ID=60273333
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710509553.3A Pending CN107357761A (en) | 2017-06-28 | 2017-06-28 | A kind of minimal error entropy computational methods of quantization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107357761A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107977651A (en) * | 2017-12-21 | 2018-05-01 | 西安交通大学 | Based on the shared spatial model spatial feature extracting method for quantifying minimal error entropy |
| CN108960106A (en) * | 2018-06-25 | 2018-12-07 | 西安交通大学 | A kind of human eye fixation point estimation method based on quantization Minimum error entropy criterion |
| CN109001789A (en) * | 2018-06-05 | 2018-12-14 | 西安交通大学 | A kind of unmanned vehicle positioning fusion method based on cross-correlation entropy registration |
| WO2019173972A1 (en) * | 2018-03-13 | 2019-09-19 | Beijing Didi Infinity Technology And Development Co., Ltd. | Method and system for training non-linear model |
| CN112086100A (en) * | 2020-08-17 | 2020-12-15 | 杭州电子科技大学 | Quantization error entropy based urban noise identification method of multilayer random neural network |
| CN112953607A (en) * | 2021-02-22 | 2021-06-11 | 西安交通大学 | Method, medium and equipment for eliminating quantization noise of MIMO-OFDM system |
-
2017
- 2017-06-28 CN CN201710509553.3A patent/CN107357761A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107977651A (en) * | 2017-12-21 | 2018-05-01 | 西安交通大学 | Based on the shared spatial model spatial feature extracting method for quantifying minimal error entropy |
| WO2019173972A1 (en) * | 2018-03-13 | 2019-09-19 | Beijing Didi Infinity Technology And Development Co., Ltd. | Method and system for training non-linear model |
| CN109001789A (en) * | 2018-06-05 | 2018-12-14 | 西安交通大学 | A kind of unmanned vehicle positioning fusion method based on cross-correlation entropy registration |
| CN108960106A (en) * | 2018-06-25 | 2018-12-07 | 西安交通大学 | A kind of human eye fixation point estimation method based on quantization Minimum error entropy criterion |
| CN112086100A (en) * | 2020-08-17 | 2020-12-15 | 杭州电子科技大学 | Quantization error entropy based urban noise identification method of multilayer random neural network |
| CN112086100B (en) * | 2020-08-17 | 2022-12-02 | 杭州电子科技大学 | Quantization error entropy based urban noise identification method of multilayer random neural network |
| CN112953607A (en) * | 2021-02-22 | 2021-06-11 | 西安交通大学 | Method, medium and equipment for eliminating quantization noise of MIMO-OFDM system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107357761A (en) | A kind of minimal error entropy computational methods of quantization | |
| Zeng et al. | Outlier-Robust Matrix Completion via $\ell _p $-Minimization | |
| WO2017197626A1 (en) | Extreme learning machine method for improving artificial bee colony optimization | |
| CN103514443A (en) | Single sample face identification transfer learning method based on LPP feature extraction | |
| Li et al. | Modified hidden factor analysis for cross-age face recognition | |
| CN110442726A (en) | Social media short text on-line talking method based on physical constraints | |
| Zhang et al. | Dictionary learning and face recognition based on sample expansion | |
| CN109685044A (en) | Recognition of face search method based on k means clustering algorithm | |
| CN106845538A (en) | A kind of sparse Subspace clustering method for declining optimization based on selective coordinate | |
| Zhang et al. | Face alignment based on fusion subspace and 3D fitting | |
| CN105701775A (en) | Image denoising method based on improved adaptive dictionary learning | |
| Wang et al. | Extrinsic Least Squares Regression with Closed‐Form Solution on Product Grassmann Manifold for Video‐Based Recognition | |
| CN116680415A (en) | Long-tail entity alignment method based on entity structure characteristics | |
| Liang et al. | Different lighting processing and feature extraction methods for efficient face recognition | |
| CN108647259B (en) | Naive Bayes text classification method based on improved depth feature weighting | |
| Hua-Yan et al. | Accelerating EM missing data filling algorithm based on the k-means | |
| CN113537308B (en) | Two-stage k-means clustering processing system and method based on localized differential privacy | |
| CN114564587A (en) | Data enhancement method based on countermeasure training under text classification scene | |
| Wang et al. | Virtual samples based robust block-diagonal dictionary learning for face recognition | |
| Boxiang et al. | State: A clustering algorithm focusing on edges instead of centers | |
| CN107480199B (en) | Query reconstruction method, device, equipment and storage medium of database | |
| CN112149053A (en) | Multi-view image characterization method based on low-rank correlation analysis | |
| CN111399838A (en) | Data modeling method and device based on spark SQ L and materialized view | |
| Cao | An Expression Recognition Model Based on Channel and Spatial Attention Fusion | |
| Gao et al. | Mining frequent sets using fuzzy multiple-level association rules |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171117 |
|
| RJ01 | Rejection of invention patent application after publication |