CN105389599A - Feature selection approach based on neural-fuzzy network - Google Patents

Abstract

The invention provides a feature selection approach based on a neural-fuzzy network. The feature selection approach includes the following steps of: the first step, adjusting parameters of fuzzy membership functions to obtain a membership function set shown in the description, based on an x training neural fuzzy network, wherein [mu]fimj represents an mj-th membership function of features fi; the second step, calculating the output oq of the neural-fuzzy network when the input is xq; the third step, modifying the neural-fuzzy network to allow a fuzzy mapping layer to map the xq as that shown in the description, wherein the values of all the membership functions of the fi are identically equal to 0.5, and at this moment, the input of the neural-fuzzy network is that shown in the description; the fourth step, calculating measurement values FQJi shown in the description of the feature fi; and the fifth step, sorting the feature measurement values FQJi in a descending order.

Description

Based on the feature selection approach of Neuro-Fuzzy Network

Technical field

The present invention relates to mode identification technology, more particularly, the present invention relates to a kind of feature selection approach based on Neuro-Fuzzy Network.

Background technology

Feature selecting is the key issue of the technology such as pattern-recognition.In recent years, artificial neural network being used for feature selecting is a hot issue.By having the artificial neural network of good learning performance, the significance level of each feature or character subset can be inferred, but also there are some problems.

In feature selecting field, major part can be regarded as a special case of network pruning algorithms based on the feature selecting of artificial neural network, unlike being input node by what wipe out, instead of hidden node or weights.The method of current widespread use be utilize network before and after beta pruning export between variable quantity as by the tolerance of beta pruning feature importance, this method also can be called feature significance measure.Generally all suppose that the significance level of a feature is lower for a study good network, it is less on the output impact of network.

The neural network type being mainly used in feature significance tolerance is at present divided into two large classes, one class is the Multilayer Feedforward Neural Networks based on multilayer perceptron, as a kind of feature selecting algorithm based on multilayer perceptron that professor Ruck proposes, define a kind of feature significance measure, namely weighed the significance level of feature by the output of computational grid relative to the rate of change that network inputs; Professor R.K.De on this basis, proposes another kind of method, and the minimum value of namely first fetching data and maximal value are normalized, and then weighs the significance level of feature relative to the rate of change that network inputs by the output of computational grid.

Another large class is exactly fuzzy neural network, as professor MK.Jia proposes to utilize membership function that data are mapped to degree of membership space from original feature space, and in the meaning of degree of membership, carries out network beta pruning and computational grid to the susceptibility of certain feature.Professor D.Chakraborty utilizes front a kind of method design based on fuzzy rule Neuro-Fuzzy Network, can carry out feature selecting and classification simultaneously.This network has 4 layers, and point 3 stages train, and wherein the 2nd stage completes feature selecting, and then in the end the stage completes in membership function adjustment, and that is, before membership function study, feature selecting completes.

Said method all carries out feature selecting based on artificial neural network.First class is mainly main based on the feature selecting of multilayer perceptron.Although achieve some good classifying qualities to feature selecting, this simple method is for the data of some special distribution, and its result can not reflect truth.If be normalized data in advance, although can ensure some unchangeability, such as displacement and convergent-divergent constant, may lose other unchangeability, such as rotatory, and may lose the vital role of information some have to(for) classification.Second largest class methods, based on fuzzy logic and artificial neural network, make some key characters lose problem although avoid due to the normalization of data, also can reflect certain authenticity to special distributed data.But the definition due to its membership function is actually and completed before e-learning, so degree of membership maps the data distortion after still may causing mapping.The four-layer structure network that professor D.Chakraborty proposes, the 2nd stage just completes feature selecting, and then in the end the stage completes in membership function adjustment, and that is, before membership function study, feature selecting completes.So the result of feature selecting will depend critically upon the initial value of membership function.

Summary of the invention

Technical matters to be solved by this invention is for there is above-mentioned defect in prior art, provides a kind of feature selection approach based on Neuro-Fuzzy Network; Wherein Neuro-Fuzzy Network object of the present invention is, feature selecting is after whole e-learning completes, and namely degree of membership self study is carried out after completing, and avoids the problem that characteristic results depends on membership function initial parameter value.In addition, fuzzy membership function is completed by self study, avoids the problem that the key character that brings due to data normalization is lost.Further, also define a kind of new characteristic measure method, make beta pruning process be carry out at the mapping layer of fuzzy neural network, instead of carry out at input layer.The method easily and other searching algorithms combine the complete pattern classification system of composition one.

In order to realize above-mentioned technical purpose, according to the present invention, providing a kind of feature selection approach based on Neuro-Fuzzy Network, comprising:

First step: train Neuro-Fuzzy Network based on χ, the parameter of adjustment fuzzy membership function is to obtain membership function collection:

$\mathrm{\μ}=\{{\mathrm{\μ}}_{11},...{\mathrm{\μ}}_{1m_1},......,{\mathrm{\μ}}_{i1},...{\mathrm{\μ}}_{{\mathrm{im}}_j},......,{\mathrm{\μ}}_{R1},...,{\mathrm{\μ}}_{{\mathrm{Rm}}_R}\},$

Wherein representation feature f _im _jindividual membership function;

Second step: calculate and be input as x _qtime Neuro-Fuzzy Network output o _q;

Third step: amendment Neuro-Fuzzy Network, makes FUZZY MAPPING layer by x _qbe mapped as

Wherein make f _iall membership function value perseverances be 0.5, the now output of Neuro-Fuzzy Network is

4th step: calculate feature f _imetric FQJ _i

${\mathrm{FQJ}}_i=\frac{1}{Q}\underset{q=1}{\overset{Q}{\Σ}}{||o_q-q_q^{\left(i\right)}||}^2;$

5th step: by characteristic measure value FQJ _iby descending sort.

Preferably, by metric FQJ _ilarger feature f _ibe defined as more important.

Accompanying drawing explanation

By reference to the accompanying drawings, and by reference to detailed description below, will more easily there is more complete understanding to the present invention and more easily understand its adjoint advantage and feature, wherein:

Fig. 1 schematically shows the structure of Neuro-Fuzzy Network.

Fig. 2 schematically shows according to the preferred embodiment of the invention based on the process flow diagram of the feature selection approach of Neuro-Fuzzy Network.

It should be noted that, accompanying drawing is for illustration of the present invention, and unrestricted the present invention.Note, represent that the accompanying drawing of structure may not be draw in proportion.Further, in accompanying drawing, identical or similar element indicates identical or similar label.

Embodiment

In order to make content of the present invention clearly with understandable, below in conjunction with specific embodiments and the drawings, content of the present invention is described in detail.

The present invention proposes a kind of novel Neuro-Fuzzy Network, this network is the combination based on fuzzy set theory and artificial neural network, is mainly used in new Feature Selection, and the method can be applicable to the fields such as pattern-recognition, data mining, image procossing.

Principle of the present invention and embodiment will be explained below from Neuro-Fuzzy Network structure.

(1) Neuro-Fuzzy Network structure:

To C class (ω ₁, ω ₂... ω _l..., ω _c) identification problem, note training sample set is, $\mathrm{\χ}=\left\{x_q\right|x_q={(x_{q1},x_{q2},...,x_{qi},...,x_{qR})}^T\∈R^R,q=1,2,...,Q\},$ Q is the number of training sample set, and feature set is Φ={ f ₁..., f _i... f _r, x _ijfeature f _ian observed reading.Fuzzy inference system is according to rule set to x _qdifferentiate, wherein fuzzy rule there is following form:

ifx ₁isA _1kandx ₂isA _2k…andx _RisA _Rkthenω _l(1)

Wherein, A _ikbe defined in feature f _ion a kth fuzzy set, and be certain Fuzzy Logic Operators; As k ≠ h, not necessarily always there is A _ik≠ A _ihset up." if " part is called the former piece of fuzzy inference system, and " then " part is called the consequent of fuzzy inference system.Traditional fuzzy inference system needs expert to set up these rules, and Neuro-Fuzzy Network can obtain these rules by the representational training sample set of study automatically.The present invention adopts pruning algorithms to calculate the metric of feature in third layer, and utilization has the learning method of supervision to upgrade the parameter value of Neuro-Fuzzy Network.The structure of Neuro-Fuzzy Network is illustrated in fig. 1 shown below.

(2) principle of work of Neuro-Fuzzy Network:

As shown in Figure 1.Input layer L1 is the input block of network, and it is transferred to lower one deck input quantity, and its nodes is the dimension of input feature vector, i.e. d ₁=R.

FUZZY MAPPING layer L2 is membership function mapping layer, and the output of input layer L1 is projected to membership function space at this layer.The output of the node j of FUZZY MAPPING layer L2 is:

$a_j^2={\mathrm{\μ}}_A\left(a_i^1\right)---\left(2\right)$

Namely its membership function adopts bell (bell shape) function:

${\mathrm{\μ}}_A\left(x\right)=\frac{1}{1+{\[{\left(\frac{x-\mathrm{\ξ}}{\mathrm{\σ}}\right)}^2\]}^{\mathrm{\τ}}},\mathrm{\σ}\≠0,\mathrm{\τ}\≥0---\left(3\right)$

Here ξ is the convergent-divergent of the center of this function, σ control function, and τ controls the width of the bell flat top of function, and τ is larger, and bell flat top is wider.The nodes of FUZZY MAPPING layer L2 wherein m _ibe defined in feature f _ion the number of fuzzy membership function.Input layer L1 is set to 1 to all weights of FUZZY MAPPING layer L2, and the node j of FUZZY MAPPING layer L2 exports and is:

$a_j^2={\stackrel{\‾}{z}}_n^{1-e^{-{\mathrm{\β}}_i^2}}---\left(4\right)$

Wherein gaussian membership function, one and input feature vector f _ithe adjustable parameter be associated, is called adjustment of features device, for the importance of measures characteristic.When when getting higher value, value trends towards for less trend towards 1, namely make feature convergent.Based on this, if for good feature get higher value, for bad feature get less value, just can with one about β _i ²function ${\mathrm{\γ}}_i=1-e^{-{\mathrm{\β}}_i^2}$ Select feature.

Each node of former piece layer L3 correspond to the if part of a fuzzy rule, and the output of its node j is:

$a_j^3=M(a_j^2,\mathrm{\Θ})---\left(5\right)$

Wherein M is defined as:

$M(x_1,x_2,...,x_R,r)=\frac{{(x_1^r+x_2^r+...+x_R^r)}^{\frac{1}{r}}}{R}---\left(6\right)$

Wherein r >=1.The nodes of former piece layer L3 is fUZZY MAPPING layer L2 is set to 1 to all weights of former piece layer L3.In network training, the information that the value of three parameter ξ of bell shaped function, σ, τ can get according to the sample learning concentrated from training sample and be adjusted to suitable value.

The network structure of former piece layer L3 is the same with D.Chakraborty's, but, the parameter still participation in learning of former piece layer L3 of the present invention.

Each node of consequent layer L4, from " then " part that represent fuzzy rule the angle of fuzzy reasoning, represents a classification from the angle of pattern classification.Therefore, the nodes d of consequent layer L4 ₄=C.The weight of each rule that former piece layer L3 exports by the node of consequent layer L4 is joined together, and calculates and finally show that a value represents that the sample x of input belongs to the determination degree of class l with S type function.The transport function of consequent layer L4 is S type function.

Former piece layer L3 to consequent layer L4 is one and entirely connects, and weights press the initialization of Nguyen-Widrow method, and in e-learning subsequently, adjust to best value.The output of L4 layer:

$a_j^4=S\left(j^{W^4}{a}^3\right)---\left(7\right)$

Wherein W ⁴for the weight matrix of former piece layer L3 to consequent layer L4, dimension is d ₄× d ₃, the jth row vector of this matrix, a ³it is the output vector of former piece layer L3.

L4 transport function uses S type function, instead of as D.Chakraborty, use max function.

(3) learning process of Neuro-Fuzzy Network:

Network learning procedure is divided into 3 stages.1st stage fixed membership function parameter ξ, σ, τ, the weights between adjustment former piece layer L3 and consequent layer L4 and adjustment of features device parameter beta _i, the 2nd stage, to γ _ithe feature f of < th (th is a threshold value) _i, wipe out its all nodes at FUZZY MAPPING layer L2, then re-training network.In 3rd stage, according to certain criterion, beta pruning is carried out to the node of former piece layer L3, and again trains, upgrade the parameter ξ of the bell membership degree function of FUZZY MAPPING layer L2, σ, τ simultaneously.

Fig. 2 schematically shows according to the preferred embodiment of the invention based on the process flow diagram of the feature selection approach of Neuro-Fuzzy Network.

As shown in Figure 2, comprise based on the feature selection approach of Neuro-Fuzzy Network according to the preferred embodiment of the invention:

First step S1: train Neuro-Fuzzy Network based on χ, make the parameter adjustment of fuzzy membership function to suitable value, obtain membership function collection:

$\mathrm{\&mu;}=\{{\mathrm{\&mu;}}_{11},...{\mathrm{\&mu;}}_{1m_1},......,{\mathrm{\&mu;}}_{i1},...{\mathrm{\&mu;}}_{{\mathrm{im}}_j},......,{\mathrm{\&mu;}}_{R1},...,{\mathrm{\&mu;}}_{{\mathrm{Rm}}_R}\},$

Wherein representation feature f _im _jindividual membership function;

Second step S2: calculate and be input as x _qtime Neuro-Fuzzy Network output o _q;

Third step S3: amendment Neuro-Fuzzy Network, makes FUZZY MAPPING layer by x _qbe mapped as,

Even f _iall membership function value perseverances be 0.5, if now the output of Neuro-Fuzzy Network is

4th step S4: calculate feature f _imetric FQJ _i

${\mathrm{FQJ}}_i=\frac{1}{Q}\underset{q=1}{\overset{Q}{\&Sigma;}}{||o_q-q_q^{\left(i\right)}||}^2$

5th step S5: by characteristic measure value by descending sort, the larger feature of metric is more important.

Neuro-Fuzzy Network of the present invention combines the advantage of artificial neural network and fuzzy logic inference system, has broad application prospects in pattern-recognition and feature selecting field.The Neuro-Fuzzy Network four-layer structure that the present invention proposes has a self-adaptation membership function learning process, the method overcoming D.Chakraborty professor completes problem for feature selecting before membership function study, the result avoiding feature selecting will depend critically upon the problem of the initial value of membership function, meanwhile, it also avoid the data degradation difficult problem brought of data normalization.Neuro-Fuzzy Network combines with pruning algorithms, and it also avoid the data of simple algorithm for some special distribution of general neural network, its result can not reflect a difficult problem for truth.

Tool of the present invention has the following advantages: 1) fuzzy membership function is self study, avoids the problem of data normalization; 2) calculate simply, can feature selecting is recycled and reused for and need not again train after network training; 3) characteristic measure value has simple, the univocal advantage of calculating; 4) can be used for pattern classification, due to based on fuzzy rule, so network behavior easy understand, analysis; 5) easy and various searching algorithm combines the complete feature selecting system of composition one.Application surface of the present invention is comparatively wide, can be applied in the aspects such as image recognition, speech recognition, data mining, machine vision.

In addition, it should be noted that, unless stated otherwise or point out, otherwise the term " first " in instructions, " second ", " the 3rd " etc. describe only for distinguishing each assembly, element, step etc. in instructions, instead of for representing logical relation between each assembly, element, step or ordinal relation etc.

Be understandable that, although the present invention with preferred embodiment disclose as above, but above-described embodiment and be not used to limit the present invention.For any those of ordinary skill in the art, do not departing under technical solution of the present invention ambit, the technology contents of above-mentioned announcement all can be utilized to make many possible variations and modification to technical solution of the present invention, or be revised as the Equivalent embodiments of equivalent variations.Therefore, every content not departing from technical solution of the present invention, according to technical spirit of the present invention to any simple modification made for any of the above embodiments, equivalent variations and modification, all still belongs in the scope of technical solution of the present invention protection.

Claims (2)

1., based on a feature selection approach for Neuro-Fuzzy Network, it is characterized in that comprising:

First step: train Neuro-Fuzzy Network based on χ, the parameter of adjustment fuzzy membership function is to obtain membership function collection:

$\mathrm{\&mu;}=\{{\mathrm{\&mu;}}_{11},...{\mathrm{\&mu;}}_{1m_1},......,{\mathrm{\&mu;}}_{i1},...{\mathrm{\&mu;}}_{{\mathrm{im}}_j},......,{\mathrm{\&mu;}}_{R1},...,{\mathrm{\&mu;}}_{{\mathrm{Rm}}_R}\},$

Wherein representation feature f _im _jindividual membership function;

Second step: calculate and be input as x _qtime Neuro-Fuzzy Network output o _q;

Third step: amendment Neuro-Fuzzy Network, makes FUZZY MAPPING layer by x _qbe mapped as,

Wherein make f _iall membership function value perseverances be 0.5, the now output of Neuro-Fuzzy Network is ;

4th step: calculate feature f _imetric FQJ _i

${\mathrm{FQJ}}_i=\frac{1}{Q}\underset{q=1}{\overset{Q}{\&Sigma;}}{||o_q-q_q^{\left(i\right)}||}^2;$

5th step: by characteristic measure value FQJ _iby descending sort.

2. the feature selection approach based on Neuro-Fuzzy Network according to claim 1, is characterized in that, by metric FQJ _ilarger feature f _ibe defined as more important.