CN1945602A - Characteristic selecting method based on artificial nerve network - Google Patents
Characteristic selecting method based on artificial nerve network Download PDFInfo
- Publication number
- CN1945602A CN1945602A CNA2006100195700A CN200610019570A CN1945602A CN 1945602 A CN1945602 A CN 1945602A CN A2006100195700 A CNA2006100195700 A CN A2006100195700A CN 200610019570 A CN200610019570 A CN 200610019570A CN 1945602 A CN1945602 A CN 1945602A
- Authority
- CN
- China
- Prior art keywords
- mrow
- mtd
- centerdot
- msubsup
- msup
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims description 37
- 210000005036 nerve Anatomy 0.000 title 1
- 238000013528 artificial neural network Methods 0.000 claims abstract description 101
- 238000012549 training Methods 0.000 claims abstract description 75
- 238000010187 selection method Methods 0.000 claims abstract description 14
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims abstract description 6
- 238000013507 mapping Methods 0.000 claims description 87
- 230000006870 function Effects 0.000 claims description 47
- 238000013138 pruning Methods 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 8
- 238000005259 measurement Methods 0.000 claims description 5
- 230000009471 action Effects 0.000 claims description 4
- 230000035945 sensitivity Effects 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 238000013459 approach Methods 0.000 claims description 2
- 238000003909 pattern recognition Methods 0.000 abstract description 11
- 238000010606 normalization Methods 0.000 abstract description 7
- 238000004364 calculation method Methods 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 206010065042 Immune reconstitution inflammatory syndrome Diseases 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000002945 steepest descent method Methods 0.000 description 2
- 244000140747 Iris setosa Species 0.000 description 1
- 235000000827 Iris setosa Nutrition 0.000 description 1
- 241001627144 Iris versicolor Species 0.000 description 1
- 241001136653 Iris virginica Species 0.000 description 1
- 235000014676 Phragmites communis Nutrition 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000002939 conjugate gradient method Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses an features selection method based on the artificial neural network, including: (1) the user gives all the features for selection, and the sample for training the artificial neural network; (2) selecting the number of blurring subjection function, setting the number of nodes on each layer of artificial neural network, link weight among layers and the initial value of blurring subjection function; (3) using the back-propagation algorithm to train the network in the mode of batch processing, to adjust the network linking weights and parameters of blurring subjection function; (4) calculating all the importance of characteristics, and ranking the features. The invention avoids the problem of data normalization, in which, calculating is simple and training network is just once, and is easy to combine with various algorithm to be a complete features selection system. The invention has been successfully used in the pattern recognition and target classification with a variety of multi-dimensional characteristics, and also can be applied to all types of pattern recognition with the data characteristics.
Description
Technical Field
The invention belongs to the field of pattern recognition, relates to a feature selection method, and particularly relates to a feature selection method based on an artificial neural network.
Background
Feature selection (feature selection) is an important aspect in the field of pattern recognition, because the complexity of the pattern recognition algorithm tends to increase exponentially with the increase of the data dimension, and if the data dimension is not reduced, the size of the classifier becomes extremely large, and the computational overhead required for classification becomes too large to bear. Therefore, the data features are selected, important features in the data features are selected, and reduction of dimensionality of the data features is an indispensable link. Moreover, most of the features used by most of the current pattern recognition algorithms are automatically extracted by machines, so that the features such as redundancy, noise and the like inevitably exist, and the problem can be effectively eliminated by utilizing feature selection.
Feature selection is the process of selecting a subset from a set of all features without or with little reduction in classifier recognition rate. The key point of the feature selection technique is what criteria are chosen to measure the importance of the features. Traditional metric criteria, such as distance-based metrics, information (or uncertainty) based metrics, dependency-based metrics, and the like, focus on analyzing the characteristics of the data, and such methods do not work well in practice. With the continuous progress of the field of artificial intelligence, some feature measurement methods using the technologies of artificial neural networks (artificalneural networks), fuzzy math (fuzzy math), and the like are proposed. This class of methods is based on classification error rate, i.e., measures the importance of features to classification error rate, and is therefore more efficient than the previous class of methods. In particular operation, most of these methods utilize artificial neural network techniques for feature selection.
Feature selection based on artificial Neural Networks can be considered as a special case of pruning Algorithms (prune Algorithms), i.e. pruning nodes of the input layer instead of nodes or weights of the hidden layer, as in Reed r.pruning Algorithms-a surface.ieee Transactions on Neural Networks, 1993, 4 (5): 740-746. A common idea is to use the variation of the output value of the artificial neural network before and after pruning as a sensitivity measure for features, such as Verikas a, Baeauskiene m.featurechoice with neural networks, pattern Recognition Letters, 2002, 23 (11): 1323-1335. The basic assumptions of this idea are: a well-learned neural network will have a greater, i.e. more sensitive, change in the corresponding output value for the more important feature changes, and vice versa. This assumption is most directly and accurately reflected by a feature selection method based on the sensitivity metric Aj, such as the method of Ruck D W, Rogers S K and Kabrisky m.feature selection using a multilayerpicervetron, Journal of Neural Network Computing, 1990, 9(1) of document 3: 40-48.
When the importance of a certain feature is specifically considered, the change of the output of the artificial neural network before and after the feature is deleted is calculated to be used as a feature metric. The deletion Feature is an observation that the Feature is constantly zero in the sample, as described in De r.k, Basak J and Pal S k, neuro-Fuzzy Feature evaluation with the Theoretical analysis, neural Networks, 1999, 12 (10): 1429 to 1455. This approach requires that the data be normalized first, which can corrupt the data. To avoid the normalization problem, an ambiguity mapping (fuzzy mapping) layer can be added in the artificial neural network, and the layer maps each feature in one-to-many manner, and the domain of the mapped new feature, namely the ambiguity feature, is limited to [0, 1], so that the normalization problem is avoided, such as Jia P and Sangg N.feature selection a radial basis functions networks and fuzzy set of the mechanical measures, in: proceedings of SPIE 5281(1) -the Third International Symposium on Multispectral Image Processing and Pattern Recognition, Beijing, China: the International Society of Optical Engineering Press, 2003.109-114. In this method, a fuzzy membership function (fuzzy membership function) is obtained before the artificial neural network learns, and it depends on the first and second moments of the data, which has substantially the same problem as the normalization of document 4. In fact, the fuzzy mapping layer proposed in document 5 can be completely separated from the network, and is used as a normalization method for preprocessing data.
Disclosure of Invention
The invention aims to provide a feature selection method based on an artificial neural network, which avoids the problem of data normalization, has high robustness and has good effect on noise features and redundancy features.
The invention provides a feature selection method based on an artificial neural network, which comprises the following steps:
(1) user specifies the feature f to be selectediI-1, …, N, giving a training sample set for training an artificial neural network:
the training samples have the same dimension R, R ═ N, and are classified into K categories: omega1，…，ωKThe qth training sample xqX of the ith dimension ofqiI.e. the specified i-th feature fiThe q-th observation of (1);
(2) constructing an artificial neural network sequentially consisting of an input layer, a fuzzy mapping layer, a hidden layer and an output layer according to the training sample; the neural network data is input into the neural network from the input layer through the connection weight w2Transferring to the fuzzy mapping layer, acting by the fuzzy mapping layer, and passing through the connection weight w3Transferred to the hidden layer, acted by the hidden layer and then passed through the connection weight w4Transmitting the output layer to obtain an output, wherein m is 2, 3, 4;
(3) training the initialized artificial neural network by using a training sample set given by a user, wherein the processing procedure is as follows:
(3.1) selecting the estimator e of mean square error as a performance index in the learning process:
wherein, ti m(q) is an output of the node i of the mth layer when the qth sample is inputTarget value, ai m(q) is the actual output of node i at the mth level when the qth sample is input, and G is the number of nodes at that level;
(3.2) adopting a back propagation algorithm to carry out connection weight matrix w between each layer of the artificial neural networkmTraining, wherein m is 3, 4;
(3.3) updating parameters xi, sigma and tau in the function of the fuzzy mapping layer node;
(3.4) when e meets the convergence condition, entering the step (4), and otherwise, repeating the steps (3.2) - (3.3);
(4) and carrying out fuzzy pruning on the features by using the trained artificial neural network, calculating the importance measurement of each feature, and sequencing the features according to the measurement values of the importance.
The invention only needs the user to give the original characteristic set and the sample for training, and can obtain the ranking of all the characteristics in the original characteristic set to the importance of classification. Compared with the existing feature selection method, the feature selection method of the invention has the advantages that: the problem of data normalization is well avoided; the calculation is simple, and the neural network only needs to be trained once; the method is easy to combine with various search algorithms to form a complete feature selection system. The method is successfully applied to various pattern recognition and target classification with multi-dimensional characteristics, and can also be applied to the field of pattern recognition of various data type characteristics.
Drawings
FIG. 1 is a flow chart of a feature selection method based on an artificial neural network with an adaptive fuzzy mapping layer;
FIG. 2 is a schematic diagram of an artificial neural network with an adaptive fuzzy mapping layer;
FIG. 3 is a schematic diagram of an artificial neural network with an adaptive fuzzy mapping layer built in an example
FIG. 4 is a graph of the fuzzy membership function (initial value) for the feature seqal length.
Detailed Description
The feature selection method of the present invention starts the feature selection process on the premise that the user gives the data set for training and the feature set to be selected, and the feature selection process is described in detail below.
Feature selection is performed to obtain a measure of the importance of the features. In the feature selection method provided by the invention, the artificial neural network with the fuzzy mapping layer is trained by using the data set provided by the user, and then the importance metric of each feature is calculated by means of the trained network, so that the purpose of feature selection is achieved. As shown in fig. 1, the method of the present invention comprises the steps of:
(1) user specifies the feature f to be selectedi(i ═ 1, …, N), training samples for training the artificial neural network are given.
(1.1) specification of features
The specified characteristics must be data-type characteristics that directly reflect the actual physical or geometric meaning of the object, such as weight, speed, length, etc. The number N of features is a natural number, that is, the number of features is one or more.
(1.2) definition of training samples
Training samples for training the artificial neural network are also of a data type, and all samples have the same dimension R (R ═ N) and are classified into K categories: omega1，…，ωK. Dimension R is equal to the number of features specified in step (1.1). The q training sample xqX of the ith dimension ofqiIs the specified i-th feature fiThe q-th observation. The specific mathematical description of the training sample set is:
wherein Q is the number of training samples, and Q is more than or equal to K, each class omegal(l-1, …, K) at least one sample,representing a set of real numbers, R being a sample xqIs equal to the number of features N of the training sample set X.
(2) And constructing an artificial neural network consisting of a characteristic layer A, a fuzzy mapping layer B, a hidden layer C and an output layer D according to the training sample, and initializing.
As shown in FIG. 2, the artificial neural network structure comprises an input layer A (i.e. a characteristic layer), a fuzzy mapping layer B, a hidden layer C and an output layer D, wherein a connection weight w is used between layersm(m-2, 3, 4) are linked. Data is input into the neural network from the input layer, then is transmitted to the fuzzy mapping layer through the connection weight, is transmitted to the hidden layer through the connection weight after being acted by the fuzzy mapping layer, and is transmitted to the output layer through the connection weight after being acted by the hidden layer, so that output is obtained. The construction of an artificial neural network with a fuzzy mapping layer requires the setting of the node numbers of an input layer (characteristic layer), a hidden layer and an output layer; determining each feature fiNumber m of corresponding fuzzy membership functioniAnd defining the fuzzy membership functions. The initialization operation needs to determine the initial value of the connection weight between each layer of the artificial neural network and the initial value of the parameter of the fuzzy membership function in each node in the fuzzy mapping layer.
The specific process is as follows:
(2.1) input layer A
(2.1.1) selection of number of input layer nodes
Input layer A node number S1Equal to the dimension R of the training samples.
(2.1.2) input and output of input layer nodes
Per node input trainingA certain dimension of the sample. When the q sample is input by the neural network, the node A of the input layeriThe inputs of (a) are:
the output is:
(2.2) fuzzy mapping layer B
(2.2.1) selection of the number of fuzzy membership functions corresponding to each feature
For feature fiF can be defined according to its specific physical meaningiCorresponding miAnd each fuzzy membership function forms a fuzzy mapping layer node. That is, the number of nodes of the mapping layer B is blurred <math> <mrow> <msup> <mi>S</mi> <mn>2</mn> </msup> <mo>=</mo> <munderover> <mi>Σ</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <msup> <mi>S</mi> <mn>1</mn> </msup> </munderover> <msub> <mi>m</mi> <mi>i</mi> </msub> <mo>,</mo> </mrow> </math> miThe selection of the value needs to satisfy the following conditions:
wherein Q ismin＝min{Ql}，QlRepresenting the class omega in the training sample given by the userlThe number of samples of (1).
(2.2.2) connection weights between input layer and fuzzy mapping layer
Node A of the input layeriNode B with fuzzy mapping layeri1，…，BimiNode B connected by connection weights and having fuzzy mapping layeri1，…，BimiDo not interact with except AiAnd other nodes of the other input layers are connected, namely a 1-to-many connection mode. Feature level a node aiAnd fuzzy mapping layer node BijThe connection weight value between the feature layer A and the fuzzy mapping layer B is constant to 1, namely, the connection weight matrix w between the feature layer A and the fuzzy mapping layer B2The training of the artificial neural network is not participated.
(2.2.3) node B of fuzzy mapping layerijIs inputted
Blurring node B of mapping layer when q sample is inputted by neural networkijThe inputs of (a) are:
(2.2.4) node B of fuzzy mapping layerijFunction of (2)
Node B of fuzzy mapping layerijThe function of (a) is a fuzzy membership function muijI.e. characteristic fiThe jth membership function of (a). In the present invention, the feature f of the ith dimension is giveniA fuzzy membership function of (1) means that a map is givenMu shootingi：fi→[0，1]。
Node BijThe fuzzy membership function of (a) is of the form:
here, n isij 2(q) node B of the fuzzy mapping layer when the q-th sample is inputijInput of aij 2(q) is the corresponding actual output. XiijIs a node BijClass of conditional probability density of σijIs a node BijStandard deviation of the class of conditional probability densities, τijIs a node BijIs measured by the measurement unit. The action of tau is shown in: even if ξ and σ of the two membership functions are equal, it is still possible to avoid that the two membership functions are exactly the same by adjusting τ.
For σijAnd τijIs not particularly limited, ξijGenerally taken in the corresponding characteristic fiIs randomly selected on the value range of (1).
(2.3) hidden layer C
(2.3.1) selection of the number of hidden layer nodes
Number of nodes S of hidden layer C3The selection of (2) is not particularly required, and generally, the selection is not less than the number K of the classes of the training samples.
(2.3.2) obfuscating the connection weights between the mapping layer and the hidden layer
The fuzzy mapping layer B is fully connected with the hidden layer C, that is, each node of the fuzzy mapping layer B is connected with all nodes of the hidden layer C, and each node of the hidden layer C is also connected with all nodes of the fuzzy mapping layer B. Fuzzy mapping layer B and implicit layer C
(2.3.3) input of hidden layer node
When the q sample is input to the neural network, node C of the hidden layeru(u＝1，…，S3) The inputs of (a) are:
wherein, ap 2(q) node B which is a fuzzy mapping layerp(p＝1，…，S2) Output at the input of the q sample of the neural network, wpu 3Node B being a fuzzy mapping layerpAnd hidden layer node CuThe right of connection between.
(2.3.4) Effect function of hidden layer nodes
The role function of the hidden layer node is selected as a Sigmoid function:
wherein n isu 3(q) node C of the hidden layer at the input of the q-th sample for the neural networkuInput of au 3(q) is the corresponding output.
It can also be chosen as a hyperbolic tangent function:
wherein n isu 3(q) node C of the hidden layer at the input of the q-th sample for the neural networkuInput of au 3(q) is the corresponding output.
(2.4) output layer D
(2.4.1) selection of number of output layer nodes
Node number S of output layer D4Equal to the number of classes K of the training samples.
(2.4.2) connection rights between hidden layer and output layer
The hidden layer C and the output layer D are fully connected, that is, each node of the hidden layer C is connected to all nodes of the output layer D, and each node of the output layer D is also connected to all nodes of the hidden layer C. Connection weight between hidden layer C and output layer D <math> <mrow> <msup> <mi>w</mi> <mn>4</mn> </msup> <mo>=</mo> <msub> <mfenced open='[' close=']'> <mtable> <mtr> <mtd> <msubsup> <mi>w</mi> <mn>11</mn> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mrow> <mn>1</mn> <mi>l</mi> </mrow> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <msup> <mi>lS</mi> <mn>4</mn> </msup> <mn>4</mn> </msubsup> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <msubsup> <mi>w</mi> <mrow> <mi>u</mi> <mn>1</mn> </mrow> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mi>ul</mi> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <msup> <mi>uS</mi> <mn>4</mn> </msup> <mn>4</mn> </msubsup> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <msubsup> <mi>w</mi> <mrow> <msup> <mi>S</mi> <mn>3</mn> </msup> <mn>1</mn> </mrow> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mrow> <msup> <mi>S</mi> <mn>3</mn> </msup> <mi>l</mi> </mrow> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mrow> <msup> <mi>S</mi> <mn>3</mn> </msup> <msup> <mi>S</mi> <mn>4</mn> </msup> </mrow> <mn>4</mn> </msubsup> </mtd> </mtr> </mtable> </mfenced> <mrow> <msup> <mi>S</mi> <mn>3</mn> </msup> <mo>×</mo> <msup> <mi>S</mi> <mn>4</mn> </msup> </mrow> </msub> </mrow> </math> (u＝1，…，S3，l＝1，…，S4) The initialization of (2) adopts a random method, and the value range of the weight is [0, 1]]。
(2.4.3) input and output of output layer nodes
Output layer node Dl(l＝1，…，S4) Input and output of (D) are equallOutput value n ofl 4(q) is that the q-th sample of the neural network input belongs to the class ωlThe probability of (c).
Wherein, wul 4Node C being a hidden layeruAnd an output layer node DlThe right of connection between.
(3) And training the artificial neural network after initialization by using a training sample set given by a user.
Training the artificial neural network by using a back propagation algorithm in a batch learning mode according to a training sample set given by a user, and updating the connection weight between each layer of the neural network and the parameters of the fuzzy membership function in each training until the artificial neural network meets the convergence condition set by the user.
The specific training method is as follows.
(3.1) selection of Convergence Condition
Firstly, an estimator e of mean square error is selected as a performance index in the learning process:
wherein, ti m(q) is a target value of an output of the node i of the mth layer when the qth sample is input, ai m(q) is the actual output of node i at the mth level when the qth sample is input, and G is the number of nodes at that level.
The user can set e less than a small positive number as a convergence condition according to the requirement on the calculation precision. For example, setting e < 0.001 as a convergence condition, calculating the value of e after the artificial neural network completes the steps (3.2) and (3.3) in a certain training, and stopping the training if the value of e is less than 0.001; otherwise, the next training is carried out.
(3.2) updating of connection weights between layers
The connection weight between the input layer A and the fuzzy mapping layer B is constant to 1, and the training is not participated in. Connection weight w between fuzzy mapping layer B and hidden layer C3Connection w between hidden layer C and output layer D4All need to take part in training, and w3And w4The updating method in training is the same.
The sensitivity of the estimator of the mean square error e in the back-propagation algorithm to the input of the mth layer is defined as
Wherein S ismIs the number of nodes at the mth layer of the artificial neural network, nmIs one size of SmXQ matrix representing the mth of the artificial neural networkInputting a layer; n isi m(q) represents the input of the node i of the mth layer at the time when the q-th sample is input to the neural network. Furthermore, it is possible to provide a liquid crystal display device, <math> <mrow> <mfrac> <mrow> <mo>∂</mo> <mi>e</mi> </mrow> <mrow> <msubsup> <mrow> <mo>∂</mo> <mi>n</mi> </mrow> <mi>i</mi> <mi>m</mi> </msubsup> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>=</mo> <mfrac> <mrow> <mo>∂</mo> <mi>e</mi> </mrow> <mrow> <msubsup> <mrow> <mo>∂</mo> <mi>a</mi> </mrow> <mi>i</mi> <mi>m</mi> </msubsup> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>×</mo> <mfrac> <mrow> <msubsup> <mrow> <mo>∂</mo> <mi>a</mi> </mrow> <mi>i</mi> <mi>m</mi> </msubsup> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> <mrow> <msubsup> <mrow> <mo>∂</mo> <mi>n</mi> </mrow> <mi>i</mi> <mi>m</mi> </msubsup> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>.</mo> </mrow> </math>
the connection weight is updated according to the steepest descent method, and a minimum modulus estimation algorithm such as a conjugate gradient method can also be adopted here. Connection weight matrix w between mth layer and m-1 layer (m is 3, 4) of artificial neural networkm(dimension is S)m-1×Sm) Updated to (r +1) th training start
wm(r+1)＝wm(r)-αgm(am-1)T.
Wherein, alpha is weight learning rate, the value range is more than 0 and less than or equal to 1, and is generally selected to be 0.05. r is the number of training sessions. a ismIs one size of SmA matrix of x Q, representing the actual output of the mth layer of the artificial neural network:
(3.3) updating of parameters xi, sigma, tau of the Functions of nodes of the fuzzy mapping layer
Node B of fuzzy mapping layer Bp(p＝1，…，S2) Three parameters xi of the action function ofp，σp，τpUpdated as follows, where θ is ξpThe learning rate of (a) is determined,
is σpIs the learning rate of, ρ is τpThe learning rate of the method adopts parameter selection methods such as a trial and error method and the like.
Wherein,pa2is the output matrix a of the fuzzy mapping layer B when inputting Q samples to the artificial neural network2Row p. And also
Wherein, ai 1(q) is with node BpConnected input layer node AiThe output at the input of the q sample of the neural networkIs xqi。
(3.4) termination of training
And (3) carrying out the operations of the steps (3.2) and (3.3) in each training of the artificial neural network. After each training is completed, calculating the value of e, and stopping the training if the convergence condition set in the step (3.1) is met; otherwise, the next training is carried out.
(4) And carrying out fuzzy pruning on the features by using the trained artificial neural network, calculating the importance measure of each feature, and sequencing.
(4.1) pairs of features fiPerforming fuzzy pruning
So-called pair feature fiFuzzy pruning (fuzzy prune algorithm) of (1), namely, the feature f is obtainediThe output value of all corresponding fuzzy membership function is set to 0.5, namely the output of the fuzzy mapping layer is set to be
Then, the artificial neural network under the condition is obtained for the input sample xqOutput vector a given by time-output layer4(xq，i)。
(4.2) calculating the importance measure of the feature FQJ (i)
The characteristic metric function FQJ (i) provided by the invention represents the ith dimension characteristic fiFor the importance of classification, feature fiA larger value of fqj (i) indicates that the feature is more important for classification. FQJ (i) is defined as follows:
wherein, a4(xq) Representing an artificial neural network for an input sample xqOutput vector given by time input layer, a4(xqI) represents a pair of features fiInput sample x for artificial neural network after fuzzy pruningqGiven the output vector. Using the artificial neural network trained in step (3) to apply to all features f given by the user in step (1.1)iCalculating the corresponding FQJ (i), characteristic f according to the formulaiThe value of fqj (i) is a measure of its importance.
(4.2) for all characteristics fiSorted according to their importance measures FQJ (i)
For all features fiThe sorting of the importance of all features to the classification is obtained in descending order of the magnitude of the corresponding fqj (i) values. The user can select one or more characteristics with the top rank for identification according to the actual needs or the constraints of objective conditions, so that the purpose of characteristic selection is achieved.
Example (c):
the user wishes to investigate the following four features: the importance of the Sepal length, the Sepal width, the Petal length and the Petal width to the classification of people is provided, and a training sample is given: the data set IRIS. The IRIS data set is used by many researchers for research in pattern recognition and has become a benchmark. The data set contains 3 classes, each class has 50 samples, each sample has 4 characteristics, sequentially Sepal length, Sepal width, Petal length and Petal width.
The specific steps for feature selection are as follows:
(1) user specifies the feature f to be selectedi(i ═ 1, …, N), training samples for training the artificial neural network are given.
(1.1) specification of features
User-specified 4 features: the Sepal length, Sepal width, Petal length, and Petalwidth are all datalogical features. Then N is 4.
(1.2) giving a training sample
Training samples given by the user are divided into 3 classes: iris Setosa, Iris Versicolor and Iris virginica, i.e., K ═ 3. Each class has 50 samples for a total of 150 samples, i.e., Q150. Each sample has 4-dimensional features: sepal length, Sepal width, Petal length, and Petal width. The dimension R of the sample is 4.
(2) And constructing an artificial neural network consisting of a characteristic layer A, a fuzzy mapping layer B, a hidden layer C and an output layer D according to the training sample, and initializing.
(2.1) construction of the input layer A
(2.1.1) selection of number of input layer nodes
Input layer A node number S1Equal to the dimension R, i.e. S, of the training sample1＝4。
(2.2) constructing a fuzzy mapping layer B
(2.2.1) selection of the number of fuzzy membership functions corresponding to each feature
Defining 3 fuzzy membership functions, m, for each feature1＝m2＝m3＝m4The number of nodes in the fuzzy mapping layer is 3 <math> <mrow> <msup> <mi>S</mi> <mn>2</mn> </msup> <mo>=</mo> <munderover> <mi>Σ</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <msup> <mi>S</mi> <mn>1</mn> </msup> </munderover> <msub> <mi>m</mi> <mi>i</mi> </msub> <mo>=</mo> <mn>3</mn> <mo>+</mo> <mn>3</mn> <mo>+</mo> <mn>3</mn> <mo>+</mo> <mn>3</mn> <mo>=</mo> <mn>12</mn> <mo>,</mo> </mrow> </math> Is provided with <math> <mrow> <mfrac> <msub> <mi>Q</mi> <mi>min</mi> </msub> <mrow> <munderover> <mi>Σ</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <msup> <mi>s</mi> <mn>1</mn> </msup> </munderover> <msub> <mi>m</mi> <mi>i</mi> </msub> </mrow> </mfrac> <mo>=</mo> <mfrac> <mn>50</mn> <mn>12</mn> </mfrac> <mo>></mo> <mn>3</mn> <mo>,</mo> </mrow> </math> The constraints are satisfied.
(2.2.2) connection weights between input layer and fuzzy mapping layer
Node A of the input layer1Node B with fuzzy mapping layer only11，B12，B13Connected by means of connection rights, node A of the input layer2Node B with fuzzy mapping layer only21，B22，B23Connected by means of connection rights, node A of the input layer3Node B with fuzzy mapping layer only31，B32，B33Connected by means of connection rights, node A of the input layer4Node B with fuzzy mapping layer only41，B42，B43Are connected by a connection right.
(2.2.3) selecting Functions of fuzzy mapping layer nodes
Selecting a node BijFuzzy membership function of (1):
parameter xi of membership functionijIs generally in the characteristic fiIs randomly selected over the range of values. Taking the feature Sepal length as an example, the value range of the feature is [4.3, 7.9 ]]Then f is1In the corresponding 3 fuzzy membership functions, the initial value of xi selected may be: xi11＝5.2，ξ12＝6.1，ξ137.0. σ may be set to σ11＝σ12＝σ13τ may be set to τ 0.4511＝τ12＝τ13The resulting membership function is shown in fig. 4 below, which is 2.
(2.3) hidden layer C
(2.3.1) selection of the number of hidden layer nodes
Empirically, S is selected3＝6。
(2.3.2) obfuscating the connection weights between the mapping layer and the hidden layer
Fuzzy mapping layer B and implicit layer C <math> <mrow> <msup> <mi>w</mi> <mn>3</mn> </msup> <mo>=</mo> <msub> <mfenced open='[' close=']'> <mtable> <mtr> <mtd> <msubsup> <mi>w</mi> <mn>11</mn> <mn>3</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mrow> <mn>1</mn> <mi>u</mi> </mrow> <mn>3</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mn>1,6</mn> <mn>3</mn> </msubsup> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <msubsup> <mi>w</mi> <mrow> <mi>p</mi> <mn>1</mn> </mrow> <mn>3</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mi>pu</mi> <mn>3</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mrow> <mi>p</mi> <mo>,</mo> <mn>6</mn> </mrow> <mn>3</mn> </msubsup> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <msubsup> <mi>w</mi> <mn>12,1</mn> <mn>3</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mrow> <mn>12</mn> <mo>,</mo> <mi>u</mi> </mrow> <mn>3</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mn>12,6</mn> <mn>3</mn> </msubsup> </mtd> </mtr> </mtable> </mfenced> <mrow> <mn>12</mn> <mo>×</mo> <mn>6</mn> </mrow> </msub> </mrow> </math> The initialization of (p 1, …, 12, u 1, …, 6) adopts a random method, and the value range of the connection weight is [0, 1]. Can make wpu＝0.5。
(2.3.3) selecting the Functions of the hidden layer nodes
The role function of the hidden layer node is selected as a Sigmoid function:
wherein n isu 3(q) node C of the hidden layer at the input of the q-th sample for the neural networkuInput of au 3(q) is the corresponding output.
(2.4) output layer D
(2.4.1) selection of number of output layer nodes
Node number S of output layer D4Equal to the number of classes K, i.e. S, of the training samples4＝K＝3。
(2.4.2) connection rights between hidden layer and output layer
Connection weight between hidden layer C and output layer D <math> <mrow> <msup> <mi>w</mi> <mn>4</mn> </msup> <mo>=</mo> <msub> <mfenced open='[' close=']'> <mtable> <mtr> <mtd> <msubsup> <mi>w</mi> <mn>11</mn> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mn>12</mn> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mn>13</mn> <mn>4</mn> </msubsup> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <msubsup> <mi>w</mi> <mrow> <mi>u</mi> <mn>1</mn> </mrow> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mrow> <mi>u</mi> <mn>2</mn> </mrow> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mrow> <mi>u</mi> <mn>3</mn> </mrow> <mn>4</mn> </msubsup> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> <mtd> </mtd> <mtd> <mo>·</mo> </mtd> </mtr> <mtr> <mtd> <msubsup> <mi>w</mi> <mn>61</mn> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mn>62</mn> <mn>4</mn> </msubsup> </mtd> <mtd> <mo>·</mo> <mo>·</mo> <mo>·</mo> </mtd> <mtd> <msubsup> <mi>w</mi> <mn>63</mn> <mn>4</mn> </msubsup> </mtd> </mtr> </mtable> </mfenced> <mrow> <mn>6</mn> <mo>×</mo> <mn>3</mn> </mrow> </msub> </mrow> </math> The initialization of (u is 1, …, 6) adopts a random method, and the value range of the weight is [0, 1%]. Can make wul＝0.5(l＝1，2，3)。
So far, the artificial neural network with the fuzzy mapping layer is constructed, and the structure diagram is shown in fig. 3.
(3) And training the artificial neural network after initialization by using a training sample set given by a user.
(3.1) selection of Convergence Condition
The convergence condition is set to e < 0.001.
(3.2) updating of connection weights between layers
And selecting the weight learning rate alpha to be 0.05 according to experience.
According to the steepest descent method, a connection weight matrix w between the mth layer and the (m is 3, 4) th layer of the artificial neural networkm(dimension is S)m-1×Sm) Updated to (r +1) th training start
wm(r+1)＝wm(r)-0.05gm(am-1)T.
Wherein
(3.3) updating parameters xi, sigma and tau of the function of fuzzy mapping layer node selects the learning rate theta of each parameter as 0.1,
ρ＝0.1。
node B for updating fuzzy mapping layer B using the following formulap(p＝1，…，S2) Three parameters xi of the action function ofp，σp，τp。
Wherein,pa2is the output matrix a of the fuzzy mapping layer B when inputting Q samples to the artificial neural network2Row p.
(3.4) termination of training
After the 1037 th training is finished, the calculation finds that e is 0.000999, the convergence condition is met, and the training is terminated.
(4) And carrying out fuzzy pruning on the features by using the trained artificial neural network, calculating the importance measure of each feature, and sequencing.
(4.1) pairs of features fiPerforming fuzzy pruning
Take the feature Sepal length as an example, for fiPruning, i.e. making node B of the mapping layer fuzzy11，B12，B13The output value of (d) is set to 0. For example, the observed value of the feature Sepal length is 5.1, the fuzzy mapping layer node B before pruning12，B12，B13The output of (1) is [0.117, 0.005, 0.009]The observed value of the characteristic Sepal width is 3.5, the fuzzy mapping layer node B before pruning21，B22，B23Is [0.100, 0.500 ]]The observed value of the characteristic Petal length is 1.4, and node B of the fuzzy mapping layer before pruning31，B32，B33The output of (1) is [0.141, 0.974, 0.028 ]]The observed value of the characteristic Petal width is 0.2, and the node B of the fuzzy mapping layer before pruning41，B42，B43Is [0.265, 0.069, 0.030 ]]Thus sample [5.1, 3.5, 1.4, 0.2%]The output of the fuzzy mapping layer before pruning is
[0.117，0.005，0.009，0.100，0.500，0.500，0.141，0.974，0.028，0.265，0.069，0.030]。
When pruning is to be performed, the output is modified to
[0.500，0.500，0.500，0.100，0.500，0.500，0.141， 0.974，0.028，0.265，0.069，0.030]。
Then, the artificial neural network after such modification is calculated for the input sample xqOutput vector a given by time-output layer4(xq,1). Pruning of other features and so on.
(4.2) calculating the importance measure of the feature FQJ (i)
Still take the feature Sepal length as an example, for f1Calculation FQJ (1):
similarly, FQJ (2) ═ 0.095858, FQJ (3) ═ 0.491984, and FQJ (4) ═ 0.511002 were calculated.
For all features fiAnd sorting the obtained features in descending order according to the sizes of the corresponding FQJ (i) values, wherein the obtained features have the following importance sequence for the classification task: petal width, Petal length, Sepal width, Sepal length.
Claims (5)
1. A feature selection method based on an artificial neural network comprises the following steps:
(1) user specifies the feature f to be selectediI-1, …, N, giving a training sample set for training an artificial neural network:
the training samples have the same dimension R, R ═ N, and are classified into K categories: omega1，…，ωKThe qth training sample xqX of the ith dimension ofqiI.e. the specified i-th feature fiThe q-th observation of (1);
(2) constructing an artificial neural network sequentially consisting of an input layer, a fuzzy mapping layer, a hidden layer and an output layer according to the training sample; the neural network data is input into the neural network from the input layer through the connection weight w2Transferring to the fuzzy mapping layer, acting by the fuzzy mapping layer, and passing through the connection weight w3Transferred to the hidden layer, acted by the hidden layer and then passed through the connection weight w4Transmitting the output layer to obtain an output, wherein m is 2, 3, 4;
(3) training the initialized artificial neural network by using a training sample set given by a user, wherein the processing procedure is as follows:
(3.1) selecting the estimator e of mean square error as a performance index in the learning process:
wherein, ti m(q) is a target value of an output of the node i of the mth layer when the qth sample is input, ai m(q) is the actual output of node i at the mth level when the qth sample is input, and G is the number of nodes at that level;
(3.2) adopting a back propagation algorithm to carry out connection weight matrix w between each layer of the artificial neural networkmTraining, wherein m is 3, 4;
(3.3) updating parameters xi, sigma and tau in the function of the fuzzy mapping layer node;
(3.4) when e meets the convergence condition, entering the step (4), and otherwise, repeating the steps (3.2) - (3.3);
(4) and carrying out fuzzy pruning on the features by using the trained artificial neural network, calculating the importance measurement of each feature, and sequencing the features according to the measurement values of the importance.
2. The method of claim 1, wherein: the step (2) comprises the following steps:
(2.1) input layer A
Input layer A node number S1Equal to the dimension R of the training sample, each node inputs a certain dimension of the training sample, and when the q sample is input by the neural network, the node A of the input layeriThe inputs of (a) are:
the output is:
(2.2) fuzzy mapping layer B
Fuzzy mapping of node number of layer B <math> <mrow> <msup> <mi>S</mi> <mn>2</mn> </msup> <mo>=</mo> <munderover> <mi>Σ</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <msup> <mi>S</mi> <mn>1</mn> </msup> </munderover> <msub> <mi>m</mi> <mi>i</mi> </msub> <mo>,</mo> </mrow> </math> miThe selection of the value needs to satisfy the following conditions:
wherein Q ismin＝min{Ql}，QlRepresenting the class omega in the training sample given by the userlThe number of samples of (a);
node A of the input layeriNode B with fuzzy mapping layeri1，…，BimiNode B connected by connection weights and having fuzzy mapping layeri1，…，BimlDo not interact with except AiOther nodes of the other input layer are connected; feature level a node aiAnd fuzzy mapping layer node BijThe connection weight between the two is constantly 1; blurring node B of mapping layer when q sample is inputted by neural networkijThe inputs of (a) are:
node B of fuzzy mapping layerijThe function of (a) is a fuzzy membership function muijGiven the ith dimension feature fiA fuzzy membership function of (1) means that a mapping mu is giveni：fi→[0，1]；
Node BijThe fuzzy membership function of (a) is of the form:
nij 2(q) node B of the fuzzy mapping layer when the q-th sample is inputijInput of aij 2(q) is the corresponding actual output, ξijIs a node BijClass of conditional probability density of σijIs a node BijStandard deviation of the class of conditional probability densities, τijIs a node BijOne of the parameters of (a);
(2.3) hidden layer C
Number of nodes S of hidden layer C3The number of classes K of the samples is greater than or equal to; the fuzzy mapping layer B and the hidden layer C are all connected, and the connection weight between the fuzzy mapping layer B and the hidden layer C
Wherein p is 1, …, S2，u＝1，…，S3The initialization of (a) takes a random approach,the value range of the connection weight is [0, 1]]；
When the q sample is input to the neural network, node C of the hidden layeru(u＝1，…，S3) The inputs of (a) are:
wherein, ap 2(q) node B which is a fuzzy mapping layerp(p＝1，…，S2) Output at the input of the q sample of the neural network, wpu 3Node B being a fuzzy mapping layerpAnd hidden layer node CuThe right of connection between;
the role function of the hidden layer node is selected as a Sigmoid function or a hyperbolic tangent function:
wherein n isu 3(q) node C of the hidden layer at the input of the q-th sample for the neural networkuInput of au 3(q) is the corresponding output;
wherein n isu 3(q) node C of the hidden layer at the input of the q-th sample for the neural networkuInput of au 3(q) is the corresponding output;
(2.4) output layer D
Node number S of output layer D4Is equal to the number of classes K of the training sample; the hidden layer C and the output layer D are fully connected; the connection weight between the hidden layer C and the output layer D is as follows:
wherein u is 1, …, S3，l＝1，…，S4The initialization of (2) adopts a random method, and the value range of the weight is [0, 1]]；
Output layer node Dl(l＝1，…，S4) Input and output of (D) are equallOutput value n ofl 4(q) is that the q-th sample of the neural network input belongs to the class ωlProbability of (c):
wherein, wul 4Node C being a hidden layeruAnd an output layer node DlThe right of connection between.
3. The method according to claim 1 or 2, characterized in that: the treatment processes of the steps (3.2) and (3.3) are as follows:
(3.2) training the connection weight w between the fuzzy mapping layer B and the hidden layer Cm：
The sensitivity of the estimator of the mean square error e to the input of the mth layer is defined as
Wherein S ismIs the number of nodes at the mth layer of the artificial neural network, nmIs one size of SmA matrix of xQ representing the input of the mth layer of the artificial neural network; n isi m(q) represents a node i of the m-th layerThe input at the time when the q-th sample is input to the neural network, and, <math> <mrow> <mfrac> <mrow> <mo>∂</mo> <mi>e</mi> </mrow> <mrow> <msubsup> <mrow> <mo>∂</mo> <mi>n</mi> </mrow> <mi>i</mi> <mi>m</mi> </msubsup> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>=</mo> <mfrac> <mrow> <mo>∂</mo> <mi>e</mi> </mrow> <mrow> <msubsup> <mrow> <mo>∂</mo> <mi>a</mi> </mrow> <mi>i</mi> <mi>m</mi> </msubsup> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>×</mo> <mfrac> <mrow> <msubsup> <mrow> <mo>∂</mo> <mi>a</mi> </mrow> <mi>i</mi> <mi>m</mi> </msubsup> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> <mrow> <msubsup> <mrow> <mo>∂</mo> <mi>n</mi> </mrow> <mi>i</mi> <mi>m</mi> </msubsup> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>;</mo> </mrow> </math>
connection weight matrix w between mth layer and m-1 layer of artificial neural networkmIs Sm-1×SmAnd m is 3, 4, and is updated to be at the beginning of the (r +1) th training
wm(r+1)＝wm(r)-αgm(am-1)T.
Wherein alpha is weight learning rate, the value range is more than 0 and less than or equal to 1, r is the training frequency, amIs one size of SmA matrix of x Q, representing the actual output of the mth layer of the artificial neural network:
(3.3) node B of fuzzy mapping layer Bp(p＝1，…，S2) Three parameters xi of the action function ofp，σp，τpUpdated according to the following formula, wherein theta is xipThe learning rate of (a) is determined,is σpRho is τpLearning rate of (d):
wherein,pa2fuzzy mapping layer B output matrix a when inputting Q samples to artificial neural network2Line p of (2), and
wherein, ai 1(q) is with node BpConnected input layer node AiThe output when the q sample is input into the neural network is xqi。
4. The method according to claim 1 or 2, characterized in that: the processing procedure of the step (4) is as follows:
(4.1) pairs of features fiPerforming fuzzy pruning to make the output of the fuzzy mapping layer as
Obtaining the input sample x of the artificial neural network under the conditionqOutput vector a given by time-output layer4(xq，i)；
(4.2) calculating the importance measure of the feature FQJ (i)
The feature metric function FQJ (i) represents the ith dimension feature fiFor the importance of classification, fqj (i) is defined as follows:
wherein, a4(xq) Representing an artificial neural network for an input sample xqOutput vector given by time input layer, a4(xqI) represents perthSign fiInput sample x for artificial neural network after fuzzy pruningqThe given output vector uses the artificial neural network trained in the step (3) to give all the features f given by the user in the step (1.1)iCalculating the corresponding FQJ (i), characteristic f according to the formulaiThe value of (fqj), (i) is a measure of its importance;
(4.3) for all characteristics fiSorted by their importance measure fqj (i).
5. The method of claim 3, wherein: the processing procedure of the step (4) is as follows:
(4.1) pairs of features fiPerforming fuzzy pruning to make the output of the fuzzy mapping layer as
Obtaining the input sample x of the artificial neural network under the conditionqOutput vector a given by time-output layer4(xq，i)；
(4.2) calculating the importance measure of the feature FQJ (i)
The feature metric function FQJ (i) represents the ith dimension feature fiFor the importance of classification, fqj (i) is defined as follows:
wherein, a4(xq) Representing an artificial neural network for an input sample xqOutput vector given by time input layer, a4(xqI) represents a pair of features fiInput sample x for artificial neural network after fuzzy pruningqThe given output vector uses the artificial neural network trained in the step (3) to give all the features f given by the user in the step (1.1)iCalculating the corresponding FQJ (i), characteristic f according to the formulaiThe value of (fqj), (i) is a measure of its importance;
(4.3) for all characteristics fiSorted by their importance measure fqj (i).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100195700A CN100367300C (en) | 2006-07-07 | 2006-07-07 | Characteristic selecting method based on artificial nerve network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006100195700A CN100367300C (en) | 2006-07-07 | 2006-07-07 | Characteristic selecting method based on artificial nerve network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1945602A true CN1945602A (en) | 2007-04-11 |
| CN100367300C CN100367300C (en) | 2008-02-06 |
Family
ID=38045000
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2006100195700A Active CN100367300C (en) | 2006-07-07 | 2006-07-07 | Characteristic selecting method based on artificial nerve network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100367300C (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101187649B (en) * | 2007-12-12 | 2010-04-07 | 哈尔滨工业大学 | Heterogeneous material diffusion welding interface defect automatic identification method |
| CN101441728B (en) * | 2007-11-21 | 2010-09-08 | 新乡市起重机厂有限公司 | Neural network method of crane optimum design |
| CN101882238A (en) * | 2010-07-15 | 2010-11-10 | 长安大学 | Wavelet neural network processor based on SOPC (System On a Programmable Chip) |
| CN101350069B (en) * | 2007-06-15 | 2010-11-17 | 三菱电机株式会社 | Computer implemented method for constructing classifier from training data and detecting moving objects in test data using classifier |
| CN101079109B (en) * | 2007-06-26 | 2011-11-30 | 北京中星微电子有限公司 | Identity identification method and system based on uniform characteristic |
| CN101510262B (en) * | 2009-03-17 | 2012-05-23 | 江苏大学 | Automatic measurement method for separated-out particles in steel and morphology classification method thereof |
| CN102609612A (en) * | 2011-12-31 | 2012-07-25 | 电子科技大学 | Data fusion method for calibration of multi-parameter instruments |
| CN103425994A (en) * | 2013-07-19 | 2013-12-04 | 淮阴工学院 | Feature selecting method for pattern classification |
| CN103606007A (en) * | 2013-11-20 | 2014-02-26 | 广东省电信规划设计院有限公司 | Target identification method and apparatus based on Internet of Things |
| CN103759290A (en) * | 2014-01-16 | 2014-04-30 | 广东电网公司电力科学研究院 | Large coal-fired unit online monitoring and optimal control system and implementation method thereof |
| CN104504443A (en) * | 2014-12-09 | 2015-04-08 | 河海大学 | Feature selection method and device based on RBF (Radial Basis Function) neural network sensitivity |
| WO2016123409A1 (en) * | 2015-01-28 | 2016-08-04 | Google Inc. | Batch normalization layers |
| CN106885228A (en) * | 2017-02-10 | 2017-06-23 | 青岛高校信息产业股份有限公司 | A kind of boiler coal-air ratio optimization method and system |
| CN107292387A (en) * | 2017-05-31 | 2017-10-24 | 汪薇 | A kind of method that honey quality is recognized based on BP |
| CN107480686A (en) * | 2016-06-08 | 2017-12-15 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of screening machine learning characteristic |
| CN107707657A (en) * | 2017-09-30 | 2018-02-16 | 苏州涟漪信息科技有限公司 | Safety custody system based on multisensor |
| CN108229533A (en) * | 2017-11-22 | 2018-06-29 | 深圳市商汤科技有限公司 | Image processing method, model pruning method, device and equipment |
| CN109587248A (en) * | 2018-12-06 | 2019-04-05 | 腾讯科技（深圳）有限公司 | User identification method, device, server and storage medium |
| CN109726812A (en) * | 2017-10-31 | 2019-05-07 | 通用电气公司 | Feature ordering neural network and method generate the method for simplifying feature set model |
| CN109754077A (en) * | 2017-11-08 | 2019-05-14 | 杭州海康威视数字技术股份有限公司 | Network model compression method, device and the computer equipment of deep neural network |
| CN114398983A (en) * | 2022-01-14 | 2022-04-26 | 腾讯科技（深圳）有限公司 | Classification prediction method, classification prediction device, classification prediction equipment, storage medium and computer program product |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6947586B2 (en) * | 2000-04-24 | 2005-09-20 | International Remote Imaging Systems, Inc. | Multi-neural net imaging apparatus and method |
| WO2002061678A2 (en) * | 2001-01-31 | 2002-08-08 | Prediction Dynamics Limited | Feature selection for neural networks |
| JP3896868B2 (en) * | 2002-02-27 | 2007-03-22 | 日本電気株式会社 | Pattern feature selection method, classification method, determination method, program, and apparatus |
-
2006
- 2006-07-07 CN CNB2006100195700A patent/CN100367300C/en active Active
Cited By (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101350069B (en) * | 2007-06-15 | 2010-11-17 | 三菱电机株式会社 | Computer implemented method for constructing classifier from training data and detecting moving objects in test data using classifier |
| CN101079109B (en) * | 2007-06-26 | 2011-11-30 | 北京中星微电子有限公司 | Identity identification method and system based on uniform characteristic |
| CN101441728B (en) * | 2007-11-21 | 2010-09-08 | 新乡市起重机厂有限公司 | Neural network method of crane optimum design |
| CN101187649B (en) * | 2007-12-12 | 2010-04-07 | 哈尔滨工业大学 | Heterogeneous material diffusion welding interface defect automatic identification method |
| CN101510262B (en) * | 2009-03-17 | 2012-05-23 | 江苏大学 | Automatic measurement method for separated-out particles in steel and morphology classification method thereof |
| CN101882238A (en) * | 2010-07-15 | 2010-11-10 | 长安大学 | Wavelet neural network processor based on SOPC (System On a Programmable Chip) |
| CN102609612A (en) * | 2011-12-31 | 2012-07-25 | 电子科技大学 | Data fusion method for calibration of multi-parameter instruments |
| CN102609612B (en) * | 2011-12-31 | 2015-05-27 | 电子科技大学 | Data fusion method for calibration of multi-parameter instruments |
| CN103425994A (en) * | 2013-07-19 | 2013-12-04 | 淮阴工学院 | Feature selecting method for pattern classification |
| CN103425994B (en) * | 2013-07-19 | 2016-09-21 | 淮阴工学院 | A kind of feature selection approach for pattern classification |
| CN103606007A (en) * | 2013-11-20 | 2014-02-26 | 广东省电信规划设计院有限公司 | Target identification method and apparatus based on Internet of Things |
| CN103606007B (en) * | 2013-11-20 | 2016-11-16 | 广东省电信规划设计院有限公司 | Target identification method based on Internet of Things and device |
| CN103759290A (en) * | 2014-01-16 | 2014-04-30 | 广东电网公司电力科学研究院 | Large coal-fired unit online monitoring and optimal control system and implementation method thereof |
| CN104504443A (en) * | 2014-12-09 | 2015-04-08 | 河海大学 | Feature selection method and device based on RBF (Radial Basis Function) neural network sensitivity |
| US10902319B2 (en) | 2015-01-28 | 2021-01-26 | Google Llc | Batch normalization layers |
| US10417562B2 (en) | 2015-01-28 | 2019-09-17 | Google Llc | Batch normalization layers |
| US12033073B2 (en) | 2015-01-28 | 2024-07-09 | Google Llc | Batch normalization layers |
| US11893485B2 (en) | 2015-01-28 | 2024-02-06 | Google Llc | Batch normalization layers |
| US11853885B2 (en) | 2015-01-28 | 2023-12-26 | Google Llc | Image classification using batch normalization layers |
| US11308394B2 (en) | 2015-01-28 | 2022-04-19 | Google Llc | Image classification using batch normalization layers |
| RU2666308C1 (en) * | 2015-01-28 | 2018-09-06 | Гугл Инк. | Package normalization layers |
| US11281973B2 (en) | 2015-01-28 | 2022-03-22 | Google Llc | Batch normalization layers |
| WO2016123409A1 (en) * | 2015-01-28 | 2016-08-04 | Google Inc. | Batch normalization layers |
| US10628710B2 (en) | 2015-01-28 | 2020-04-21 | Google Llc | Image classification using batch normalization layers |
| EP3483795A1 (en) * | 2015-01-28 | 2019-05-15 | Google LLC | Batch normalization layers |
| CN107480686A (en) * | 2016-06-08 | 2017-12-15 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus of screening machine learning characteristic |
| CN106885228A (en) * | 2017-02-10 | 2017-06-23 | 青岛高校信息产业股份有限公司 | A kind of boiler coal-air ratio optimization method and system |
| CN107292387A (en) * | 2017-05-31 | 2017-10-24 | 汪薇 | A kind of method that honey quality is recognized based on BP |
| CN107707657B (en) * | 2017-09-30 | 2021-08-06 | 苏州涟漪信息科技有限公司 | Safety monitoring system based on multiple sensors |
| CN107707657A (en) * | 2017-09-30 | 2018-02-16 | 苏州涟漪信息科技有限公司 | Safety custody system based on multisensor |
| CN109726812A (en) * | 2017-10-31 | 2019-05-07 | 通用电气公司 | Feature ordering neural network and method generate the method for simplifying feature set model |
| CN109754077A (en) * | 2017-11-08 | 2019-05-14 | 杭州海康威视数字技术股份有限公司 | Network model compression method, device and the computer equipment of deep neural network |
| CN109754077B (en) * | 2017-11-08 | 2022-05-06 | 杭州海康威视数字技术股份有限公司 | Network model compression method and device of deep neural network and computer equipment |
| CN108229533A (en) * | 2017-11-22 | 2018-06-29 | 深圳市商汤科技有限公司 | Image processing method, model pruning method, device and equipment |
| CN109587248A (en) * | 2018-12-06 | 2019-04-05 | 腾讯科技（深圳）有限公司 | User identification method, device, server and storage medium |
| CN109587248B (en) * | 2018-12-06 | 2023-08-29 | 腾讯科技（深圳）有限公司 | User identification method, device, server and storage medium |
| CN114398983A (en) * | 2022-01-14 | 2022-04-26 | 腾讯科技（深圳）有限公司 | Classification prediction method, classification prediction device, classification prediction equipment, storage medium and computer program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN100367300C (en) | 2008-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1945602A (en) | Characteristic selecting method based on artificial nerve network | |
| CN1214340C (en) | Multi-neural net imaging appts. and method | |
| CN1053247C (en) | Apparatus for predicting flow of water supplying | |
| CN100336070C (en) | Method of robust human face detection in complicated background image | |
| CN1846218A (en) | An artificial neural network | |
| CN100346856C (en) | Rectification tower automatic control and optimization method | |
| CN1818927A (en) | Fingerprint identifying method and system | |
| CN1100541A (en) | Neural network and method of using same | |
| CN1286066C (en) | Paper securities true-false distinguishing method and paper securities input direction distinguishing method | |
| CN101050971A (en) | Dynamic soft measuring and form establishing method base pulse response formwork and parameter optumization | |
| CN1287641A (en) | Method and apparatus for performing pattern dictionary formation for use in sequence homology detection | |
| CN1794266A (en) | Biocharacteristics fusioned identity distinguishing and identification method | |
| CN1894703A (en) | Pattern recognition method, and device and program therefor | |
| CN1691740A (en) | Magnified display apparatus and magnified image control apparatus | |
| CN1869992A (en) | Person searching device, person searching method and access control system | |
| CN1912891A (en) | Face meta-data creation equipment and method, face distinguishing system and method | |
| CN1310825A (en) | Methods and apparatus for classifying text and for building a text classifier | |
| CN1696959A (en) | Detector for special shooted objects | |
| CN1211769A (en) | Method and equipment for file retrieval based on Bayesian network | |
| CN1871622A (en) | Image collation system and image collation method | |
| CN101030257A (en) | File-image cutting method based on Chinese characteristics | |
| CN1195830A (en) | Stripe pattern matching system | |
| CN1170251C (en) | Beer-bottle raised character fetching-identifying hardware system and processing method | |
| CN1905383A (en) | Shared frequency cell channel estimating apparatus and method | |
| CN101030366A (en) | Information processing apparatus, information processing method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: SHENZHEN RUIMING TECHNOLOGY CO., LTD. Free format text: FORMER OWNER: HUAZHONG SCINECE AND TECHNOLOGY UNIV Effective date: 20100804 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: 430074 NO.1037, LUOYU ROAD, HONGSHAN DISTRICT, WUHAN CITY, HUBEI PROVINCE TO: 518057 5/F, BUILDING 3, SHENZHEN SOFTWARE PARK, KEJIZHONGER ROAD, NEW + HIGH-TECHNOLOGY ZONE, NANSHAN DISTRICT, SHENZHEN CITY, GUANGDONG PROVINCE |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20100804 Address after: 518057, five floor, building three, Shenzhen Software Park, two road, Nanshan District hi tech Zone, Shenzhen, Guangdong Patentee after: Shenzhen Streaming Video Technology Co., Ltd. Address before: 430074 Hubei Province, Wuhan city Hongshan District Luoyu Road No. 1037 Patentee before: Huazhong University of Science and Technology |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Characteristic selecting method based on artificial nerve network Effective date of registration: 20130110 Granted publication date: 20080206 Pledgee: Shenzhen SME credit financing guarantee Group Co., Ltd. Pledgor: Shenzhen Streaming Video Technology Co., Ltd. Registration number: 2013990000024 |
|
| PLDC | Enforcement, change and cancellation of contracts on pledge of patent right or utility model | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20140318 Granted publication date: 20080206 Pledgee: Shenzhen SME credit financing guarantee Group Co., Ltd. Pledgor: Shenzhen Streaming Video Technology Co., Ltd. Registration number: 2013990000024 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Characteristic selecting method based on artificial nerve network Effective date of registration: 20140318 Granted publication date: 20080206 Pledgee: Shenzhen SME credit financing guarantee Group Co., Ltd. Pledgor: Shenzhen Streaming Video Technology Co., Ltd. Registration number: 2014990000174 |
|
| PLDC | Enforcement, change and cancellation of contracts on pledge of patent right or utility model | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20150528 Granted publication date: 20080206 Pledgee: Shenzhen SME credit financing guarantee Group Co., Ltd. Pledgor: Shenzhen Streaming Video Technology Co., Ltd. Registration number: 2014990000174 |
|
| PLDC | Enforcement, change and cancellation of contracts on pledge of patent right or utility model | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Characteristic selecting method based on artificial nerve network Effective date of registration: 20150603 Granted publication date: 20080206 Pledgee: Shenzhen SME financing Company limited by guarantee Pledgor: Shenzhen Streaming Video Technology Co., Ltd. Registration number: 2015990000430 |
|
| PLDC | Enforcement, change and cancellation of contracts on pledge of patent right or utility model | ||
| C56 | Change in the name or address of the patentee |
Owner name: SHENZHEN STREAMAX TECHNOLOGY CO., LTD. Free format text: FORMER NAME: SHENZHEN RUIMING TECHNOLOGY CO., LTD. |
|
| CP03 | Change of name, title or address |
Address after: Nanshan District Xueyuan Road in Shenzhen city of Guangdong province 518000 No. 1001 Nanshan Chi Park building B1 building 21-23 Patentee after: STREAMAX TECHNOLOGY CO., LTD. Address before: 518057, five floor, building three, Shenzhen Software Park, two road, Nanshan District hi tech Zone, Shenzhen, Guangdong Patentee before: Shenzhen Streaming Video Technology Co., Ltd. |
|
| DD01 | Delivery of document by public notice |
Addressee: Chen Dan Document name: Notification of Passing Examination on Formalities |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Date of cancellation: 20160718 Granted publication date: 20080206 Pledgee: Shenzhen SME financing Company limited by guarantee Pledgor: STREAMAX TECHNOLOGY CO., LTD. Registration number: 2015990000430 |
|
| PLDC | Enforcement, change and cancellation of contracts on pledge of patent right or utility model | ||
| PM01 | Change of the registration of the contract for pledge of patent right |
Change date: 20160718 Registration number: 2015990000430 Pledgor after: STREAMAX TECHNOLOGY CO., LTD. Pledgor before: Shenzhen Streaming Video Technology Co., Ltd. |