JP2012238075A - Feature selecting device, feature selecting method, and feature selecting program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a feature selecting device with improved generalizing property.SOLUTION: The feature selecting device for selecting features of samples using genetic algorithm includes: means for calculating distances of all learning samples from a hyperplane that is a determination boundary between categories; means for selecting one of all the learning samples, defining it as a reference sample, and setting two hyperplanes in both the sides of the determined boundary at positions of a distance between the reference sample and the determined boundary; means for calculating penalty values for learning samples projecting from the set two hyperplanes, according to the lengths projecting from the two hyperplanes; means for calculating an evaluation measure of robustness based on the interval between the two hyperplanes and the penalty values and defining a value of the evaluation measure as an evaluation value of the reference sample; and means for calculating the evaluation value, for each of all the learning samples, when using them as the reference sample, using the reference sample evaluation means and outputting a minimal value among them as the selected feature evaluation value.

Description

  The present invention relates to a technique for selecting features (hereinafter referred to as feature selection), which is one of the techniques for improving the recognition accuracy of pattern recognition, and in particular, a technique using a genetic algorithm (hereinafter referred to as GA). About.

  As a conventional technique for feature selection using GA, a database consisting of many patterns for learning and each result category information is used, and the combination of features is changed and the identification and aggregation of the identification rate are performed using the database. Is known, and a method for performing a search with the identification rate as fitness is performed by GA (see, for example, Non-Patent Document 1).

Yoshihiko Hamamoto, Yuri Furusato, Tomoya Kanayama, Satoshi Tomita: “Feature selection method using genetic algorithm”, theory of theory (A), vol. J78-A, no. 10, pp. 1385-1389 (1995).

  Conventional feature selection techniques cannot increase generalization ability. The generalization ability in identification is the ability to identify not only a learning sample but also an unknown sample with a high identification rate, and is a very important characteristic for practical use. According to the idea of linear SVM, which is a discriminator that emphasizes generalization ability, increasing the “margin” that is the Euclidean distance between the decision boundary and the adjacent learning sample as much as possible increases the generalization ability of the discriminator. Although effective, it has never been a feature selection technique based on this concept.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a feature selection device, a feature selection method, and a feature selection program that can realize a feature selection technique that increases generalization ability. .

  The present invention is a feature selection device that improves the recognition accuracy of pattern recognition by selecting features of a sample using a genetic algorithm, and is used for all learning from a hyperplane that is a decision boundary between categories. A distance calculation means for calculating the distance of the sample and one of all the learning samples is selected as a reference sample, and two hyperplanes are provided at each of the reference sample and the determination boundary distance positions on both sides of the determination boundary. Margin setting means for setting, penalty calculation means for calculating a penalty value according to the length of the learning sample that protrudes from the two hyperplanes that is set, and the length that protrudes from the two hyperplanes; Based on an interval between two hyperplanes and the penalty value, a robustness evaluation scale is calculated, and the value of the evaluation scale is set as the evaluation value of the reference sample. For each learning sample using the reference sample evaluation means, the evaluation value when using them as a reference sample is calculated, and the minimum value among them is selected for the selected feature. An evaluation means for outputting as an evaluation value is provided.

  The present invention provides a distance calculation means, a margin setting means, a penalty calculation means, a reference sample evaluation means, in order to improve pattern recognition identification accuracy by selecting sample features using a genetic algorithm. A distance selection step in which the distance calculation means calculates the distances of all learning samples from the hyperplane that is a decision boundary between categories; The margin setting means selects one of all learning samples as a reference sample, and sets margins for setting two hyperplanes at each of the distance between the reference sample and the decision boundary on both sides of the decision boundary. A step and a penalty calculating means for the learning sample that protrudes from the set two hyperplanes. A penalty calculation step for calculating a penalty value according to a length protruding from the hyperplane, and the reference sample evaluation means determines a robustness evaluation scale based on the interval between the two hyperplanes and the penalty value. A reference sample evaluation step that calculates and uses the value of the evaluation scale as the evaluation value of the reference sample and the evaluation means use the reference sample evaluation means for each learning sample as a reference sample. And an evaluation step for calculating a minimum evaluation value as an evaluation value of the selected feature.

  The present invention is a feature selection program for causing a computer on a feature selection device to improve pattern recognition discrimination accuracy by selecting sample features using a genetic algorithm, A distance calculating step for calculating distances of all learning samples from the hyperplane which is a decision boundary, and selecting one of all learning samples as a reference sample, and the reference samples on both sides of the decision boundary; A margin setting step for setting two hyperplanes at each position of the decision boundary distance, and a length of the learning sample that protrudes from the two hyperplanes set to a length that protrudes from the two hyperplanes. Based on the penalty calculation step for calculating the corresponding penalty value, the interval between the two hyperplanes, and the penalty value, And a reference sample evaluation step using the evaluation scale value as the evaluation value of the reference sample and the reference sample evaluation step to use them as reference samples for all learning samples. An evaluation value is calculated by the computer, and an evaluation step of outputting a minimum value among them as an evaluation value of the selected feature is performed.

  According to the present invention, in the feature selection using the genetic algorithm, since the evaluation scale based on the soft margin of the linear SVM is used as the fitness of the chromosome, the feature selection having a high generalization ability is performed as in the linear SVM. The effect that it can be obtained.

It is a block diagram which shows the structure of one Embodiment of this invention. It is explanatory drawing which shows the structure of a chromosome. It is a block diagram which shows the structure of the chromosome evaluation means 8 shown in FIG. FIG. 5 is a schematic diagram showing the distribution of ψ in a selected feature space, where ψ is a set of learning samples and W ₁ and W ₂ are standard patterns. It is a conceptual diagram of the standard pattern in a feature space, a decision boundary, and the positional relationship between samples. It is a flowchart which shows the processing operation of the feature selection apparatus 1 shown in FIG. It is explanatory drawing which shows the structure of the all-generation chromosome set storage means 4 shown in FIG.

  A feature selection apparatus according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 1 denotes a feature selection device configured by a computer device. Reference numeral 2 denotes overall control means for controlling the operation by controlling the entire apparatus. Reference numeral 3 denotes an initial chromosome set creating means for creating an initial chromosome set. Reference numeral 4 denotes all generation chromosome set storage means for storing all generations of chromosome sets. Reference numeral 5 denotes selection probability value calculation means for calculating a selection probability value. Reference numeral 6 denotes a generation number counter for counting the number of generations. Reference numeral 7 denotes an individual generation execution unit. Reference numeral 71 denotes crossover execution means for performing crossover processing. Reference numeral 72 denotes a mutation execution means for performing a mutation process. Reference numeral 73 denotes a duplication execution means for executing duplication processing. Reference numeral 74 denotes chromosome set rearranging means for rearranging chromosome sets. Reference numeral 8 is a chromosome evaluation means for evaluating a chromosome.

  First, the identification method will be described. There are various identification methods. In this embodiment, the minimum distance identification is performed with the category corresponding to the standard pattern having the minimum value among the Euclidean distances between the input pattern and the standard pattern of each category as the identification result. We will use it. In the case of the two-category problem in the minimum distance identification, the decision boundary is a hyperplane that bisects a line segment connecting two standard patterns. The standard pattern is the average of the learning samples.

Next, the structure of the chromosome will be described with reference to FIG. FIG. 2 is a diagram showing the structure of a chromosome. Assuming that the number of dimensions of the original feature is L, a vector expressing a dimension to be selected in the L dimension is assumed, and this is called a feature selection vector λ. In the present embodiment, this feature selection vector is used as it is as a chromosome in GA. The number of features selected by λ is represented by K, and a feature selection vector is constructed as shown in FIG. This vector indicates whether or not the feature corresponding to each individual element is used. “1” indicates “use” and “0” indicates “not use”. In FIG. 2, the applicable range of “use” is represented by a dashed frame indicated by “1” in each dimension. The value of the individual _{elements, λ 1, λ 2, ···} , expressed in lambda _L.

Next, the configuration of the chromosome evaluation means 8 shown in FIG. 1 will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the chromosome evaluation means 8 shown in FIG. The set of original features of the learning sample is Ψ, the number of learning samples is N, the individual elements of Ψ are X _i (i = 1, 2,..., N), and the information of the category to which X _i belongs is y _i . And category 1 is represented by y _i = 1 and category 2 is represented by y _i = −1. This y _i is called a teacher signal. Assume that the number of categories to be identified is 2, each category is category 1 and category 2, and the number of dimensions of the original feature is L. The learning sample set storage means 81 stores Ψ and all teacher signals in advance as a set of original features of the learning sample.

The standard pattern creation means 82 in the original feature space operates first when the feature selection device 1 is activated. The operation of the overall control unit 2 is started after the processing of the standard pattern creating unit in the original feature space is completed. Creating means 82 standard patterns in the original feature space, the standard pattern in the original feature space created as follows, X _1, X _2, · · ·, category 1 and category elements of X _N with the teacher signal Dividing into two, the average is calculated for each, and these are represented by M ₁ and M ₂ as standard patterns of categories 1 and 2 in the original feature space.

The sample-to-category distance calculation unit 85 calculates the distance between the sample input from the sample distance calculation unit 84 from the decision boundary and the category k (k = 1, 2) on the assumption that λ is applied. Calculate as follows. First, the basic distance between vectors based on the application of λ is as follows. Assume that two vectors X _A and X _B in the original feature space and a feature selection vector λ are given. Let X _A = (X _A1 , X _A2 ,..., X _AL ), X _B = (X _B1 , X _B2 ,..., X _BL ). In this case, among the elements _{X Ai} of the vector _{X A (1 ≦ i ≦ L} ), the corresponding lambda _i extracts only elements equal to 1, are collectively K-dimensional vector _{to X} A = (to X _{A1, ~X A2, ···, ~X} AK) (~ is attached to the head of the X, to create a hereinafter the same). A K-dimensional vector ˜X _B = (˜X _B1 , ˜X _B2 ,..., ˜X _BK ) is created from X _{B in the} same manner. Using these two, the distance between X _A and X _{B on} the assumption of λ is calculated by the function f _{D in the} equation (1). Equation (1) indicates that f _d is the Euclidean distance calculated using only the feature selected by the feature selection vector λ.

Next, with reference to the distance between the standard patterns of category 1 and category 2, a method of scaling the coordinate system of the feature space so that it always becomes a constant value “1” even when λ changes will be described. First, let X be an original feature of an arbitrary sample, and create a vector from the feature selected by λ from this, and let this be ~ X (~ is attached to the head of X). Further, a vector is created with the feature selected by λ from the standard pattern M _k (k = 1, 2) of the original feature, and this is added to ~ M _k (k = 1, 2) (˜ is the head of M). The same shall apply hereinafter. Using this ~ Mk (k = 1, 2), ~ X receives the conversion of equation (2) and is converted to a K-dimensional vector x.

The same conversion as that of the equation (2) is also performed by the equation (3) for the standard pattern of the category k (k = 1, 2).

W _{k in} equation (3) is a standard pattern after normalization, which is also a K-dimensional vector. According to the equation (3), a standard pattern is newly created every time λ changes. The distance calculation means 85 between the sample and the category calculates and outputs the distance between an arbitrary sample and the category k (k = 1, 2) using f _d (x, W _k ).

Next, with reference to FIG. 3, the operation | movement which evaluates the chromosome (lambda) by the chromosome evaluation means 8 is demonstrated. When the chromosome λ is inputted, the chromosome evaluation means 8 operates in the order of the sample distance calculation means 84 and the robustness evaluation means 86 from the decision boundary, evaluates them, and outputs a score. This score is called fitness. The sample distance calculation means 84 from the decision boundary first creates a set obtained by converting each element of the learning sample set Ψ by the equation (2), and this is set as {x ₁ , x ₂ ,..., X _N } And represented by ψ.

In the feature space that is scale-transformed so that the distance between the standard patterns of category 1 and category 2 is 1, if the set of learning samples is ψ and the standard patterns are W ₁ and W ₂ , A schematic diagram of the distribution of ψ is shown in FIG. In FIG. 4, black circles represent category 1 samples, and white circles represent category 2 samples. H ₀ represents the hyperplane of the decision boundary between categories.

Next, the sample distance calculation means 84 from the decision boundary includes the Euclidean distance h (x _i ) (i = 1, 1) from the decision boundary H ₀ to x _i (i = 1, 2,..., N). 2, ..., N). Specifically, the distance calculation means between the sample and the category is activated to calculate the distances f _D (x _i , W ₁ ) and f _D (x _i , W ₂ ) between the categories 1 and 2, and the calculation result Is used to calculate h (x _i ) according to equation (4).

FIG. 5 shows a conceptual diagram of a standard pattern, a decision boundary, and a positional relationship between samples in the feature space. H (x _i ) calculated by equation (4) has a positive or negative sign, and when x _i is on the category 1 side when viewed from H _0, h (x _i )> 0, on the category 2 side H (x _i ) <0. The correctness of the identification result can be determined by the sign of the product with the teacher signal _yi . If h (x _i ) · y _i > 0, the correct identification result is obtained. If h (x _i ) · y _i <0, the error is incorrect. Means the identification result.

Next, the evaluation means 86 of the robustness checks the sign of _{h (x i)} · _y i for the individual _{x i,} described below, if _{h (x i) · y i} > 0 (1) ~ ( 3) by calculating the evaluation measure _{S i} of robustness in _{x i, h (x i)} · y if i <0 (1) ~ ( 3) sufficiently larger positive value than the range of values calculated by Let S _i be a constant. Then, after the calculation of the S _i for all x _i, calculate the minimum value from a set of S _i, and outputs the value as the fitness of the chromosome lambda.

(1) Margin setting Hyperplanes H ₁ and H ₂ are set at positions | h (x _i ) | on both sides of H ₀ . X _i is included in either H ₁ or H ₂ . Then, an interval 2 · | h (x _i ) | between H ₁ and H ₂ is calculated, and this is represented by 2 / ω _i . That is, the x _i corresponds to support vector linear SVM, referred to herein as "support point". Further, 2 / ω _i means a margin as in the case of the linear SVM.

(2) Penalty calculation Penalty is calculated for the sample protruding from the margin set in (1) above. The feature space is divided into two by H ₁ into a region on the H ₀ side and a region on the opposite side, and the sample of category 1 included in the former region is regarded as reducing the degree of distribution separation from H _1. A penalty proportional to the distance is given. A similar process is performed on the category 2 sample using H ₂ . Formula (5) formulates this.

In equation (5), ξ _ij is a penalty given to the individual element x _{j of} ψ. (5) (if the ie _{h (x j)> 1 /} ω i) _{if x j} is protruding from the margin, the length protruding _{(1 / ω i -y (j} ) · h It means that (1-y (j) · h (x _j ) · ω _i ), which is the ratio of (x _j ) to 1 / ω _i , is given as a penalty, and 0 is given if it does not protrude. Then, the total value Σ ^N _{j = 1} ξ _ij is calculated as a penalty as a whole.

(3) Final calculation of S _{i Using} the margin value 2 / ω _i calculated in (1) and the penalty value Σ ^N _{j = 1} ξ _ij calculated in (2), let S _{i be} Calculate with equation (6).

In the equation (6), the first term is a term for increasing the margin, and the second term is a penalty term for the sample protruding from the margin. C is a constant for balancing the first term and the second term, and is determined experimentally. Since _Si is a better value as it is lower, the minimum value is selected according to Equation (7).

  Next, the processing operation of the feature selection device 1 shown in FIG. 1 will be described with reference to FIG. FIG. 6 is a flowchart showing the processing operation of the feature selection device 1 shown in FIG. The feature selection device 1 basically operates based on a genetic algorithm (GA). The generation number (number of generations) in GA is GN. The overall control means 2 first sets GN = 1, and sets this value in the generation number counter 6 (step S1). Subsequently, the overall control means 2 instructs the initial chromosome set creation means 3 to start operation.

Here, the configuration of the all generation chromosome set storage means 4 will be described. FIG. 7A shows a configuration example of the all-generation chromosome set storage means 4, and GN _max chromosome sets are stored in individual chromosome set storage units. The individual chromosome set storage section in all generation chromosome set storage means 4 corresponds to each generation number GN, and the chromosome set storage means of GN is represented by A (GN). As shown in FIG. 7B, the individual chromosome set storage unit is composed of a region for storing chromosomes, a region for storing fitness, and a region for storing the probability of selection. . The chromosome set storage means can store a maximum of K _β chromosome storage units. All individual chromosome set storage units are empty when the feature selection device 1 is activated.

The initial chromosome set creation means 3 sets A (1) as follows. The value of each bit of the feature selection bit string of each individual of K chromosomes is set to “0” with a constant probability P _f0 and “1” with 1-P _f0 . Subsequently, the value of each bit of the sample selection bit string of each individual is set to “0” with a constant probability P _p0 and “1” with 1-P _p0 , and all these individuals are elements of A (1). . The identification rate P _alpha training samples in each individual A (1) and fitness rearranges the individual A (1) to the descending order (Step S2). Then, the probability that the chromosomes of each rank are selected is calculated by the selection probability value calculation means 5 and written in the area for storing the selection probabilities of the all generation chromosome set storage means 4.

上式においてＭａｘは２変数の中の最大値を出力する関数である。 Here, the operation of the selection probability value calculation means 5 will be described. The selection probability value calculation means 5 calculates a probability value P _S (r) by which the r-th chromosome is selected by the equation (8).

In the above equation, Max is a function that outputs the maximum value of the two variables.

  Next, after the operation of the initial chromosome set creating means 3 is completed, the operation of the individual generation execution unit 7 is started. The overall control means 2 first adds 1 to the value of GN stored in the generation number counter 6 (step S3). Then, the overall control means 2 instructs the cross execution means 71 to execute. The crossover execution means 71 selects a pair of individual chromosome storage units at random from the probability value written in the selection probability area of each individual chromosome storage unit from A (GN-1), and duplicates two chromosomes therefrom. Crossover is performed using a pair of replicated chromosomes to generate two new chromosomes, which are sent to the chromosome evaluation means 8 to calculate each fitness ψ, together with the fitness value, the chromosome A (GN ) To add.

The process of _{K 1} single individuals by repeating _K 1/2 times is generated as an element of A (GN) (step S4). Crossover is a two-point crossover, and is performed independently for each of the sample selection bit string and the feature selection bit string of the chromosome. This is because, if crossover is simply applied to the entire chromosome, only one of both bit strings is crossed, and there is a possibility that simultaneous selection of both cannot be performed.

Next, the overall control means 2 instructs the mutation execution means 72 to execute. The mutation execution means 72 selects one chromosome storage part at random according to the probability value written in the selection probability area of the individual chromosome storage part from A (GN-1), and replicates it to the replicated chromosome. On the other hand, the chromosomal bit string is inverted with a certain probability to be an element of A (GN) (step S5). By repeating this process _{K 2} times, _{K 2} pieces of individual is added to A (GN). The probabilities of inverting the bits of the feature selection bit sequence and the sample selection bit sequence are P _fm and P _pm , respectively.

Next, the overall control unit 2 instructs the copy execution unit 73 to execute. The replication executing means 73 randomly selects K ₃ chromosome storage units from the probability value written in the selection probability area of each individual chromosome storage unit from A (GN-1), and assigns each chromosome and fitness to A (GN) is added (step S6). In the above, the sum of K ₁ , K ₂ , and K ₃ is a value equal to k _β . Finally, the overall control means 2 instructs the chromosome set rearranging means 74 to execute. The chromosome set rearranging means 74 rearranges each individual of A (GN) in descending order of fitness (step S7). The above is the processing of one generation of GA.

Then, it is determined whether or not GN = GN _max (step S8). If GN = GNmax, the operation is completed, and the chromosome in the first-order chromosome storage means of A (GNmax) is taken out and finally processed. Output as a result. If GN = GNmax is not satisfied, the process returns to step S3 and is repeated.

  As described above, in the feature selection using the GA, since the evaluation scale based on the soft margin of the linear SVM is used as the fitness of the chromosome, the feature selection having a high generalization ability is performed similarly to the linear SVM. be able to.

  1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to execute the feature selection process. You may go. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  It can be applied to applications where it is essential to improve the recognition accuracy of pattern recognition by selecting features using a genetic algorithm.

  DESCRIPTION OF SYMBOLS 1 ... Feature selection apparatus, 2 ... Overall control means, 3 ... Initial chromosome set creation means, 4 ... All generation chromosome set storage means, 5 ... Selection probability value calculation means, 6 ... Generation number counter, 7 ... individual generation execution unit, 71 ... cross execution means, 72 ... mutation execution means, 73 ... replication execution means, 74 ... chromosome set rearrangement means, 8 ... Chromosome evaluation means

Claims (3)

A feature selection device that improves the recognition accuracy of pattern recognition by selecting features of a sample using a genetic algorithm,
A distance calculating means for calculating the distances of all learning samples from the hyperplane that is a decision boundary between categories;
Margin setting means for selecting one of all learning samples as a reference sample, and setting two hyperplanes at each of the distance between the reference sample and the decision boundary on both sides of the decision boundary;
Penalty calculation means for calculating a penalty value corresponding to a length protruding from the two hyperplanes for the learning sample protruding from the two hyperplanes set,
A reference sample evaluation unit that calculates a robustness evaluation scale based on the interval between the two hyperplanes and the penalty value, and uses the value of the evaluation scale as the evaluation value of the reference sample;
Evaluation means for calculating an evaluation value when using them as a reference sample for every learning sample using the reference sample evaluation means, and outputting the minimum value among them as the evaluation value of the selected feature And a feature selection device. In order to improve the identification accuracy of pattern recognition by selecting features of a sample using a genetic algorithm, a distance calculation means, a margin setting means, a penalty calculation means, a reference sample evaluation means, an evaluation means, A feature selection method in a feature selection device comprising:
A distance calculating step in which the distance calculating means calculates the distances of all learning samples from the hyperplane which is a decision boundary between categories;
A margin setting step in which the margin setting means selects one of all the learning samples as a reference sample, and sets two hyperplanes at each of the distance between the reference sample and the determination boundary on both sides of the determination boundary. When,
A penalty calculating step in which the penalty calculating means calculates a penalty value corresponding to a length protruding from the two hyperplanes for the learning sample protruding from the set two hyperplanes;
The reference sample evaluation means calculates a robust evaluation measure based on the interval between the two hyperplanes and the penalty value, and uses the value of the evaluation measure as the evaluation value of the reference sample. The step and the evaluation means calculate an evaluation value when using them as a reference sample for every learning sample using the reference sample evaluation means, and the minimum value among them is selected. An evaluation step for outputting as an evaluation value. A feature selection program that causes a computer on a feature selection device to improve pattern recognition discrimination accuracy by selecting a feature of a sample using a genetic algorithm,
A distance calculating step for calculating distances of all learning samples from a hyperplane that is a decision boundary between categories;
A margin setting step of selecting one of all the learning samples as a reference sample, and setting two hyperplanes at each of the reference sample and decision boundary distance positions on both sides of the decision boundary;
A penalty calculating step for calculating a penalty value corresponding to a length protruding from the two hyperplanes with respect to the learning sample protruding from the set two hyperplanes;
Based on the interval between the two hyperplanes and the penalty value, a robustness evaluation measure is calculated, and a reference sample evaluation step using the value of the evaluation measure as an evaluation value of the reference sample; and the reference sample evaluation step Then, for each of the learning samples, an evaluation value when using them as a reference sample is calculated, and an evaluation step for outputting the minimum value among them as the evaluation value of the selected feature is performed on the computer. A feature selection program characterized by

Images