JP2010282367A - Learning device and learning method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To configure a discriminator unit to perform pattern identification of input data with high accuracy at high speed. <P>SOLUTION: A branch node determines the next node to be initiated based on a parameter. A discrimination node discriminates whether the input data belongs to a second class, based on the parameter. The problem is solved by the provision of: a multivariate analysis means for obtaining a direction vector by performing multivariate analysis on the feature vector of a learning data belonging to a first class; a division plane decision means for deciding a division plane perpendicular to the direction vector obtained in the multivariate analysis means, to divide the feature space of the learning data; and based on the division plane decided by the division plane decision means, a parameter decision means for deciding the parameter of the branch node. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a learning apparatus and a learning method.

Conventionally, pattern identification using linear identification has been actively performed. Non-Patent Document 1 describes some examples of linear discrimination.
Briefly, in linear identification, input data is represented as feature vectors as vectors in a multidimensional space, and the feature space spanned by these feature vectors is divided by a hyperplane. Then, the input data is identified depending on which side of the hyperplane the feature vector corresponding to the input data is located. Furthermore, if a plurality of hyperplanes are prepared, feature vectors in a region surrounded by these hyperplanes can be identified as one class. Non-Patent Document 2 discloses such an example.
The discriminator has a relatively high processing speed, but adopts a structure in which linear discrimination is integrated by logical product. Therefore, a set of feature vectors to be identified can be expressed only as one side of a hyperplane or a convex polyhedron in the feature space. In other words, it is not possible to represent a set with unevenness.

  Several methods have been proposed to overcome this problem. The most common method is a method of integrating the results of individual linear discrimination by operations other than logical product. One method is to use a piecewise discriminant function described in Non-Patent Document 1. This is the idea of covering the surface of the set with a plurality of polygons. There is also a method using decision trees. A typical decision tree is supervised learning. When constructing a decision tree, it is common to use the concept of impurity of branch destination nodes. This is a variation in the type of input data that reaches the branch destination node. Usually, when a decision tree is constructed, a branch condition is determined so that this impurity is reduced. Non-Patent Document 3 proposes a method for constructing a decision tree without a teacher. However, Non-Patent Document 3 also introduces the concept of impurity and determines the branching condition so that this is reduced.

Increasing the dimension of the feature space is another way to represent an uneven set. For example, Non-Patent Document 2 introduces the concept of a strong classifier by adding the results of a plurality of weak classifiers. However, even if the dimension of the feature space is increased, the essential problem that only the non-concave polyhedron can be expressed in the feature space having the increased dimension cannot be solved.
The method of building a decision tree without a teacher is similar to clustering. In clustering, data whose class is unknown is divided into a plurality of sets. Non-Patent Document 4 proposes one method of clustering in which input data is divided hierarchically. At that time, the completed cluster is divided into two one after another. The resulting two clusters are made as close to a Gaussian distribution as possible.

Ishii, Ueda, Maeda, Murase (1998) "Intuitive pattern recognition", Ohmsha. Viola & Jones (2001) "Rapid Object Detection using a Boosted Cascade of Simple Features", Proceedings of the IEEE Conference on Computer and Veteran. 1, p. 551. Basak & Krishnapuram (2005) "Interpretable Hierarchical Clustering by Constructing an Undisclosed Decision Tree." IEEE Transactions in Knowledge. 17 (1), p. 121. Miasnikov, Rome & Harrick (2004) "A Hierarchical Projection Pursuing Clustering Algorithm", Pattern Recognition, Proceedings of the International Conference. 1, p. 268.

  As described above, the conventional technique has a problem that it is not possible to configure a discriminator that identifies input data at high speed and with high accuracy.

  The present invention has been made in view of such problems, and an object of the present invention is to constitute a discriminator for identifying patterns of input data at high speed and with high accuracy.

Therefore, the present invention determines a parameter of each node of a classifier having a tree structure in which a plurality of identification nodes and branch nodes are connected and identifying input data into two classes S and ˜S of the feature space. The branching node is a node that determines a node to be activated next based on the parameter, and the identification node determines whether the input data belongs to ~ S based on the parameter. A multivariate analysis for obtaining a direction vector by performing multivariate analysis on a feature vector of learning data belonging to S based on learning data belonging to any of S and ˜S. And a dividing plane determining unit that determines a dividing plane that is perpendicular to the direction vector obtained by the multivariate analyzing unit and that divides the feature space of the learning data, and the dividing Based on the dividing plane which is determined by the determining means, and having a a parameter determining means for determining a parameter of the branch node.
With such a configuration, for example, when a decision tree for pattern identification is constructed, it is possible to appropriately determine the parameters of the branch node, so that the identifier for identifying the pattern of input data at high speed and with high accuracy is provided. Can be configured.
Further, the present invention may be a learning method.

  According to the present invention, it is possible to configure a discriminator for identifying patterns of input data at high speed and with high accuracy.

It is a block diagram which shows an example of the hardware constitutions of the information processing apparatus which concerns on embodiment. It is a flowchart showing the flow of the process at the time of detecting a face. It is a data flowchart of FIG. It is a figure which shows the structure of the data showing the parameter 211 for pattern identification. It is a figure showing the data structure of the node of type T1. It is a figure showing the data structure of the node of type T2. It is a flowchart showing the detail of step S203 of FIG. It is a figure showing the image which drew a mode that the input image was branched. It is a flowchart which shows an example of the rough process of the learning for node N3. It is a flowchart showing the detail of step F01 of FIG. It is a flowchart (the 1) showing the detail of step F03 of FIG. It is a flowchart (the 2) showing the detail of step F03 of FIG. It is a flowchart (the 1) showing the detail of step F0311 of FIG. It is a flowchart which shows an example of the process which calculates | requires the projection vector q 'which minimizes an objective function. It is a flowchart (the 2) showing the detail of step F0311 of FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Embodiment 1>
The example of the information processing apparatus which determines whether a face exists in the input image is shown. In order to simplify the embodiment, it is assumed that the input image is a grayscale image, and if there is a face, it is arranged at a substantially determined size at the center, like a passport photo. Note that a face of an arbitrary size at an arbitrary position can be detected by scanning the image or enlarging / reducing the image. It is assumed that the luminance value is also normalized. As a normalization method, there is a method of taking a difference from the average luminance or dividing by a standard deviation of luminance.
FIG. 1 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus according to the embodiment. In FIG. 1, a CPU (Central Processing Unit) 100 executes a pattern identification parameter learning method described in the embodiment according to a program. The program memory 101 stores a program executed by the CPU 100. The RAM 102 provides a memory for temporarily storing various types of information when the CPU 100 executes a program. The hard disk 103 is a storage medium for storing image files, pattern identification parameters, and the like. The display 104 is a device that presents the processing result of the present embodiment to the user. A bus 110 is a control bus / data bus that connects these units to the CPU 100.

FIG. 2 is a flowchart showing the flow of processing when a face is detected.
First, in step S <b> 201, the CPU 100 reads an image from the hard disk 103 into the RAM 102. The image is held on the RAM 102 as a two-dimensional array. In the next step S <b> 202, the CPU 100 reads pattern identification parameters created by a learning method described later from the hard disk 103 into the RAM 102. In step S203, the CPU 100 determines whether there is a face in the image read in step S201, using the pattern identification parameter read in step S202. In step S204, the CPU 100 displays the result on the display 104.

  FIG. 2 is represented as a data flowchart as shown in FIG. FIG. 3 is a data flowchart of FIG. Reference numeral 205 denotes an image stored in the hard disk 103. In the image reading process 201, the image 205 in the hard disk is stored as an input image I on the RAM 102. Reference numeral 209 denotes a pattern identification parameter stored in the hard disk 103. In the pattern identification parameter reading process 210, the pattern identification parameter 209 in the hard disk 103 is stored on the RAM 102 as the pattern identification parameter 211. In the detection process 203, the CPU 100 determines whether or not there is a face in the input image I using the previous input image I and the pattern identification parameter 211, and detects whether or not there is a face 207. As a result, the data is written in the RAM 102. In the detection result display process 204, the CPU 100 displays the contents of the detection result 207 on the display 104.

  Here, the contents of the pattern identification parameter 211 will be briefly described with reference to FIGS. 4, 5, and 6. A method for creating the pattern identification parameter 211 will be described later. FIG. 4 is a diagram illustrating the structure of data representing the pattern identification parameter 211. In FIG. 4, the square represents each node of the tree structure. Moreover, the arrow represents the order in which the processing of each node is executed. The pattern identification parameter 211 has a structure in which two types of nodes represented by type T1 and type T2 are connected in a tree shape. A node of type T1 is an identification node, after which only one node is connected. The type T2 node is a branch node, and a plurality of nodes are connected after the node. The node labeled N3 is also a type T2 node. The present embodiment can be applied to various types of detectors (identifiers) regardless of the type of T1, but here, a weak classifier as described in Non-Patent Document 2 is used as a type T1. An example of using this node is shown below. In addition to this, a detector using linear discriminant analysis (LDA), support vector machine (SVM), or the like can be used. Moreover, the detector which connected these may be sufficient.

In the following description, although using a term parameter branch destination A · branch target B-set F ^+-set G ^+, These are the contents by the node of interest are different. The value obtained using these also differs depending on the node. In this embodiment, subscripts indicating nodes are omitted in order to avoid complexity.

  The weak classifier described in Non-Patent Document 2 corresponds to a node of type T1 in FIG. FIG. 5 is a diagram illustrating a data structure of a node of type T1. Multiple pieces of this data are stored on the memory of the RAM 102. Individual data usually have different values. First, the node type is stored at the top. Since this node is of type T1, a code representing T1 is stored as the node type. Next, rectangle information is stored. The number n of rectangles is stored at the beginning of the rectangle information, and then the coordinates of the rectangles corresponding to the number n (the upper left point and the lower right point) are stored. These multiple rectangles are collectively referred to as a rectangle group. Next, parameters for censoring are stored. Here, “censoring” is described later with reference to FIG. 7, but simply speaking, it is determined that there is no face in the input image at an early stage. A threshold value θ is stored at the head of the censoring parameter. After that, codes used for calculation for truncation by the number n of the previous rectangles are arranged. The reference sign here means +1 or -1. Finally, a pointer to the next node is stored.

  FIG. 6 is a diagram illustrating a data structure of a node of type T2. A plurality of this data is also stored on the memory of the RAM 102. Individual data usually have different values. First, the node type is stored at the top. Since this node is of type T2, a code representing T2 is stored as the node type. Next, rectangle information is stored. The number n of rectangles is stored at the beginning of the rectangle information, and then the coordinates of the rectangles corresponding to the number n (the upper left point and the lower right point) are stored. Next, parameters for branch destination A are arranged. In the parameter for the branch destination A, a threshold and a rectangular coefficient are stored as in the case of the abort parameter, but a pointer to the branch destination node A is also stored. The parameter of another node is stored at the destination indicated by this pointer. Finally, a pointer to another branch destination node B is stored.

  Before explaining how to create the parameters, a method for detecting a face using these parameters will be explained. The overall flow of the detection process is performed by the CPU 100 sequentially following each node in FIG. 4 from the root node (the node drawn at the top in the drawing). If the node to be processed is a node of type T1, the CPU 100 determines whether or not a face is included in the input image I using parameters specific to the node illustrated in FIG. When it is determined that there is a high possibility that there is no face, the CPU 100 interrupts the processing there. Otherwise, the CPU 100 moves to the processing of the next node. When the node to be processed is a node of type T2, the CPU 100 determines which node to transfer the process to next using the parameters specific to the node illustrated in FIG. By following the nodes sequentially in this way, the CPU 100 determines whether the type T1 node is aborted or continued, and the type T2 node selects a branch destination node. Here, a node of type T1, that is, an identification node, is a classifier that identifies input data into two classes of S (first class) and ~ S (second class) of the feature space, based on parameters. , A node that identifies whether the input data belongs to ~ S. A type T2 node, that is, a branch node, is a node that determines a node to be activated next based on parameters.

FIG. 7 is a flowchart showing details of step S203 in FIG. In the first step D10, the CPU 100 initializes the pointer variable p to point to the first node. In the next step D02, the CPU 100 confirms the type of node indicated by p. When the node indicated by p is of type T1, the CPU 100 proceeds to step D11. Conversely, in the case of type T2, the CPU 100 proceeds to step D21.
In step D11, the CPU 100 initializes the variable c with 0. Then, the CPU 100 repeats the loop from step D12 to D15 for the number n of the rectangles. In the loop, the CPU 100 sets i as a loop variable representing a rectangle. In step D13, the CPU 100 obtains the coordinates (x _iL , y _iT ) − (x _iR , y _iB ) of the diagonal line of the rectangle i from the node information in FIG. 5 and sums the luminance values in the rectangle in the input image I. Ask for. CPU100, this is referred to as b _i. b _i can be determined at high speed by using the accumulated information (integral image) as written in Non-Patent Document 2. In step D14, the CPU 100 adds the product of b _i and the code a _i of the rectangle i to the variable c. In summary, the CPU 100 calculates the following sum in this loop.

In step D16, the CPU 100 determines whether or not the sum c exceeds the threshold value θ in FIG. If it exceeds θ, the CPU 100 proceeds to step D17 and writes a “false” value in the detection result 207. This represents that no face was detected. Here, the processing of the tree shown in FIG. 4 is aborted. If the CPU 100 determines in step D16 that the sum c does not exceed the threshold θ, the process proceeds to the next step D18. Here, the CPU 100 confirms whether or not all nodes have been processed. When the processing of all the nodes has been completed, the CPU 100 writes a “true” value in the detection result 207 in step D19. As a result, a face is detected. On the other hand, if all the nodes have not been processed in step D18, the CPU 100 stores a pointer to the next node in the pointer variable p in step D05. Then, the CPU 100 returns the control to step D02.
If the type of the node pointed to by the pointer variable p is T2 in step D02, the CPU 100 executes the processing from step D21. First, in step D21, the CPU 100 initializes the variable c with 0. Then, in the loop from step D22 to D25, the CPU 100 obtains the next inner product value. Note that a _Ai is a rectangular coefficient in FIG.

In step D26, the CPU 100 confirms whether the inner product value c exceeds the threshold value θ _A in FIG. When exceeding, the CPU 100 proceeds to next Step D28. In step D28, the CPU 100 assigns the pointer value to the branch destination node A in FIG. And CPU100 starts the process from step D02 again. If the threshold is not exceeded in step D26, the CPU 100 proceeds to step D30. Here, the CPU 100 substitutes the pointer value to the branch destination node B in FIG. And CPU100 starts the process from step D02 again. FIG. 8 is an image of this situation. FIG. 8 is a diagram illustrating an image depicting a state where an input image is branched. In FIG. 8, these are feature vectors bi of the input image I that are drawn with circles and triangles. When the input image I is a face, it is drawn as a circle (E00 or E01), and when it is not a face, it is drawn as a triangle (E10 or E11). E02 is a hyperplane where c = θ _A. E03 is a vector a _A = (a _A1 , a _A2 ,..., A _An ) and is a normal vector of the hyperplane E02. Due to the above branching condition, the face image displayed as the black circle E01 and the non-face image displayed as the black triangle E11 are distributed to the branch destination A. Further, the face image displayed as the white circle E00 and the non-face image displayed as the white triangle E10 are distributed to the branch destination B. With the above processing, the nodes of the tree in FIG. 4 are transitioned. As shown in FIG. 4, nodes of type T2 can be continuous. By doing so, more complex branches are possible. Alternatively, by preparing a plurality of threshold values, one of three or more branch destinations can be selected.

The learning procedure for obtaining the parameter of the node of type T1 shown in FIG. 5 is as shown in Non-Patent Document 2. Here, it is easy to understand if the candidates for each rectangle in FIG. 5 are presented in advance before learning. These rectangular set R = | a _{{r i i = 1 ··· N} r}. Of course, the set R may be generated regularly or by random numbers.
FIG. 7 shows a learning procedure in the present embodiment for obtaining a parameter of a node of type T2 shown in FIG. First, as a premise, there is a set F = {f _j | j = 1... N _f } of learning face images f _j , and a set G = {g _j | g _{j of} learning images g _j without a face. = shall 1 ··· N _g} are prepared. Furthermore, it is assumed that the tree structure of FIG. 4 is determined in advance, and a memory for securing parameters is secured on the RAM 102. For example, as an example only, the branch nodes can be arranged so that one branch occurs every two times until the number of branches reaches three as shown in FIG. At this time, the pointer values in FIGS. 5 and 6 are also determined and can be stored. Therefore, it is assumed that learning has been completed from the node indicated as T1 in FIG. 4 to the node immediately before the node indicated as N3 (that is, the node indicated as T2 here). When the detection process described above is applied, some of the learning images are rejected (canceled) as having no face at the nodes up to N3, or distributed to other branch destinations by the node of type T2. Therefore, the CPU 100, at the node N3, sets F ⁺ = {f _j ⁺ | j = 1... N _f ⁺ } of face images f _j ⁺ that have not been rejected or distributed to other branch destinations so far. And a set G ⁺ = {g _j ⁺ | j = 1... N _g ⁺ } of non-face images g _j ⁺ are used for learning.

A general flow of learning for the node N3 is shown in FIG. First, in step F01, CPU 100 represents a learning image F ⁺ as a set of feature vectors. Next, in step F03, the CPU 100 determines a hyperplane in the feature space that divides the feature space of the learning data using the set of feature vectors (determination plane determination), and writes it as a parameter of the node N3 (parameter determination). .
Next, details of each of these steps will be described.
FIG. 10 is a flowchart showing details of step F01 in FIG. A loop from steps F0101 to F0107 is processing relating to each face image f _j ⁺ belonging to the learning image F ⁺ . The loop from step F0103 to step F0105 is repeated for each rectangle r _i belonging to the rectangle candidate set R. In step F0104, the CPU 100 assigns the sum of the luminance values of the pixels in the rectangle r _i on the face image f _j ⁺ to the element b _ji ^f of the two-dimensional array. With the above processing, _Nr- dimensional feature vectors are associated with each image in the learning image set F ⁺ . Each dimension of the feature space corresponds to the sum of luminance values in a certain rectangle. A set of feature vectors corresponding to the set of learning images F ⁺ is _assumed to be B ^{F +} = {b _j ^f | b _j ^f = (b _j1 ^f , b _j2 ^f ,..., B _jNr ^f )}.

In step F03 in FIG. 9, the CPU 100 determines a divided hyperplane that is perpendicular to the vector obtained in step F01 and divides the feature space of the learning data (division plane determination). A divided hyperplane is an example of a divided surface. The flow of step F03 is shown in the flowchart of FIG. FIG. 11 is a flowchart (part 1) showing details of step F03 in FIG.
First, in step F0301, the CPU 100 obtains a first principal component direction vector of the face feature vector set B ^{F +} . Here, the first principal component direction is a direction in which the dispersion of the set B ^{F +} is maximized. The CPU 100 can obtain a principal component direction vector using a multivariate analysis technique such as single-value decomposition (SVD) or principal component analysis (PCA). Alternatively, the CPU 100 can also obtain the principal component direction vector using a multivariate analysis method such as independent component analysis (ICA). The principal component direction vector obtained here is d = (d ₁ , d ₂ ,..., D _Nr ). Next, in step F0302, the CPU 100 reduces the dimension of the principal component direction vector d. To be more specific, CPU 100 is a predetermined value n, taken out the top n components having a large absolute value among the components of _{d, a A = (a A1} , a A2, ··· , A _An ). Since n takes a long time to calculate if the value is large, it is necessary not to make it too large. Each dimension of d corresponds to one rectangle in R. Focusing on this fact, CPU 100 may be arranged a rectangle corresponding to each element of a _A. Let this be r _A = (r _A1 , r _A2 ,..., R _An ). Each component of a _A is written by the CPU 100 in the corresponding area of the branch destination A parameter in FIG. 6, and the coordinates of the rectangles n and r _A are written as rectangle information in FIG. The dimension reduction method is not limited to the above method. For example, the CPU 100 can extract the top n components having a large product of the absolute value in the vector d and the corresponding rectangular area. Further, the CPU 100 may not perform dimension reduction.
The principal component direction vector is an example of a direction vector.

The remaining threshold value θ _A is obtained in step F0303 in FIG. An example of a method for obtaining the threshold value θ _A is expressed by the following equation using the centroid c of B ^{F +} described above.

Here, b _jA ^f is a vector obtained by extracting components corresponding to r _A = (r _A1 , r _A2 ,..., R _An ) from b _j ^f . Although w _j = 1 may be used, it may be an Adaboost weight as described in Non-Patent Document 2.

  The parameters of the tree structure shown in FIG. 4 can be prepared by learning the parameters of the nodes of type T1 and type T2 by the above method. By using this parameter, a face in the input image can be detected by processing with a relatively light calculation load. In the present embodiment, PCA or the like is used to obtain the principal component direction. However, as a matter of course, a nonlinear method such as kernel PCA can also be used. From the above description, it is clear that the discriminator is not limited to face identification, but can also handle other images such as a person, a figure, and a character.

<Embodiment 2>
In the first embodiment, the dimension reduction is performed after the principal component analysis or the like is performed. In the second embodiment, an example is shown in which independent component analysis (ICA) is performed after the dimension reduction is performed.
In this embodiment, FIG. 12 is used instead of FIG. FIG. 12 is a flowchart (part 2) showing details of step F03 in FIG.
In step F0311, the CPU 100 obtains a normal vector of the divided hyperplane. In step F0313, the CPU 100 obtains the threshold θ _A by the same procedure (same process) as step F0303 in FIG.

FIG. 13 is a flowchart (part 1) showing details of step F0311 in FIG.
In step G01, the CPU 100 selects several rectangle combinations from the rectangle set R, and sets the combination set as RC. The number m of rectangles in each combination may be constant, for example, 2 but may be uneven. If they are not uniform, the CPU 100 may select m to increase monotonously according to the number of nodes counted from the root node. The CPU 100 performs steps G02 to G06 for each combination r _C = (r _C1 , r _C2 ,..., R _Cm ) (C _{1 to} C _m are indexes representing rectangle numbers) of the rectangular combination set RC. Repeat the loop.

を生成する。もし、ｒ_C＝（ｒ₅，ｒ₂₃₆，ｒ₅₄₆₈
）の場合、

は、（ｂ_j,5 ^f，ｂ_j,236 ^f，ｂ_j,5468 ^f）となる。つまり、学習画像ｆ
_j ⁺上の矩形ｒ₅とｒ₂₃₆とｒ₅₄₆₈内の輝度値総和を並べたベクトルとなる。 Next, in step G03, the CPU 100 extracts a component corresponding to each element (rectangle) of r _C = (r _C1 , r _C2 ,..., R _Cm ) for each feature vector b _j ^f of the set B ^{F +.} Feature vector

Is generated. If r _C = (r ₅ , r ₂₃₆ , r ₅₄₆₈
)in the case of,

Becomes (b _{j, 5} ^f , b _{j, 236} ^f , b _{j, 5468} ^f ). That is, the learning image f
_This is a vector in which the luminance value sums in the rectangles r ₅ , r _236, and r ₅₄₆₈ on _j ⁺ are arranged.

に対してＩＣＡを適用し、最大でｍ本のｍ次元ベクトルｑ_k（ｋ＝１，・・・，ｍ_q；ｍ_q≦ｍ）を得る。ＩＣＡを適用する際の目的関数には例えば次のような関数Ｊ（ｑ）を選ぶことができる。ここで、ｖは平均０、分散１の正規分布に従う確率変数である。 In step G04, the CPU 100 determines a set of these m-dimensional vectors.

ICA is applied to a maximum of m m-dimensional vectors q _k (k = 1,..., M _q ; m _q ≦ m). For example, the following function J (q) can be selected as an objective function when applying ICA. Here, v is a random variable that follows a normal distribution with an average of 0 and a variance of 1.

の尖度（ｋｕｒｔｏｓｉｓ）の符号を反転したものを計算する。より具体的に説明すると、ＣＰＵ１００は、次の値を求める（射影評価）。 Next, in step G05, the CPU 100 sets the evaluation value as a set.

Compute the inverse of the sign of kurtosis. More specifically, the CPU 100 obtains the following value (projection evaluation).

Alternatively, the CPU 100 may use a contrast function or an objective function as the evaluation value. Once out of the loop, CPU 100 is a vector q _k evaluation value was greatest in step G07 and the rectangular combined r at that time, respectively selected as a _A and r _A (optimization). The vector q _k is an example of a statistic regarding the set of inner product values of the projection vector and the feature vector of the learning data. That is, the CPU 100 obtains a projection vector q that maximizes or minimizes the vector q _k and sets it as a direction vector.

The parameters of the tree structure shown in FIG. 4 can be prepared by learning the parameters of nodes of type T1 and type T2 by the above method. By using this parameter, a face in the input image can be detected by processing with a relatively light calculation load. In particular, in the present embodiment, not only the normal vector of the divided hyperplane is obtained by calculating and comparing evaluation values using an evaluation function common to all combinations r _C , but also the components used in dimension reduction are selected. ing. In the present embodiment, ICA is used to obtain a hyperplane normal vector, but other methods such as PCA and SVD can also be used.

<Embodiment 3>
In the second embodiment, a method has been described in which a projection vector q whose kurtosis is more deviated from the normal distribution is obtained using ICA. This method requires not only a projection vector with a low kurtosis but also a projection vector with a high kurtosis. In the present embodiment, a method for obtaining only a projection vector having a small direct kurtosis using a projection tracking method will be described. Also in this embodiment, the kurtosis is used as an example of an index that represents the degree of deviation from the normal distribution. Although the configuration is almost the same as that of the second embodiment, FIG. 15 is used in this embodiment instead of FIG.

First, the projection vector q ′ is expressed in a polar coordinate system.

を次式に従って平行移動させる。

Also,

Is translated according to the following equation.

The objective function is as follows.

FIG. 14 shows a flowchart of a method for obtaining the projection vector q ′ for minimizing the objective function. FIG. 14 is a flowchart illustrating an example of processing for obtaining a projection vector q ′ that minimizes the objective function.
In step K01, the CPU 100 initializes θ _i (i = 1,..., M) with a predetermined value. For example, the CPU 100 can set θ _i = 0 (i = 1,..., M). Alternatively, the CPU 100 can generate the value with a random number. Further, the CPU 100 initializes a counter variable s for convergence condition to 0.

The CPU 100 enters repetitive processing from step K02. First, in step K02, the CPU 100 generates θ ⁺ . More specifically, the CPU 100 first generates natural numbers u (1 ≦ u ≦ m−1) and Δ using random numbers. For example, Δ has a normal distribution with an average of 0. Θ ⁺ is obtained by adding Δ to the u-th component of θ. That is, the following formula is assumed. _{θ i (i = 1, ...} , m) is the i-th component of theta.
θ _i ⁺ = θ _i (i ≠ u)
θ _u ⁺ = θ _u + Δ

Next, in step K03, the CPU 100 checks the increase / decrease of the objective function. More specifically, the CPU 100 substitutes θ into (Equation 1) to obtain a projection vector q ′, and calculates J (q ′). Next, the CPU 100 calculates a projection vector q ⁺ 'using θ ⁺ instead of θ, and calculates J (q ⁺ '). The CPU 100 proceeds to step K06 if J (q ′) ≦ J (q ⁺ ′), and proceeds to step K04 if vice versa.
In step K04, the CPU 100 initializes the counter variable s to 0. In step K05, the CPU 100 substitutes θ ⁺ for θ and repeats the loop from step K02 again. In step K06, the CPU 100 increments the counter variable s by one. In step K07, the CPU 100 compares a predetermined constant S with s, and if s <S, repeats the loop from step K02. Conversely, if s ≦ S, the CPU 100 stops the minimization process. As a result, if the value of the objective function is not improved S times, the loop is exited. The projection vector q ′ obtained from θ at this time is treated as a value that minimizes the objective function.

  In step K03, since the CPU 100 sequentially evaluates J (q ′), it is not necessary to recalculate the evaluation value (the value of J (q ′)) as in step G05 of FIG. Therefore, instead of FIG. 13, the present embodiment follows FIG. Details of step L04 are as shown in FIG.

The method for obtaining the projection vector that minimizes the kurtosis has been described above. In this embodiment, the division hyperplane of the type T2 node is determined using this projection vector.
In addition to the optimization method described above, the projection vector may be obtained by another optimization method such as Newton's method.
Moreover, although the example using the rectangular feature was shown in the embodiments so far, the present invention is not limited to this. Examples of the feature amount include a pixel value of the input image, a feature amount obtained by applying a Gabor filter to the input image, and a feature called a local feature that assigns a vector to each pixel of the input image. As described above, many feature quantities are defined in the field of pattern recognition. Further, although face identification has been taken up as an example, the present invention can be applied not only to images but also to other information such as voice information and questionnaire results.

  As described above, according to each of the above-described embodiments, when building a decision tree for pattern identification, it is possible to appropriately determine the parameters of the branch node. A discriminator can be constructed.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

100 CPU, 101 program memory, 102 RAM, 103 hard disk, 104 display

Claims (12)

Determine a parameter of each node of a classifier having a tree structure in which a plurality of identification nodes and branch nodes are connected and identifying input data into two classes of a first class and a second class of feature space A learning device,
The branch node is a node that determines a node to be started next based on a parameter;
The identification node is a node that identifies whether input data belongs to the second class based on a parameter;
Multivariate analysis means for performing a multivariate analysis on a feature vector of learning data belonging to the first class to obtain a direction vector;
A dividing plane determining unit that determines a dividing plane that is perpendicular to the direction vector obtained by the multivariate analyzing unit and divides the feature space of the learning data;
Parameter determining means for determining a parameter of the branch node based on the split plane determined by the split plane determining means;
A learning apparatus comprising:   The dividing plane determining means determines a hyperplane in the feature space that is perpendicular to the direction vector obtained by the multivariate analyzing means and divides the feature space of the learning data as a dividing plane. The learning apparatus according to claim 1, wherein The multivariate analysis means includes:
Projection means for obtaining an inner product value of a projection vector and a feature vector of learning data belonging to the first class;
A projection evaluation means for obtaining a statistic relating to the set of inner product values;
An optimization means for determining the projection vector for maximizing or minimizing the statistic and setting it as the direction vector;
The learning apparatus according to claim 1, comprising: The projection evaluation means obtains a statistic representing a degree of deviation from a normal distribution of the set of inner product values,
The learning device according to claim 3, wherein the optimization unit obtains the projection vector that maximizes the statistic and uses the projection vector as the direction vector.   The learning apparatus according to claim 3, wherein the projection evaluation unit obtains a kurtosis of the set of inner product values as a statistic. Dimensional reduction means for reducing elements based on the magnitude of the absolute value from the direction vector determined by the multivariate analysis means,
The learning apparatus according to claim 1, wherein the division plane determination unit determines the division plane that is perpendicular to the vector whose dimension is reduced by the dimension reduction unit. Determine a parameter of each node of a classifier having a tree structure in which a plurality of identification nodes and branch nodes are connected, and identifying input data into two classes, a first class and a second class in the feature space. A learning method in a learning device,
The branch node is a node that determines a node to be started next based on a parameter;
The identification node is a node that identifies whether input data belongs to the second class based on a parameter;
The learning device is
A multivariate analysis step of performing a multivariate analysis on a feature vector of learning data belonging to the first class to obtain a direction vector;
A dividing plane determining step that determines a dividing plane that is perpendicular to the direction vector determined in the multivariate analysis step and divides the feature space of the learning data;
A parameter determining step for determining a parameter of the branch node based on the split plane determined in the split plane determining step;
The learning method characterized by including.   In the dividing plane determining step, a hyperplane in the feature space that is perpendicular to the direction vector obtained in the multivariate analyzing step and divides the feature space of the learning data is determined as a dividing plane. The learning method according to claim 7, wherein the learning method is characterized. In the multivariate analysis step,
A projection step for obtaining an inner product value of a certain projection vector and a feature vector of learning data belonging to the first class;
A projection evaluation step for obtaining a statistic relating to the set of inner product values;
Determining the projection vector that maximizes or minimizes the statistic, and optimizing it as the direction vector;
The learning method according to claim 7, further comprising: In the projection evaluation step, a statistic representing a degree of deviation from the normal distribution of the set of inner product values is obtained,
The learning method according to claim 9, wherein in the optimization step, the projection vector that maximizes the statistic is obtained and used as the direction vector.   The learning method according to claim 9, wherein, in the projection evaluation step, a kurtosis of the set of inner product values is obtained as a statistic. A dimension reduction step of reducing elements based on the magnitude of the absolute value from the direction vector obtained in the multivariate analysis step;
12. The learning method according to claim 7, wherein, in the division plane determination step, the division plane that is perpendicular to the vector whose dimension is reduced in the dimension reduction step is determined.

Images