JP6147003B2 - Information processing apparatus, information processing method, and program - Google Patents

Description

  In particular, the present invention relates to an information processing apparatus, an information processing method, and a program suitable for use in improving object recognition performance.

  When performing personal recognition using face image data, it is an important task to determine the positions of facial organs or multiple characteristic parts (hereinafter referred to as feature points) on the image, It greatly affects the recognition performance.

  As a method for determining the positions of a plurality of feature points, a method for determining position candidates for each feature point and a feature point position candidate based on the arrangement relationship of feature points specific to an object such as a face to be processed are used. In many cases, it is constituted by means for correcting. For example, Non-Patent Documents 1 and 2 disclose a method for determining the positions of feature points of a plurality of facial organs according to a geometric constraint technique based on statistics. Here, “restraint” refers to restricting the position of the feature point based on the arrangement relationship peculiar to the target object.

  In the method disclosed in Non-Patent Document 1, geometrical correction of feature point arrangement is performed using a relational model of feature point position coordinates called PDM (Point Distribution Model). In PDM, a vector obtained by concatenating the position coordinates of a plurality of feature points is projected onto an eigenspace generated by principal component analysis, and the projected vector is back-projected onto the space of the position coordinates of the feature points. Get position coordinates. At that time, by restricting the amplitude of the projection vector, the effect of restricting the position of the feature point based on the statistical arrangement relationship can be obtained.

  On the other hand, the method disclosed in Non-Patent Document 2 is also a method for correcting the position coordinates of feature points using principal component analysis. However, in this method, the restriction effect of the position of the feature point is obtained by limiting the number of dimensions of the projection vector. These methods give geometric constraints to the arrangement of the feature point positions by limiting the variation of the vector projected on the eigenspace by the range or the number of dimensions.

T. T. F. Cootes, C.I. J. et al. Taylor, Active Shape Models-'Smart Snakes'. in Proc. British Machine Vision Conference. Springer-Verlag, 1992, pp. 266-275 Beumer, G.G. M.M. Tao, Q .; Bazen, A .; M.M. Veldhuis, R .; N. J. et al. “A Landmark Paper In Face Recognition”, “Automatic Face and Gesture Recognition, 2006. FGR 2006. 7th International Conference, pp. 73-78 Y. LeCun, L.M. Bottow, G.M. Orr and K.K. Muller: “Efficient BackProp”, in Orr, G.M. and Muller K.M. (Eds), Neural Networks: Tricks of the trade, Springer, 1998.

  However, each of the above methods is an excellent method for efficiently correcting the position of feature points (correcting outliers) based on statistical processing (using principal component analysis). Power) could not be controlled for each feature point. For example, when the accuracy of determining feature point position candidates is different for each feature point, it is desirable to change the degree of correction for each feature point. However, the degree of correction cannot be changed by the method according to the conventional example.

  An object of the present invention is to make it possible to change the degree of correction of feature point position candidates for each feature point.

The information processing apparatus according to the present invention includes a position candidate determining unit that obtains position candidates of a plurality of feature points from an object region, and a feature point vector obtained by connecting the coordinates of the plurality of position candidates obtained by the position candidate determining unit. Generating means, projecting means for projecting the feature point vector generated by the generating means onto a space of a predetermined dimension to obtain a projection vector, and using the plurality of feature point position candidates for back projection An element number determining unit that determines the number of elements of a projection vector, and each of the plurality of feature point position candidates is converted into the position candidates by the element number determining unit among the elements of the projection vector obtained by the projecting unit. reverse projection means for inverse projection with projection vector composed of elements of the determined number of elements for, using the results of said inverse projection unit to correct the position candidate of the plurality of feature points And having a position determining means for determining the positions of a plurality of feature points, the.

  According to the present invention, the degree of correction of feature point position candidates can be controlled for each feature point, and the position of the feature point can be determined with higher accuracy.

5 is a flowchart illustrating an example of a processing procedure for determining a position of a feature point in the first embodiment of the present invention. It is a block diagram which shows the structural example of the information processing apparatus for determining the position of a feature point in embodiment. It is a figure explaining an example of the cutting-out process of a face image. It is a figure explaining the feature point position candidate relevant to a facial organ. It is a figure explaining the outline | summary of the process of step S101 of FIG. It is a figure explaining the example of a geometric correction process. It is a figure which illustrates typically the projection process and reverse projection process by the 1st Embodiment of this invention. It is a figure which shows the example of the table information of a reverse projection dimension number. It is a flowchart which shows an example of the process sequence which determines the position of the feature point in the 2nd Embodiment of this invention. It is a figure which illustrates typically the projection process and reverse projection process by the 2nd Embodiment of this invention. It is a figure explaining the example of the projection matrix for reverse projection.

(First embodiment)
The operation of the first embodiment of the present invention will be described below.
FIG. 2 is a block diagram illustrating a configuration example of the information processing apparatus 200 for determining the position of the feature point in the present embodiment. The information processing apparatus 200 according to the present embodiment first extracts a region including a face from image data. Hereinafter, the extracted area is referred to as face image data. Then, a plurality of feature point positions (here, feature positions related to the facial organs) are determined from the obtained face image data.

  In FIG. 2, an image input unit 201 includes an optical system device, a photoelectric conversion device, a driver circuit that controls a sensor, an AD converter, a signal processing circuit that controls various image corrections, a frame buffer, and the like. The pre-processing unit 202 executes various pre-processes in order to effectively perform various processes in the subsequent stage. Specifically, correction such as color conversion processing and contrast correction processing is processed by hardware on the image data acquired by the image input unit 201.

  The face image data cutout processing unit 203 performs face detection processing on the image data corrected by the preprocessing unit 202. Various conventionally proposed methods can be applied to the face detection method. Further, the face image data cutout processing unit 203 normalizes and cuts out a face image to a predetermined size for each detected face.

  FIG. 3 is a diagram illustrating an example of a face image cut-out process. A face region 302 is detected from the image 301 corrected by the preprocessing unit 202, and an erect face image 303 normalized to a predetermined size is cut out. Therefore, the size of the face image 303 is constant regardless of the face. Data of the cut face image is stored in a RAM (Random Access Memory) 209 via a DMAC (Direct Memory Access Controller) 205. Hereinafter, the position of the feature point is defined as the coordinate of the feature point in the face image 303. The coordinates are expressed in a coordinate system (x coordinate, y coordinate) with the upper left corner of the face image 303 as the origin.

  A CPU (Central Processing Unit) 207 executes main processing according to the present embodiment and controls the operation of the entire apparatus. The bridge 204 provides a bus bridge function between the image bus 210 and the CPU bus 206. A ROM (Read Only Memory) 208 stores a program that defines the operation of the CPU 207. A RAM 209 is a work memory necessary for the operation of the CPU 207. The CPU 207 executes processing on the face image data stored in the RAM 209.

FIG. 1 is a flowchart illustrating an example of a processing procedure for determining the position of a feature point in the present embodiment. The flowchart shows the operation of the CPU 207 and describes the process of determining the position of the feature point with respect to the normalized face image 303.
First, in step S101, the CPU 207 functions as a position candidate determination unit, and determines a position candidate of a predetermined feature point. In this process, for example, position candidates of feature points 401 to 415 related to 15 facial organs as shown in FIG. 4 are determined.

  FIG. 5 is a diagram for explaining the outline of the processing in step S101 in FIG. FIG. 5 shows a processing configuration in the case where position candidates for two feature points are determined for the sake of explanation. In the present embodiment, candidate positions of feature points are determined by CNN (Convolutional Neural Networks). The CNN is configured by hierarchical feature extraction processing, and FIG. 5 shows an example of a two-layer CNN in which the number of features in the first layer 506 is 3 and the number of features in the second layer 510 is 2.

  In FIG. 5, a face image 501 corresponds to the face image 303 shown in FIG. Characteristic surfaces 503 a to 503 c indicate characteristic surfaces of the first hierarchy 506. The feature plane is an image data plane that stores a result of calculation while scanning data of the previous layer with a predetermined feature extraction filter (cumulative sum of convolution calculations and nonlinear processing). Since the feature plane is a detection result for raster-scanned image data, the detection result is also represented by a plane.

  The feature surfaces 503a to 503c are calculated by different feature extraction filters with reference to the face image 501. The characteristic surfaces 503a to 503c are generated by two-dimensional convolution filter operations corresponding to the convolution filters 504a to 504c and non-linear transformation (sigmoid function or the like) of the operation results, respectively. A reference image area 502 indicates an area necessary for the convolution calculation. For example, in a convolution filter operation with a filter size (horizontal length and vertical height) of 11 × 11, processing is performed by a product-sum operation as shown in the following equation (1).

  Here, input (x, y) represents a reference pixel value at coordinates (x, y), and output (x, y) represents a calculation result at coordinates (x, y). Also, weight (column, row) represents a weighting coefficient at coordinates (x + column, y + row), and columnSize = 11 and rowSize = 11 represent a filter size (the number of filter taps).

  The convolution filters 504a to 504c are filters having different coefficients, and the sizes of the convolution filters 504a to 504c are different depending on the feature plane. In the CNN operation, a product-sum operation is repeated while scanning a plurality of filters in units of pixels, and a final product-sum result is nonlinearly transformed to generate a feature plane. For example, when calculating the feature plane 503a, the convolution filter 504a is used because the number of connections with the previous layer is one. On the other hand, when calculating the feature planes 507a and 507b, since the number of connections with the previous layer is 3, the calculation results of the three convolution filters are cumulatively added. That is, the feature plane 507a is obtained by accumulating all the outputs of the convolution filters 508a to 508c and finally performing nonlinear conversion processing.

  Reference image areas 505a to 505c indicate areas necessary for the convolution calculation of the second hierarchy 510, respectively. The coefficient of the convolution filter is determined in advance by a conventionally known neural network learning method such as backpropagation. For learning CNN, for example, the method disclosed in Non-Patent Document 3 is used.

  The feature surfaces 507a and 507b obtained by the CNN calculation correspond to an image map indicating the possibility of the existence of target feature points. That is, the image map has a high value at the position where the feature point exists. In step S101, the center of gravity of the feature planes 507a and 507b is further calculated, and the coordinate value is used as the coordinates of the feature point position candidate.

  In steps S102 to S106, geometric correction processing (correction of outliers due to the performance of step S101) is performed on the 15 feature point position candidates obtained in step S101.

  FIG. 6 is a diagram for explaining an example of the geometric correction process. In the example shown in FIG. 6, the feature point 402a is a feature point characterized by the corner of the eye but is erroneously determined as the position of the eyebrow end. In the present embodiment, the position is corrected by statistical processing based on the arrangement relationship of human face features. That is, based on the knowledge obtained by learning (principal component analysis or the like), the position candidate indicated by the feature point 402a is corrected to the position indicated by the feature point 402b. Hereinafter, specific examples of the correction processing according to the present embodiment will be described in order.

First, in step S102, the CPU 207 functions as a generation unit, and generates a single feature point vector by simply connecting the coordinates of the plurality of feature point position candidates obtained in step S101. In this embodiment, a 30-dimensional feature vector V is generated from 15 feature point position coordinates. A data string obtained by simply connecting the position coordinates (x _i , y _i ) (i: feature point numbers 1 to 15) of each feature point is defined as a feature point vector V (element v _j : j = 1 to 30). The feature point numbers 1 to 15 correspond to the feature points 401 to 415 in this embodiment. Therefore, for example, elements v ₁ and v ₂ of the feature point vector correspond to the x coordinate value and the y coordinate value of the feature point 401, respectively. The feature vector V is defined by the following equation (2).

  In the formula, f represents the number of feature points, and T represents transposition.

  In step S103, the CPU 207 functions as a projecting unit, and calculates a projection vector P using the average vector A and the projection matrix E. The projection vector P is calculated by the following equation (3) using vector data obtained by subtracting the average vector A from the feature point vector V and the projection matrix E. Note that the projection matrix E and the average vector A are matrices calculated in advance by principal component analysis using feature point vectors (learning feature point vectors) for a large number of face images. Here, the learning feature points and vectors are vectors generated by connecting all the correct feature point position coordinates of the face image in the same manner. The average vector A is defined by the following equation (4), and the projection matrix E is defined by the following equation (5).

P = E ^T (V−A) (3)
A = (a ₁ , a ₂ , a ₃ ,..., A ₂ × _f ) ^T (4)
E = (u ₁ , u ₂ ,..., U _p ) (5)

In the equation, u ₁ , u ₂ ,..., U _p are 2 × f-dimensional orthonormal vectors (eigenvectors) obtained by principal component analysis. In the case of this embodiment, it is a 30-dimensional vector. p represents the dimension of the projection vector, and is set to 8, for example, as a value smaller than 2 × f dimension. In this case, the projection matrix E uses eight eigenvectors in descending order of the corresponding eigenvalues among orthogonal vectors obtained by principal component analysis. It is assumed that information of the projection matrix E and the average vector A is stored in advance in the ROM 208 or RAM 209.

As described above, in step S103, the dimension of the 2 × f-dimensional feature point vector is reduced to a p-dimensional projection vector by the calculation of Expression (3). That is, it projects onto a p-dimensional eigenspace. In the method described in Non-Patent Document 2, the original feature point vector dimension data, that is, the coordinate position is restored from the projection vector P. Here, the restoration vector V ′ is calculated by the following equation (6) using the projection matrix E and the average vector A described above. Hereinafter, the process (EP) of the first term of the equation (6) is referred to as a reverse projection process.
V ′ = EP + A (6)

  On the other hand, in this embodiment, the CPU 207 functions as a reverse projection unit in steps S104 to S106, and restores the feature point vector using a different number of projection vector elements for each feature point. FIG. 7 is a diagram schematically illustrating the projection process and the reverse projection process according to the present embodiment. Here, for the sake of explanation, a case will be described in which processing is performed on feature point vectors corresponding to three feature points.

In FIG. 7, a feature point vector 701 is a vector corresponding to feature point coordinate data. That is, feature point vectors (v ₁ , v ₂ , v ₂ ) generated from the coordinates (x ₁ , y ₁ ), (x ₂ , y ₂ ), (x ₃ , y ₃ ) of the feature point position candidates determined in step S101. _{v 3, v 4, v 5} , v 6) it is. For the sake of explanation, it is assumed that the mean vector A has already been subtracted from the feature point vector here. That is, it is assumed that VA in the expression (3) is processed. Reference numerals 702a to 702c denote a projection matrix E. Here, three eigenvectors are used to calculate a three-dimensional projection vector. The projection matrices 702a, 702b, and 702c are configured by three eigenvectors (e _1i , e _2i , e _3i : i = 1 to 6) in descending order of eigenvectors by principal component analysis. Projection vectors 703a to 703c are calculated by inner products of eigenvectors of projection matrices 702a, 702b, and 702c corresponding to feature point vector 701. That is, the elements p _n of the projection vector is calculated by the following equation at Step S103 (7).

Next, in step S104, the dimension of the feature point to be back-projected for each feature point (the number of elements of the projection vector used during back-projection) is determined. In this process, it is determined by referring to table information stored in the RAM 209 or ROM 208 in advance. FIG. 8 is a diagram showing an example of table information, and the number of reverse projection dimensions (number of elements of a projection vector used during reverse projection) is designated for each of the feature points 1, 2, and 3. In the case of the example shown in FIG. 7, the feature points corresponding to the coordinates (x ₁ , y ₁ ) and (x ₃ , y ₃ ) of the feature point position candidates are two-dimensionally back-projected, and the coordinates (x ₂ , y ₂ ) performs reverse projection processing in three dimensions.

In step S105, corrected feature point vectors 704 (v ₁ ′, v ₂ ′, v ₃ ′, v ₄ ′, v ₅ ′, v ₆₎ using the projection vectors 703 a to 703 c and the projection matrices 702 a to 702 c. '). For example, corrected feature point vectors v ₁ ′ and v ₂ ′ corresponding to the feature point (x ₁ , y ₁ ) are calculated by the following equation (8) using the projection vector elements p ₁ and p _2. .
v ₁ '= (p ₁ × e ₁₁ ) + (p ₂ × e ₂₁ )
v ₂ '= (p ₁ × e ₁₂ ) + (p ₂ × e ₂₂ ) (8)

On the other hand, the corrected feature point vectors v ₃ ′ and v ₄ ′ corresponding to the coordinates (x ₂ , y ₂ ) are calculated by the following equation (9).
_{v 3 '= (p 1 ×} e 13) + (p 2 × e 23) + (p 3 × e 33)
v ₄ ′ = (p ₁ × e ₁₄ ) + (p ₂ × e ₂₄ ) + (p ₃ × e ₃₄ ) (9)

The corrected feature point vector (v ₁ ′, v ₂ ′) is selected as dimension number 2 in step S104, and is subjected to reverse projection processing in step S105 using two projection vector elements. The elements of the projection vector are selected in order from the element projected by the corresponding eigenvector having a large eigenvalue. That is, an element projected by a corresponding eigenvector having a small eigenvalue is not used. On the other hand, for the corrected feature point vector (v ₃ ′, v ₄ ′), the dimension number 3 is selected in step S104, and reverse projection processing is performed in step S105 using all projection vector elements.

  In step S106, it is determined whether or not the above processing has been performed for all feature points. As a result of the determination, if there is a feature point that has not been processed, the process returns to step S104, and if processing has been performed for all feature points, the process proceeds to step S107. In step S107, the CPU 207 functions as a determination unit, adds the average vector A to the back-projected restored vector V ′, extracts the corrected coordinate data of the position corresponding to the feature point, and stores it in the RAM 209 or the like. .

  As described above, by the processing from step S102 to step 106, the feature point vector obtained by connecting the coordinates of the position candidate positions of the plurality of feature points is projected onto the eigenspace with reduced dimensions, and then back-projected, so that the statistical deviation is obtained. The value can be corrected. That is, an outlier that cannot be expressed in the projected space, which is caused by erroneous detection when determining the coordinates of the position candidate of the feature point, is statistically corrected.

  Here, when the projection dimension number is high, the degree of freedom of the eigenspace projected by the projection matrix is high, and when the projection dimension number is low, the degree of freedom of the eigenspace projected by the projection matrix is low. That is, when the projection dimension number is high, the constraint force of the geometric arrangement relationship is weak, and when the projection dimension number is low, the constraint force is strong. In this embodiment, the number of dimensions to be back-projected for each feature point can be changed at the time of back-projection, so that, for example, when the performance of the process for determining the position candidate of the feature point is low, the back-projection of the feature point When performing, reverse projection is performed by reducing the number of projection vectors to be used. Thereby, control, such as raising restraint force, is realizable by simple processing.

  Note that the number of reverse projection dimensions required for each feature point is determined in advance using a plurality of evaluation data. For example, it is determined based on a statistical value of a deviation amount from the correct answer of the feature point vector after correction, using data having position information of the correct feature point. For example, the number of dimensions of reverse projection for feature points having a large variance is reduced.

  As described above, according to the present embodiment, in the process of correcting the position of the feature point using the eigenspace, the number of elements of the projection vector used for back projection from the eigenspace is controlled for each feature point. did. This eliminates the need for complicated processing including multiplication and division, makes it possible to perform appropriate correction for each feature point by simple processing, and improve the performance of detecting the position of the feature point.

(Second Embodiment)
FIG. 9 is a flowchart illustrating an example of a processing procedure for determining the position of a feature point in the present embodiment. The configuration of the information processing apparatus according to the present embodiment is the same as that shown in FIG. 9 is also performed by the operation of the CPU 207 in FIG.
First, in step S901, feature point position candidates are determined. In this process, similar to step S101 of FIG. 1 described in the first embodiment, the candidate position of the feature point is obtained by CNN calculation and the centroid search for the result as the candidate position of the plurality of feature points.

  In the next step S902, as in step S102, the feature point vectors are generated by connecting the position candidate coordinates of a plurality of feature points. In step S903, a projection matrix for projecting the feature point vector to the eigenspace is selected. The projection matrix selected here is the same as the projection matrix used in step S103, and a projection matrix for projection stored in advance in the ROM 208 or RAM 209 is selected. Next, in step S904, the feature point vector is projected onto the eigenspace using the selected projection matrix. The processing here is also the same as the processing content described in step S103.

  Next, in step S905, a projection matrix for back projection from the eigenspace to the original space is selected. Here, the case where the projection matrix to be used is processed with respect to the feature point vectors corresponding to the three feature points as in the first embodiment will be described with reference to FIG.

  In FIG. 10, the feature point vector 1001, the projection matrices 1002a to 1002c, the projection vectors 1003a to 1003c, and the corrected feature point vector 1004 are the same as in the example shown in FIG. On the other hand, in the example shown in FIG. 10, a part of the inverse projection matrices 1005a to 1005c is different from the example shown in FIG.

  In the first embodiment, the projection matrix for projection and the projection matrix for reverse projection use common data, and by selecting the number of elements of the projection vector used during the reverse projection processing, the constraint force is set for each feature point. I have control. On the other hand, in the present embodiment, reverse projection processing is executed using three projection vectors for all feature points in accordance with the calculation of the above-described equation (6). However, in the present embodiment, the constraint force is controlled for each feature point by using an inverse projection matrix different from the projection matrix used at the time of projection.

In the example shown in FIG. 10, the inverse projection matrix 1005c is partially different from the projection matrix 1002c. Specifically, the values e ₂₁ , e ₂₂ , e ₂₁ , e ₂₂ are all 0. As a result, the corrected feature point vectors v ₁ ′, v ₂ ′, v ₅ ′, v ₆ ′ are equivalent to the reverse projection using the _two projection vectors p ₁ , p ₂ . Further, the corrected feature point vectors v ₃ ′ and v ₄ ′ are equivalent to reverse projection using the _three projection vectors p ₁ , p ₂ , and p ₃ . For example, the corrected feature point vector v ₁ ′ is calculated by the following formula (10) and is equivalent to the above-described formula (8).
v ₁ '= (p ₁ × e ₁₁ ) + (p ₂ × e ₂₁ ) + (p ₃ × 0) (10)

  Thus, by setting a part of the projection matrix for reverse projection to 0, it is possible to obtain a result equivalent to that of the first embodiment without changing the processing content for each feature point.

  In the next step S906, a feature point vector after correction in the real space is generated from the projection vector using the inverse projection matrix selected in step S905. In step S907, the position of the corresponding feature point is extracted from the corrected feature point vector, and the information is stored in the RAM 209 or the like.

  As described above, according to the present embodiment, by generating a projection matrix for back projection in advance, when performing back projection processing, the number of elements of the projection vector to be used is determined as in step S104. The processing to be performed can be made unnecessary. Thereby, the geometrical restraint force for every feature point can be controlled by simpler processing. That is, as compared with the conventional example described in Non-Patent Document 2, geometric correction with different correction force is processed for each feature point by simple change by simply selecting the matrix for back projection at the time of back projection. be able to.

(Other embodiments)
In the first and second embodiments, the case of performing reverse projection using a predetermined number (projection dimension number) of projection vector elements for each feature point has been described. However, the projection vector used for each feature point at the time of execution is described. The number of elements may be changed. For example, when a value corresponding to the detection reliability is output in the process of determining the candidate position of the feature point in step S101 or S901, the reverse projection dimension number is set based on the value.

  Further, in each embodiment, when determining a feature point candidate by the CNN, the reliability may be determined using the output value of the CNN. For example, when the output value of CNN is high, it is determined that there is a high possibility of the target feature point, and the number of reverse projection dimensions is increased. This processing can be easily realized by preliminarily storing table information indicating the relationship between the reliability and the reverse projection dimension number in the ROM 208 or the RAM 209 and determining the reverse projection dimension number in step S104.

  In the configuration of the second embodiment, a plurality of projection matrices for reverse projection are prepared, and the projection matrix is selected according to the reliability output in step S901. In that case, table information indicating the relationship between the reliability and the projection matrix is stored in the ROM 208 or RAM 209 in advance. Also in this case, by setting a predetermined element to 0 for each projection matrix, it is possible to perform reverse projection processing with different dimensions (elements of different numbers of projection vectors) for each feature point.

FIG. 11 is a diagram for explaining an example of a projection matrix for reverse projection. 11 (a) to 11 (c) show cases in which the number of inverse projection dimensions of the feature point vectors (v ₁ , v ₂ ), (v ₃ , v ₄ ), (v ₅ , v ₆ ) is reduced, respectively. It is a projection matrix. For example, when the reliability of the detection of the position candidate corresponding to the feature point vector (v ₁ , v ₂ ) is low, the projection matrix shown in FIG. 11A is selected at the time of reverse projection. In the first and second embodiments, the case where the candidate position of the feature point is determined by the CNN has been described. However, the present invention is not limited to this and can be applied to any method.

  In the first and second embodiments, the case where the position of the feature point corresponding to the specific organ position is detected from the face image in the image has been described. However, the present invention is not limited to this. It can be applied to detection of feature points of various object regions.

  Moreover, although the case where the position of the feature point on two-dimensional data was determined was demonstrated in 1st and 2nd embodiment, it does not necessarily restrict to this. For example, in the case of processing based on three-dimensional data, it can be applied to position determination of feature points in a three-dimensional space. In this case, a feature point vector V is created by connecting three-dimensional coordinate values.

  In the first and second embodiments, only the method of calculating the corrected feature point positions has been described. Actually, the corrected feature point is not the final feature point position, but the final feature point position is determined in combination with other processing not described.

  Furthermore, in the second embodiment, a case has been described in which a projection vector used for reverse projection is generated by rewriting some of the elements of the projection vector to 0. However, the present invention is not limited to 0, and a value smaller than 1 is used. If it exists, the effect similar to 2nd Embodiment can be acquired.

  In the first and second embodiments, the case where the geometric arrangement relationship is constrained using the projection matrix calculated by the principal component analysis has been described. The method based on principal component analysis can best bring out the effects of the present invention by a simple method, but a projection matrix generated by another method may be used.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

207 CPU
208 ROM
209 RAM

Claims (7)

Position candidate determination means for obtaining position candidates of a plurality of feature points from the object region;
Generating means for connecting the coordinates of a plurality of position candidates obtained by the position candidate determining means to generate a feature point vector;
Projecting means for projecting the feature point vector generated by the generating means onto a space of a predetermined dimension to obtain a projection vector;
Element number determination means for determining the number of elements of a projection vector used for back projection for each position candidate of the plurality of feature points;
Each of the plurality of feature point position candidates is composed of elements of the number of elements determined for the position candidates by the element number determination means among the elements of the projection vector obtained by the projection means. A reverse projection means for performing reverse projection using a vector ;
Position determining means for determining positions of a plurality of feature points obtained by correcting position candidates of the plurality of feature points using a result of the reverse projection means;
An information processing apparatus comprising:   The projecting means projects the feature point vector using a projection matrix, and the inverse projection means reversely projects the projection vector using a projection matrix different from the projection matrix used by the projection means. The information processing apparatus according to claim 1. The projection matrix used by the inverse projection means is a matrix in which the elements of the matrix corresponding to the elements of the projection vector not used when performing the reverse projection among the elements of the projection matrix used by the projection means are zero. The information processing apparatus according to claim 2 . When the position candidate determination means obtains position candidates of the plurality of feature points, the position candidate determination means further includes a determination means for determining reliability for each position candidate ,
Said element number determination means, on the basis of the reliability determined by the determining means, according to claim 1, characterized in that to determine the number of elements of the projection vector for use in the inverse projection for each position candidate of feature points Information processing device. The projection matrix projecting means is employed, according to claim 2 or 3, characterized in that a matrix generated by the principal component analysis of a plurality of feature points vector obtained from the coordinates of the position of a feature point as a learning Information processing device. A position candidate determination step for obtaining position candidates of a plurality of feature points from the object region;
Generating a feature point vector by connecting the coordinates of a plurality of position candidates obtained in the position candidate determination step;
A projection step of projecting the feature point vector generated in the generation step to a space of a predetermined dimension to obtain a projection vector;
An element number determination step for determining the number of elements of a projection vector used for back projection for each of the plurality of feature point position candidates;
Each of the plurality of feature point position candidates is composed of elements having the number of elements determined for the position candidates in the element number determination step among the elements of the projection vector obtained in the projection step. A reverse projection process using a vector for reverse projection;
A determination step for determining positions of a plurality of feature points obtained by correcting the position candidates of the plurality of feature points using a result in the reverse projection step;
An information processing method characterized by comprising: A position candidate determination step for obtaining position candidates of a plurality of feature points from the object region;
Generating a feature point vector by connecting the coordinates of a plurality of position candidates obtained in the position candidate determination step;
A projection step of projecting the feature point vector generated in the generation step to a space of a predetermined dimension to obtain a projection vector;
An element number determination step for determining the number of elements of a projection vector used for back projection for each of the plurality of feature point position candidates;
Each of the plurality of feature point position candidates is composed of elements having the number of elements determined for the position candidates in the element number determination step among the elements of the projection vector obtained in the projection step. A reverse projection process using a vector for reverse projection;
A determination step for determining positions of a plurality of feature points obtained by correcting the position candidates of the plurality of feature points using a result in the reverse projection step;
A program that causes a computer to execute.

Images