JP2007164669A - Three-dimensional pattern position/attitude detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To precisely and surely detect the position and attitude of a three-dimensional pattern of a three-dimensional object imaged. <P>SOLUTION: A reference template PR is decided with respect to the detection object (human face). The reference template PR is formed into a three-dimensional pattern by using a three-dimensional pattern of a marked portion (head of a nose, line of nose) as a reference position and a reference attitude. The three-dimensional pattern PX of the detection object imaged is aligned to the reference template PR by an ICP algorithm, so as to detect the position and the attitude of the three-dimensional pattern PX of the detection object, in a position/attitude detecting part 3. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a three-dimensional pattern position / posture detection apparatus that picks up a three-dimensional object such as a human face as a detection target and detects the position and posture of the three-dimensional pattern of the picked-up three-dimensional object.

  In recent years, in fields that require personal authentication, such as access control to computer rooms and important machine rooms, and access control to computer terminals and financial terminals of banks, instead of conventional passwords and ID cards, Devices for performing personal authentication based on three-dimensional biological information (biometrics) such as a face and a vein have been developed.

  For example, the detection target (three-dimensional object) is imaged by two cameras from different positions, the unevenness of each part of the three-dimensional object imaged based on the stereo image is measured, and the position of the three-dimensional object in the three-dimensional space There has been proposed a three-dimensional pattern matching device that creates a three-dimensional pattern including a posture and performs matching between a “registered three-dimensional pattern” obtained at the time of registration and a “collation three-dimensional pattern” obtained at the time of matching. (See Patent Document 1).

  In this three-dimensional pattern collation apparatus, the collation three-dimensional pattern and the registered three-dimensional pattern are used to increase the collation accuracy, and the positional deviation (X, Y, Z-axis direction deviation) or rotational deviation (X It is important to accurately obtain the deviations of the Y, Z, and Z axes and match the positions and orientations of the matching three-dimensional pattern and the registered three-dimensional pattern.

  Conventionally, there is an ICP (Iterative Closest Point) algorithm as a method for matching the position and orientation of a three-dimensional pattern (see Patent Document 2). In this ICP algorithm, when the registered three-dimensional pattern data is M and the collation three-dimensional pattern data is M ′, the positions and orientations of the collation three-dimensional pattern and the registered three-dimensional pattern are the following (1) to (4 ).

(1) For all the points mi included in M, the closest point mi ′ is searched from M ′, and the set of obtained mi and mi ′ is set as a set of corresponding points.
(2) Based on the current correspondence relationship, an optimal motion pattern (rotation R, translation t) between the two three-dimensional pattern data M and M ′ is estimated by the least square method.
(3) M ′ is coordinate-transformed using R and t obtained in (2) above.
(4) The above (1) to (3) are repeated until the change in R and t becomes sufficiently small and converges with respect to M and M ′ after coordinate transformation.

  In this case, by matching the position and orientation of the matching 3D pattern with the position and orientation of the registered 3D pattern, the position and orientation of the matching 3D pattern based on the registered 3D pattern are detected from the matching result. can do.

JP-A-9-259271 JP-A-9-277184

  However, in the above-described conventional method for detecting the position and orientation of the three-dimensional pattern, for example, when the detection target is biological information such as a human face, individual differences are significant, and characteristic portions in the registered three-dimensional pattern The position and posture of the are not always fixed positions. For example, the head of the nose may be low or the nasal muscles may be bent. In such a case, the matching three-dimensional pattern and the registered three-dimensional pattern may be misaligned and corresponding positions may be aligned, and the position and orientation of the matching three-dimensional pattern may not be detected correctly. It was.

  The present invention has been made to solve such a problem, and an object of the present invention is to be able to correctly and reliably detect the position and posture of a three-dimensional pattern of a three-dimensional imaged object. An object of the present invention is to provide a dimension pattern position / posture detection apparatus.

  In order to achieve such an object, the first invention (the invention according to claim 1) captures a three-dimensional object as a detection target, and detects the position and orientation of a three-dimensional pattern of the captured three-dimensional object. In the three-dimensional pattern position / posture detection apparatus, a reference template storage means for storing, as a reference template, a reference template storage means for storing a three-dimensional pattern that is determined for a detection target and whose three-dimensional pattern of the target portion is a reference position and reference posture; And a position / posture detecting means for detecting the position and posture of the three-dimensional pattern of the three-dimensional object imaged by performing the above positioning. In the present invention, “alignment” means matching both “position” and “posture”.

In the present invention, for example, when a detection target is a human face, a captured three-dimensional pattern of a face and a reference template (for example, “nose head” as a reference position and “nasal muscle” as a reference posture as a target portion) The position and orientation of the captured three-dimensional pattern of the face are detected by performing alignment with the three-dimensional pattern) using an ICP algorithm or the like.
In the reference template, attention portions such as “nose head” and “nasal muscle” are set as a reference position and a reference posture, and are standard with no individual differences. Accordingly, the alignment of the captured three-dimensional pattern of the face and the reference template is performed so as to correspond to the target portion without individual differences, and the alignment may be performed by mistaken corresponding points. As a result, the position and orientation of the captured three-dimensional pattern of the face can be detected correctly and reliably.

  In the present invention (first invention), for example, the detection target is a human face, the reference template includes a “hairstyle”, and the “hairstyle” of the reference template is the same as the “hairstyle” of the imaged face. In such a case, there is a possibility that the alignment is performed by taking the correspondence with the “hairstyle”, not the attention portion such as the “nose head” or the “nasal muscle”. Therefore, in the second invention (the invention according to claim 2), the reference template is a three-dimensional pattern of the attention area including the attention portion. For example, a 3D pattern of a limited area including “head of nose” and “nose muscle” is used as a reference template, and an extra portion such as “hairstyle” is excluded from the reference template, thereby 3 Increase the detection accuracy of the position and orientation of the dimensional pattern.

  According to the first aspect of the present invention, the reference template is defined with the three-dimensional pattern of the target portion as the reference position and the reference posture for the detection target, and the three-dimensional image of the three-dimensional object imaged by performing alignment with the reference template. Since the position and orientation of the pattern are detected, there is less possibility that the corresponding points will be mistaken for positioning, and the position and orientation of the three-dimensional pattern of the captured three-dimensional object are detected correctly and reliably. It becomes possible.

  According to the second invention, in the first invention, since the three-dimensional pattern of the attention area including the attention portion is used as the reference template, for example, 3 of the limited area including only the attention portion such as “nose” and “nasal muscle”. By using a dimensional pattern as a reference template and excluding extra portions such as “hairstyle” from the reference template, it is possible to improve the detection accuracy of the position and orientation of the three-dimensional pattern of the captured three-dimensional object.

Hereinafter, the present invention will be described in detail with reference to the drawings.
[Embodiment 1: Three-dimensional pattern position / posture detection apparatus]
FIG. 1 is a block diagram showing a main part of an embodiment of a three-dimensional pattern position / posture detection apparatus according to the present invention.

  In the figure, reference numeral 1 denotes a human face as a detection target, which is imaged by two cameras 1-1 and 1-2, and the three-dimensional pattern ( Detection target 3D face pattern) 3D pattern generation unit for generating PX, 2 is a reference template storage unit for storing a reference template PR determined for the detection target (human face), 3 is a 3D pattern generation unit Position / attitude detection for detecting the position and orientation of the detection target 3D face pattern PX by aligning the detection target 3D face pattern PX sent from 1 with the reference template PR read from the reference template storage unit 2 Part.

  In this embodiment, the reference template PR creates, for example, an average three-dimensional pattern (average three-dimensional face pattern) of a human face that is a detection target, and “nose” as a target portion from the average three-dimensional face pattern. 3D pattern of the attention area including “head of nose” and “nose muscle” is extracted, and “nose head (A)” of the extracted three-dimensional pattern of the attention area is set as X = 0, Y = 0, Z = 0. The reference position is set, and the "nose muscle (B)" is positioned on the Y-axis to obtain the reference posture. FIG. 4 shows the relationship between the attention area AR on the human face and the attention portion TA included in the attention area AR. In the present embodiment, the attention area AR is defined as an area centering on the nose and not including eyes, mouth and hair.

  If the three-dimensional average is simply obtained by combining the position of the head of the nose and the nose muscle line, an average three-dimensional face pattern (average face) can be obtained. Also, the average face may be created by normalizing and averaging the sizes of the positions of the two eyes and the nose.

  In this embodiment, the position / posture detection unit 3 detects the position and posture of the detection target three-dimensional face pattern PX by aligning the detection target three-dimensional face pattern PX and the reference template PR using the ICP algorithm. . In this case, the data of the reference template PR is M, the data of the detection target three-dimensional face pattern PX is M ′, and the positions and orientations of the detection target three-dimensional face pattern PX and the reference template PR are the following (1) to (4 ) To detect the position and orientation of the detection target three-dimensional face pattern PX with reference to the reference template PR.

(1) For all the points mi included in M, the closest point mi ′ is searched from M ′, and the set of obtained mi and mi ′ is set as a set of corresponding points.
(2) Based on the current correspondence relationship, an optimal motion pattern (rotation R, translation t) between the two three-dimensional pattern data M and M ′ is estimated by the least square method.
(3) M ′ is coordinate-transformed using R and t obtained in (2) above.
(4) The above (1) to (3) are repeated until changes in R and t become sufficiently small and converge with respect to M and M ′ after coordinate transformation.

  In this embodiment, in order to speed up the calculation, in the initial stage of the ICP algorithm, rough alignment is performed using a small number of corresponding points, and the number of corresponding points is gradually increased as the processing proceeds. Increase and adjust the position.

  In the reference template PR, “head of nose (A)” and “nose muscle (B)” are set as reference positions and reference postures, and are standard with no individual differences. Therefore, the alignment between the detection target three-dimensional face pattern PX and the reference template PR corresponds to “nose head (A)” and “nasal muscle (B)” in the reference template PR without individual differences. Will be done. As a result, the possibility that the corresponding points are mistakenly aligned is reduced, and the position and orientation of the detection target three-dimensional face pattern PX can be detected correctly and reliably.

  In the present embodiment, the three-dimensional pattern of a limited area including “nose head (A)” and “nose muscle (B)” is used as the reference template PR, that is, from the reference template PR to “hairstyle” and the like. Since unnecessary portions are excluded, there is no fear that the alignment is performed so as to correspond to the “hairstyle”, and the detection accuracy of the position and orientation of the detection target three-dimensional face pattern PX is increased.

  In the present embodiment, the reference template PR may include not only “nose” but also “eyes”, “mouth”, and the like as attention parts. Also, instead of creating an average three-dimensional pattern (average face) of a human face, a reference portion PR is created by extracting a target portion from a three-dimensional pattern of a human face that seems to be an average. Also good. Further, when viewed globally, it is considered that the average three-dimensional pattern of a human face differs depending on the country or region, and a representative reference template PR may be created for each country or region. Further, the target portion to be included in the reference template PR may be manually designated.

  In the present embodiment, the detection target is a human face, but the detection target is not limited to a human face, and there are always high and low parts as human features. In the case of a three-dimensional object, the position and posture of the three-dimensional object can be detected in the same manner as in the case of a human face.

  Further, in the present embodiment, the alignment of the detection target three-dimensional face pattern PX and the reference template PR is performed by the ICP algorithm, but the ICP algorithm need not necessarily be used.

[Embodiment 2: Three-dimensional pattern matching device (1: 1 matching)]
FIG. 2 is a block diagram showing a main part of an embodiment of a three-dimensional pattern matching apparatus including a three-dimensional pattern position / attitude detection apparatus according to the present invention. 1, the same reference numerals as those in FIG. 1 denote the same or equivalent components as those described with reference to FIG. 1, and the description thereof will be omitted.

  The three-dimensional pattern matching device includes a three-dimensional pattern creation unit 1, a reference template storage unit 2, a first position / posture detection unit 3-1, a second position / posture detection unit 3-2, and a database. 4 and a collation unit 5.

  The first position / posture detection unit 3-1 reads the reference template PR from the reference template storage unit 2, and the verification target 3D face pattern (matching target 3D face pattern) sent from the 3D pattern creation unit 1. ) PX and reference template PR are aligned by the ICP algorithm, and sent to collation unit 5 as collation three-dimensional face pattern PX ′ whose position and orientation are corrected.

  The second position / posture detection unit 3-2 reads the reference template PR from the reference template storage unit 2, and the registration target 3D face pattern (registration target 3D face pattern) sent from the 3D pattern creation unit 1. ) Position alignment between PY and reference template PR is performed by the ICP algorithm, and stored in the database 4 as a registered three-dimensional face pattern PY ′ whose position and orientation are corrected.

  When the collation 3D face pattern PX ′ is sent from the position / posture detection unit 3-1, the collation unit 5 reads the registered 3D face pattern PY ′ from the database 4 and registers it as the collation 3D face pattern PX ′. Collation with the three-dimensional face pattern PY ′ is performed. In this collation, the collation 3D face pattern PX ′ and the registered 3D face pattern PY ′ correspond to the “nasal head (A)” and “nasal muscle (B)” of the reference template PR, so that the reference The alignment with the template PR is performed correctly and reliably. Therefore, the collation three-dimensional face pattern PX ′ and the registered three-dimensional face pattern PY ′ are collated with their positions and postures correctly matched.

  The ICP algorithm is very time consuming because the operation is repeated many times until the operation converges. In the present embodiment, the registration target three-dimensional face pattern PY registered by the ICP algorithm is stored in the database 4 as the registered three-dimensional face pattern PY ′, and the registered three-dimensional face pattern PY ′ stored in the database 4 is stored. Is read out as necessary, and collation with the collation three-dimensional face pattern PX ′ is performed. Therefore, alignment by the ICP algorithm at the time of collation may be performed once by the position / posture detection unit 3-1. Time is short.

  In this embodiment, the position and posture of the registration target three-dimensional face pattern PY are corrected, and the registered three-dimensional face pattern PY ′ having the corrected position and posture is stored in the database 4. The target three-dimensional face pattern PY may be stored, the position and orientation of the registration target three-dimensional face pattern PY may be corrected at the time of matching, and matching with the matching three-dimensional face pattern PX ′ may be performed. In this case, as the alignment at the time of collation, in addition to the alignment between the verification target 3D face pattern PX and the reference template PR, the registration target 3D face pattern PY and the reference template PR need to be aligned. In the case of 1 collation, the alignment speed is sufficient, so the collation speed does not drop so much. Of course, it is needless to say that the registered three-dimensional face pattern PY ′ is stored in the database 4 and collation is performed.

[Embodiment 3: Three-dimensional pattern matching device (1: N matching)]
The ICP algorithm is very time consuming because the operation is repeated many times until the operation converges. When 1: N collation is performed, alignment by the ICP algorithm N + 1 times is necessary. If this is performed at the time of collation, the processing time becomes enormous and the collation speed decreases. Therefore, in the 1: N collation, as shown in FIG. 3, the position / posture detection unit 3-2 performs ICP registration for each of the registration target 3D face patterns PY1, PY2,... PYN and the reference template PR. The registered three-dimensional face patterns PY1 ′, PY2 ′,... PYN ′ corrected by the algorithm and corrected in position and orientation are stored in the database 4.

  When the collation unit 5 receives the collation three-dimensional face pattern PX ′, the collation unit 5 sequentially reads the registered three-dimensional face patterns PY1 ′, PY2 ′,... PYN ′ from the database 4, and performs the collation three-dimensional face pattern PX ′. Is checked. At the time of this collation, the registered three-dimensional face patterns PY1 ', PY2',... PYN 'have already been registered by the ICP algorithm at the time of registration. Therefore, the processing time can be significantly reduced and 1: N matching can be performed at a high speed, compared with the case where alignment is performed by the ICP algorithm each time at the time of matching.

  In the above-described three-dimensional pattern matching apparatus, the first position / posture detection unit 3-1 is provided for verification, and the second position / posture detection unit 3-2 is used for registration. The posture detection unit 3-1 may be used for both verification and registration, and the second position / posture detection unit 3-2 may be omitted. In addition, the collation by the collation unit 5 may adopt any collation method such as a feature point method or a correlation method.

  Further, before collation between the collation 3D face pattern PX ′ and the registered 3D face pattern PY ′, the “nose head” and “nasal muscle” of the collation 3D face pattern PX ′ and the registered 3D face pattern PY ′. It is also possible to weight each pattern data of the attention area including As a result, individual pattern data of the area of interest including “nose head” and “nasal muscles” can be emphasized more than pattern data of other areas, and matching accuracy can be improved.

  In addition, as an average three-dimensional pattern (average three-dimensional face pattern) of a human face when determining the reference template PR, for example, a registered average three-dimensional face pattern that is an average of the registered three-dimensional face patterns PY1 to PYN is created. Then, the reference template PR may be created by extracting a target part from the created registered average three-dimensional face pattern. In this way, the user's face information is reflected in the reference template PR, so that the alignment accuracy is increased and the matching accuracy is further improved.

It is a block diagram which shows the principal part of one Embodiment (Embodiment 1) of the three-dimensional pattern position and attitude | position detection apparatus which concerns on this invention. It is a block diagram which shows the principal part of one Embodiment (Embodiment 2) of the three-dimensional pattern collation apparatus containing the three-dimensional pattern position and attitude | position detection apparatus which concerns on this invention. It is a block diagram which shows the principal part of the three-dimensional pattern collation apparatus (Embodiment 3) in the case of performing 1: N collation. It is a figure which shows the relationship between the attention area in a human face, and the attention part contained in this attention area.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Three-dimensional pattern preparation part, 1-1, 1-2 ... Camera, 2 ... Reference | standard template memory | storage part, 3,3-1, 3-2 ... Position and attitude | position detection part, 4 ... Database, 5 ... Collation part, PR: reference template, PX: detection target 3D face pattern (matching target 3D face pattern), PY: registration target 3D face pattern, PX ′: matching 3D face pattern, PY ′: registration 3D face pattern, AR ... attention area, TA ... attention part.

Claims (2)

In a three-dimensional pattern position / posture detection device that picks up a three-dimensional object as a detection target and detects the position and posture of the three-dimensional pattern of the picked-up three-dimensional object,
A reference template storage means for storing, as a reference template, a three-dimensional pattern that is defined for the detection target and in which the three-dimensional pattern of the target portion is a reference position and a reference posture;
A three-dimensional pattern position / posture detection device comprising: position / posture detection means for detecting the position and posture of the three-dimensional pattern of the imaged three-dimensional object by performing alignment with the reference template. In the three-dimensional pattern position / attitude detection device according to claim 1,
The reference template storage unit stores a three-dimensional pattern of an area of interest including the portion of interest as a reference template.

Images