JP4929828B2 - Three-dimensional authentication method, three-dimensional authentication device, and three-dimensional authentication program - Google Patents

Description

  The present invention relates to a three-dimensional authentication method, a three-dimensional authentication apparatus, and a three-dimensional authentication program for detecting fraud authentication during personal authentication using a face image.

  As a problem at the time of personal authentication using a face image, there is an unauthorized authentication using cracking. Unauthorized authentication by cracking means that a person other than the person “spoofs” the person by some means and passes through the authentication system. As a specific method, a photograph taken by the person is often used because of the ease of execution. Impersonation using a photo can be used by printing on a medium such as paper or using it on a display device such as a monitor.

  The face on the display object is a plane in the three-dimensional world. That is, in the three-dimensional world that is the real world, the face displayed on paper or a monitor is two-dimensional information (plane). Also, in the real world, there are movements that can be used. Japanese Patent Application Laid-Open No. 2004-151561 describes one method of determining based on the movement of a partial area of a face. In Patent Document 1, attention is paid to the opening and closing of eyes and mouths. This method is one countermeasure because the eyes and mouth are easy to find among the facial features and easily move.

  As another approach, there is a method using the fact that the face on the display object is a plane in the three-dimensional world, whereas the real face is three-dimensional.

  Patent Document 2 describes a method based on this policy, a method of irradiating light, a method of stereo vision, and a method of registering a plurality of face images during specific operations. Japanese Patent Application Laid-Open No. H10-228561 discloses a method of using a distance measurement sensor more directly.

  In the method of irradiating light, the shadow generated from the fact that the face has a three-dimensional shape is used. The method using stereo vision is triangulation using each of images taken by a plurality of cameras whose positional relationships are known, or using a plurality of images taken from different viewpoints by one camera whose position is controlled. This is a method for measuring a three-dimensional shape based on the principle.

  In the method of registering multiple face images for specific actions, if images of multiple postures are registered at the time of registration and a front image and a certain or more posture change image are used at the time of authentication, impersonation by a plane object such as a photo is performed. It is easy to eliminate. The distance measuring sensor includes a sensor that irradiates pattern light and measures the image, and in principle, is close to a combination of a light irradiation method and a stereo method.

JP 2002-251614 A (paragraphs 0045-0055) JP 2003-178306 A (paragraphs 0015, 0039, 0055) Japanese Patent Laying-Open No. 2005-135271 (paragraphs 0022-0026)

  However, the method described in Patent Document 1 has the following problems. The movement of the kite is small and difficult to find depending on the person or the captured image. The movement of the eyes and mouth greatly changes the facial expression of the person, so there is also a psychological resistance that is difficult to do where the surrounding eyes are concerned. Difficult to implement.

  In addition, the method of irradiating light, which is the method described in Patent Document 2, is easily affected by illumination conditions, so the place of use is limited and it is difficult to expect stability of performance. In the stereo method, the problem of shape measurement by stereo view is applied as it is, and a camera with multiple positional relationships or a device for controlling the position of one camera is required, and the use place and mounting conditions are limited. The

  In the method of registering face images at a plurality of specific operations, which is a method described in Patent Document 3, registration at a plurality of specific operations is troublesome, and there is a problem that face authentication other than the front is difficult. Although the method using the distance measuring sensor is reliable, it is expensive and difficult to mount on a small device such as a portable terminal.

  One of the objects of the present invention is to provide a three-dimensional authentication method, a three-dimensional authentication device, and a three-dimensional authentication program for preventing unauthorized authentication by a photograph or the like in face authentication.

  One of the objects of the present invention is to provide a three-dimensional authentication method, a three-dimensional authentication device, and a three-dimensional authentication program for preventing unauthorized authentication by a photograph or the like in face authentication in a small device such as a portable terminal.

  One of the objects of the present invention is to provide a three-dimensional authentication method, a three-dimensional authentication device, and a three-dimensional authentication program that prevent unauthorized authentication by a photograph or the like in face authentication using only a single camera.

The three-dimensional authentication method according to the present invention is a three-dimensional authentication method for determining the three-dimensionality of a photographing object, and acquires a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects. And feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired in the input step, A feature point extraction step for acquiring the coordinates of the point, and among the feature points extracted in the feature point extraction step, a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane A solid amount calculation step for calculating a three-dimensionality evaluation amount defined as an amount for evaluating the three-dimensionality of a certain three-dimensional evaluation point, and a photographing pair based on the three-dimensionality evaluation amount calculated in the three-dimensional amount calculation step. A three-dimensionality determination step for determining the three-dimensionality of the object, and the coordinates of the feature point in the three-dimensional coordinate system of the three-dimensional evaluation point converted into the two-dimensional coordinate system in the three-dimensional amount calculation step; The difference between the coordinate of the feature point in the three-dimensional coordinate system and the coordinate converted into the two-dimensional coordinate system is calculated for each captured image, and the difference calculated for each captured image is combined into one as a matrix. calculating a rank of a judgment matrix as a stereoscopic evaluation quantity, with the three-dimensional determination step, when the rank of the judgment matrix is 3, the three-dimensional evaluation point and judging not on a plane .

  In the input step, a photographed image obtained by photographing a person's face as a photographing object may be acquired. According to such a configuration, it is possible to determine the three-dimensionality of a person's face, and the photographed image is an image in which a person's face photograph is captured as a subject, or a real three-dimensional face is captured as a subject. It can be determined whether the image.

  In the feature point extraction step, feature points corresponding to four or more points of the center of the left eye of the human face, the center of the right eye, the left end of the lips, the right end of the lips, the apex of the nose, the bottom of the nose, and the upper end of the lips May be extracted. According to such a configuration, the three-dimensionality can be determined without changing the facial expression.

The three-dimensional authentication apparatus according to the present invention is a three-dimensional authentication apparatus for determining the three-dimensionality of a photographing object, and acquires a plurality of photographed images photographed by a single photographing apparatus from a plurality of viewpoints having different photographing objects. And feature points corresponding to four or more points on the object to be photographed that are not in the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired by the input means, A feature point extracting means for acquiring the coordinates of the point, and among the feature points extracted by the feature point extracting means, a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane. A three-dimensional amount calculation means for calculating a three-dimensionality evaluation amount defined as an amount for evaluating the three-dimensionality of a certain three-dimensional evaluation point, and a three-dimensionality of an object to be photographed based on the three-dimensionality evaluation amount calculated by the three-dimensional amount calculation means To determine And a three-dimensional calculation unit that converts the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation points into a two-dimensional coordinate system, and the feature points in the three-dimensional coordinate system of each plane component point. The difference between the coordinates of the two-dimensional coordinate system and the coordinates converted into a two-dimensional coordinate system is calculated for each captured image, and the rank of the determination matrix, which is a matrix in which the differences calculated for each captured image are combined into one element, is represented by three-dimensionality. It is calculated as an evaluation amount, and the three-dimensionality determination means determines that the three-dimensionality evaluation point is not on a plane when the rank of the determination matrix is 3.

  The input unit may acquire a captured image in which a person's face is captured as a capturing target. According to such a configuration, it is possible to determine the three-dimensionality of a person's face, and the photographed image is an image in which a person's face photograph is captured as a subject, or a real three-dimensional face is captured as a subject. It can be determined whether the image.

  The feature point extraction means is a feature point corresponding to any four or more points of the center of the left eye of the person's face, the center of the right eye, the left end of the lips, the right end of the lips, the top of the nose, the bottom of the nose, and the top of the lips May be extracted. According to such a configuration, the three-dimensionality can be determined without changing the facial expression.

The present invention relates to a three-dimensional authentication program installed in a computer for determining the three-dimensionality of an object to be imaged, and a plurality of images captured by a single image capturing device from a plurality of viewpoints with different object to be imaged. Extracting feature points corresponding to four or more points on the object to be photographed that are assumed not to be on the same plane in the three-dimensional coordinate system from the photographed image obtained in the input process A feature point extraction process for obtaining the coordinates of each feature point, and out of the plane constituted by the plane constituent points selected as the feature points constituting the plane among the feature points extracted in the feature point extraction process Shooting based on the three-dimensionality calculation processing for calculating the three-dimensionality evaluation amount defined as the amount to evaluate the three-dimensionality of the three-dimensional evaluation point, which is a feature point, and the three-dimensionality evaluation amount calculated in the three-dimensional calculation processing And a three-dimensionality determination process for determining the three-dimensionality of the figurine, and the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensionality evaluation point converted into the two-dimensional coordinate system in the three-dimensional amount calculation process, and each plane A matrix in which the difference between the coordinate of the feature point in the three-dimensional coordinate system of the component point and the coordinate converted into the two-dimensional coordinate system is calculated for each captured image, and the difference calculated for each captured image is combined into one matrix the rank of the judgment matrix is then calculated as a stereoscopic evaluation quantity, with the three-dimensional determination process, when the rank of the judgment matrix is 3, and characterized in that the three-dimensional evaluation point is determined not on a plane To do.

  A preferable aspect of the three-dimensional authentication method according to the present invention is, for example, that a plurality of face images with different postures of the authenticator are captured by a single camera, and each of them is characterized by three or more feature points and three-dimensional space. One or more feature points that are not on a plane composed of three points are acquired. Since normal stereo vision measures distance and shape, use a plurality of cameras with known camera positional relationships as described in Patent Document 2, or move a single camera while controlling the position. Need to image. However, it is not necessary to measure the exact distance and shape for the determination of the three-dimensionality, and it is sufficient if the flatness can be determined. Whether it is a face on an artificial object or a real face. In particular, in the face, the plane created by the center of both eyes and the left and right ends of the lips, which is relatively easy to extract, and the top of the nose, the lower point of the nose, and the central point of the upper end of the upper lip are obvious on the real face. It is only necessary to detect this because of the three-dimensionality. There is no camera with multiple positional relationships or a device that controls the position of one camera. Even in stereo viewing with one uncalibrated camera, it is composed of three points: the center of both eyes and the left and right ends of the lips. One point of three-dimensionality due to the apex of the nose with respect to the plane can be detected by the above three-dimensionality evaluation amount, and this is utilized. As described above, the object of the present invention can be achieved.

  The first effect of the present invention is to detect and prevent unauthorized authentication using a photograph during personal authentication using a face image. This is because the three-dimensionality of the face in the captured image is calculated at the time of authentication.

  The second effect of the present invention is to detect and prevent unauthorized authentication using a photograph during personal authentication using a face image on a small device such as a portable terminal. The reason is that fraud authentication is detected and prevented only with a face image taken with a small camera attached to a portable terminal or the like.

  The third effect of the present invention is that it is possible to perform face image authentication in which unauthorized authentication by a photograph is detected and prevented even in places where surrounding eyes are concerned. The reason is that, during authentication, the three-dimensionality of the face in the photographed image is calculated, and a face motion that changes the facial expression such as eyes and mouth is not required.

  The fourth effect of the present invention is to detect and prevent fraud authentication using photographs at the time of personal authentication using face images at low cost. The reason is that it can be realized by a single camera and does not require a stereo camera, a camera control device, a light irradiator, a distance sensor, or the like.

  A fifth effect of the present invention is to detect and prevent unauthorized authentication by a photo at the time of personal authentication while reducing calculation processing at the time of face authentication. The reason is that the registered image may be a single front face, and there is no need to use a side face image that makes face authentication difficult. Furthermore, it is not necessary to detect movements that are difficult to detect depending on the person such as the movements of the eyes and mouth.

Embodiment 1 FIG.
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of a three-dimensional authentication apparatus according to the present invention.

  The three-dimensional authentication apparatus shown in FIG. 1 captures and accumulates face image groups S10a and S10b having a plurality of postures, and specific feature points S20a associated between the images in each of the face images S10a and S10b. , S20b, the feature point extraction means 20, the feature quantity coordinate means S20a, S20b, the feature quantity extraction means 30 that computes the feature value S30 of the feature point outside the plane, and the feature value S30. And a three-dimensionality determining means 40 for determining whether or not the unauthorized authentication is performed by an artificial object such as a photograph.

  The imaging unit 10 captures and accumulates a plurality of face images S10a and S10b having different postures. That is, the imaging unit 10 captures a plurality of face images having different viewpoints and compositions (postures) at the time of imaging. The imaging means 10 is realized by a normal camera. A small one attached to the portable terminal is also possible. The camera may be a still camera for still images or a video camera for moving images. When the imaging unit 10 is a video camera for moving images, the imaging unit 10 needs to extract a plurality of frames having different face postures from the captured moving image as a face image.

  The feature point extraction unit 20 extracts feature points in the face from each of the face images S10a and S10b having a plurality of postures imaged by the imaging unit 10. The feature points may be extracted manually, but it is preferable to automatically extract the feature points using a feature extraction filter or the like in consideration of application.

  In the present invention, feature points are used as corresponding points in stereo vision, so correspondence between images is more important than accurate feature position acquisition. As feature points to be extracted, arbitrary three feature points (hereinafter referred to as plane composing points) and out-of-plane points (hereinafter referred to as three-dimensional evaluation points) constituted by the three points in the real face. ) Is required at least one point. That is, the three-dimensional evaluation point is a point corresponding to a point on the real face in the three-dimensional world and corresponding to a point that is not on the plane composed of the three feature points extracted in the three-dimensional world. It is.

  Theoretically, there are an infinite number of combinations of three arbitrary feature points and three-dimensionality evaluation points. However, when feature points are automatically extracted using a feature extraction filter or the like, the points to be extracted are limited. FIG. 2 is an explanatory diagram illustrating an example of feature points extracted by the feature point extraction unit 20. As illustrated in FIG. 2, as a feature point candidate that can be easily extracted by automatic extraction, as a plane composing point, for example, any one of a total of four points including the center of the right eye, the center of the left eye, the right end of the lips, and the left end of the lips. Three points are given. Examples of the three-dimensionality evaluation point include the apex of the nose, the bottom of the nose, and the center of the upper end of the lips.

  In the case of a real human face, the latter three-dimensionality is recognized with respect to the former plane. That is, if the face images S10a and S10b are not images in which face photographs are taken as subjects, but are images in which a real three-dimensional face is taken as a subject, the three-dimensionality evaluation extracted by the feature point extraction means 20 The point is calculated not to be a point on a plane constituted by three plane composing points.

The three-dimensional amount calculation unit 30 calculates the three-dimensionality evaluation amount S30 of the three-dimensionality evaluation point from the feature point coordinates S20a and S20b of each image extracted by the feature point extraction unit 20. As shown in FIG. 2, using the right eye center (P ₀ ), the left eye center (P ₁ ), and the right lip edge (P ₂ ) as the plane composing points, the three points (for example, the nose) as the three-dimensionality evaluation points P are used. In the following, an example will be described in which images of the face postures 1 and 2 (images 1 and 2) are used.

  In the following description, a case where the camera model (perspective projection model) can be approximated by a general pinhole camera model in stereo view will be exemplified. FIG. 3 is an explanatory diagram for explaining an example of a pinhole camera model. The pinhole camera model is a general camera model in stereo view, and when an imaging target in a three-dimensional space is projected onto a two-dimensional image, the positional relationship between the imaging target and image formation on the image is simplified. This is an ideal model. A pinhole camera is a camera in which only light incident through a pinhole (a small hole opened by a pin) reaches the imaging surface. In a pinhole camera, the pinhole can be considered as the focal point. In the pinhole camera, since the image projected on the imaging surface is upside down, a model in which the imaging surface is moved between the pinhole and the imaging target is used as shown in FIG.

As shown in FIG. 3, a three-dimensional coordinate system (X, Y, Z) having a pinhole as an origin is taken, and a two-dimensional image coordinate system (u, v) is taken on the imaging surface. The coordinates of the plane composing point P _i (i = 0, 1, 2) in the three-dimensional space are P _i (X _i , Y _i , Z _i ), and the coordinates of the three-dimensional evaluation point P in the three-dimensional space are P (X, Y, Z).

Also, the coordinates in the image of the posture j (j = 1, 2) of P _i (i = 0, 1, 2) are (u _i ^j , v _i ^j ), and the posture j of P (j = 1, 2). ) In the image is (u ^j , v ^j ). That is, it is assumed that P _i (X _i , Y _i , Z _i ) in the three-dimensional coordinate system is mapped to (u _i ^j , v _i ^j ) in the image coordinate system. FIG. 3 illustrates an imaging surface of posture j (j = 1, 2), P (X, Y, Z) is mapped to (u ^j , v ^j ), and P ₀ (X ₀ , Y _{The case where 0} , Z ₀ ) is mapped to (u ₀ ^j , v ₀ ^j ) is illustrated.

  When the focal length when the camera is assumed to be a pinhole camera is f, the following formula 1 is established. T means transposition. The focal length f is a distance from the pinhole to the imaging surface.

In Expression (1), ξ _i ^j = Z _i ^j / f, and in Expression (2), ξ ^j = Z ^j / f. The transformations shown in equations (1) and (2) are called perspective projection (perspective transformation). ξ ₁ ^j and ξ ₂ ^j are magnification values in perspective projection.

Here, when the three-dimensionality evaluation point P is on the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ , A and B are real numbers.

This is a linearly dependent relationship. Therefore, the expressions (1) and (2) are substituted into the expression (3), and S ₁ ^j = ξ ₁ ^j / ξ ₀ ^j , S ₂ ^j = ξ ₂ ^j / ξ ₀ ^j , S ^j = ξ ^j / ξ ^Assuming ₀ ^j , the following equation (4) is obtained.

  Expanding the third line of equation (4),

  Therefore, if the real number C is defined as C = 1−A−B, and the formula (5) is substituted into the first and second rows of the formula (4) and rearranged, the first and second rows of the formula (4) about,

  Is established. When the formula (6) calculated from the postures 1 and 2 (j = 1, 2) is put together, the following formula (7) is obtained.

  The matrix in equation (7) is represented by equation (8), where D is D.

Since the vector [AS ₁ ^j , BS ₂ ^j , C] ^T in equation (7) cannot be a 0 vector from C = 1−A−B, the rank (rank) of the matrix D shown in equation (8) ) Becomes 2 for the following reason.

The matrix D is a mapping acting on the three-dimensional space V. When the point P is on the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ , the third dimension C of the vector [AS ₁ ^j , BS ₂ ^j , C] ^T is 1-A-B. As a result, the dimension of ker (D), which is the core of the matrix D, is dim (ker (D)) = 1. Also, dimV = 3. Dimensional theorem

  dimV = dim (ker (D)) + dim (D (V)) Equation (9)

  , Dim (D (V)), that is, the rank of the matrix D is 2.

Next, the rank of the matrix D in the case where the three-dimensionality evaluation point P is not on the plane formed by the plane configuration points P ₀ , P ₁ , P ₂ is obtained. It can also be determined from the rank of the matrix D that the point P is not on the plane. In this case, since there is no constraint in the third dimension C of the vector [AS ₁ ^j , BS ₂ ^j , C] ^T , dim (ker (D )) = 0, and dim (D (V)), that is, the rank of the matrix D is 3, from the dimensional theorem shown in Equation (9).

As described above, an amount indicating the rank of the matrix D may be adopted as the three-dimensional evaluation amount S30 for evaluating the three-dimensionality of the three-dimensional evaluation point. Examples of candidates for the quantity indicating the rank of the matrix D include, for example, the smallest singular value of the matrix D, the smallest eigenvalue of the matrix D ^T D (matches the square of the smallest singular value of D), and 0 of the characteristic polynomial of the matrix D ^T D There are fluctuations in nearby values.

Looking at the elements of the matrix D, S ₁ ² / S ₁ ¹ and S ₂ ² / S ₂ ¹ are usually unknown. However, ξ ₁ ^j and ξ ₂ ^j are magnification values obtained by perspective projection, and S ₁ ^j = ξ ₁ ^j / ξ ₀ ^j , S ₂ ^j = ξ ₂ ^j / ξ ₀ ^j is the value between plane composing points. Is the ratio. That is, S ₁ ^j is a ratio between ξ ₁ ^j and ξ ₀ ^j, and S ₂ ^j is a ratio between ξ ₂ ^j and ξ ₀ ^j . Since ξ ₁ ^j and ξ ₂ ^j are magnification values obtained by perspective projection, S ₁ ^j is a ratio between Z ₁ and Z _0, and S ₂ ^j is a ratio between Z ₂ and Z ₀ . Therefore, the unknown quantities S ₁ ² / S ₁ ¹ and S ₂ ² / S ₂ ¹ are values in the vicinity of 1 in the relationship between the plane plane composing point and the imaging surface of the camera, As long as the three-dimensional evaluation point that shows the three-dimensionality around the center of the upper edge of the lips under the nose is used, it can be regarded as 1.

As a result, when the minimum singular value of the matrix D and the minimum eigenvalue of the matrix D ^T D (matching the square of the minimum singular value of D) are used for the three-dimensionality evaluation amount S30, a simple coordinate difference is used as an element. It can be obtained by calculating singular values and eigenvalues of the matrix D.

When the value near 0 of the characteristic polynomial of the matrix D ^T D is used for the three-dimensionality evaluation amount S30, it is possible to perform the calculation only by integer arithmetic, which is effective when it is desired to reduce the calculation amount.

Hereinafter, as a stereoscopic evaluation amount S30, will be described that may be a value near 0 in the smallest eigenvalue and the characteristic polynomial of the matrix D ^{T D} of the matrix D ^{T D.} The characteristic polynomial of the matrix D ^T D is f (x), and the solution of f (x) = 0 is λ. λ is an eigenvalue of the matrix D ^T D. Note that the value near 0 in the characteristic polynomial of the matrix D ^T D is the value of f (x) near x = 0.

FIG. 4 is an explanatory diagram illustrating an example of the characteristic polynomial f (x) of the matrix D ^TD . In FIG. 4, the value of f (x) when the three-dimensional evaluation point P is in the plane formed by the plane constituent points P ₀ , P ₁ and P ₂ is represented by a solid line, and the three-dimensional evaluation point P is the nose. The value of f (x) when the point is the center of the lower or upper lip upper end is represented by a broken line, and the value of f (x) when the three-dimensional evaluation point P is a point on the nose is represented by a dashed line. ing.

Since the matrix D ^T D is a semi-positive target matrix, all eigenvalues do not take negative values. Therefore, since the minimum eigenvalue of the matrix D ^T D takes a value of 0 or more, f (x) takes a value as shown in FIG. 4, for example. Here, the minimum eigenvalue of the matrix D ^T D when the three-dimensional evaluation point P is in the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ is λ ₁ , and the three-dimensional evaluation point P is Let λ _{2 be} the minimum eigenvalue of the matrix D ^T D when it is the center of the upper lip upper end, and let λ _{3 be} the minimum eigenvalue of the matrix D ^T D when the three-dimensionality evaluation point P is a point on the nose. λ ₁ , λ ₂ , and λ ₃ are values of x at the point where f (x) intersects the horizontal axis (x-axis) in FIG.

Considering that the eigenvalue of the matrix D ^T D, that is, λ, which is a solution of f (x) = 0, coincides with the square of the singular value of D, the three-dimensional evaluation point P becomes the plane constituent point P ₀ , P _1. , P ₂ , the minimum eigenvalue λ ₁ is a value near zero. Further, when the three-dimensionality evaluation point P is outside the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ , the minimum eigenvalues λ ₂ , λ ₃ become values larger than 0 to some extent. That is, based on the minimum eigenvalue of the matrix D ^T D, it can be determined whether or not the three-dimensionality evaluation point P is within the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ .

  Further, when passing through the eigenvalue (when x = λ), the characteristic polynomial f (x) whose function value is 0 is, as shown in FIG. The behavior near x = 0 changes. That is, whether or not the sign of f (x) changes near x = 0 depends on whether or not the three-dimensional evaluation point P is in the plane. Therefore, if function values f (−ε) and f (ε) (where ε is a small positive integer) near x = 0 are calculated, the calculation can be performed only by integer arithmetic. The determination of the three-dimensionality is performed by the sign change of both values by the three-dimensionality determination means 40 to which these values are sent.

That is, by appropriately taking the value of ε, when f (ε) is positive and f (−ε) is a negative value, the three-dimensionality evaluation point P becomes the plane constituent points P ₀ and P _1. , P ₂ . When both f (ε) and f (−ε) are negative values, it is determined that the three-dimensionality evaluation point P is not on the plane formed by the plane constituent points P ₀ , P ₁ , P _2. Can do.

  Based on the three-dimensional evaluation amount S30 calculated by the three-dimensional calculation unit 30 and a predetermined threshold value, the three-dimensionality determination unit 40 determines the three-dimensionality of the three-dimensional evaluation point, that is, the face on the artifact by a photograph or the like. Whether it is a real face or not.

  The determination method differs depending on the three-dimensionality evaluation amount S30 calculated by the three-dimensional amount calculation means 30. When the three-dimensionality evaluation amount S30 is the minimum singular value of D, if the absolute value of the value is less than a certain value (threshold value), the three-dimensionality evaluation amount S30 is regarded as a planar object, and the absolute value is constant. If it is more than that, it is regarded as a three-dimensional object and is determined to be real.

If the three-dimensionality evaluation amount S30 is the minimum eigenvalue of the matrix D ^T D (matches the square of the minimum singular value of D), if the value is less than a certain value, it is considered to be a plane and is determined to be an artifact. If it is equal to or greater than a certain value, it is regarded as a three-dimensional object and is determined to be real.

In the example illustrated in FIG. 4, when the threshold is ε, for example, if the minimum eigenvalue of the matrix D ^T D is λ ₁ , λ ₁ is a value less than ε. It is determined that it is in the plane formed by the plane composing points P ₀ , P ₁ , P ₂ . Therefore, the face image is determined to be an image obtained by capturing a face photograph as a subject. Further, for example, taking the minimum eigenvalue lambda ₂ values of the matrix D ^{T D,} lambda ₂ is because it is a value greater than epsilon, stereo evaluation point P formed by the planar configuration point P _0, P _1, P ₂ It is determined that it is not on the plane. Therefore, the face image is determined to be an image obtained by capturing an actual three-dimensional face as a subject.

When the three-dimensionality evaluation amount S30 is a fluctuation of a value near 0 of the characteristic polynomial of the matrix D ^T D, the characteristic equation f (x) has different values as shown in FIG. Therefore, if there is a change in the sign of the function values f (−ε) and f (ε) near x = 0, it is determined as a planar object, and if not, it is determined as a three-dimensional object and determined as a real object.

In the example shown in FIG. 4, for example, f (−ε), which is a value near x = 0, of the characteristic polynomial f (x) of the matrix D ^T D takes a negative value, and f (ε) is a positive value. If it is taken, the three-dimensional evaluation point P is determined to be in the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ . Therefore, the face image is determined to be an image obtained by capturing a face photograph as a subject. For example, if f (−ε) takes a negative value and f (ε) also takes a negative value, the three-dimensionality evaluation point P is on the plane formed by the plane constituent points P ₀ , P ₁ , P _2. It is judged that it is not. Therefore, the face image is determined to be an image obtained by capturing an actual three-dimensional face as a subject.

  As described above, according to the first embodiment, it is possible to determine whether or not the subject is a facial photograph by calculating the three-dimensionality of the subject of the captured image. The unauthorized authentication used can be detected.

Embodiment 2. FIG.
Next, a second embodiment of the present invention will be described with reference to the drawings. In the second embodiment, a three-dimensional authentication apparatus will be described in which each unit of the first embodiment performs each method in the three-dimensional authentication method. FIG. 5 is a block diagram showing a second embodiment of the three-dimensional authentication apparatus according to the present invention. 5 includes an imaging device 100, a feature point extraction device 200, a three-dimensional amount calculation device 300, and a three-dimensionality determination device 400.

  The imaging apparatus 100 may be a popular digital camera, a still camera for still images, or a video camera for moving images. In the case of a still camera, two or more images are taken when the face posture is different, and two or more images are sent to the feature point extraction apparatus 200. In the case of a video camera, the moving image during the face posture operation is sent to the feature point extraction device 200 as it is.

  FIG. 6 is an explanatory diagram illustrating an example of the feature point extraction apparatus 200. As shown in FIG. 6, the feature point extraction device 200 is a storage device such as a memory or a disk that stores image data captured by the imaging device 100, and an arithmetic device such as a CPU that performs calculation of feature point extraction in the image. And can be realized. When the imaging apparatus 100 is configured by a video camera, two or more frames having different face postures in a moving image are cut out as still images by calculation by the CPU. After two or more still images having different face postures, the feature point extraction means 20 of the first embodiment extracts the feature points, and the CPU performs calculation for feature point coordinate extraction. The feature point coordinate data is sent to the three-dimensional amount calculation device 300.

  FIG. 7 is an explanatory diagram illustrating an example of the three-dimensional amount calculation apparatus 300. As shown in FIG. 7, the three-dimensional amount calculation device 300 includes a storage device such as a memory or a disk that stores the feature point coordinate data extracted by the feature point extraction device 200, and an operation such as a CPU that calculates a three-dimensionality evaluation amount. It can be realized with a device. For the feature point coordinate data extracted from the two still images, the CPU calculates the three-dimensional evaluation amount by the method of calculating the three-dimensional evaluation amount by the three-dimensional amount calculation unit 30 of the first embodiment. After performing the above, data indicating the three-dimensionality evaluation amount is sent to the three-dimensionality determination device 400.

  The three-dimensionality determination device 400 can be realized by a simple logic circuit such as an LSI that determines whether the three-dimensionality evaluation amount calculated by the three-dimensional amount calculation device 300 is above or below a predetermined threshold value. When the function values f (−ε) and f (ε) in the vicinity of 0 are used as the three-dimensionality evaluation amount S30 in the first embodiment, no threshold value is required. Judge with.

Embodiment 3 FIG.
Next, a third embodiment of the present invention will be described with reference to the drawings. In an embodiment of the third invention, the feature point extraction method of the first embodiment is configured as a program executable on a computer, and the program is stored in an information storage medium readable by a computer. It is executed on the computer in the procedure shown in FIG.

  FIG. 8 is a flowchart showing the operation of the third embodiment. First, face images with different postures are photographed (step S100). In the procedure shown in FIG. 6, as a case where the minimum two postures are photographed, the face posture is changed (step S101), it is determined whether or not the image is taken twice (step S102), and when the image is taken twice. (Yes) When the process proceeds to step S200 and the image is not captured twice (No), the case is illustrated where the process proceeds to step S100.

  Next, in step S200, the coordinate value of the feature point is extracted from each of the two or more face images by the method using the feature point extraction unit 20 of the first embodiment. When the coordinate values of the feature points are extracted, the three-dimensionality evaluation amount is calculated by the method using the three-dimensional amount calculation unit 30 of the first embodiment using them (step S300). Finally, it is determined whether the face is an actual face or a face impersonated by an artificial object such as a photograph, based on the magnitude relationship between the three-dimensionality evaluation amount and a predetermined threshold (step S400). When the function values f (−ε) and f (ε) in the vicinity of 0 are used as the three-dimensionality evaluation amount S30 in the first embodiment, no threshold value is required. Judge with.

  In the first to third embodiments, the authentication of the three-dimensionality of the face image has been described. However, the similar embodiment can be used for discrimination between a photograph of a general scene and an actual scene for the purpose of monitoring or the like. Can also be done.

  According to the present invention, the present invention can be applied when detecting unauthorized authentication at the time of personal authentication using a face image. For example, it can be used for face image authentication in an unattended entrance / exit system or the like, and face image authentication in a small device such as a portable terminal.

It is a block diagram which shows 1st Embodiment of the three-dimensional authentication apparatus by this invention. It is explanatory drawing which shows the example of the feature point which a feature point extraction means extracts. It is explanatory drawing for demonstrating the example of a pinhole camera model. It is explanatory drawing which shows the example of characteristic equation f (x). It is a block diagram which shows 2nd Embodiment of the three-dimensional authentication apparatus by this invention. It is explanatory drawing which shows the example of a feature point extraction apparatus. It is explanatory drawing which shows the example of a solid amount calculation apparatus. It is a flowchart which shows operation | movement of 3rd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging means 20 Feature point extraction means 30 Three-dimensional amount calculation means 40 Three-dimensionality determination means

Claims (7)

A three-dimensional authentication method for determining the three-dimensionality of a shooting object,
An input step of acquiring a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired in the input step, and the coordinates of each feature point are obtained. A feature point extraction step to be acquired;
Among the feature points extracted in the feature point extraction step, the three-dimensionality of the three-dimensionality evaluation point that is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. A solid quantity calculation step for calculating a three-dimensionality evaluation quantity defined as a quantity;
A three-dimensionality determination step for determining the three-dimensionality of the object to be photographed based on the three-dimensionality evaluation amount calculated in the three-dimensional amount calculation step,
In the three-dimensional amount calculation step, the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point converted into the two-dimensional coordinate system, and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points are two-dimensional. The difference from the coordinate converted into the coordinate system is calculated for each captured image, and the rank of the determination matrix, which is a matrix in which the difference calculated for each captured image is combined into one element, is calculated as the stereoscopic evaluation amount. ,
In the three-dimensionality determination step, when the rank of the determination matrix is 3, it is determined that the three-dimensional evaluation point is not on a plane. The three-dimensionality authentication method according to claim 1, wherein in the input step, a captured image in which a person's face is captured as a capturing object is acquired. In the feature point extraction step, feature points corresponding to four or more points of the center of the left eye of the human face, the center of the right eye, the left end of the lips, the right end of the lips, the apex of the nose, the bottom of the nose, and the upper end of the lips The three-dimensionality authentication method according to claim 2, wherein: A three-dimensional authentication device for determining the three-dimensionality of a photographing object,
Input means for acquiring a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired by the input means, and the coordinates of the feature points are extracted. A feature point extraction means to obtain;
Among the feature points extracted by the feature point extracting means, the three-dimensionality of the three-dimensionality evaluation point which is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. A solid amount calculation means for calculating a three-dimensionality evaluation amount defined as a quantity;
Three-dimensionality determination means for determining the three-dimensionality of the object to be photographed based on the three-dimensionality evaluation amount calculated by the three-dimensional amount calculation means;
The three-dimensional amount calculation means is configured to two-dimensionally convert the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point into the two-dimensional coordinate system and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points. The difference from the coordinate converted into the coordinate system is calculated for each captured image, and the rank of the determination matrix, which is a matrix in which the difference calculated for each captured image is combined into one element, is calculated as the stereoscopic evaluation amount. ,
The three-dimensionality determination unit determines that the three-dimensional evaluation point is not on a plane when the rank of the determination matrix is three. The three-dimensional authentication apparatus according to claim 4 , wherein the input unit acquires a captured image in which a person's face is captured as an object to be captured. The feature point extraction means is a feature point corresponding to any four or more points of the center of the left eye of the person's face, the center of the right eye, the left end of the lips, the right end of the lips, the top of the nose, the bottom of the nose, and the top of the lips The three-dimensional authentication device according to claim 5 , wherein the three-dimensional authentication device is extracted. A three-dimensional authentication program installed in a computer for determining the three-dimensionality of a shooting object,
In the computer,
An input process for acquiring a plurality of captured images captured by a single imaging device from a plurality of viewpoints having different imaging objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired in the input process, and the coordinates of each feature point are obtained. A feature point extraction process to be acquired;
Among the feature points extracted in the feature point extraction process, the three-dimensionality of the three-dimensionality evaluation point which is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. Solid amount calculation processing for calculating a three-dimensionality evaluation amount defined as a quantity;
A three-dimensionality determination process for determining the three-dimensionality of the object to be photographed based on the three-dimensionality evaluation amount calculated in the three-dimensional amount calculation process;
In the three-dimensional amount calculation process, the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point converted into the two-dimensional coordinate system, and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points are two-dimensional. The difference between the coordinates converted into the coordinate system is calculated for each captured image, and the rank of the determination matrix, which is a matrix in which the differences calculated for each captured image are combined into one element, is calculated as the three-dimensionality evaluation amount. ,
In the three-dimensionality determination process, when the rank of the determination matrix is 3, the three-dimensionality authentication program for determining that the three-dimensionality evaluation point is not on a plane.

Images