JP5035467B2 - Three-dimensional authentication method, three-dimensional authentication device, and three-dimensional authentication program - Google Patents

Description

  The present invention relates to a three-dimensional authentication method, a three-dimensional authentication apparatus, and a three-dimensional authentication program for detecting fraud authentication during personal authentication using a face image.

  As a problem at the time of personal authentication using a face image, there is an unauthorized authentication using cracking. Unauthorized authentication by cracking means that a person other than the person “spoofs” the person by some means and passes through the authentication system. As a specific method, a photograph taken by the person is often used because of the ease of execution. Impersonation using a photo can be used by printing on a medium such as paper or using it on a display device such as a monitor.

  The face on the display object is a plane in the three-dimensional world. That is, in the three-dimensional world that is the real world, the face displayed on paper or a monitor is two-dimensional information (plane). Also, in the real world, there are movements that can be used. Japanese Patent Application Laid-Open No. 2004-151561 describes one method of determining based on the movement of a partial area of a face. In Patent Document 1, attention is paid to the opening and closing of eyes and mouths. This method is one countermeasure because the eyes and mouth are easy to find among the facial features and easily move.

  As another approach, there is a method using the fact that the face on the display object is a plane in the three-dimensional world, whereas the real face is three-dimensional.

  Patent Document 2 describes a method based on this policy, a method of irradiating light, a method of stereo vision, and a method of registering a plurality of face images during specific operations. Japanese Patent Application Laid-Open No. H10-228561 discloses a method of using a distance measurement sensor more directly.

  In the method of irradiating light, the shadow generated from the fact that the face has a three-dimensional shape is used. The method using stereo vision is triangulation using each of images taken by a plurality of cameras whose positional relationships are known, or using a plurality of images taken from different viewpoints by one camera whose position is controlled. This is a method for measuring a three-dimensional shape based on the principle.

  In the method of registering multiple face images for specific actions, if images of multiple postures are registered at the time of registration and a front image and a certain or more posture change image are used at the time of authentication, impersonation by a plane object such as a photo is performed. It is easy to eliminate. The distance measuring sensor includes a sensor that irradiates pattern light and measures the image, and in principle, is close to a combination of a light irradiation method and a stereo method.

JP 2002-251614 A (paragraphs 0045-0055) JP 2003-178306 A (paragraphs 0015, 0039, 0055) Japanese Patent Laying-Open No. 2005-135271 (paragraphs 0022-0026)

  However, the method described in Patent Document 1 has the following problems. The movement of the kite is small and difficult to find depending on the person or the captured image. The movement of the eyes and mouth greatly changes the facial expression of the person, so there is also a psychological resistance that is difficult to do where the surrounding eyes are concerned. Difficult to implement.

  In addition, the method of irradiating light, which is the method described in Patent Document 2, is easily affected by illumination conditions, so the place of use is limited and it is difficult to expect stability of performance. In the stereo method, the problem of shape measurement by stereo view is applied as it is, and a camera with multiple positional relationships or a device for controlling the position of one camera is required, and the use place and mounting conditions are limited. The

  In the method of registering face images at a plurality of specific operations, which is a method described in Patent Document 3, registration at a plurality of specific operations is troublesome, and there is a problem that face authentication other than the front is difficult. Although the method using the distance measuring sensor is reliable, it is expensive and difficult to mount on a small device such as a portable terminal.

  One of the objects of the present invention is to provide a three-dimensional authentication method, a three-dimensional authentication device, and a three-dimensional authentication program for preventing unauthorized authentication by a photograph or the like in face authentication.

  One of the objects of the present invention is to provide a three-dimensional authentication method, a three-dimensional authentication device, and a three-dimensional authentication program for preventing unauthorized authentication by a photograph or the like in face authentication in a small device such as a portable terminal.

  One of the objects of the present invention is to provide a three-dimensional authentication method, a three-dimensional authentication device, and a three-dimensional authentication program that prevent unauthorized authentication by a photograph or the like in face authentication using only a single camera.

The three-dimensional authentication method according to the present invention is a three-dimensional authentication method for determining the three-dimensionality of a photographing object, and acquires a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects. And feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired in the input step, A feature point extraction step for acquiring the coordinates of the point, and among the feature points extracted in the feature point extraction step, a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane A solid amount calculation step for calculating a three-dimensionality evaluation amount defined as an amount for evaluating the three-dimensionality of a certain three-dimensional evaluation point, and a photographing pair based on the three-dimensionality evaluation amount calculated in the three-dimensional amount calculation step. Anda steric determination step of determining three-dimensional properties of the object, in the three-dimensional volume calculation step, a coordinate obtained by converting the coordinates of the feature points in the two-dimensional coordinate system in the three-dimensional coordinate system of the three-dimensional evaluation points, each planar configuration The minimum of the determination matrix, which is a difference between coordinates of feature points in the three-dimensional coordinate system of the points converted into coordinates in the two-dimensional coordinate system, and is a matrix in which differences calculated for each captured image are combined into one element A singular value is calculated as a three-dimensionality evaluation amount, and in the three-dimensionality determination step, when the absolute value of the minimum singular value is equal to or greater than a threshold value, it is determined that the three-dimensionality evaluation point is not on a plane.

  According to such a configuration, it is possible to easily calculate the minimum singular value based on the matrix having the coordinate difference as an element, and determine the three-dimensionality.

Another three-dimensional authentication method according to the present invention is a three-dimensional authentication method for determining the three-dimensionality of a photographing object, and a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints with different photographing objects. And a feature point corresponding to four or more points on the object to be photographed that are not in the same plane in the three-dimensional coordinate system are extracted from the photographed image obtained in the input step. A feature point extracting step for obtaining the coordinates of each feature point, and out of the plane constituted by the plane constituent points selected as the feature points constituting the plane among the feature points extracted in the feature point extracting step A three-dimensional amount calculation step for calculating a three-dimensionality evaluation amount defined as an amount for evaluating the three-dimensionality of the three-dimensionality evaluation point that is a feature point of the three-dimensionality evaluation point, and the three-dimensionality evaluation amount calculated in the three-dimensional amount calculation step Anda steric determination step of determining three-dimensional properties of the object to be shot by Zui and, in the three-dimensional volume calculation step, by converting the coordinates of the feature points in 3-dimensional coordinate system of the three-dimensional evaluation points in a two-dimensional coordinate system A matrix which is a difference between coordinates and coordinates obtained by converting the coordinates of feature points in the three-dimensional coordinate system of each plane composing point into a two-dimensional coordinate system, and a difference calculated for each photographed image as one element. When the minimum eigenvalue of a regular matrix that is the product of the transposed matrix of the determination matrix and the determination matrix is calculated as a stereoscopicity evaluation amount, and the minimum eigenvalue is equal to or greater than a threshold in the stereoscopicity determination step, It is determined that the three-dimensional evaluation point is not on a plane. According to such a configuration, it is possible to easily calculate the minimum eigenvalue based on the matrix having the coordinate difference as an element and the transposed matrix to determine the three-dimensionality.

Still another three-dimensional authentication method according to the present invention is a three-dimensional authentication method for determining the three-dimensionality of a subject to be photographed, and a plurality of photographings taken by a single photographing device from a plurality of viewpoints with different photographing subjects. An input step for acquiring an image and feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the captured image acquired in the input step. A feature point extracting step for obtaining the coordinates of each feature point, and the plane constituted by the plane constituent points selected as the feature points constituting the plane among the feature points extracted in the feature point extracting step. A three-dimensional amount calculation step for calculating a three-dimensionality evaluation amount defined as an amount for evaluating the three-dimensionality of the three-dimensionality evaluation point which is an external feature point; and the three-dimensionality evaluation calculated in the three-dimensional amount calculation step. Anda steric determination step of determining three-dimensional properties of the object to be shot on the basis of the amount, converted in the three-dimensional volume calculation step, the coordinates of the feature points in 3-dimensional coordinate system of the three-dimensional evaluation points in a two-dimensional coordinate system The difference between the calculated coordinates and the coordinates obtained by converting the coordinates of the feature points in the three-dimensional coordinate system of each plane composing point into a two-dimensional coordinate system, and the differences calculated for each captured image are combined into one element . the characteristic polynomial of the regular matrix is the product of the matrix and is determined for matrix transpose and the determination matrix, calculated code different 0 a value obtained by substituting the 2 values of neighboring as each three-dimensional evaluation amount, the stereoscopic In the sex determination step, when the sign does not change in the two three-dimensionality evaluation quantities, it is determined that the three-dimensionality evaluation point is not on a plane. According to such a configuration, the three-dimensionality evaluation amount can be calculated with a small amount of calculation, based on the matrix having the coordinate difference as an element and the transposed matrix.

The three-dimensional authentication apparatus according to the present invention is a three-dimensional authentication apparatus for determining the three-dimensionality of a photographing object, and acquires a plurality of photographed images photographed by a single photographing apparatus from a plurality of viewpoints having different photographing objects. And feature points corresponding to four or more points on the object to be photographed that are not in the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired by the input means, A feature point extracting means for acquiring the coordinates of the point, and among the feature points extracted by the feature point extracting means, a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane. A three-dimensional amount calculation means for calculating a three-dimensionality evaluation amount defined as an amount for evaluating the three-dimensionality of a certain three-dimensional evaluation point, and a three-dimensionality of an object to be photographed based on the three-dimensionality evaluation amount calculated by the three-dimensional amount calculation means To determine It includes and sex determination means, a solid amount calculation means, the coordinates obtained by converting the coordinates of the feature points in the two-dimensional coordinate system in the three-dimensional coordinate system of the three-dimensional evaluation points, characterized in a three-dimensional coordinate system of the planar configuration point The minimum singular value of the determination matrix, which is a difference between the coordinates of the points converted into the two-dimensional coordinate system and the difference calculated for each captured image as one element, is used as the stereoscopic evaluation amount. The three-dimensionality determination means calculates and determines that the three-dimensionality evaluation point is not on a plane when the absolute value of the minimum singular value is equal to or greater than a threshold value.

  According to such a configuration, it is possible to easily calculate the minimum singular value based on the matrix having the coordinate difference as an element, and determine the three-dimensionality.

Another three-dimensional authentication apparatus according to the present invention is a three-dimensional authentication apparatus for determining the three-dimensionality of a photographing object, and a plurality of photographed images photographed by a single photographing apparatus from a plurality of viewpoints with different photographing objects. And feature points corresponding to four or more points on the object to be photographed that are not in the same plane in the three-dimensional coordinate system are extracted from the photographed image obtained by the input means. A feature point extracting means for acquiring the coordinates of each feature point; and out of the plane constituted by the plane constituting points selected as the feature points constituting the plane among the feature points extracted by the feature point extracting means. A three-dimensional amount calculation means for calculating a three-dimensionality evaluation amount defined as an amount for evaluating the three-dimensionality of the three-dimensionality evaluation point that is a feature point of the three-dimensionality, and photographing based on the three-dimensionality evaluation amount calculated by the three-dimensional amount calculation means Standing object And a three-dimensional determination means for determining sex, the stereoscopic amount calculation means, the coordinates obtained by converting the coordinates of the feature points in the two-dimensional coordinate system in the three-dimensional coordinate system of the three-dimensional evaluation points, each planar configuration point The transpose matrix of the determination matrix, which is a difference between the coordinates of the feature points in the three-dimensional coordinate system and the coordinates converted into the two-dimensional coordinate system, and the differences calculated for each captured image as one element. A minimum eigenvalue of a regular matrix, which is the product of the matrix and the determination matrix, is calculated as a three-dimensionality evaluation amount, and when the three-dimensionality determination unit has the minimum eigenvalue equal to or greater than a threshold value, the three-dimensionality evaluation point is It is characterized in that it is determined not to be present. According to such a configuration, it is possible to easily calculate the minimum eigenvalue based on the matrix having the coordinate difference as an element and the transposed matrix to determine the three-dimensionality.

Still another three-dimensionality authentication device according to the present invention is a three-dimensionality authentication device for determining the three-dimensionality of an object to be imaged, and a plurality of images taken by a single image-taking device from a plurality of viewpoints where the object to be imaged is different. Extracting feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system from the photographed image obtained by the input means And a feature point extracting means for obtaining the coordinates of each feature point, and the plane constituted by the plane constituent points selected as the feature points constituting the plane among the feature points extracted by the feature point extracting means. Based on the three-dimensional amount calculation means for calculating the three-dimensionality evaluation amount defined as the amount for evaluating the three-dimensionality of the three-dimensional evaluation point, which is an external feature point, and the three-dimensionality evaluation amount calculated by the three-dimensional amount calculation means Shooting target And a three-dimensional determination means for determining the three-dimensional properties of the solid amount calculation means, the coordinates obtained by converting the coordinates of the feature points in the two-dimensional coordinate system in the three-dimensional coordinate system of the three-dimensional evaluation point, each plane A determination matrix that is a difference between coordinates of feature points in the three-dimensional coordinate system of the component points and converted into two-dimensional coordinate systems, and is a matrix in which differences calculated for each captured image are combined into one element. the characteristic polynomial of the regular matrix is the product of the transposed matrix and the determination matrix, codes of different 0 a value obtained by substituting the 2 values of the neighboring calculated as each three-dimensional evaluation amount, is the three-dimensional determination device, said 2 When the sign does not change in one three-dimensionality evaluation amount, it is determined that the three-dimensionality evaluation point is not on a plane. According to such a configuration, the three-dimensionality evaluation amount can be calculated with a small amount of calculation, based on the matrix having the coordinate difference as an element and the transposed matrix.

  The first effect of the present invention is to detect and prevent unauthorized authentication using a photograph during personal authentication using a face image. This is because the three-dimensionality of the face in the captured image is calculated at the time of authentication.

  The second effect of the present invention is to detect and prevent unauthorized authentication using a photograph during personal authentication using a face image on a small device such as a portable terminal. The reason is that fraud authentication is detected and prevented only with a face image taken with a small camera attached to a portable terminal or the like.

  The third effect of the present invention is that it is possible to perform face image authentication in which unauthorized authentication by a photograph is detected and prevented even in places where surrounding eyes are concerned. The reason is that, during authentication, the three-dimensionality of the face in the photographed image is calculated, and a face motion that changes the facial expression such as eyes and mouth is not required.

  The fourth effect of the present invention is to detect and prevent fraud authentication using photographs at the time of personal authentication using face images at low cost. The reason is that it can be realized by a single camera and does not require a stereo camera, a camera control device, a light irradiator, a distance sensor, or the like.

  A fifth effect of the present invention is to detect and prevent unauthorized authentication by a photo at the time of personal authentication while reducing calculation processing at the time of face authentication. The reason is that the registered image may be a single front face, and there is no need to use a side face image that makes face authentication difficult. Furthermore, it is not necessary to detect movements that are difficult to detect depending on the person such as the movements of the eyes and mouth.

It is a block diagram which shows 1st Embodiment of the three-dimensional authentication apparatus by this invention. It is explanatory drawing which shows the example of the feature point which a feature point extraction means extracts. It is explanatory drawing for demonstrating the example of a pinhole camera model. It is explanatory drawing which shows the example of characteristic equation f (x). It is a block diagram which shows 2nd Embodiment of the three-dimensional authentication apparatus by this invention. It is explanatory drawing which shows the example of a feature point extraction apparatus. It is explanatory drawing which shows the example of a solid amount calculation apparatus. It is a flowchart which shows operation | movement of 3rd Embodiment.

Embodiment 1 FIG.
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of a three-dimensional authentication apparatus according to the present invention.

  The three-dimensional authentication apparatus shown in FIG. 1 captures and accumulates face image groups S10a and S10b having a plurality of postures, and specific feature points S20a associated between the images in each of the face images S10a and S10b. , S20b, the feature point extraction means 20, the feature quantity coordinate means S20a, S20b, the feature quantity extraction means 30 that computes the feature value S30 of the feature point outside the plane, and the feature value S30. And a three-dimensionality determining means 40 for determining whether or not the unauthorized authentication is performed by an artificial object such as a photograph.

  The imaging unit 10 captures and accumulates a plurality of face images S10a and S10b having different postures. That is, the imaging unit 10 captures a plurality of face images having different viewpoints and compositions (postures) at the time of imaging. The imaging means 10 is realized by a normal camera. A small one attached to the portable terminal is also possible. The camera may be a still camera for still images or a video camera for moving images. When the imaging unit 10 is a video camera for moving images, the imaging unit 10 needs to extract a plurality of frames having different face postures from the captured moving image as a face image.

  The feature point extraction unit 20 extracts feature points in the face from each of the face images S10a and S10b having a plurality of postures imaged by the imaging unit 10. The feature points may be extracted manually, but it is preferable to automatically extract the feature points using a feature extraction filter or the like in consideration of application.

  In the present invention, feature points are used as corresponding points in stereo vision, so correspondence between images is more important than accurate feature position acquisition. As feature points to be extracted, arbitrary three feature points (hereinafter referred to as plane composing points) and out-of-plane points (hereinafter referred to as three-dimensional evaluation points) constituted by the three points in the real face. ) Is required at least one point. That is, the three-dimensional evaluation point is a point corresponding to a point on the real face in the three-dimensional world and corresponding to a point that is not on the plane composed of the three feature points extracted in the three-dimensional world. It is.

  Theoretically, there are an infinite number of combinations of three arbitrary feature points and three-dimensionality evaluation points. However, when feature points are automatically extracted using a feature extraction filter or the like, the points to be extracted are limited. FIG. 2 is an explanatory diagram illustrating an example of feature points extracted by the feature point extraction unit 20. As illustrated in FIG. 2, as a feature point candidate that can be easily extracted by automatic extraction, as a plane composing point, for example, any one of a total of four points including the center of the right eye, the center of the left eye, the right end of the lips, and the left end of the lips. Three points are given. Examples of the three-dimensionality evaluation point include the apex of the nose, the bottom of the nose, and the center of the upper end of the lips.

  In the case of a real human face, the latter three-dimensionality is recognized with respect to the former plane. That is, if the face images S10a and S10b are not images in which face photographs are taken as subjects, but are images in which a real three-dimensional face is taken as a subject, the three-dimensionality evaluation extracted by the feature point extraction means 20 The point is calculated not to be a point on a plane constituted by three plane composing points.

The three-dimensional amount calculation unit 30 calculates the three-dimensionality evaluation amount S30 of the three-dimensionality evaluation point from the feature point coordinates S20a and S20b of each image extracted by the feature point extraction unit 20. As shown in FIG. 2, using the right eye center (P ₀ ), the left eye center (P ₁ ), and the right lip edge (P ₂ ) as the plane composing points, the three points (for example, the nose) as the three-dimensionality evaluation points P are used. In the following, an example will be described in which images of the face postures 1 and 2 (images 1 and 2) are used.

  In the following description, a case where the camera model (perspective projection model) can be approximated by a general pinhole camera model in stereo view will be exemplified. FIG. 3 is an explanatory diagram for explaining an example of a pinhole camera model. The pinhole camera model is a general camera model in stereo view, and when an imaging target in a three-dimensional space is projected onto a two-dimensional image, the positional relationship between the imaging target and image formation on the image is simplified. This is an ideal model. A pinhole camera is a camera in which only light incident through a pinhole (a small hole opened by a pin) reaches the imaging surface. In a pinhole camera, the pinhole can be considered as the focal point. In the pinhole camera, since the image projected on the imaging surface is upside down, a model in which the imaging surface is moved between the pinhole and the imaging target is used as shown in FIG.

As shown in FIG. 3, a three-dimensional coordinate system (X, Y, Z) having a pinhole as an origin is taken, and a two-dimensional image coordinate system (u, v) is taken on the imaging surface. The coordinates of the plane composing point P _i (i = 0, 1, 2) in the three-dimensional space are P _i (X _i , Y _i , Z _i ), and the coordinates of the three-dimensional evaluation point P in the three-dimensional space are P (X, Y, Z).

Also, the coordinates in the image of the posture j (j = 1, 2) of P _i (i = 0, 1, 2) are (u _i ^j , v _i ^j ), and the posture j of P (j = 1, 2). ) In the image is (u ^j , v ^j ). That is, it is assumed that P _i (X _i , Y _i , Z _i ) in the three-dimensional coordinate system is mapped to (u _i ^j , v _i ^j ) in the image coordinate system. FIG. 3 illustrates an imaging surface of posture j (j = 1, 2), P (X, Y, Z) is mapped to (u ^j , v ^j ), and P ₀ (X ₀ , Y _{The case where 0} , Z ₀ ) is mapped to (u ₀ ^j , v ₀ ^j ) is illustrated.

  When the focal length when the camera is assumed to be a pinhole camera is f, the following formula 1 is established. T means transposition. The focal length f is a distance from the pinhole to the imaging surface.

In Expression (1), ξ _i ^j = Z _i ^j / f, and in Expression (2), ξ ^j = Z ^j / f. The transformations shown in equations (1) and (2) are called perspective projection (perspective transformation). ξ ₁ ^j and ξ ₂ ^j are magnification values in perspective projection.

Here, when the three-dimensionality evaluation point P is on the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ , A and B are real numbers.

This is a linearly dependent relationship. Therefore, the expressions (1) and (2) are substituted into the expression (3), and S ₁ ^j = ξ ₁ ^j / ξ ₀ ^j , S ₂ ^j = ξ ₂ ^j / ξ ₀ ^j , S ^j = ξ ^j / ξ ^Assuming ₀ ^j , the following equation (4) is obtained.

  Expanding the third line of equation (4),

  Therefore, if the real number C is defined as C = 1−A−B, and the formula (5) is substituted into the first and second rows of the formula (4) and rearranged, the first and second rows of the formula (4) about,

  Is established. When the formula (6) calculated from the postures 1 and 2 (j = 1, 2) is put together, the following formula (7) is obtained.

  The matrix in equation (7) is represented by equation (8), where D is D.

Since the vector [AS ₁ ^j , BS ₂ ^j , C] ^T in equation (7) cannot be a 0 vector from C = 1−A−B, the rank (rank) of the matrix D shown in equation (8) ) Becomes 2 for the following reason.

The matrix D is a mapping acting on the three-dimensional space V. When the point P is on the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ , the third dimension C of the vector [AS ₁ ^j , BS ₂ ^j , C] ^T is 1-A-B. As a result, the dimension of ker (D), which is the core of the matrix D, is dim (ker (D)) = 1. Also, dimV = 3. Dimensional theorem

  dimV = dim (ker (D)) + dim (D (V)) Equation (9)

  , Dim (D (V)), that is, the rank of the matrix D is 2.

Next, the rank of the matrix D in the case where the three-dimensionality evaluation point P is not on the plane formed by the plane configuration points P ₀ , P ₁ , P ₂ is obtained. It can also be determined from the rank of the matrix D that the point P is not on the plane. In this case, since there is no constraint in the third dimension C of the vector [AS ₁ ^j , BS ₂ ^j , C] ^T , dim (ker (D )) = 0, and dim (D (V)), that is, the rank of the matrix D is 3, from the dimensional theorem shown in Equation (9).

As described above, an amount indicating the rank of the matrix D may be adopted as the three-dimensional evaluation amount S30 for evaluating the three-dimensionality of the three-dimensional evaluation point. Examples of candidates for the quantity indicating the rank of the matrix D include, for example, the smallest singular value of the matrix D, the smallest eigenvalue of the matrix D ^T D (matches the square of the smallest singular value of D), and 0 of the characteristic polynomial of the matrix D ^T D There are fluctuations in nearby values.

Looking at the elements of the matrix D, S ₁ ² / S ₁ ¹ and S ₂ ² / S ₂ ¹ are usually unknown. However, ξ ₁ ^j and ξ ₂ ^j are magnification values obtained by perspective projection, and S ₁ ^j = ξ ₁ ^j / ξ ₀ ^j , S ₂ ^j = ξ ₂ ^j / ξ ₀ ^j is the value between plane composing points. Is the ratio. That is, S ₁ ^j is a ratio between ξ ₁ ^j and ξ ₀ ^j, and S ₂ ^j is a ratio between ξ ₂ ^j and ξ ₀ ^j . Since ξ ₁ ^j and ξ ₂ ^j are magnification values obtained by perspective projection, S ₁ ^j is a ratio between Z ₁ and Z _0, and S ₂ ^j is a ratio between Z ₂ and Z ₀ . Therefore, the unknown quantities S ₁ ² / S ₁ ¹ and S ₂ ² / S ₂ ¹ are values in the vicinity of 1 in the relationship between the plane plane composing point and the imaging surface of the camera, As long as the three-dimensional evaluation point that shows the three-dimensionality around the center of the upper edge of the lips under the nose is used, it can be regarded as 1.

As a result, when the minimum singular value of the matrix D and the minimum eigenvalue of the matrix D ^T D (matching the square of the minimum singular value of D) are used for the three-dimensionality evaluation amount S30, a simple coordinate difference is used as an element. It can be obtained by calculating singular values and eigenvalues of the matrix D.

When the value near 0 of the characteristic polynomial of the matrix D ^T D is used for the three-dimensionality evaluation amount S30, it is possible to perform the calculation only by integer arithmetic, which is effective when it is desired to reduce the calculation amount.

Hereinafter, as a stereoscopic evaluation amount S30, will be described that may be a value near 0 in the smallest eigenvalue and the characteristic polynomial of the matrix D ^{T D} of the matrix D ^{T D.} The characteristic polynomial of the matrix D ^T D is f (x), and the solution of f (x) = 0 is λ. λ is an eigenvalue of the matrix D ^T D. Note that the value near 0 in the characteristic polynomial of the matrix D ^T D is the value of f (x) near x = 0.

FIG. 4 is an explanatory diagram illustrating an example of the characteristic polynomial f (x) of the matrix D ^TD . In FIG. 4, the value of f (x) when the three-dimensional evaluation point P is in the plane formed by the plane constituent points P ₀ , P ₁ and P ₂ is represented by a solid line, and the three-dimensional evaluation point P is the nose. The value of f (x) when the point is the center of the lower or upper lip upper end is represented by a broken line, and the value of f (x) when the three-dimensional evaluation point P is a point on the nose is represented by a dashed line. ing.

Since the matrix D ^T D is a semi-positive target matrix, all eigenvalues do not take negative values. Therefore, since the minimum eigenvalue of the matrix D ^T D takes a value of 0 or more, f (x) takes a value as shown in FIG. 4, for example. Here, the minimum eigenvalue of the matrix D ^T D when the three-dimensional evaluation point P is in the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ is λ ₁ , and the three-dimensional evaluation point P is Let λ _{2 be} the minimum eigenvalue of the matrix D ^T D when it is the center of the upper lip upper end, and let λ _{3 be} the minimum eigenvalue of the matrix D ^T D when the three-dimensionality evaluation point P is a point on the nose. λ ₁ , λ ₂ , and λ ₃ are values of x at the point where f (x) intersects the horizontal axis (x-axis) in FIG.

Considering that the eigenvalue of the matrix D ^T D, that is, λ, which is a solution of f (x) = 0, coincides with the square of the singular value of D, the three-dimensional evaluation point P becomes the plane constituent point P ₀ , P _1. , P ₂ , the minimum eigenvalue λ ₁ is a value near zero. Further, when the three-dimensionality evaluation point P is outside the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ , the minimum eigenvalues λ ₂ , λ ₃ become values larger than 0 to some extent. That is, based on the minimum eigenvalue of the matrix D ^T D, it can be determined whether or not the three-dimensionality evaluation point P is within the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ .

  Further, when passing through the eigenvalue (when x = λ), the characteristic polynomial f (x) whose function value is 0 is, as shown in FIG. The behavior near x = 0 changes. That is, whether or not the sign of f (x) changes near x = 0 depends on whether or not the three-dimensional evaluation point P is in the plane. Therefore, if function values f (−ε) and f (ε) (where ε is a small positive integer) near x = 0 are calculated, the calculation can be performed only by integer arithmetic. The determination of the three-dimensionality is performed by the sign change of both values by the three-dimensionality determination means 40 to which these values are sent.

That is, by appropriately taking the value of ε, when f (ε) is positive and f (−ε) is a negative value, the three-dimensionality evaluation point P becomes the plane constituent points P ₀ and P _1. , P ₂ . When both f (ε) and f (−ε) are negative values, it is determined that the three-dimensionality evaluation point P is not on the plane formed by the plane constituent points P ₀ , P ₁ , P _2. Can do.

  Based on the three-dimensional evaluation amount S30 calculated by the three-dimensional calculation unit 30 and a predetermined threshold value, the three-dimensionality determination unit 40 determines the three-dimensionality of the three-dimensional evaluation point, that is, the face on the artifact by a photograph or the like. Whether it is a real face or not.

  The determination method differs depending on the three-dimensionality evaluation amount S30 calculated by the three-dimensional amount calculation means 30. When the three-dimensionality evaluation amount S30 is the minimum singular value of D, if the absolute value of the value is less than a certain value (threshold value), the three-dimensionality evaluation amount S30 is regarded as a planar object, and the absolute value is constant. If it is more than that, it is regarded as a three-dimensional object and is determined to be real.

If the three-dimensionality evaluation amount S30 is the minimum eigenvalue of the matrix D ^T D (matches the square of the minimum singular value of D), if the value is less than a certain value, it is considered to be a plane and is determined to be an artifact. If it is equal to or greater than a certain value, it is regarded as a three-dimensional object and is determined to be real.

In the example illustrated in FIG. 4, when the threshold is ε, for example, if the minimum eigenvalue of the matrix D ^T D is λ ₁ , λ ₁ is a value less than ε. It is determined that it is in the plane formed by the plane composing points P ₀ , P ₁ , P ₂ . Therefore, the face image is determined to be an image obtained by capturing a face photograph as a subject. Further, for example, taking the minimum eigenvalue lambda ₂ values of the matrix D ^{T D,} lambda ₂ is because it is a value greater than epsilon, stereo evaluation point P formed by the planar configuration point P _0, P _1, P ₂ It is determined that it is not on the plane. Therefore, the face image is determined to be an image obtained by capturing an actual three-dimensional face as a subject.

When the three-dimensionality evaluation amount S30 is a fluctuation of a value near 0 of the characteristic polynomial of the matrix D ^T D, the characteristic equation f (x) has different values as shown in FIG. Therefore, if there is a change in the sign of the function values f (−ε) and f (ε) near x = 0, it is determined as a planar object, and if not, it is determined as a three-dimensional object and determined as a real object.

In the example shown in FIG. 4, for example, f (−ε), which is a value near x = 0, of the characteristic polynomial f (x) of the matrix D ^T D takes a negative value, and f (ε) is a positive value. If it is taken, the three-dimensional evaluation point P is determined to be in the plane formed by the plane constituent points P ₀ , P ₁ , P ₂ . Therefore, the face image is determined to be an image obtained by capturing a face photograph as a subject. For example, if f (−ε) takes a negative value and f (ε) also takes a negative value, the three-dimensionality evaluation point P is on the plane formed by the plane constituent points P ₀ , P ₁ , P _2. It is judged that it is not. Therefore, the face image is determined to be an image obtained by capturing an actual three-dimensional face as a subject.

  As described above, according to the first embodiment, it is possible to determine whether or not the subject is a facial photograph by calculating the three-dimensionality of the subject of the captured image. The unauthorized authentication used can be detected.

Embodiment 2. FIG.
Next, a second embodiment of the present invention will be described with reference to the drawings. In the second embodiment, a three-dimensional authentication apparatus will be described in which each unit of the first embodiment performs each method in the three-dimensional authentication method. FIG. 5 is a block diagram showing a second embodiment of the three-dimensional authentication apparatus according to the present invention. 5 includes an imaging device 100, a feature point extraction device 200, a three-dimensional amount calculation device 300, and a three-dimensionality determination device 400.

  The imaging apparatus 100 may be a popular digital camera, a still camera for still images, or a video camera for moving images. In the case of a still camera, two or more images are taken when the face posture is different, and two or more images are sent to the feature point extraction apparatus 200. In the case of a video camera, the moving image during the face posture operation is sent to the feature point extraction device 200 as it is.

  FIG. 6 is an explanatory diagram illustrating an example of the feature point extraction apparatus 200. As shown in FIG. 6, the feature point extraction device 200 is a storage device such as a memory or a disk that stores image data captured by the imaging device 100, and an arithmetic device such as a CPU that performs calculation of feature point extraction in the image. And can be realized. When the imaging apparatus 100 is configured by a video camera, two or more frames having different face postures in a moving image are cut out as still images by calculation by the CPU. After two or more still images having different face postures, the feature point extraction means 20 of the first embodiment extracts the feature points, and the CPU performs calculation for feature point coordinate extraction. The feature point coordinate data is sent to the three-dimensional amount calculation device 300.

  FIG. 7 is an explanatory diagram illustrating an example of the three-dimensional amount calculation apparatus 300. As shown in FIG. 7, the three-dimensional amount calculation device 300 includes a storage device such as a memory or a disk that stores the feature point coordinate data extracted by the feature point extraction device 200, and an operation such as a CPU that calculates a three-dimensionality evaluation amount. It can be realized with a device. For the feature point coordinate data extracted from the two still images, the CPU calculates the three-dimensional evaluation amount by the method of calculating the three-dimensional evaluation amount by the three-dimensional amount calculation unit 30 of the first embodiment. After performing the above, data indicating the three-dimensionality evaluation amount is sent to the three-dimensionality determination device 400.

  The three-dimensionality determination device 400 can be realized by a simple logic circuit such as an LSI that determines whether the three-dimensionality evaluation amount calculated by the three-dimensional amount calculation device 300 is above or below a predetermined threshold value. When the function values f (−ε) and f (ε) in the vicinity of 0 are used as the three-dimensionality evaluation amount S30 in the first embodiment, no threshold value is required. Judge with.

Embodiment 3 FIG.
Next, a third embodiment of the present invention will be described with reference to the drawings. In an embodiment of the third invention, the feature point extraction method of the first embodiment is configured as a program executable on a computer, and the program is stored in an information storage medium readable by a computer. It is executed on the computer in the procedure shown in FIG.

  FIG. 8 is a flowchart showing the operation of the third embodiment. First, face images with different postures are photographed (step S100). In the procedure shown in FIG. 6, as a case where the minimum two postures are photographed, the face posture is changed (step S101), it is determined whether or not the image is taken twice (step S102), and when the image is taken twice. (Yes) When the process proceeds to step S200 and the image is not captured twice (No), the case is illustrated where the process proceeds to step S100.

  Next, in step S200, the coordinate value of the feature point is extracted from each of the two or more face images by the method using the feature point extraction unit 20 of the first embodiment. When the coordinate values of the feature points are extracted, the three-dimensionality evaluation amount is calculated by the method using the three-dimensional amount calculation unit 30 of the first embodiment using them (step S300). Finally, it is determined whether the face is an actual face or a face impersonated by an artificial object such as a photograph, based on the magnitude relationship between the three-dimensionality evaluation amount and a predetermined threshold (step S400). When the function values f (−ε) and f (ε) in the vicinity of 0 are used as the three-dimensionality evaluation amount S30 in the first embodiment, no threshold value is required. Judge with.

  In the first to third embodiments, the authentication of the three-dimensionality of the face image has been described. However, the similar embodiment can be used for discrimination between a photograph of a general scene and an actual scene for the purpose of monitoring or the like. Can also be done.

  According to the present invention, the present invention can be applied when detecting unauthorized authentication at the time of personal authentication using a face image. For example, it can be used for face image authentication in an unattended entrance / exit system or the like, and face image authentication in a small device such as a portable terminal.

DESCRIPTION OF SYMBOLS 10 Imaging means 20 Feature point extraction means 30 Three-dimensional amount calculation means 40 Three-dimensionality determination means

Claims (8)

A three-dimensional authentication method for determining the three-dimensionality of a shooting object,
An input step of acquiring a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired in the input step, and the coordinates of each feature point are obtained. A feature point extraction step to be acquired;
Among the feature points extracted in the feature point extraction step, the three-dimensionality of the three-dimensionality evaluation point that is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. A solid quantity calculation step for calculating a three-dimensionality evaluation quantity defined as a quantity;
Anda steric determination step of determining three-dimensional properties of the object to be shot on the basis of the three-dimensional evaluation amount calculated by the three-dimensional volume calculation step,
In the three-dimensional amount calculation step, the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point converted into the two-dimensional coordinate system, and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points are two-dimensional. Calculate the minimum singular value of the determination matrix, which is a difference between the coordinates converted into the coordinate system and the difference calculated for each captured image as a single element, as a three-dimensionality evaluation amount,
In the three-dimensionality determination step, when the absolute value of the minimum singular value is greater than or equal to a threshold value, it is determined that the three-dimensionality evaluation point is not on a plane. A three-dimensional authentication method for determining the three-dimensionality of a shooting object,
An input step of acquiring a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired in the input step, and the coordinates of each feature point are obtained. A feature point extraction step to be acquired;
Among the feature points extracted in the feature point extraction step, the three-dimensionality of the three-dimensionality evaluation point that is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. A solid quantity calculation step for calculating a three-dimensionality evaluation quantity defined as a quantity;
Anda steric determination step of determining three-dimensional properties of the object to be shot on the basis of the three-dimensional evaluation amount calculated by the three-dimensional volume calculation step,
In the three-dimensional amount calculation step, the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point converted into the two-dimensional coordinate system, and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points are two-dimensional. A regular matrix that is a product of a transpose matrix of a determination matrix that is a difference between coordinates converted into a coordinate system and that is a matrix in which differences calculated for each captured image are combined as elements. Is calculated as the three-dimensionality evaluation amount,
In the three-dimensionality determination step, when the minimum eigenvalue is greater than or equal to a threshold value, it is determined that the three-dimensional evaluation point is not on a plane. A three-dimensional authentication method for determining the three-dimensionality of a shooting object,
An input step of acquiring a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired in the input step, and the coordinates of each feature point are obtained. A feature point extraction step to be acquired;
Among the feature points extracted in the feature point extraction step, the three-dimensionality of the three-dimensionality evaluation point that is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. A solid quantity calculation step for calculating a three-dimensionality evaluation quantity defined as a quantity;
Anda steric determination step of determining three-dimensional properties of the object to be shot on the basis of the three-dimensional evaluation amount calculated by the three-dimensional volume calculation step,
In the three-dimensional amount calculation step, the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point converted into the two-dimensional coordinate system, and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points are two-dimensional. A regular matrix that is a product of a transpose matrix of a determination matrix that is a difference between coordinates converted into a coordinate system and that is a matrix in which differences calculated for each captured image are combined as elements. the characteristic polynomial, and calculates code different 0 a value obtained by substituting the 2 values of neighboring as each three-dimensional evaluation quantity,
In the three-dimensionality determination step, when the sign does not change in the two three-dimensionality evaluation amounts, it is determined that the three-dimensionality evaluation point is not on a plane. The three-dimensionality authentication method according to claim 3, wherein in the three-dimensional amount calculation step, a value obtained by substituting two near-zero values having the same absolute value with different signs into a characteristic polynomial is calculated as the three-dimensionality evaluation amount. A three-dimensional authentication device for determining the three-dimensionality of a photographing object,
Input means for acquiring a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired by the input means, and the coordinates of the feature points are extracted. A feature point extraction means to obtain;
Among the feature points extracted by the feature point extracting means, the three-dimensionality of the three-dimensionality evaluation point which is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. A solid amount calculation means for calculating a three-dimensionality evaluation amount defined as a quantity;
And a three-dimensional determination means for determining the three-dimensional properties of the object to be shot on the basis of the calculated three-dimensional evaluation amount by the stereoscopic amount calculating means,
The three-dimensional amount calculation means is configured to two-dimensionally convert the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point into the two-dimensional coordinate system and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points. Calculate the minimum singular value of the determination matrix, which is a difference between the coordinates converted into the coordinate system and the difference calculated for each captured image as a single element, as a three-dimensionality evaluation amount,
The three-dimensionality determination apparatus, wherein the three-dimensionality determination unit determines that the three-dimensionality evaluation point is not on a plane when the absolute value of the minimum singular value is equal to or greater than a threshold value. A three-dimensional authentication device for determining the three-dimensionality of a photographing object,
Input means for acquiring a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired by the input means, and the coordinates of the feature points are extracted. A feature point extraction means to obtain;
Among the feature points extracted by the feature point extracting means, the three-dimensionality of the three-dimensionality evaluation point which is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. A solid amount calculation means for calculating a three-dimensionality evaluation amount defined as a quantity;
And a three-dimensional determination means for determining the three-dimensional properties of the object to be shot on the basis of the calculated three-dimensional evaluation amount by the stereoscopic amount calculating means,
The three-dimensional amount calculation means is configured to two-dimensionally convert the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point into the two-dimensional coordinate system and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points. A regular matrix that is a product of a transpose matrix of a determination matrix that is a difference between coordinates converted into a coordinate system and that is a matrix in which differences calculated for each captured image are combined as elements. Is calculated as the three-dimensionality evaluation amount,
The three-dimensionality determination unit determines that the three-dimensionality evaluation point is not on a plane when the minimum eigenvalue is equal to or greater than a threshold value. A three-dimensional authentication device for determining the three-dimensionality of a photographing object,
Input means for acquiring a plurality of photographed images photographed by a single photographing device from a plurality of viewpoints having different photographing objects;
Feature points corresponding to four or more points on the object to be photographed that are not on the same plane in the three-dimensional coordinate system are extracted from the photographed image acquired by the input means, and the coordinates of the feature points are extracted. A feature point extraction means to obtain;
Among the feature points extracted by the feature point extracting means, the three-dimensionality of the three-dimensionality evaluation point which is a feature point outside the plane constituted by the plane constituent points selected as the feature points constituting the plane is evaluated. A solid amount calculation means for calculating a three-dimensionality evaluation amount defined as a quantity;
And a three-dimensional determination means for determining the three-dimensional properties of the object to be shot on the basis of the calculated three-dimensional evaluation amount by the stereoscopic amount calculating means,
The three-dimensional amount calculation means is configured to two-dimensionally convert the coordinates of the feature points in the three-dimensional coordinate system of the three-dimensional evaluation point into the two-dimensional coordinate system and the coordinates of the feature points in the three-dimensional coordinate system of the respective plane constituent points. A regular matrix that is a product of a transpose matrix of a determination matrix that is a difference between coordinates converted into a coordinate system and that is a matrix in which differences calculated for each captured image are combined as elements. the characteristic polynomial, and calculates code different 0 a value obtained by substituting the 2 values of neighboring as each three-dimensional evaluation quantity,
The three-dimensionality determination unit determines that the three-dimensionality evaluation point is not on a plane when the sign does not change in the two three-dimensionality evaluation amounts. The three-dimensionality authentication apparatus according to claim 7, wherein the three-dimensional amount calculation means calculates a value obtained by substituting two near-zero values having the same absolute value with different signs into a characteristic polynomial as a three-dimensionality evaluation amount.

Images