JP2018036965A - Living body detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a living body detection device that can easily detect an impersonation, using an imaging device attached to a mobile terminal even with respect to an impersonation attack.SOLUTION: A living body detection device 1 according to the present invention, which is the living body detection device that coordinates with a mobile terminal M having an imaging device 14 capable of shooting a motion image, and detects an impersonation of a legitimate person of an authenticated person handling a mobile terminal M1, comprises: an action unit P11 that acts on a subject a; an acquisition unit P21 that photographs images having the subject a included in time series, using an imaging device 14; a calculation unit P31 that calculates a matching level of between an action on the subject a by the action unit P11, and the image in time series photographed by the acquisition unit P21; and a judgement unit P41 that judges whether the authenticated person is a person impersonating a legitimate person, using the matching level calculated by the calculation unit P31.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a living body detection apparatus.

  In addition to logging in to PCs and entering and exiting buildings, biometric authentication technology is being used in a wide range of fields including the financial field. With the spread of mobile terminals such as smartphones, such technology is becoming more important in personal authentication from mobile terminals. On the other hand, it has also been reported that fraudulent impersonation is possible with a finger artificially reconstructed from residual fingerprints, photographs, videos, and the like.

  In recent years, as a biometric authentication method using a mobile terminal such as a smartphone, there has been a movement of performing biometric authentication using a general-purpose device such as a camera attached to the mobile terminal instead of a special device for biometric authentication. In particular, in the technology that performs biometric authentication such as fingers, palm prints, and fingerprints using the attached camera, there is a possibility of being subjected to a pretending attack by a photograph or video taken in advance. It is necessary to improve safety by doing.

  As a living body detection technique for impersonation using such a general-purpose device, for example, a face image is taken by a camera, and a pulse is detected by a time-series change of representative values of R and G components in a specific region in the face image. A method for performing living body detection (for example, refer to Patent Document 1) and a method for confirming that a luminance value changes only in a specific region by moving a finger as a subject back and forth under LED illumination (for example, non-patent) Document 1) is disclosed.

JP 2014-184002 A

Chris Stein, “Video-based Fingerprint Recognition with Anti-spoofing Technologies with Smartphone Cameras”, BIOSIG, 2013

  However, in the conventional technique described above, in the method of detecting the pulse, the time-series change of the color component can be reproduced by photographing the finger in advance with a moving image, and there is a possibility that it is impersonated improperly. is there. On the other hand, in the method of checking the change of the brightness value, it is possible to detect the change of the brightness value in advance under the LED lighting with a moving image and present it at the time of authentication. There is a possibility that.

  The present invention has been made based on the circumstances as described above, and the purpose thereof is attached to the mobile terminal against a pretending attack that presents a photograph or a moving picture such as a fingerprint or a palm print that has been taken in advance. An object of the present invention is to provide a living body detection device that can easily detect impersonation using an imaging device.

The present invention
(1) A living body detection device that detects impersonation of an authenticated person who handles a mobile terminal in cooperation with a mobile terminal having a shooting device capable of shooting a moving image,
An action part acting on the subject;
An acquisition unit that shoots an image including the subject in time series using the imaging device;
A calculation unit that calculates the degree of coincidence between the action of the action unit on the subject and a time-series image captured by the acquisition unit;
A living body detection device comprising: a determination unit that determines whether or not the person to be authenticated is impersonating the right person using the degree of coincidence calculated by the calculation unit;
(2) The action unit instructs the person to be authenticated to move the mobile terminal relative to the subject.
The degree of coincidence calculated by the calculation unit is calculated using the direction in which the mobile terminal is instructed by the action unit and the moving direction of the subject obtained from the time-series images taken by the acquisition unit. The living body detection device according to (1),
(3) The action unit irradiates the subject multiple times with a predetermined timing,
In the above (1), the degree of coincidence calculated by the calculating unit is calculated using the timing and a change in luminance value of the subject obtained from the time-series images taken by the acquiring unit. The living body detection device according to the description,
(4) A first image determination unit is further provided,
The first image determination unit binarizes the image photographed by the acquisition unit, and obtains the number of connected components by performing a morphological process on the binarized image.
The living body detection device according to any one of (1) to (3), wherein the determination unit determines whether or not the person to be authenticated is impersonating a valid person based on the degree of coincidence and the number of connected components. And (5) further comprising a second image determination unit,
The second image determination unit calculates a gradient direction of luminance for each local region constituting the image from the image photographed by the acquisition unit, and uses the calculated gradient direction to quantize each predetermined angle. To obtain a normalized histogram,
The living body detection according to any one of (1) to (3), wherein the determination unit determines whether or not the person to be authenticated is impersonating a valid person based on the degree of coincidence and the variance of the frequencies in the histogram. apparatus,
About.

  In the present specification, “mobile terminal” means a mobile terminal such as a smartphone or a tablet terminal used by a user. Further, the “degree of coincidence” means an index indicating whether or not the action such as an instruction by the action unit is consistent with the change mode of the subject calculated from the time-series image calculated by the calculation unit. . The “number of connected components” means the number of connected components in one image. However, “connected component” refers to a region visually connected to one in a binarized image. Further, the “gradient direction” means a direction in which a change in luminance value is large in a partitioned partial area in the image.

  The present invention provides a living body detection device that can easily detect impersonation using a photographing device attached to a mobile terminal even for a pretending attack that presents a photograph or a moving image such as a fingerprint or a palm print that has been photographed in advance. Can be provided. As a result, the present invention can improve safety when performing biometric authentication.

1 is a schematic block diagram showing a first embodiment of the present invention. It is a schematic sequence diagram which shows an example of the synchronous detection process by the synchronous detection program in the biometric detection apparatus of FIG. It is a flowchart of the biological body detection process which the biological body detection apparatus of FIG. 1 performs. It is the schematic which shows an example of the display screen which detects using a mobile terminal incorporating the living body detection apparatus of FIG. 1 according to a moving direction. It is the schematic which shows an example of the display screen of the mobile terminal which concerns on 2nd Embodiment. It is the schematic which shows an example of the display screen of the mobile terminal which concerns on 3rd Embodiment. It is a flowchart of the biological body detection process which the biological body detection apparatus of 4th Embodiment performs. FIG. 8 is a schematic diagram illustrating an example of a display screen when a luminance value changes in FIG. 7, where (a) shows a screen when no light is irradiated, and (b) shows a screen when light is irradiated. Each is shown. It is the schematic for demonstrating the biological body detection method in 4th Embodiment, (a) is the change of the luminance value by LED, (b) has shown the synchronization confirmation method, respectively. It is a flowchart of the biological body detection process in 5th Embodiment. FIG. 11 is a schematic diagram illustrating an example of an image generated in each step of FIG. It is a flowchart of the biological body detection process in 6th Embodiment. It is the schematic which shows the usage example of the biometrics apparatus which applied the biometric detection apparatus of this invention.

  A living body detection apparatus according to the present invention is a living body detection apparatus that detects impersonation of an authenticated person who handles a mobile terminal in cooperation with a mobile terminal having a shooting apparatus capable of shooting a moving image. An action unit that acts on the subject, an acquisition unit that takes an image including the subject in time series using the photographing device, an action on the subject by the action unit, and a time-series image taken by the acquisition unit And a determination unit for determining whether or not the person to be authenticated is impersonating the right person using the degree of coincidence calculated by the calculation unit. Features.

  Note that the “subject” in this specification is preferably configured such that at least a part on the side imaged by the imaging device is configured in a three-dimensional shape. As such a subject, for example, the finger (fingerprint) or palm (palmprint) of the person to be authenticated exemplified below can be adopted.

  In addition, the living body detection device is not particularly limited as long as it cooperates with the mobile terminal, but it is preferably incorporated in the mobile terminal from the viewpoint of improving portability and convenience.

  Hereinafter, although the 1st-6th embodiment of the said living body detection apparatus is described with reference to drawings, this invention is not limited only to embodiment described in the said drawing.

[First Embodiment]
The living body detection device 1 cooperates with a mobile terminal used by a user such as a smartphone or a tablet terminal having a photographing device capable of photographing a moving image, and roughly includes an action unit P11, an acquisition unit P21, and a calculation. A part P31 and a determination part P41 are included. In the present embodiment, an example in which the living body detection device is incorporated in a mobile terminal will be described.

  The action unit P11 acts on the subject, and in this embodiment, instructs the person to be authenticated to move the mobile terminal M1 relative to the subject. As this action part P11, as shown in FIG. 1, for example, an audio output device 151 such as a speaker capable of giving an instruction to the person to be authenticated, a display device 12 capable of giving an instruction visually, and the like are adopted.

  The acquisition unit P21 uses the imaging device 14 to capture an image including the subject in time series. The photographing apparatus 14 is not particularly limited as long as it can shoot a moving image, but is preferably attached to the end of the mobile terminal M1 from the viewpoint of convenience.

  The calculation unit P31 calculates the degree of coincidence between the action on the subject by the action part P11 (in this embodiment, an operation instruction to the person to be authenticated) and the time-series images taken by the acquisition part P21. The degree of coincidence calculated by the calculation unit P31 of the living body detection apparatus 1 is the direction of moving the mobile terminal M1 instructed by the action unit P11 and the movement of the subject obtained from the time-series images taken by the acquisition unit P21. It is calculated using the direction. Specifically, the calculation unit P31 of the living body detection apparatus 1 obtains the direction in which the mobile terminal M1 actually moved by examining the increase or decrease of the background area of each area in the captured image, and the direction indicated above. The degree of coincidence is obtained by comparing the direction in which the mobile terminal M1 actually moved.

  The determination unit P41 uses the degree of coincidence calculated by the calculation unit P31 to determine whether or not the person to be authenticated is impersonating a valid person.

  Here, the action unit P11 described above is an audio output device 151 and a display device 12 described later, an acquisition unit is a shooting device 14 described later, a calculation unit P31 and a determination unit P41 are stored in a CPU 13, work area 16, and information holding unit 17 described later. Although generally corresponding, it is not necessarily limited to this.

  Next, the hardware configuration of the living body detection apparatus 1 will be described. FIG. 1 is a schematic block diagram showing a first embodiment of the present invention. As shown in FIG. 1, the living body detection apparatus 1 is roughly configured by an input device 11, a display device 12, a CPU (Central Processing Unit) 13, an imaging device 14, a work area 16, and information holding means 17. Yes.

  The input device 11 receives an input such as an instruction from the user. The input device 11 employs a keyboard, a mouse, a touch panel, or the like, and receives a desired process from a plurality of boxes (buttons) displayed on the display device 12, for example.

  The display device 12 outputs information such as various characters and images. For example, an image display device such as a liquid crystal display is employed as the display device 12. The input device 11 and the display device 12 described above may be integrated.

  The CPU 13 implements various functions by executing programs stored in the work area 16. Specifically, the CPU 13 executes processing of each program described later.

  The photographing device 14 photographs a subject such as a finger (fingerprint) or a palm (palmprint). Specifically, the photographing device 14 includes a camera (camera 14) attached to the mobile terminal M1.

  The work area 16 is a storage area for storing programs executed by the CPU 13. The work area 16 typically employs a high-speed and volatile storage device such as a DRAM (Dynamic Random Access Memory). In the present embodiment, the work area 16 stores an OS (Operating System) 161, a communication program 162, a synchronization detection process 163, subject time-series data 164, authentication device time-series data 165, and the like.

  It should be noted that at least a part of each program and data stored in the information holding means 17 may be temporarily copied to the work area 16 as necessary when the CPU 13 executes various processes. And the reference data thereof may be stored. Further, the work area 16 may store the result of the process executed by the CPU 13. Processing based on these programs will be described later.

  The information holding unit 17 includes an operation instruction program 171, a subject time-series data acquisition program 172, an authentication device time-series data acquisition program 173, a synchronization detection program 174, and the like that are referred to by the CPU 13 to execute various processes based on each program. The program etc. are stored. As this information holding means 17, typically, a large-capacity nonvolatile storage device such as an HDD (Hard Disk Drive) or a flash memory is employed.

  The above-described operation instruction program 171 gives an operation instruction to the subject, for example, “Move the mobile terminal M1 to the left or right”. The subject time-series data acquisition program 172 acquires time-series information of image features such as a moving direction of a subject and a change in luminance value based on a plurality of continuous subject images. When the mobile terminal M1 is an authentication device such as a smartphone, the authentication device time-series data acquisition program 173 acquires time-series information about the direction and the operation direction of the mobile terminal M1 using an attached acceleration sensor. The synchronization detection program 174 detects whether or not the operation instruction matches the movement of the subject. The information holding unit 17 may store other programs such as a program for performing biometric authentication (identity confirmation) based on the biometric detection result.

  Next, a biological detection sequence in the biological detection device 1 will be described. Here, the identity verification process when the user A receives the service provided by the servicer B using the mobile terminal M1 in the presence of the user A, the mobile terminal M1, and the servicer B will be described as an example. FIG. 2 is a schematic sequence diagram illustrating an example of synchronization detection processing by the synchronization detection program in the living body detection apparatus of FIG.

  As shown in FIG. 2, first, the user A makes a service start request to the servicer B in order to receive the service provided by the servicer B (step S201). In response to this, the servicer B requests the mobile terminal M1 to confirm the identity of the service requester (step S202). Next, the mobile terminal M1 requests biometric authentication for user identification from the user A (step S203). On the other hand, the user A presents biometric information such as a finger or a palm as biometric information used for identity verification (step S204).

  Next, the mobile terminal M1 instructs the user A to move the mobile terminal M1 relative to the subject in order to detect whether the presented biometric information is a fake such as a pre-captured video or photo ( Step S205). In response to this, the user A moves the mobile terminal M1 according to the instruction of step S205 (step S206).

  Next, the mobile terminal M1 detects whether or not the subject is impersonating (forgery) by performing biometric detection described later. Here, if the biometric detection is successful, the user A is notified of the biometric detection success, and the process proceeds to biometric authentication processing (step S207).

  In the biometric authentication process, after performing identity verification using biometric authentication technology, if the identity verification is successful, the user A and the servicer B are notified of “successful” identity confirmation (steps S208 and S209). The requested service is started by the servicer B (step S210). On the other hand, if the above-described biometric detection fails, the user A and the servicer B are notified of “failure” identity confirmation, and the process ends (not shown).

  Next, a specific flow of biological detection performed using the biological detection device 1 will be described with reference to FIG. In the living body detection by the living body detection apparatus 1, the impersonation detection of the subject is performed by checking the degree of coincidence between the user A (subject) and the operation direction of the mobile terminal M1.

  As shown in FIG. 3, first, the mobile terminal M1 issues an instruction for terminal operation to the user A (step S301). As this instruction, for example, the movement target and direction may be clearly indicated, such as “Move the terminal left and right”, and the direction in which the user moves to the user side, such as “Move the terminal in any direction”. You may leave it.

  Next, in accordance with the above instruction, the user A moves the mobile terminal M1 while photographing a finger or palm that is a detection target (subject) (step S302). Next, after determining the moving direction of the subject (finger or palm) (step S303), a determination is made (step S304). At this time, if the direction of movement is different from the instruction, it is determined that there is a possibility of impersonation, and authentication NG is determined (S305). If the direction matches, the process proceeds to the subsequent biometric authentication process (step S306). User authentication is performed (step S307). In addition, as a technique for performing biometric authentication processing, a known technique such as fingerprint, finger vein, and palm authentication method (for example, “Biometrics textbook—from principle to programming, Hanya et al., 2012” etc.) should be adopted. Can do.

  Here, the determination method of the moving direction of the subject in the present embodiment in the above-described step S303 will be described with reference to FIG. FIG. 4 is a schematic diagram illustrating an example of a display screen that performs detection based on a moving direction using a mobile terminal in which the living body detection apparatus of FIG. 1 is incorporated. FIG. 4 shows a state in which a subject a to be subjected to living body detection is photographed using the camera 14 attached to the mobile terminal M1. In FIG. 4, an axis 211 and an axis 212 indicate center lines in directions orthogonal to each other (for example, the vertical direction and the horizontal direction). In addition, areas 213 to 216 indicate the four areas divided by the two center lines in the shooting area of the subject.

In the determination method according to the present embodiment, for example, the mobile terminal M1 is moved vertically and horizontally with respect to the palm that is the subject a in accordance with an instruction from the mobile terminal M1. For example, when the mobile terminal M1 is moved to the right with respect to the subject a, the subject a moves to the left, so that the area of the background image (background area) in the areas 215 and 216 increases. On the other hand, when the mobile terminal M1 is moved to the left side, the subject a moves to the right, so that the background areas of the area 213 and the area 214 increase. Similarly, when the mobile terminal M1 is moved upward, the background areas of the areas 213 and 215 are increased, and when the mobile terminal M1 is moved downward, the background areas of the areas 214 and 216 are increased. Therefore, the movement direction of the mobile terminal M1 is estimated (calculated) by examining the increase / decrease in the background area of each of the areas 213 to 216. Here, it is assumed that the background image area of the area 213 is the background image F10, the background image area of the area 214 is the background image F14, the background image area of the area 215 is the background image F12, and the background image area of the area 216 is the background image F16. At this time, for example, when the hand is held in the vertical direction, the moving direction vector of the subject is calculated as follows.

  In the above formula (1), MoveX represents movement in the X-axis (horizontal) direction, MoveY represents movement in the Y-axis (vertical) direction, and ObjMovVec represents the moving direction of the subject. A method of calculating the degree of coincidence between the moving direction of the subject and the moving direction of the mobile terminal M1 is expressed by Expression (2).

  Here, for example, the degree of coincidence matches the direction instructed from the mobile terminal M1 and the direction in which the mobile terminal M1 has moved based on this instruction (the estimated moving direction of the mobile terminal M1 described above). It may be expressed by two choices of a case and a case where they do not match, and impersonation can be detected using this.

  In the determination of the moving direction, an image after separating the subject image (finger or palm image) F11 and the background image (image other than the subject image in the photographed image) F12 is used. Such separation (background separation) can be performed using, for example, a skin color component or a luminance value difference between the subject image F11 and the background image F12.

  As described above, the living body detection apparatus 1 includes the action unit P11, the acquisition unit P21, the calculation unit P31, and the determination unit P41. Even for an attack, impersonation can be easily detected using the camera 14 attached to the mobile terminal M1. In particular, since the living body detection device 1 calculates the degree of coincidence according to the moving direction of the mobile terminal M1 as described above, impersonation can be detected more reliably.

  In the first embodiment described above, the determination method for moving the mobile terminal M1 with respect to the subject a according to an instruction from the mobile terminal M1 is illustrated. However, the subject a with respect to the mobile terminal M1 according to an instruction from the mobile terminal M1. Or a determination method performed by moving both the mobile terminal M1 and the subject a.

  For example, in the determination method performed by moving both the mobile terminal M1 and the subject a, the determination can be made by determining whether the moving direction of the subject a and the moving direction of the mobile terminal M1 are within a certain angle. . In such a case, for example, a difference in angle in the moving direction between the mobile terminal M1 and the subject a is obtained. This difference can be calculated using the following equation (2).

  In the above formula (2), A indicates the moving direction vector of the mobile terminal M1, and B indicates the moving direction vector of the subject a. If the difference between the angles calculated by the above equation (2) using these two movement direction vectors is outside the predetermined range, it is determined that there is a possibility of impersonation and authentication NG is determined, and the difference between the angles is within the predetermined range. If so, the process proceeds to the subsequent biometric authentication process.

  Here, a usage example of the mobile terminal M1 in which the living body detection apparatus 1 is incorporated will be described with reference to FIG. In FIG. 13, a screen G11 represents a service start screen displayed on the mobile terminal M1.

  First, the user A selects a desired service from the service menu displayed on the screen G11. Next, when, for example, the “remittance” service is selected from the service menu, a list of remittance destination candidates is displayed, and the user selects a remittance target person from the displayed remittance destination candidates (see screen G12). Next, the desired remittance amount is input. Here, “15000” yen is input as the amount of money to be sent (see screen G13).

  Next, after a confirmation screen for confirming the remittance target and the remittance amount is displayed (see screen G14), an authentication method ("fingerprint authentication" (biometric authentication) or "password" for verifying the identity of the remittance transaction is displayed. A screen for selecting “authentication”) is displayed (see screen G15). Here, “fingerprint authentication” (biometric authentication) is selected in order to confirm that the person has instructed the remittance.

  Next, a finger | toe is image | photographed using the camera 14 attached to the mobile terminal M1 (refer screen G16). At this time, the living body detection apparatus 1 performs living body detection (see steps S203 to S207 in FIG. 2) to detect whether or not impersonation is performed using a previously captured moving image or still image. As a result, if it is not impersonation, the biometric authentication process is continued, and after successful identity verification, the remittance is executed and the remittance result is displayed on the display device (see screen G17).

  In this way, impersonation can be easily detected by the biometric detection device 1 in cooperation with the mobile terminal M1, and as a result, safety when performing biometric authentication can be improved.

[Second Embodiment]
The living body detection apparatus 2 of the present embodiment estimates (calculates) in which direction the terminal has been moved by tracking the fingertip position while moving the mobile terminal. The action part P11, the acquisition part P21, the calculation part P32, and the determination part P41 are comprised. The living body detection apparatus 2 is different from the first embodiment in the configuration of the calculation unit. In addition, since the action part P11, the acquisition part P21, the determination part P41, the hardware configuration, and the biometric detection sequence are the same as those in the first embodiment, detailed descriptions thereof are omitted.

  The calculation unit P32 calculates the degree of coincidence between the action on the subject a by the action part P11 (in this embodiment, an instruction of an operation for the person to be authenticated) and the time-series images taken by the acquisition part P21. . The degree of coincidence calculated by the calculation unit P32 of the living body detection device 2 is the direction of moving the mobile terminal M2 instructed by the action unit P11 and the subject a obtained from the time-series images taken by the acquisition unit P21. It is calculated using the moving direction. Specifically, the calculation unit P32 of the living body detection apparatus 2 extracts the fingertip coordinates of each finger in the captured image and examines the change with time of the fingertip coordinates, so that the mobile terminal M2 has actually moved. And the degree of coincidence is obtained by comparing the instructed direction with the direction in which the mobile terminal M2 has actually moved.

  Next, a method for determining the moving direction of the subject a in the present embodiment in step S303 described above will be described with reference to FIG. FIG. 5 is a schematic diagram illustrating an example of a display screen of the mobile terminal according to the second embodiment. FIG. 5 shows a state in which a subject a to be subjected to living body detection is photographed using the camera 14 attached to the mobile terminal M2. In FIG. 5, the positions of the fingertips a1 to a3 indicate the fingertip coordinates of each finger for detecting the movement (movement of the hand) of the mobile terminal M2.

  In the determination method according to the present embodiment, an image with a background separated in advance is used as in the first embodiment. First, a palm image photographed by the camera 14 is binarized, and a peak position as a fingertip position is calculated from the binarized image. At this time, when the palm is held vertically toward the camera 14, the peak position in the Y-axis direction (vertical direction of the screen) is calculated, and when the palm is held sideways, the peak position in the X-axis direction (horizontal direction of the screen) is calculated. Is calculated. Next, in the former case, three points are selected from the one with the larger peak position in the Y-axis direction (when the upward direction on the screen is the positive direction), and the X and Y coordinates of these three points are respectively selected from the fingertips a1, a2, and a3. It is recorded as the coordinates (position).

  Next, the coordinates (positions) of the fingertips a1 to a3 are recorded in the same manner for the image after the mobile terminal M2 has been moved by the instruction. At this time, the vector of each fingertip a1 to a3 is calculated from the difference in the coordinates of the corresponding three fingertips a1 to a3 before and after the movement, and the moving direction vector is calculated by calculating the average of these vectors and recorded. To do. Here, the actual movement direction of the mobile terminal M2 is calculated by inverting the movement vector by 180 degrees. Next, after comparing the instructed direction with the actual moving direction of the mobile terminal M2 to obtain the degree of coincidence, the determination unit P41 determines whether or not the subject a is impersonating.

  Here, for example, the degree of coincidence matches the direction instructed from the mobile terminal M2 and the direction in which the mobile terminal M2 has moved based on this instruction (the moving direction of the mobile terminal M2 determined from the movement vector). It may be expressed by two choices, that is, the case of not matching, and impersonation can be detected using this.

  As described above, the living body detection apparatus 2 includes the action unit P11, the acquisition unit P21, the calculation unit P32, and the determination unit P41 described above. Even for an attack, impersonation can be easily detected using the camera 14 attached to the mobile terminal M2. In particular, since the living body detection device 2 calculates the degree of coincidence according to the moving direction of the mobile terminal M2 as described above, impersonation can be detected more reliably.

  In the second embodiment described above, the correspondence relationship between the fingertips a1 to a3 between the two images is obtained from the peak positions of the X coordinate and the Y coordinate. The correspondence relationship may be obtained by selecting a combination that minimizes. Thereby, each fingertip can be made to correspond more reliably, and the movement of the fingertip and the movement direction vector based thereon can be calculated more accurately. Further, the fingertips to be used are not limited to the three points described above, and may be increased or decreased, or may be only fingertips that are within the display screen both before and after movement.

[Third Embodiment]
The living body detection apparatus 3 of the present embodiment estimates (calculates) in which direction the terminal is moved by tracking the position between background local feature points while the mobile terminal is moving. Schematically, it is comprised by the action part P11, the acquisition part P21, the calculation part P33, and the determination part P41. The living body detection device 3 is different from the first embodiment in the configuration of the calculation unit. In addition, since the action part P11, the acquisition part P21, the determination part P41, the hardware configuration, and the biometric detection sequence are the same as those in the first embodiment, detailed descriptions thereof are omitted.

  The calculation unit P33 calculates the degree of coincidence between the action of the action part P11 on the subject a (in this embodiment, an instruction to the user to be authenticated) and the time-series images taken by the acquisition part P21. . The degree of coincidence calculated by the calculation unit P33 of the living body detection device 3 is the direction of moving the mobile terminal M3 instructed by the action unit P11 and the subject a obtained from the time-series images captured by the acquisition unit P21. It is calculated using the moving direction. Specifically, the calculation unit P33 of the living body detection device 3 obtains the direction in which the mobile terminal M3 actually moved by extracting the position coordinates of the background local feature point and examining the temporal change of the position coordinates, The degree of coincidence is obtained by comparing the instructed direction with the direction in which the mobile terminal M3 has actually moved.

  Next, a method for determining the moving direction of the subject a in the present embodiment in step S303 described above will be described with reference to FIG. FIG. 6 is a schematic diagram illustrating an example of a display screen of the mobile terminal according to the third embodiment. FIG. 6 shows a state where the subject a is photographed using the camera 14 attached to the mobile terminal M3, as in FIG. In FIG. 6, each point in the background image F32 indicates a background local feature point. This background local feature point uses a local image feature (see, for example, “Information Processing”, September 2008 (Vol. 49, No. 9) “High-Speed Specific Object Recognition System Made in 3 Days”). Can be derived.

  In the determination method according to the present embodiment, an image with a background separated in advance is used as in the first embodiment. First, a local image feature point before an instruction (before moving the mobile terminal M3) is extracted from the local image feature, and its X coordinate and Y coordinate are calculated.

  Next, for the image after moving the mobile terminal M3 according to the instruction, the X and Y coordinates of the corresponding local image feature point are calculated, and then each local image is calculated according to the difference in the coordinates of the corresponding local image feature point before and after the movement. A vector of image feature points is calculated, and a moving direction vector is calculated by obtaining an average of these vectors and recorded. Here, the actual movement direction of the mobile terminal M3 is calculated by inverting the movement vector by 180 degrees. Next, after comparing the instructed direction with the actual moving direction of the mobile terminal M3 to obtain a degree of coincidence, the determination unit P41 determines whether or not the subject is impersonating.

  Here, for example, the degree of coincidence matches the direction instructed from the mobile terminal M3 and the direction in which the mobile terminal M3 has moved based on this instruction (the moving direction of the mobile terminal M3 determined from the movement vector). It may be expressed by two choices, that is, the case of not matching, and impersonation can be detected using this.

  As described above, since the living body detection device 3 includes the action unit P11, the acquisition unit P21, the calculation unit P33, and the determination unit P41 described above, it is assumed to present a photograph or a moving image such as a fingerprint or a palm print that has been captured in advance. Even for an attack, impersonation can be easily detected using the camera 14 attached to the mobile terminal M3. In particular, since the living body detection device 3 calculates the degree of coincidence according to the moving direction of the mobile terminal M3 as described above, impersonation can be detected more reliably.

[Fourth Embodiment]
The living body detection apparatus 4 of the present embodiment is roughly configured by an action part P14, an acquisition part P21, a calculation part P34, and a determination part P41. The living body detection apparatus 4 is different from the first embodiment in the configuration of the action part P14 and the calculation part P34. Note that the hardware configuration other than the acquisition unit P21, the determination unit P41, and the light emitting device 152 is the same as that of the first embodiment, and thus detailed description thereof is omitted.

  The action part P14 acts on the subject a by irradiating the subject a with a plurality of times at a predetermined timing.

  The calculation unit P34 calculates the degree of coincidence between the action of the action part P14 on the subject a (in this embodiment, irradiation of light to the subject a) and the time-series images taken by the acquisition part P21. The degree of coincidence calculated by the calculation unit P34 of the living body detection device 4 includes the timing of light irradiation multiple times and the change in luminance value of the subject a obtained from the time-series images captured by the acquisition unit P21. Is used to calculate. Specifically, the calculation unit P34 of the living body detection device 4 compares the timing of the LED irradiation with the timing of the luminance change of the subject a photographed by the camera 14 to obtain the degree of coincidence.

  Next, the hardware configuration of the living body detection device 4 will be described. The living body detection device 4 includes a light emitting device 152 together with / with the voice output device 151 of the living body detection device 1 described above. The light emitting device 152 is not particularly limited as long as it can irradiate the subject a a plurality of times at a predetermined timing. For example, a light emitting element such as an LED can be employed.

  Next, a specific flow of living body detection performed using the living body detection device 4 will be described with reference to FIGS. In the living body detection by the living body detection device 4, impersonation detection of the subject a is performed by confirming the degree of coincidence (synchronization) between the timing of light irradiation by the mobile terminal M4 and the timing of the luminance value change of the subject a.

  First, as shown in FIG. 7, the light emitting device 152 attached to the mobile terminal M4 irradiates the subject (finger or palm) a with light at a predetermined timing (step S701). At this time, the timing (time) of the irradiation is recorded. Here, in the schematic diagram shown in FIG. 8, (a) shows a screen G41 when light is not irradiated, and (b) shows a screen G42 when light is irradiated.

  Next, the ratio of the area where the change in the luminance value is equal to or greater than the threshold is calculated with respect to the entire screen of the mobile terminal M4 (step S702), and impersonation is determined using the ratio of the area (step S703). At this time, if the ratio of the area is equal to or greater than the threshold value, it is determined that there is a possibility that an impersonated display display or a printed display on paper is presented. The detection is terminated (step S708).

  On the other hand, when the ratio of the region is less than the threshold, the peak timing of the change in luminance value is calculated, and the degree of coincidence (synchronization) with the timing of LED irradiation of the mobile terminal M4 is obtained using the method described later (S704). ) And impersonation is determined using this degree of coincidence (step S705). At this time, if the peak timing of the change in luminance value and the timing of LED irradiation are not synchronized, it is determined that there is a possibility of impersonation, so that the biometric detection ends as authentication NG without proceeding to the subsequent processing. (Step S709).

  On the other hand, if the timing is synchronized, the process proceeds to biometric authentication processing (step S706) and authentication is performed (step S707). This biometric authentication process can be executed using a known technique similar to that of the first embodiment. In the biometric authentication process, usually, biometric information to be authenticated (for example, a finger area) is extracted from the entire image. However, in the biometric authentication process in step S706, only the area where the luminance value changes is targeted. May be.

  Next, an example of a method for confirming the degree of coincidence (synchronization) between the timing of light irradiation by the mobile terminal M4 and the timing of change of the luminance value of the subject a will be described. FIG. 9 is a schematic diagram for explaining a living body detection method according to the fourth embodiment. A graph K1 in FIG. 9A shows changes in luminance values due to LEDs, the horizontal axis indicates time, and the vertical axis indicates luminance values.

  FIG. 9B shows a synchronization confirmation method. In FIG. 9B, graph K2 represents time-series data expressing the luminance peak time value as “1” and the other time values as “0” with respect to the change in the luminance value in FIG. 9A. (Hereinafter, the time when the value is 1 is referred to as “bit”). Therefore, in this graph K2, the horizontal axis represents time, and the vertical axis represents the magnitude (0 or 1) of brightness (for example, time 903 represents the peak time of the brightness value of the captured image of the subject a). On the other hand, the graph K3 represents time-series data in which the time value when the mobile terminal M4 irradiates the subject a with light is expressed as “1”, and the other time values are expressed as “0”. Therefore, in this graph K3, the horizontal axis represents time and the vertical axis represents the presence or absence of irradiation (0 or 1).

  Next, the relationship between the light irradiation and the luminance value is illustrated according to FIG. 9B. In FIG. 9B, a time difference 911 represents a time difference between the luminance value peak time 903 of the subject a and the LED irradiation time 905 by the mobile terminal M4. When checking the degree of coincidence (synchronization), one of the graphs K2 and K3 is correlated with the time having the value “1” while shifting the X axis between −Δt to Δt with respect to the other. After extraction (simply matching the bits with the closest time), the total value of the time differences between the corresponding bits is calculated.

  If there is a bit that does not have a corresponding bit, for convenience, the closest bits are made to correspond to each other (thus, it is not always one-to-one correspondence). For example, if there is a correspondence between the luminance value peak time 903 of the subject a and the LED irradiation time 905 by the mobile terminal M4, and the time of the graph K3 is delayed by δt, the difference is calculated by the following equation (3). Is done.

  In the above formula (3), t represents time and δt represents time difference.

  Next, the determination unit P41 determines the presence / absence of impersonation. At this time, if the value obtained by the above equation (3) is equal to or greater than a preset threshold value, the change in the luminance value of the subject a and the timing of the LED irradiation of the mobile terminal M4 are not synchronized (impersonation is possible). Determined). That is, the degree of coincidence in this example can be expressed by two choices of a case where the value obtained by the above formula (3) is greater than or equal to a preset threshold value and a case where the value is less than the threshold value. Accordingly, it can be confirmed that what the mobile terminal M4 is photographing is not a reproduction of a moving image obtained by photographing a finger or a hand.

  As described above, the living body detection apparatus 4 includes the above-described action unit P14, acquisition unit P21, calculation unit P34, and determination unit P41, so that it pretends to present photographs and videos such as fingerprints and palm prints that have been taken in advance. Even for an attack, impersonation can be easily detected using the camera 14 attached to the mobile terminal M4. In particular, since the living body detection device 4 calculates the degree of coincidence based on the change in the luminance value of the subject a as described above, it can detect impersonation more reliably.

  In the fourth embodiment described above, the ratio of the area where the change in luminance value is greater than or equal to the threshold is calculated for the entire screen of the mobile terminal M4 (step S702), and the possibility of impersonation based on the ratio of this area is calculated. Although it was determined (step S703), the living body detection device that does not perform these steps S702 and S703 may be used.

[Fifth Embodiment]
The living body detection device 5 of this embodiment schematically further includes an action part P11, an acquisition part P21, a calculation part P31, a determination part P45, and a first image determination part P55. The living body detection device 5 is different from the first embodiment in that it includes a determination unit P45 and a first image determination unit 55. In the present embodiment, the configuration other than the first image determination unit 55 and the determination unit 45 is the same as that of the first embodiment, and thus detailed description thereof is omitted.

  The first image determination unit P55 binarizes the image captured by the acquisition unit P21, and obtains the number of connected components by performing a morphological process on the binarized image.

  The determination unit P45 determines whether or not the person to be authenticated is impersonating a valid person based on the degree of coincidence calculated by the calculation unit P31 and the number of connected components obtained by the first image determination unit P55.

  Next, a specific flow of living body detection performed using the living body detection apparatus 5 will be described with reference to FIG. The processing by the first image determination unit P55 is normally performed after determining whether or not the person to be authenticated is impersonating a legitimate person. For this reason, for example, in 1st Embodiment, it carries out between step S304 and S306 (refer FIG. 3).

  In this process, first, a direction (hereinafter also referred to as “gradient direction”) in which the luminance value changes greatly is calculated for each partial area (area) (size w × h) in the image captured by the acquisition unit P21. (Step S1001). In step S1001, not only the maximum gradient direction but also several gradient directions may be calculated.

  Next, a histogram is calculated by projecting pixel components in the gradient direction calculated for each partial region and in the vertical direction (step S1002). When a plurality of gradient directions are calculated in step 1001, the final gradient direction may be calculated by a weighted average corresponding to the strength of the gradient.

  Next, a binarization threshold value is calculated for each partial region (area) to generate a binarized image (step S1003) (see the binarized images 1102 and 1105 illustrated in FIG. 11). Next, morphological processing is performed on the binarized image created in step S1003, and island regions (connected components) are extracted (step S1004). This morphological process is usually composed of an expansion process and a contraction process. In the dilation process, the output pixel value is set to the maximum value of all pixels in the vicinity of the input pixel (for example, the vicinity of 8). On the other hand, in the contraction process, the output pixel value is the minimum value of all pixels in the vicinity of the input pixel (for example, the vicinity of 8).

  Next, after extracting island areas, the number of extracted island areas is counted (step S1005). Next, the presence / absence of impersonation is determined using the number of extracted island regions (step S1006). At this time, if the number of island areas is equal to or greater than the threshold, it is determined that there is a possibility of impersonation, and authentication NG is determined. If the number of island areas is less than the threshold, the process proceeds to subsequent biometric authentication processing.

  Here, a specific example of the island region extraction method described above will be described with reference to FIG. In FIG. 11, an image 1101 shows an example of an image obtained by photographing a real finger (subject) with the mobile terminal M5. On the other hand, the image 1104 shows an example of an image obtained by photographing a finger image (subject) printed on paper with the mobile terminal M5.

  First, the above-described photographed image is binarized. Images 1102 and 1105 show an example in which the images 1101 and 1104 are binarized in step S1003, respectively. As a method of binarization in step S1003, for example, Otsu's method (URL: http://imaginingsolution.blog107.fc2.com/blog-entry-113.html) or the like can be employed. In these binarized images, in the image 1105, the gradient vector could not be calculated well due to the influence of the texture information included in the printed matter, and therefore many areas that fail to binarize appear.

  Next, the above-described binarized image morphological processing is performed. Images 1103 and 1106 show an example in which the images 1102 and 1105 are morphologically processed in step S1004, respectively. By performing such morphological processing, island regions can be extracted, and the number of island regions described above can be counted.

  As described above, the living body detection device 5 further includes the first image determination unit 55, so that the forgery of the subject a can be determined based on the number of connected components obtained from the image. , Impersonation can be detected even more reliably.

[Sixth Embodiment]
The living body detection device 6 of this embodiment schematically further includes an action unit P11, an acquisition unit P21, a calculation unit P31, a determination unit P46, and a second image determination unit P56. The living body detection device 6 is different from the first embodiment in that it includes a determination unit P46 and a second image determination unit P56. In the present embodiment, the configuration other than the second image determination unit P56 and the determination unit 46 is the same as that of the first embodiment, and thus detailed description thereof is omitted.

  The second image determination unit P56 calculates the luminance gradient direction for each local region constituting the image from the image captured by the acquisition unit P21, and uses the calculated gradient direction for each predetermined angle. Find the quantized histogram.

  The determination unit P46 determines whether or not the person to be authenticated is impersonating a right person based on the degree of coincidence calculated by the calculation unit P31 and the variance of the frequencies in the histogram obtained by the second image determination unit P56. .

  Next, a specific flow of biological detection performed using the biological detection device 6 will be described with reference to FIG. The processing by the second image determination unit P56 is normally performed after determining whether or not the person to be authenticated is impersonating a valid person. Therefore, as in the fifth embodiment, for example, in the first embodiment, the process is performed between steps S305 and S306 (see FIG. 3).

  In this process, as in the fifth embodiment, the gradient direction is calculated for each partial region (step S1201), and a histogram in which pixel components are projected in the gradient direction and the vertical direction is calculated (step S1202). A digitized image is generated (step S1203). In addition, these calculations use the description of 5th Embodiment, and a detailed description here is abbreviate | omitted.

  Next, the gradient direction of the luminance value is calculated for the binarized image generated in step S1203 by the same process as in step S1201 (step S1204). At this time, the size (w ′ × h ′) of the partial region for calculating the gradient may be the same as or different from the size (w × h) in step S1201.

  Next, a histogram obtained by quantizing the gradient direction for each angle π / 16, for example, is calculated, and the variance of this histogram is calculated (step S1205). Next, the presence / absence of impersonation is determined using the calculated variance (step S1206). At this time, if the calculated variance is less than or equal to a certain value, it is determined that there is a possibility of impersonation, and authentication NG is determined. If the variance exceeds the certain value, the process proceeds to the subsequent biometric authentication processing. .

  As described above, the living body detection apparatus 6 further includes the second image determination unit P56, so that the forgery of the subject a can be determined based on the size of the dispersion obtained from the image, and impersonation is performed. Can be detected more reliably.

  In addition, this invention is not limited to the structure of embodiment mentioned above, is shown by the claim, and intends that all the changes within the meaning and range equivalent to a claim are included. Is done.

  For example, in the embodiment described above, the aspect in which the living body detection apparatus is incorporated in the mobile terminal has been described as an example, but may be separate from the mobile terminal as long as the effects of the present invention are not impaired. In such a case, the living body detection apparatus includes, for example, one or more computers (authentication servers) having the same configuration and the same function as the hardware configuration described above, and the computer and the mobile terminal are connected via a communication network. It is possible to employ a configuration in which a plurality of computers that share and execute similar functions are provided and the computer and the mobile terminal are connected via a communication network. In the case of executing by sharing the above, for example, the subject time-series data acquisition program is stored in one of the file servers instead of being stored in the information holding means, and the subject image, the subject time-series data, the authentication device time-series data, etc. For example, a configuration that is transmitted and received by a communication program can be used.

1-6 Living body detection device 14 Imaging device (camera)
M1 to M5 Mobile terminal a Subject P11, P14 action unit P21 acquisition unit P31 to P34 calculation unit P41, P45, P46 determination unit P55 first image determination unit P56 second image determination unit

Claims (5)

A biological detection device that detects impersonation of an authenticated person who handles a mobile terminal in cooperation with a mobile terminal having a shooting device capable of shooting a video,
An action part acting on the subject;
An acquisition unit that shoots an image including the subject in time series using the imaging device;
A calculation unit that calculates the degree of coincidence between the action of the action unit on the subject and a time-series image captured by the acquisition unit;
A living body detection apparatus comprising: a determination unit that determines whether or not the person to be authenticated impersonates the right person using the degree of coincidence calculated by the calculation unit. The action unit instructs the person to be authenticated to move the mobile terminal relative to the subject,
The degree of coincidence calculated by the calculation unit is calculated using the direction in which the mobile terminal is instructed by the action unit and the moving direction of the subject obtained from the time-series images taken by the acquisition unit. The living body detection device according to claim 1, wherein The action unit irradiates the subject multiple times with a predetermined timing,
The degree of coincidence calculated by the calculation unit is calculated using the timing and a change in luminance value of the subject obtained from a time-series image captured by the acquisition unit. Living body detection device. A first image determination unit;
The first image determination unit binarizes the image photographed by the acquisition unit, and obtains the number of connected components by performing a morphological process on the binarized image.
The living body detection device according to any one of claims 1 to 3, wherein the determination unit determines whether or not the person to be authenticated is impersonating a valid person based on the degree of coincidence and the number of connected components. A second image determination unit;
The second image determination unit calculates a gradient direction of luminance for each local region constituting the image from the image photographed by the acquisition unit, and uses the calculated gradient direction to quantize each predetermined angle. To obtain a normalized histogram,
The living body detection device according to any one of claims 1 to 3, wherein the determination unit determines whether or not the person to be authenticated is impersonating a valid person based on the degree of coincidence and the variance of the frequencies in the histogram. .