JP2020095299A - Authentication apparatus and program - Google Patents

Abstract

To provide a novel and improved authentication apparatus and a program capable of preventing "spoofing" with a simple apparatus configuration.SOLUTION: An authentication apparatus includes a first camera that picks-up a person to be authenticated, a second camera that picks-up an area including at least a part of a background of a person to be authenticated 302 picked-up by the first camera, an acquisition unit that acquires a first still-picture 420b picked-up by the first camera and a second still-picture 420a picked-up by the second camera, a person collation unit that performs person collation to the person to be authenticated, a background collation unit that compares an image of a first video area that is a video of a part of the background included in the first still-picture that is an image matching area 402 with an image of a second video area corresponding to the first video area included in the second still-picture to perform background collation, and an authentication unit that determines that authentication for the person to be authenticated succeeds when the person collation to the person to be authenticated succeeds and the background collation succeeds.SELECTED DRAWING: Figure 15

Description

The present invention relates to an authentication device and a program.

In an authentication device that acquires a face image of a person to be authenticated and performs personal authentication based on the acquired face image, by putting a photograph or a still image of the face of another person in front of the camera that acquires the face image, the other person It is possible to cheat by impersonating or "spoofing". Therefore, various measures are taken to prevent such “spoofing”. For example, Patent Document 1 below discloses an example of a technique for preventing the above "spoofing" by performing authentication using two cameras installed so as to face each other. Further, in Patent Document 2 below, by performing authentication using a moving image, a natural movement of the face (for example, blinking, a slight shake of the face, etc.) is detected, thereby preventing the above “spoofing”. Further, in Japanese Patent Application Laid-Open No. 2003-242242, the "impersonation" is prevented by changing the background of the person to be authenticated according to a predetermined rule when the face image is acquired.

JP, 2008-107942, A JP, 2004-259255, A JP, 2007-26330, A

However, in the techniques for preventing “spoofing” that have been proposed so far, “spoofing” can be performed by, for example, holding a tablet or the like displaying a moving image or the like in front of the camera. Further, in the techniques for preventing “spoofing” that have been proposed so far, it is difficult to avoid a complicated configuration of the authentication device and the like because a projection device or the like that displays a predetermined background is required.

Therefore, the present invention has been made in view of the above situation, and an object of the present invention is to provide a new and improved authentication capable of preventing “spoofing” while simplifying the device configuration. To provide a device and a program.

In order to solve the above problems, according to an aspect of the present invention, a first camera that captures an image of a person to be authenticated, and at least a part of the background of the person to be authenticated captured by the first camera are included. A second camera that captures an area; a first moving image captured by the first camera; and an acquisition unit that acquires a second moving image captured by the second camera; A person matching unit that performs person matching on the person to be authenticated based on a moving image; an image of a first image area that is a partial image of the background included in the first moving image; A background collation unit that performs background collation by comparing an image of a second video region corresponding to the first video region included in the moving image of Further, there is provided an authentication device, comprising: an authentication unit that determines that the authentication of the person to be authenticated has succeeded when the background collation is successful.

The background matching unit determines whether or not the image in the first video area included in the first moving image and the image in the second video area included in the second moving image match each other. Therefore, the background collation may be performed.

The background matching unit may extract each of a plurality of still images captured at predetermined time intervals from the first and second moving images.

The background matching unit includes an image of the first video region of the first still image included in the first moving image at a predetermined time, and a second image included in the second moving image at the predetermined time. It may be determined whether or not the image of the second video area of the still image matches.

The background matching unit includes images of the first video regions of the first still images included in the first moving image, which correspond to each other, and the second still images included in the second moving image. It may be determined that the background matching has succeeded when all the images in the second video area of the image match.

The authentication device acquires area information for acquiring position information of the first and second video areas based on an overlap between an image captured by the first camera and an image captured by the second camera. You may further provide a part.

The first camera and the second camera may be arranged vertically.

The first camera and the second camera may be arranged along the horizontal direction.

In order to solve the above problems, according to another aspect of the present invention, a computer is used to capture a first moving image captured by a first camera that captures a person to be authenticated and a first movie captured by the first camera. An acquiring unit for acquiring a second moving image captured by a second camera that captures an area including at least a part of the background of the authenticated person; and for the authenticated person based on the first moving image. A person matching unit that performs person matching by means of a person, a first image area image that is a part of the background image included in the first moving image, and the first moving image included in the second moving image. A background collation unit that performs background collation by comparing an image of a second video region corresponding to the video region, and a person collation for the person to be authenticated succeeds, and when the background collation succeeds. , And a program for functioning as an authentication unit that determines that the authentication of the person to be authenticated is successful.

As described above, according to the present invention, it is possible to provide an authentication device and a program capable of preventing “spoofing” while simplifying the device configuration.

It is a figure for demonstrating the outline of the authentication system 1 which concerns on embodiment of this invention. It is a block diagram of the authentication device 10 which concerns on the same embodiment. It is a flowchart figure of the authentication processing method which concerns on the same embodiment. It is a flowchart figure of the preset step of the authentication processing method which concerns on the same embodiment. It is a figure which shows an example of the information stored in the camera pair information table 152 which concerns on the same embodiment. FIG. 7 is an explanatory diagram (Part 1) for explaining an example of an aspect of shooting according to the same embodiment. It is an explanatory view explaining an example of still pictures 400a and 400b concerning the embodiment. It is a figure which shows an example of the information stored in the position information table 154 which concerns on the same embodiment. It is a flowchart figure of the face authentication step of the authentication processing method which concerns on the same embodiment. FIG. 7 is an explanatory diagram (part 2) explaining an example of a shooting mode according to the embodiment. It is a figure which shows an example of the information stored in the face authentication implementation management table 162 which concerns on the same embodiment. It is explanatory drawing explaining an example of the still image 410b which concerns on the same embodiment. It is a flowchart figure of the background matching stage of the authentication processing method which concerns on the same embodiment. It is a figure which shows an example of the information stored in the still image management table 172a which concerns on the same embodiment. It is an explanatory view explaining an example of still picture 420a and 420b concerning the embodiment. It is a figure which shows an example of the information preserve|saved at the background image matching result detailed table 174 which concerns on the same embodiment. It is a figure which shows an example of the information preserve|saved at the background image matching implementation management table 176 which concerns on the same embodiment. It is an explanatory view explaining an example of the mode of photography concerning the modification of the embodiment. It is an explanatory view explaining an example of still pictures 420a and 420b concerning the modification of the embodiment. It is a block diagram showing the hardware constitutions of authentication device 10 concerning an embodiment of the present invention. It is an explanatory view explaining an example of a mode of impersonation.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present specification and the drawings, constituent elements having substantially the same functional configuration are designated by the same reference numerals, and a duplicate description will be omitted. Further, in the present specification and the drawings, substantially the same or similar components may be distinguished by attaching different alphabets after the same reference numerals. However, if there is no particular need to distinguish between similar components, only the same reference numerals will be given.

In the following description, the embodiment of the present invention will be described in order to ensure security, face authentication upon entering and leaving the room, face authentication at an airport, face authentication performed by an automated teller machine, local government, etc. It can be applied to a system or the like in which the security lock is released or the device is activated by performing authentication on the person to be authenticated, such as face authentication of the resident card issuing device.

<<Background of Forming an Embodiment of the Present Invention>>
First, prior to the description of the embodiments of the present invention, the background of how the present inventor forms the embodiments of the present invention will be described with reference to FIG. FIG. 21 is an explanatory diagram illustrating an example of a spoofing mode.

As described above, for example, while authenticating with a video containing a face image, by detecting natural movements of the face (for example, blinking, subtle shaking of the face, etc.), the act of impersonating another person There is a technology to prevent fraud caused by. However, it is possible to "spoof" another person using such a technique. For example, as shown in FIG. 21, a person 304 who is going to perform a fraudulent activity is a tablet PC (personal computer) that displays a moving image including another person's face image in front of the camera 90 that acquires the face image of the person to be authenticated. Hold 92 etc. In such a case, the authentication device can perform authentication based on the face image of another person included in the acquired moving image, and can detect the natural movement of the face because it is a moving image. Therefore, the person 304 who tries to perform the fraud can succeed in the face authentication by "impersonating" the above-mentioned other person.

Further, as described above, a projection device or the like that displays a predetermined background may be required in order to prevent “spoofing”. Therefore, when trying to prevent “spoofing”, an authentication system or an authentication device is required. It is difficult to avoid complication of the configuration.

In view of the above situation, the present inventor has created an embodiment of the present invention capable of preventing “spoofing” while simplifying the device configuration. Specifically, in the embodiment of the present invention, the person matching (authentication) is performed using a moving image including a face image, and a moving image of the background of the person to be authenticated is acquired by two cameras and acquired by two cameras. By determining whether or not the images in the regions where the images should match in the moving image match, it is possible to prevent the above “spoofing”. Hereinafter, such embodiments of the present invention will be sequentially described.

<<Embodiment>>
<Outline of Authentication System 1>
First, an outline of an authentication system (authentication device) 1 according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram for explaining the outline of the authentication system 1 according to the present embodiment. As shown in FIG. 1, the authentication system 1 includes an authentication device 10 that performs face authentication, and cameras 20a and 20b for capturing a face image of a person to be authenticated 302 (see FIG. 10) and a background 300 (see FIG. 6). Including and In the following description, as shown in FIG. 1, it is assumed that two cameras 20 are installed in the authentication system 1, but the present embodiment is not limited to this. The number of cameras 20 is not limited as long as at least two cameras 20 are included. The outline of each device included in the authentication system 1 will be described below.

(Authentication device 10)
The authentication device 10 is a personal computer or the like that manages information such as a face image used for authentication and can perform face authentication in response to a request from the person to be authenticated 302 or the like. As illustrated in FIG. 1, the authentication device 10 includes a face authentication application 112 that performs face authentication (person verification) of the person to be authenticated 302, and a background matching application 114 that performs image matching (matching) of the background 300. The details of the authentication device 10 will be described later.

(Cameras 20a, 20b)
The cameras 20a and 20b are installed along the up-down direction and can take a moving image or the like of the space to be taken. It is preferable that the cameras 20a and 20b have the same various camera performances such as resolution and the like, and for example, the cameras 20a and 20b are installed apart from each other by several tens cn in the vertical direction. Furthermore, in the present embodiment, the cameras 20a and 20b can move if their position coordinates can be acquired, but it is preferable that their relative positional relationship be fixed. Specifically, for example, the camera (first camera) 20b can capture a face image of the person to be authenticated 302 (see FIG. 10). The camera (second camera) 20a can capture an area including at least a part of the background 300 of the person to be authenticated 302 captured by the camera 20b in synchronization with the camera 20b.

More specifically, for example, the cameras 20a and 20b condense the light emitted from the target space to form an optical image on the imaging surface, and convert the optical image formed on the imaging surface into an electrical image signal. An image pickup unit (for example, an image pickup optical system such as a taking lens and a zoom lens that collects light emitted from a subject, and a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor)) 200a and 200b, which are configured by image pickup devices and the like, and recording units 202a and 202b, which store captured images (moving images and still images), respectively.

Although the two cameras 20a and 20b and the authentication device 10 are illustrated as one authentication system 1 in FIG. 1, the present embodiment is not limited to such a configuration. For example, in this embodiment, the authentication system 1 may include three or more cameras 20, and the authentication device 10 includes an application server (not shown) and a database server (not shown). However, it is not particularly limited.

<Structure of Authentication Device 10>
Next, with reference to FIG. 2, a functional configuration of the above-described authentication device 10 will be described. FIG. 2 is a block diagram of the authentication device 10 according to the present embodiment. As shown in FIG. 2, the authentication device 10 according to the present embodiment mainly includes a display unit 100, a processing unit 110, and a storage unit 130. The details of each functional block of the authentication device 10 will be described below.

(Display unit 100)
The display unit 100 can display the authentication result and the like to the person to be authenticated 302. For example, a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD: Liquid Crystal Display) device, an OLED (Organic Light Emitting Diode) device, and the like. ) Can be realized by the device. Further, in the present embodiment, a plurality of display units 100 may be provided, and in this case, the other display unit 100 displays the image of the camera 20 or the like to the administrator of the authentication device 10. You can The display unit 100 may be configured as a device integrated with the authentication device 10, or may be configured as a device separate from the authentication device 10.

(Processing unit 110)
The processing unit 110 controls each functional unit of the authentication device 10 using hardware such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) built in the authentication device 10. Have the function to Specifically, as shown in FIG. 2, the processing unit 110 serves as an acquisition unit 120, a coordinate acquisition unit (area acquisition unit) 122, a person matching unit 124, a background matching unit (background matching unit) 126, and an authentication unit 128. The authentication device 10 is controlled by mainly functioning. The details of each functional unit of the processing unit 110 will be described below.

~ Acquisition unit 120 ~
The acquisition unit 120 controls the camera 20 by outputting control information such as image acquisition timing so that the camera 20 can perform image capturing. At this time, the control information such as the image acquisition timing instructed by the acquisition unit 120 may be output to the storage unit 130 described below by the acquisition unit 120 and may be stored in the storage unit 130. Then, the acquisition unit 120 acquires data and the like (first and second moving images and the like) of images captured by the cameras 20a and 20b, and a storage unit 130, a coordinate acquisition unit 122, a person collation unit 124, and a background, which will be described later. The data is output to the matching unit (background matching unit) 126.

~ Coordinate acquisition unit 122 ~
The coordinate acquisition unit 122 can previously extract a region where the images acquired by the two cameras 20a and 20b should match. Specifically, the coordinate acquisition unit 122 is based on, for example, matching (overlapping) of images of a still image 400a (see FIG. 7) captured by the camera 20a and a still image 400b (see FIG. 7) captured by the camera 20b. Position information of the image matching area (first and second video areas) 402 (see FIG. 7) that is the area of the matching image (for example, coordinate information of the image matching area 402 in each of the still images 400a and 400b). To get. Then, the coordinate acquisition unit 122 saves the position information of the image matching area 402 in the storage unit 130 by outputting the acquired position information of the image matching area 402 to the storage unit 130 described below. More specifically, for example, the coordinate acquisition unit 122 extracts the common feature points of the still images 400a and 400b and calculates the positional relationship between the extracted feature points to determine the image matching area 402 where the images match. Extract. Further, the coordinate acquisition unit 122 acquires the position information of the image matching area 402 by acquiring the coordinates specifying the image matching area 402 extracted in the still images 400a and 400b. Alternatively, the administrator of the authentication device 10 may manually overlap the two still images 400a and 400b, and the coordinate acquisition unit 122 may extract the image matching area 402 from the overlapped still images 400a and 400b. That is, in the present embodiment, the method of acquiring the position information of the image matching area 402 is not particularly limited.

~Person collation unit 124~
The person matching unit 124 can perform face authentication (person matching) on the person to be authenticated 302 based on the moving image (first moving image) of the camera 20b. For example, the person matching unit 124 acquires one still image 410b (see FIG. 12) from the moving image of the camera 20b, and extracts the feature points and the feature amounts indicating the features of the face of the person included in the still image 410b. The feature amount is the positions of the feature points such as eyes, nose, and mouth detected from the face image and their positional relationship, and is obtained by normalizing the data such as the extracted position information. Further, the person matching unit 124 compares the acquired feature amount with the feature amount of the face image stored in the storage unit 130 described later (for example, calculates the similarity of each feature amount), and the still image 410b. The person corresponding to the face image included in is stored in the storage unit 130. Then, the person matching unit 124 determines that the person matching has succeeded when the corresponding person exists in the stored information, and determines the person matching when the corresponding person does not exist in the stored information. It is determined that it has not succeeded, and the determination result is output to the background matching unit 126 and the authentication unit 128, which will be described later.

~ Background matching unit 126 ~
The background matching unit 126 executes background matching (background matching) when the person matching unit 124 succeeds in face authentication for the person to be authenticated 302. In the background matching, the image of the image matching area 402 which is a part of the image of the background 300 included in the moving image acquired by the camera 20b is compared with the image of the image matching area 402 included in the moving image acquired by the camera 20a. , It is possible to determine whether these images match. Then, the background matching unit 126 performs background matching when the image in the image matching area 402 included in the moving image acquired by the camera 20b and the image in the image matching area 402 included in the moving image acquired by the camera 20a match. Determined as successful. On the other hand, the background matching unit 126 performs background matching when the image in the image matching area 402 included in the moving image acquired by the camera 20b does not match the image in the image matching area 402 included in the moving image acquired by the camera 20a. It is determined that it was not successful. Further, the background matching unit 126 outputs the determination result to the authentication unit 128 described later.

More specifically, the background matching unit 126 extracts a plurality of still images 420a and 420b (see FIG. 15) captured at predetermined time intervals from the moving images acquired by the cameras 20a and 20b, respectively. Next, the background matching unit 126 and the image of the image matching area (first video area) 402 of the still image (first still image) 420b included in the moving image acquired by the camera 20b at a predetermined time, and the predetermined image. It is determined whether or not the image in the image matching area (second video area) 402 of the still image 420a (second still image) included in the moving image acquired by the camera 20a at the time (that is, the same time) matches. To do. The determination is performed for each pair of corresponding still images 420a and 420b at the same time. Then, the background matching unit 126 determines that the background matching has succeeded when the images in the image matching area 402 of the still images 420a and 420b match in all the pairs of still images 420a and 420b at the same time. On the other hand, the background matching unit 126 determines that the background matching has not succeeded if there is a pair of still images 420a and 420b that do not match even at the same time.

~ Authentication unit 128 ~
The authentication unit 128 determines that the authentication of the authenticated person 302 has succeeded when the person verification has succeeded with respect to the authenticated person 302 and the background matching has succeeded, and outputs the determination result to the display unit 100 or the like. Or control other devices (eg opening doors, unlocking security locks, etc.).

(Storage unit 130)
As illustrated in FIG. 2, the storage unit 130 is acquired by the image position information database (Data Base; DB) 150 used for performing background matching, the face image database 160 used for performing face authentication, and the cameras 20a and 20b. A background image database 170 for registering a moving image is stored. Details of these databases will be described later. Further, the storage unit 130 may store various data and programs for the authentication device 10 to execute processing.

More specifically, the storage unit 130 stores the camera position information table 152 (see FIG. 5), the position information table 154 (see FIG. 8), and the like in the image position information DB 150. The face image DB 160 stores a face authentication execution management table 162 (see FIG. 11) and a face authentication image table (not shown) that stores images used for face authentication, and the background image DB 170 stores a still image management table. 172a (see FIG. 14), a background image matching result detail table 174 (see FIG. 16), a background image matching execution management table 176 (see FIG. 17) and the like are stored. Details of these tables will be described later.

<Outline of authentication processing method>
Next, an outline of the authentication processing method according to this embodiment will be described with reference to FIG. FIG. 3 is a flowchart of the authentication processing method according to this embodiment. More specifically, the authentication processing method according to the present embodiment includes three steps from step S100 to step S300, as shown in FIG. The details of each step will be described below.

(Step S100)
First, in step S100, the authentication system 1 performs pre-setting before performing authentication. The authentication device 10 acquires images (for example, still images 400a and 400b (see FIG. 7)) that will be the background 300 of the person to be authenticated 302 (see FIG. 10) with the two cameras 20a and 20b, and these images are acquired. The position information (see FIG. 7) of the area image matching area 402, which is the image area that matches in (1), is acquired and saved.

(Step S200)
Next, in step S200, the authentication device 10 acquires a moving image including the face image of the person to be authenticated 302, and performs face authentication based on the acquired face image.

(Step S300)
Further, in step S300, the authentication device 10 performs background matching by using the success of face authentication in step S200 as a trigger.

The details of each of steps S100, S200, and S300 described above will be sequentially described below.

<Pre-setting stage>
Next, the presetting according to the present embodiment will be described with reference to FIGS. 4 to 8. FIG. 4 is a flowchart diagram of a preset stage of the authentication processing method according to the present embodiment. FIG. 5 is a diagram showing an example of information stored in the camera pair information table 152 according to this embodiment. FIG. 6 is an explanatory diagram illustrating an example of a shooting mode according to the present embodiment, and specifically shows a shooting mode at the time of setting. FIG. 7 is an explanatory diagram illustrating an example of the still images 400a and 400b according to the present embodiment. FIG. 8 is a diagram showing an example of information stored in the position information table 154 according to this embodiment.

At the pre-setting stage, the authentication device 10 determines the position of the image matching area 402 which is an area where the images of the still images 400a and 400b (see FIG. 7) acquired by the cameras 20a and 20b (see FIG. 7) match. Information (for example, coordinates) is acquired and stored. Specifically, the presetting step includes steps S101 to S111 as shown in FIG. The details of each step will be described below.

First, the authentication device 10 refers to the information stored in the camera pair information table 152 to identify the cameras 20 that form a pair (combination).

For example, as shown in FIG. 5, the camera pair information table 152 is a pair of cameras that can acquire the image of the same background 300 among the plurality of cameras 20 included in the authentication system 1 according to the present embodiment. To save the information. More specifically, since the camera pair information table 152 can acquire the image of the same background 300 with the camera 20a and the camera 20b, for example, the combination of the cameras 20a and 20b is associated with the pair number “0001”. I keep it. Then, the authentication device 10 instructs the pair of cameras 20a and 20b to capture a still image.

(Step S101)
The camera 20a captures a still image 400a (see FIG. 7).

(Step S103)
The camera 20b captures a still image 400b (see FIG. 7) so as to be synchronized with the camera 20a.

Note that, in the present embodiment, as shown in FIG. 6, the cameras 20a and 20b are installed along the vertical direction and are capable of shooting the background 300 that is the shooting target.

(Step S105)
Next, the authentication device 10 displays the still images 400a and 400b acquired in steps S101 and S103 side by side.

(Step S107)
Furthermore, the authentication device 10 shifts the still images 400a and 400b displayed side by side in the vertical direction so that a part of the images coincide with each other and superimposes them.

(Step S109)
The authentication device 10 completes the overlapping operation when a part of the two still images 400a and 400b completely match. For example, at the time when the superposition operation is completed, the still images 400a and 400b are in the state as shown in FIG. In this state, the images of the still images 400a and 400b coincide with each other in the region 402 located at the center in the vertical direction in the drawing.

Therefore, the authentication device 10 extracts the area where the images match as the image matching area 402, acquires the position coordinates of the extracted image matching area 402 in the still image 400a, and uses the acquired position coordinates as the position information. Save in table 154 (see FIG. 8).

(Step S111)
Further, the authentication device 10 acquires the position coordinates of the extracted image matching area 402 in the still image 400b, and stores the acquired position coordinates in the position information table 154 (see FIG. 8).

For example, as shown in FIG. 8, the position information table 154 stores the position coordinates of the image matching area 402 where the images match in the combination of the cameras 20 defined by the camera pair information table 152 described above. More specifically, the camera pair information table 152 includes, for example, a still image in the image matching area 402, which is an area where the image of the still image 400a acquired by the camera 20a and the image of the still image 400b acquired by the camera 20b match. The position coordinates in the images 400a and 400b are stored respectively.

As described above, the position information of the image matching area 402 used for background matching can be acquired and stored in advance.

<Face recognition stage>
Next, the face authentication step according to this embodiment will be described with reference to FIGS. 9 to 12. FIG. 9 is a flowchart of the face authentication step of the authentication processing method according to this embodiment. FIG. 10 is an explanatory diagram illustrating an example of a shooting mode according to the present embodiment, and specifically shows a shooting mode in face authentication in detail. FIG. 11 is a diagram showing an example of information stored in the face authentication implementation management table 162 according to the embodiment. FIG. 12 is an explanatory diagram illustrating an example of the still image 410b according to the present embodiment.

In the present embodiment, face authentication is performed before background matching is performed, and whether or not background matching is performed is determined according to the result of face authentication, thereby saving processing load and processing time. However, the present embodiment is not limited to this, and may be performed at the same time, the order of performing may be changed, and there is no particular limitation.

Specifically, the face authentication method according to this embodiment includes steps S201 to S219, as shown in FIG. The details of each step will be described below.

First, the person to be authenticated 302 stands in front of the camera 20b, and the face authentication application 112 starts. For example, as shown in FIG. 10, the authenticated person 302 stands in front of the camera 20b with the background 300 as a back. In the present embodiment, since the camera 20a is located above the camera 20a, it is possible to obtain only the image of the head portion of the person to be authenticated 302, or the image of the person to be authenticated 302 is displayed. You will not be able to obtain it.

(Step S201)
The authentication device 10 acquires a unique face authentication implementation number.

(Step S203)
The authentication device 10 determines a recording start time and a recording end time for recording a moving image including a face image of the person to be authenticated 302, and associates the recording start time and the recording end time with the face authentication execution number acquired in step S201 in the face authentication execution management table 162. save. The recording time is preferably set to, for example, about 5 to 10 seconds.

For example, as shown in FIG. 11, the face authentication implementation management table 162 stores various information for managing the implemented authentication. Specifically, as illustrated in FIG. 11, the face authentication implementation management table 162 includes a face authentication implementation number and a video recording time (for example, “recording start time”) of a video of the person to be authenticated 302 for performing face authentication. "Recording end time", etc.) are associated with each other and saved. Further, the face authentication execution management table 162 can also store the result of face authentication obtained by the steps described below and the result of background matching.

(Step S205)
The authentication device 10 instructs the camera 20a to start and end recording (for example, "recording start time" and "recording end time").

(Step S207)
The authentication device 10 instructs the camera 20b to start and end recording (for example, “recording start time” and “recording end time”) so as to synchronize with the camera 20a.

(Step S209)
The cameras 20a and 20b record a moving image from the instructed recording start time to the recording end time.

(Step S211)
The authentication device 10 acquires moving images from the cameras 20a and 20b, and stores the acquired moving images in association with the face authentication implementation number acquired in step S201.

(Step S213)
The authentication device 10 acquires the still image 410b from the moving image acquired by the camera 20b, and stores it as face authentication data. For example, the still image 410b is a still image as shown in FIG. 12, and includes a face image of the person to be authenticated 302.

(Step S215)
The authentication device 10 collates the face image included in the still image 410b acquired in step S213 with the face image stored in advance in the face image database 160 to perform face authentication. At this time, the authentication device 10 determines the size, the sharpness, and the like of the face image included in the still image 410b acquired in step S213. Alternatively, the moving image may be recorded again by guiding the user to the position near 20b.

(Step S217)
The authentication device 10 determines whether the face image included in the still image 410b acquired in step S213 and the face image stored in advance in the face image database 160 match, that is, whether the face authentication is successful. If the face authentication is successful, the process proceeds to step S219, and if the face authentication is not successful, the process proceeds to step S221.

(Step S119)
The authentication device 10 activates the background matching application 114 to perform background matching. At this time, the authentication device 10 may notify the cameras 20a and 20b of the start time and end time of recording and the face authentication implementation number acquired at the start of authentication to acquire a corresponding moving image.

(Step S221)
The authentication device 10 terminates the process and notifies the person to be authenticated 302 that the face authentication was not successful (NG).

As described above, the authentication device 10 performs the face authentication before performing the background matching, and performs the background matching when the face authentication is successful. In this embodiment, by doing so, the processing load and processing time can be saved.

<Background matching stage>
Next, the background matching method according to this embodiment will be described with reference to FIGS. 13 to 17. FIG. 13 is a flowchart of the background matching step of the authentication processing method according to this embodiment. FIG. 14 is a diagram showing an example of information stored in the still image management table 172a according to this embodiment. FIG. 15 is an explanatory diagram illustrating an example of the still images 420a and 420b according to the present embodiment. FIG. 16 is a diagram showing an example of information stored in the background image matching result detail table 174 according to this embodiment. FIG. 17 is a diagram showing an example of information stored in the background image matching execution management table 176 according to this embodiment.

In the background matching stage, the authentication device 10 uses the images in the image matching area 402 of the still images 420a and 420b (see FIG. 15) extracted from the moving images acquired by the two cameras 20a and 20b in order to prevent “spoofing”. Are determined to match. Specifically, the background matching method according to this embodiment includes steps S301 to S323 as shown in FIG. The details of each step will be described below.

(Step S301)
The authentication device 10 receives a background matching start instruction from the face authentication application 112.

(Step S303)
The authentication device 10 acquires a moving image associated with the notified face authentication implementation number from the plurality of moving images recorded in the camera 20a, and saves the moving image as background matching data.

(Step S305)
The authentication device 10 acquires a moving image associated with the notified face authentication implementation number from a plurality of moving images recorded in the camera 20b, and stores it as background matching data.

(Step S307)
The authentication device 10 extracts a plurality of still images 420a and 420b for each predetermined unit time (for example, 1 second, 0.5 seconds, 2 seconds, etc.) from the acquired moving image, and extracts the still images 420a and 420b at the earlier time. Save the last number as n with a serial number.

Specifically, as shown in FIG. 14, the still image management table 172a stores a plurality of still images 420a extracted from the moving image acquired by the camera 20a. Each still image 420a is stored in association with the face authentication implementation number corresponding to the moving image and the branch number assigned to each still image 420a. Further, the plurality of still images 420b extracted from the moving image acquired by the camera 20b are also stored in the still image management table 172 in the same manner.

(Step S309)
The authentication device 10 sets i=1.

(Step S311)
The authentication device 10 compares i set in step S309 with n set in step S307, and determines whether i is n or less. If i is n or less, the process proceeds to step S303, and if i is greater than n, the process proceeds to step S323.

(Step S313)
The authentication device 10 compares the image in the image matching area 402 of the still image 420a captured by the camera 20a with the image in the image matching area 402 of the still image 420b captured by the camera 20b. For example, as shown in FIG. 15, since the still image 420a obtained by the camera 20a and the still image 420b obtained by the camera 20b are acquired at the same time, if "spoofing" is not performed, the image matching area 402 The images should match each other.

(Step S315)
If the image of the image matching area 402 of the still image 420a captured by the camera 20a and the image of the image matching area 402 of the still image 420b captured by the camera 20b match, the authentication device 10 proceeds to step S317. move on. On the other hand, if the image in the image matching area 402 of the still image 420a captured by the camera 20a and the image in the image matching area 402 of the still image 420b captured by the camera 20b do not match, the authentication device 10 performs the step. Proceed to S319.

(Step S317)
When the image matching area 402 of the still image 420a captured by the camera 20a and the image matching area 402 of the still image 420b captured by the camera 20b match, the authentication device 10 recognizes the i-th data. Register that matching is OK.

(Step S319)
If the image matching area 402 of the still image 420a captured by the camera 20a and the image matching area 402 of the still image 420b captured by the camera 20b do not match, the authentication device 10 determines that the i-th data And register that it is a matching NG.

Specifically, the authentication device 10 stores the registration information in step S317 and step S319 in the background image matching result detail table 174 as shown in FIG. The background image matching result detail table 174 stores the result of whether or not the images in the image matching region 402 match for each pair of corresponding still images 420a and 420b (at the same time). For example, the authentication apparatus 10 compares the images in the image matching area 402 of the still images 420a and 420b having the same branch number stored in the still image management table 172 described above, and if the images match, “OK”, the image If the two do not match, the information “NG” is saved.

(Step S321)
The authentication device 10 sets the value of i+1 to i, and returns to step S311 described above.

(Step S323)
When i is larger than n, the authentication device 10 exits from the loop illustrated in FIG. 13 and refers to the background image matching result detail table 174 to confirm the background matching result. More specifically, the authentication apparatus 10 sets the background matching result to NG when at least one of the pairs of still images 420a and 420b is NG, and all the pairs of still images 420a and 420b are OK. In this case, the background matching result is OK.

Then, the authentication device 10 stores the background matching result in the background image matching execution management table 176. For example, as shown in FIG. 17, the background image matching execution management table 176 stores the result of background matching for each face authentication number.

Further, the authentication device 10 refers to the background image matching execution management table 176, determines that the authentication is successful (OK) if the result of the background matching is OK, and determines that the result of the background matching is NG. It is determined that the authentication has not succeeded (NG), and the authenticated person 302 is notified of the authentication result.

As described above, in the present embodiment, the authentication device 10 can save the processing load and the processing time by performing the background matching using the still images 420a and 420b extracted from the moving image. Note that the time interval when extracting the still images 420a and 420b may be appropriately changed according to the processing load and the processing time. Note that the embodiment of the present invention is not limited to such matching of the still images 420a and 420b, and may be performed directly between moving images.

In the present embodiment, background matching is performed to prevent “spoofing”, but the background 300 of the authenticated person 302 at this time is changed such that a passerby is included. It is preferable that However, when such a change in the background 300 cannot be expected, the authentication system 1 controls the lighting device (not shown) that illuminates the background 300 to blink so that the image of the background 300 changes. You can In addition, when the blinking interval of the lighting equipment is known in advance, the authentication system 1 determines the still images 420a and 420b when the illumination is on and the still images 420a and 420b when the illumination is off. A plurality of still images 420a and 420b may be extracted from the moving image at a time interval that allows acquisition of

<<Summary>>
As described above, in the present embodiment, the face authentication is performed using the moving image including the face image, the images of the background 300 of the person to be authenticated 302 are acquired by the two cameras 20a and 20b, and the two cameras 20a and 20a are used. It is possible to prevent “spoofing” by determining whether the images in the image matching area 402 among the images of the background 300 acquired in 20b match.

Specifically, in the present embodiment, by using the moving images acquired by the two cameras 20a and 20b, it is possible to prevent “spoofing” due to, for example, holding the tablet PC 92 or the like displaying the moving images or the like over the camera 20b. .. That is, according to the present embodiment, it is possible to carry out more secure and reliable personal authentication. In particular, by applying this embodiment, it is possible to replace the work that cannot be performed unless it is originally face-to-face with non-face-to-face work, which can lead to personnel reduction measures such as reducing the number of work workers. ..

Further, according to the present embodiment, although it is required to install two or more cameras 20a and 20b, it is possible to implement with a simple configuration, so that it is possible to suppress an increase in manufacturing cost and introduction cost.

Further, in the above description, the face image is used for authentication, but the present embodiment is not limited to this, and may be an image of the body of the person to be authenticated 302. The authentication is not limited to the authentication based on the image 410b, and may be the authentication based on the moving image.

<<Variations>>
Although the cameras 20a and 20b have been described as being arranged in the vertical direction in the above description, the cameras 20a and 20b are limited to being arranged in the vertical direction in the embodiment of the present invention. Not a thing. Therefore, a modified example of the present embodiment will be described with reference to FIGS. 18 and 19. FIG. 18 is an explanatory diagram illustrating an example of a shooting mode according to a modified example of the present embodiment, and FIG. 19 is an explanatory diagram illustrating an example of still images 420a and 420b according to a modified example of the present embodiment. is there.

As shown in FIG. 18, in this modification, the cameras 20a and 20b are arranged along the horizontal direction. In such a case, the cameras 20a and 20b have still images 420a and 420b (may be moving images) as shown in FIG. 19 and an image matching area 402a common to these still images 420a and 420b. That is, in the present modification, the image matching area 402a where the images match is located at the center in the left-right direction in the drawing. Even in this modification, the authentication can be performed in the same manner as in the above-described embodiment.

<<Hardware configuration of the authentication device 10 according to the present embodiment>>
The embodiments of the present invention have been described above. Next, the hardware configuration of the authentication device 10 according to the present embodiment will be described with reference to FIG. FIG. 20 is a block diagram showing the hardware configuration of the authentication device 10 according to the embodiment of the present invention.

As shown in FIG. 20, the authentication device 10 includes a CPU 950, a ROM 952, a RAM 954, an internal bus 956, an input/output interface 958, an input device 960, an output device 962, an HDD (Hard Disk Drive) 964, a network interface 966, a drive 970, and a drive 970. It has a connection port 974.

(CPU950)
The CPU 950 is configured by a processor such as a microprocessor, and controls overall operations in the authentication device 10 according to various programs as an arithmetic processing device and a control device. The CPU 950 realizes, for example, a function as the above-described processing unit 110 and the like.

(ROM952, RAM954)
The ROM 952 stores programs used by the CPU 950, calculation parameters, and the like. Further, the RAM 954 temporarily stores a program used in the execution of the CPU 950, parameters that appropriately change in the execution, and the like. The ROM 952 and the RAM 954 realize the functions of the storage unit 130 and the like described above.

(Internal bus 956)
The internal bus 956 is composed of a CPU bus and the like. The internal bus 956 connects the CPU 950, the ROM 952, and the RAM 954 to each other.

(Input/output interface 958)
The input/output interface 958 connects the HDD 964 and the network interface 966 to the internal bus 956. For example, the HDD 964 exchanges data with the RAM 954 and the like via the input/output interface 958 and the internal bus 956.

(Input device 960)
The input device 960 is configured as an example of an input unit for previously inputting various parameters (for example, information on the camera 20 to be paired) and the like to the authentication device 10. The input device 960 includes, for example, input means for the user to input parameters to the authentication device 10 in advance, an input control circuit that generates an input signal based on the input by the user, and outputs the input signal to the CPU 950.

(Output device 962)
The output device 962 is configured as an example of the display unit of the authentication device 10. The output device 962 includes a display device such as a liquid crystal display device or an OLED (Organic Light Emitting Diode) device as a display unit. This display device displays the result of authentication and the like obtained by the CPU 950.

(HDD964)
The HDD 964 is a data storage device that realizes the function of the storage unit 130 and the like described above. The HDD 964 can also store programs executed by the CPU 950 and various data.

(Network interface 966)
The network interface 966 is a communication interface including a communication device for communicating with the camera 20.

(Drive 970)
The drive 970 is a reader/writer for a removable recording medium 972 such as a magnetic disk, an optical disk, or a semiconductor memory, and is built in or externally attached to the authentication device 10. The drive 970 can write information in the mounted removable recording medium 972. The drive 970 can also read the information recorded in the mounted removable recording medium 972 and output it to the RAM 954.

(Connection port 974)
The connection port 974 is a port for directly connecting an external device (not shown) to the authentication device 10. The connection port 974 is, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface) port, or the like. By connecting an external device to the connection port 974, the connection port 974 can exchange various data between the authentication device 10 and the external device.

Heretofore, an example of the hardware configuration of the authentication device 10 according to the present embodiment has been shown. Each component described above may be configured by using a general-purpose member, or may be configured by hardware specialized for the function of each component. Such a configuration can be appropriately changed according to the technical level at the time of implementation.

<<Supplement>>
Although the preferred embodiments of the present invention have been described above in detail with reference to the accompanying drawings, the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various alterations or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

Further, the steps in the authentication processing method according to the embodiment of the present invention described above do not necessarily have to be processed in the order described. For example, the order of the steps may be appropriately changed and processed. Further, each step may be processed partially in parallel or individually instead of being processed in time series. Further, the processing method of each step does not necessarily have to be processed according to the described method, and may be processed by another function unit by another method, for example.

Furthermore, at least a part of the authentication processing method according to the above-described embodiment can be configured by software as an information processing program that causes a computer to function, and when configured by software, at least one of these methods is used. A program that realizes the unit is stored in a recording medium such as a flexible disk or a CD-ROM (Compact Disc Read Only Memory), and is read by the authentication device 10 or another device connected to the authentication device 10 and executed. May be. The recording medium is not limited to a removable medium such as a magnetic disk or an optical disk, and may be a fixed recording medium such as a hard disk device or a memory. Further, a program that realizes at least a part of these methods may be distributed via a communication line (including wireless communication) such as the Internet. Further, the program may be distributed in the state of being encrypted, modulated, or compressed, via a wired line such as the Internet or a wireless line, or stored in a recording medium.

1 Authentication System 10 Authentication Device 20a, 20b, 90 Camera 92 Tablet PC
100 display unit 110 processing unit 112 face authentication application 114 background matching application 120 acquisition unit 122 coordinate acquisition unit 124 person matching unit 126 background matching unit 128 authentication unit 130 storage unit 150 image position information database 152 camera pair information table 154 position information table 160 Face image database 162 Face authentication implementation management table 170 Background image database 172a Still image management table 174 Background image matching result detail table 176 Background image matching implementation management table 200a, 200b Imaging unit 202a, 202b Recording unit 300 Background 302 Authenticated person 304 Person 400a, 400b, 410a, 420a, 420b Still image 402, 402a Image matching area 950 CPU
952 ROM
954 RAM
956 Internal bus 958 Input/output interface 960 Input device 962 Output device 964 HDD
966 Network interface 970 Drive 972 Removable recording medium 974 Connection port

Claims (9)

A first camera for photographing the authenticated person,
A second camera for capturing an area including at least a part of the background of the person to be authenticated captured by the first camera;
An acquisition unit that acquires a first moving image shot by the first camera and a second moving image shot by the second camera;
A person matching unit that performs person matching on the person to be authenticated based on the first moving image;
An image of a first video area which is a video of a part of the background included in the first video, and a second video area corresponding to the first video area included in the second video. A background matching unit that performs background matching by comparing the image of
An authentication unit that determines that the person to be authenticated has succeeded, and, if the background collation has succeeded, determines that the person to be authenticated has succeeded in authentication.
An authentication device including. The background matching unit determines whether or not the image in the first video area included in the first moving image and the image in the second video area included in the second moving image match each other. The authentication device according to claim 1, wherein the background verification is performed by doing so. The authentication device according to claim 2, wherein the background matching unit extracts a plurality of still images captured at predetermined time intervals from the first and second moving images, respectively. The background matching unit includes an image of the first video region of the first still image included in the first moving image at a predetermined time, and a second image included in the second moving image at the predetermined time. The authentication device according to claim 3, wherein it is determined whether or not the image of the second video area of the still image matches. The background matching unit includes images of the first video regions of the first still images included in the first moving image, which correspond to each other, and the second still images included in the second moving image. The authentication device according to claim 4, wherein the background collation is determined to be successful when all of the images in the second video area of the image match. An area acquisition unit that acquires position information of the first and second video areas based on an overlap between an image captured by the first camera and an image captured by the second camera, The authentication device according to claim 1. The authentication device according to claim 1, wherein the first camera and the second camera are arranged along a vertical direction. The authentication device according to claim 1, wherein the first camera and the second camera are arranged along a horizontal direction. Computer,
A first video captured by a first camera that captures a person to be authenticated, and a second camera that captures an area including at least a part of the background of the subject captured by the first camera An acquisition unit that acquires the second video captured in
A person matching unit that performs person matching on the person to be authenticated based on the first moving image;
An image of a first video area which is a video of a part of the background included in the first video, and a second video area corresponding to the first video area included in the second video. A background matching unit that performs background matching by comparing the image of
An authentication unit that determines that the person to be authenticated has succeeded, and, if the background collation has succeeded, determines that the person to be authenticated has succeeded in authentication.
A program to function as.

Images