JP2023079948A - Information processing apparatus and method for controlling the same, and program - Google Patents

Abstract

To reduce reduction in authentication accuracy due to opening and closing of a door or efficiently perform face authentication and human authentication in an entry-exit management system.SOLUTION: An information processing apparatus comprises: face authentication means that performs face authentication on the basis of a feature amount of the face of a subject included in an image of the subject picked up by imaging means; biological determination means that determines if the subject is a living body by using the image; and control means that, when the face authentication means succeeds in the face authentication and the biological determination means determines that the subject is a living body, instructs an external device controlling opening and closing a door to open the door. The biological determination means determines whether to perform processing of determining if the subject is a living body by using the image acquired by image acquisition means according to the opening and closing state of the door.SELECTED DRAWING: Figure 1

Description

TECHNICAL FIELD The present invention relates to an information processing apparatus, its control method, and program, and in particular, a technique for reducing deterioration in authentication accuracy due to door opening/closing, or for efficiently performing face authentication and human body authentication in an entrance/exit management system. Regarding.

Face authentication is used as one of methods for specifying a person by image processing. 2. Description of the Related Art In recent years, an entry/exit management system has been proposed that controls opening and closing of a door by photographing a person positioned near a door using a network camera and identifying the person by performing face authentication from the photographed image.

When face recognition technology based on image processing is used, there is a problem of fraudulent acts of pretending to be another person by presenting a photograph of another person's face to the camera.

As solutions to these problems, the following techniques have been disclosed.

For example, Patent Literature 1 discloses a method of determining whether a color change extracted from a time-series image of a face is based on a pulse by comparing a score calculated by a predetermined evaluation function with a threshold value. ing.

JP 2019-194888

However, in the conventional technology disclosed in the above-mentioned Patent Document 1, noise is generated in color changes due to pulse included in time-series images of the face depending on shooting conditions such as the surrounding environment in which the face authentication system operates and camera settings. There is a risk that it will be mixed in and the judgment accuracy will decrease. In addition, when considering application to an entrance/exit management system, the spoofing determination accuracy is affected by the open/closed state of the door. For example, changes in lighting conditions due to the opening and closing of doors, shadows of passers-by, and other factors may cause noise to be mixed into the time-series images of faces that can be acquired.

The present invention has been made to solve the above-mentioned problems, and it is possible to reduce the decrease in authentication accuracy due to opening and closing of doors in an entrance/exit management system, or to efficiently perform face authentication and human body authentication. With the goal.

The present invention comprises open/closed state acquisition means for acquiring an open/closed state of a door, image acquisition means for acquiring an image of a subject captured by an imaging means, and a face of the subject included in the image acquired by the image acquisition means. face authentication means for performing face authentication based on a feature amount; biometric determination means for determining whether the subject is a living body using the image acquired by the image acquisition means; and face authentication performed by the face authentication means. and a control means for instructing an external device for controlling opening/closing of the door to open the door when the object is determined to be a living body by the living body determination means. The living body determination means determines whether the subject is a living body using the image acquired by the image acquisition means according to the open/closed state of the door acquired by the open/closed state acquisition means. It is characterized by determining whether or not to perform.

Further, the present invention includes open/closed state acquisition means for acquiring an open/closed state of a door, image acquisition means for acquiring an image of a subject captured by an imaging means, and a subject included in the image acquired by the image acquisition means. face authentication means for performing face authentication based on facial features; biometric determination means for determining whether the subject is a living body using the image acquired by the image acquisition means; a calculation means for calculating a reliability of a determination result; and the confidence calculated by the calculation means when the face authentication means succeeds in the face authentication and the biometric determination means determines that the subject is a living body. control means for instructing an external device that controls opening and closing of the door to open the door when the degree is equal to or greater than a predetermined threshold; indicates that the door is closed, the first reliability threshold is used as the predetermined threshold, and the opening/closing state acquisition means acquires A second threshold lower than the first threshold is used as the predetermined threshold when the open/closed state of the door indicates that the door is open.

Advantageous Effects of Invention According to the present invention, in an entrance/exit management system, it is possible to reduce deterioration in authentication accuracy due to door opening/closing, or to efficiently perform face authentication and human body authentication.

Block diagram showing a configuration example of an entrance/exit management system according to the present embodiment Flowchart of registrant data storage processing according to the present invention Flowchart of authentication processing at the time of entry/exit according to the first embodiment FIG. 11 is a diagram showing an output example in the first embodiment; Flowchart of details of spoofing determination processing according to the present invention Flowchart of authentication processing at the time of entry/exit according to the second embodiment

Preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. It should be noted that the present invention is not limited to the following embodiments, but only specific examples advantageous for carrying out the present invention are shown. Also, not all combinations of features described in the following embodiments are essential for solving the problems of the present invention.

<Example 1>
The image processing apparatus according to this embodiment can be applied to an image processing circuit incorporated in an imaging apparatus such as a surveillance camera, or a computer apparatus such as a personal computer. The video processing device 102 is an application example of the information processing device of the present invention.

A functional configuration of the entrance/exit management system according to the present embodiment will be described with reference to the block diagram of FIG. The entrance/exit management system 100 includes an imaging device 101 (imaging means), a video processing device 102 and an external device 103 . The video processing device 102 is provided with an image acquisition unit 111, a face registration unit 112, a face feature amount storage unit 113, a face authentication unit 114, a biometric determination unit 115, an external device control unit 116, and an analysis result output unit 117. . In this embodiment, a pattern in which these are configured in the video processing device 102 will be described, but the functional configuration is not limited to this. For example, the face registration unit 112, the face feature quantity holding unit 113, and the like may be configured in separate servers.

The imaging device 101 captures an image of a subject for face authentication and spoofing determination, and sends the image to the video processing device 102 via a network. In addition, a camera or the like built in the video processing device 102 may be used instead of being limited to the outside.

The external device 103 is a device for controlling doors managed by the entrance/exit management system. For example, a known electric lock is configured to maintain a locked state in response to an input of a lock signal and perform an unlocking operation in response to an input of an unlock signal.

A configuration of the video processing device 102 will be described.

The image acquisition unit 111 acquires a moving image or still image supplied from the imaging device 101 . In the following description, whether the video processing device 102 acquires a moving image or a still image, one image is sent to the face registration unit 112, the face authentication unit 114, and the biometric determination unit 115. will be described. In the former case, the one image corresponds to each frame forming a moving image, and in the latter case, the one image corresponds to a still image. In the case of moving images, similar processing may be repeated for each frame.

The face registration unit 112 extracts an image of a person's face area from the video received from the imaging unit 111 by image processing. Further, the face registration unit 112 performs feature amount extraction processing for extracting a face feature amount based on a predetermined calculation from an image of a person's face area. In this embodiment, a process of detecting edges of eyes, mouth, etc. from an image to detect a characteristic portion of a person's face is performed, but other embodiments are not limited to this. The extracted facial feature amount is sent to the facial feature amount holding unit 113 .

The facial feature amount holding unit 113 associates the registrant information input by the user with the facial feature amount acquired from the face registration unit 112 to generate registrant data, and stores the registrant data in the facial feature amount holding unit 113. do.

The face authentication unit 114 extracts an image of a person's face area from the video received from the imaging unit 111 by image processing. Further, the face authentication unit 114 performs a feature amount extraction process for extracting a face feature amount based on a predetermined calculation from the image of the face area of the person, and extracts the face feature amount of the person to be authenticated. The facial features of each registrant held in the holding unit 113 are collated to determine whether or not they are the same person. The determination result is sent to the external device control section 116 and the analysis result output section 117 .

The living body determination unit 115 (living body determination means) executes spoofing determination processing for determining whether or not the person detected from the image received from the imaging unit 111 is a living body. In other words, the identity of the person to be authenticated is verified, and it is determined whether or not impersonation is taking place. The spoofing determination can be performed by, for example, observing the color change due to the pulse from the time-series data of the face image, and distinguishing whether it is a photograph or a living body.

The generation determination process by the living body determining unit 115 (biological determining means) determines whether the subject is a living body by estimating the pulse of the subject included in the image.

The determination result is sent to the external device control section 116 and the analysis result output section 117 .

The external device control unit 116 controls the external device 103 based on the face authentication result received from the face authentication unit 114 and the spoofing determination result received from the biometric determination unit 115 . For example, it is connected to the external device 103 via a contact cable that transmits a contact signal, and the external device control unit 116 controls opening and closing of the door by transmitting an unlock signal/lock signal. Further, the external device control unit 116 acquires the open/closed state of the door by receiving the unlock signal/lock signal.

The analysis result output unit 117 notifies the output destination of the face authentication result received from the face authentication unit 114 and the spoofing determination result received from the biometric determination unit 115 . The output destination is not limited to a specific output destination, and may be output to an external device via a wireless or wired connection. The external device may be a display device such as a display. Also, the information to be output may be output as it is, or visible graphics may be output.

Next, details of operations performed by the video processing apparatus according to the present embodiment will be described.

A registrant information saving processing procedure for performing face authentication will be described with reference to the flowchart of FIG.

The processing of each step shown in the flowchart shown in FIG. 2 is executed by the CPU of the video processing device 102 .

In S201, the face registration unit 112 determines whether or not the user has input information about the registrant. In this embodiment, the registrant information is the face image of the registrant input by the image acquisition unit 111, and the name and sex of the registrant, but other embodiments are not limited to this. If it is determined that the registrant information has been input, the process proceeds to S202.

In S202, the face registration unit 112 checks the information input in S201. Specifically, facial feature amount extraction processing is performed on the input face image of the registrant. If the face feature amount can be extracted, it is determined that the input information is correct, and the process proceeds to S203. On the other hand, if the face feature amount to be detected cannot be extracted, it is determined that the input information is insufficient for registration, and the process returns to S201.

In S203, the face registration unit 112 associates the information on the registrant input in S201 with the facial feature amount extracted in S202, and registers them in the facial feature amount holding unit 113 as registrant information.

The authentication processing procedure at the time of entry/exit will be described with reference to the flowchart of FIG.

The processing of each step shown in the flowchart shown in FIG. 3 is executed by the CPU of the video processing device 102 .

Here, it is assumed that an authentication camera is installed in front of the door for authenticating the target person in front of the door.

In S301, the image acquiring unit 111 (image acquiring means) acquires an image input from the front door authentication camera (imaging device 101), which is the imaging device 101. FIG. S301 is an application example of an image acquisition unit that acquires an image of a subject captured by an imaging unit.

If the image has been acquired, the process proceeds to S302 and S304. In addition, hereinafter, the living/non-biological determination processing in S302 and S303 and the person determining processing in S304 are performed in parallel.

In S<b>302 , the external device control unit 116 acquires the open/closed state of the door from the external device 103 . If the door is closed, it is assumed that a frame image suitable for spoofing determination has been acquired, and the frame at that timing is sent to the living body determination unit 115 to perform living body/non-living body determination. If the door is open, it is assumed that a frame image unsuitable for spoofing determination has been acquired, and the process returns to S301.

S302 is an application example of an open/closed state acquisition unit that acquires the open/closed state of the door. Further, S302 is an application example of the living body determination means of the present invention, and whether or not the subject is a living body is determined using the image acquired in S301 according to the open/closed state of the door acquired by the open/closed state acquisition means. It is determined whether or not to perform determination processing.

Specifically, when the open/closed state of the door acquired by the open/closed state acquisition unit indicates that the door is open, the living body determination unit uses the image acquired by the image acquisition unit to determine whether the subject is If it is determined not to perform the process of determining whether the subject is a living body, and if the open/closed state of the door indicates that the door is closed, the image obtained by the image obtaining means is used to determine whether the subject is It decides to perform the process of judging whether it is a living body.

In S303, the living body determining unit 115 (living body determining means) executes a process of determining whether the person detected from the received frame image is a living body or a non-living body.

S303 is an application example of a living body determination unit that uses the image acquired in S301 to determine whether the subject included in the image is a living body.

Determination is performed using a plurality of frame image groups, and the obtained determination result is sent to the external device control unit 116 .

In S304, the face authentication unit 114 uses the image acquired in S301 to perform identity determination by face authentication. The face authentication unit 114 detects a face area from the obtained image, and extracts a feature amount from the detected face area image. The extracted facial feature amount is collated with the registrant information held in the facial feature amount holding unit 113 . If the collation is successful, it is determined that the person is the registrant, and the external device control unit 116 is notified of the result. If the collation fails, the person is determined to be an unregistered person, and the external device control unit 116 is notified of the result.

S304 is an application example of a face authentication unit that performs face authentication based on the feature amount of the subject's face included in the image acquired in S301.

In S<b>305 , the external device control unit 116 (control unit) controls the external device 103 based on the biometric/non-biometric determination result received from the biometric determination unit 115 and the identity determination result received from the face authentication unit 114 . The external device control unit 116 transmits an unlocking signal to the external device 103 when it is determined that the person is a living body and the person himself/herself. Otherwise, the process returns to S301.

S305 is an application example of the control means of the present invention. An instruction to open the door (unlock signal) is issued to the external device that controls opening and closing.

In S306, the external device 103 receives an unlocking signal from the external device control unit 116, and the door is unlocked. Also, the analysis result output unit 117 receives the result of identification by the face authentication unit 114 and the result of the biometric/non-biological determination by the biometric determination unit 115, and outputs the results to an output destination such as a tablet terminal. With reference to FIG. 4, a specific example of the analysis result output content will be described. When the external device control unit 116 performs external device control, the analysis result output unit 117 prompts the target person in front of the door to pass through the door by displaying a screen such as that shown in FIG. . In addition, the analysis result output unit 117 prompts the target person in front of the door to temporarily stand still in front of the door by displaying a screen as shown in FIG. good.

S306 is an application example of the control means of the present invention, in which face authentication is successful in S304, the subject is determined to be a living body in S303, and the reliability calculated in S503 is equal to or greater than a predetermined threshold. If there is (S504: YES), an instruction to open the door is given to the external device that controls opening and closing of the door.

Here, details of the living body/non-living body determination (S303) by the living body determination unit 115 will be described with reference to the flowchart of FIG.

The processing of each step shown in the flowchart shown in FIG. 5 is executed by the CPU of the video processing device 102 .

In S501, the subject image is acquired from S302.

In S502, it is checked whether or not the number of acquired images has reached the number required for determination processing. As the required number of images, a value set in advance is used so that the color change due to the pulse of the subject can be accurately detected. If the number of acquired images has not reached the required number, the process returns to S501 to acquire the next image. If the number of acquired images has reached the required number, the process proceeds to S503.

In S503, the acquired image group is used to determine the living/non-living body and calculate the reliability of the determination. For the determination of living body/non-living body, for example, a color change due to pulse is observed from an obtained image group to distinguish between a photograph and a living body. The reliability is calculated based on the frame rate of the acquired image group and the degree of detected color change.

S303 uses the image acquired in S301 to apply a living body determining means for determining whether the subject included in the image is a living body and a calculating means for calculating the reliability of the living body determination result by the living body determining means. For example.

In S504, it is determined whether or not the reliability calculated in S503 exceeds (or is equal to or exceeds) a preset predetermined threshold. If it is determined that the reliability exceeds the threshold, the process proceeds to S505. If it is determined that the reliability is equal to or less than (or less than) the predetermined threshold, the process returns to S501.

In S505, the living/non-living body determination result by the living body determination unit 115 is sent to the external device control unit 116, and the process proceeds to S305.

As described above, according to the present embodiment, in the entrance/exit management system, it is possible to prevent a decrease in spoofing determination accuracy regardless of whether the door is opened or closed.

<Example 2>
Only points different from the first embodiment will be described below, and the same as the first embodiment is assumed unless otherwise specified.

In the first embodiment, when the external device control unit 116 determines in S302 that the door is open, the frame image sent at that timing is used for the living body/non-living body determination by the living body determination unit 115. I explained the case where it is not used. Here, a case will be described using FIG. 6 in which the frame image at the timing at which the external device control unit 116 determines that the door is open is used for the living body/non-living body determination.

The processing of each step shown in the flowchart shown in FIG. 6 is executed by the CPU of the video processing device 102 .

Since S601, S602, S605, S606, and S607 are the same processes as S301, S302, S304, S305, and S306, respectively, description thereof is omitted. Also, S604 corresponds to S303.

If the external device control unit 116 determines in S602 that the door is closed, it is assumed that a frame image suitable for spoofing determination has been obtained, and the living/non-living determination in S604 is performed using a normal reliability threshold value. (first threshold similar to S303).

If the external device control unit 116 determines in S602 that the door is open, it is assumed that a frame image that is not suitable for spoofing determination has been acquired, and in S603 the reliability level used for biometric/non-biological determination is determined. Lower the threshold and use a second threshold that is lower than the first. In the living body/non-living body determination in S604, the living body/non-living body determination is performed using the reliability threshold lowered in S603.

In this way, when the open/closed state of the door acquired by the open/closed state acquisition unit in S602 indicates that the door is closed (S602: YES), the living body determining means in S604 sets the predetermined threshold to the If a reliability threshold of 1 is used, and if the open/closed state of the door acquired by the open/closed state acquiring means in S602 indicates that the door is open (S602: NO), the predetermined threshold is A second threshold (S603) lower than the threshold of 1 is used.

By adopting the above embodiment, the reliability of the living/non-living determination is lowered, but compared to the first embodiment, the living/non-living determination result can be obtained more efficiently in a shorter time.

<Example 3>
In the first embodiment, when the external device control unit 116 determines in S302 that the door is open, the frame image sent at that timing is used for the living body/non-living body determination by the living body determination unit 115. I explained the case where it is not used. In the second embodiment, when the external device control unit 116 determines in S302 that the door is open, the frame image sent at that timing is used for the living body/non-living body determination by the living body determination unit 115. I explained the use cases. Here, a case in which these processes are switched according to changes in illumination conditions will be described.

The entrance/exit management system according to the present embodiment varies in magnitude of change in lighting conditions due to door opening/closing depending on the environment in which it is installed. A change in lighting conditions due to opening and closing of a door, for example, due to a change in sunlight, changes regularly depending on the time of day. In such a case, it is preferable to switch between the processing flow of the first embodiment and the processing flow of the second embodiment according to the time zone of the day designated by the user.

For example, the process (control process) described in the second embodiment is executed in the first time period, and the process described in the first embodiment is executed in the second time period different from the first time period. do.

For example, the processing flow of the first embodiment is applied because it is expected that the living/non-living object determination accuracy with the door open is reduced during times of the day when there are large changes in lighting conditions. In addition, the processing flow of the second embodiment is applied because it is expected that the live/non-living determination accuracy does not deteriorate even when the door is open during the time period during which the lighting condition changes are small.

Further, the image processing apparatus 102 may include setting means for setting the first time period and the second time period according to the magnitude of illumination fluctuation.

By adopting the above-described embodiment, it is possible to perform living/non-living determination according to changes in illumination conditions.

Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and modifications and the like are possible within the scope of the present invention, and the above-described embodiments can be combined as appropriate. may

Also, part or all of the functional configuration of the system described above may be implemented as hardware in the imaging device, video processing device, or external device.

The present invention is also realized by executing the following processing. That is, a program that implements the functions of the above-described embodiments is supplied to a system or device via a network or various recording media, and a computer (CPU, MPU, etc.) of the system or device reads and executes the program code. be. In this case, the program and the recording medium storing the program constitute the present invention.

100 entrance/exit management system 101 imaging device 102 image processing device 103 external device 111 image acquisition unit 112 face registration unit 113 face feature amount holding unit 114 face authentication unit 115 biometric determination unit 116 external device control unit 117 analysis result output unit

Claims (9)

an open/closed state obtaining means for obtaining an open/closed state of the door;
an image acquiring means for acquiring an image of a subject captured by the imaging means;
face authentication means for carrying out face authentication based on facial features of a subject included in the image acquired by the image acquisition means;
a living body determining means for determining whether the subject is a living body using the image acquired by the image acquiring means;
Instructing an external device controlling opening/closing of the door to open the door when the face authentication means succeeds in face authentication and the biometric determination means determines that the subject is a living body. a control means for
with
The living body determining means performs processing for determining whether the subject is a living body using the image acquired by the image acquiring means according to the open/close state of the door acquired by the open/close state acquiring means. An information processing apparatus characterized by determining whether or not. When the open/closed state of the door acquired by the open/closed state acquisition unit indicates that the door is open, the living body determination unit uses the image acquired by the image acquisition unit to determine the subject. is a living body, and if the open/closed state of the door indicates that the door is closed, the image acquired by the image acquisition means is used 2. The information processing apparatus according to claim 1, wherein the processing for determining whether the subject is a living body is determined. an open/closed state obtaining means for obtaining an open/closed state of the door;
an image acquiring means for acquiring an image of a subject captured by the imaging means;
face authentication means for carrying out face authentication based on facial features of a subject included in the image acquired by the image acquisition means;
a living body determining means for determining whether the subject is a living body using the image acquired by the image acquiring means;
Calculation means for calculating the reliability of the result of the biometric determination by the biometric determination means;
When the face recognition means succeeds in face recognition, the biometric determination means determines that the subject is a living body, and the reliability calculated by the calculation means is equal to or greater than a predetermined threshold, a control means for instructing an external device for controlling the opening and closing of the door to open the door;
with
When the open/closed state of the door acquired by the open/closed state acquisition unit indicates that the door is closed, the living body determining means uses a first reliability threshold as the predetermined threshold, Further, when the open/close state of the door acquired by the open/close state acquiring means indicates that the door is open, a second threshold lower than the first threshold is used as the predetermined threshold. An information processing device characterized by: In the first time zone, the control process is executed by the control means,
In a second time period different from the first time period,
When the open/closed state of the door acquired by the open/closed state acquisition unit indicates that the door is open, the living body determination unit uses the image acquired by the image acquisition unit to determine the subject. is a living body, and if the open/closed state of the door indicates that the door is closed, the image acquired by the image acquisition means is used 4. The information processing apparatus according to claim 3, wherein the processing for determining whether the subject is a living body is determined. 5. The information processing apparatus according to claim 4, further comprising setting means for setting said first time period and said second time period according to the magnitude of illumination variation. 6. The generation determination process by said living body determination means determines whether said subject is a living body by estimating a pulse of said subject included in said image. The information processing device according to . an open/closed state acquiring step of acquiring the open/closed state of the door;
an image acquisition step of acquiring an image of the subject captured by the imaging means;
a face authentication step of performing face authentication based on the feature amount of the subject's face included in the image acquired by the image acquisition step;
a living body determination step of determining whether the subject is a living body using the image acquired by the image acquisition step;
Instructing an external device for controlling opening/closing of the door to open the door when the face authentication is successful in the face authentication step and the subject is determined to be a living body in the biometric determination step. a control step for
with
The living body determination step uses the image acquired by the image acquisition step to determine whether the subject is a living body according to the open/closed state of the door acquired by the open/closed state acquisition step. A control method characterized by determining whether or not an open/closed state acquiring step of acquiring the open/closed state of the door;
an image acquisition step of acquiring an image of the subject captured by the imaging means;
a face authentication step of performing face authentication based on the feature amount of the subject's face included in the image acquired by the image acquisition step;
a living body determination step of determining whether the subject is a living body using the image acquired by the image acquisition step;
a calculation step of calculating the reliability of the biometric determination result of the biometric determination step;
When face authentication is successful in the face authentication step, the subject is determined to be a living body in the biometric determination step, and the reliability calculated in the calculation step is equal to or greater than a predetermined threshold, a control step of instructing an external device that controls opening and closing of the door to open the door;
with
When the open/closed state of the door acquired by the open/closed state acquiring step indicates that the door is closed, the living body determining step uses a first reliability threshold as the predetermined threshold, Further, when the open/close state of the door acquired by the open/close state acquiring step indicates that the door is open, a second threshold lower than the first threshold is used as the predetermined threshold. A control method characterized by: A program for functioning as the information processing apparatus according to any one of claims 1 to 6.

Images