JP2022100394A - Information processing system, terminal device, authentication object management method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To register efficiently a group of authentification objects for face authentification to a system.

SOLUTION: An information processing system 1 has: a face detection unit 2 for detecting a face from an image obtained by capturing faces of a plurality of persons; a determination unit 3 for determining whether or not the face detected by the face detection unit 2 satisfies a predetermined condition; and a registration information generating unit 4 for generating registration information associating partial images of the plurality of faces determined as one satisfying the predetermined condition with an identifier for use in identifying a group of authentication objects in a face authentication.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to an information processing system, a terminal device, a management method for an authentication target, and a program.

In recent years, various systems using face recognition have been developed, and their uses are also diverse. In the face recognition system, it is necessary to register the face information of the person to be authenticated in advance in order to perform face recognition. In face authentication for identity verification purposes, face images registered as authentication targets are often registered on an individual basis. However, for example, in a stamp rally game in which all group members have reached a predetermined point as a condition for stamping, when the arrival of a participant is judged by face recognition in advance, the group unit It is required to register the face image.

As a technique related to this, there is an authentication device described in Patent Document 1. In this authentication device, a plurality of faces detected from the image are associated and registered.
However, when the face of the person to be authenticated is photographed outdoors or in a place with many people and multiple faces are detected from the image at the same time to register the face information, the face of a person other than the person to be authenticated is reflected and the face is authenticated. Faces of people other than the target person may also be erroneously associated. In such a case, it is necessary to change the shooting location and re-shoot the face image, which is troublesome.

Further, as a related technique, there is an image information processing apparatus described in Patent Document 2. This image information processing device groups a part of a plurality of people in the image based on the direction in which the person in the image is facing.

Japanese Patent No. 578309 International Publication No. 2011/148562

In the apparatus described in Patent Document 1 or Patent Document 2, the group to be authenticated in face recognition cannot be efficiently registered in the system.
One of the objectives to be achieved by the embodiments disclosed in the present specification is an information processing system, a terminal device, and a management method of an authentication target that can efficiently register a group of authentication targets in face recognition in the system. , And to provide the program.

The information processing system according to the first aspect is
A face detector that detects faces from images of multiple people's faces,
A determination unit for determining whether or not the face detected by the face detection unit satisfies a predetermined condition, and a determination unit.
A registration information generation unit that generates registration information that is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face authentication. Has.

The terminal device according to the second aspect is
A face detector that detects faces from images of multiple people's faces,
A determination unit for determining whether or not the face detected by the face detection unit satisfies a predetermined condition, and a determination unit.
Registration information is generated by associating each partial image of a plurality of faces determined to satisfy the predetermined conditions with an identifier that identifies a group to be authenticated in face authentication, and the registration information is generated. It has a registration information generator to be registered in the face authentication server.

In the management method of the authentication target according to the third aspect,
Faces are detected from images of multiple people's faces,
It is determined whether or not the detected face meets the predetermined conditions, and the detected face is determined.
Registration information is generated, which is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face recognition.

The program according to the fourth aspect is
A face detection step that detects faces from images of multiple people's faces,
A determination step for determining whether or not the detected face satisfies a predetermined condition, and
A registration information generation step that generates registration information that is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face authentication. Let the computer run.

According to the above-described aspect, it is possible to provide an information processing system, a terminal device, a management method of an authentication target, and a program capable of efficiently registering a group of authentication targets in face recognition in the system.

It is a block diagram which shows an example of the structure of the information processing system which concerns on the outline of embodiment. It is a block diagram which shows an example of the structure of the information processing system which concerns on Embodiment 1. FIG. It is a block diagram which shows an example of the hardware composition of the face authentication server and the face registration terminal which concerns on Embodiment 1. FIG. It is a flowchart which shows an example of the registration process which concerns on Embodiment 1. FIG. It is a block diagram which shows an example of the structure of the information processing system which concerns on Embodiment 2. It is a block diagram which shows an example of the hardware composition of the face registration terminal which concerns on Embodiment 2. FIG. It is a flowchart which shows an example of the registration process which concerns on Embodiment 2. It is a block diagram which shows an example of the structure of the information processing system which concerns on Embodiment 3. FIG. It is a flowchart which shows an example of the registration process which concerns on Embodiment 3. It is a block diagram which shows an example of the structure of the information processing system which concerns on Embodiment 4. It is a flowchart which shows an example of the registration process which concerns on Embodiment 4. It is a block diagram which shows an example of the structure of the information processing system which concerns on Embodiment 5. It is a flowchart which shows an example of the registration process which concerns on Embodiment 5.

<Outline of Embodiment>
Prior to the detailed description of the embodiment, the outline of the embodiment will be described. FIG. 1 is a block diagram showing an example of the configuration of the information processing system 1 according to the outline of the embodiment. As shown in FIG. 1, the information processing system 1 has a face detection unit 2, a determination unit 3, and a registration information generation unit 4.

The face detection unit 2 detects a face from an image in which the faces of a plurality of people are captured. The determination unit 3 determines whether or not the face detected by the face detection unit 2 satisfies a predetermined condition. The registration information generation unit 4 generates registration information which is information for managing the authentication target in the face recognition system. The registration information generation unit 4 associates a partial image of each of a plurality of faces determined by the determination unit 3 to satisfy a predetermined condition with an identifier that identifies a group to be authenticated in face authentication, thereby registering information. To generate. That is, the registration information is, specifically, information in which a partial image of each of a plurality of faces determined to satisfy a predetermined condition is associated with an identifier that identifies a group to be authenticated in face recognition. ..

The predetermined condition is an arbitrary condition for determining whether or not the face is the face of the authentication target person in face recognition. Specifically, for example, as will be described later, the predetermined condition may be a condition in the direction of the line of sight, a condition in the direction of the face (face), or a subject distance of the face. It may be a condition of fluctuation of the position of the face.

According to the information processing system 1, among the faces included in the image, only the set of faces determined to satisfy the predetermined conditions is associated with the identifier that identifies the group to be authenticated. Therefore, it is possible to prevent a partial image of a face that does not satisfy a predetermined condition from being associated with the identifier. Therefore, according to the information processing system 1, appropriate registration information can be generated. Therefore, the group to be authenticated in face recognition can be efficiently registered in the system.

<Details of the embodiment>
Next, the details of the embodiment will be described.
<Embodiment 1>
FIG. 2 is a block diagram showing an example of the configuration of the information processing system 10 according to the first embodiment. The information processing system 10 corresponds to the information processing system 1 of FIG. As shown in FIG. 2, the information processing system 10 includes a face authentication server 100 and a face registration terminal 200. The face authentication server 100 and the face registration terminal 200 are communicably connected by a wired or wireless network.

The face recognition server 100 is a device that performs face recognition processing. As shown in FIG. 2, the face recognition server 100 has a registration information management unit 101, an authentication determination image acquisition unit 102, and an authentication unit 103.

The registration information management unit 101 manages the registration information described above. That is, the registration information management unit 101 manages the pre-registered face image used for the collation process in face authentication. Specifically, the registration information management unit 101 stores each face image of a member belonging to the group to be authenticated in a storage device in association with an identifier for identifying the group (hereinafter referred to as a group ID). , Manage these. For example, the registration information management unit 101 stores the face image and the group ID in the storage device 54 (see FIG. 3) of the face recognition server 100. The registration information management unit 101 receives the registration information transmitted from the face registration terminal 200 and manages it.

The authentication determination image acquisition unit 102 acquires a face image to be determined for face authentication. The authentication determination image acquisition unit 102 acquires, for example, a face image transmitted from another device (not shown).

The authentication unit 103 collates the face image acquired by the authentication determination image acquisition unit 102 with the face image managed as registration information. As a result, the authentication unit 103 authenticates the person (group member) corresponding to each of the face images associated with the same group ID. The authentication unit 103 may output whether or not all the members of the group have been successfully authenticated as the final authentication result.

The face registration terminal 200 is a terminal device that registers the above-mentioned registration information in the face authentication server 100. As shown in FIG. 2, the face registration terminal 200 includes a related information creation unit 201, a registration image acquisition unit 202, a face detection unit 203, a face line-of-sight direction determination unit 204, a registration target selection unit 205, and a face. It has a registration unit 206.

When generating registration information for a new group, the related information creation unit 201 issues a group ID that identifies the group. Specifically, the related information creation unit 201 generates an arbitrary ID that does not overlap with the issued group ID.

The registration image acquisition unit 202 acquires a face image (hereinafter referred to as a registration image) used for generating registration information for a new group. That is, the registration image acquisition unit 202 acquires an image including the face of the authentication target person. The registration image is an image obtained by photographing the faces of all the members belonging to the group by a photographing device such as a camera. However, there is a possibility that a person other than the person to be authenticated (registration target) is reflected in the registration image. The registration image acquisition unit 202 may acquire a face image transmitted from another device such as a photographing device, or may read it from a storage device 54 (see FIG. 3) provided in the face registration terminal 200. You may acquire the image for registration by.

The face detection unit 203 corresponds to the face detection unit 2 in FIG.
The face detection unit 203 detects a face from the registration image acquired by the registration image acquisition unit 202. The face detection unit 203 detects a region representing a person's face by, for example, a known image recognition process.

The facial line-of-sight direction determination unit 204 corresponds to the determination unit 3 in FIG.
The face line-of-sight direction determination unit 204 determines whether or not the direction of the line-of-sight of the face detected by the face detection unit 203 is a predetermined direction and the face (face) is facing the predetermined direction. The predetermined direction is, for example, the shooting direction. The shooting direction is the direction in which a shooting device such as a camera that shoots an image exists. However, the shooting direction referred to here does not have to exactly match the direction in which a shooting device such as a camera for shooting an image exists, and may be a direction in which the subject is considered to be facing the front. Therefore, it is not necessary that the predetermined direction required as the direction of the line of sight and the predetermined direction required as the direction of the face exactly match.

As described above, in the present embodiment, the face line-of-sight direction determination unit 204 determines whether or not the face detected by the face detection unit 203 is facing the shooting direction and the line of sight is also facing the shooting direction. do. A state in which the face is oriented in a predetermined direction (shooting direction) and the line of sight is also oriented in a predetermined direction (shooting direction) is referred to as a positive state. Further, a state in which the face does not face a predetermined direction (shooting direction) or the line of sight of the subject does not face a predetermined direction (shooting direction) is referred to as a negative state. That is, the face line-of-sight direction determination unit 204 determines whether the face is in a positive state or a negative state for all the faces detected by the face detection unit 203.

The facial line-of-sight direction determination unit 204 makes a determination using, for example, a machine learning technique. Specifically, for example, the facial line-of-sight direction determination unit 204 makes a determination using a model trained using the learning data classified into the positive state and the learning data classified into the negative state. The face line-of-sight direction determination unit 204 inputs an image of the face detected by the face detection unit 203 into this model, and determines which state the image is in.

In the present embodiment, the face line-of-sight direction determination unit 204 determines whether or not both the face direction and the line-of-sight direction satisfy the conditions, but only one of them may be the determination target. For example, the face line-of-sight direction determination unit 204 may only determine whether or not the face line-of-sight direction detected by the face detection unit 203 is a predetermined direction, or the face detected by the face detection unit 203. It may be determined only whether or not the direction of is a predetermined direction. In other words, whether the face is in a positive state or a negative state may be defined only by the orientation of the face or may be defined only by the orientation of the line of sight.

The registration target selection unit 205 serves as registration information among all the faces detected by the face detection unit 203 (that is, a partial image in which the face portion cut out from the image acquired by the registration image acquisition unit 202 is copied). Select the face to register. That is, the registration target selection unit 205 selects the face of the authentication target (registration target) for the new group from all the faces detected by the face detection unit 203. In the present embodiment, the registration target selection unit 205 selects a face determined to be in the positive state by the face line-of-sight direction determination unit 204 as the registration target.

The face registration unit 206 corresponds to the registration information generation unit 4 of FIG.
The face registration unit 206 generates registration information by associating each partial image of the plurality of faces selected by the registration target selection unit 205 with the group ID created by the related information creation unit 201.
Further, the face registration unit 206 transmits the generated registration information to the face authentication server 100. The registration information transmitted to the face authentication server 100 is managed by the registration information management unit 101. As a result, the registration information is registered in the face authentication server 100.

Next, the hardware configuration of the face authentication server 100 and the face registration terminal 200 will be described.
FIG. 3 is a block diagram showing an example of the hardware configuration of the face authentication server 100 and the face registration terminal 200. The face recognition server 100 and the face registration terminal 200 each have a network interface 51, a memory 52, a processor 53, and a storage device 54, respectively. The face registration terminal 200 may be, for example, a personal computer, a tablet, a smartphone, or the like. That is, the face registration terminal 200 is an arbitrary information processing device having a function as a computer. Further, the face registration terminal 200 may include an image pickup device (camera) for providing a registration image in the registration image acquisition unit 202. Further, the image pickup device may be connected to the face registration terminal 200 as an external device of the face registration terminal 200.

The network interface 51 is used to communicate with other devices. The network interface 51 may include, for example, a network interface card (NIC).

The memory 52 is composed of, for example, a combination of a volatile memory and a non-volatile memory. The memory 52 is used to store software (computer program) including one or more instructions executed by the processor 53.

This program can be stored and supplied to a computer using various types of non-transitory computer readable media. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer readable media include magnetic recording media (eg flexible disks, magnetic tapes, hard disk drives), optomagnetic recording media (eg optomagnetic disks), Compact Disc Read Only Memory (CD-ROM), CD- Includes R, CD-R / W, semiconductor memory (eg, mask ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Flash ROM, Random Access Memory (RAM)). The program may also be supplied to the computer by various types of temporary computer readable media. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

The processor 53 may be, for example, a microprocessor, an MPU (Micro Processor Unit), or a CPU (Central Processing Unit). The processor 53 may include a plurality of processors.
The processor 53 of the face recognition server 100 reads a computer program from the memory 52 and executes it to process each component of the face recognition server 100 shown in FIG.
Further, the processor 53 of the face registration terminal 200 reads a computer program from the memory 52 and executes it to process each component of the face registration terminal 200 shown in FIG.

The storage device 54 is a device for storing data or the like used for processing, and is composed of, for example, an HDD (Hard Disk Drive), a magnetic tape, an optical disk, an SSD (Solid State Drive), or the like.

Next, the flow of the registration process according to the first embodiment will be described. FIG. 4 is a flowchart showing an example of the registration process according to the first embodiment. Hereinafter, the flow of the registration process will be described with reference to FIG.

In step S101, the related information creation unit 201 issues a new group ID.
Next, in step S102, the registration image acquisition unit 202 acquires a registration image for the new group.
Next, in step S103, the face detection unit 203 detects all the faces in the registration image acquired in step S102.
Next, in step S104, the face line-of-sight direction determination unit 204 determines whether or not each of the faces detected in step S103 is in a positive state.
Next, in step S105, the registration target selection unit 205 selects the face to be registered from all the faces detected in step S103. Specifically, the registration target selection unit 205 selects a face determined to be in a positive state in step S104 as a registration target.
Next, in step S106, the face registration unit 206 generates registration information by associating each partial image of the plurality of faces selected in step S105 with the group ID issued in step S101. Then, the face registration unit 206 transmits the generated registration information to the face authentication server 100. On the other hand, the registration information management unit 101 of the face authentication server 100 associates the image received as the registration information with the group ID and stores it in the storage device 54 of the face authentication server 100.

The first embodiment has been described above. According to the information processing system 10 according to the first embodiment, among the faces included in the registration image, only the set of faces in the positive state is associated with the group ID, and the registration information is generated. Generally, when the image for registration is taken, the faces of the members of the group face the front and the line of sight faces the direction of the taking device. Therefore, according to the present embodiment, even if the face of a person who should not be registered is reflected in the registration image, the faces of the members of the group can be efficiently extracted. That is, according to the information processing system 10, appropriate registration information can be generated, so that the group to be authenticated in face recognition can be efficiently registered in the system.

<Embodiment 2>
Next, the second embodiment will be described. In the first embodiment, as described above, whether or not the face of the person to be registered is the face is determined by the direction of the face and the line of sight. In this case, if a person who should not be registered, such as a passerby, accidentally satisfies the conditions for the direction of the face and the line of sight, the face of the person may be mistakenly included in the registration information. Therefore, a configuration in which an inappropriate face as a registration target can be excluded based on a user's instruction will be described.

FIG. 5 is a block diagram showing an example of the configuration of the information processing system 20 according to the second embodiment. The information processing system 20 is different from the information processing system 10 according to the first embodiment in that the face registration terminal 200 is replaced with the face registration terminal 210. Also in this embodiment, the face authentication server 100 and the face registration terminal 210 are communicably connected by a wired or wireless network. Hereinafter, configurations and operations different from those of the first embodiment will be described, and duplicate description will be omitted as appropriate.

Further, FIG. 6 is a block diagram showing an example of the hardware configuration of the face registration terminal 210 according to the second embodiment. As shown in FIG. 6, the face registration terminal 210 has an input device 55 and an output device 56 in addition to the network interface 51, the memory 52, the processor 53, and the storage device 54 as hardware.

The input device 55 is a device that receives an instruction input by the user, and is, for example, an input device such as a mouse or a keyboard. Further, the output device 56 is an output device such as a display that outputs information to the outside. The input device 55 and the output device 56 may be integrally configured as a touch panel.

As shown in FIG. 5, the face registration terminal 210 includes a related information creation unit 201, a registration image acquisition unit 202, a face detection unit 203, a face line-of-sight direction determination unit 204, a registration target selection unit 205, and a face. It has a registration unit 206 and a selection reception unit 211. That is, the face registration terminal 210 is different from the face registration terminal 200 in that it further has a selection reception unit 211. The processing of each component shown in FIG. 5 of the face registration terminal 210 is performed by the processor 53 of the face registration terminal 210 reading a computer program from the memory 52 and executing the processing.

Whether or not the selection reception unit 211 sets the face determined to be in the positive state by the face line-of-sight direction determination unit 204, that is, the face determined to satisfy the predetermined condition, as the authentication target (registration target). Accepts instructions from the user to select. Specifically, the selection receiving unit 211 receives an instruction from the user via, for example, the input device 55.

When the determination by the face line-of-sight direction determination unit 204 is completed for all the faces detected from the registration image, the selection reception unit 211 performs the following processing in more detail. The selection reception unit 211 superimposes a frame line surrounding the face determined to be in the positive state by the face line-of-sight direction determination unit 204 on the registration image, and outputs the image to the output device 56. Whether or not the user makes the face to be authenticated (registration target) by selecting the face to be authenticated (registration target) or the face to be excluded from the authentication target (registration target) from the faces surrounded by the frame line. Enter the instructions to select. As a result, the selection reception unit 211 receives the instruction from the user. The selection reception unit 211 notifies the registration target selection unit 205 of the received instruction.

The registration target selection unit 205 according to the present embodiment registers the face selected as the registration target by the instruction received by the selection reception unit 211 among the faces determined to be in the positive state by the face line-of-sight direction determination unit 204. Select as the face to include in the information. That is, the face registration unit 206 according to the present embodiment has a partial image selected according to an instruction from the user among the partial images of each of the plurality of faces determined to satisfy the predetermined conditions, and the group ID. Generate registration information in association with.

Next, the flow of the registration process according to the second embodiment will be described. FIG. 7 is a flowchart showing an example of the registration process according to the second embodiment. Hereinafter, the flow of the registration process will be described with reference to FIG. 7.

In step S201, the related information creation unit 201 issues a new group ID.
Next, in step S202, the registration image acquisition unit 202 acquires a registration image for the new group.
Next, in step S203, the face detection unit 203 detects all the faces in the registration image acquired in step S202.
Next, in step S204, the face line-of-sight direction determination unit 204 determines whether or not each of the faces detected in step S203 is in a positive state.
Next, in step S205, the registration target selection unit 205 selects, among all the faces detected in step S203, the face determined to be in the positive state in step S204 as a candidate for registration target.
Next, in step S206, the selection reception unit 211 displays a registration image in which the face selected as a candidate for registration in step S205 is surrounded by a frame line. On the other hand, for example, when there is an unnecessary face as a registration target, the user inputs to select the face.
Next, in step S207, the selection reception unit 211 determines whether or not the user has instructed an unnecessary face as a registration target. When instructed about an unnecessary face as a registration target (Yes in step S207), the process proceeds to step S208. On the other hand, when no unnecessary face is instructed as a registration target (No in step S207), the process proceeds to step S209.
In step S208, the registration target selection unit 205 excludes the face instructed by the user from the registration target among the faces of the candidates to be registered. After step S208, the process proceeds to step S209.
In step S209, the face registration unit 206 generates registration information by associating each partial image of the face not excluded from the faces of the candidates to be registered with the group ID issued in step S201. Then, the face registration unit 206 transmits the generated registration information to the face authentication server 100. On the other hand, the registration information management unit 101 of the face authentication server 100 associates the image received as the registration information with the group ID and stores it in the storage device 54 of the face authentication server 100.

The second embodiment has been described above. According to the information processing system 20 according to the second embodiment, the user can instruct whether or not the face determined to be in the positive state is to be registered. Therefore, even if the face of a person who should not be registered as a passerby or the like accidentally satisfies a predetermined condition for determining the registration target, the registration information can be appropriately generated. Therefore, it is possible to more accurately register the group to be authenticated in face recognition.

The selection of the image based on the user instruction described in the present embodiment is not limited to the first embodiment, and can be combined with any of the embodiments described later.

<Embodiment 3>
Next, the third embodiment will be described. In the second embodiment, when the face of a person who should not be registered as a passerby or the like accidentally meets a predetermined condition for determining a registration target, the face is registered based on an instruction from the user. Excluded inappropriate faces. In the present embodiment, faces inappropriate for registration are excluded by a method different from that of the second embodiment.

Specifically, in the present embodiment, among a plurality of registration images continuously captured at predetermined time intervals, a plurality of registration images (for example, all of the continuously captured registration images). ), If the faces of the same person are all in a positive state, they are subject to registration.

FIG. 8 is a block diagram showing an example of the configuration of the information processing system 30 according to the third embodiment. The information processing system 30 has a face authentication server 110 and a face registration terminal 220. The face authentication server 110 and the face registration terminal 220 are communicably connected by a wired or wireless network. The face recognition server 110 is different from the face recognition server 100 of the first embodiment in that the same person determination unit 111 is added. Further, the face registration terminal 220 is different from the face registration terminal 200 according to the first embodiment in that the registration target selection unit 205 is replaced with the registration target selection unit 221.

The hardware configuration of the face authentication server 110 and the face registration terminal 220 is, for example, the configuration shown in FIG. The processing of each component shown in FIG. 8 of the face recognition server 110 is performed by the processor 53 of the face recognition server 110 reading a computer program from the memory 52 and executing the processing. Similarly, the processing of each component shown in FIG. 8 of the face registration terminal 220 is performed by the processor 53 of the face registration terminal 220 reading a computer program from the memory 52 and executing the processing.
Hereinafter, configurations and operations different from those of the first embodiment will be described, and duplicate description will be omitted as appropriate.

The registration image acquisition unit 202 in the present embodiment acquires N images (where N is an integer of 2 or more) continuously captured at a predetermined time interval (for example, several seconds). Then, the face detection unit 203 detects a face from each of the plurality of continuous registration images acquired by the registration image acquisition unit 202. Then, the face detection unit 203 outputs the detected partial image of the face to the same person determination unit 111 of the face authentication server 110.

The same person determination unit 111 performs matching processing between the registration images for the faces detected from a plurality of consecutive registration images. The same person determination unit 111 performs a matching process using any known image recognition process (face recognition technique). The function corresponding to the same person determination unit 111 may be provided by the authentication unit 103. The same person determination unit 111 associates faces determined to match by matching as faces of the same person. The same person determination unit 111 outputs the association result to the registration target selection unit 221 of the face registration terminal 220.

In the registration target selection unit 221, all the faces extracted from the M registration images associated with the same person's face image by the same person determination unit 111 are in a positive state by the face line-of-sight direction determination unit 204. Check if it is determined to be there. Note that M is an integer of 2 or more. As an example, M = N, but M <N may be used. When all the faces extracted from the M registration images are in a positive state, the registration target selection unit 221 registers a partial image of any of the faces extracted from the M registration images. Select as a partial image of the face of. For example, the registration target selection unit 221 selects a partial image of the registration image taken first among the M registration images as a partial image of the face to be registered. The registration target selection unit 221 performs the above-mentioned processing on the facial images of all the persons associated with each other by the same person determination unit 111.

The facial line-of-sight direction determination unit 204 and the registration target selection unit 221 in the present embodiment correspond to the determination unit 3 in FIG. Then, the face line-of-sight direction determination unit 204 and the registration target selection unit 221 determine whether or not a plurality of faces of the same person detected from the plurality of registration images are in a positive state. Then, the face registration unit 206 associates the partial image of the face of a person determined to satisfy a predetermined condition (positive state) for the plurality of faces detected from the plurality of registration images with the group ID. , Generate registration information.

Next, the flow of the registration process according to the third embodiment will be described. FIG. 9 is a flowchart showing an example of the registration process according to the third embodiment. Hereinafter, the flow of the registration process will be described with reference to FIG.

In step S301, the related information creation unit 201 issues a new group ID.
Next, in step S302, the registration image acquisition unit 202 acquires a plurality of registration images continuously captured at predetermined time intervals (for example, several seconds).
Next, in step S303, the face detection unit 203 detects all the faces in the registration image acquired in step S302. That is, the face detection unit 203 detects the face in the image for all the registration images acquired in step S302.
Next, in step S304, the same person determination unit 111 performs matching processing between the registration images for the faces detected in step S303, and associates the faces determined to match by matching as the faces of the same person. ..
Next, in step S305, the face line-of-sight direction determination unit 204 determines whether or not each of the faces detected in step S303 is in a positive state.
Next, in step S306, the registration target selection unit 221 determines whether each of the faces extracted from the M registration images associated with the same person's face image is in a positive state. When both are in the positive state, the registration target selection unit 221 selects any partial image of the face extracted from each of the M registration images as the partial image of the face to be registered.
In step S307, the face registration unit 206 generates registration information by associating the partial image of the face selected in step S306 with the group ID issued in step S301. Then, the face registration unit 206 transmits the generated registration information to the face authentication server 110. On the other hand, the registration information management unit 101 of the face authentication server 110 associates the image received as the registration information with the group ID and stores it in the storage device 54 of the face authentication server 110.

The third embodiment has been described above. It is assumed that the person to be registered will maintain a positive state for the period during which the image for registration is continuously taken. On the other hand, it is assumed that the face of a person who happens to be facing the camera and is inappropriate for registration, such as a passerby, is only temporarily in a positive state. According to the information processing system 30 according to the third embodiment, it is possible to determine whether or not the positive state continues by using a plurality of continuously captured images for registration. Therefore, even if the face of a person who should not be registered, such as a passerby, accidentally becomes positive, the registration information can be appropriately generated. Therefore, it is possible to more accurately register the group to be authenticated in face recognition.

<Embodiment 4>
In the above-described embodiment, the face to be registered is selected depending on whether or not the face detected from the registration image is in the positive state. On the other hand, in the present embodiment, the face to be registered is selected based on the distance between the photographing device and the subject when the image for registration is photographed.

FIG. 10 is a block diagram showing an example of the configuration of the information processing system 40 according to the fourth embodiment. The information processing system 40 is different from the information processing system 10 according to the first embodiment in that the face registration terminal 200 is replaced with the face registration terminal 230. Also in this embodiment, the face authentication server 100 and the face registration terminal 230 are communicably connected by a wired or wireless network.
The face registration terminal 230 is provided with a distance estimation unit 231 and a distance determination unit 232 instead of the face line-of-sight direction determination unit 204, and a registration target selection unit 233 is provided instead of the registration target selection unit 205. It is different from the face registration terminal 200 according to the first embodiment.

The hardware configuration of the face registration terminal 230 is, for example, the configuration shown in FIG. The processing of each component shown in FIG. 10 of the face registration terminal 230 is performed by the processor 53 of the face registration terminal 230 reading a computer program from the memory 52 and executing the processing.
Hereinafter, configurations and operations different from those of the first embodiment will be described, and duplicate description will be omitted as appropriate.

The distance estimation unit 231 estimates the subject distance of the face for each of the faces detected by the face detection unit 203. The distance estimation unit 231 estimates the subject distance for each of the detected faces by, for example, performing a known image analysis process on the partial image of the face detected by the face detection unit 203. For example, the distance estimation unit 231 may estimate the subject distance based on the length of a predetermined portion of the body (for example, the length between the eyes). In this case, it is presumed that the longer the length of the predetermined part of the body, the shorter the subject distance. The predetermined part of the body may be, for example, the entire face, a part of the face (for example, nose, eyes, ears, etc.), or a part other than the face (for example, shoulders, etc.). It may be (neck, etc.). The subject estimation technique is not limited to the above estimation method, and other known estimation techniques may be used.

The distance determination unit 232 corresponds to the determination unit 3 in FIG. The distance determination unit 232 determines whether or not the subject distance estimated by the distance estimation unit 231 satisfies a predetermined distance condition. That is, the distance determination unit 232 determines whether or not the subject distance of the face estimated from the face detected by the face detection unit 203 satisfies the predetermined distance condition as a determination of whether or not the predetermined condition is satisfied. To judge. In the present embodiment, the distance determination unit 232 uses the subject distance of the face having the shortest subject distance among the faces obtained from the registration image as a reference distance for each subject distance of the face obtained from the registration image. , It is determined whether or not the difference from the reference distance is equal to or less than a predetermined threshold. The distance determination unit 232 outputs a determination result indicating whether or not the difference between the subject distance and the reference distance is equal to or less than a predetermined threshold value for each of the faces obtained from the registration image to the registration target selection unit 233.

The registration target selection unit 233 selects a partial image of the face whose subject distance is determined by the distance determination unit 232 to satisfy the above conditions as the registration target. When shooting the registration image, it is assumed that the members of the group are close to each other in the shooting direction. Therefore, by determining whether or not the difference between the subject distance and the reference distance is less than or equal to a predetermined threshold value, a person who is inappropriate for registration, such as a person passing through a position away from the members of the group, is selected. Faces can be excluded.

The above-mentioned distance condition is an example, and the distance determination unit 232 may make a determination using another distance condition. For example, the distance determination unit 232 may determine whether or not the subject distance estimated by the distance estimation unit 231 is equal to or less than a predetermined threshold value. It is assumed that the image for registration is taken in an environment where the subject distance is equal to or less than a predetermined distance. Therefore, by determining whether or not the subject distance is equal to or less than a predetermined threshold value, it is possible to exclude faces of persons who are inappropriate for registration, such as persons who are passing through a place far from the photographing device.

Next, the flow of the registration process according to the fourth embodiment will be described. FIG. 11 is a flowchart showing an example of the registration process according to the fourth embodiment. Hereinafter, the flow of the registration process will be described with reference to FIG.

In step S401, the related information creation unit 201 issues a new group ID.
Next, in step S402, the registration image acquisition unit 202 acquires a registration image for the new group.
Next, in step S403, the face detection unit 203 detects all the faces in the registration image acquired in step S402.
Next, in step S404, the distance estimation unit 231 estimates the subject distance of the face for each of the faces detected in step S403.
Next, in step S405, the distance determination unit 232 determines whether or not the subject distance estimated in step S404 satisfies a predetermined distance condition for each face detected in step S403.
Next, in step S406, the registration target selection unit 233 selects, among all the faces detected in step S403, the face determined to satisfy the distance condition in step S405 as the registration target.
Next, in step S407, the face registration unit 206 generates registration information by associating each partial image of the plurality of faces selected in step S406 with the group ID issued in step S401. Then, the face registration unit 206 transmits the generated registration information to the face authentication server 100. On the other hand, the registration information management unit 101 of the face authentication server 100 associates the image received as the registration information with the group ID and stores it in the storage device 54 of the face authentication server 100.

The embodiment 4 has been described above. According to the information processing system 40 according to the fourth embodiment, the face to be registered is selected depending on whether or not the subject distance satisfies a predetermined distance condition. Therefore, it is possible to exclude a person with a subject distance that is not suitable as a person to be registered, such as a group member or a person who is away from the photographing device. Therefore, it is possible to appropriately register the group to be authenticated in face recognition.

<Embodiment 5>
In the present embodiment, when the fluctuation distance of the face position for the same person calculated from a plurality of registration images continuously taken at a predetermined time interval is equal to or less than a predetermined threshold value, the registration target is selected. do.

FIG. 12 is a block diagram showing an example of the configuration of the information processing system 50 according to the fifth embodiment. The information processing system 50 has a face authentication server 110 and a face registration terminal 240. That is, the information processing system 50 is different from the information processing system 30 according to the third embodiment in that the face registration terminal 220 is replaced with the face registration terminal 240. The face registration terminal 240 is implemented in that a coordinate measurement unit 241 and a coordinate determination unit 242 are provided in place of the face line-of-sight direction determination unit 204, and a registration target selection unit 243 is provided in place of the registration target selection unit 221. It is different from the face registration terminal 220 according to the third form. Here, the coordinate determination unit 242 in the present embodiment corresponds to the determination unit 3 in FIG. The face authentication server 110 and the face registration terminal 240 are communicably connected by a wired or wireless network.

The hardware configuration of the face registration terminal 240 is, for example, the configuration shown in FIG. The processing of each component shown in FIG. 12 of the face registration terminal 240 is performed by the processor 53 of the face registration terminal 240 reading a computer program from the memory 52 and executing the processing.
Hereinafter, configurations and operations different from those of the third embodiment will be described, and duplicate description will be omitted as appropriate.

The registration image acquisition unit 202 in the present embodiment acquires N images (where N is an integer of 2 or more) continuously captured at a predetermined time interval (for example, several seconds). It is assumed that the shooting range of N images for registration is the same. That is, when shooting a series of N images for registration, it is assumed that the shooting device has a constant shooting area. Then, the face detection unit 203 detects a face from each of the plurality of continuous registration images acquired by the registration image acquisition unit 202. Then, the face detection unit 203 outputs the detected partial image of the face to the same person determination unit 111 of the face authentication server 110.

The same person determination unit 111 performs matching processing between the registration images for the faces detected from a plurality of continuous registration images, and associates the faces determined to match by matching as the faces of the same person. The same person determination unit 111 outputs the association result to the coordinate determination unit 242 of the face registration terminal 240.

The coordinate measurement unit 241 measures the positions of all the faces detected by the face detection unit 203 in the registration image. That is, the coordinate measuring unit 241 measures the coordinates in the image plane for all the faces detected by the face detecting unit 203. The coordinate measuring unit 241 detects the position (coordinates) of the feature point corresponding to a predetermined part of the face (for example, the center point between the eyes) by, for example, a known image recognition process, and the detected position (coordinates). ) Is the position in the registration image of the face.

The coordinate determination unit 242 determines whether or not the fluctuation distance between the registration images of the positions of the plurality of faces of the same person detected from the plurality of registration images is equal to or less than a predetermined threshold value. The coordinate determination unit 242 determines whether or not the fluctuation of the positions of the faces of the same person for each registration image specified from the association result by the same person determination unit 111 is equal to or less than a predetermined threshold value.

The registration target selection unit 243 selects a partial image of the face of a person whose fluctuation distance is determined by the coordinate determination unit 242 to be equal to or less than a predetermined threshold value as a registration target. At this time, the registration target selection unit 243 selects a partial image of the face of the same person extracted from each of the N registration images as a partial image of the face to be registered. For example, the registration target selection unit 243 selects a partial image of the registration image taken first among the N registration images as a partial image of the face to be registered. The registration target selection unit 243 performs the above-mentioned processing on the facial images of all the persons associated with each other by the same person determination unit 111.

When taking the registration image, it is assumed that the position of each face of the group members is almost constant. Therefore, by confirming the change in the position of the face in the continuous registration images, it is possible to exclude the face of a person who is inappropriate for registration, such as a passing person.

Next, the flow of the registration process according to the fifth embodiment will be described. FIG. 13 is a flowchart showing an example of the registration process according to the fifth embodiment. Hereinafter, the flow of the registration process will be described with reference to FIG.

In step S501, the related information creation unit 201 issues a new group ID.
Next, in step S502, the registration image acquisition unit 202 acquires a plurality of registration images continuously captured at predetermined time intervals (for example, several seconds).
Next, in step S503, the face detection unit 203 detects all the faces in the registration image acquired in step S502. That is, the face detection unit 203 detects the face in the image for all the registration images acquired in step S502.
Next, in step S504, the same person determination unit 111 performs matching processing between the registration images for the faces detected in step S503, and associates the faces determined to match by matching as the faces of the same person. ..
Next, in step S505, the coordinate measuring unit 241 measures the coordinates in the image plane for all the faces detected in step S503.
Next, in step S506, whether or not the fluctuation distance between the registration images of the positions of a plurality of faces of the same person detected from the plurality of registration images by the coordinate determination unit 242 is equal to or less than a predetermined threshold value. To judge.
Next, in step S507, the registration target selection unit 243 selects, among all the faces detected in step S503, the face whose fluctuation distance satisfies the threshold condition in step S506 as the registration target.
Next, in step S508, the face registration unit 206 generates registration information by associating each partial image of the plurality of faces selected in step S507 with the group ID issued in step S501. Then, the face registration unit 206 transmits the generated registration information to the face authentication server 110. On the other hand, the registration information management unit 101 of the face authentication server 110 associates the image received as the registration information with the group ID and stores it in the storage device 54 of the face authentication server 110.

The embodiment 5 has been described above. According to the information processing system 50 according to the fifth embodiment, the face to be registered is selected depending on whether or not the fluctuation distance of the face position is equal to or less than a predetermined threshold value. Therefore, it is possible to exclude faces of people who are inappropriate for registration, such as people who are passing by. Therefore, it is possible to appropriately register the group to be authenticated in face recognition.

The present invention is not limited to the above embodiment, and can be appropriately modified without departing from the spirit. For example, in the above-described embodiment, the same person determination unit is provided in the face authentication server, but may be provided in the face registration terminal. That is, the processing of the same person determination unit may be performed on the face registration terminal. Further, a part or all of any component in the face registration terminal may be provided in the face authentication server. That is, the processing of any of the components of the face registration terminal shown in FIGS. 2, 5, 8, 10, or 12 may be performed in the face authentication server. In addition, the methods for determining the face to be registered in each of the above-described embodiments may be appropriately combined.

In addition, some or all of the above embodiments may be described as in the following appendix, but are not limited to the following.

(Appendix 1)
A face detector that detects faces from images of multiple people's faces,
A determination unit for determining whether or not the face detected by the face detection unit satisfies a predetermined condition, and a determination unit.
A registration information generation unit that generates registration information that is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face authentication. Information system with.
(Appendix 2)
According to Appendix 1, the determination unit determines whether or not the direction of the line of sight of the face detected by the face detection unit is a predetermined direction as a determination of whether or not the predetermined condition is satisfied. Information processing system.
(Appendix 3)
As a determination of whether or not the predetermined condition is satisfied, the determination unit determines that the direction of the line of sight of the face detected by the face detection unit is a predetermined direction, and the face is in the predetermined direction. The information processing system according to Appendix 2, which determines whether or not it is suitable.
(Appendix 4)
The face detection unit detects a face from each of a plurality of images continuously taken at predetermined time intervals.
The determination unit determines whether or not a plurality of faces of the same person detected from the plurality of images satisfy the predetermined conditions.
The registration information generation unit associates the partial image of the face of a person determined to satisfy the predetermined conditions with respect to the plurality of faces detected from the plurality of images with the identifier, and the registration information. The information processing system according to Appendix 2 or 3.
(Appendix 5)
Further, it has a selection reception unit that receives an instruction to select whether or not to make a face to be authenticated for a face determined to satisfy the predetermined condition.
The registration information generation unit associates the partial image selected according to the instruction with the identifier among the partial images of each of the plurality of faces determined to satisfy the predetermined conditions, and the registration information. The information processing system according to any one of Supplementary note 1 to 4.
(Appendix 6)
The determination unit determines whether or not the subject distance of the face estimated from the face detected by the face detection unit satisfies the predetermined distance condition as a determination of whether or not the predetermined condition is satisfied. The information processing system according to Appendix 1.
(Appendix 7)
The face detection unit detects a face from each of a plurality of images continuously taken at predetermined time intervals.
The determination unit determines whether or not the predetermined condition is satisfied, and whether or not the fluctuation distance of the positions of a plurality of faces of the same person detected from the plurality of images is equal to or less than a predetermined threshold value. Judging whether
The information according to Appendix 1 in which the registration information generation unit associates a partial image of each of the faces of a plurality of persons determined to satisfy the predetermined conditions with the identifier and generates the registration information. Processing system.
(Appendix 8)
A face detector that detects faces from images of multiple people's faces,
A determination unit for determining whether or not the face detected by the face detection unit satisfies a predetermined condition, and a determination unit.
Registration information is generated by associating each partial image of a plurality of faces determined to satisfy the predetermined conditions with an identifier that identifies a group to be authenticated in face authentication, and the registration information is generated. A terminal device having a registration information generator to be registered in the face authentication server.
(Appendix 9)
As described in Appendix 8, the determination unit determines whether or not the direction of the line of sight of the face detected by the face detection unit is a predetermined direction as a determination of whether or not the predetermined condition is satisfied. Terminal device.
(Appendix 10)
Faces are detected from images of multiple people's faces,
It is determined whether or not the detected face meets the predetermined conditions, and the detected face is determined.
A method of managing an authentication target that generates registration information, which is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face authentication. ..
(Appendix 11)
A face detection step that detects faces from images of multiple people's faces,
A determination step for determining whether or not the detected face satisfies a predetermined condition, and
A registration information generation step for generating registration information, which is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face authentication. A program that causes a computer to execute.

1, 10, 20, 30, 40, 50 Information processing system 2 Face detection unit 3 Judgment unit 4 Registration information generation unit 51 Network interface 52 Memory 53 Processor 54 Storage device 55 Input device 56 Output device 100, 110 Face authentication server 101 registration Information management unit 102 Authentication judgment image acquisition unit 103 Authentication unit 111 Same person determination unit 200, 210, 220, 230, 240 Face registration terminal 201 Related information creation unit 202 Registration image acquisition unit 203 Face detection unit 204 Face line-of-sight direction determination Unit 205, 221, 233, 243 Registration target selection unit 206 Face registration unit 211 Selection reception unit 231 Distance estimation unit 232 Distance determination unit 241 Coordinate measurement unit 242 Coordinate determination unit

Claims (10)

A face detector that detects faces from images of multiple people's faces,
A determination unit for determining whether or not the face detected by the face detection unit satisfies a predetermined condition, and a determination unit.
A registration information generation unit that generates registration information that is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face authentication. Information system with. The first aspect of claim 1, wherein the determination unit determines whether or not the direction of the line of sight of the face detected by the face detection unit is a predetermined direction as a determination of whether or not the predetermined condition is satisfied. Information processing system. As a determination of whether or not the predetermined condition is satisfied, the determination unit determines that the direction of the line of sight of the face detected by the face detection unit is a predetermined direction, and the face is in the predetermined direction. The information processing system according to claim 2, wherein it is determined whether or not it is suitable. The face detection unit detects a face from each of a plurality of images continuously taken at predetermined time intervals.
The determination unit determines whether or not a plurality of faces of the same person detected from the plurality of images satisfy the predetermined conditions.
The registration information generation unit associates the partial image of the face of a person determined to satisfy the predetermined conditions with respect to the plurality of faces detected from the plurality of images with the identifier, and the registration information. The information processing system according to claim 2 or 3. Further, it has a selection reception unit that receives an instruction to select whether or not to make a face to be authenticated for a face determined to satisfy the predetermined condition.
The registration information generation unit associates the partial image selected according to the instruction with the identifier among the partial images of each of the plurality of faces determined to satisfy the predetermined conditions, and the registration information. The information processing system according to any one of claims 1 to 4. The determination unit determines whether or not the subject distance of the face estimated from the face detected by the face detection unit satisfies the predetermined distance condition as a determination of whether or not the predetermined condition is satisfied. The information processing system according to claim 1. The face detection unit detects a face from each of a plurality of images continuously taken at predetermined time intervals.
The determination unit determines whether or not the predetermined condition is satisfied, and whether or not the fluctuation distance of the positions of a plurality of faces of the same person detected from the plurality of images is equal to or less than a predetermined threshold value. Judging whether
The registration information generation unit according to claim 1, wherein the registration information generation unit associates a partial image of each of the faces of a plurality of persons determined to satisfy the predetermined conditions with the identifier and generates the registration information. Information processing system. A face detector that detects faces from images of multiple people's faces,
A determination unit for determining whether or not the face detected by the face detection unit satisfies a predetermined condition, and a determination unit.
Registration information is generated by associating each partial image of a plurality of faces determined to satisfy the predetermined conditions with an identifier that identifies a group to be authenticated in face authentication, and the registration information is generated. A terminal device having a registration information generator to be registered in the face authentication server. Faces are detected from images of multiple people's faces,
It is determined whether or not the detected face meets the predetermined conditions, and the detected face is determined.
A method of managing an authentication target that generates registration information, which is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face authentication. .. A face detection step that detects faces from images of multiple people's faces,
A determination step for determining whether or not the detected face satisfies a predetermined condition, and
A registration information generation step that generates registration information that is information in which a partial image of each of a plurality of faces determined to satisfy the predetermined conditions is associated with an identifier that identifies a group to be authenticated in face authentication. A program that causes a computer to execute.

Images