JP7327571B2 - Information processing system, terminal device, authentication target management method, and program - Google Patents

Description

The present invention relates to an information processing system, a terminal device, an authentication target management method, and a program.

In recent years, various systems using face authentication have been developed, and their applications are diversified. In a face authentication system, it is necessary to register face information of a person to be authenticated in advance in order to perform face authentication. In face authentication for personal identification, face images registered as authentication targets are often registered on an individual basis. However, for example, in a stamp rally game where the condition for giving stamps is that all group members have arrived at a predetermined point, if the arrival of the participants is determined by facial recognition, each group Face images are required to be registered.

As a technology related to this, there is an authentication device described in Patent Document 1. In this authentication device, multiple faces detected from an image are associated and registered.
However, when the face of a person to be authenticated is photographed outdoors or in a place with many people, and multiple faces are detected from the image at the same time and face information is registered, the face of a person other than the person to be authenticated may appear in the image. There is a risk that the face of a person other than the target person will also be erroneously associated. In such a case, it is necessary to change the photographing location or retake photographing of the face image, which is troublesome.

Also, as a related technique, there is an image information processing apparatus described in Japanese Patent Application Laid-Open No. 2002-200023. This image information processing apparatus groups some persons out of a plurality of persons in the image based on the direction in which the persons in the image are facing.

Japanese Patent No. 5783009 WO2011/148562

The device described in Patent Document 1 or Patent Document 2 cannot efficiently register a group to be authenticated in the face authentication in the system.
One of the objects to be achieved by the embodiments disclosed in this specification is an information processing system, a terminal device, and an authentication target management method capable of efficiently registering a group to be authenticated in face authentication in the system. , and to provide a program.

An information processing system according to a first aspect includes:
a face detection unit that detects a face from an image containing the faces of a plurality of people;
a determination unit that determines whether the face detected by the face detection unit satisfies a predetermined condition;
a registration information generation unit that generates registration information that is information that associates each partial image of a plurality of faces determined to satisfy the predetermined condition with an identifier that identifies a group to be authenticated in face authentication; have

A terminal device according to a second aspect includes:
a face detection unit that detects a face from an image containing the faces of a plurality of people;
a determination unit that determines whether the face detected by the face detection unit satisfies a predetermined condition;
Registered information that is information that associates each partial image of a plurality of faces determined to satisfy the predetermined condition with an identifier that identifies a group to be authenticated in face authentication is generated, and the registered information is generated. and a registration information generation unit for registering in the face authentication server.

In the authentication target management method according to the third aspect,
Detect faces from an image of multiple people's faces,
determining whether the detected face satisfies a predetermined condition;
Registered information is generated that is information that associates each partial image of a plurality of faces determined to satisfy the predetermined condition with an identifier that identifies a group to be authenticated in face authentication.

A program according to a fourth aspect,
a face detection step of detecting a face from an image showing faces of a plurality of persons;
a determination step of determining whether the detected face satisfies a predetermined condition;
a registration information generation step of generating registration information that is information that associates each partial image of a plurality of faces determined to satisfy the predetermined condition with an identifier that identifies a group to be authenticated in face authentication; run on the computer.

According to the above aspect, it is possible to provide an information processing system, a terminal device, an authentication target management method, and a program that can efficiently register groups to be authenticated in face authentication in the system.

1 is a block diagram showing an example of a configuration of an information processing system according to an outline of an embodiment; FIG. 1 is a block diagram showing an example of a configuration of an information processing system according to Embodiment 1; FIG. 2 is a block diagram showing an example of hardware configurations of a face authentication server and a face registration terminal according to the first embodiment; FIG. 6 is a flowchart showing an example of registration processing according to the first embodiment; 2 is a block diagram showing an example of the configuration of an information processing system according to a second embodiment; FIG. FIG. 11 is a block diagram showing an example of a hardware configuration of a face registration terminal according to a second embodiment; FIG. FIG. 11 is a flow chart showing an example of registration processing according to the second embodiment; FIG. FIG. 11 is a block diagram showing an example of the configuration of an information processing system according to a third embodiment; FIG. FIG. 11 is a flow chart showing an example of registration processing according to the third embodiment; FIG. FIG. 12 is a block diagram showing an example of the configuration of an information processing system according to a fourth embodiment; FIG. FIG. 12 is a flow chart showing an example of registration processing according to the fourth embodiment; FIG. FIG. 12 is a block diagram showing an example of the configuration of an information processing system according to a fifth embodiment; FIG. FIG. 16 is a flow chart showing an example of registration processing according to the fifth embodiment; FIG.

<Overview of Embodiment>
Prior to detailed description of the embodiments, an outline of the embodiments will be described. FIG. 1 is a block diagram showing an example of the configuration of an information processing system 1 according to the outline of the embodiment. As shown in FIG. 1, the information processing system 1 has a face detection section 2, a determination section 3, and a registered information generation section 4. FIG.

A face detection unit 2 detects faces from an image in which faces of a plurality of persons are photographed. The determination unit 3 determines whether the face detected by the face detection unit 2 satisfies a predetermined condition. The registration information generation unit 4 generates registration information, which is information for managing authentication targets in the face authentication system. The registration information generation unit 4 generates registration information by associating partial images of each of the plurality of faces determined by the determination unit 3 to satisfy a predetermined condition with an identifier that identifies a group to be authenticated in face authentication. to generate Specifically, the registration information is information that associates each partial image of a plurality of faces determined to satisfy a predetermined condition with an identifier that identifies a group to be authenticated in face authentication. .

Note that the predetermined condition is an arbitrary condition for determining whether or not it is the face of a person to be authenticated in face authentication. Specifically, for example, as will be described later, the predetermined condition may be a condition of the direction of the line of sight, a condition of the direction of the face (face), or a condition of the subject distance of the face. may be the condition, or the condition may be the change in the position of the face.

According to the information processing system 1, only a set of faces determined to satisfy a predetermined condition among faces included in an image is associated with an identifier that identifies a group to be authenticated. Therefore, it is possible to prevent a partial face image that does not satisfy a predetermined condition from being associated with the identifier. Therefore, according to the information processing system 1, appropriate registration information can be generated. Therefore, groups to be authenticated in face authentication can be efficiently registered in the system.

<Details of embodiment>
Next, details of the embodiment will be described.
<Embodiment 1>
FIG. 2 is a block diagram showing an example of the configuration of the information processing system 10 according to the first embodiment. The information processing system 10 corresponds to the information processing system 1 in FIG. As shown in FIG. 2, the information processing system 10 includes a face authentication server 100 and a face registration terminal 200. As shown in FIG. The face authentication server 100 and the face registration terminal 200 are communicably connected via a wired or wireless network.

The face authentication server 100 is a device that performs face authentication processing. The face authentication server 100 has a registration information management unit 101, an authentication determination image acquisition unit 102, and an authentication unit 103, as shown in FIG.

The registration information management unit 101 manages the registration information described above. That is, the registration information management unit 101 manages pre-registered face images used for matching processing in face authentication. Specifically, the registration information management unit 101 associates face images of members belonging to a group to be authenticated with identifiers for identifying the groups (hereinafter referred to as group IDs) and stores them in a storage device. , manage these. For example, the registration information management unit 101 stores the face image and the group ID in the storage device 54 (see FIG. 3) of the face authentication server 100. FIG. The registration information management unit 101 receives the registration information transmitted from the face registration terminal 200 and manages it.

The authentication determination image acquisition unit 102 acquires a face image to be determined for face authentication. The authentication determination image acquisition unit 102 acquires, for example, a face image transmitted from another device (not shown).

Authentication unit 103 collates the face image acquired by authentication determination image acquisition unit 102 with the face image managed as registration information. Thereby, the authentication unit 103 authenticates a person (group member) corresponding to each face image associated with the same group ID. Note that the authentication unit 103 may output whether or not authentication has been successful for all members of the group as a final authentication result.

The face registration terminal 200 is a terminal device that registers the registration information described above in the face authentication server 100 . As shown in FIG. 2, the face registration terminal 200 includes a related information creation unit 201, a registration image acquisition unit 202, a face detection unit 203, a face gaze direction determination unit 204, a registration target selection unit 205, a face and a registration unit 206 .

The related information creation unit 201 issues a group ID for identifying the new group when creating registration information for a new group. Specifically, the related information creating unit 201 creates an arbitrary ID that does not overlap with the issued group ID.

The registration image acquisition unit 202 acquires a face image (hereinafter referred to as a registration image) used for generating registration information for a new group. That is, the registration image acquiring unit 202 acquires an image including the face of the person to be authenticated. The registration image is an image obtained by photographing the faces of all members belonging to the group with a photographing device such as a camera. However, there is a possibility that a person other than a person to be authenticated (registered) appears in the registration image. The registration image acquisition unit 202 may, for example, acquire a face image transmitted from another device such as an imaging device, or read it from the storage device 54 (see FIG. 3) provided in the face registration terminal 200. You may acquire the image for registration by.

A face detection unit 203 corresponds to the face detection unit 2 in FIG.
A face detection unit 203 detects a face from the registration image acquired by the registration image acquisition unit 202 . The face detection unit 203 detects an area representing a person's face by, for example, known image recognition processing.

The face line-of-sight direction determination unit 204 corresponds to the determination unit 3 in FIG.
The face line-of-sight direction determination unit 204 determines whether the line-of-sight direction of the face detected by the face detection unit 203 is in a predetermined direction and whether the face (face) is oriented in the predetermined direction. Note that the predetermined direction is, for example, the shooting direction. The shooting direction is the direction in which a shooting device such as a camera for shooting an image exists. However, the photographing direction referred to here does not have to strictly match the direction in which the photographing device such as a camera that photographs the image exists, and may be any direction in which the subject is considered to be facing the front. Therefore, the predetermined direction required as the direction of the line of sight does not need to strictly match the predetermined direction required as the direction of the face.

As described above, in the present embodiment, the face line-of-sight direction determination unit 204 determines whether the face detected by the face detection unit 203 faces the shooting direction, and whether the line of sight also faces the shooting direction. do. A subject whose face faces a predetermined direction (shooting direction) and whose line of sight also faces a predetermined direction (shooting direction) is referred to as a positive state. Also, a state of a subject whose face does not face a predetermined direction (shooting direction) or whose line of sight does not face a predetermined direction (shooting direction) will be referred to as a negative state. That is, the face gaze direction determination unit 204 determines whether all faces detected by the face detection unit 203 are in a positive state or a negative state.

The face line-of-sight direction determination unit 204 performs determination using, for example, machine learning technology. Specifically, for example, the face line-of-sight direction determination unit 204 makes determination using a model learned using learning data classified into the positive state and learning data classified into the negative state. The face line-of-sight direction determination unit 204 inputs the image of the face detected by the face detection unit 203 to this model, and determines in which state the image is.

In the present embodiment, the face line-of-sight direction determination unit 204 determines whether or not both the face direction and the line-of-sight direction satisfy the conditions, but only one of them may be the target of determination. For example, the face line-of-sight direction determination unit 204 may determine only whether or not the line-of-sight direction of the face detected by the face detection unit 203 is in a predetermined direction. It may be determined only whether or not the direction of is a predetermined direction. In other words, whether the face is in the positive state or the negative state may be defined only by the direction of the face or only by the direction of the line of sight.

The registration target selection unit 205 selects, as registration information, all faces detected by the face detection unit 203 (that is, partial images in which face parts are extracted from images acquired by the registration image acquisition unit 202). Select a face to be registered. That is, the registration target selection unit 205 selects, from all the faces detected by the face detection unit 203, faces to be authenticated (registered) for the new group. In this embodiment, the registration target selection unit 205 selects a face determined to be in a positive state by the face gaze direction determination unit 204 as a registration target.

A face registration unit 206 corresponds to the registration information generation unit 4 in FIG.
The face registration unit 206 generates registration information by associating the partial images of each of the faces selected by the registration target selection unit 205 with the group ID created by the related information creation unit 201 .
The face registration unit 206 also transmits the generated registration information to the face authentication server 100 . The registration information transmitted to the face authentication server 100 is managed by the registration information management section 101 . Thereby, the registration information is registered in the face authentication server 100 .

Next, hardware configurations of the face authentication server 100 and the face registration terminal 200 will be described.
FIG. 3 is a block diagram showing an example of the hardware configuration of the face authentication server 100 and the face registration terminal 200. As shown in FIG. The face authentication server 100 and the face registration terminal 200 each have a network interface 51, a memory 52, a processor 53, and a storage device . The face registration terminal 200 may be, for example, a personal computer, tablet, smartphone, or the like. That is, the face registration terminal 200 is any information processing device having a function as a computer. Further, the face registration terminal 200 may incorporate an imaging device (camera) for providing the registration image to the registration image acquisition unit 202 . Also, the imaging device may be connected to the face registration terminal 200 as an external device of the face registration terminal 200 .

A network interface 51 is used to communicate with other devices. Network interface 51 may include, for example, a network interface card (NIC).

The memory 52 is configured by, for example, a combination of volatile memory and non-volatile memory. The memory 52 is used to store software (computer program) including one or more instructions to be executed by the processor 53 .

The program can be stored and provided to the computer using various types of non-transitory computer readable medium. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., floppy disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical discs), Compact Disc Read Only Memory (CD-ROM), CD- R, CD-R/W, semiconductor memory (eg mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM)). The program may also be supplied to the computer on various types of transitory computer readable medium. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can deliver the program to the computer via wired channels, such as wires and optical fibers, or wireless channels.

The processor 53 may be, for example, a microprocessor, an MPU (Micro Processor Unit), or a CPU (Central Processing Unit). Processor 53 may include multiple processors.
The processor 53 of the face authentication server 100 reads the computer program from the memory 52 and executes it, thereby performing the processing of each component of the face authentication server 100 shown in FIG.
Further, the processor 53 of the face registration terminal 200 reads the computer program from the memory 52 and executes it, so that the processing of each component of the face registration terminal 200 shown in FIG. 2 is performed.

The storage device 54 is a device that stores data used for processing, and is configured by, for example, an HDD (Hard Disk Drive), a magnetic tape, an optical disk, or an SSD (Solid State Drive).

Next, the flow of registration processing according to the first embodiment will be described. 4 is a flowchart illustrating an example of registration processing according to the first embodiment; FIG. The flow of registration processing will be described below with reference to FIG.

In step S101, the related information creating unit 201 issues a new group ID.
Next, in step S102, the registration image acquisition unit 202 acquires a registration image for the new group.
Next, in step S103, the face detection unit 203 detects all faces appearing in the registration image acquired in step S102.
Next, in step S104, the face line-of-sight direction determination unit 204 determines whether each face detected in step S103 is in a positive state.
Next, in step S105, the registration target selection unit 205 selects a registration target face from all the faces detected in step S103. Specifically, the registration target selection unit 205 selects the face determined to be in the positive state in step S104 as the registration target.
Next, in step S106, the face registration unit 206 generates registration information by associating the partial images of each of the faces selected in step S105 with the group ID issued in step S101. The face registration unit 206 then transmits the generated registration information to the face authentication server 100 . On the other hand, the registration information management unit 101 of the face authentication server 100 associates the image received as the registration information with the group ID, and stores them in the storage device 54 of the face authentication server 100 .

The first embodiment has been described above. According to the information processing system 10 according to the first embodiment, among the faces included in the registration image, only a set of faces in the positive state is associated with the group ID, and registration information is generated. In general, when the image for registration is captured, the faces of the members of the group face the front and the line of sight is directed toward the photographing device. Therefore, according to the present embodiment, even if the face of a person who should not be registered is included in the registration image, the face of the member of the group can be efficiently extracted. That is, according to the information processing system 10, since appropriate registration information can be generated, groups to be authenticated in face authentication can be efficiently registered in the system.

<Embodiment 2>
Next, Embodiment 2 will be described. In Embodiment 1, as described above, whether or not a face is a person's face to be registered is determined based on the direction of the face and line of sight. In this case, if a person who should not be registered, such as a passerby, accidentally satisfies the face and line-of-sight direction conditions, the person's face may be erroneously included in the registration information. Therefore, a configuration for excluding faces inappropriate for registration based on user's instructions will be described.

FIG. 5 is a block diagram showing an example of the configuration of the information processing system 20 according to the second embodiment. The information processing system 20 differs from the information processing system 10 according to the first embodiment in that the face registration terminal 200 is replaced with a face registration terminal 210 . Also in this embodiment, the face authentication server 100 and the face registration terminal 210 are communicably connected via a wired or wireless network. Configurations and operations different from those of the first embodiment will be described below, and overlapping descriptions will be omitted as appropriate.

FIG. 6 is a block diagram showing an example hardware configuration of the face registration terminal 210 according to the second embodiment. As shown in FIG. 6, the face registration terminal 210 has an input device 55 and an output device 56 in addition to a network interface 51, a memory 52, a processor 53, and a storage device 54 as hardware.

The input device 55 is a device that receives instructions input by a user, and is, for example, an input device such as a mouse or keyboard. The output device 56 is an output device such as a display that outputs information to the outside. Note that the input device 55 and the output device 56 may be configured integrally as a touch panel.

As shown in FIG. 5, the face registration terminal 210 includes a related information creation unit 201, a registration image acquisition unit 202, a face detection unit 203, a face gaze direction determination unit 204, a registration target selection unit 205, a face It has a registration unit 206 and a selection acceptance unit 211 . In other words, the face registration terminal 210 differs from the face registration terminal 200 in that it further has a selection accepting section 211 . Processing of each component shown in FIG. 5 of face registration terminal 210 is performed by processor 53 of face registration terminal 210 reading a computer program from memory 52 and executing it.

The selection reception unit 211 determines whether or not the face determined to be in the positive state by the face line-of-sight direction determination unit 204, that is, the face determined to satisfy a predetermined condition, is to be authenticated (registered). An instruction to select is received from the user. Specifically, the selection accepting unit 211 accepts an instruction from the user via the input device 55, for example.

After the determination by the face line-of-sight direction determination unit 204 is completed for all faces detected from the registration image, the selection reception unit 211 more specifically performs the following processing. The selection reception unit 211 superimposes a frame line surrounding the face determined to be in the positive state by the face gaze direction determination unit 204 on the registration image, and outputs the image to the output device 56 . The user selects a face to be authenticated (registered) or a face to be excluded from authentication (registration) from among the faces enclosed by the frame lines, thereby determining whether or not the face is to be authenticated (registered). Enter instructions to select Thereby, the selection accepting unit 211 accepts an instruction from the user. Selection receiving unit 211 notifies registration target selection unit 205 of the received instruction.

The registration target selection unit 205 according to the present embodiment registers a face selected as a registration target by an instruction received by the selection reception unit 211 from among the faces determined to be in the positive state by the face line-of-sight direction determination unit 204. Select a face to include in the information. That is, the face registration unit 206 according to the present embodiment selects a partial image according to an instruction from the user, from partial images of each of a plurality of faces determined to satisfy a predetermined condition, and a group ID. to generate registration information.

Next, the flow of registration processing according to the second embodiment will be described. FIG. 7 is a flowchart of an example of registration processing according to the second embodiment. The flow of registration processing will be described below with reference to FIG.

In step S201, the related information creating unit 201 issues a new group ID.
Next, in step S202, the registration image acquiring unit 202 acquires a registration image for the new group.
Next, in step S203, the face detection unit 203 detects all faces appearing in the registration image acquired in step S202.
Next, in step S204, the face line-of-sight direction determination unit 204 determines whether each face detected in step S203 is in a positive state.
Next, in step S205, the registration target selection unit 205 selects the face determined to be in the positive state in step S204 from all the faces detected in step S203 as registration target candidates.
Next, in step S206, the selection receiving unit 211 displays a registration image in which the face selected as a candidate to be registered in step S205 is surrounded by a frame. On the other hand, for example, if there is an unnecessary face to be registered, the user performs an input to select that face.
Next, in step S207, the selection accepting unit 211 determines whether or not the user has instructed an unnecessary face to be registered. If a face that is not required to be registered has been instructed (Yes in step S207), the process proceeds to step S208. On the other hand, if a face that is not required to be registered has not been instructed (No in step S207), the process proceeds to step S209.
In step S<b>208 , the registration target selection unit 205 excludes, from the registration target candidate faces, the face designated by the user as an unnecessary face from the registration target. After step S208, the process proceeds to step S209.
In step S209, the face registration unit 206 generates registration information by associating the partial images of the faces that have not been excluded among the candidate faces to be registered with the group ID issued in step S201. The face registration unit 206 then transmits the generated registration information to the face authentication server 100 . On the other hand, the registration information management unit 101 of the face authentication server 100 associates the image received as the registration information with the group ID, and stores them in the storage device 54 of the face authentication server 100 .

The second embodiment has been described above. According to the information processing system 20 according to the second embodiment, the user can instruct whether or not the face determined to be in the positive state is to be registered. Therefore, even if the face of a person, such as a passerby, who should not be registered, accidentally satisfies a predetermined condition for determining registration, registration information can be appropriately generated. Therefore, it is possible to more accurately register a group to be authenticated in face authentication.

Note that the image selection based on the user's instruction described in the present embodiment is not limited to the first embodiment, and can be combined with any of the embodiments described later.

<Embodiment 3>
Next, Embodiment 3 will be described. In the second embodiment, when the face of a person who should not be registered, such as a passerby, accidentally satisfies a predetermined condition for determining whether to be registered, registration is performed based on an instruction from the user. Inappropriate faces were excluded. In the present embodiment, faces inappropriate for registration are excluded by a method different from that of the second embodiment.

Specifically, in the present embodiment, among a plurality of registration images continuously captured at predetermined time intervals, a plurality of registration images (for example, all of the continuously captured registration images ) for the same person are in a positive state, they are registered.

FIG. 8 is a block diagram showing an example of the configuration of the information processing system 30 according to the third embodiment. The information processing system 30 has a face authentication server 110 and a face registration terminal 220 . The face authentication server 110 and the face registration terminal 220 are communicably connected via a wired or wireless network. The face authentication server 110 differs from the face authentication server 100 of Embodiment 1 in that a same person determination unit 111 is added. Further, face registration terminal 220 differs from face registration terminal 200 according to the first embodiment in that registration target selection section 205 is replaced with registration target selection section 221 .

The hardware configuration of the face authentication server 110 and face registration terminal 220 is, for example, the configuration shown in FIG. The processing of each component shown in FIG. 8 of face authentication server 110 is performed by processor 53 of face authentication server 110 reading a computer program from memory 52 and executing it. Similarly, the processing of each component shown in FIG. 8 of face registration terminal 220 is performed by processor 53 of face registration terminal 220 reading a computer program from memory 52 and executing it.
Configurations and operations different from those of the first embodiment will be described below, and overlapping descriptions will be omitted as appropriate.

The registration image acquisition unit 202 in the present embodiment acquires N registration images (where N is an integer equal to or greater than 2) continuously photographed at predetermined time intervals (for example, several seconds). Then, the face detection unit 203 detects a face from each of the plurality of continuous registration images acquired by the registration image acquisition unit 202 . The face detection unit 203 then outputs the detected partial image of the face to the same person determination unit 111 of the face authentication server 110 .

The same person determination unit 111 performs matching processing between registration images for faces detected from a plurality of continuous registration images. The same person determination unit 111 performs matching processing using any known image recognition processing (face recognition technology). A function corresponding to the same person determination unit 111 may be provided by the authentication unit 103 . The same person determination unit 111 associates faces determined to match by matching as faces of the same person. The same person determination unit 111 outputs the association result to the registration target selection unit 221 of the face registration terminal 220 .

The registration target selection unit 221 determines whether all of the faces extracted from the M registration images associated by the same person determination unit 111 as images of the face of the same person are in a positive state by the face line-of-sight direction determination unit 204. Check if it is determined that there is Note that M is an integer of 2 or more. As an example, M=N, but M<N is also possible. When all of the faces extracted from the M registration images are in the positive state, the registration target selection unit 221 selects any partial image of the face extracted from the M registration images as the registration target. to select as a partial image of the face of For example, the registration target selection unit 221 selects the partial image of the registration image photographed first among the M registration images as the registration target face partial image. Registration target selection section 221 performs the above-described processing on face images of all persons associated by same person determination section 111 .

Note that the face line-of-sight direction determination unit 204 and the registration target selection unit 221 in the present embodiment correspond to the determination unit 3 in FIG. Then, the face line-of-sight direction determination unit 204 and the registration target selection unit 221 determine whether or not the plurality of faces of the same person detected from the plurality of images for registration are in the positive state. Then, the face registration unit 206 associates a partial face image of a person determined to satisfy a predetermined condition (positive state) with respect to a plurality of faces detected from a plurality of registration images, and a group ID. , to generate registration information.

Next, the flow of registration processing according to the third embodiment will be described. FIG. 9 is a flowchart of an example of registration processing according to the third embodiment. The flow of registration processing will be described below with reference to FIG.

In step S301, the related information creating unit 201 issues a new group ID.
Next, in step S302, the registration image acquisition unit 202 acquires a plurality of registration images continuously captured at predetermined time intervals (for example, several seconds).
Next, in step S303, the face detection unit 203 detects all faces appearing in the registration image acquired in step S302. That is, the face detection unit 203 detects faces in all images for registration acquired in step S302.
Next, in step S304, the same person determination unit 111 performs matching processing between the registration images for the faces detected in step S303, and associates the faces determined to match by matching as the faces of the same person. .
Next, in step S305, the face line-of-sight direction determination unit 204 determines whether each face detected in step S303 is in a positive state.
Next, in step S306, the registration target selection unit 221 determines whether all of the faces extracted from the M registration images associated as images of the face of the same person are in the positive state. Then, if all of them are in the positive state, the registration target selection unit 221 selects one of the partial images of the face extracted from each of the M registration images as the partial image of the face to be registered.
In step S307, the face registration unit 206 generates registration information by associating the partial image of the face selected in step S306 with the group ID issued in step S301. The face registration unit 206 then transmits the generated registration information to the face authentication server 110 . On the other hand, the registration information management unit 101 of the face authentication server 110 associates the image received as the registration information with the group ID, and stores them in the storage device 54 of the face authentication server 110 .

The third embodiment has been described above. It is assumed that the person to be registered is maintained in a positive state during the period in which the image for registration continues to be captured. On the other hand, it is assumed that the face of a person inappropriate for registration, such as a passerby who happens to face the direction of the camera, is only temporarily in a positive state. According to the information processing system 30 according to the third embodiment, it is possible to determine whether or not the positive state continues by using a plurality of images for registration that are continuously captured. Therefore, even if the face of a person who should not be registered, such as a passerby, happens to be in a positive state, it is possible to appropriately generate registration information. Therefore, it is possible to more accurately register a group to be authenticated in face authentication.

<Embodiment 4>
In the above-described embodiment, a face to be registered is selected depending on whether or not the face detected from the registration image is in the positive state. On the other hand, in the present embodiment, a face to be registered is selected based on the distance between the photographing device and the subject when the registration image is photographed.

FIG. 10 is a block diagram showing an example of the configuration of an information processing system 40 according to the fourth embodiment. The information processing system 40 differs from the information processing system 10 according to the first embodiment in that the face registration terminal 200 is replaced with a face registration terminal 230 . Also in this embodiment, the face authentication server 100 and the face registration terminal 230 are communicably connected via a wired or wireless network.
The face registration terminal 230 is provided with a distance estimation unit 231 and a distance determination unit 232 instead of the face gaze direction determination unit 204, and is provided with a registration target selection unit 233 instead of the registration target selection unit 205. It differs from the face registration terminal 200 according to the first embodiment.

The hardware configuration of the face registration terminal 230 is, for example, the configuration shown in FIG. 10 of face registration terminal 230 is performed by processor 53 of face registration terminal 230 reading a computer program from memory 52 and executing it.
Configurations and operations different from those of the first embodiment will be described below, and overlapping descriptions will be omitted as appropriate.

The distance estimation unit 231 estimates the subject distance of each face detected by the face detection unit 203 . The distance estimation unit 231 estimates the subject distance for each detected face by, for example, performing known image analysis processing on the partial image of the face detected by the face detection unit 203 . For example, the distance estimating section 231 may estimate the subject distance based on the length of a predetermined part of the body (for example, the length between the eyes). In this case, it is estimated that the longer the length of the predetermined part of the body, the shorter the subject distance. The predetermined part of the body may be, for example, the entire face, a part of the face (for example, nose, eyes, ears, etc.), or a part other than the face (for example, shoulders, ears, etc.). neck, etc.). Note that the subject estimation technique is not limited to the above estimation method, and other known estimation techniques may be used.

The distance determination section 232 corresponds to the determination section 3 in FIG. The distance determination section 232 determines whether or not the subject distance estimated by the distance estimation section 231 satisfies a predetermined distance condition. That is, the distance determination unit 232 determines whether or not the subject distance of the face detected by the face detection unit 203 satisfies the predetermined distance condition as a determination as to whether or not the predetermined condition is satisfied. judge. In the present embodiment, the distance determination unit 232 uses the subject distance of the face with the shortest subject distance among the faces obtained from the registration image as the reference distance, and determines the subject distance of each face obtained from the registration image. , whether or not the difference from the reference distance is equal to or less than a predetermined threshold. The distance determination unit 232 outputs to the registration target selection unit 233 a determination result indicating whether or not the difference between the subject distance and the reference distance is equal to or less than a predetermined threshold for each face obtained from the registration image.

The registration target selection unit 233 selects, as a registration target, a partial face image for which the distance determination unit 232 determines that the subject distance satisfies the above conditions. It is assumed that the members of the group are close to each other in the shooting direction when the registration image is captured. Therefore, by determining whether or not the difference between the subject distance and the reference distance is equal to or less than a predetermined threshold, a person inappropriate for registration, such as a person passing through a position away from the members of the group, can be detected. Faces can be excluded.

Note that the distance conditions described above are merely examples, and the distance determination unit 232 may perform determination using other distance conditions. For example, the distance determination section 232 may determine whether or not the subject distance estimated by the distance estimation section 231 is equal to or less than a predetermined threshold. It is assumed that the image for registration is captured in an environment where the subject distance is equal to or less than a predetermined distance. Therefore, by determining whether or not the subject distance is equal to or less than a predetermined threshold value, it is possible to exclude faces of persons inappropriate for registration, such as persons passing far away from the photographing apparatus.

Next, the flow of registration processing according to the fourth embodiment will be described. FIG. 11 is a flowchart of an example of registration processing according to the fourth embodiment. The flow of registration processing will be described below with reference to FIG.

In step S401, the related information creating unit 201 issues a new group ID.
Next, in step S402, the registration image acquisition unit 202 acquires a registration image for the new group.
Next, in step S403, the face detection unit 203 detects all faces appearing in the registration image acquired in step S402.
Next, in step S404, the distance estimation unit 231 estimates the subject distance of each face detected in step S403.
Next, in step S405, the distance determination unit 232 determines whether or not the subject distance estimated in step S404 satisfies a predetermined distance condition for each face detected in step S403.
Next, in step S406, the registration target selection unit 233 selects, as a registration target, the face determined to satisfy the distance condition in step S405 from all the faces detected in step S403.
Next, in step S407, the face registration unit 206 generates registration information by associating the partial images of each of the faces selected in step S406 with the group ID issued in step S401. The face registration unit 206 then transmits the generated registration information to the face authentication server 100 . On the other hand, the registration information management unit 101 of the face authentication server 100 associates the image received as the registration information with the group ID, and stores them in the storage device 54 of the face authentication server 100 .

The fourth embodiment has been described above. According to the information processing system 40 according to the fourth embodiment, a face to be registered is selected depending on whether or not the subject distance satisfies a predetermined distance condition. Therefore, it is possible to exclude persons at subject distances unsuitable as persons to be registered, such as members of a group or persons who are distant from the photographing device. Therefore, it is possible to appropriately register a group to be authenticated in face authentication.

<Embodiment 5>
In the present embodiment, when the variation distance of the facial position of the same person calculated from a plurality of images for registration that are continuously captured at predetermined time intervals is equal to or less than a predetermined threshold value, do.

FIG. 12 is a block diagram showing an example of the configuration of an information processing system 50 according to the fifth embodiment. The information processing system 50 has a face authentication server 110 and a face registration terminal 240 . That is, the information processing system 50 differs from the information processing system 30 according to the third embodiment in that the face registration terminal 220 is replaced with the face registration terminal 240 . The face registration terminal 240 is provided with a coordinate measurement unit 241 and a coordinate determination unit 242 instead of the face gaze direction determination unit 204, and is provided with a registration target selection unit 243 instead of the registration target selection unit 221. It is different from the face registration terminal 220 according to form 3 of . Here, the coordinate determination section 242 in this embodiment corresponds to the determination section 3 in FIG. The face authentication server 110 and the face registration terminal 240 are communicably connected via a wired or wireless network.

The hardware configuration of the face registration terminal 240 is, for example, the configuration shown in FIG. 12 of face registration terminal 240 is performed by processor 53 of face registration terminal 240 reading a computer program from memory 52 and executing it.
Configurations and operations different from those of the third embodiment will be described below, and overlapping descriptions will be omitted as appropriate.

The registration image acquisition unit 202 in the present embodiment acquires N registration images (where N is an integer equal to or greater than 2) continuously photographed at predetermined time intervals (for example, several seconds). It is assumed that the photographing ranges of the N registration images are the same. That is, it is assumed that the photographing area of the photographing device is fixed when photographing a series of N images for registration. Then, the face detection unit 203 detects a face from each of the plurality of continuous registration images acquired by the registration image acquisition unit 202 . The face detection unit 203 then outputs the detected partial image of the face to the same person determination unit 111 of the face authentication server 110 .

The same person determination unit 111 performs matching processing between registration images for faces detected from a plurality of continuous registration images, and associates faces determined to match by matching as faces of the same person. Same person determination section 111 outputs the association result to coordinate determination section 242 of face registration terminal 240 .

The coordinate measurement unit 241 measures the positions of all faces detected by the face detection unit 203 within the registration image. That is, the coordinate measurement unit 241 measures coordinates on the image plane for all faces detected by the face detection unit 203 . The coordinate measurement unit 241 detects the position (coordinates) of a feature point corresponding to a predetermined part of the face (for example, the center point between the eyes) by, for example, known image recognition processing, and calculates the detected position (coordinates ) is the position of the face in the registration image.

The coordinate determination unit 242 determines whether or not the variation distance between the plurality of registration images of the face positions of the same person detected from the plurality of registration images is equal to or less than a predetermined threshold. The coordinate determination unit 242 determines whether or not the positional variation of the face of the same person in each registration image specified from the association result by the same person determination unit 111 is equal to or less than a predetermined threshold.

The registration target selection unit 243 selects, as a registration target, a partial image of a person's face for which the coordinate determination unit 242 determines that the variation distance is equal to or less than a predetermined threshold. At this time, the registration target selection unit 243 selects any partial image of the face of the same person extracted from each of the N registration images as a partial image of the face to be registered. For example, the registration target selection unit 243 selects the partial image of the registration image photographed first among the N registration images as the registration target face partial image. Registration target selection section 243 performs the above-described processing on face images of all persons associated by same person determination section 111 .

It is assumed that the positions of the faces of the members of the group are almost constant when the images for registration are taken. Therefore, by confirming the change in the position of the face in successive images for registration, it is possible to exclude the face of a person who is inappropriate for registration, such as a person passing by.

Next, the flow of registration processing according to the fifth embodiment will be described. FIG. 13 is a flowchart of an example of registration processing according to the fifth embodiment. The flow of registration processing will be described below with reference to FIG.

In step S501, the related information creating unit 201 issues a new group ID.
Next, in step S502, the registration image acquisition unit 202 acquires a plurality of registration images continuously captured at predetermined time intervals (for example, several seconds).
Next, in step S503, the face detection unit 203 detects all faces appearing in the registration image acquired in step S502. That is, the face detection unit 203 detects faces in all images for registration acquired in step S502.
Next, in step S504, the same person determination unit 111 performs matching processing between the registration images for the faces detected in step S503, and associates the faces determined to match by matching as the faces of the same person. .
Next, in step S505, the coordinate measurement unit 241 measures coordinates on the image plane for all faces detected in step S503.
Next, in step S506, the coordinate determination unit 242 determines whether the variation distance between the plurality of registration images of the face positions of the same person detected from the plurality of registration images is equal to or less than a predetermined threshold. judge.
Next, in step S507, the registration target selection unit 243 selects, from all the faces detected in step S503, the face whose variation distance has been determined to satisfy the threshold condition in step S506 as a registration target.
Next, in step S508, the face registration unit 206 generates registration information by associating the partial images of each of the faces selected in step S507 with the group ID issued in step S501. The face registration unit 206 then transmits the generated registration information to the face authentication server 110 . On the other hand, the registration information management unit 101 of the face authentication server 110 associates the image received as the registration information with the group ID, and stores them in the storage device 54 of the face authentication server 110 .

The fifth embodiment has been described above. According to the information processing system 50 according to the fifth embodiment, a face to be registered is selected depending on whether the variation distance of the face position is equal to or less than a predetermined threshold. Therefore, faces of persons inappropriate for registration, such as persons passing by, can be excluded. Therefore, it is possible to appropriately register a group to be authenticated in face authentication.

It should be noted that the present invention is not limited to the above embodiments, and can be modified as appropriate without departing from the scope of the invention. For example, in the above-described embodiments, the same person determination unit is provided in the face authentication server, but may be provided in the face registration terminal. That is, the processing of the same person determination unit may be performed in the face registration terminal. Also, some or all of the optional components in the face registration terminal may be provided in the face authentication server. That is, the processing of any component of the face registration terminal shown in FIGS. 2, 5, 8, 10, or 12 may be performed in the face authentication server. Further, the techniques for determining faces to be registered in each of the above-described embodiments may be combined as appropriate.

In addition, part or all of the above-described embodiments can be described as the following additional remarks, but are not limited to the following.

(Appendix 1)
a face detection unit that detects a face from an image containing the faces of a plurality of people;
a determination unit that determines whether the face detected by the face detection unit satisfies a predetermined condition;
a registration information generation unit that generates registration information that is information that associates each partial image of a plurality of faces determined to satisfy the predetermined condition with an identifier that identifies a group to be authenticated in face authentication; An information processing system having
(Appendix 2)
The determination unit determines whether or not a line-of-sight direction of the face detected by the face detection unit is a predetermined direction as a determination as to whether or not the predetermined condition is satisfied. Information processing system.
(Appendix 3)
The determination unit determines whether or not the predetermined condition is satisfied, and the direction of the line of sight of the face detected by the face detection unit is a predetermined direction, and the face is oriented in the predetermined direction. The information processing system according to appendix 2, wherein it is determined whether or not it is facing.
(Appendix 4)
The face detection unit detects a face from each of a plurality of images continuously captured at predetermined time intervals,
The determination unit determines whether a plurality of faces of the same person detected from the plurality of images satisfy the predetermined condition,
The registration information generation unit associates a partial image of a face of a person determined to satisfy the predetermined condition with respect to the plurality of faces detected from the plurality of images with the identifier, and creates the registration information by associating the identifier with the partial image of the face of the person. The information processing system according to appendix 2 or 3.
(Appendix 5)
further comprising a selection reception unit that receives an instruction to select whether or not the face determined to satisfy the predetermined condition is to be authenticated;
The registration information generation unit associates a partial image selected according to the instruction from among the partial images of each of the plurality of faces determined to satisfy the predetermined condition with the identifier, and creates the registration information by associating the identifier with the partial image. 5. The information processing system according to any one of appendices 1 to 4.
(Appendix 6)
The determination unit determines whether or not the subject distance of the face detected by the face detection unit satisfies a predetermined distance condition as a determination of whether or not the predetermined condition is satisfied. The information processing system according to appendix 1.
(Appendix 7)
The face detection unit detects a face from each of a plurality of images continuously captured at predetermined time intervals,
The determination unit determines whether or not the predetermined condition is satisfied, and determines whether or not a variation distance of positions of the faces of the same person detected from the plurality of images is equal to or less than a predetermined threshold value. determine whether
The registration information generation unit generates the registration information by associating the partial images of the faces of the plurality of persons determined to satisfy the predetermined condition with the identifiers. processing system.
(Appendix 8)
a face detection unit that detects a face from an image containing the faces of a plurality of people;
a determination unit that determines whether the face detected by the face detection unit satisfies a predetermined condition;
Registered information that is information that associates each partial image of a plurality of faces determined to satisfy the predetermined condition with an identifier that identifies a group to be authenticated in face authentication is generated, and the registered information is generated. A terminal device having a registration information generation unit for registration in a face authentication server.
(Appendix 9)
8. The determination unit determines whether or not the line-of-sight direction of the face detected by the face detection unit is a predetermined direction as the determination of whether or not the predetermined condition is satisfied. Terminal equipment.
(Appendix 10)
Detect faces from an image of multiple people's faces,
determining whether the detected face satisfies a predetermined condition;
An authentication target management method for generating registration information that is information that associates each partial image of a plurality of faces determined to meet the predetermined condition with an identifier that identifies an authentication target group in face authentication. .
(Appendix 11)
a face detection step of detecting a face from an image showing faces of a plurality of persons;
a determination step of determining whether the detected face satisfies a predetermined condition;
a registration information generation step of generating registration information that is information that associates each partial image of a plurality of faces determined to satisfy the predetermined condition with an identifier that identifies a group to be authenticated in face authentication; A program that makes a computer run

1, 10, 20, 30, 40, 50 information processing system 2 face detection unit 3 determination unit 4 registration information generation unit 51 network interface 52 memory 53 processor 54 storage device 55 input device 56 output device 100, 110 face authentication server 101 registration Information management unit 102 Authentication determination image acquisition unit 103 Authentication unit 111 Same person determination unit 200, 210, 220, 230, 240 Face registration terminal 201 Related information creation unit 202 Registration image acquisition unit 203 Face detection unit 204 Face line-of-sight direction determination Units 205, 221, 233, 243 Registration target selection unit 206 Face registration unit 211 Selection reception unit 231 Distance estimation unit 232 Distance determination unit 241 Coordinate measurement unit 242 Coordinate determination unit

Claims (9)

a face detection unit that detects a face from an image containing the faces of a plurality of people;
a determination unit that determines whether the face detected by the face detection unit satisfies a predetermined condition;
an output unit for outputting a superimposed image in which a frame line surrounding each partial image of the face determined to satisfy the predetermined condition is superimposed for each of the partial images;
a selection reception unit that receives an instruction to select whether the face is to be authenticated;
Information that associates a partial image selected according to the instruction from among the partial images of each of the plurality of faces determined to satisfy the predetermined condition, and an identifier that identifies a group to be authenticated in face authentication. An information processing system comprising: a registration information generation unit that generates registration information. 2. The determination unit according to claim 1, wherein the determination unit determines whether or not the line-of-sight direction of the face detected by the face detection unit is a predetermined direction as the determination of whether or not the predetermined condition is satisfied. information processing system. The determination unit determines whether or not the predetermined condition is satisfied, and the direction of the line of sight of the face detected by the face detection unit is a predetermined direction, and the face is oriented in the predetermined direction. 3. The information processing system according to claim 2, which determines whether or not it is facing. The face detection unit detects a face from each of a plurality of images continuously captured at predetermined time intervals,
The determination unit determines whether a plurality of faces of the same person detected from the plurality of images satisfy the predetermined condition,
The registration information generation unit associates a partial image of a face of a person determined to satisfy the predetermined condition with respect to the plurality of faces detected from the plurality of images with the identifier, and creates the registration information by associating the identifier with the partial image of the face of the person. The information processing system according to claim 2 or 3, which generates The determination unit determines whether or not the subject distance of the face detected by the face detection unit satisfies a predetermined distance condition as a determination of whether or not the predetermined condition is satisfied. The information processing system according to claim 1. The face detection unit detects a face from each of a plurality of images continuously captured at predetermined time intervals,
The determination unit determines whether or not the predetermined condition is satisfied, and determines whether or not a variation distance of positions of the faces of the same person detected from the plurality of images is equal to or less than a predetermined threshold value. determine whether
2. The registration information generation unit according to claim 1, wherein the registration information generation unit generates the registration information by associating partial images of faces of a plurality of persons determined to satisfy the predetermined condition with the identifiers. Information processing system. a face detection unit that detects a face from an image containing the faces of a plurality of people;
a determination unit that determines whether the face detected by the face detection unit satisfies a predetermined condition;
an output unit for outputting a superimposed image in which a frame line surrounding each partial image of the face determined to satisfy the predetermined condition is superimposed for each of the partial images;
a selection reception unit that receives an instruction to select whether the face is to be authenticated;
Information that associates a partial image selected according to the instruction from among the partial images of each of the plurality of faces determined to satisfy the predetermined condition, and an identifier that identifies a group to be authenticated in face authentication. and a registration information generator that generates registration information and registers the registration information in a face authentication server. Detect faces from an image of multiple people's faces,
determining whether the detected face satisfies a predetermined condition;
outputting a superimposed image in which a frame line surrounding each partial image of the face determined to satisfy the predetermined condition is superimposed for each partial image;
Receiving an instruction to select whether the face is to be authenticated,
Information that associates a partial image selected according to the instruction from among the partial images of each of the plurality of faces determined to satisfy the predetermined condition, and an identifier that identifies a group to be authenticated in face authentication. How to manage authentication targets that generate registration information. a face detection step of detecting a face from an image showing faces of a plurality of persons;
a determination step of determining whether the detected face satisfies a predetermined condition;
an output step of outputting a superimposed image in which a frame line surrounding each partial image of the face determined to satisfy the predetermined condition is superimposed for each of the partial images;
a selection receiving step of receiving an instruction to select whether the face is to be authenticated;
Information that associates a partial image selected according to the instruction from among the partial images of each of the plurality of faces determined to satisfy the predetermined condition, and an identifier that identifies a group to be authenticated in face authentication. A program for causing a computer to execute a registration information generation step for generating registration information.

Images