JP2018195067A - Information processing apparatus and information processing program - Google Patents

Abstract

To provide an information processing apparatus capable of storing a face image for storage by using a face image taken for verification, without newly causing a user to be conscious of the imaging for storage, when collation with the stored face image fails.SOLUTION: When a captured face image satisfies a first condition, first collation means of an information processing apparatus collates the face image with a face image stored in advance. When the collation by the first collation means fails, second collation means collates a face image satisfying a second condition that is more severe than the first condition with the face image stored in advance. When the collation by the second collation means fails, third collation means performs collation according to a collation method different from the collation of the face image. When the collation by the third collation means is successful, first storage means stores a face image that satisfies the second condition.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus and an information processing program.

  In Patent Document 1, it is an object to provide a face image registration apparatus that can capture a plurality of face images having different face orientations and that requires only one camera, and the control unit is registered in the input unit. When an instruction to start the operation is input, the image capturing unit is controlled to capture a plurality of face images, and one or more of the captured face images are stored in the storage unit as registered images respectively. The control unit controls the instruction unit during the registration operation to instruct the registrant to change the orientation of the face, and performs authentication using a plurality of face images thus obtained. The device is easy to use, such as being able to receive authentication without changing the orientation of the face during movement, and the orientation of the face is changed by issuing an instruction to the registrant. One camera is enough for the department There has been disclosed.

  Patent Document 2 discloses an authentication system capable of correctly authenticating under any environment by preparing a plurality of face information under different environments, and a computer-readable storage medium used for the authentication system The authentication person's face is photographed in a plurality of different environments, the face information is registered in the face authentication database with a pattern name corresponding to the environment, An authentication person identification table that associates the registered pattern name with the authentication person photographed with this pattern name is provided, and at the time of authentication, the face of the person to be authenticated is photographed with the camera, The information and a plurality of face information registered in the face authentication database are subjected to 1: N comparison by the authentication processing means, face information with the highest similarity is specified, and the specified face information From the pattern name, to identify the corresponding authentication person by person identification table, thus also different authentication time environment, it is disclosed that it is possible to authenticate the authentication target person reliably whether authentication person or not.

  A device is known in which a face image obtained by photographing a user of a device with a camera is compared with a face image of a user that has been stored in advance, and authentication is performed by the user who has been verified when the comparison is successful. ing. In addition, in the related art, there is a technique that captures a plurality of face images having different face orientations when storing a face image in order to increase the accuracy of collation.

JP 2009-258991 A JP-A-2005-115481

If the shooting environment at the time of storage is different from that at the time of verification, there are cases where the verification cannot be performed even when compared with the stored face image. Therefore, a method of additionally storing a face image under a situation where the photographing environment is different is conceivable. However, it is not desirable and difficult to cause a user who has already stored the face image to be stored many times.
In the present invention, when a face image that has been stored cannot be verified, the face for storage can be stored using the face image captured for verification without causing the user to be aware of a new image for storage (including shooting). An object of the present invention is to provide an information processing apparatus and an information processing program capable of storing an image.

The gist of the present invention for achieving the object lies in the inventions of the following items.
According to the first aspect of the present invention, when the captured face image satisfies the first condition, the first matching means for matching the face image with the previously stored face image, and the first matching means A second collating unit that collates a face image satisfying a second condition, which is stricter than the first condition, with the previously stored face image when the collation by When the collation by the collation unit fails, a third collation unit that performs collation by a collation method different from the collation of the face image, and when the collation by the third collation unit is successful, the second condition is An information processing apparatus having first storage means for storing a face image to be satisfied.

  According to a second aspect of the present invention, the first condition or the second condition includes at least elements of a face size in a face image, a face orientation, a degree of stillness, blinking, and a facial expression acceptance range. The information processing apparatus according to claim 1, comprising at least one.

  According to a third aspect of the present invention, at least one of the elements is different between the first condition and the second condition, and the elements of the second condition different from each other are imaging of the face image stored in advance. The information processing apparatus according to claim 2, which is similar to a condition.

  The invention of claim 4 further includes second storage means for storing a face image after imaging satisfying the second condition, and the first storage means is stored in the second storage means. The information processing apparatus according to claim 1, wherein a face image is stored.

  The invention according to claim 5 deletes the face image stored in the second storage means when the exit detecting means detects that the user of the face image has left, and the exit detecting means detects the exit. The information processing apparatus according to claim 4, further comprising deletion means for performing the deletion.

  The invention according to claim 6 is the information processing apparatus according to claim 1, further comprising display means for displaying an imaging guide under the second condition when the collation by the first collation means fails.

  The invention according to claim 7 further includes an exit detection means for detecting that the user of the face image has exited, and the display means displays the guidance displayed when the exit is detected by the exit detection means. Is an information processing apparatus according to claim 6.

The invention of claim 8 is characterized in that the evacuation detecting means detects the evacuation by any one of the following or a combination thereof.
(1) The face is not included in the captured image,
(2) that the user cannot be detected by a sensor;
(3) a person is not included in an image captured by an image capturing apparatus different from the image capturing apparatus capturing the face image;
An information processing apparatus according to claim 5 or 7.

  According to a ninth aspect of the present invention, when the captured face image satisfies the first condition, the computer compares the face image with the face image stored in advance, and the first matching means. A second collating unit that collates a face image satisfying a second condition that is stricter than the first condition and the previously stored face image when the collation by the collating unit fails, When the collation by the second collating unit fails, a third collating unit that collates by a collation method different from the collation of the face image, and when the collation by the third collating unit succeeds, the second It is an information processing program for functioning as a 1st memory | storage means to memorize | store the face image which satisfy | fills these conditions.

  According to the information processing apparatus of the first aspect, when it is not possible to collate with a stored face image, the face image captured for collation is used for storage without making the user newly aware of the image for storage. Can be stored.

  According to the information processing apparatus of claim 2, as the first condition or the second condition, the size of the face in the face image, the orientation of the face, the degree of stillness, the presence or absence of blinking, and the elements of the acceptance range of facial expressions At least one of the above can be used.

  According to the information processing apparatus of claim 3, the first condition and the second condition are different in at least one of the elements, and different elements of the second condition are pre-stored face image capturing conditions. Those similar to can be used.

  According to the information processing apparatus of the fourth aspect, it is possible to store the captured face image that satisfies the second condition.

  According to the information processing apparatus of the fifth aspect, when the retreat is detected, the face image stored in the second storage unit can be deleted.

  According to the information processing apparatus of the sixth aspect, it is possible to display an imaging guide under the second condition.

  According to the information processing apparatus of the seventh aspect, when leaving is detected, the displayed guidance can be hidden.

  According to the information processing apparatus of claim 8, (1) the face is not included in the captured image, (2) the user cannot be detected by the sensor, and (3) the imaging apparatus that captures the face image. The fact that no person is included in an image captured by an image capturing apparatus different from the above can be detected.

  According to the information processing program of claim 9, when it is not possible to collate with a stored face image, the user uses the face image captured for verification without making the user newly aware of the image for storage. Can be stored.

It is a conceptual module block diagram about the structural example of 1st Embodiment. It is explanatory drawing which shows the system configuration example using this Embodiment. It is a flowchart which shows the process example by this Embodiment. It is a flowchart which shows the process example by 1st Embodiment. It is explanatory drawing which shows the example of a display of a screen. It is explanatory drawing which shows the example of a display of a screen. It is explanatory drawing which shows the example of a display of a screen. It is explanatory drawing which shows the example of a data structure of a user ID and face image correspondence table. It is explanatory drawing which shows the example of a data structure of a user ID / password correspondence table. It is a conceptual module block diagram about the structural example of 2nd Embodiment. It is a flowchart which shows the process example by 2nd Embodiment. It is a flowchart which shows the process example by 2nd Embodiment. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Hereinafter, examples of various preferred embodiments for realizing the present invention will be described with reference to the drawings.
First, before describing the present embodiment, the premise or an image processing apparatus using the present embodiment will be described. This description is intended to facilitate understanding of the present embodiment.

2. Description of the Related Art An image processing apparatus that performs verification by comparing an image captured by a user of an image processing apparatus with a user's face image stored in advance and performs authentication when the verification is successful is known. ing.
In such an image processing apparatus, it is possible to reduce the occurrence of erroneous recognition by capturing a user's face stored in advance with a stable and high-quality image. For example, if the user's face image is taken as close to the camera as possible, the resolution will be finer, and the proportion of the face image in the entire image will be higher, so it will be less affected by the brightness of the background image, etc. It becomes an image of image quality.
For this reason, by having the user follow a guidance such as pointing in a specific direction or shooting in a stationary state, a stable image can be taken under the same conditions, and the occurrence of erroneous recognition can be reduced. Also, as in the technique disclosed in Patent Document 1, a plurality of face images with different face orientations can be taken according to guidance stored in advance, and the accuracy at the time of recognition can be increased. However, when the light source environment is significantly different, there are cases where the collation cannot be performed even when compared with the stored face image. In the technique disclosed in Patent Document 2, a method of additionally storing a face image under a situation where photographing environments are different has been proposed. However, it is time consuming and undesirable for the user who has already been stored to perform the step of registering the face image many times.
On the other hand, when the image processing apparatus is used, the occurrence of misrecognition can be reduced if shooting is performed under the same conditions as those stored in advance. However, since the image processing apparatus is frequently used at the time of recognition, unlike the case where it is stored in advance, it is inconvenient for the user to give instructions or restrictions to the user's shooting. Therefore, it is desirable that a user who has come to use the image processing apparatus can authenticate by using an image reflected on the digital camera in a natural posture without limiting as much as possible.

Thus, in the present embodiment, for example, the following processing is performed. In particular, this description is intended to facilitate understanding of the present embodiment, and is not intended to be interpreted in a limited manner using this description. And it should not be judged that the invention to be patented is described in the detailed explanation of the invention (Patent Act Article 36 (6) (i)) using only this explanation part. It is natural.
When verification fails with an image captured with a digital camera, the camera guides the user to capture the face under the same strict conditions (face size, orientation, position, etc.) as when the face image was stored in advance. If the face image at that time is unsuccessful even after collating for a set period, the face image at that time is temporarily stored, and after authenticating using a method other than the face image, the face image of the authenticated user must be stored in advance. For example, the temporarily stored face image is stored as an additional face image of the user. Note that “temporarily store” is deleted when it is no longer necessary to store a face image, for example, when a predetermined period has elapsed, or when a user cannot be detected. There are things to do.
As a result, if face authentication fails when using the image processing apparatus, the image is captured under the same conditions as those stored in advance, and verification is facilitated by performing collation. If the authentication cannot be performed even if the images are combined, it is determined that the imaging environment has changed significantly, and a face image is added. At this time, since the face image has already been captured under the image capturing conditions stored in advance, there is no need to newly store the face image, and the user can easily store a new face image. Become.

FIG. 1 is a conceptual module configuration diagram of a configuration example according to the first embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment is a computer program for causing these modules to function (a program for causing a computer to execute each procedure, a program for causing a computer to function as each means, and a function for each computer. This also serves as an explanation of the program and system and method for realizing the above. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used. However, when the embodiment is a computer program, these words are stored in a storage device or stored in memory. This means that control is performed so as to be stored in the apparatus. Modules may correspond to functions one-to-one, but in mounting, one module may be configured by one program, or a plurality of modules may be configured by one program, and conversely, one module May be composed of a plurality of programs. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.). “Predetermined” means that the process is determined before the target process, and not only before the process according to this embodiment starts but also after the process according to this embodiment starts. Also, if it is before the target processing, it is used in accordance with the situation / status at that time or with the intention to be decided according to the status / status up to that point. When there are a plurality of “predetermined values”, they may be different values, or two or more values (of course, including all values) may be the same. In addition, the description of “do B when A” is used to mean “determine whether or not A and do B when A”. However, the case where it is not necessary to determine whether or not A is excluded. In addition, when enumerating things such as “A, B, C”, etc., it is an enumerated list unless otherwise specified, and includes cases where only one of them is selected (for example, only A).
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by a communication means such as a network (including one-to-one correspondence communication connection), and the like. The case where it implement | achieves by etc. is included. “Apparatus” and “system” are used as synonymous terms. Of course, the “system” does not include a social “mechanism” (social system) that is an artificial arrangement.
In addition, when performing a plurality of processes in each module or in each module, the target information is read from the storage device for each process, and the processing result is written to the storage device after performing the processing. is there. Therefore, description of reading from the storage device before processing and writing to the storage device after processing may be omitted. Here, the storage device may include a hard disk, a RAM (Random Access Memory), an external storage medium, a storage device via a communication line, a register in a CPU (Central Processing Unit), and the like.

  The authentication apparatus 100, which is the information processing apparatus according to the first embodiment, performs verification of the user in order to authenticate the user. As shown in the example of FIG. First condition detection module 110, verification A module 115, face image storage module 120, second condition detection module 125, verification B module 135, storage processing module 140, authentication processing module 145, guide display module 150, and control module 155 Have. Note that authentication refers to making the authentication device 100 (or an information processing device 200 including the authentication device 100 described later, an image processing device 250, etc.) usable (so-called login processing). The collation is to compare information related to the user (for example, a face image, a combination of a user ID and a password) and data stored in advance, and a case where they match is referred to as success, and a case where they do not match is referred to as failure. The verification needs to succeed for authentication.

The first condition and the second condition will be described. The first condition and the second condition are that the face image captured by the imaging module 105 must be satisfied when collating with the face image stored in advance (including the concept of registration) in the face image storage module 120. It shows the conditions that must be met. Specifically, the first condition is a condition required for collating face images. The second condition is a condition necessary for storing a face image (so-called original face image pattern) in the face image storage module 120. The second condition is stricter than the first condition. Here, “the condition is severe” indicates that there are many (or large) restrictions on imaging.
As the first condition or the second condition, the size of the face in the face image, the direction of the face, the degree of stillness, the presence / absence of blinking, the acceptable range of facial expressions (for example, allowing smiles, Or at least one of the other elements). For example, when the size of the face in the face image is adopted as a different condition, the number of pixels of the face image is A pixels or more under the first condition, and the number of pixels of the face image is B under the second condition. (A <B) It is greater than or equal to pixels. Further, for example, when the degree of stillness is adopted as a different condition, the first condition allows even if there is a blur at the time of imaging, and the second condition requires that the person is stationary at the time of imaging. Is.
The first condition and the second condition may be different in at least one of the elements described above. The different second condition elements may be similar to the face image capturing conditions stored in the face image storage module 120 in advance. For example, the first condition may be a minimum condition for recognizing a face image (specifically, a face image is captured), and the second condition is a face image storage. It may be the same as the imaging condition when the face image stored in the module 120 is captured.
This will be described more specifically.
The first condition is that, for example, parts (face elements) such as eyes and mouth can be detected, and the size of the face is 80 pixels or more. In addition to the first condition, the second condition includes that the face size is 120 pixels or more, the face angle is within plus or minus 5 degrees in both the left and right, and top and bottom directions.
For example, when the degree of stillness is used, the amount of blurring of the face portion on the image is detected, the condition is satisfied regardless of the amount of blurring in the first condition, and the amount of blurring is predetermined in the second condition. It is determined that the condition is not satisfied unless it is less than the threshold value. When using the presence / absence of blinking, the degree of opening / closing of the eyes on the image is detected, and the first condition satisfies the condition regardless of the degree of opening / closing, and the second condition sets a predetermined threshold value. Otherwise, it is determined that the condition is not satisfied.

  The imaging module 105 is connected to the first condition detection module 110, the second condition detection module 125, and the storage processing module 140. The imaging module 105 captures a user's face image. For example, a digital camera or the like as an imaging device. The imaging module 105 supplies the captured face image to the first condition detection module 110 and the second condition detection module 125, and causes the storage processing module 140 to store the face image that satisfies the second condition. Note that the imaging device may be built in the authentication device 100 or may be connected as an external device of the authentication device 100. There may be a plurality of imaging devices. In that case, any imaging device may be used for imaging in (collation A-1 processing) described later, but a predetermined imaging device may be used in imaging in (collation A-2 processing). Good. Examples of the “predetermined imaging device” include an imaging device with a higher resolution than other imaging devices. Further, the image to be captured may be one or a plurality of images. In the case of a plurality of sheets, specifically, one imaging device may capture a plurality of images, a plurality of imaging devices may capture one image at a time, or a plurality of imaging devices capture a plurality of images. Also good. In the collation, a plurality of face images may be targeted.

  The first condition detection module 110 is connected to the imaging module 105 and the verification A module 115. The first condition detection module 110 detects whether the face image captured by the imaging module 105 satisfies the first condition, and passes the detection result to the verification A module 115. The detection process may use a known face image processing technique (recognition technique). In the example described above, it is detected whether or not the number of pixels of the face image is greater than or equal to A pixels.

The verification A module 115 is connected to the first condition detection module 110, the face image storage module 120, the second condition detection module 125, the verification B module 135, and the authentication processing module 145. When the face image captured by the imaging module 105 satisfies the first condition using the detection result of the first condition detection module 110, the verification A module 115 stores the face image and the face image storage module 120 in advance. The face image is collated (collation A-1 process). The matching process may use a known facial image recognition technique.
Then, when the collation by the (collation A-1 process) fails, the collation A module 115 uses the detection result by the second condition detection module 125 to determine the second condition that is stricter than the first condition. The face image to be satisfied is collated with the face image stored in advance in the face image storage module 120 (collation A-2 processing). The matching process may use a known facial image recognition technique.

  The face image storage module 120 is connected to the collation A module 115 and the storage processing module 140. The face image storage module 120 stores a face image for verification. Then, the stored face image is provided to the collation A module 115, and the face image is added and stored by the storage processing module 140. Here, in “addition and storage”, the number of stored face images is increased (the face images already stored are kept as they are), and the already stored face images are replaced with the face images from the storage processing module 140. It is included.

  The second condition detection module 125 is connected to the imaging module 105 and the verification A module 115. The second condition detection module 125 detects whether the face image captured by the imaging module 105 satisfies the second condition, and passes the detection result to the verification A module 115. The detection process may use a known face image processing technique (recognition technique). In the example described above, it is detected whether or not the number of pixels of the face image is greater than or equal to B pixels.

  The verification B module 135 is connected to the verification A module 115, the storage processing module 140, and the authentication processing module 145. When the collation by the collation A module 115 (collation A-2 processing) fails, the collation B module 135 performs collation by a collation method different from the collation of face images. For example, there are an IC card owned by the user, collation of a user ID and a password, biometric information other than a face image (fingerprint, iris, etc.) and the like. Note that it is preferable to perform collation in which the probability of successful collation is higher than collation of face images (that the person can be correctly collated without being influenced by imaging conditions as in collation of face images).

The storage processing module 140 is connected to the imaging module 105, the face image storage module 120, and the verification B module 135. The storage processing module 140 stores a face image that satisfies the second condition after the face registration instruction is given.
Specifically, when the control module 155 receives a face registration instruction and selects a user to register a face, the imaging module 105 captures the face of the user to be registered, and the second condition is satisfied. The face image stored in the storage processing module 140 (the face image satisfying the second condition) is stored in the face image storage module 120.
The storage processing module 140 may store a face image that satisfies the second condition when the collation by the collation B module 135 is successful. The storage in the storage processing module 140 includes temporary storage as described above. As for “whether or not the second condition is satisfied”, the detection result by the second condition detection module 125 may be used.
Specifically, when the collation by the collation B module 135 is successful, the storage processing module 140 displays the face image (face image satisfying the second condition) stored in the storage processing module 140 as the face image storage module 120. Remember me.

  The authentication processing module 145 is connected to the verification A module 115 and the verification B module 135. The authentication processing module 145 performs authentication processing when the verification is successful by the verification A module 115 or the verification B module 135. As a result, the user can use the information processing apparatus 200 or the image processing apparatus 250 including the authentication apparatus 100. Of course, the functions permitted for the user (for example, color printing is possible, fax transmission is not possible) can be used. Then, the usage fee may be charged to the user (or the organization to which the user belongs).

The guide display module 150 displays an imaging guide (also referred to as a guide) when imaging is performed by the imaging module 105. Further, the guide display module 150 displays guidance for collation by the collation B module 135.
In particular, the guide display module 150 displays imaging guidance under the second condition when the collation by the collation A module 115 (collation A-1 process) fails. As a result, even if the collation by (collation A-1 process) fails, the probability that the collation by (collation A-2 process) succeeds increases. And even if collation by (collation A-2 process) fails, it becomes possible to capture a face image that satisfies the second condition. As a result, a face image can be added and stored in the face image storage module 120. “Guidance for imaging under the second condition” is to prompt the user to satisfy the second condition. For example, a captured face image is displayed and a frame indicating the size of the face image is displayed. Then, a guidance prompting to fill the frame with the face image is displayed. The guidance may be presented with sound output by a speaker, tactile sensation using a tactile device, and the like.
The control module 155 controls processing of each module in the authentication device 100. It also controls the processing order of modules.

FIG. 2 is an explanatory diagram showing a system configuration example using the present embodiment.
The information processing apparatus 200 includes an authentication apparatus 100 (authentication apparatus 900) and a functional module 210. The authentication device 100 (authentication device 900) and the function module 210 are connected. A user who is authenticated by the authentication device 100 (authentication device 900) can use the information processing device 200 (function of the function module 210). The information processing apparatus 200 includes an image processing apparatus 250, which will be described later, but is not limited thereto. Examples of the information processing apparatus 200 include devices that require authentication, such as an entrance / exit management device, a health management device, and a household electrical device.
The image processing apparatus 250 includes an authentication apparatus 100 (authentication apparatus 900) and an image processing module 260. The authentication device 100 (authentication device 900) and the image processing module 260 are connected. A user who is authenticated by the authentication apparatus 100 (authentication apparatus 900) can use the image processing apparatus 250 (function of the image processing module 260). Examples of the image processing apparatus 250 include a copying machine, a fax machine, a scanner, a printer, and a multifunction machine (an image processing apparatus having any two or more functions such as a scanner, a printer, a copying machine, and a fax machine). For example, it is used as follows. A user issues a print instruction using a personal computer connected to the image processing apparatus 250 via a communication line. From the viewpoint of security, in order to avoid leaving the printed matter, the printing process is performed after the user is authenticated. That is, the user is authenticated by the face image at the image processing apparatus 250 in order to receive the printed matter. Then, the image processing module 260 executes a print instruction by the user corresponding to the face image, and performs printing.

For example, the authentication device 100 (authentication device 900) performs the following processing. In particular, this description is intended to facilitate understanding of the present embodiment, and is not intended to be interpreted in a limited manner using this description. And it should not be judged that the invention to be patented is described in the detailed explanation of the invention (Patent Act Article 36 (6) (i)) using only this explanation part. It is natural.
When the activation is completed, the image processing device 250 displays an image captured by the digital camera on a panel (display device, for example, the screen 400 shown in the example of FIG. 5) and performs processing for detecting a face from the captured image. At this time, if the first condition is not satisfied, it is determined that the image cannot be detected because it is determined to be an image that is insufficient for later face matching. If the face cannot be detected, the imaging by the digital camera, the display on the panel, and the detection of the face are repeated. On the other hand, when a face can be detected, a process of collating with a face image of a user who has already been registered is performed. If the collation is successful, authentication is performed as a user who has registered the collated face image. If verification is not possible, processing is performed again from the digital camera. If verification is not possible after a predetermined period of time, a guide is displayed on the panel, and the original face image pattern A face can be imaged under the same condition (specific example of the second condition) as that during storage (described later using the example of FIG. 3). For example, the following is displayed on the panel. “Still so that the face appears as large as possible in the image”. In addition, a guide other than characters may be used such as displaying a frame superimposed on an image of a digital camera.
Thereafter, the face is detected again. At this time, if the second condition which is the same as the condition used when registering the face image is not satisfied, it is considered that the face cannot be detected. If the face is detected, a process for further collating the face is performed. If collation fails here, the captured face image is temporarily stored, and the screen on the panel is further changed to encourage the user to authenticate with a user ID and password (for example, as shown in the example of FIG. 6). Screen 500). If the user ID and password are keyed in, verification is performed, and if there is a matching user, authentication is performed as that user. At this time, if a face image has already been registered with the target user, the face image temporarily stored is associated as an additional face image. As a result, when an image is captured in the same external light environment in the future, this face image is also collated, so that the face collation is successful and the authentication is successful.

When associating a face image with a user in advance, it is performed as follows. While the administrator is logged in to the image processing apparatus, one user ID is selected and the association of the face image is instructed from the panel. If a face can be detected under the conditions used when registering the face image, the captured face image is associated with the selected user ID.
FIG. 3 is a flowchart showing a processing example (an example of face image storage processing) according to this embodiment.
In step S350, the face image storage process is started.
In step S352, a face image is captured.
In step S354, it is determined whether or not the captured face image satisfies the second condition. If satisfied, the process proceeds to step S356. Otherwise, the process returns to step S352.
In step S356, the captured face image is stored in the face image storage module 120 in association with the user ID. That is, the user ID / face image correspondence table 700 shown in the example of FIG. 8 to be described later is generated. The user ID may be selected before imaging, or may be selected after imaging and before storing in the face image storage module 120.
In step S359, the face image storage process ends.
Note that this processing is desirably performed by the information processing apparatus 200 or the image processing apparatus 250 that performs collation. This is because the conditions for capturing the original face image pattern and the face image for collation are the same. However, it is not excluded to perform this process in another device, and the face image subjected to this process in another device may be stored in the face image storage module.

FIG. 4 is a flowchart illustrating a processing example according to the first exemplary embodiment. An example of face image authentication processing (an example of face recognition processing) is shown.
In step S302, a face image is captured. For example, a display like a screen 400 shown in FIG. 5 is performed. FIG. 5 is an explanatory diagram showing a display example of the screen 400. On the screen 400, a guide display area 410, a face image display area 420, a face image display designation area 430, and a help button 440 are displayed. In the guide display area 410, for example, “Please keep your face in the frame while looking at the screen” is displayed. The display content in step S302 is guidance for satisfying the first condition. In the face image display area 420, the captured face image is displayed in real time. This is so that the user can check his / her face captured. When the face image display designation area 430 is selected by the user, the face image is displayed in the face image display area 420. When the face image display designation area 430 is not selected, the face image display area 420 is displayed. Does not display face images. When the help button 440 is selected (clicked) by the user, an instruction manual is displayed. For example, the first condition may be displayed.

  In step S304, it is determined whether the captured face image satisfies the first condition. If satisfied, the process proceeds to step S306. Otherwise, the process returns to step S302. For example, if “No” is determined in the process in step S304 for a predetermined period (or a predetermined number of times of imaging), the process may proceed to step S308. Of course, this processing is an example and is not necessarily performed. When returning to step S302, guidance for satisfying the first condition may be displayed in the guide display area 410 in the screen 400 shown in the example of FIG. For example, when the size of the face in the face image is less than the threshold, “Please get closer” or the like may be displayed.

  In step S306, it is determined whether or not the collation with the face image stored in the face image storage module 120 is successful. If successful, the process proceeds to step S326, and otherwise, the process proceeds to step S308. In this example, as soon as the collation fails in step S306, the process proceeds to step S308. However, the imaging may be repeated by returning to step S302 until the collation fails for a certain period. In step S306, for example, the user ID / face image correspondence table 700 may be used to collate with the captured face image. FIG. 8 is an explanatory diagram showing an example of the data structure of the user ID / face image correspondence table 700. The user ID / face image correspondence table 700 has a user ID field 710, a face image 1 field 720, and a face image 2 field 730. The user ID column 710 stores information (user ID: IDentification) for uniquely identifying a user in the present embodiment. The face image 1 field 720 stores the face image 1 of the user. The face image 2 column 730 stores the face image 2 of the user. In the initial state after the face registration (immediately after the face image storing process shown in the example of FIG. 3), the face image 2 field 730 may not store a face image. In step S324, the face image may be stored in the face image 2 column 730. Further, the face image stored in the face image 1 field 720 and the face image 2 field 730 may be the face image itself or feature data extracted from the face image. In this example, there is only one additional face image, but there may be a plurality of additional face images.

In step S308, a guide is displayed. As a guide, there is a guide that prompts the user to satisfy the second condition. For example, you may display on the guide display area 410 in the screen 400 shown in FIG. Furthermore, for example, a captured face image may be displayed, a frame indicating the size of the face image may be displayed, and guidance for filling the frame with the face image may be displayed.
In step S310, a face image is captured.

  In step S312, it is determined whether or not the captured face image satisfies the second condition. If satisfied, the process proceeds to step S314. Otherwise, the process returns to step S308. For example, if “No” is determined in the process in step S312 during a predetermined period (or a predetermined number of times of imaging), error processing may be performed. For example, the process of step S318, step S320, and step S322 (that is, the process of step S316 and step S324 is not performed) is performed.

In step S314, it is determined whether or not the collation with the face image stored in the face image storage module 120 is successful. If successful, the process proceeds to step S326. Otherwise, the process proceeds to step S316. For example, a process equivalent to the process in step S306, that is, a user ID / face image correspondence table 700 may be used to collate with the captured face image.
In step S316, the detected face image is stored in the storage processing module 140 as the second condition adaptive face image.

  In step S318, another collation method other than the face image is guided. As another verification method, for example, there is verification by a combination of a user ID and a password. Specifically, the screen 500 shown in FIG. 6 may be displayed to prompt the user ID and password to be entered. FIG. 6 is an explanatory diagram showing a display example of the screen 500. The screen 500 displays a keyboard authentication area 510, a guide display area 520, a user ID reception field 530, a password reception field 540, a login button 550, and a close button 560. In the guide display area 520, for example, “Authentication failed. Enter user information with the keyboard to authenticate. Press [Close] to return to the face recognition screen.” Is displayed. A user ID is keyed in the user ID reception field 530. A password is keyed in the password reception field 540. When the login button 550 is selected, the combination of the user ID and the password is verified (step S320). When the close button 560 is selected, the process returns to step S302 (or step S308).

  In step S320, it is determined whether or not the collation is successful. If successful, the process proceeds to step S322. Otherwise, the process returns to step S308. For example, the user ID / password correspondence table 800 may be used to collate the keyed-in user ID and password combination. FIG. 9 is an explanatory diagram showing an example of the data structure of the user ID / password correspondence table 800. The user ID / password correspondence table 800 has a user ID column 810 and a password column 820. The user ID column 810 stores a user ID. The password column 820 stores a password corresponding to the user ID.

In step S322, an authentication process is performed. A so-called login process is performed. Thereafter, the authenticated user can use the information processing apparatus 200 (or the image processing apparatus 250).
In step S324, the face image is additionally stored in the authenticated user. That is, the storage processing module 140 causes the face image storage module 120 to store the face image in the storage processing module 140 as the face image of the user verified in step S320. For example, the face image in the storage processing module 140 is stored in the face image 2 field 730 of the user ID / face image correspondence table 700. Further, the face image in the face image 1 column 720 of the user ID / face image correspondence table 700 may be replaced with the face image in the storage processing module 140.
Further, the screen 600 may be displayed when performing the process of step S324. FIG. 7 is an explanatory diagram illustrating a display example of the screen 600. The screen 600 includes a face information addition area 610, a guide display area 620, a face image display area 630, a user ID display area 640, a user name display area 650, a yes (add) button 660, and a no (do not add) button 670. Is displayed. In the guide display area 620, for example, “This time face information can be added to increase the probability of face recognition. Do you want to add face information?” Is displayed. In the face image display area 630, a face image to be stored in the face image storage module 120 (a face image in the storage processing module 140 that satisfies the second condition) is displayed. In the user ID display area 640, the user ID keyed in the user ID reception field 530 of the screen 500 shown in the example of FIG. 6 may be displayed, or the user ID may be keyed in again. In the user name display area 650, the password keyed in the password reception field 540 of the screen 500 shown in the example of FIG. 6 may be displayed, or the password may be keyed in again. When the Yes (add) button 660 is selected, a feedback display indicating that a face image has been added is displayed, and the screen transitions to the initial screen when authentication is successful. When the No (not added) button 670 is selected, the screen transitions to the initial screen when authentication is successful.

In step S326, an authentication process is performed. A so-called login process is performed. Thereafter, the authenticated user can use the information processing apparatus 200 (or the image processing apparatus 250).
In step S399, the face recognition process ends.

<< Second Embodiment >>
FIG. 10 is a conceptual module configuration diagram of a configuration example according to the second embodiment.
The authentication apparatus 900, which is the information processing apparatus according to the second embodiment, performs user verification. As illustrated in the example of FIG. 10, the imaging module 105, the first condition detection module 110, and verification are performed. A module 115, face image storage module 120, second condition detection module 125, collation B module 135, storage processing module 140, authentication processing module 145, guide display module 150, control module 155, and withdrawal module 960.
In addition, the same code | symbol is attached | subjected to the site | part of the same kind as 1st Embodiment, and the overlapping description is abbreviate | omitted.

The imaging module 105 is connected to the first condition detection module 110, the second condition detection module 125, and the storage processing module 140.
The first condition detection module 110 is connected to the imaging module 105 and the verification A module 115.
The verification A module 115 is connected to the first condition detection module 110, the face image storage module 120, the second condition detection module 125, the verification B module 135, and the authentication processing module 145.
The face image storage module 120 is connected to the collation A module 115 and the storage processing module 140.
The second condition detection module 125 is connected to the imaging module 105 and the verification A module 115.
The verification B module 135 is connected to the verification A module 115, the storage processing module 140, and the authentication processing module 145.
The storage processing module 140 is connected to the imaging module 105, the face image storage module 120, and the verification B module 135.
The authentication processing module 145 is connected to the verification A module 115 and the verification B module 135.

The leaving module 960 includes a leaving detection module 965 and a leaving processing module 970.
The leaving detection module 965 detects that the user of the face image (the user who captured the face image) has left.
For example, the leaving detection module 965 may detect the leaving by any one of the following methods or a combination thereof.
(1) The face is not included in the image captured by the imaging module 105.
(2) The user cannot be detected by the sensor. As this sensor, for example, there is a sensor for detecting the location of a person using infrared rays, ultrasonic waves, visible light, or the like.
(3) No person is included in the image captured by the image capturing apparatus B different from the image capturing module 105 that captured the face image. As the “imaging device B different from the imaging module 105”, for example, an authentication device 900 (or an information processing device 200 including the authentication device 900, an image processing device 250) and a user detection imaging device. It is. For example, when it is detected by the imaging device B that the user has approached the authentication device 900, the authentication device 900 can be used from a power saving mode (also referred to as an energy saving mode, a sleep mode, a low power mode, etc.). (A state in which the functions of the information processing apparatus 200 and the image processing apparatus 250 can be used immediately).

The retreat processing module 970 performs processing for preventing erroneous registration of face images. For example, the following processing is performed.
The leaving processing module 970 deletes the face image stored by the storage processing module 140 when the leaving detection module 965 detects the leaving.
Further, the evacuation processing module 970 controls the guidance displayed on the guide display module 150 to be hidden when the eviction is detected by the evacuation detection module 965. For example, the screen 400 shown in FIG. 5, the screen 500 shown in FIG. 6, and the screen 600 shown in FIG. 7 are hidden.

  FIG. 11 is a flowchart illustrating a processing example according to the second exemplary embodiment. An example of face image authentication processing (an example of face recognition processing) is shown. The process of step S1026 and step S1028 is added to the flowchart shown in the example of FIG. That is, processes other than step S1026 and step S1028 are equivalent to the processes shown in the example of FIG.

In step S1002, a face image is captured.
In step S1004, it is determined whether the captured face image satisfies the first condition. If satisfied, the process proceeds to step S1006. Otherwise, the process returns to step S1002. For example, if “No” is determined in the process in step S1004 for a predetermined period (or a predetermined number of times of imaging), the process may proceed to step S1008.
In step S1006, it is determined whether or not the collation with the face image stored in the face image storage module 120 is successful. If successful, the process proceeds to step S1030. Otherwise, the process proceeds to step S1008.

In step S1008, a guide is displayed.
In step S1010, a face image is captured.
In step S1012, it is determined whether or not the captured face image satisfies the second condition. If satisfied, the process proceeds to step S1014. Otherwise, the process proceeds to step S1026. For example, if “No” is determined in the process in step S1012 during a predetermined period (or a predetermined number of times of imaging), error processing may be performed. For example, the process of step S1018, step S1020, and step S1022 (that is, the process of step S1016 and step S1024 is not performed) is performed.
In step S1014, it is determined whether or not the collation with the face image stored in the face image storage module 120 is successful. If successful, the process proceeds to step S1030. Otherwise, the process proceeds to step S1016.
In step S1016, the detected face image is stored in the storage processing module 140 as the second condition adaptive face image.
In step S1018, another collation method other than the face image is guided.
In step S1020, it is determined whether or not the collation is successful. If successful, the process proceeds to step S1022, and otherwise, the process proceeds to step S1026.
In step S1022, authentication processing is performed. A so-called login process is performed. Thereafter, the authenticated user can use the information processing apparatus 200 (or the image processing apparatus 250).
In step S1024, the face image is additionally stored in the authenticated user.

In step S1026, it is detected whether the user has left the information processing apparatus 200 (or the image processing apparatus 250) (the absence of a person before the information processing apparatus 200 (or the image processing apparatus 250)). If it is detected, the process proceeds to step S1028. Otherwise, the process proceeds to step S1008.
In step S1028, a leaving process is performed, and the process returns to step S1002. Detailed processing in step S1028 will be described later using the flowchart shown in the example of FIG.

In step S1030, an authentication process is performed. A so-called login process is performed. Thereafter, the authenticated user can use the information processing apparatus 200 (or the image processing apparatus 250).
In step S1099, the face recognition process ends.

  In the example illustrated in FIG. 11, step S1026 is performed when “No” is determined in steps S1012 and S1020, but step S1026 may be performed at another location. For example, step S1026 may be performed in any one of steps after step S1008. Further, when it is detected that the user has left, the process of step S1028 may be performed as an interrupt process.

FIG. 12 is a flowchart illustrating a processing example according to the second exemplary embodiment.
In step S1100, the leaving process is started.
In step S1102, the storage processing module 140 is deleted (cleared). That is, the face image (face image satisfying the second condition) temporarily stored in the storage processing module 140 is deleted.
In step S1104, the guide by the guide display module 150 is hidden.
In step S1106, the matching process and the authentication process are returned to the initial values. For example, in step S1004 and step S1012, in order to compare with a predetermined period (or a predetermined number of times of imaging, etc.), the measured imaging time (or the number of times of imaging) is returned to the initial value, etc. Is applicable.
In step S1199, the leaving process is terminated.

  A hardware configuration example of the information processing apparatus according to the present embodiment will be described with reference to FIG. The configuration shown in FIG. 13 is configured by a personal computer (PC), for example, and shows a hardware configuration example including a data reading unit 1217 such as a scanner and a data output unit 1218 such as a printer.

  A CPU (Central Processing Unit) 1201 includes various modules described in the above-described embodiments, that is, a first condition detection module 110, a verification A module 115, a second condition detection module 125, a verification B module 135, and a storage processing module. 140 is a control unit that executes processing according to a computer program describing an execution sequence of each module such as an authentication processing module 145, a guide display module 150, a control module 155, and a leaving module 960.

  A ROM (Read Only Memory) 1202 stores programs used by the CPU 1201, calculation parameters, and the like. A RAM (Random Access Memory) 1203 stores a program used in the execution of the CPU 1201, parameters that change as appropriate during the execution, a face image that satisfies the second condition stored in the storage processing module 140, and the like. These are connected to each other by a host bus 1204 configured by a CPU bus or the like.

  The host bus 1204 is connected to an external bus 1206 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 1205.

  A keyboard 1208 and a pointing device 1209 such as a mouse are devices operated by an operator. The display 1210 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text or image information. Further, a touch screen or the like having both functions of the pointing device 1209 and the display 1210 may be used. In that case, with respect to the realization of the keyboard function, even if the keyboard 1208 is not physically connected, a keyboard (so-called software keyboard, screen keyboard, etc.) is drawn on the screen (touch screen) by software, You may make it implement | achieve the function of a keyboard.

  An HDD (Hard Disk Drive) 1211 includes a hard disk (may be a flash memory or the like), drives the hard disk, and records or reproduces a program executed by the CPU 1201 and information. The hard disk realizes a function as the face image storage module 120 or the like. Further, various other data, various computer programs, and the like are stored.

  The drive 1212 reads data or a program recorded in a removable recording medium 1213 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out to the interface 1207 and the external bus 1206. , The bridge 1205, and the RAM 1203 connected via the host bus 1204. Note that the removable recording medium 1213 can also be used as a data recording area.

  The connection port 1214 is a port for connecting the external connection device 1215 and has a connection unit such as USB, IEEE1394. As the external connection device 1215, for example, there is an imaging module 105 that is a digital camera or the like. The connection port 1214 is connected to the CPU 1201 and the like via the interface 1207, the external bus 1206, the bridge 1205, the host bus 1204, and the like. The communication unit 1216 is connected to a communication line and executes data communication processing with the outside. The data reading unit 1217 is a scanner, for example, and executes document reading processing. The data output unit 1218 is a printer, for example, and executes document data output processing.

  Note that the hardware configuration of the information processing apparatus illustrated in FIG. 13 shows one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 13 and executes the modules described in the present embodiment. Any configuration is possible. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line Alternatively, a plurality of systems shown in FIG. 13 may be connected to each other via a communication line so as to cooperate with each other. In addition, in addition to personal computers, entrance / exit management devices, health management devices, household electrical devices, portable information communication devices (including mobile phones, smartphones, mobile devices, wearable computers, etc.), robots, copiers, fax machines, It may be incorporated in a scanner, printer, multifunction device or the like.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standard “DVD + R, DVD + RW, etc.”, compact disc (CD), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), Blu-ray disc ( Blu-ray (registered trademark) Disc), magneto-optical disk (MO), flexible disk (FD), magnetic tape, hard disk, read-only memory (ROM), electrically erasable and rewritable read-only memory (EEPROM (registered trademark)) )), Flash memory, Random access memory (RAM) SD (Secure Digital) memory card and the like.
Then, the whole or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, or a wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part or all of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

DESCRIPTION OF SYMBOLS 100 ... Authentication apparatus 105 ... Imaging module 110 ... 1st condition detection module 115 ... Collation A module 120 ... Face image storage module 125 ... 2nd condition detection module 135 ... Collation B module 140 ... Storage processing module 145 ... Authentication processing module 150 ... Guide display module 155 ... Control module 200 ... Information processing device 210 ... Function module 250 ... Image processing device 260 ... Image processing module 900 ... Authentication device 960 ... Leave module 965 ... Leave detection module 970 ... Leave process module

Claims (9)

A first collation means for collating the face image with a previously stored face image when the captured face image satisfies the first condition;
Second collation means for collating a face image satisfying a second condition, which is stricter than the first condition, with the face image stored in advance when collation by the first collation means fails. When,
A third verification unit that performs verification by a verification method different from the verification of a face image when verification by the second verification unit fails;
An information processing apparatus comprising: a first storage unit that stores a face image that satisfies the second condition when the third verification unit succeeds in verification. The first condition or the second condition is configured by at least one of elements of a face size, a face direction, a degree of stillness, blinking presence / absence, and a facial expression acceptance range in a face image. Yes,
The information processing apparatus according to claim 1. The first condition and the second condition are different in at least one of the elements, and different elements of the second condition are similar to the pre-stored face image capturing conditions.
The information processing apparatus according to claim 2. Second storage means for storing a face image after imaging satisfying the second condition;
The first storage means stores the face image stored in the second storage means;
The information processing apparatus according to claim 1. A leaving detection means for detecting that the user of the face image has left,
The information processing apparatus according to claim 4, further comprising: a deletion unit that deletes the face image stored in the second storage unit when the departure detection unit detects a departure. The information processing apparatus according to claim 1, further comprising a display unit configured to display an imaging guide under the second condition when the verification by the first verification unit fails. It further has a leaving detection means for detecting that the user of the face image has left,
The display means hides the displayed guidance when the withdrawal is detected by the withdrawal detection means.
The information processing apparatus according to claim 6. The exit detection means detects the exit by any one of the following, or a combination thereof:
(1) The face is not included in the captured image,
(2) that the user cannot be detected by a sensor;
(3) a person is not included in an image captured by an image capturing apparatus different from the image capturing apparatus capturing the face image;
The information processing apparatus according to claim 5 or 7. Computer
A first collation means for collating the face image with a previously stored face image when the captured face image satisfies the first condition;
Second collation means for collating a face image satisfying a second condition, which is stricter than the first condition, with the face image stored in advance when collation by the first collation means fails. When,
A third verification unit that performs verification by a verification method different from the verification of a face image when verification by the second verification unit fails;
An information processing program for causing a face image satisfying the second condition to function as a first storage unit when the collation by the third collating unit is successful.

Images