JP2019168929A - Information processor, authentication method in information processor, and program - Google Patents

Abstract

To provide an information processor including a face authentication system which solves the problem that when an operator is photographed while wearing sunglasses, a mask and a hut, authentication may be denied irrespective of a registered authorized person registered, for this reason, there is a possibility that efficiency of an operation work of MFP aggregates.SOLUTION: The photographed face image is divided into three portions of an upper portion, a middle portion and a lower portion, and authentication is performed for each of the portions. When authentication succeeds in all portions, the face authentication is regarded to pass. In addition, even when authentication in one portion fails in the three portions, when authentication succeeds in other two portions succeeds, the face authentication is not quickly regarded to fail. In this case, the portion in which the authentication fails is clearly displayed, and an operator is promoted to photograph its face image again.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an information processing apparatus including a face authentication system.

Conventionally, there is a multifunction peripheral (MFP) having a user authentication function based on face authentication. There is a face authentication system that determines whether or not to authenticate an operator by photographing the face of the operator standing in front of the MFP and comparing the face image with a pre-registered face image. It is used.
When collating face images, a captured face image target extraction unit that extracts an image of a target region of a captured face image, and a registered face that extracts a region corresponding to the target extraction unit from a pre-registered face image The image target extraction unit is checked by the matching unit.

  Further, Patent Document 1 divides a face region, and among the divided regions, eyes, nose and mouth are normal in an upper region including eyes and a lower region including nose and mouth, or It is determined whether the face is partially hidden by sunglasses or a mask. If it is determined that the face is partially hidden, a warning message is transmitted. If this warning continues multiple times, the authentication is rejected.

JP 2004-158013 A

In the face authentication system, a part of the photographed face of the person to be recognized may be hidden, which may cause a problem that authentication cannot be performed.
One possible reason for this is when another person hides a part of the face in an attempt to impersonate the person to be authenticated. In addition, while the registered person tries to authenticate, he / she shoots with sunglasses, a mask, a hat, etc., or the spectacles reflect, making it impossible to recognize part of the face. If you have, there are.

For example, in Japanese Patent Laid-Open No. 2004-260260, authentication is performed because a person who is trying to hide a part of a face by impersonation and a person who is trying to authenticate himself / herself are in a poor shooting state. There is no distinction from the case where it is not possible. In both cases, it is determined that a part of the face is hidden, and the authentication is rejected by performing a plurality of warnings.
For this reason, when the registered person tries to authenticate, the efficiency of the operation operation of the MFP may be deteriorated.

  The present invention is an information processing apparatus that is permitted to be used based on a determination by a face authentication unit, wherein the face authentication unit includes a captured face image divided into N (N is a natural number of 3 or more) parts, The registered face image divided into the parts is compared for each divided part, and if the authentication is successful in all the N parts, a first determination is made that the photographed face image matches the registered face image, and M (M is a natural number less than or equal to 2 and less than N) If authentication is successful at less than a part, a second determination is made that the captured face image does not match the registered face image, and M or more and N-1 or less When the authentication is successful at the part, a third determination is made that a part of the part has failed in the authentication.

  According to the present invention, even if authentication fails in some parts of the divided face images, prompting the operator to take a face image again without immediately turning the face authentication to NG. Can do. For this reason, when the operator is the registered person, the efficiency of the authentication procedure can be improved.

This is the appearance of the MFP. 2 is a block diagram illustrating a hardware configuration of an MFP. FIG. It is a block diagram which shows the function structure of a controller. It is a block diagram which shows the hardware constitutions of an operation part. 3 is a flowchart showing an overall operation of the MFP. It is a flowchart which shows the detail of a face authentication process. This is an example in which face authentication fails. It is a display example on the operation unit after face authentication.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is an overall view of an image forming apparatus which is an example of an information processing apparatus of the present invention. FIG. 1A is a front view of the image forming apparatus. FIG. 1B is a view of the image forming apparatus as viewed from above.

In this embodiment, the image forming apparatus is an MFP. The MFP 100 includes an operation unit 206, a reading unit 204, a paper feed cassette 104, an ADF 109, and the like. The MFP 100 can execute functions such as copying, printing, faxing, and scanning.
The MFP 100 can operate by switching between a normal power mode and a power saving mode in which power consumption is lower than that in the normal power mode.
Further, the MFP 100 is provided with a camera 202 for photographing the operator in order to perform face authentication.

FIG. 2 is a block diagram showing a hardware configuration of MFP 100.
The MFP 100 includes a camera unit 202, a controller 203, a reading unit 204, a printing unit 205, an operation unit 206, a power supply control unit 207, an HDD 209, and the like.
The camera unit 202 is provided for photographing the authenticator. Then, the face of the authenticator is photographed and the face image data is sent to the controller 203.

A controller 203 controls the entire MFP 100.
A reading unit 204 reads a paper document input into the image forming apparatus.
The printing unit 205 performs printing on output paper.
The operation unit 206 is an interface for an operator to perform an operation, and includes a display unit and the like.
The power control unit 207 controls power supply to the camera 202, the controller 203, the reading unit 204, the printing unit 205, and the operation unit 206 based on an input from a sleep return sensor (not shown).
An HDD (Hard Disk Drive) 209 is a storage device that stores various data.

FIG. 3 is a block diagram illustrating a functional configuration of the controller 203.
The controller 203 includes an MFP control unit 301, a read image processing unit 302, a read control signal I / F unit 303, a print image processing unit 304, a print control signal I / F unit 305, a UI transmission / reception data I / F unit 308, and a power control. An I / F unit 309 and the like.

An MFP control unit 301 controls each unit in the controller 203. In addition, the MFP control unit 301 includes a CPU, a ROM, a RAM, and the like, all not shown.
The read image processing unit 302 processes the image read by the reading unit 204.
The reading control signal I / F unit 303 is an interface unit that communicates with the reading unit 204 using the reading control signal.

A print image processing unit 304 performs image processing of image data to be printed by the printing unit 205.
A print control signal I / F unit 305 is an interface that communicates with the print unit 205 using a print control signal.
The UI transmission / reception data I / F unit 308 is an interface that transmits display data to the operation unit 206.
The power control I / F unit 309 is an interface that receives a control signal from the power control unit 207 and performs power control in the controller 203.

  Various settings and controls in this embodiment are executed by the CPU in the MFP control unit 301 according to a control program stored in a ROM (not shown) in the MFP control unit 301 or expanded in the RAM. .

FIG. 4 is a block diagram illustrating a hardware configuration of the operation unit 206.
The operation unit 206 includes a controller I / F unit 401, a UI control unit 402, a display unit 403, a key input unit 404, a speaker unit 405, a light emitting unit 406, a memory 407, a face authentication unit 408, a power control I / F unit 409, Etc.

The controller I / F unit 401 is an interface that communicates with the controller 203.
The UI control unit 402 is a user interface control unit that controls each unit in the operation unit 206 such as the display unit 403. In addition, the UI control unit 402 performs control to divide a face image into upper, middle, and lower parts for face authentication and extract feature amounts of each part.
The display unit 403 is a display unit including a panel for confirming display of characters and images, various settings, and the like.

The key input unit 404 is an input unit including buttons and numeric keys.
The speaker unit 405 includes a speaker that outputs a buzzer sound.
The light emitting unit 406 is configured by an LED or the like.
A memory 407 stores the face image data captured by the camera unit 202 sent from the camera unit 202 and the face image data registered in advance in the HDD 209 sent from the controller 203. Part.
The face authentication unit 408 is an authentication unit that performs authentication by collating face photographing data and registered face data.
A power control I / F unit 409 is an interface that receives a control signal from the power control unit 207 and performs power control in the operation unit 206.

Next, the face authentication process will be described.
FIG. 5 is a flowchart showing the overall operation of MFP 100.
The processing based on this flowchart is executed by the CPU in the MFP control unit 301 in accordance with a control program stored in a ROM (not shown) in the MFP control unit 301 or expanded in the RAM.

First, when the MFP 100 is turned on, the MFP 100 waits for an operation from the operator in S501.
In step S502, the operator reads the ID card in order to authenticate that the operator is registered. Thereby, the ID information recorded in the ID card is input as the personal authentication information of the operator. Further, in order to input personal authentication information, a password may be input.
When the personal authentication information of the operator is input, the camera 202 captures the face of the operator in S503. Then, the face authentication unit 408 collates the captured face image (hereinafter also referred to as “captured face image”) with a previously registered face image (hereinafter also referred to as “registered face image”), Perform face authentication processing. Details of the face authentication processing will be described later with reference to FIG.

If the face authentication in S503 is successful, the face authentication is OK in S504.
As a result, the authenticated operator is permitted to use the MFP.
Thereafter, in S505, the operator executes the job.
If it is determined in S506 that the job has been completed, this flowchart ends.

  On the other hand, if the face authentication fails in S503, the face authentication is determined to be NG in S504. Then, the flowchart ends without allowing the operator to use the MFP.

FIG. 6 is a flowchart showing details of the face authentication process (S503) in the flowchart of FIG.
In S502 of FIG. 5, when the operator stands in front of the MFP and reads the ID card or inputs a password, the camera 202 captures the face of the operator in S601.
The face image data captured by the camera is transferred to the memory 407 and stored.
In step S <b> 602, the UI control unit 402 divides the photographed face image stored in the memory 407 into three parts, that is, an upper part, a middle part, and a lower part. Then, the captured face image data for each part is stored in the memory 407, and the feature amount of each part is extracted. Note that the technique for extracting the feature amount is a known technique, and thus the description thereof is omitted here.

In step S <b> 603, the MFP control unit 301 refers to the personal authentication information input in step S <b> 502 and searches for and extracts registered face image data registered in the HDD 209.
In step S604, the extracted registered face image is divided into three parts, an upper part, a middle part, and a lower part. Then, the registered face image data for each part is stored in the HDD 209, and the feature amount of each part is extracted. The divided registered face data is stored in the memory 407 in the operation unit 206 via the UI transmission / reception data I / F unit 308.
Thus, the captured face image data divided into the upper, middle, and lower three parts and the registered face image data divided into the upper, middle, and lower three parts are stored in the memory 407 for each part. The

In step S <b> 605, the face authentication unit 408 performs comparison for each of the upper, middle, and lower parts divided into three using the captured face image data and the registered face image data read from the memory 407.
At this time, the face authentication unit 408 obtains the upper feature quantity of the captured face image data divided into three parts and the upper feature quantity of the registered face image data divided into three parts. In comparison, a recognition score is calculated.
Similarly, a recognition score is calculated by comparing the feature quantity of the middle part of the captured face image data with the feature quantity of the middle part of the registered face image data.
Also, the recognition score is calculated by comparing the lower feature amount of the photographed face image data with the lower feature amount of the registered face image.

In addition, since the pattern matching method of the image data in the case of calculating a recognition score is a well-known technique, description is abbreviate | omitted here.
When the recognition score exceeds a certain threshold value, the face authentication unit 408 determines that the authentication is successful. On the other hand, if the predetermined threshold is not exceeded, it is determined that face authentication has failed.
Then, it is determined whether the face authentication has succeeded or failed for each of the upper, middle, and lower parts.

If it is determined in step S606 that all of the upper, middle, and lower parts of the photographed face image have been successfully authenticated, the face authentication unit 408 determines that the photographed face image matches the registered face image.
In step S608, the face authentication is OK and the face authentication ends.
If authentication of at least one of the upper, middle, and lower three parts fails in S606, the process proceeds to S607.

  If it is determined in step S607 that all of the upper, middle, and lower three parts are unsuccessful, or if two parts of the upper, middle, and lower parts are unsuccessful, the face authentication unit In step 408, it is determined that the captured face image does not match the registered face image. That is, the photographed face image is determined to be a face image of a person different from the person having the input personal authentication information. In step S609, the face authentication is NG, and the face authentication ends.

On the other hand, in S607, even if authentication fails in one of the upper, middle, and lower parts, if authentication of the other two parts is determined to be successful, face authentication is immediately performed. Not NG.
This is because, in this case, it is considered that there is a possibility that some parts have not been photographed accurately due to some cause at the time of photographing, regardless of whether the operator is a registered person. .

Therefore, in step S610, the UI control unit 402 displays a notification that a part of the captured face image has failed to be authenticated on the display unit 403.
Specifically, the photographed face image is displayed on the display unit 403 so that the part where the authentication has failed is clearly indicated.
As a result, the operator can be aware that if the person himself / herself has been registered in advance, the imaging of the part determined to have failed in authentication has not been performed accurately. At this time, the recognition score calculated for each part may be displayed on the display unit 403.

In S612, if the operator recognizes that he / she is registered in advance, the operator can return to S601 and perform face recognition again.
Thereby, even if the operator registered in advance fails in the authentication in one part, it is possible to perform face recognition again by re-imaging after recognizing the part in which the authentication has failed. For this reason, at the time of the face recognition again, there is a high possibility that the authentication is successful in all the parts and the face authentication is OK.

In the flowchart of FIG. 6, when performing face authentication, the registered face image and the photographed face image are each divided into upper, middle, and lower three parts. However, the number of divisions is not limited to 3, and may be divided into N (natural numbers greater than or equal to 3) sites.
In this case, if the authentication is successful for all the N parts, it is determined that the photographed face image belongs to a person registered in advance, and the use of the MFP is permitted.

Further, even if the authentication is not successful for all of the N sites, if the authentication is successful for the M (2 or more and natural number smaller than N) site (that is, the authentication is performed for the sites of M or more and N-1 or less) If it succeeds), the face authentication is not immediately judged as NG. Then, the photographed face image is displayed on the display unit so that the part where the authentication has failed is clearly indicated. Then, the operator is prompted to take a face image again.
For example, each of the registered face image and the photographed face image is divided into four parts, and when only three parts are successfully authenticated, the photographed face image is displayed and the face recognition is performed again for the operator. Prompt.
If the authentication is successful for less than the M part, it is determined that the photographed face image is not for a person registered in advance, and the use of the MFP is rejected.

With reference to FIG. 7, an example will be described in which authentication is not OK due to failure of authentication of one part of a face image, regardless of whether an operator who performed face authentication is registered.
FIG. 7A shows an example of registered face images. As the registered face image, a face image in a state where there is no hidden part as a whole is registered.

On the other hand, FIGS. 7B to 7D are examples of photographed face images.
In the photographed face image of FIG. 7B, the hair is hidden by a hat. This is an example where authentication fails at the top of the face.
In the photographed face image of FIG. 7C, eyes are hidden by sunglasses. This is an example in which authentication fails in the middle of the face.
In the photographed face image of FIG. 7D, the mouth is hidden by a mask. This is an example where authentication fails at the bottom of the face.

As shown in FIGS. 7B to 7D, even if the authentication fails at one part, if the authentication is successful at the other two parts, the possibility that the operator is registered is high.
For this reason, in the flowchart of FIG. 7 described above, in S607, the face authentication is not changed to NG, and the display of S610 is performed. Then, the process returns to S601, and the operator is given an opportunity to take a face image again. I will do it.

FIG. 8 shows various examples when the result of face authentication is displayed on the display unit 403.
FIG. 8A shows a display example when the face authentication is successful.
In this case, the fact that the face authentication has been successful is displayed, and that the screen is shifted to the operation menu in order to allow the operator to execute the job.
FIG. 8B is a display example when face authentication fails.
In this case, a message indicating that the face authentication has failed and a return to the login menu screen (FIG. 8F) is displayed.

FIG. 8C shows a display example when face authentication fails for only the upper part of the face.
In this case, a message indicating that some face authentication has failed is displayed. In addition, the captured face image is displayed together with the registered face image so that the part (upper part) where the authentication has failed is clearly indicated.
FIG. 8D shows a display example when face authentication has failed for only the middle part of the face.
In this case as well, a message indicating that some face authentication has failed is displayed. In addition, the captured face image is displayed together with the registered face image so that the part (central part) where the authentication has failed is clearly indicated.
FIG. 8E shows a display example when face authentication fails for only the lower part of the face.
In this case as well, a message indicating that some face authentication has failed is displayed. In addition, the captured face image is displayed together with the registered face image so that the part (lower part) where the authentication has failed is clearly indicated.
FIG. 8F is an example of a login menu screen for entering personal authentication information that is returned when authentication fails.

As described above, even if authentication fails in one part, such as when the operator has taken a picture while wearing sunglasses, a mask, a hat, etc. regardless of being registered, the other two If authentication is successful at one part, face authentication is not immediately judged as NG. Then, by explicitly displaying the part where the authentication has failed, it is possible to prompt the operator to take a face image again. Accordingly, it is possible to make the operation in the MFP authentication procedure more efficient for the registered operator.
On the other hand, when another person who is not registered attempts to impersonate a registrant, the use of the MFP can be reliably refused.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
Further, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device.
The present invention is not limited to the above-described embodiments, and various modifications can be made based on the spirit of the present invention, and they are not excluded from the scope of the present invention. That is, all the configurations in which the above-described embodiments and modifications thereof are combined are also included in the present invention.

100 MFP
203 Controller 202 Camera Unit 206 Operation Unit 403 Display Unit

Claims (18)

An information processing apparatus that is permitted to be used based on the determination by the face authentication means,
The face authentication means
The captured face image divided into N (N is a natural number of 3 or more) parts and the registered face image divided into N parts are compared for each divided part,
If authentication is successful for all N sites, a first determination is made that the captured face image matches the registered face image;
When authentication is successful in less than M (M is a natural number less than or equal to 2 and less than N) part, a second determination is made that the captured face image does not match the registered face image;
An information processing apparatus characterized by performing a third determination that a part of a part has failed authentication when authentication succeeds in parts not less than M and not more than N-1. The information processing apparatus according to claim 1, wherein when the face authentication unit makes the third determination, a notification that a part of the authentication has failed is displayed on the display unit. The information processing apparatus according to claim 2, wherein the notification includes information on the captured face image. The information processing apparatus according to claim 3, wherein a part that has failed in authentication is specified in the information related to the captured face image included in the notification. The information processing apparatus according to claim 3, wherein a recognition score is included in the notification. 6. The comparison performed by the face authentication means is performed by extracting a feature amount for each divided part of the photographed face image and the registered face image. The information processing apparatus according to item 1. The information processing apparatus according to any one of claims 1 to 6, wherein the N is 3 and the M is 2. The information processing apparatus according to claim 7, wherein each of the photographed face image and the registered face image is divided into an upper part, a middle part, and a lower part. The information processing apparatus according to any one of claims 1 to 8, wherein the information processing apparatus is an image processing apparatus. The information processing apparatus according to claim 1, wherein the face authentication unit is provided in an information processing apparatus. It further has a photographing means,
The information processing apparatus according to claim 1, wherein the photographed face image is photographed by the photographing unit. A storage means;
The information processing apparatus according to claim 1, wherein the registered face image is stored in the storage unit. The information processing apparatus according to claim 12, wherein the captured face image is stored in the storage unit. It further has an input means,
The information processing apparatus according to claim 12, wherein the personal authentication information of the operator can be input using the input unit. The information processing apparatus according to claim 14, wherein the personal authentication information is a password or ID information recorded on an ID card. The one registered face image is extracted from the registered face images stored in the storage unit on the basis of the personal authentication information. Information processing device. An authentication method in an information processing apparatus that is permitted to be used based on determination by a face authentication process,
The face authentication process includes
A step of comparing the captured face image divided into N (N is a natural number of 3 or more) parts and the registered face image divided into N parts for each divided part;
If authentication is successful for all N sites, a first determination is made that the captured face image matches the registered face image;
When authentication is successful in less than M (M is a natural number less than or equal to 2 and less than N) part, a second determination is made that the captured face image does not match the registered face image;
A step of performing a third determination that a part of the parts has failed in the authentication when the authentication has succeeded at a part of M or more and N-1 or less;
An authentication method in an information processing apparatus characterized by comprising:   A program for causing a face authentication method in the information processing apparatus according to claim 17 to be executed by a computer.