JP2016110244A - Face recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a face recognition device that alleviates a load of the device.SOLUTION: In a registration mode, a face recognition device executes processing including a first step of: acquiring a face photograph image of a person scheduled to be registered; extracting a first amount of characteristic composed of a variable of a relatively few number on the basis of the face photograph image; calculating a degree of similarity between the first amount of characteristic of the person scheduled to be registered and a first amount of characteristic of a registered person; and when the registered person approximate to the first amount of characteristic does not exist, registering the first amount of characteristic of the person scheduled to be registered, and a second step of: when the registered person approximate to the first amount of characteristic exists, extracting a second amount of characteristic having a new variable added to the variable consisting of the first amount of characteristic on the basis of the face photograph image of the person scheduled to be registered; calculating a degree of similarity between the second amount of characteristic of the person scheduled to be registered and a second amount of characteristic of the registered person; and when the registered person approximate to the second amount of characteristic does not exist in the registered person approximate to the first amount of characteristic, registering the second amount of characteristic of the person scheduled to be registered.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a face recognition device.

  Japanese Patent Application Laid-Open No. H10-260260 discloses storing shooting conditions at the time of face shooting.

  Patent Document 2 discloses that a plurality of faces under a plurality of different environments are photographed and stored.

  Patent Document 3 discloses extracting information from a plurality of faces taken from a plurality of angles.

JP 2006-192093 A JP-A-2005-115481 Japanese Patent Laid-Open No. 2008-065651

  An object of this invention is to provide the face recognition apparatus which reduced the load of the apparatus.

Claim 1
A shooting unit for shooting a face and acquiring a face shot image on the data;
An extraction unit that extracts a facial feature amount based on a face-captured image acquired by the imaging unit;
A registration unit for registering the feature amount extracted by the extraction unit in association with an identification number for identifying each registrant;
The similarity between the new feature amount newly extracted by the extraction unit and the feature amount of the registered person registered in the registration unit is calculated based on the face photographed image newly acquired by the shooting unit. A calculation unit;
The similarity calculated by the calculation unit is compared with a predetermined threshold value, and among the registered users, an approximate registered user having a feature amount that is closer to the new feature amount than the predetermined threshold value A determination unit for determining whether or not it exists;
In the registration mode in which the feature amount of the prospective registration person is registered in association with the identification code of the prospective registration person, the photographing unit acquires the face photographing image of the prospective registration person, and based on the photographing image, the extraction unit , Extracting a first feature amount composed of a relatively small number of variables, and causing the calculation unit to extract the first feature amount of the prospective registered person and the registered person registered in the registration unit. A degree of similarity between the first feature amount and the determination unit to determine whether or not there is a first feature amount approximate registered user who is the approximate registered user regarding the first feature amount; When the first feature quantity approximate registered person does not exist, the registration unit includes a first step of registering the first feature quantity of the prospective registration person in association with the identification code of the prospective registration person. And when the first feature quantity approximate registered person exists, the face of the prospective registered person is photographed. Based on the image, the extraction unit extracts the second feature amount of the prospective registrant by adding a new variable to the variable constituting the first feature amount, and causes the calculation unit to register the registration plan. A similarity between the second feature amount of the person registered and the second feature amount of the registered person registered in the registration unit is calculated, and the determination unit causes the first feature amount approximate registration When the second feature quantity approximate registrant who is the approximate registered person relating to the second feature quantity in the person is determined to exist, the registration is performed when the second feature quantity approximate registrant does not exist. And a control unit that executes a process including a second step of registering the second feature amount of the prospective registration person in association with the identification code of the prospective registration person. Device.

Claim 2
In the registration mode, the control unit approximates a maximum variable feature amount including a maximum number of variables extracted by the extraction unit based on the first face photographed image of a prospective registration person among registered users. When there is a first image approximate registered person who is the approximate registrant having the maximum variable feature amount, the photographing unit acquires the second face photographed image of the prospective registration person, and the extracting unit A second image feature amount based on the second face-captured image is extracted, and the determination unit causes the second image to be the approximate registered user regarding the second image feature amount in the first image approximate registered user. Whether or not there is an approximate registered person is determined, and when the second image approximate registered person does not exist, the registration unit is extracted based on the first face photograph image of the prospective registered person Extracted based on the maximum variable feature amount and the second face image Both 2 image feature quantity, a face authentication apparatus according to claim 1, wherein the executing further comprises processing a third step of registering in association with the identification code of the intended registrant.

Claim 3
When the second feature quantity unregistered registered person in which only the first feature quantity of the second feature quantity is recorded exists in the first feature quantity approximate registered person, the control unit The second feature amount is extracted by the extraction unit on the basis of a face photographed image acquired at the time of the next face photographing relating to the second feature amount unregistered registered person, and the second calculation unit is caused to extract the second feature amount. A similarity between the second feature amount of a registered person who has not registered a feature amount and the second feature amount of a registered person registered in the registration unit is calculated, and the determination unit causes the second feature amount to be calculated. When the second feature quantity approximate registered user is present, the registration unit, when there is no second registered feature quantity registered, The second feature quantity of the second feature quantity unregistered registered person is registered in association with the identification code of the second feature quantity unregistered registered person. A face authentication apparatus according to claim 1 or 2, characterized in that to execute the process.

Claim 4
When the second image feature amount is not registered in the first image approximate registered person, the control unit relates to the second image unregistered registered person. At the time of the next face photographing, the photographing unit is made to acquire a second face photographing image of the second image unregistered registered person, and based on the second face photographing image, the extracting unit causes the second face photographing image to be acquired. An image feature amount is extracted, and the calculation unit calculates the second image feature amount of the second image unregistered registered user and the second image feature amount of the registered user registered in the registration unit. The second image approximate registration is performed by causing the determination unit to determine whether or not there is a second image approximate registered user who is the approximate registered user regarding the second image feature amount. When the user does not exist, the second image feature amount of the registrant who has not registered the second image is stored in the registration unit. A face authentication apparatus according to any one of claims 1 to 3, characterized in that to execute a process of registering in association with the identification codes of registered registered person.

  According to the face recognition apparatus of the first aspect, the load at the time of registration is reduced as compared with the system that always registers the feature quantity including the maximum number of variables.

  According to the face recognition device 2 of the second aspect, recognition processing with higher accuracy can be performed as compared with a system without the third step.

  According to the face recognition device 2 of claims 3 and 4, it is possible to avoid the recognition accuracy from subsequently decreasing.

1 is a diagram illustrating an appearance of a multi-function peripheral including a face recognition device according to an embodiment of the present invention. FIG. 2 is a functional block diagram of the multifunction machine shown in FIG. 1. FIG. 3 is a block diagram showing an internal configuration of a face recognition unit shown by one block in FIG. 2. It is the figure which showed the registration content in a registration part. It is the flowchart which showed the operation | movement procedure in a registration process. It is the flowchart which showed the operation | movement procedure in a registration process. It is the flowchart which showed the process sequence in authentication mode. It is the flowchart which showed the process sequence in authentication mode. It is the flowchart which showed the process sequence in authentication mode. It is the flowchart which showed the process sequence in authentication mode. It is the flowchart which showed the process sequence in authentication mode. It is the flowchart which showed the process sequence in authentication mode.

  Embodiments of the present invention will be described below.

  FIG. 1 is a diagram showing the external appearance of a multi-function peripheral incorporating a face recognition device as one embodiment of the present invention.

  FIG. 2 is a functional block diagram of the multifunction peripheral shown in FIG.

  As shown in FIG. 2, the multifunction machine 10 includes a scanner unit 21 that reads an image on a document and generates image data, a print unit 22 that prints an image on a sheet based on the image data, and a telephone line 51. A facsimile transmission / reception unit 23 that performs facsimile transmission / reception via an I / O interface, and an I / O interface unit 24 connected to one or more personal computers (hereinafter abbreviated as “PC”) via a communication line 52. And a processing unit 20 including:

  Further, as shown in FIG. 1, the multifunction machine 10 includes a UI (User Interface) 10A. The UI 10 </ b> A includes a display unit 11 that displays setting contents and an internal state of the multifunction machine 10 and an operation unit 12 that gives various instructions to the multifunction machine 1. Further, the UI 10A further includes a camera 13 and an IC card reader 14. The camera 13 is responsible for photographing the face of the user of the multifunction machine 10. Further, the IC card reader 14 has a function of reading information in the IC card when an IC card (not shown) owned by a person who has the authority to use the multifunction machine 10 is deceived. The IC card stores ID information of the IC card owner and information certifying that the IC card owner has the right to use the multifunction machine 10.

  Further, the multifunction device 10 is provided with a human sensor 15. The human sensor 15 is a sensor that detects that a person has approached the multifunction device 10 by detecting infrared rays. When the multifunction device 10 is not in use, it is in an energy saving mode, and when the human sensor 15 detects that a person is approaching, supply of electric power or the like to make the multifunction device 10 usable. Preparation for use begins.

  As shown in FIG. 2, the multifunction machine 10 further includes an operation control unit 30 and a face recognition unit 40 in addition to the processing unit 20 described above.

  As described above, the processing unit 20 includes the scanner unit 21, the print unit 22, the facsimile transmission / reception unit 23, and the I / O interface 24.

  The multifunction machine 10 has a copy mode. In this copy mode, the scanner unit 21 reads an image on an original to generate image data, and the print unit 22 prints out a copy image based on the image data on paper.

  The multifunction machine 10 has a facsimile transmission mode and a facsimile reception mode. In the facsimile transmission mode, an image on an original is read by the scanner unit 21 to generate image data, and the image data is converted into image data for facsimile transmission by the facsimile transmission / reception unit 23 and transmitted via the telephone line 51. Is done. Further, in the facsimile reception mode, image data for facsimile transmission input via the telephone line 51 is received by the facsimile transmission / reception unit 23 and converted into image data for the inside of the multifunction machine 10, and the printing unit 22. In FIG. 5, an image based on the image data is printed out.

  Further, the multifunction machine 10 has a print mode. In the print mode, image data transmitted from a PC or the like connected to the communication circuit 52 is received by the I / O interface 24 and converted into image data for the inside of the multifunction machine 10, and the image is displayed in the print unit 22. An image based on the data is printed out.

  Further, the multifunction machine 10 has a scanner mode. In the scanner mode, the scanner unit 21 reads an image on a document and generates image data. The I / O interface unit converts the image data into image data for communication with a PC or the like. Sent to a PC or the like.

  The operation control unit 30 is responsible for controlling the operation of the processing unit 20. The operation control unit 30 exchanges necessary information with the face recognition unit 40.

  A display unit 11, an operation unit 12, an IC card reader 14, and a human sensor 15 shown in FIG. 1 are connected to the operation control unit 30.

  The human sensor 15 is a sensor that detects that a person has approached the multifunction device 10 as described above. When the motion control unit 30 is notified from the human sensor 15 that the person is approaching, the operation unit 30 starts up the processing unit 20 and the face recognition unit that have been in the sleep state until then, and the person has moved away. When notified, the processing unit 20 and the face recognition unit 40 are shifted to the sleep state after a predetermined time. However, at this stage, the operation control unit 30 puts the processing unit 20 into an operating state, but keeps it in a locked state that cannot be used yet.

  The IC card reader 14 reads the information recorded therein from the IC card placed there, and notifies the operation control unit 30 of the presence / absence of the right of use of the multifunction device 10 and the user's ID. The face recognition unit 40 takes a picture of the face of a person standing in front of the multifunction machine 10 with the camera 13 shown in FIG. 1 and recognizes whether or not the person has the right to use the multifunction machine 10. And the recognition result is transmitted to the operation control unit 30.

  When the operation control unit 30 is notified from the IC card reader 14 or the face recognition unit 40 that it is a person who has the right to use the multifunction machine 10, the operation control unit 30 unlocks the processing unit 20 and makes it usable.

  The operation unit 12 is a tool for operating and inputting various instructions to the multifunction machine 10, and the operation content is displayed on the display unit 11 via the operation control unit 30. Further, the operation control unit 30 controls the processing unit 20 according to the operation content, or transmits the operation content to the face recognition unit 40.

  The face recognition unit 40 in the multifunction machine 10 of the present embodiment is an example of a face recognition device, and the present embodiment is characterized by the processing in the face recognition unit 40. Therefore, the details will be described below focusing on the face recognition unit 40.

  FIG. 3 is a block diagram showing an internal configuration of the face recognition unit shown by one block in FIG.

  As shown in FIG. 3, the face recognition unit 40 includes a photographing unit 41, an extraction unit 42, a registration unit 43, a calculation unit 44, a determination unit 45, and a registration / authentication control unit 46.

  The imaging unit 41 includes the camera 13 shown in FIGS. 1 and 2, and is an element that captures a face of a person standing in front of the multifunction machine 10 with the camera 13 and acquires a captured image on the data. .

  The extraction unit 42 is an element that extracts a facial feature amount based on the captured image acquired by the imaging unit 41.

  The registration unit 43 is an element that registers the feature amount extracted by the extraction unit 42 in association with identification numbers for identifying registrants.

  Further, the calculation unit 44 calculates a value between the new feature amount newly extracted by the extraction unit 42 based on the photographed image newly acquired by the photographing unit 41 and the feature amount of the registered person registered in the registration unit 43. Is an element for calculating the similarity.

  Furthermore, the determination unit 45 compares the similarity calculated by the calculation unit 43 with a predetermined threshold value, and finds a feature amount that is closer to the new feature amount than the predetermined threshold value among registered users. This is an element for determining whether or not there is an approximate registered user.

  The registration / authentication control unit 46 is an element that controls all the operations of the photographing unit 41 to the determination unit 44.

  Here, the control operation of the registration / authentication control unit 46 will be generally described, and then the control operation will be specifically described.

  The registration / authentication control unit 46 performs a control operation as a registration mode and a control operation as an authentication mode. However, additional registration processing may be performed even in the authentication mode. In the registration mode, the registration / authentication control unit 46 controls the registration unit 43 to register the feature amount of the prospective registration person in association with the identification code of the prospective registration person. That is, the registration / authentication control unit 46 performs the control from the first step to the third step described below in the registration mode. However, further different steps may be inserted between the second step and the third step.

  As the first step in the registration mode, the registration / authentication control unit 46 causes the photographing unit 41 to acquire a face photographed image of the person who is scheduled to be registered, and based on the face photographed image, the extraction unit 42 relatively compares A first feature amount consisting of a small number of variables is extracted, and the calculation unit 43 has the first feature amount of the prospective registered person and the first feature amount of the registered person already registered in the registration unit 43. And the determination unit determines whether or not there is a first feature quantity approximate registered person who is an approximate registered person related to the first feature quantity, and the first feature quantity approximate registered person When it does not exist, control is performed to cause the registration unit 43 to register the first feature amount of the prospective registration person in association with the identification code of the prospective registration person.

  In addition, as a second step in the registration mode, the registration / authentication control unit 46, when there is a registered person who is the first feature amount approximate registered, the extraction unit 42 based on the face photographed image of the prospective registration person. Then, the second feature amount obtained by adding a new variable to the variable constituting the first feature amount of the prospective registration person is extracted, and the calculation unit 43 causes the second feature quantity of the prospective registration person and the registration part to be extracted. 43, the degree of similarity between the second registered feature quantity and the second feature quantity of the registered person registered in 43 is calculated, and the determination unit 44 causes the approximate registered person related to the second feature quantity in the first feature quantity approximate registered person. When there is no second feature quantity approximate registrant, the registration unit 43 sets the second feature quantity of the prospective registration person to Control is performed in association with the identification code of the prospective registration person.

  The registration / authentication control unit 46 adds a new variable to the variable constituting the second feature amount before the third step described later, following the second step. For the second feature amount, the same step as the second step may be repeated. In this way, after executing the same step as the second step described above using the maximum variable feature amount consisting of the maximum number of variables as a new second feature amount, the third step described below is executed. . However, when the second feature amount executed in the second step is the maximum variable feature amount, the third step is executed following the second step.

  As a third step in the registration mode, the registration / authentication control unit 46 includes the maximum number of variables extracted by the extraction unit 42 based on the first face-captured image of the registered person among registered persons. When there is a first image approximate registered person who is an approximate registrant having a maximum variable feature quantity that approximates the maximum variable feature quantity, the imaging unit 41 is made to acquire and extract the second face photographed image of the prospective registration person The second image feature amount based on the second face-captured image is extracted by the unit 42, and the determination unit 44 causes the second image that is an approximate registered person regarding the second image feature amount in the first image approximate registered person. Whether or not there is an approximate registered user is determined, and when there is no second image approximate registered person, the registration unit 43 causes the maximum variable feature extracted based on the first face photographed image of the prospective registered person The second extracted based on the amount and the second face image Both the image feature amount, performs control to register in association with the identification code of the intended registrant.

  Here, the second image feature amount may be a feature amount composed of all variables based on the second face photographed image, or may be a feature amount composed of only some variables. When the second image feature amount is a feature amount consisting of only some of the variables based on the second face photographed image, as in the relationship between the first feature amount and the second feature amount, Also for the second face photographed image, a variable may be sequentially increased in a plurality of stages to determine whether or not there is a similar person.

  In the multifunction machine 10 shown in FIGS. 1 and 2, a plurality of operations are performed in parallel, for example, when a new registration is made for a prospective registration person, a print request is received from a PC or the like, or a facsimile reception request is received. Sometimes done. For this reason, it is desirable to reduce the load required for the registration process and the load required for the authentication process as much as possible.

  In the face recognition unit 40, as described above, when a face can be identified with a feature amount consisting of a small number of variables, the feature amount consisting of a small number of variables can be extracted to reduce the load.

  However, the following problems may occur if the above first to third steps are maintained. That is, among the registered users, there is a registered user (herein referred to as “first registered user”) in which only the first feature amount is registered. As a result of extracting the feature amount, it is assumed that the first feature amount is approximate to the first registrant. At this time, for the new registrant, the second feature value in which the new variable is added to the variable constituting the first feature value is extracted. Since the second feature quantity is not registered for the first registrant, this new prospective registered person is not similar to the first registrant. For this reason, the existing registered person who approximates the second feature amount does not exist at that time, and the new registered person is registered (here, this new registrant is referred to as “second registration”. Called "registrant"). However, if the first registrant who has only registered the first feature quantity extracts the second feature quantity, the second feature quantity may be similar to the second registrant. The same thing can occur not only between the first feature quantity and the second feature quantity but also between the maximum variable feature quantity and the second image feature quantity. Therefore, in the face recognition unit 40 in the present embodiment, the registration / authentication control unit 46 performs the following control.

  The registration / authentication control unit 46 includes, among the first feature quantity approximate registered users, a second feature quantity unregistered registered person in which only the first feature quantity among the second feature quantities is recorded. When the second feature amount is not registered, the second feature amount is extracted by the extraction unit 42 based on the face photographed image acquired at the time of the next face photographing regarding the unregistered registered person, and the calculation unit 43 causes the first feature amount to be extracted. (2) The similarity between the second feature amount of the unregistered registered user and the second feature amount of the registered user registered in the registration unit 43 is calculated, and the determination unit 44 causes the second feature amount to be calculated. Whether or not there is a second feature quantity approximate registered person who is an approximate registered person regarding the feature quantity is determined, and when there is no second feature quantity approximate registered person, the registration unit 43 causes the second feature quantity to be registered. A system for registering the second feature quantity of an unregistered registered person in association with the identification code of the registered person who has not registered the second feature quantity. It is carried out.

  This control is executed in the authentication mode in a specific example to be described later. However, when a person who should register the second feature amount comes to use the multi-function device 10, the control is separated from the authentication mode. This registration process may be executed.

  Similarly, the registration / authentication control unit 46, when there is a second image unregistered registered person in which the second image feature quantity is not registered among the first image approximate registered users, At the time of the next face photographing relating to the second image unregistered registered person, the photographing unit 41 is caused to acquire the second face photographed image of the second image unregistered registered person, and based on the second face photographed image. The extraction unit 42 extracts the second image feature amount, the calculation unit 43 extracts the second image feature amount of the second image unregistered registered user and the second registered user registered in the registration unit 43. The degree of similarity between the image feature amount is calculated, and the determination unit 44 determines whether or not there is a second image approximate registered user who is an approximate registered user with respect to the second image feature amount. When there is no registered person, the registration unit 43 displays the second image feature amount of the registered person who has not registered the second image. Performs control of registration in association with the identification code of the second image unregistered registered person.

  This control is also executed in the authentication mode in the specific example described later, but may be executed separately from the authentication mode.

  The face recognition unit 40 according to the present embodiment avoids the presence of a plurality of registrants having feature quantities that are similar to each other as described above by performing such control.

  In addition, when another step similar to the second step described above exists between the second step and the third step, the additional registration process for the already registered person is also performed for that step. Done.

  Hereinafter, specific examples of processing in the registration mode and the authentication mode will be described.

  FIG. 4 is a diagram showing registration contents in the registration unit.

  Here, an ID, which is an identification code for identifying registrants, and a feature amount extracted by the extraction unit 42 related to the registrant are registered in association with each other. Here, “# 1”, “# 2”, and “# 3” in the “feature amount” column are “eye feature amount of the forward image”, “mouth feature amount of the forward image”, and “landscape orientation”, respectively. This is the “total feature amount of the image”. Here, the “forward-facing image” and the “laterally-facing image” are examples of “face photographed image (first face photographed image)” and “second face photographed image”, respectively. The “feature feature amount of the eye of the forward image” corresponds to an example of the first feature amount, and the combination of the “feature feature amount of the eye of the forward image” and the “feature feature amount of the mouth of the forward image” This corresponds to an example of a second feature amount. Further, here, the combination of “the feature amount of the eyes of the forward image” and “the feature amount of the mouth of the forward image” corresponds to an example of the maximum variable feature amount. Further, the “feature amount of the entire landscape image” corresponds to an example of the second image feature amount.

  Further, in FIG. 4, in addition to “ID” and “feature”, columns of “first addition flag”, “second addition flag”, and “NG flag” are provided. Here, the “first additional flag” is a registered person who has already registered the “feature feature amount of the forward-looking image” and should be additionally registered the “feature feature amount of the mouth of the forward-looking image”. It represents that. The “second additional flag” is a registered person who has already registered both the “feature feature amount of the forward-looking image” and the “feature feature amount of the mouth of the forward-looking image”. The “feature value” of “is a registered person who should be additionally registered”. Further, the “NG flag” is used by a registration applicant or an already registered person who approximates the “feature feature amount of the forward image”, the “feature feature amount of the mouth of the forward image”, and the “total feature amount of the landscape image”. Therefore, it indicates that the person should give up authentication by face recognition and perform the authentication process by IC card.

  This FIG. 4 will be further described in the description after FIG.

  5 and 6 are flowcharts showing an operation procedure in the registration process.

  The prospective registration person applies to the administrator of the multifunction machine 10 and obtains an IC card. This IC card records the ID of the prospective registration person and that the prospective registration person is the person who has the authority to use the multifunction machine 10. The person who intends to register stands in front of the multifunction device 10 with the IC card and operates the operation unit 12 to instruct the multifunction device 10 to perform registration processing. Then, the registration processing shown in FIGS. 5 and 6 is started in the face recognition unit 40 of the multifunction machine 10.

  Here, first, an IC card is inserted into the IC card reader 14, and information indicating that there is an ID and a right of use is read from the IC card (step S101).

  Next, a face is photographed by the camera 13. Here, the “forward-looking image” is shot (step S102). Then, based on the forward-looking image, the eye feature amount is extracted (step S103), and the similarity with each registered person is calculated for the eye feature amount (step S104), and the registered user 1 For each person, based on the degree of similarity regarding the registered person, it is determined whether or not the registered person is more similar than a predetermined similarity threshold (step S105). If there is no similar person (step S106), the feature amount of the eyes of the prospective registration person is registered in association with the ID of the prospective registration person (step S107). In the “first additional flag”, “second additional flag”, and “NG flag” shown, “0” is recorded as an initial value. That is, here, the same recording as the one line of “ID = 0001” in FIG. 4 is performed.

  If it is determined in step S105 that there is a similar person, the forward feature image obtained by the face photographing in step S102 is used as it is, and the mouth feature amount is extracted (step S108). Among them, the degree of similarity of the mouth feature amount between each similar person of the eye feature amount is calculated (step S109). Further, the similarity of the mouth feature value is compared with a predetermined threshold value, and whether or not each eye similar person is a similar person whose mouth feature value is more similar than the threshold value. Is determined (step S110). When it is determined that there is no similar person for the mouth (step S111), both the eye feature amount and the mouth feature amount of the prospective registration person are registered in association with the ID of the prospective registration person. (Step S112). “ID = 0002” in FIG. 4 is an example of this registration.

  However, in the case of registration in step S112, this is not the end, and it is further determined whether or not the mouth feature amount has been registered for each registered person who has been determined to have similar eyes in step S105. (Step S113). Then, the first additional flag is rewritten to ON “1” for the registered person who is similar to the eyes and has not registered the feature value of the mouth (step S114). “ID = 0004” in FIG. 4 shows an example of this rewriting. The fact that “0 → 1” is written in the “first additional flag” field of “ID = 0004” indicates that “0” has been changed to “1” in step S114. It represents. This is because an already registered person who is similar to the eyes and has no registration of the feature amount of the mouth may be similar not only to the eyes but also to the mouth as compared to the person registered this time. Here, the first additional flag is set to ON ‘1’, and the registered feature value of the mouth is registered later for this registered person. This will be described later.

  If it is determined in step S110 that there is at least one mouth similar among similar eyes, the face of the prospective registered person is photographed again (step S115). Here, the face of the person who is scheduled to register is photographed to obtain a landscape image. Next, the feature amount is extracted from the landscape image (step S116), and the degree of similarity regarding the profile is calculated for each registered person whose eyes and mouth are both similar (step S117), and similarity determination by threshold processing is performed. Is performed (step S118).

  Then, when there is no profile person in the profile (FIG. 6, step S119), the registration feature person's eye and mouth feature value and profile feature value are associated with the registration planner's ID. And registered (step S120). This corresponds to one line of “ID = 0003” in FIG. After that, in step S121, when there is a registered person who is similar to both eyes and mouth and has no profile feature registration, the second additional flag is set for each registered person. ON is changed to “1” (step S122). This corresponds to one line of “ID = 0005” in FIG.

  Even if the registered person is similar in both eyes and mouth, the similarity cannot be calculated in step S117 if the registered person has no profile feature registration. For this reason, even if it is determined that the profile feature amount is similar to that of the registered person, the profile feature amount is not registered, so this time in step S120. Then, registration of the prospective registration person is performed. Therefore, the second additional flag is set to “1” for a registered person who does not have a profile feature amount registration and may be similar to the profile, and for this registered person, Thus, the profile value of the profile is registered.

  When there is a registered person who is similar not only in eyes and mouth but also in profile (step S119), the feature quantity of eyes and mouth and the profile quantity of profile of the person scheduled for registration are While being registered in association with the ID (step S123), ON '1' is registered in the column of “NG flag” (step S124). This NG flag on “1” means that for this registrant, the authentication by face is given up and only the authentication by IC card is enabled. The registrant with “ID = 0007” in FIG. 4 corresponds to this example.

  Further, in this step S122, the NG flag is also applied to the registered person who is similar to the new registrant of this time (“ID = 0007”) in both the feature quantity of both eyes and mouth and the profile quantity of the profile. Is turned on '1'. This is because the registered person is similar in all aspects of eyes, mouth, and profile as compared to the new registrant. Here, the registered person of “ID = 0006” in FIG. 4 corresponds to this, and the NG flag for “ID = 0006” is changed to ON “1”.

  7 to 12 are flowcharts showing the processing order in the authentication mode.

  In this authentication mode, when a person who is authenticated and intends to use the multifunction device 10 approaches the multifunction device 10, the human sensor 15 reacts to start up the multifunction device 10, and the face authentication unit 40 ( In FIG. 2 and FIG. 3, the face authentication process shown in FIG. Further, as will be described later, an additional registration process may be performed in the face authentication process.

  When this authentication process is started, first, the face is photographed by the camera 13 as in the registration process (step S201), and a forward-looking image is acquired here. Next, an eye feature amount is extracted based on the forward image (step S202), and a similarity between each eye feature amount of a registered person is calculated (step S203), and similar by comparison with a threshold value. It is determined whether or not (step S204).

  Next, it is determined whether or not there is a similar person among the registered persons (step S205).

  If it is determined that there is no similar person, the face authentication has failed and a display prompting the user to pick up the IC card is displayed on the display screen 11 shown in FIG. 1 (step S206).

  Here, there is a difference between the captured images depending on the shooting conditions (brightness of the surroundings and the direction of light rays), the facial expression of the person, and the like in the face shooting at registration (see step S102 in FIG. 5) and the face shooting at the time of this authentication. There is also a risk that even if the person is born and already registered, it is determined that they are not similar.

  When an IC card is inserted on the IC card reader 14 (see FIG. 1) and it is authenticated that the person who has the right to use the multifunction device 10 (see FIG. 1) is authenticated, the processing unit 20 of the multifunction device 10 is used. (Refer to FIG. 2) The lock is released to enable the use (step S208).

  On the other hand, if a predetermined time elapses without the IC card being tampered with or the IC card that is correctly authenticated not being tampered with (step S209), the authentication process is performed while the multifunction machine 10 remains locked. End.

  As a result of the determination in step S204, if there is only one similar person among the registered persons (step S210) and the “NG flag” is not turned on for that one person (step S211), the authentication is OK. The multifunction device 10 is unlocked (step S212).

  As described above, “NG flag on” means a person who cannot correctly authenticate a photographed image. When the NG flag is on (step S211), the process proceeds to step S206, and authentication with the IC card is performed as described above.

  If the result of determination in step S204 is that there are a plurality of similar persons (step S210), it is determined whether or not the “first additional flag” of the current authentication applicant is on / off (step S213). When the first additional flag is on, it means that it is required to extract and register the mouth feature amount.

  Here, the description of the case where the first additional flag of the authentication applicant is turned on will be deferred later, and the first additional flag is turned off. It is assumed that the feature amount has already been registered. At this time, the process proceeds to step S214 in FIG. 8, and this time, the feature amount of the mouth of the authentication applicant is calculated based on the forward-looking image obtained by the face photographing in step S201, and the similarity is calculated (step S215). ). Then, it is determined whether or not each of a plurality of similar persons determined for the eye feature amount in step S204 is a mouth similar person (step S216).

  If there is no similar person in the mouth, authentication using an IC card is performed (steps S218 to 221).

  The authentication process using the IC card (steps S218 to S221) here is the same as the authentication process using the IC card including steps S206 to S209 in FIG.

  When there is only one similar person in the mouth and the NG club of the one similar person is not on (step S223), the authentication is OK (step S224), and when the NG club is on (step S223), The process proceeds to authentication processing with an IC card (steps S218 to S221).

  When it is determined in step S216 that there are a plurality of persons who are similar not only in the eyes but also in the mouth (step S222), it is determined whether or not the “second additional flag” of the current applicant for authentication is on / off ( Step S225). The second additional flag on means that it is required to extract and register the feature amount of the landscape image. Here, the description of the case where the second additional flag for the current authentication applicant is turned on will be deferred later, and the second additional flag is turned off. In addition, the case where the profile amount of the profile is already registered will be described first.

  At this time, the process proceeds to step S226 in FIG. 9, where a landscape image of the authentication applicant is acquired by face photographing. Then, the profile amount of the profile is extracted based on the landscape image (step S227), and the profile similarity is calculated for each person determined to be similar in the determination in step S216 of FIG. Step S228) It is determined whether or not the profile is similar for each person (Step S229). When there is no similar person (step S230) and when there are a plurality of similar persons (step S231), the process proceeds to an authentication process using an IC card (steps S232 to S235), and steps S232 to S235. The authentication process by the IC card in FIG. 7 is the same as the authentication process in steps S206 to S209 in FIG.

  When there is one similar person (step S231), authentication is OK (step S235).

  Next, a case will be described in which it is determined in step S213 in FIG. 7 that the first additional flag is on. The fact that the first flag is on means that the applicant who is seeking authentication needs to register the mouth feature value but not the mouth feature value, but register the mouth feature value additionally. Is meant to be.

  At this time, the process proceeds to step S237 in FIG. 10, and the mouth feature amount is extracted from the forward image obtained by the face photographing in step S201 in FIG. 7, and it is determined that the eyes are similar in step S203 in FIG. The mouth similarity is calculated for all registered persons (step S238), and it is determined whether or not they are similar (step S239). When it is determined that there is no similar person (step S240), the mouth feature amount is additionally registered for the current applicant for authentication (step S241), and the first additional flag is reset to OFF for the applicant for authentication. (Step S242), and the first addition is performed for those whose mouth feature values are not registered among those determined to have similar eyes to the current authentication applicant in step S204 of FIG. The flag is set on (step S244). This is because there is a possibility that there is a person whose mouth is similar among those whose eyes are similar and whose mouth feature value is not registered.

  Thereafter, the authentication requester for this time is authenticated (step S253).

  If the result of determination in step S239 is that there is a similar person (step S240), it means that there are similar persons for both eyes and mouth, so that a face image is taken again for the person who wishes to authenticate. Here, a landscape image is acquired (step S246), and then a feature value of the profile is extracted based on the landscape image (step S247), and the profile of the profile between each similar person in both eyes and mouth is extracted. The similarity is calculated (step S248), and it is determined whether or not the profile is similar (step S249).

  When there is no similar person (FIG. 11, step S250), the mouth feature amount and the profile feature amount are additionally registered for the current applicant for authentication (step S251), and the first addition of the applicant for authentication is added. The flag is reset to off (step S252).

  Next, when there is a registered person who is similar to both eyes and mouth and has no profile feature added (step S253), the second additional flag is turned on for all such registered users. (Step S254). Thereafter, the authentication request for the current applicant is accepted (step S255).

  On the other hand, if a similar person exists for the profile (step S250), the mouth feature value and the profile feature value are registered for the current authentication base person (step S256), and the NG flag is set for the authentication applicant. Set to ON (step S257). In addition, in order to perform the authentication using the IC card instead of the authentication based on the face photography for the similar person whose profile is similar to that of the applicant for authentication, in this step S257, the NG flag is also turned on for the similar person. Set to

  Thereafter, an authentication process using an IC card is performed for the current authentication applicant (steps S258 to S261). The authentication process using the IC card (steps S258 to S261) is the same as the authentication process of steps S206 to S209 in FIG.

  Next, a process when it is determined in step S255 in FIG. 8 that the second additional flag is ON will be described. The second additional flag on means that the feature amount of the eyes and mouth is registered for the authentication applicant but the profile feature amount is not registered, and registration of the profile feature amount is requested. ing.

  If it is determined in step S225 in FIG. 8 that the second additional flag is ON, the process proceeds to step S262 in FIG. 12, and a landscape image is acquired by performing another face photographing for the authentication applicant (step S262). Then, the profile amount of the profile is extracted based on the landscape image (step S263), and the profile of the profile between similar persons determined by the determination in step S216 in FIG. 8, that is, persons similar in both eyes and mouth. Is calculated (step S264), and it is determined whether or not the side faces are similar (step S265).

  If there is no similar person (step S266), the profile feature amount is additionally registered for the authentication applicant (step S267), and the second additional flag is reset to OFF (step S268). Further, the second additional flag is set to ON (step S270) for those who have no profile feature amount registered among the similarities of both eyes and mouth (step S270). After that, the authentication applicant is authenticated (step 271).

  If the result of determination in step S265 is that there is a similar person for the profile (step S266), the profile feature amount is registered for the authentication applicant (step S272), and the NG flag is turned on for the authentication applicant. (Step S273).

  In this step S273, the NG flag of the similar person who is similar to the authentication applicant for the profile is also set to ON, and the similar person is also authenticated by the IC card from the next time.

  Thereafter, an authentication process using an IC card is performed for the current applicant for authentication (steps S274 to S277). The authentication process using the IC card here is the same as the authentication process in steps S206 to S209 in FIG.

  In the above-described embodiment, it has been described that the registration applicant receives the IC card from the administrator in advance. However, the administrator has the IC card, and when the registration applicant is registered, the administrator uses the IC card. It may be a system in which a person who wants to register is accompanied by a card or a person who wants to register borrows an IC card from an administrator to register. In the case of this system, the administrator may be called for additional registration to perform additional registration, and only those who cannot be authenticated by face (the person with the NG flag turned on) may be issued an IC card.

  As described above, according to the present embodiment, the load associated with the registration and authentication process is reduced, and only a person who has a similar person adds a feature amount to perform strict authentication.

  Here, the example in which the face recognition device of the present invention is applied to the multifunction machine 10 shown in FIGS. 1 and 2 has been described. However, the face recognition device of the present invention performs face recognition regardless of the device to which it is applied. Can be widely applied to suitable devices.

10 MFP 10A UI (User Interface)
DESCRIPTION OF SYMBOLS 11 Display part 12 Operation part 13 Camera 14 IC card reader 15 Human sensor 20 Processing part 21 Scanner part 22 Print part 23 Facsimile transmission / reception part 24 I / O interface 30 Operation control part 40 Face recognition part 41 Image pick-up part 42 Extraction part 43 Registration Unit 44 Calculation unit 45 Judgment unit 46 Registration / authentication control unit 51 Telephone line 52 Communication line

Claims (4)

A shooting unit for shooting a face and acquiring a face shot image on the data;
An extraction unit that extracts a facial feature amount based on a face-captured image acquired by the imaging unit;
A registration unit for registering the feature amount extracted by the extraction unit in association with an identification number for identifying each registrant;
The similarity between the new feature amount newly extracted by the extraction unit and the feature amount of the registered person registered in the registration unit is calculated based on the face photographed image newly acquired by the shooting unit. A calculation unit;
The similarity calculated by the calculation unit is compared with a predetermined threshold value, and among the registered users, an approximate registered user having a feature amount that is closer to the new feature amount than the predetermined threshold value A determination unit for determining whether or not it exists;
In the registration mode in which the feature amount of the prospective registration person is registered in association with the identification code of the prospective registration person, the photographing unit acquires the face photographing image of the prospective registration person, and based on the photographed image, the extraction unit , Extracting a first feature amount composed of a relatively small number of variables, and causing the calculation unit to extract the first feature amount of the prospective registered person and the registered person registered in the registration unit. A degree of similarity between the first feature amount and the determination unit to determine whether or not there is a first feature amount approximate registered user who is the approximate registered user regarding the first feature amount; When the first feature quantity approximate registered person does not exist, the registration unit includes a first step of registering the first feature quantity of the prospective registration person in association with the identification code of the prospective registration person. And when the first feature quantity approximate registered person exists, the face of the prospective registered person is photographed. Based on the image, the extraction unit extracts the second feature amount of the prospective registrant by adding a new variable to the variable constituting the first feature amount, and causes the calculation unit to register the registration plan. A similarity between the second feature amount of the person registered and the second feature amount of the registered person registered in the registration unit is calculated, and the determination unit causes the first feature amount approximate registration When the second feature quantity approximate registrant who is the approximate registered person relating to the second feature quantity in the person is determined to exist, the registration is performed when the second feature quantity approximate registrant does not exist. And a control unit that executes a process including a second step of registering the second feature amount of the prospective registration person in association with the identification code of the prospective registration person. apparatus.   In the registration mode, the control unit approximates a maximum variable feature amount including a maximum number of variables extracted by the extraction unit based on the first face photographed image of a prospective registration person among registered users. When there is a first image approximate registered person who is the approximate registrant having the maximum variable feature amount, the photographing unit acquires the second face photographed image of the prospective registration person, and the extracting unit A second image feature amount based on the second face-captured image is extracted, and the determination unit causes the second image to be the approximate registered user regarding the second image feature amount in the first image approximate registered user. Whether or not there is an approximate registered person is determined, and when the second image approximate registered person does not exist, the registration unit is extracted based on the first face photograph image of the prospective registered person Extracted based on the maximum variable feature amount and the second face image Both 2 image feature amount, the face authentication apparatus according to claim 1, wherein the executing further comprises processing a third step of registering in association with the identification code of the intended registrant.   When the second feature quantity unregistered registered person in which only the first feature quantity of the second feature quantity is recorded exists in the first feature quantity approximate registered person, the control unit The second feature amount is extracted by the extraction unit on the basis of a face photographed image acquired at the time of the next face photographing relating to the second feature amount unregistered registered person, and the second calculation unit is caused to extract the second feature amount. A similarity between the second feature amount of a registered person who has not registered a feature amount and the second feature amount of a registered person registered in the registration unit is calculated, and the determination unit causes the second feature amount to be calculated. When the second feature quantity approximate registered user is present, the registration unit, when there is no second registered feature quantity registered, The second feature quantity of the second feature quantity unregistered registered person is registered in association with the identification code of the second feature quantity unregistered registered person. Face authentication apparatus according to claim 1 or 2, characterized in that to execute the process. When the second image feature amount is not registered in the first image approximate registered person, the control unit relates to the second image unregistered registered person. At the time of the next face photographing, the photographing unit is made to acquire a second face photographing image of the second image unregistered registered person, and based on the second face photographing image, the extracting unit causes the second face photographing image to be acquired. An image feature amount is extracted, and the calculation unit calculates the second image feature amount of the second image unregistered registered user and the second image feature amount of the registered user registered in the registration unit. The second image approximate registration is performed by causing the determination unit to determine whether or not there is a second image approximate registered user who is the approximate registered user regarding the second image feature amount. When the user does not exist, the second image feature amount of the registrant who has not registered the second image is stored in the registration unit. Face authentication apparatus according to any one of the those to execute a process of registering in association with the identification codes of registered registered person from claim 1, wherein 3.

Images