CN111199029A

CN111199029A - Face recognition device and face recognition method

Info

Publication number: CN111199029A
Application number: CN201811367426.5A
Authority: CN
Inventors: 顾炯; 周文; 曹永刚
Original assignee: Ricoh Co Ltd
Current assignee: Ricoh Co Ltd
Priority date: 2018-11-16
Filing date: 2018-11-16
Publication date: 2020-05-26
Anticipated expiration: 2038-11-16
Also published as: CN111199029B

Abstract

The invention relates to a face recognition device and a face recognition method, which are used for recognizing the face of a recognition object, and the face recognition device comprises: a reference image storage unit; an image pickup unit; a similarity calculation unit; an angle deviation calculating section; a comprehensive similarity calculation unit; a comprehensive similarity comparison and judgment unit; and a recognition object determination unit. The reference image storage part stores a plurality of groups of reference face images, the camera part acquires the face images to obtain current face images, the similarity calculation part compares the current face images with each group of reference face images in sequence respectively in similarity, the angle deviation calculation part compares the current face images with each group of reference face images in sequence in an angle mode, the comprehensive similarity calculation part calculates comprehensive similarities corresponding to the plurality of current face images, the comprehensive similarity comparison judgment part judges whether the highest comprehensive similarity is larger than or equal to a similarity threshold value, and the identification object judgment part judges the identity information of the current identification object.

Description

Face recognition device and face recognition method

Technical Field

The present invention relates to a face recognition apparatus and a face recognition method, and more particularly, to a face recognition apparatus and a face recognition method for recognizing a face of a recognition target.

Background

Identity authentication techniques based on biological characteristics have increasingly important positions and roles in social life. Among various biometric authentication methods, identification and authentication based on human facial features are widely concerned and paid attention to because of the advantages of no invasion, low cost, good concealment, no need of special cooperation of a tested person and the like, and have wide application prospects.

In the existing face recognition device, one or a few face images of a recognition object are required to be collected in advance as a reference face image, when a face recognition operation is performed, a current face image of the recognition object in a current state is collected and compared with the reference face image to obtain a similarity, and then the similarity is compared with a preset similarity threshold value to judge identity information of the current recognition object. In practical application, the scheme may have misjudgment, and the reasons mainly include the following: 1. the number of collected reference face images of one recognition object is small, and the related features of the face are single; 2. the acquired current face image may have lower definition; 3. the reference face image is not updated, but the recognition target may change the hair style, make up, or wear glasses to change the appearance.

Disclosure of Invention

The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a face recognition apparatus and a face recognition method capable of preparing for rapid recognition of a face to be recognized.

In order to achieve the purpose, the invention adopts the following structure:

< Structure I >

The present invention provides a face recognition apparatus for recognizing a face of a recognition object, comprising: a reference image storage unit; an image pickup unit; a similarity calculation unit; an angle deviation calculating section; a comprehensive similarity calculation unit; a comprehensive similarity comparison and judgment unit; and a recognition object determination unit in which the reference image storage unit stores a plurality of sets of reference face images corresponding to a plurality of recognition objects, respectively, each set of reference face images including a reference front face image, a reference left face image, a reference right face image, a reference overhead face image, and a reference overhead face image of the recognition object, the image pickup unit acquires a face image of the recognition object in a current state to obtain a current face image, the similarity calculation unit compares the current face image with the reference front face image, the reference left face image, the reference right face image, the reference overhead face image, and the reference overhead face image in each set of reference face images in order to obtain a plurality of front face similarities, left side similarities, right side similarities, overhead similarities, and similarities corresponding to each set of reference face images, respectively, an angle deviation calculating part compares the current face image with a reference front face image, a reference left face image, a reference right face image, a reference upward face image and a reference downward face image in each group of reference face images in turn to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward deviation angles and downward deviation angles respectively corresponding to each group of reference face images, a comprehensive similarity calculating part calculates comprehensive similarities between the plurality of current face images and each group of reference face images according to a predetermined calculation rule and a corresponding front face deviation angle, left side deviation angle, right side deviation angle, upward similarity, downward similarity and corresponding downward deviation angle of each group of front face images, a comprehensive similarity comparing and judging part compares the highest comprehensive similarity as the highest comprehensive similarity with a predetermined similarity threshold, and judging whether the highest comprehensive similarity is greater than or equal to a similarity threshold, and when the highest comprehensive similarity is greater than or equal to the similarity threshold, the identification object judging part judges the corresponding identity information as the identity information of the current identification object according to the highest comprehensive similarity.

< Structure two >

The present invention also provides a face recognition apparatus for recognizing a face of a recognition object, comprising: a reference image storage unit; an image pickup unit; a control unit; a similarity calculation unit; an angle deviation calculating section; a comprehensive similarity calculation unit; a highest integrated similarity acquisition unit; an identity information acquisition unit; an information determination unit; a definition calculating section; a comprehensive definition calculating section; and a recognition object determination unit in which the reference image storage unit stores a plurality of sets of reference face images corresponding to a plurality of recognition objects, each set of reference face images including a reference front face image, a reference left face image, a reference right face image, a reference overhead face image, and a reference overhead face image of the recognition object, the image pickup unit acquires a plurality of current face images by performing a plurality of face image acquisitions on the recognition object in a current state, and the control unit controls the similarity calculation unit to compare the current face image with the reference front face image, the reference left face image, the reference right face image, the reference overhead face image, and the reference overhead face image in each set of reference face images, respectively, to obtain a plurality of front face similarities, left side similarities, and top face similarities corresponding to the reference face images of each set, respectively, Right side similarity, look-up similarity, and look-down similarity; the control angle deviation calculation part compares the current face image with a reference front face image, a reference left face image, a reference right face image, a reference upward-looking face image and a reference downward-looking face image in each group of reference face images in sequence to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward-looking deviation angles and downward-looking deviation angles which respectively correspond to each group of reference face images; controlling a comprehensive similarity calculation part to calculate according to a preset calculation rule and according to the face similarity, the left side similarity, the right side similarity, the looking-up similarity and the looking-down similarity of each group and the corresponding face deviation angle, the left side deviation angle, the right side deviation angle, the looking-up deviation angle and the looking-down deviation angle to obtain a plurality of comprehensive similarities; a control highest comprehensive similarity obtaining part obtains a highest comprehensive similarity as a highest comprehensive similarity according to the plurality of comprehensive similarities; the control part also controls the information judging part to judge whether the plurality of identity information which corresponds to the plurality of current face images respectively and is acquired by the identity information acquiring part corresponds to different identification objects or not, when the identity information is judged to correspond to different identification objects, the definition calculating part carries out image analysis on the plurality of current face images respectively to obtain a plurality of definitions, the comprehensive definition calculating part calculates the corresponding comprehensive definition aiming at different identification objects respectively according to the plurality of definitions, and the identification object judging part judges that the corresponding identification object is the current identification object according to the highest comprehensive definition.

The invention also provides a face recognition method for recognizing the face of a recognition object, which is characterized by comprising the following steps: a reference image storage step of acquiring and storing a plurality of sets of reference face images corresponding to a plurality of recognition objects, respectively, each set of reference face image including a reference front face image, a reference left side face image, a reference right side face image, a reference upward-looking face image, and a reference downward-looking face image of the recognition object; a shooting step, in which a face image of a recognition object in a current state is acquired to obtain a current face image; a similarity calculation step, in which the current face image is sequentially compared with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image and the reference downward-looking face image in each group of reference face images respectively to obtain a plurality of front face similarities, left side similarities, right side similarities, upward-looking similarities and downward-looking similarities corresponding to each group of reference face images respectively; an angle deviation calculation step, namely sequentially comparing the current face image with a reference front face image, a reference left face image, a reference right face image, a reference upward-looking face image and a reference downward-looking face image in each group of reference face images to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward-looking deviation angles and downward-looking deviation angles which respectively correspond to each group of reference face images; a comprehensive similarity calculation step, according to a preset calculation rule, calculating according to the front face similarity, the left side similarity, the right side similarity, the upward view similarity, the downward view similarity of each group and the corresponding front face deviation angle, the left side deviation angle, the right side deviation angle, the upward view deviation angle and the downward view deviation angle to obtain comprehensive similarities between the plurality of current face images and each group of reference face images; a comprehensive similarity comparison and judgment step, wherein the highest comprehensive similarity is used as the highest comprehensive similarity to be compared with a preset similarity threshold, and whether the highest comprehensive similarity is greater than or equal to the similarity threshold is judged; and an identification object judgment step, namely judging corresponding identity information as the identity information of the current identification object according to the highest comprehensive similarity when the highest comprehensive similarity is larger than or equal to the similarity threshold.

Action and Effect of the invention

According to the face recognition apparatus and the face recognition method of the present invention, since the reference image storage section stores a plurality of sets of reference face images including a reference front face image, a reference left face image, a reference right face image, a reference overhead face image, and a reference overhead face image of the recognition object, the similarity calculation section compares the current face image with each set of reference face images in turn, respectively, the angle deviation calculation section compares the current face image with each set of reference face images in turn, respectively, the integrated similarity calculation section obtains the integrated similarity between the current face image and each set of reference face images according to a predetermined calculation rule based on the comparison results of the similarity calculation section and the angle deviation calculation section, the recognition object determination section determines the corresponding identity information as the identity information of the current recognition object based on the highest integrated similarity, the face recognition device can calculate the similarity of different reference pictures and calculate the highest comprehensive similarity, thereby reducing the influence of the face orientation of the current face image on the recognition result and ensuring that the result is more accurate when the face of the recognition object is recognized.

Drawings

Fig. 1 is a block diagram of a face recognition apparatus according to a first embodiment of the present invention;

FIG. 2 is a diagram of an identity information display screen according to an embodiment of the present invention;

FIG. 3 is a diagram of an identity information display screen according to an embodiment of the present invention;

FIG. 4 is a flowchart illustrating operations of a face recognition apparatus according to an embodiment of the present invention;

FIG. 5 is a diagram illustrating steps of an update process performed by a face recognition device according to an embodiment of the present invention;

fig. 6 is a block diagram of a face recognition apparatus according to a second embodiment of the present invention; and

fig. 7 is a flowchart illustrating operations of a face recognition apparatus according to a second embodiment of the present invention.

Detailed Description

In order to make the technical means, the creation features, the achievement purposes and the effects of the invention easy to understand, the face recognition device of the invention is specifically described below with reference to the embodiments and the accompanying drawings.

As a first aspect, the present invention provides a face recognition apparatus for recognizing a face of a recognition target, comprising: a reference image storage unit; an image pickup unit; a similarity calculation unit; an angle deviation calculating section; a comprehensive similarity calculation unit; a comprehensive similarity comparison and judgment unit; and a recognition object determination unit in which the reference image storage unit stores a plurality of sets of reference face images corresponding to a plurality of recognition objects, respectively, each set of reference face images including a reference front face image, a reference left face image, a reference right face image, a reference overhead face image, and a reference overhead face image of the recognition object, the image pickup unit acquires a face image of the recognition object in a current state to obtain a current face image, the similarity calculation unit compares the current face image with the reference front face image, the reference left face image, the reference right face image, the reference overhead face image, and the reference overhead face image in each set of reference face images in order to obtain a plurality of front face similarities, left side similarities, right side similarities, overhead similarities, and similarities corresponding to each set of reference face images, respectively, an angle deviation calculating part compares the current face image with a reference front face image, a reference left face image, a reference right face image, a reference upward face image and a reference downward face image in each group of reference face images in turn to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward deviation angles and downward deviation angles respectively corresponding to each group of reference face images, a comprehensive similarity calculating part calculates comprehensive similarities between the plurality of current face images and each group of reference face images according to a predetermined calculation rule and a corresponding front face deviation angle, left side deviation angle, right side deviation angle, upward similarity, downward similarity and corresponding downward deviation angle of each group of front face images, a comprehensive similarity comparing and judging part compares the highest comprehensive similarity as the highest comprehensive similarity with a predetermined similarity threshold, and judging whether the highest comprehensive similarity is greater than or equal to a similarity threshold, and when the highest comprehensive similarity is greater than or equal to the similarity threshold, the identification object judging part judges the corresponding identity information as the identity information of the current identification object according to the highest comprehensive similarity.

In the first aspect, the present invention may further include: an update recording part, an update control part, a similarity judging part, a current time acquiring part, an update interval judging part, a reference similarity calculating part, a reference similarity judging part and a reference image updating part, wherein the update recording part stores the update time of each group of reference face images, once the identification object judging part judges the corresponding identity information according to the highest comprehensive similarity of the current face image, the update control part takes the corresponding group of reference face images as the current reference face image group, the control similarity judging part judges whether the highest comprehensive similarity between the current face image and the current reference face image group is between 80% and 90%, when the highest comprehensive similarity is between 80% and 90%, the update control part controls the current time acquiring part to acquire the current time, and controls the update interval judging part to judge the latest update time corresponding to the current reference face image group stored in the update recording part and the current time If the interval is larger than the preset time interval, the updating control part controls the reference similarity calculation part to respectively calculate the current reference similarity between the current face image and each reference face image in the current reference face image group and calculate a plurality of reference similarities between each pair of reference face images in the current reference face image group, further controls the reference similarity judgment part to judge whether the minimum value in the current reference similarity is smaller than the average value of the reference similarities, and when the minimum value is smaller than the average value, the updating control part controls the reference image updating part to perform updating operation on the corresponding reference face image stored in the reference image storage part according to the current face image and controls the updating recording part to correspondingly record the updating time of the group of reference face images.

In the first aspect, the present invention may further include: a reference image deleting section in which each set of reference face images in the reference image storage section includes at least one reference front face image, at least one reference left face image, at least one reference right face image, at least one reference upward-looking face image, and at least one reference downward-looking face image, the reference image updating section includes an orientation determining unit that determines a face orientation of the current face image as a current face orientation based on a front face deviation angle, a left side deviation angle, a right side deviation angle, an upward-looking deviation angle, and a downward-looking deviation angle corresponding to the current reference face image group, and a number determining unit; the number judging unit judges whether the number of reference face images having the same orientation as the orientation of the current face in the current reference face image group is smaller than a predetermined number, when the number is smaller than the predetermined number, the update control unit controls the reference image storage unit to additionally store the current face image as a new reference face image corresponding to the orientation of the current face and the current time as the storage time of the new reference face image, when the number is not smaller than the predetermined number, the update control unit controls the reference image storage unit to additionally store the new reference face image and the storage time thereof, when the number is not smaller than the predetermined number, the update control unit controls the reference image storage unit to additionally store the current face image as a new reference face image corresponding to the orientation of the current face and the current time as the storage time of the new reference face image, and controls the reference image deleting unit to delete the reference face image corresponding to the orientation of the current face and having the earliest storage time in the reference image storage unit And (4) removing.

In the first aspect, the present invention may further have a feature that: the preset calculation rule comprises a similarity coefficient endowing rule and a comprehensive similarity calculation rule, wherein the similarity coefficient endowing rule is as follows: firstly, a front face corresponding to each group of reference face images is formed by adopting a coordinate modeThe deviation angle, the left deviation angle, the right deviation angle, the upward deviation angle, and the downward deviation angle are respectively expressed as (x)₀,y₀)、(x₁,y₁)、(x₂,y₂)、(x₃,y₃) And (x)₄,y₄) Thereby calculating a set of deviation degrees of the face deviation angle, the left deviation angle, the right deviation angle, the upward deviation angle and the downward deviation angle respectively

And

then, adding a fixed value 1e-4 to a group of deviation degrees respectively and obtaining a corresponding group of sequence values by calculating the reciprocal

And

finally, a group of similarity coefficients corresponding to the corresponding face similarity, left side similarity, right side similarity, look-up similarity and look-down similarity are obtained through calculation and are respectively

And

wherein s ═ w₀+w₁+w₂+w₃+w₄The comprehensive similarity calculation rule is that the face similarity, the left side similarity, the right side similarity, the looking-up similarity and the looking-down similarity are multiplied by corresponding similarity coefficients respectively and then added to obtain the comprehensive similarity.

In the first aspect, the present invention may further have a feature that: the camera shooting part comprises a camera, a camera shooting control unit, a real face judging unit and an output unit, once the camera shoots a face image, the camera shooting control unit controls the real face judging unit to judge whether the face image is a real face or not, and further controls the output unit to output the face image as a current face image when the face image is judged to be the real face.

In the first aspect, the present invention may further have a feature that: wherein the face orientation of the reference front face image is a front side, the face orientation of the reference left side face image is a left side, the face orientation of the reference right side face image is a right side, the face orientation of the reference upward face image is an upper side, the face orientation of the reference downward face image is a lower side, the angular deviation of the face orientation of the reference left side face image from the face orientation of the reference front face image and the angular deviation of the face orientation of the reference right side face image from the face orientation of the reference front face image are 10 ° -30 °, and the angular deviation of the face orientation of the reference upward face image from the face orientation of the reference front face image and the angular deviation of the face orientation of the reference downward face image from the face orientation of the reference front face image are 10 ° -20 °.

In the first aspect, the present invention may further include: the human face image similarity comparison and judgment system comprises a preset threshold storage part, a threshold updating control part, a threshold accuracy calculation part and a similarity threshold storage part, wherein the preset threshold storage part stores a plurality of preset similarity thresholds, the threshold updating control part controls the threshold accuracy calculation part to calculate the accuracy value of each preset similarity threshold according to the preset threshold updating time point and a plurality of groups of reference human face images stored in a reference image storage part aiming at the preset similarity thresholds based on the accuracy value calculation rule and controls the similarity threshold storage part to store the preset similarity threshold with the highest accuracy value as the similarity threshold with which a comprehensive similarity comparison and judgment part is used as the judgment reference.

In the first aspect, the present invention may further have a feature that: wherein, the accurate value calculation rule is as follows: first, all of the reference image storage part is traversedThe method comprises the following steps that part of reference face images form a plurality of same-group pairs by pairwise matching of reference face images belonging to the same group, a plurality of different-group pairs by pairwise matching of reference face images not belonging to the same group are formed, then, the similarity between two reference face images of each same-group pair and each different-group pair is calculated, and finally, the accurate value a of each preset similarity threshold is calculated as follows:

wherein a is₁Is the logarithm of the pairings with a similarity above a preset similarity threshold, a₂Is the logarithm of the heterogeneous pairs with a similarity below a preset similarity threshold, a₃The logarithm of the same pair is the logarithm of the same pair and the logarithm of the different pair is the sum.

As a second aspect, the present invention provides a face recognition apparatus for recognizing a face of a recognition target, comprising: a reference image storage unit; an image pickup unit; a control unit; a similarity calculation unit; an angle deviation calculating section; a comprehensive similarity calculation unit; a highest integrated similarity acquisition unit; an identity information acquisition unit; an information determination unit; a definition calculating section; a comprehensive definition calculating section; and a recognition object determination unit in which the reference image storage unit stores a plurality of sets of reference face images corresponding to a plurality of recognition objects, each set of reference face images including a reference front face image, a reference left face image, a reference right face image, a reference overhead face image, and a reference overhead face image of the recognition object, the image pickup unit acquires a plurality of current face images by performing a plurality of face image acquisitions on the recognition object in a current state, and the control unit controls the similarity calculation unit to compare the current face image with the reference front face image, the reference left face image, the reference right face image, the reference overhead face image, and the reference overhead face image in each set of reference face images, respectively, to obtain a plurality of front face similarities, left side similarities, and top face similarities corresponding to the reference face images of each set, respectively, Right side similarity, look-up similarity, and look-down similarity; the control angle deviation calculation part compares the current face image with a reference front face image, a reference left face image, a reference right face image, a reference upward-looking face image and a reference downward-looking face image in each group of reference face images in sequence to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward-looking deviation angles and downward-looking deviation angles which respectively correspond to each group of reference face images; controlling a comprehensive similarity calculation part to calculate according to a preset calculation rule and according to the face similarity, the left side similarity, the right side similarity, the looking-up similarity and the looking-down similarity of each group and the corresponding face deviation angle, the left side deviation angle, the right side deviation angle, the looking-up deviation angle and the looking-down deviation angle to obtain a plurality of comprehensive similarities; a control highest comprehensive similarity obtaining part obtains a highest comprehensive similarity as a highest comprehensive similarity according to the plurality of comprehensive similarities; the control part also controls the information judging part to judge whether the plurality of identity information which corresponds to the plurality of current face images respectively and is acquired by the identity information acquiring part corresponds to different identification objects or not, when the identity information is judged to correspond to different identification objects, the definition calculating part carries out image analysis on the plurality of current face images respectively to obtain a plurality of definitions, the comprehensive definition calculating part calculates the corresponding comprehensive definition aiming at different identification objects respectively according to the plurality of definitions, and the identification object judging part judges that the corresponding identification object is the current identification object according to the highest comprehensive definition.

In the second aspect, the present invention may further have a feature that: wherein, synthesize definition calculation portion includes: a definition coefficient storage unit which stores a plurality of image definitions and corresponding definition coefficients; the definition coefficient acquisition unit retrieves and acquires a corresponding definition coefficient from the definition coefficient storage unit according to the definition of each current face image in sequence; and the comprehensive accumulation unit is used for sequentially accumulating the definition coefficients of the current face image corresponding to each different recognition object to obtain the comprehensive definition of each recognition object.

As a third aspect, the present invention provides a face recognition method for recognizing a face of a recognition target, comprising: a reference image storage step of acquiring and storing a plurality of sets of reference face images corresponding to a plurality of recognition objects, respectively, each set of reference face image including a reference front face image, a reference left side face image, a reference right side face image, a reference upward-looking face image, and a reference downward-looking face image of the recognition object; a shooting step, in which a face image of a recognition object in a current state is acquired to obtain a current face image; a similarity calculation step, in which the current face image is sequentially compared with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image and the reference downward-looking face image in each group of reference face images respectively to obtain a plurality of front face similarities, left side similarities, right side similarities, upward-looking similarities and downward-looking similarities corresponding to each group of reference face images respectively; an angle deviation calculation step, namely sequentially comparing the current face image with a reference front face image, a reference left face image, a reference right face image, a reference upward-looking face image and a reference downward-looking face image in each group of reference face images to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward-looking deviation angles and downward-looking deviation angles which respectively correspond to each group of reference face images; a comprehensive similarity calculation step, according to a preset calculation rule, calculating according to the front face similarity, the left side similarity, the right side similarity, the upward view similarity, the downward view similarity of each group and the corresponding front face deviation angle, the left side deviation angle, the right side deviation angle, the upward view deviation angle and the downward view deviation angle to obtain comprehensive similarities between the plurality of current face images and each group of reference face images; a comprehensive similarity comparison and judgment step, wherein the highest comprehensive similarity is used as the highest comprehensive similarity to be compared with a preset similarity threshold, and whether the highest comprehensive similarity is greater than or equal to the similarity threshold is judged; and an identification object judgment step, namely judging corresponding identity information as the identity information of the current identification object according to the highest comprehensive similarity when the highest comprehensive similarity is larger than or equal to the similarity threshold.

< example one >

In the first embodiment, the face recognition device is used for recognizing the face of the recognition object. Specifically, for example, the face recognition device may be an attendance checking device of an enterprise, and a set of reference face images of each employee of the enterprise is stored in advance. When someone attendance was checked card, attendance device carried out face identification to this personnel, discerned this personnel's identity information when attendance device, accomplished attendance operation of checking card promptly to this personnel. If the person is a non-enterprise person, the attendance checking device displays the prompt information of a stranger.

Fig. 1 is a block diagram of a face recognition apparatus according to a first embodiment of the present invention.

As shown in fig. 1, the face recognition apparatus 100 includes a reference image storage unit 1, an imaging unit 2, a similarity calculation unit 3, an angle deviation calculation unit 4, a comprehensive similarity calculation unit 5, a comprehensive similarity comparison determination unit 6, a recognition target determination unit 7, a screen storage unit 8, an output display unit 9, an output display control unit 10, a similarity determination unit 11, a current time acquisition unit 12, an update recording unit 13, an update interval determination unit 14, a reference similarity calculation unit 15, a reference similarity determination unit 16, a reference image update unit 17, a reference image deletion unit 18, an update control unit 19, a preset threshold storage unit 20, a threshold accuracy calculation unit 21, a similarity threshold storage unit 22, a threshold update control unit 23, a communication unit 24, and a control unit 25.

The reference image storage unit 1 stores a plurality of sets of reference face images which are registered in advance and correspond to a plurality of recognition targets, respectively, each set of reference face images including at least one reference front face image, at least one reference left face image, at least one reference right face image, at least one reference upward-looking face image, and at least one reference downward-looking face image of the recognition targets. The face orientation of the reference front face image is the front side, the face orientation of the reference left side face image is the left side, the face orientation of the reference right side face image is the right side, the face orientation of the reference upward-looking face image is the upper side, and the face orientation of the reference downward-looking face image is the lower side. In this embodiment, the number of the reference front face image, the reference left side face image, the reference right side face image, the reference overhead face image, and the reference overhead face image of each group of reference face images is one, but may be set to be plural according to actual needs of an enterprise.

The face orientation of the reference front face image is taken as a reference face orientation, the angle deviation between the face orientation of the reference left side face image and the face orientation of the reference right side face image and the reference face orientation is 10-30 degrees, and the angle deviation between the face orientation of the reference upward-looking face image and the face orientation of the reference downward-looking face image and the reference face orientation is 10-20 degrees.

The camera part 2 acquires a face image of the recognition object in the current state to obtain a current face image, and comprises a camera, a camera control unit, a real face judgment unit and an output unit. The camera is used for shooting and collecting face images. The camera control unit is used for controlling the real face judging unit to judge whether the face image is a real face or not, and when the real face judging unit judges that the face image is a real face, the camera control unit controls the output unit to output the face image as a current face image. For example, in an attendance card punching system of an enterprise, after a camera shoots an image, a real face judgment unit judges whether the image is a real face image or not so as to prevent the camera from collecting a photo image of a certain employee which is prepared in advance.

The similarity calculation unit 3 compares the similarity of the current face image with each of the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference upward-looking face image in each of the sets of reference face images in sequence, and obtains a plurality of front face similarities, left side similarities, right side similarities, upward-looking similarities, and downward-looking similarities corresponding to each of the sets of reference face images.

The angle deviation calculation unit 4 compares the current face image with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference downward-looking face image in each set of reference face images in order of angle, and obtains a plurality of front face deviation angles, left deviation angles, right deviation angles, upward-looking deviation angles, and downward-looking deviation angles corresponding to each set of reference face images.

The integrated similarity calculation unit 5 calculates, according to a predetermined calculation rule, the integrated similarity between the plurality of current face images and each group of reference face images based on the front face similarity, the left side similarity, the right side similarity, the upward view similarity, the downward view similarity of each group, and the corresponding front face deviation angle, left side deviation angle, right side deviation angle, upward view deviation angle, and downward view deviation angle. The predetermined calculation rule includes a similarity coefficient assignment rule and a comprehensive similarity calculation rule.

The similarity coefficient assignment rule is:

first, a face deviation angle, a left deviation angle, a right deviation angle, an elevation deviation angle, and an overhead deviation angle corresponding to each set of reference face images are respectively expressed as (x) in the form of coordinates₀,y₀)、(x₁,y₁)、(x₂,y₂)、(x₃,y₃) And (x)₄,y₄) Thereby calculating the front face deviation angle b₀Left deviation angle b₁Right deviation angle b₂Elevation deviation angle b₃And an angle of deviation b in plan view₄A set of deviation degrees of

And

wherein x is₀A deviation angle in the horizontal direction, x, of the right front of the face of the current face image and the right front of the face of the reference face image₁A deviation angle in the horizontal direction, x, of the right front of the face of the current face image and the right front of the face of the reference left face image₂A deviation angle in the horizontal direction, x, of a position directly in front of the face of the current face image and a position directly in front of the face of the reference right face image₃Is a deviation angle, x, in the horizontal direction between the right front of the face of the current face image and the right front of the face of the reference upward face image₄The deviation angle in the horizontal direction between the front of the face of the current face image and the front of the face of the reference top-view face image is shown. y is₀Is at presentDeviation angle in vertical direction, y, of the front of the face image and the front of the face of the reference front face image₁Is a deviation angle in the vertical direction, y, of the right front of the face of the current face image and the right front of the face of the reference left face image₂Is a deviation angle in the vertical direction, y, of the right front of the face of the current face image and the right front of the face of the reference right face image₃Is a deviation angle in the vertical direction, y, of the right front of the face of the current face image and the right front of the face of the reference upward face image₄The deviation angle between the right front of the face of the current face image and the right front of the face of the reference top-view face image in the vertical direction is shown.

Then, adding a fixed value 1e-4 to the group of deviation degrees respectively and obtaining a corresponding group of sequence values by reciprocal calculation

And

wherein, w₀Is a sequence value corresponding to the angle of deviation of the face, w₁Is a sequence value corresponding to the left deviation angle, w₂Is a sequence value corresponding to the right deviation angle, w₃Is a sequence value, w, corresponding to the upward deviation angle₄Is a sequence value corresponding to the overlooking deviation angle.

And

wherein, w₀Similarity coefficient, w, being the similarity of a positive face₁Similarity coefficient, w, for left side similarity₂Similarity coefficient, w, for the right side similarity₃Similarity coefficient for looking up at similarity, w₄Similarity system for overlooking similarityNumber s is the corresponding sequence value w₀、w₁、w₂、w₃And w₄Is i.e. s ═ w₀+w₁+w₂+w₃+w₄。

The comprehensive similarity calculation rule is as follows: and multiplying the face similarity, the left side similarity, the right side similarity, the upward view similarity and the downward view similarity with corresponding similarity coefficients respectively, and then adding to obtain the comprehensive similarity.

The integrated similarity comparison and determination unit 6 compares the highest integrated similarity, which is the highest integrated similarity, with a predetermined similarity threshold value, and determines whether or not the highest integrated similarity is equal to or greater than the similarity threshold value.

The identification object determination unit 7 is configured to determine, based on the highest integrated similarity, corresponding identity information as the identity information of the current identification object when the highest integrated similarity is equal to or greater than the similarity threshold.

The screen storage unit 8 stores a stranger prompt screen 8a and a plurality of identification information display screens 8b corresponding to a plurality of identification objects, respectively.

FIG. 2 is a diagram of an identity information display screen according to an embodiment of the present invention.

As shown in fig. 2, the stranger prompt screen 8a includes a stranger prompt box 81a, and the stranger prompt box 81a is used to display a prompt message "the current recognition object is a stranger".

FIG. 3 is a diagram of an identity information display screen according to an embodiment of the present invention.

As shown in fig. 3, each of the plurality of identification information display screens 8b includes an identification information display frame 81b for displaying identification information of a corresponding recognition target. For example, the content displayed in the identity information display box corresponding to employee a of the enterprise is "the current identification object is employee a".

The output display unit 9 is used for displaying the screen.

The output display control unit 10 controls the output display unit 9 to display the screen based on the determination result of the recognition target determination unit 7.

The similarity judging part 11 is used for judging whether the highest comprehensive similarity is between 80% and 90%.

The current time acquisition unit 12 is for acquiring a current time.

The update recording section 13 is used to record and store the update time of each set of reference face images.

The update interval determination unit 14 determines whether or not the interval between the latest update time corresponding to the current reference face image group stored in the update recording unit 13 and the current time acquired by the current time acquisition unit 12 is a predetermined time interval. In the embodiment, the predetermined time interval is 1 day, but may be set to other days according to the actual needs of the enterprise.

The reference similarity calculation unit 15 is configured to calculate a current reference similarity between the current face image and each of the reference face images in the current reference face image group, and calculate a plurality of reference similarities between each pair of reference face images in the current reference face image group, when the interval between the latest update time and the current time is greater than a predetermined time interval.

The reference similarity determination unit 16 is configured to determine whether or not the minimum value of the current reference similarities is smaller than the average value of the reference similarities.

The reference image updating section 17 is configured to perform an updating operation on the corresponding reference face image stored in the reference image storage section 1 based on the current face image when the minimum value of the current reference similarity is smaller than the average value of the reference similarities. The reference image update unit 17 includes a direction determination unit and a number determination unit.

The orientation judging unit is used for judging the face orientation of the current face image as the current face orientation according to the front face deviation angle, the left side deviation angle, the right side deviation angle, the upward view deviation angle and the downward view deviation angle corresponding to the current reference face image group. For example, when the value of the left deviation angle is the largest, the face orientation of the reference left face image in the current reference face image group is the current face orientation.

The number judgment unit is used for judging whether the number of the reference face images with the same orientation as the orientation of the current face in the current reference face image group is less than a preset number.

The reference image deleting unit 18 deletes the reference face image that corresponds to the current face orientation and is stored at the earliest time in the reference image storage unit 1.

The update control unit 19 controls the operation of a component related to image update, and includes: when the identification object judging part 7 judges the identity information of the current identification object, the corresponding group of reference face images is used as the current reference face image group to control the similarity judging part 11 to judge whether the highest comprehensive similarity between the current face image and the current reference face image group is between 80 and 90 percent; when the current time is 80-90%, controlling the current time obtaining part to obtain the current time, and controlling the updating interval judging part 14 to judge whether the interval between the latest updating time corresponding to the current reference face image group and the current time is larger than a preset time interval; when the current reference similarity is judged to be greater than the reference similarity, the control reference similarity calculation part 15 calculates the current reference similarity and the plurality of reference similarities, and the control reference similarity judgment part 16 further judges whether the minimum value of the current reference similarity is smaller than the average value of the plurality of reference similarities; when the face orientation is smaller than the preset face orientation, the control orientation judging unit judges the face orientation of the current face image, and the control number judging unit judges whether the number of the standard face images in the same direction as the current face orientation is smaller than the preset number; when the face image is judged to be smaller than the current face image, the reference image storage part 1 is controlled to additionally store the current face image and the current time; when it is determined that the current face image is not less than the reference face image, the reference image storage unit 1 is controlled to additionally store the current face image and the current time, the reference image deleting unit 18 is controlled to delete the reference face image whose corresponding storage time is the earliest, and the update recording unit 13 is controlled to record the update time correspondingly.

The preset threshold value storage section 20 stores a plurality of preset threshold values of the degree of similarity,

the accuracy calculation section 21 is configured to calculate an accuracy value of each preset similarity threshold value based on the accuracy value calculation rule and the plurality of sets of reference face images stored in the reference image storage section 1 with respect to the preset similarity threshold value.

Wherein, the accurate value calculation rule is as follows:

firstly, all the reference face images in the reference image storage part are traversed, the reference face images belonging to the same group are paired pairwise to form a plurality of same-group pairs, and the reference face images not belonging to the same group are paired pairwise to form a plurality of different-group pairs.

Then, the similarity between the two reference face images of each same-pair group and each different-pair group is calculated.

Finally, calculating the accurate value a of each preset similarity threshold as follows:

The similarity threshold storage unit 22 is configured to store a preset similarity threshold having the highest accuracy as a similarity threshold for the integrated similarity comparison and determination unit 6 to use as a determination reference.

The threshold update control unit 23 controls the operation of a means for updating the similarity threshold, and includes: according to the preset threshold update time point, the control threshold accuracy calculation section 21 calculates the accuracy value of each preset similarity threshold, and controls the similarity threshold storage section 22 to store the similarity threshold having the integrated similarity comparison determination section 6 as the determination reference. In this embodiment, the preset threshold updating time point is 24 points per day of the week, but may be set to other times according to the actual needs of the customers.

The communication unit 24 exchanges data between the respective components of the face recognition apparatus 100.

The control unit 25 controls operations of the respective components of the face recognition apparatus 100.

The face recognition device of this embodiment acquires a current face image of a recognition object in a current state, performs similarity analysis on the current face image and each group of reference face images, calculates the comprehensive similarity, determines the identity information of the recognition object according to the highest comprehensive similarity, and performs image update operation on a group of reference face images corresponding to the recognition object.

Fig. 4 is a flowchart illustrating operations of a face recognition apparatus according to an embodiment of the present invention.

As shown in fig. 4, in the first embodiment, the operation flow of the face recognition apparatus 100 includes the following steps:

and step S1-1-1, shooting a human face image by the camera, and then entering step S1-1-2.

In step S1-1-2, the camera control unit controls the real face judgment unit to judge whether the face image captured by the camera is a real face, and when the judgment is yes, the processing proceeds to step S1-1-3, and when the judgment is no, the processing proceeds to step S1-1-1.

In step S1-1-3, the imaging control unit controls the output unit to output the face image as the current face image, and then proceeds to step S1-1-4.

In step S1-1-4, the similarity calculation unit 3 compares the similarity between the current face image and each of the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference upward-looking face image in each of the reference face image groups in sequence to obtain a plurality of front face similarities, left side similarities, right side similarities, upward-looking similarities, and upward-looking similarities corresponding to each of the reference face image groups, and then proceeds to step S1-1-5.

In step S1-1-5, the angle deviation calculation unit 4 compares the current face image sequentially with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference downward-looking face image in each set of reference face images, respectively, to obtain a plurality of front face deviation angles, left deviation angles, right deviation angles, upward-looking deviation angles, and downward-looking deviation angles corresponding to each set of reference face images, respectively, and then proceeds to step S1-1-6.

In step S1-1-6, the integrated similarity calculation section 5 calculates the integrated similarity between the plurality of current face images and each set of reference face images according to a predetermined calculation rule, and then proceeds to step S1-1-7.

In step S1-1-7, the integrated similarity comparison and determination section 6 compares the highest integrated similarity, which is the highest integrated similarity, with a predetermined similarity threshold, determines whether or not the highest integrated similarity is equal to or greater than the predetermined threshold, and if so, proceeds to step S1-1-8, and if not, proceeds to step S1-1-10.

In step S1-1-8, the identification object determining unit 7 determines the corresponding identification information as the identification information of the current identification object based on the highest integrated similarity, and then proceeds to step S1-1-9.

In step S1-1-9, the output display control unit 10 controls the output display unit 9 to display the identification information display screen 8b corresponding to the identification information of the current recognition target, and then the process proceeds to step S1-2, where a set of reference face images corresponding to the identification information of the current recognition target is updated.

In step S1-1-10, the recognition target determination unit 7 determines that the current recognition target is a stranger, and then proceeds to step S1-1-11.

In step S1-1-11, the output display control section 10 controls the output display section 9 to display the stranger prompt screen 8a, and then enters an end state.

In the above operation flow, the reference front face image, the reference left side face image, the reference right side face image, the reference upward face image, and the reference downward face image of each group of reference face images are all one, and when there are a plurality of reference front face images, reference left side face images, reference right side face images, reference upward face images, and reference downward face images of each group of reference face images, the above steps S1-1-4 may be repeated a plurality of times to obtain a plurality of front face similarities, a plurality of left side similarities, a plurality of right side similarities, a plurality of upward view similarities, and a plurality of downward view similarities of each group of reference face images, and the plurality of front face similarities, the plurality of left side similarities, the plurality of right side similarities, the plurality of upward view similarities, and the plurality of downward view similarities are respectively averaged to obtain an average front face similarity, an average left side similarity, an average upward view similarity, and an average downward view similarity, The average right-side similarity, the average upward-view similarity and the average downward-view similarity are used as the corresponding front face similarity, the left-side similarity, the right-side similarity, the upward-view similarity and the downward-view similarity of each group of the reference face images, the step S1-1-5 is repeated for a plurality of times, the average front face deviation angle, the average left-side deviation angle, the average right-side deviation angle, the average upward-view deviation angle and the average downward-view deviation angle are calculated and used as the corresponding front face deviation angle, left-side deviation angle, right-side deviation angle, upward-view deviation angle and downward-view deviation angle of each group of the reference face images, and then the process proceeds to the step S1-1-6.

Fig. 5 is a diagram of steps of an update process of a face recognition apparatus according to an embodiment of the present invention.

As shown in fig. 5, the face recognition apparatus 100 of the present embodiment further performs an update process on the current reference face image group in the reference image storage unit with the current reference face image as the face image to be updated after the face recognition operation is completed, where the update process S1-2 includes the steps of:

in step S1-2-1, the update control unit 19 sets a set of reference face images corresponding to the identification information of the current recognition object as a current reference face image group, and controls the similarity determination unit 11 to determine whether the highest integrated similarity between the current face image and the current reference face image group is between 80% and 90%, and if yes, the processing proceeds to step S1-2-2, and if no, the processing proceeds to an end state.

In step S1-2-2, the update control section 19 controls the current time acquisition section 12 to acquire the current time, and then proceeds to step S1-2-3.

In step S1-2-3, the update controller 19 controls the update interval determiner 14 to determine whether the interval between the latest update time corresponding to the current reference face image group stored in the update recorder 13 and the current time is greater than a predetermined time interval, and if so, the processing proceeds to step S1-2-4, and if not, the processing proceeds to the end state.

In step S1-2-4, the update control section 19 controls the reference similarity calculation section 15 to calculate the current reference similarities between the current face image and each of the reference face images in the current reference face image group and calculate the plurality of reference similarities between each pair of reference face images in the current reference face image group, respectively, and then proceeds to step S1-2-5.

In step S1-2-5, the update control section 19 controls the reference similarity determination section 16 to determine whether or not the minimum value of the current reference similarities is smaller than the average value of the reference similarities, and proceeds to step S1-2-6 when the determination is yes, and proceeds to the end state when the determination is no.

In step S1-2-6, the update control unit 19 controls the orientation determination unit to determine the face orientation of the current face image as the current face orientation based on the front face deviation angle, the left side deviation angle, the right side deviation angle, the upward view deviation angle, and the downward view deviation angle corresponding to the current reference face image group, and then proceeds to step S1-2-7.

In step S1-2-7, the update control section 19 controls the number judgment unit to judge whether the number of reference face images oriented in the same direction as the current face in the current reference face image group is smaller than a predetermined number, and proceeds to step S1-2-8 when judged yes, and proceeds to step S1-2-9 when judged no.

In step S1-2-8, the update control unit 19 controls the reference image storage unit 1 to additionally store the new reference face image and the storage time thereof, using the current face image as the new reference face image corresponding to the orientation of the current face and the current time as the storage time of the new reference face image, and then the process proceeds to step S1-2-11.

In step S1-2-9, the update control unit 19 controls the reference image storage unit 1 to additionally store the new reference face image and the storage time thereof, using the current face image as the new reference face image corresponding to the orientation of the current face and the current time as the storage time of the new reference face image, and then the process proceeds to step S1-2-10.

In step S1-2-10, the update control section 19 controls the reference image deleting section to delete the reference face image corresponding to the current face orientation and having the earliest storage time in the reference image storage section, and then proceeds to step S1-2-11.

In step S1-2-11, the update control section 19 controls the update recording section 13 to record the current time as the update time of the set of reference face images correspondingly, and then enters the end state.

Effect of the first embodiment

According to the face recognition apparatus and the face recognition method of the present embodiment, since the reference image storage section stores a plurality of sets of reference face images including a reference front face image, a reference left face image, a reference right face image, a reference overhead face image, and a reference overhead face image of a recognition object, the similarity calculation section compares the current face image with each set of reference face images in order, respectively, the angle deviation calculation section compares the current face image with each set of reference face images in order, respectively, the integrated similarity calculation section obtains the integrated similarity between the current face image and each set of reference face images according to a predetermined calculation rule based on the comparison results of the similarity calculation section and the angle deviation calculation section, the recognition object determination section determines corresponding identity information as the identity information of the current recognition object based on the highest integrated similarity, therefore, the face recognition device of the embodiment can calculate the similarity of different reference pictures and calculate the highest comprehensive similarity, and identity information is judged through the highest comprehensive similarity and the similarity threshold, so that the influence of the face orientation of the current face image on the recognition result is reduced, and the face recognition device has higher accuracy.

In addition, the update control unit controls the reference image update unit to perform an update operation on the corresponding reference face image according to the current face image, so that the face recognition apparatus of the present embodiment can periodically update the reference face image, and the situation that face recognition cannot be performed due to a face change of the face of the recognition target is avoided.

And the updating control part controls the reference image updating part to update the reference face image when the similarity judging part judges that the highest comprehensive similarity of the current face image is 80-90 percent, the reference similarity judging part judges that the minimum value of the current reference similarity is smaller than the average value of the reference similarities and the updating interval judging part judges that the interval between the latest updating time of the current reference image group and the current time is not in the preset time interval, so that the current reference face image group has higher image quality and can regularly update the reference face image.

Further, the orientation determination unit can determine the face orientation of the current face image as the current face orientation; the updating control part can take the current face image as a new reference face image corresponding to the current face orientation, and control the reference image storage part to additionally store the new reference face image and the storage time thereof, so that the embodiment can judge the face orientation of the current face image and add the corresponding face orientation to the corresponding current reference face image group, thereby ensuring that the image characteristics of the original five different face orientations are still kept in the current reference face image group.

In addition, the face orientation of the reference front face image is the front side, the face orientation of the reference left side face image is the left side, the face orientation of the reference right side face image is the right side, the face orientation of the reference upward-looking face image is the upper side, and the face orientation of the reference downward-looking face image is the lower side.

In addition, the threshold updating control part controls the threshold accuracy calculation part to calculate the accuracy value of each preset similarity threshold according to the preset threshold updating time point, and controls the similarity threshold storage part to store the preset similarity threshold with the highest accuracy value as the similarity threshold which allows the comprehensive similarity comparison and judgment part to serve as the judgment reference.

< example two >

In contrast to the first embodiment, in the second embodiment, the same reference numerals are given to the constituent elements having the same structures as those in the first embodiment, and the corresponding descriptions are omitted.

Fig. 6 is a block diagram of a face recognition apparatus according to a second embodiment of the present invention.

As shown in fig. 6, in the present embodiment, the face recognition apparatus 200 is an apparatus for performing face recognition based on a plurality of captured face images, and specifically includes a reference image storage unit 1, an imaging unit 2, a similarity calculation unit 3, an angle deviation calculation unit 4, a comprehensive similarity calculation unit 5, a highest comprehensive similarity acquisition unit 26, an identification information acquisition unit 27, an information determination unit 28, a sharpness calculation unit 29, a comprehensive sharpness calculation unit 30, a comprehensive similarity comparison determination unit 6, a recognition target determination unit 7, a screen storage unit 8, an output display unit 9, an output display control unit 10, a similarity determination unit 11, a current time acquisition unit 12, an update recording unit 13, an update interval determination unit 14, a reference similarity calculation unit 15, a reference similarity determination unit 16, a reference image update unit 17, a reference image deletion unit 18, an update control unit 19, a preset threshold storage unit 20, and a computer program, A threshold accuracy calculation unit 21, a similarity threshold storage unit 22, a threshold update control unit 23, a communication unit 24, and a control unit 25.

The image pickup part 2 is configured to acquire a plurality of face images of the recognition object in the current state for a plurality of times to obtain a plurality of current face images, and an operation flow of each face image acquisition is the same as that in the first embodiment, which is not described herein again. In this embodiment, the image capturing unit 2 performs five times of face acquisition image acquisition on the recognition target in the current state to obtain 5 current face images.

For each acquired current face image, the similarity calculation unit 3 is configured to calculate similarities between the current face image and each group of reference images, and the angle deviation calculation unit 4 is configured to calculate an angle deviation between the current face image and each group of reference images.

The highest integrated similarity obtaining unit 26 is configured to obtain a highest integrated similarity from the multiple integrated similarities calculated by the integrated similarity calculating unit 5, and the highest integrated similarity is used as the highest integrated similarity corresponding to each current face image.

The identity information obtaining part 27 is configured to obtain corresponding identity information according to the highest comprehensive similarity corresponding to each current face image.

The information judging section 28 is for judging whether or not a plurality of pieces of identification information, which correspond to the plurality of current face images respectively and are acquired by the identification information acquiring section 27, correspond to different recognition objects.

The sharpness calculation unit 29 is configured to perform image analysis on each of the plurality of current face images to obtain a plurality of sharpness degrees when the plurality of pieces of identification information acquired by the identification information acquisition unit 27 correspond to different identification objects.

The integrated sharpness calculation unit 30 is configured to calculate respective integrated sharpness for different recognition targets, and includes sharpness coefficient storage means, sharpness coefficient acquisition means, and integrated accumulation means.

The definition coefficient storage unit stores a plurality of image definitions and corresponding definition coefficients.

And the definition coefficient acquisition unit sequentially retrieves and acquires the corresponding definition coefficients from the definition coefficient storage unit according to the definition of each current face image.

And the comprehensive accumulation unit sequentially accumulates the definition coefficients of the current face image corresponding to each different recognition object to obtain the comprehensive definition of each recognition object. For example, the number of the current face images is 5, wherein two pieces of identity information correspond to the staff a, the definition coefficients are 0.1 and 0.3, the other three pieces of identity information correspond to the staff B, and the definitions are 0.1, 0.3 and 0.2, so that the comprehensive definition corresponding to the staff a is 0.4, and the comprehensive definition corresponding to the staff B is 0.6.

The recognition object determination unit 7 is configured to determine the corresponding recognition object as the current recognition object based on the highest integrated resolution.

The screen storage unit 8 stores a plurality of identity information display screens 8b and stranger prompt screens 8a, and the configurations of the identity information display screens 8b and the stranger prompt screens 8a are the same as those in the first embodiment, and will not be described herein again.

The output display unit 9 is used for displaying the screen.

The output display control part 10 is used for controlling the output display part 9 to display the above-mentioned picture, and the specific implementation process is the same as that in the first embodiment, and will not be described again here.

The communication unit 24 exchanges data between the respective components of the face recognition apparatus 200.

The control unit 25 controls the operations of the respective components of the face recognition apparatus 200.

The face recognition apparatus 200 of this embodiment collects a plurality of current face images, then performs similarity analysis on each current face image and each group of reference face images, and calculates the corresponding highest comprehensive similarity, and by determining whether the plurality of highest comprehensive similarities correspond to different identity information, when corresponding to different identity information, respectively calculates the comprehensive sharpness of the current face image corresponding to the same identity information, and uses the current face image corresponding to the final comprehensive sharpness as the current face image corresponding to the current recognition object.

As shown in fig. 7, in the second embodiment, the operation flow of the face recognition apparatus 200 includes the following steps:

in step S2-1, the camera unit 2 acquires a plurality of face images of the recognition object in the current state, and then proceeds to step S2-2.

In step S2-2, the control unit 25 controls the similarity calculation unit 3, the angle deviation calculation unit 4, and the integrated similarity calculation unit 5 to perform corresponding operations on a plurality of current face images to obtain integrated similarities between each of the current face images and each of the sets of reference face images, and then the process proceeds to step S2-3.

In step S2-3, the control unit 25 controls the highest integrated similarity obtaining unit 26 to obtain the highest integrated similarity according to the multiple integrated similarities of each current face image, as the highest integrated similarity corresponding to each current face image, and then proceeds to step S2-4.

In step S2-4, the control unit 25 controls the identity information obtaining unit 27 to obtain corresponding identity information according to the highest comprehensive similarity corresponding to each current face image for a plurality of current face images, and then proceeds to step S2-5.

In step S2-5, the control section 25 controls the information judging section 28 to judge whether or not the plurality of pieces of identification information, which correspond to the plurality of current face images respectively and are acquired by the identification information acquiring section 27, correspond to different recognition objects, and proceeds to step S2-6 when judged yes, and proceeds to step S2-10 when judged no.

In step S2-6, the sharpness calculation unit 29 performs image analysis on each of the plurality of current face images to obtain a plurality of sharpness values, and then proceeds to step S2-7.

In step S2-7, the sharpness coefficient obtaining unit retrieves and obtains the corresponding sharpness coefficient from the sharpness coefficient storage unit according to the sharpness of each current face image in sequence, and then the step S2-8 is performed.

And S2-8, the comprehensive accumulation unit accumulates the definition coefficients of the current face image corresponding to each different recognition object in sequence to obtain the comprehensive definition of each recognition object, and then the step S2-9 is carried out.

In step S2-9, the recognition object determining unit 7 determines the corresponding recognition object as the current recognition object according to the highest integrated resolution, and then proceeds to step S2-10.

In step S2-10, the integrated similarity comparison and determination unit 6 compares the highest value of the highest integrated similarity corresponding to the current recognition target as the current highest integrated similarity with a predetermined similarity threshold, determines whether or not the current highest integrated similarity is equal to or greater than the similarity threshold, and if yes, proceeds to step S2-11, and if no, proceeds to step S2-13.

S2-11, the identification object decision unit 7 decides the corresponding identification information as the identification information of the current identification object based on the highest integrated similarity, and then proceeds to step S2-12.

In step S2-12, the output display control unit 10 controls the output display unit 9 to display the identification information display screen 8b corresponding to the identification information of the current recognition target, and then enters the end state.

In step S2-13, the identification object determining unit 7 determines that the current identification object is a stranger, and then proceeds to step S2-14.

In step S2-14, the output display control unit 10 controls the output display unit 9 to display the stranger prompt screen 8a, and then enters the end state.

In the above process, the specific process of acquiring a plurality of current face images each time in the step S2-1 is the same as the step S1-1-1 to the step S1-1-2 in the first embodiment; the specific process of the integrated similarity between the current face image and each group of reference face images in step S2-2 is the same as that of step S1-1-3 to step S1-1-5 in the first embodiment, and is not described herein again.

In this embodiment, the update control unit 19 controls the similarity determination unit 11, the current time acquisition unit 12, the update recording unit 13, the update interval determination unit 14, the reference similarity calculation unit 15, the reference similarity determination unit 16, the reference image update unit 17, the reference image storage unit 1, and the reference image deletion unit 18 to update the current reference face image group in the reference image storage unit by using the current face image with the highest comprehensive similarity as the face image to be updated, which is the same as in the first embodiment and will not be described again.

Effects and effects of example two

According to the face recognition device and the face recognition method of the second embodiment, the camera part acquires a plurality of current face images for a plurality of times, and the control part controls the similarity calculation part, the angle deviation calculation part, the comprehensive similarity calculation part and the highest comprehensive similarity acquisition part to perform corresponding operations for each current face image, so that the highest comprehensive similarity of each current face image is obtained and serves as the corresponding highest comprehensive similarity; further, the identity information acquiring part acquires corresponding identity information, the control information judging part judges whether the plurality of identity information correspond to different identification objects, when the face recognition target is judged to correspond to different recognition targets, the definition calculating part can respectively perform image analysis on a plurality of current face images to obtain a plurality of definitions, the comprehensive definition calculating part calculates the comprehensive definitions corresponding to the different recognition targets according to the plurality of definitions, the recognition target judging part judges the recognition target corresponding to the highest comprehensive definition as the current recognition target, therefore, the second embodiment not only has the same effect as the first embodiment, but also can avoid the situation that the collected current face image is unclear and the recognition error occurs, the accuracy and the efficiency of face recognition are improved, and the time spent on face recognition of the recognition object is reduced.

The above embodiments are preferred examples of the present invention, and are not intended to limit the scope of the present invention.

Claims

1. A face recognition apparatus for recognizing a face of a recognition object, comprising:

a reference image storage unit; an image pickup unit; a similarity calculation unit; an angle deviation calculating section; a comprehensive similarity calculation unit; a comprehensive similarity comparison and judgment unit; and a recognition object judging section for judging the recognition object,

wherein the reference image storage unit stores a plurality of sets of reference face images corresponding to a plurality of recognition objects, respectively, each set of the reference face images including a reference front face image, a reference left face image, a reference right face image, a reference upward-looking face image, and a reference downward-looking face image of the recognition object,

the camera shooting part acquires a face image of the recognition object in the current state to obtain a current face image,

the similarity calculation section compares the current face image with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference downward-looking face image in each of the reference face images in turn to obtain a plurality of front face similarities, left side similarities, right side similarities, upward-looking similarities, and downward-looking similarities corresponding to each of the reference face images,

the angle deviation calculation section compares the current face image sequentially with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference upward-looking face image in each of the sets of reference face images, respectively, to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward-looking deviation angles, and upward-looking deviation angles corresponding to each of the sets of reference face images, respectively,

the integrated similarity calculation section calculates an integrated similarity between the plurality of current face images and each group of the reference face images according to a predetermined calculation rule based on the front face similarity, the left side similarity, the right side similarity, the upward view similarity, the downward view similarity of each group and the corresponding front face deviation angle, the left side deviation angle, the right side deviation angle, the upward view deviation angle, and the downward view deviation angle of each group,

the integrated similarity comparison determination unit compares the highest integrated similarity serving as a highest integrated similarity with a predetermined similarity threshold value to determine whether or not the highest integrated similarity is equal to or greater than the similarity threshold value,

and when the highest comprehensive similarity is larger than or equal to the similarity threshold, the identification object judgment part judges corresponding identity information as the current identity information of the identification object according to the highest comprehensive similarity.

2. The face recognition apparatus of claim 1, further comprising:

an update recording unit, an update control unit, a similarity determination unit, a current time acquisition unit, an update interval determination unit, a reference similarity calculation unit, a reference similarity determination unit, and a reference image update unit,

wherein the update recording unit stores an update time for each set of the reference face images,

once the identification object judging part judges the corresponding identity information according to the highest comprehensive similarity of the current facial image, the updating control part takes a corresponding group of the reference facial images as a current reference facial image group, controls the similarity judging part to judge whether the highest comprehensive similarity between the current facial image and the current reference facial image group is between 80 and 90 percent,

when the current time is 80-90%, the update control part controls the current time obtaining part to obtain the current time, and controls the update interval judging part to judge whether the interval between the latest update time corresponding to the current reference face image group stored in the update recording part and the current time is in a preset time interval,

when it is judged to be greater than the reference similarity, the update control section controls the reference similarity calculation section to calculate current reference similarities between the current face image and each of the reference face images in the current reference face image group and to calculate a plurality of reference similarities between each pair of the reference face images in the current reference face image group, respectively, and further controls the reference similarity judgment section to judge whether or not the minimum value of the current reference similarities is smaller than the average value of the reference similarities,

and when the current face image is judged to be smaller than the reference face image, the updating control part controls the reference image updating part to update the corresponding reference face image stored in the reference image storage part according to the current face image, and controls the updating recording part to correspondingly record the updating time of the group of reference face images.

3. The face recognition apparatus of claim 2, further comprising:

a reference image deleting unit for deleting the reference image,

wherein each of the reference face images in the reference image storage section includes at least one reference front face image, at least one reference left face image, at least one reference right face image, at least one reference upward face image, and at least one reference downward face image,

the reference image update unit includes an orientation determination unit and a number determination unit,

the orientation determination unit determines the face orientation of the current face image as a current face orientation according to the front face deviation angle, the left side deviation angle, the right side deviation angle, the elevation deviation angle and the plan deviation angle corresponding to a current reference face image group;

the number judgment unit judges whether the number of the reference face images oriented in the same direction as the current face in the current reference face image group is smaller than a predetermined number,

when the face orientation is judged to be smaller than the reference face orientation, the update control unit controls the reference image storage unit to additionally store the new reference face image and the storage time thereof by using the current face image as a new reference face image corresponding to the current face orientation and the current time as the storage time of the new reference face image,

when the face orientation is determined to be not less than the face orientation, the update control unit controls the reference image storage unit to additionally store the new reference face image and the storage time thereof, and controls the reference image deleting unit to delete the reference face image which is associated with the current face orientation and has the earliest storage time in the reference image storage unit, by using the current face image as a new reference face image corresponding to the current face orientation and the current time as the storage time of the new reference face image.

4. The face recognition apparatus of claim 1, wherein:

wherein the predetermined calculation rule includes a similarity coefficient assignment rule and a comprehensive similarity calculation rule,

the similarity coefficient assignment rule is as follows:

first, the face deviation angle, the left deviation angle, the right deviation angle, the elevation deviation angle, and the plan deviation angle corresponding to each set of the reference face image are respectively expressed as (x) in the form of coordinates₀,y₀)、(x₁,y₁)、(x₂,y₂)、(x₃,y₃) And (x)₄,y₄) Thereby calculating the front face deviationThe deviation degrees of the difference angle, the left deviation angle, the right deviation angle, the upward deviation angle and the downward deviation angle are respectively

And

then, adding a fixed value 1e-4 to a group of deviation degrees and calculating the reciprocal to obtain a corresponding group of sequence values

And

finally, a group of similarity coefficients corresponding to the corresponding front face similarity, the left side similarity, the right side similarity, the upward view similarity and the downward view similarity are obtained through calculation and are respectively

And

wherein s ═ w₀+w₁+w₂+w₃+w₄，

And the comprehensive similarity calculation rule is that the front face similarity, the left side similarity, the right side similarity, the looking-up similarity and the looking-down similarity are multiplied by the corresponding similarity coefficients respectively and then added to obtain the comprehensive similarity.

5. The face recognition apparatus of claim 1, wherein:

wherein the camera shooting part comprises a camera, a camera shooting control unit, a real face judging unit and an output unit,

once the camera shoots a face image, the camera shooting control unit controls the real face judgment unit to judge whether the face image is a real face or not, and further controls the output unit to output the face image as the current face image when the face image is judged to be the real face.

6. The face recognition apparatus of claim 1, wherein:

wherein the face orientation of the reference front face image is a front side, the face orientation of the reference left side face image is a left side, the face orientation of the reference right side face image is a right side, the face orientation of the reference upward-looking face image is an upper side, the face orientation of the reference downward-looking face image is a lower side,

the angular deviation of the face orientation in the reference left face image from the face orientation of the reference front face image and the angular deviation of the face orientation of the reference right face image from the face orientation of the reference front face image are 10 ° -30 °,

the angular deviation of the face orientation in the reference look-up face image from the face orientation of the reference front face image and the angular deviation of the face orientation of the reference look-up face image from the face orientation of the reference front face image are 10 ° -20 °.

7. The face recognition apparatus of claim 1, further comprising:

a preset threshold value storage part, a threshold value updating control part, a threshold value accuracy calculation part and a similarity threshold value storage part,

wherein the preset threshold storage part stores a plurality of preset similarity thresholds,

the threshold updating control section controls the threshold accuracy calculation section to calculate an accuracy value of each of the preset similarity thresholds based on an accuracy value calculation rule for the preset similarity thresholds and the plurality of sets of reference face images stored in the reference image storage section according to a preset threshold updating time point, and controls the similarity threshold storage section to store the preset similarity threshold having the highest accuracy value as the similarity threshold for which the comprehensive similarity comparison judgment section serves as a judgment reference.

8. The face recognition apparatus of claim 7, wherein:

wherein the accurate value calculation rule is as follows:

firstly, traversing all the reference face images in the reference image storage part, pairwise matching the reference face images belonging to the same group to form a plurality of same-group pairs, pairwise matching the reference face images not belonging to the same group to form a plurality of different-group pairs,

then, calculating the similarity between the two reference facial images of each same-group pair and each different-group pair,

wherein a is₁Is the logarithm of the pairings with a similarity above the preset similarity threshold, a₂Is the logarithm of the heterogeneous pair with a similarity below the preset similarity threshold, a₃Is the logarithmic sum of the logarithm of the same pair and the logarithm of the different pair.

9. A face recognition apparatus for recognizing a face of a recognition object, comprising:

a reference image storage unit; an image pickup unit; a control unit; a similarity calculation unit; an angle deviation calculating section; a comprehensive similarity calculation unit; a highest integrated similarity acquisition unit; an identity information acquisition unit; an information determination unit; a definition calculating section; a comprehensive definition calculating section; and a recognition object judging section for judging the recognition object,

the camera shooting part acquires a plurality of face images of the recognition object in the current state for a plurality of times to obtain a plurality of current face images,

the control unit controls the similarity calculation unit to compare the current face image with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference downward-looking face image in each group of the reference face images in sequence to obtain a plurality of front face similarities, left side similarities, right side similarities, upward-looking similarities, and downward-looking similarities corresponding to the reference face images in each group; controlling the angle deviation calculation unit to sequentially compare the current face image with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference downward-looking face image in each of the sets of reference face images, respectively, to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward-looking deviation angles, and downward-looking deviation angles corresponding to the respective sets of reference face images; controlling the comprehensive similarity calculation part to calculate a plurality of comprehensive similarities according to a preset calculation rule and according to the front face similarity, the left side similarity, the right side similarity, the looking-up similarity, the looking-down similarity of each group and the corresponding front face deviation angle, the left deviation angle, the right deviation angle, the looking-up deviation angle and the looking-down deviation angle; controlling the highest comprehensive similarity obtaining part to obtain the highest comprehensive similarity as the highest comprehensive similarity according to the multiple comprehensive similarities; further controlling the identity information acquisition part to acquire corresponding identity information according to the highest comprehensive similarity,

the control section further controls the information judging section to judge whether or not a plurality of the identification information which correspond to the plurality of current face images respectively and which are acquired by the identification information acquiring section correspond to different recognition objects,

when the face images are judged to correspond to different recognition objects, the definition calculating part respectively carries out image analysis on the current face images to obtain a plurality of definitions,

the integrated definition calculating section calculates integrated definitions corresponding to the different recognition objects based on the plurality of definitions,

the identification object determination unit determines that the corresponding identification object is the current identification object based on the highest integrated resolution.

10. The face recognition apparatus of claim 9, wherein:

wherein the integrated definition calculating section includes:

a definition coefficient storage unit which stores a plurality of image definitions and corresponding definition coefficients;

the definition coefficient acquisition unit retrieves and acquires a corresponding definition coefficient from the definition coefficient storage unit according to the definition of each current face image in sequence;

and the comprehensive accumulation unit is used for sequentially accumulating the definition coefficients of the current face image corresponding to each different recognition object to obtain the comprehensive definition of each recognition object.

11. A face recognition method for recognizing a face of a recognition object, comprising the steps of:

a reference image storage step of acquiring and storing a plurality of sets of reference face images corresponding to a plurality of recognition objects, respectively, each set of reference face image including a reference front face image, a reference left face image, a reference right face image, a reference upward-looking face image, and a reference downward-looking face image of the recognition object;

a shooting step, in which a face image of the recognition object in the current state is acquired to obtain a current face image;

a similarity calculation step of sequentially comparing the current face image with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference downward-looking face image in each group of the reference face images to obtain a plurality of front face similarities, left side similarities, right side similarities, upward-looking similarities, and downward-looking similarities corresponding to each group of the reference face images;

an angle deviation calculation step of sequentially comparing the current face image with the reference front face image, the reference left face image, the reference right face image, the reference upward-looking face image, and the reference downward-looking face image in each of the sets of reference face images to obtain a plurality of front face deviation angles, left side deviation angles, right side deviation angles, upward-looking deviation angles, and downward-looking deviation angles respectively corresponding to each of the sets of reference face images;

a comprehensive similarity calculation step of calculating, according to a predetermined calculation rule, a comprehensive similarity between the plurality of current face images and each group of reference face images according to the front face similarity, the left side similarity, the right side similarity, the upward view similarity, the downward view similarity of each group, and the corresponding front face deviation angle, the left side deviation angle, the right side deviation angle, the upward view deviation angle, and the downward view deviation angle of each group;

a comprehensive similarity comparison and judgment step, wherein the highest comprehensive similarity is used as the highest comprehensive similarity to be compared with a preset similarity threshold value, and whether the highest comprehensive similarity is more than or equal to the similarity threshold value is judged;

and an identification object judgment step, namely judging corresponding identity information as the current identification object identity information according to the highest comprehensive similarity when the highest comprehensive similarity is larger than or equal to the similarity threshold.