CN102867173B - Human face recognition method and system thereof - Google Patents

Abstract

The invention discloses a face recognition method, which has self-learning function and space-time characteristics, and recognizes faces through the steps of face detection, face tracking, data collection and analysis, recognition, online learning, and human-computer interaction. The invention also discloses a system for realizing the above face recognition method, including a detector, a tracker, a collector, an analyzer, an online learning module, a recognizer and a human-computer interaction module. Compared with the prior art, the invention has the advantages of strong anti-interference ability and high identification efficiency.

Description

A face recognition method and system thereof

technical field

The invention relates to human-computer interaction technology, in particular to a face recognition method and system thereof.

Background technique

Human-computer interaction is currently a hot spot in the field of computer science research in the world. In human-computer interaction technology, face recognition is gradually being applied to smart home and other occasions as the most convenient method for computers to identify users and provide personalized services. Face recognition based on computer vision, its core is to use computer vision, image processing and other technologies to process the video sequences collected by image acquisition equipment, and classify users, so as to respond accordingly.

In the existing face recognition technology, the detection of faces, the tracking of detected faces, the extraction of face features from a given image or video sequence and comparison with face database data have been realized respectively. In order to identify the user, here, we call the face recognition method that can combine these three modules together as the traditional face recognition system. However, the current traditional face recognition system cannot integrate some technologies with better effects into modules, so there are the following problems:

(1) The effect is poor. Because the traditional face recognition system cannot effectively integrate the various modules of the face recognition system, the traditional face recognition system is prone to problems such as not being able to track the face, tracking drift, unrecognizable, uncertain or even wrong, which seriously affects people. Face recognition effect.

(2) Poor interactivity. Traditional face recognition systems cannot interact well with people. When the face cannot be recognized, is uncertain or even makes mistakes, the system cannot interact with people well, so when the recognition is inaccurate, the system cannot get feedback from the user in time, and the performance cannot be improved; even when the recognition is inaccurate Sometimes, because the system thinks that its own recognition is correct and uses the recognition result to modify the original parameters in the system (that is, self-learning), the more the system is used, the worse it becomes.

(3) Poor adaptability. Traditional face recognition systems are easily affected by various external conditions such as lighting, beards, glasses, hairstyles, and expressions, which reduce the recognition rate. Therefore, the system is not very practical.

Contents of the invention

In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, the object of the present invention is to provide a face recognition method with self-learning function, space-time characteristics, and strong anti-interference ability.

Another object of the present invention is to provide a face recognition system for realizing the above face recognition method.

The object of the present invention is achieved through the following technical solutions:

A face recognition method, comprising the following steps:

S1 detector detects whether there is a human face in the frame sequence; if so, proceed to step S2, if not, repeat step S1;

The S2 tracker tracks the detected faces;

The S3 collector collects face images;

S4 analyzer analyzes whether the face image collected is a reliable sample; if so, proceed to step S5, if not, repeat steps S2～S4;

The S5 analyzer extracts shape parameters and texture parameters from the target face in reliable samples, models the shape and texture of the target face respectively, and obtains the average face of the target face through model fusion;

The S6 discriminator obtains the matching degree C between the average face and the closest face class in the face class library through the linear discriminant eigenface method;

If C<B, proceed to step S7; if C>A, identify the user identity, and the recognition ends; if B<C<A, the user is required to input a name through the human-computer interaction module, if the face corresponding to the name input by the user If the class exists in the face class storehouse, proceed to step S8; otherwise, proceed to step S7; the values of A and B are determined empirically by the user;

S7 online learning module creates a new face class in the face class library, adds the average face of the target face to the newly created face class and marks the user name, and passes the face class to the recognizer, and proceeds to step S9 ;

S8 online learning module updates the corresponding face class of the name input by the user, and passes the updated face class to the recognizer; proceed to step S9;

The S9 recognizer updates the face class library.

The analyzer in step S5 extracts shape and texture features from the target face in the reliable sample, models the shape and texture of the target face respectively, and obtains the average face of the target face through model fusion, specifically including the following steps:

S5.1 trace the target face in the reliable sample;

S5.2 Modeling the shape of the face: first, perform Procrustes transformation on the face images after the points are drawn, to obtain the average shape of the face, and then obtain the shape parameters and shape model through principal component analysis;

S5.3 Modeling the texture of the face: first delaunay triangulate the average shape of the face, then use the slice affine method to fill the texture, and finally use the principal component analysis method to reduce the dimensionality to obtain the average texture model and texture parameters;

S5.4 Weighted combination of shape parameters and texture parameters, using principal component analysis method to reduce dimensionality to obtain fusion parameters, and finally obtain the average face.

The linear discrimination eigenface method described in step S6 comprises the following steps:

S6.1 Through the inter-class and intra-class nearest neighbor sample algorithm for the average face sent by the analyzer and the face class in the face class library, the measurement of the inter-class and intra-class differences is obtained;

S6.2 Obtain the inter-class scatter matrix and the intra-class scatter matrix according to the inter-class and intra-class differences obtained for each face class;

S6.3 According to the inter-class scatter matrix and the intra-class scatter matrix obtained in step S6.2, the optimal identification vector set is obtained by using the Fisher identification criterion;

S6.4 Project the average face from the analyzer to the optimal discrimination vector set to obtain low-dimensional feature data;

S6.5 According to the nearest neighbor matching principle, obtain the matching degree C between the average face and the closest face class in the face class library.

As described in step S5.1, the target face in the reliable sample is described, specifically:

Draw points on the outline, eyebrows, eyes, nose, and lips of the target face in the reliable sample, and write the coordinates of the points in vector form.

The online learning module described in step S8 updates the corresponding face class of the name of user input, specifically:

The online learning module calculates the difference between the average face passed by the analyzer and all the face samples in the face class corresponding to the name input by the user. If the difference exceeds the intra-class distance, the face class is updated, otherwise it is not. renew.

Realize the face recognition system of above-mentioned face recognition method, comprise detector, tracker, collector, analyzer, online learning module, recognizer and human-computer interaction module, described detector, tracker, collector, analyzer , the recognizer, the human-computer interaction module and the online learning module are connected in sequence; the online learning module is also connected with the recognizer.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

(1) The present invention has self-learning function and space-time characteristics. It recognizes faces by integrating six modules: face detection, face tracking, data collection and analysis, recognition, online learning and supervision, and has strong anti-interference ability.

(2) The present invention has good interactivity. By inquiring the user when the system is uncertain, the next step operation is performed according to the user's feedback, which prevents the system performance from degrading due to various reasons, and the user can use the human-computer interaction module Issue a reset command to the system. At this time, the system deletes the data added during use in the library and restores it to the initialization state, so that the system performance will not decline more and more seriously.

Description of drawings

Fig. 1 is a frame diagram of the face recognition system of the present invention.

Fig. 2 is a flow chart of face recognition in the present invention.

Figure 3 is an example of a destructive sample.

Figure 4 is an example of a reliable sample.

Figure 5 is an example of a human face that has been traced.

Detailed ways

The present invention will be described in further detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

Example

As shown in Figure 1, face recognition system of the present invention comprises detector, tracker, collector, analyzer, online learning module, recognizer and human-computer interaction module, described detector, tracker, collector, analysis The device, the recognizer, the human-computer interaction module and the online learning module are sequentially connected; the online learning module is also connected with the recognizer.

As shown in Figure 2, the face recognition method of the present invention comprises the following steps:

S1 detector detects whether there is a human face in the frame sequence; if so, proceed to step S2, if not, repeat step S1;

The face detection algorithm proposed by Viola and Jones is used in the detector detection process. In this algorithm, Adaboost is used to train the classifier, and the method of multi-pose classification can be used to detect faces quickly with a high detection rate.

The S2 tracker tracks the detected faces;

An improved Camshift combined with Kalman filter algorithm is used in the tracking process. When there is a large area of background interference similar to the target color, start the ROI (region of interest) frame difference method, only perform frame difference calculation on the Kalman prediction area, extract the moving object surface through the edge information of the moving object, and then compare it with the target probability distribution The graph performs an AND operation to filter out the non-moving interference background. When the target is severely occluded, due to the failure of the Camshift algorithm, the Kalman predicted value is used instead of the optimal position value calculated by Camshift, and the Kalman predicted value is used as the observation value updated by the Kalman filter. This algorithm can effectively overcome the severe occlusion caused by the Kalman filter. failure problem. This algorithm can perform real-time tracking based on the faces detected in the detector to ensure that the faces of the same person are in the image area.

The S3 collector collects face images.

S4 analyzer analyzes whether the face image collected is a reliable sample; if so, proceed to step S5, if not, repeat steps S2～S4;

Through reliability analysis, some destructive samples with head rotation angles exceeding 90 degrees (such as the sample shown in Figure 3) are filtered out, and reliable samples that can be used to recognize faces (such as the sample shown in Figure 4) are retained.

The S5 analyzer extracts shape parameters and texture parameters from the target face in reliable samples, models the shape and texture of the target face respectively, and obtains the average face of the target face through model fusion; specifically includes the following steps:

S5.1 Draw 68 points on the outline, eyebrows, eyes, nose, and lips of the target face in the reliable sample (as shown in Figure 5), and write the coordinate positions of the points in vector form;

x={x1,y1,x2,y2,x3,y3,...,x68,y68}.

S5.2 Modeling the shape of the face: first, perform Procrustes transformation on the face images after the points are drawn, to obtain the average shape of the face, and then obtain the shape parameters and shape model through principal component analysis;

S5.3 Modeling the texture of the face: first delaunay triangulate the average shape of the face, then use the slice affine method to fill the texture, and finally use the principal component analysis method to reduce the dimensionality to obtain the average texture model and texture parameters;

S5.4 Weighted combination of shape parameters and texture parameters, using principal component analysis method to reduce dimensionality to obtain fusion parameters, and finally obtain the average face.

The S6 discriminator obtains the matching degree C between the average face and the closest face class in the face class library through the linear discriminant eigenface method;

The linear discriminative eigenface method comprises the following steps:

S6.1 Through the inter-class and intra-class nearest neighbor sample algorithm for the average face sent by the analyzer and the face class in the face class library, the measurement of the inter-class and intra-class differences is obtained;

S6.2 Obtain the inter-class scatter matrix and the intra-class scatter matrix according to the inter-class and intra-class differences obtained for each face class;

S6.3 According to the inter-class scatter matrix and the intra-class scatter matrix obtained in step S6.2, the optimal identification vector set is obtained by using the Fisher identification criterion;

S6.4 Project the average face from the analyzer to the optimal discrimination vector set to obtain low-dimensional feature data;

S6.5 Obtain the matching degree C between the average face and the closest face class in the face class library according to the nearest neighbor matching principle;

If the matching degree C is less than 50%, proceed to step S7; if the matching degree C is greater than 95%, identify the user identity, and the identification ends; if the matching degree C is greater than 50% and less than 95%, the user is required to input a name through the human-computer interaction module , if the face class corresponding to the name input by the user already exists in the face class database, go to step S8; otherwise, go to step S7.

S7 online learning module creates a new face class in the face class library, adds the average face of the target face to the newly created face class and marks the user name, and passes the face class to the recognizer, and proceeds to step S9 .

S8 online learning module updates the corresponding face class of the name input by the user, and passes the updated face class to the recognizer; proceed to step S9;

Described online learning module updates the face class corresponding to the name input by the user, specifically:

The online learning module calculates the difference between the average face passed by the analyzer and all the face samples in the face class corresponding to the name input by the user. If the difference exceeds the intra-class distance, the face class is updated, otherwise it is not. renew.

The S9 recognizer updates the face class library.

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the embodiment, and any other changes, modifications, substitutions and combinations made without departing from the spirit and principle of the present invention , simplification, all should be equivalent replacement methods, and are all included in the protection scope of the present invention.

Claims (5)

1. a face recognition method, is characterized in that, comprises the following steps: S1 detector detects whether there is a human face in the frame sequence; if so, proceed to step S2, if not, repeat step S1; The S2 tracker tracks the detected faces; The S3 collector collects face images; S4 Analyzer analyzes whether the face image collected is a reliable sample; if so, proceed to step S5, if not, repeat steps S2～S4; The S5 analyzer extracts shape parameters and texture parameters from the target face in reliable samples, models the shape and texture of the target face respectively, and obtains the average face of the target face through model fusion, specifically including the following steps: S5.1 trace the target face in the reliable sample; S5.2 Modeling the shape of the face: first, perform Procrustes transformation on the face images after the points are drawn, to obtain the average shape of the face, and then obtain the shape parameters and shape model through principal component analysis; S5.3 Modeling the texture of the face: first delaunay triangulate the average shape of the face, then use the slice affine method to fill the texture, and finally use the principal component analysis method to reduce the dimensionality to obtain the average texture model and texture parameters; S5.4 Weighted combination of shape parameters and texture parameters, using principal component analysis method to reduce dimensionality to obtain fusion parameters, and finally obtain the average face; The S6 discriminator obtains the matching degree C between the average face and the closest face class in the face class library through the linear discriminant eigenface method; If C<B, proceed to step S7; if C>A, identify the user identity, and the recognition ends; if B<C<A, the user is required to input a name through the human-computer interaction module, if the face corresponding to the name input by the user If the class exists in the face class storehouse, proceed to step S8; otherwise, proceed to step S7; the values of A and B are determined empirically by the user; S7 online learning module creates a new face class in the face class library, adds the average face of the target face to the newly created face class and marks the user name, and passes the face class to the recognizer, and proceeds to step S9 ; S8 online learning module updates the corresponding face class of the name input by the user, and passes the updated face class to the recognizer; proceed to step S9; The S9 recognizer updates the face class library. 2. face recognition method according to claim 1, is characterized in that, the described linear discrimination eigenface method of step S6, comprises the following steps: S6.1 Through the inter-class and intra-class nearest neighbor sample algorithm for the average face sent by the analyzer and the face class in the face class library, the measurement of the inter-class and intra-class differences is obtained; S6.2 Obtain the inter-class scatter matrix and the intra-class scatter matrix according to the inter-class and intra-class differences obtained for each face class; S6.3 According to the inter-class scatter matrix and the intra-class scatter matrix obtained in step S6.2, the optimal identification vector set is obtained by using the Fisher identification criterion; S6.4 Project the average face from the analyzer to the optimal discrimination vector set to obtain low-dimensional feature data; S6.5 According to the nearest neighbor matching principle, obtain the matching degree C between the average face and the closest face class in the face class library. 3. The face recognition method according to claim 1, characterized in that, in step S5.1, the target face in the reliable sample is described, specifically: Draw points on the outline, eyebrows, eyes, nose, and lips of the target face in the reliable sample, and write the coordinates of the points in vector form. 4. The face recognition method according to claim 1, wherein the online learning module of step S8 updates the corresponding face class of the name input by the user, specifically: The online learning module calculates the difference between the average face passed by the analyzer and all the face samples in the face class corresponding to the name input by the user. If the difference exceeds the intra-class distance, the face class is updated, otherwise it is not. renew. 5. realize the face recognition system of any one described face recognition method of claim 1～4, it is characterized in that, comprise detector, tracker, collector, analyzer, online learning module, recognizer and man-machine interaction module, the detector, tracker, collector, analyzer, recognizer, human-computer interaction module, and online learning module are connected in sequence; the online learning module is also connected with the recognizer.