KR20080104565A - Method and system for recognizing a face by means of an image variation sensing - Google Patents

Abstract

A face recognition method using an image variation sensing function and a system thereof are provided to discriminate the image variation of a photographed picture, thereby increasing the efficiency in the face recognition. A face recognition method using image variation sensing comprises the following steps. A client transmits a photographed picture to a service managing server(S100). The service managing server pre-processes the picture for face recognition(S102), and then extracts a face image based on the photographed picture in order to discriminate face components forming the face image(S104). And then, the service managing server extracts face feature points formed with the face components in order to sense the image variation through the pattern change based on the extracted face feature points(S108), and executes at least one of a face recognition application which uses the face feature points based on the sensed result and a face recognition application which uses the pre-processed image(S110).

Description

METHOD AND SYSTEM FOR RECOGNIZING A FACE BY MEANS OF AN IMAGE VARIATION SENSING}

1 is a block diagram showing a face recognition system using image shift sensing according to an embodiment of the present invention,

2 is a block diagram showing the internal configuration of the service management server shown in Figure 1, and

FIG. 3 is a flowchart illustrating an operation of a face recognition system using image shift sensing shown in FIG. 1.

<Description of Symbols for Main Parts of Drawings>

100: client 200: communication network

300: service management server 310: interface unit

320: image preprocessing unit 330: face region extraction unit

340: Face component discriminator 350: Image shift sensing unit

360 control

The present invention relates to face recognition technology, and more particularly, each face recognition to be applied in response to various image variations such as face pose, lighting, facial expression or occlusion that interfere with face recognition from the captured image. The present invention relates to a face recognition method using an image variation sensing for executing an optimal face recognition application by determining an image variation of a corresponding captured image.

In general, it is very difficult to accurately recognize a face because a human face appears as a very different face image according to a change in lighting, facial expression, age, when the image was acquired, noise, and the like. The core of the face recognition system is to realize stable face recognition performance without being affected by the image variation of the face. The development of a face recognition system with stable performance is a security system or a human computer interface in a company, a laboratory, a bank, etc. It can be very important in the field.

In order to recognize a face, there was a method of using a local geometric feature of a face or applying a template matching, but this method uses a large amount of memory due to the formation of a large number of intermediate images in order to recognize a face shape having a lighting change or a facial expression change. There is a problem that increases and takes a lot of time to calculate. Therefore, in order to reduce the amount of memory and the recognition time, a face image is converted into one feature vector in a vector space, and the feature vector of the face region of the captured image and the face of the recognition target (or a face implemented by a standard template) are used. By comparing feature vectors, a method of recognizing a face has been studied and typically, a facial recognition application (FBA) using a feature point of a face included in a predetermined image and a face region from the captured image There is an Appearance Based Approach (ABA) that uses image data forming a preprocessed image to be converted for extraction. Examples of each of these face recognition applications are the Eigenface method and the Fisherface method.

The eigenface method obtains the eigenvalues and eigenvectors from the variance matrix representing the variance of the data for the intermediate processed image and expresses the face image as the coordinates of the eigenvectors corresponding to the large eigenvalues. The coordinates thus obtained are specific vectors representing one face image. In the actual recognition step, the face is recognized by comparing the feature vector obtained by linearly transforming the face image extracted from the captured image as an input with the feature vectors previously obtained offline. By applying PCA (Principal Component Analysis) on the face image, the dimension of face vector space is effectively reduced, so that the entire face images are compared by comparing only a few feature vectors that have been linearly transformed. Fast has the advantage.

However, in this method, feature vectors obtained from face images with changing lighting or facial expressions with partial occlusion or face images with partial occlusion are not easily separated between classes, resulting in poor recognition rate. There are disadvantages. This is because the eigenface method performs eigenvalue analysis of the variance matrix of the entire face image vectors instead of the variance matrix and inter-class variance matrix in each class.

The Fisher Face method is a method of applying a FLD (Fisher Linear Discrimination) using a matrix representing data distribution between classes and a matrix representing data distribution between classes in addition to a method of recognizing a face by applying PCA. FLD has the advantage of classifying feature vectors relatively accurately by using linear transformation that maximizes class separation between linearly changed feature vectors as well as the effect of dramatically reducing the dimension of face vector space. The Fischer Face method which recognizes a face by applying FLD shows better recognition performance than the eigenface method for the face image with lighting or facial expression change. In order to obtain a matrix representing intra-class dispersion and inter-class dispersion, a large amount of memory and computation time are required, and the recognition rate is still degraded for a face image with severe illumination change or occlusion.

As described above, the face recognition method according to the related art is implemented based on a face recognition application corresponding to any one image variation among various image variations such as face pose, lighting, facial expression, or occlusion. For example, when face recognition is executed based on a face recognition application focused on extracting facial expressions, the face recognition rate is significantly shaken for image variation caused by lighting or occlusion. There was a problem of very poor efficiency.

Accordingly, the present invention was created to solve the above problems, and an object of the present invention is to respond to various image variations such as face poses, lighting, facial expressions, or occlusion that interfere with face recognition from captured images. There is provided a face recognition application for each application, and to determine the image drift for the image taken to provide a face recognition method and system utilizing the image variance sensing for executing the optimal face recognition application.

In order to achieve the above object, the system of the present invention executes a client for transmitting a photographed image formed by photographing a predetermined object through a communication network and image preprocessing for extracting a face region of a person included in the image from the photographed image. Determining a face component from the extracted face region based on the pre-processed image, sensing the image variation that is an obstacle to face recognition through the pattern variation based on the facial feature points formed by the face component, and according to the sensing execution result. And a service management server for executing at least one of a face recognition application using the face feature included in a face component and a face recognition application using image data constituting the preprocessed image.

Face recognition method using the image shift sensing according to the first aspect of the present invention for achieving the above object, (A) the step of delivering the photographed image formed by the client to the service management server, b) the service management Performing image preprocessing for face recognition from the photographed image by the server; c) extracting a face region based on the preprocessed image and determining a facial component constituting the face region; Extracting facial feature points formed of elements and sensing image variations that are obstacles to face recognition through pattern variation based on the facial feature points, (e) facial recognition application using the facial feature points based on the sensing result (FBA) Feature Based Approach and facial recognition application using image data forming the preprocessed image (AB) A; executing at least one of the Appearance Based Approach) and (f) delivering the execution result of the face recognition application to the client.

Preferably, the step (a) is characterized in that the client connects to the service management server through a video communication network.

Preferably, the step (b) is characterized by executing image preprocessing including additional operations for image filtering, color transformation, coordinate transformation, and face recognition for the photographed image.

Preferably, the step (c) extracts the face region through blob analysis through matching comparison between the preprocessed image and the standard image template, and extracts the face region through a ratio operation with respect to the face region. It is characterized by verifying.

Preferably, the step (c) is characterized by determining the face component through projection-based extraction and geometric prediction through matching between the face region and the standard image template.

Preferably, the step (d) includes at least one of face pose, illumination, facial expression, and occlusion as elements of the image shift.

Preferably, in the step (d), if the pattern change amount based on the facial feature point is within a predetermined error range based on the pattern change amount corresponding to the standard image template, the step (c) is set to an image shift caused by the face pose. .

Preferably, the step (e) is to execute a face recognition application using the face feature point, when the image variation with respect to the face pose.

Preferably, the step (d) sets the image variation for the facial expression when the pattern variation of a specific portion of the pattern variation based on the facial feature point is out of a predetermined error range based on a pattern variation corresponding to a standard image template. Characterized in that.

Preferably, the step (e) is to perform a face recognition application using the face feature point, when the image variation with respect to the facial expression.

Preferably, the step (d) is to set the image variation for the illumination when the pattern variation based on the facial feature point is determined to be an intensity variation outside a predetermined error range based on the pattern variation corresponding to the standard template. It features.

Preferably, the step (e) is to execute a face recognition application using the image data constituting the pre-processed image, when the image shift for the illumination.

Preferably, in the step (d), if there is a partial image distortion based on the pattern change amount corresponding to the standard template, the pattern change amount based on the facial feature point is set as an image shift for the occlusion.

Preferably, the step (e) is to execute a face recognition application using image data constituting the pre-processed image and a face recognition application using the face feature point in case of an image shift for the occlusion. .

In addition, the face recognition system using the image shift sensing according to the second aspect of the present invention for achieving the above object, after performing the client and the image pre-processing for the photographed image to transmit the photographed image formed through the communication network Determining a face component from a face region extracted based on a preprocessed image, and sensing an image variation that is an obstacle to face recognition through a pattern variation based on a facial feature point formed by the face component, and according to the sensing result. And a service management server for executing at least one of a face recognition application using the face feature and a face recognition application using image data constituting the preprocessed image.

Preferably, the service management server includes at least one of face pose, lighting, facial expression, and occlusion as an element constituting the video transition.

Preferably, the service management server is set to an image shift due to the face pose when the pattern change amount based on the facial feature point is within a predetermined error range based on a pattern change amount corresponding to a standard image template.

Preferably, the service management server is characterized in that for executing a face recognition application using the facial feature point, if the image variation with respect to the face pose.

Preferably, the service management server is configured to set the image variation for the facial expression when the pattern variation of a specific portion of the pattern variation based on the facial feature point is out of a predetermined error range based on a pattern variation corresponding to a standard image template. It is characterized by.

Preferably, the service management server is characterized in that for running the face recognition application using the facial feature point, if the image variation for the facial expression.

Preferably, the service management server, if it is determined that the pattern change amount based on the facial feature point is a change amount of contrast outside the predetermined error range based on the pattern change amount corresponding to the standard template, it is set to the image shift for the illumination It is done.

Preferably, the service management server is characterized in that when the image shift for the illumination, the face recognition application using the image data forming the pre-processed image.

Preferably, the service management server is configured to set the image variation for the occlusion when there is a partial image distortion based on the pattern variation amount corresponding to the standard template, based on the facial feature point.

Preferably, the service management server is characterized in that when the image shift for the occlusion, the face recognition application using the image data constituting the pre-processed image and the face recognition application using the face feature point.

Hereinafter, a preferred embodiment of a face recognition system using image shift sensing according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a face recognition system using image shift sensing according to an embodiment of the present invention. As shown in FIG. 1, in the face recognition system using image shift sensing, a client 100 and a captured image provided from the client 100 transmit a captured image formed by photographing a user through a communication network 200. Image preprocessing for converting the image into an image for face recognition, extracting a face region based on the converted preprocessed image, determining a face component from the face region, and applying a face feature to the face feature point formed by the face component. Based on the result of simulating a pattern change amount and sensing the image variation that is a obstacle to face recognition through the pattern change amount, a face recognition application (FBA) using the facial feature point and the preprocessed image Face Recognition Application (ABA; Appearance Based Appr) oach) includes a service management server 300 for executing at least one.

The service management server 300 performs image preprocessing on the captured image, removes noise on the corresponding image by filtering the image, and executes additional operations for color conversion, coordinate transformation, and face recognition to facilitate face extraction. It will be processed as an image to make it.

Here, image filtering uses techniques such as edge extraction, Gaussian filtering, Laplacian filtering, Gabor filtering, and canine edge extraction, and color conversion includes RGB conversion, HLS conversion, HSV conversion, YCrCb conversion, YUV conversion, XYZ conversion, I1I2I3 conversion, etc. Use the technology of

For coordinate transformation, techniques such as Cartesian coordinate transformation, polar coordinate transformation, and log-polar coordinate transformation are used. It is preferable that techniques such as logical operation, arithmetic operation, and threshold division are applied to the execution of the additional operation.

Thereafter, the service management server 300 extracts the face region through matching comparison and blob analysis, and extracts the face region from the preprocessed image, and extracts the face region and extracts the face region from the face region. Also run the verification. In other words, the ratio calculation is the ratio of face height and face width, which are calculated based on normal statistics.

More specifically, the matching comparison is a technique such as error sum comparison, correlation coefficient comparison, correlation coefficient 2 comparison, etc., and the blow analysis can be applied to techniques such as maximum blob analysis, labeling for each blob.

In order to extract facial components such as eyes, nose and mouth from the extracted facial regions, each facial component is determined through projection-based extraction and geometric prediction through the matching matching between the extracted facial region and the standard image template. In the matching comparison, techniques such as error sum comparison, correlation coefficient comparison, and correlation coefficient 2 comparison are applied, and the projection-based extraction is applied with the technique of horizontal projection or vertical projection. In addition, geometric prediction is performed through an element ratio for each facial component formed based on the standard image template.

Furthermore, in order to extract feature points in facial components, geometric features, transformation features, statistical features, etc. are extracted, and local and global features are extracted as geometric features, and the transformation features are Fast Fourier Transform and Discrete Fourier. It is a feature formed through the transformation, and the statistical feature is a feature calculated by means of average, variance, and norm.

Thereafter, the service management server 300 senses whether it includes at least one of face pose, lighting, facial expression, and occlusion as an element constituting the image variation. In the sensing process, the face is detected. When the sensing result is formed that the pattern change amount based on the feature point is within a predetermined error range based on the pattern change amount corresponding to the standard image template, it is determined that the image change is caused by the face pose among various image variations.

Accordingly, the service management server 300 executes the face recognition application using the face feature in response to the image shift caused by the face pose.

Alternatively, when the pattern variation of a specific region of the pattern variation based on the facial feature is outside the predetermined error range when compared with the pattern variation corresponding to the standard image template, the specific region may smile, surprise, fear or embarrassment on the face. Due to such factors, the pattern change amount is concentrated compared to other parts, and in this case, it is determined that the image variation is related to facial expression.

Accordingly, the service management server 300 executes a face recognition application using face feature points in response to the image variation caused by facial expression.

In addition, when the pattern variation based on the facial feature point is compared with the pattern variation corresponding to the standard template, the correlation of each pixel is the intensity of intensity throughout the image, and the intensity variation is outside the predetermined error range. In this case, it is determined that there is an image shift for illumination.

Accordingly, the service management server 300 executes the face recognition application using all the image data forming the preprocessed image.

In addition, when there is a partial image distortion based on the standard template and the corresponding pattern variation amount based on the facial feature point, it is determined that the image variation is for the occlusion that the pattern recognition for the partial face component is not performed.

Accordingly, the service management server 300 executes the face recognition application using the image data constituting the pre-processed image and the face recognition application using the facial feature point together.

2 is a block diagram showing the internal configuration of the service management server 300 shown in FIG. As shown in FIG. 2, the service management server 300 processes the image into a face-recognized image by executing an image preprocessing on the interface 310, which receives the captured image from the client 100, and the captured image. The image preprocessing unit 320, a face region extraction unit 330 for extracting a face region based on the preprocessed image, a face component determination unit 340 for determining a face component from the face region, and a face component. A facial recognition application using the facial feature point based on the image variation sensing unit 350 and the sensing result for extracting a facial feature point to be formed and sensing an image variance that is an obstacle to face recognition through the amount of pattern change based on the facial feature point. And a controller for executing at least one of a face recognition application using image data forming the preprocessed image. 360).

Hereinafter, with reference to the accompanying drawings will be described in more detail with respect to the operation of the face recognition system using the image shift sensing according to the present invention.

FIG. 3 is a flowchart illustrating an operation of a face recognition system using image shift sensing shown in FIG. 1. As shown in FIG. 3, in the face recognition method using image shift sensing, the client 100 transmits the captured image generated by photographing the user's face to the service management server 300 through the communication network 200. It proceeds (S100).

Accordingly, the service management server 300 executes image preprocessing for recognizing a user from the received captured image (S102), extracts a face region based on the preprocessed image in step S102 (S104), and the face region. In step S106, a process for determining a face component disposed in each is performed.

Thereafter, the service management server 300 additionally determines a facial feature point constituting the extracted face component (S108), and acts as an obstacle to face recognition through a pattern change amount based on the facial feature point formed in step S108. Sensing is performed to determine what the video shift is (S110).

In step S110, when the pattern change amount based on the facial feature point is within a predetermined error range when compared with the pattern change amount corresponding to the standard image template, the facial feature point is determined to be a face pose adapted to the standard image template. The face recognition application to be used is executed (S114).

In addition, in the step S110, if the pattern change amount of the specific region of the pattern change based on the facial feature point is out of the predetermined error range compared to the pattern change amount corresponding to the standard image template (S116), the facial expression correlates to the specific region. The face recognition application using the facial feature is determined and executed (S118).

In the step S110, if the pattern change amount based on the facial feature points corresponds to the change amount of contrast throughout the image, and the contrast change is out of a predetermined error range compared to the change amount of contrast formed by the standard image template (S120), the image shift by illumination The face recognition application using the image data constituting the pre-processed image is determined as being (S122).

In addition, in the step S124, when it is determined that there is a partial image distortion based on the pattern change amount corresponding to the standard image template in step S124 (S124), the image is preprocessed as being an image variation due to occlusion. Both the face recognition application using the image data constituting the image and the face recognition application using the facial feature point are executed (S126).

Through this process, the service management server 300 executes face recognition for the captured image provided from the client 100 (S128).

Although the above has been described with reference to a preferred embodiment of the present invention, those skilled in the art will be variously modified and changed within the scope of the invention without departing from the spirit and scope of the invention described in the claims below. I can understand that you can.

Accordingly, the present invention is provided with a face recognition application to be applied in response to various image variations such as face pose, lighting, facial expression or occlusion that interfere with face recognition from the captured image, It is possible to increase the face recognition efficiency by utilizing the strengths of both the face recognition application using the facial feature and the face recognition application using the image data forming the preprocessed image by executing the optimal face recognition application by determining the image variation. There is this.

Claims (16)

(A) transmitting the photographed image formed by the client to the service management server; (B) the service management server executing image preprocessing for face recognition from the captured image; (C) extracting a face region based on the preprocessed image and determining a face component constituting the face region; (D) extracting facial feature points formed by the face components and sensing image variation that is an obstacle to face recognition through a pattern change amount based on the facial feature points; (E) executing at least one of a facial recognition application (FBA) using the facial feature point and an facial recognition application (ABA) using image data constituting the preprocessed image based on the sensing result; Doing; And (F) The face recognition method using image variation sensing comprising the step of delivering the execution result of the face recognition application to the client. The method of claim 1, wherein step (a) And the client accesses the service management server through a video communication network. The method of claim 1, wherein step (b) And image preprocessing including image filtering, color transformation, coordinate transformation, and additional operations for face recognition of the photographed image. The method of claim 1, wherein the step (c) The image variation sensing is characterized in that the face region is extracted through blob analysis through matching comparison between the preprocessed image and the standard image template, and the face region is verified through a ratio operation with respect to the face region. Facial recognition method utilized. The method of claim 4, wherein the step (c) The face recognition method using image disparity sensing, characterized in that the face component is determined through a matching comparison between the face region and the standard image template through projection-based extraction and geometric prediction. The method of claim 1, wherein step (d) The image recognition method using the image shift sensing, characterized in that it comprises at least one of facial pose, illumination, facial expression and occlusion as an element constituting the image variation. The method of claim 6, wherein the step (d) And when the pattern variation based on the facial feature is within a predetermined error range based on a standard image template and a pattern variation corresponding to the standard image template, set the image variation due to the face pose. The method of claim 7, wherein the step (e) In the case of the image variation with respect to the face pose, the face recognition method using image variation sensing, characterized in that for executing the face recognition application using the facial feature point. The method of claim 6, wherein the step (d) When the pattern variation of a specific region of the pattern variation based on the facial feature is out of a predetermined error range based on the pattern variation corresponding to the standard image template, the image variation sensing is set as an image variation for the facial expression. One face recognition method. The method of claim 9, wherein step (e) When the image variation of the facial expression, the facial recognition method using image variation sensing, characterized in that for executing the facial recognition application using the facial feature point. The method of claim 6, wherein the step (d) When it is determined that the pattern change amount based on the facial feature point is a change amount of contrast that falls outside a predetermined error range based on the pattern change amount corresponding to the standard template, the image variation sensing face is set as an image variation for the illumination. Recognition method. The method of claim 11, wherein step (e) In the case of the image shift with respect to the illumination, the face recognition method using image shift sensing, characterized in that for executing the face recognition application using the image data forming the pre-processed image. The method of claim 6, wherein the step (d) And a pattern variation based on the facial feature point, when there is a partial image distortion based on a standard template and a pattern variation corresponding to the standard template. The method of claim 13, wherein step (e) And a face recognition application using image data constituting the pre-processed image and a face recognition application using the facial feature point when the image variation is about the occlusion. A client for transmitting a photographed image formed by photographing through a communication network; And After performing the image preprocessing on the photographed image, the face component is determined from the extracted facial region based on the preprocessed image, and the image variation that is an obstacle to face recognition through the pattern variation based on the facial feature points formed by the face component. Senses, And a service management server for executing at least one of a face recognition application using the face feature and a face recognition application using image data constituting the pre-processed image according to the sensing result. Face recognition system. The method of claim 15, wherein the service management server An interface unit receiving the captured image from the client; An image preprocessing unit configured to perform image preprocessing on the captured image; A face region extractor for extracting a face region based on the preprocessed image; A face component discriminating unit for discriminating the face component from the face region; An image variation sensing unit for extracting a facial feature point formed of the face component and sensing an image variation that is an obstacle to face recognition through a pattern change amount based on the facial feature point; And And a control unit configured to execute at least one of a face recognition application using the facial feature point and a face recognition application using image data constituting the pre-processed image based on the sensing result. Recognition system.

Images