KR101484003B1 - Evaluating system for face analysis - Google Patents

Abstract

The present invention relates to a facial analysis evaluation system. The facial analysis evaluation system according to an embodiment of the present invention includes a data input device which configures and outputs facial image data based on input facial images; a face detection device which detects the face areas in the facial image data to output the face areas; a code generation device which generates and outputs codes by using code generation algorithms selected by a user; a feature expression device which expresses the facial images as facial feature vectors to output feature data; and a classification device outputting the facial recognition result.

Description

{EVALUATING SYSTEM FOR FACE ANALYSIS}

The present invention relates to a face analysis evaluation system, and more particularly, to a face analysis evaluation system capable of accurately comparing and evaluating the performance of each algorithm used in face analysis under the same environmental conditions.

Recently, facial image analysis techniques such as face recognition and facial expression recognition have been increasingly applied. The application field of facial image analysis is actively researching because it differentiates products and induces users' interest and purchase. These facial image analysis methods are classified into a geometric feature based method and an appearance feature based method.

The geometric feature method is a method of expressing the shape of the face and the change of the facial expression by using the relationship between the positions of the eyes, nose, mouth, lips, eyebrows, and the components of the face. This geometric feature method can easily represent facial features as a few factors and can easily cope with changes in position, size, and azimuth of each component, but it is very difficult to accurately detect facial components .

Appearance feature base method is a method of expressing facial image features on whole or local facial images. At this time, the image is classified into a holistic image based approach and a local image based approach according to the size of the image used to express the feature of the facial image.

The global feature-based method is a method of representing each pixel constituting a facial image as one feature element and displaying the entire image as one feature vector. Since the global feature-based method is based on the entire image without a separate feature extraction process, the image processing method is simple. However, the feature of the facial image appearing on a part of the facial image can not be accepted The face recognition performance is deteriorated.

On the other hand, the regional feature-based method sets a local window in a region where a change can occur so as to accommodate the characteristics of a fine facial image, extracts features based on the brightness distribution in the window, It is a method of constructing a vector. Since the feature - based method accepts features of facial image change, it shows stable facial image analysis result with higher recognition rate than global feature based method.

In general, the feature-based face analysis method performs a face image input process, a face image feature extraction process, a feature vector generation process, a learning process, and a test process. This outline-based feature face analysis outputs various results according to the data input every process, feature extraction algorithm, vector generation algorithm, learning and test method.

Most appearance-based features facial analysis methods use facial images obtained from a known face database or use self-generated facial images. And the feature based feature analysis method shows the facial image analysis result using its own algorithm for feature extraction, vector generation and learning / testing.

The facial image analysis method of the feature based feature extracts the image analysis result implemented by the own algorithm and the image analysis result implemented by the other type of algorithm, and outputs an incorrect comparison result. Therefore, in order to obtain an accurate comparison result of facial image analysis implemented with different algorithms, the facial image analysis method of the outline-based feature may be implemented through various verification procedures or a facial image analysis method corresponding to any one of the algorithm methods Should be.

However, since facial image analysis system based on outline based features is implemented by using its own algorithm instead of standardized algorithm, it is difficult to reuse and transfer the algorithm used in each process for facial image analysis.

In addition, although various environmental changes must be considered in order to accurately compare and evaluate the facial image analysis method, facial image analysis results implemented by the own algorithm have the problem that accurate comparison evaluation is practically impossible because the same environmental change is applied.

That is, it is impossible to confirm which algorithm is used in each process of the facial image analysis method, the optimum performance condition in which predetermined algorithms perform optimum performance, and the environmental condition in optimum performance.

When the first facial analysis system and the second facial analysis system exist, the first facial analysis system derives the facial analysis result using the first environmental condition and the first algorithm, and the second facial analysis system uses the second environmental condition And the second algorithm can be used to derive the facial analysis results.

In this case, since the first environmental condition and the second environmental condition are not the same, and the first algorithm and the second algorithm are implemented under different environmental conditions to derive the respective facial analysis results, impossible.

Therefore, a face analysis system to which various algorithms are applied needs an evaluation system that can accurately compare and evaluate the analysis results of facial images in consideration of environmental changes.

The present invention is to compare and evaluate the performance difference of each algorithm used in facial image analysis using the feature feature-based method under the same environmental conditions, thereby making it possible to select an optimal facial analysis algorithm according to the environment, And provides a facial analysis evaluation system capable of deriving recognition results.

Among the embodiments, the face analysis evaluation system includes a data input device that receives a face image used for analyzing a face image and constructs and outputs face image data; A face detection device that stores at least one face detection algorithm, receives face image data, detects a face area in the face image data using a face detection algorithm selected by a user among the face detection algorithms, and outputs the detected face area; A code generation device that stores at least one code generation algorithm, receives face image data from the face detection device and generates and outputs a code using a code generation algorithm selected by a user among the code generation algorithms; A facial feature extraction unit that extracts facial feature vectors from the facial feature vector by using at least one facial feature expression algorithm that uses local or global information, A feature expressing device for outputting data; And storing at least one learning / classifying algorithm, storing the feature data as learning data in the case of the learning algorithm, comparing the feature data with the previously stored learning data in the case of the classification algorithm, And a sorting device for outputting the classified data.

In one embodiment, the face analysis evaluation system includes at least one feature selection algorithm that is located between the feature expression device and the classification device, stores at least one feature selection algorithm, and uses the feature selection algorithm selected by the user to convert the feature data into low dimensional And a characteristic reduction device for reducing the feature data or the probability-based feature data.

At this time, the face image data includes at least one of horizontal and vertical resolution of the image, image buffer information, image data type, color order information, and the feature data includes at least one of dimension information, Or more.

Wherein the data input device comprises: a face database module including at least one database for storing face images used for face image analysis; A file storage module for storing the face image in a file format; An external input module for inputting facial images transmitted from outside in real time; And an image selection module for selecting a face image input through any one of the face data module, the file storage module, and the external input module and outputting the selected face image as face image data.

Wherein the face detection apparatus comprises: an algorithm selection module storing the at least one face detection algorithm and selecting a specific face detection algorithm by a user; And a face detection engine for detecting a face area from face image data received from the data input device by the selected face detection algorithm.

The code generation device comprising: a code selection module storing the at least one code generation algorithm and selecting a specific code generation algorithm by a user; And a code calculation engine for calculating a code for constructing an outline feature vector from the face image data input from the face detection device by the selected code generation algorithm.

Wherein the feature expression apparatus comprises an algorithm selection module storing at least one face feature expression algorithm using the regional or global information and selecting a specific face feature expression algorithm by a user; And a feature calculating module for expressing the face image data using a facial feature vector using the code input from the code generating device by the selected facial feature indicating algorithm to output feature data.

The feature reduction apparatus may further include a feature extraction unit that extracts feature data input from the feature expression apparatus by using either a probability-based feature analysis method or a principal component analysis feature selection algorithm to reduce the feature data to low-dimensional feature data or probability- And a reduction engine.

The classification device comprises: a learning database for storing learning data; A classifier selection module storing at least one learning / classifying algorithm, and wherein a classifying algorithm specific to the user is selected; And a classifier engine for comparing the feature data with the learning data stored in the learning database using the selected classification algorithm when the classification algorithm is performed, thereby outputting a face recognition result.

The facial analysis evaluation system of the present invention selects a face analysis algorithm that implements optimal performance according to environmental conditions by comparing and evaluating performance differences between algorithms of each process used in facial image analysis process using the feature feature based method under the same environmental conditions Therefore, it is possible to obtain optimal face analysis and face recognition result.

1 is a block diagram illustrating the entire configuration of a face analysis evaluation system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating the configuration of the face analysis evaluation system of FIG. 1 in detail.

The description of the present invention is merely an example for structural or functional explanation, and the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, the embodiments are to be construed as being variously embodied and having various forms, so that the scope of the present invention should be understood to include equivalents capable of realizing technical ideas. Also, the purpose or effect of the present invention should not be construed as limiting the scope of the present invention, since it does not mean that a specific embodiment should include all or only such effect.

Meanwhile, the meaning of the terms described in the present invention should be understood as follows.

The terms "first "," second ", and the like are intended to distinguish one element from another, and the scope of the right should not be limited by these terms. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected to the other element, but there may be other elements in between. On the other hand, when an element is referred to as being "directly connected" to another element, it should be understood that there are no other elements in between. On the other hand, other expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

It should be understood that the singular " include "or" have "are to be construed as including a stated feature, number, step, operation, component, It is to be understood that the combination is intended to specify that it does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used predefined terms should be interpreted to be consistent with the meanings in the context of the related art and can not be interpreted as having ideal or overly formal meaning unless explicitly defined in the present invention.

1 is a block diagram illustrating the entire configuration of a face analysis evaluation system according to an embodiment of the present invention.

1, the face analysis evaluation system 100 includes a data input device 110, a face detection device 120, a code generation device 130, a feature expression device 140, a feature reduction device 150, Device 160 as shown in FIG.

The data input device 110 receives facial images used for facial image analysis in the form of a face database, a real-time image, a facial image file, etc., and outputs facial image data.

The face detection apparatus 120 stores at least one face detection algorithm, receives facial image data from the data input apparatus 110, and extracts facial image data from facial image data using a face detection algorithm selected by the user, And outputs the detected code to the code generation device 130.

The code generation device 130 stores at least one code generation algorithm, receives facial image data from the face detection device 120, generates codes using a code generation algorithm selected by the user among the code generation algorithms, And outputs it to the device 140.

The feature expression unit 140 stores at least one facial feature expression algorithm using local or global information, receives code from the code generation unit 130, and uses a facial feature expression algorithm selected by the user in the facial feature expression algorithm And outputs the facial image data to the feature reducing apparatus 150 by expressing facial image data as a facial feature vector.

The feature reduction device 150 stores at least one feature selection algorithm and reduces the feature data to low-dimensional feature data or probability-based feature data using the feature selection algorithm selected by the user, Output.

The classifying device 160 stores at least one learning / classifying algorithm, stores the characteristic data as learning data when performing the learning algorithm, compares the characteristic data with pre-stored learning data when performing the classification algorithm The face recognition result is output. The classifying device 160 outputs learning data when performing a learning algorithm, and outputs a face recognition result when the classifying device 160 is a test.

FIG. 2 is a block diagram illustrating the configuration of the face analysis evaluation system of FIG. 1 in detail.

Referring to FIG. 2, the data input device 110 includes a face database module 111, a file storage module 112, an external input module 113, and an image selection module 114.

The face database module 111 includes a known Cohn-Kanade face database, a JAFFE face database, an extended Cohn-Kanade face database, an MMI (Multi-Machine Interface) database, a CMU You can import face images from any of the PIE (Pose, Illumination, and Expression) databases. At this time, the face database module 111 may add a known face database according to the user's needs.

The file storage module 112 stores facial images to be analyzed in a file form in a computer or an analyzing device implementing the facial analysis evaluation system, an external computer connected to the facial analysis evaluation system through an interface, or an analyzing device.

The external input module 113 inputs a face image transmitted through an external device such as a camera device in real time.

The image selection module 114 selects a face image input through the face database module 111, the file storage module 112 and the external input module 113 and outputs the face image data to the face detection device 120 Output.

The face detection apparatus 120 includes an algorithm selection module 121 and a face detection engine 122.

The algorithm selection module 121 stores a face detection algorithm such as a face detection algorithm using PCA (Principal Component Analysis), a Viola-Jones algorithm, a forward feature selection face detection algorithm, So that a specific face detection algorithm is selected.

PCA is a statistical method that reduces computational complexity by reducing high-dimensional data with low computational complexity and difficult processing.

The face detection engine module 122 detects the face area from the face image data input from the data input device 110 by the face detection algorithm selected by the algorithm selection module 121, To a resolution suitable for the display device.

The face detection apparatus 120 may store a plurality of face detection algorithms for detecting face regions, and the user may designate one of the plurality of face detection algorithms to use.

The code generation device 130 includes a code selection module 131 and a code calculation engine 132. [

The code selection module 131 stores at least one code generation algorithm such as Local Directional Pattern, Local Binary Pattern, Gabor filter, and SIFT, and a code generation algorithm specific to the user is selected.

The code calculation engine 132 calculates a code for constructing the cosmetic feature vector from the face image data input from the face detection device 120 by the code generation algorithm selected by the code selection module 131. [

The code generation device 130 is designed to be able to register and extend a plurality of code generation algorithms.

The feature expression unit 140 expresses one facial image data as a feature vector, and includes an algorithm selection module 141 and a feature calculation module 142.

The algorithm selection module 141 may include a plurality of face feature expression algorithms expressing a local histogram-based feature vector, a global feature vector, and a superimposed local histogram-based feature vector, as well as a face feature expression At least one algorithm is stored, and a face feature expression algorithm specific to the user is selected.

The feature calculating module 142 expresses facial image data as facial feature vectors using the code input from the code generating device 130 by the facial feature indicating algorithm selected by the algorithm selecting module 141 and outputs the feature data.

If the length of the feature vector input from the feature expression unit 140 is long, the feature reduction unit 150 rapidly increases the amount of operation for classification in the classification unit 160, Dimensional feature data or probability-based feature data by using either the probability-based feature analysis method or the principal component analysis method.

The classification device 160 performs learning and classification, and performs a learning function and a test function. The classification device 160 includes a learning database 161, a classifier selection module 162, and a classifier engine 163.

The learning database 161 stores learning data when performing a learning function.

The classifier selection module 162 stores classifiers such as SVM, Template Matching, Ada Boost, and Dempater-Shafer, stores at least one learning / classifying algorithm for executing the classifiers, and when the classifier is selected by the user To be executed.

The classifier engine 163 compares the feature data with the learning data stored in the learning database 161 using the classifier and classification algorithm selected by the user when performing the classifying function, and outputs the most similar facial recognition result.

Since the facial analysis evaluation system 100 can select and replace an algorithm corresponding to each of the devices 110 to 160, data to be input to and output from each device must be accurately defined in order to maintain mutual compatibility.

Table 1 describes input data and output data for each device of the facial analysis evaluation system. It defines the data used for the input and output of each device to increase the reusability of the algorithm, do.

Device Input data Output data Data input device Face image file path or database file path or real-time image input (CAM) or truth table (input during learning) Face image data Face detection device Face image data Face image data Code generation device Face image data Feature data Feature presentation device Feature data Feature data Feature reduction device Feature data Feature data Classification device Feature data Learning: Accumulated feature data
Test: Unknown recognition rate (%)

In Table 1, the facial image data includes the horizontal and vertical resolutions of the image, the image buffer information including the image information, the image data type, the color order information such as RGB or BGR, and the characteristic data includes one-, Dimensional data such as three-dimensional data, a size of each dimension, and a data type such as a standard data type of C or C ++.

If the user desires to compare the performance difference between the LDP algorithm and the LBP algorithm in the same environment, the data input device 110, the face detection device 120, and the feature expression device 140, except for the code generation device 130, The feature reducing device 150 and the classifying device 160 are identical to each other and the LDP algorithm and the LBP algorithm are selected by the code generating device 130 and then the recognition result or recognition rate output from the classifying device 160 is checked do.

Therefore, the user can accurately compare the performance difference between the LDP algorithm and the LBP algorithm under the same environmental conditions.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

110: data input device 120: face detection device
130: Code generation device 140: Characteristic expression device
150: Feature reduction device 160: Classifier

Claims (9)

A data input device for inputting and outputting facial image data used for facial image analysis;
A face detection device that stores at least one face detection algorithm, receives face image data, detects a face area in the face image data using a face detection algorithm selected by a user among the face detection algorithms, and outputs the detected face area;
A code generation device that stores at least one code generation algorithm, receives face image data from the face detection device and generates and outputs a code using a code generation algorithm selected by a user among the code generation algorithms;
A facial feature extraction unit that extracts facial feature vectors from the facial feature vector by using at least one facial feature expression algorithm that uses local or global information, A feature expressing device for outputting data; And
Storing at least one learning / classifying algorithm, storing the feature data as learning data in the case of the learning algorithm, comparing the feature data with the previously stored learning data in the case of the classification algorithm, and outputting a face recognition result And a face image analyzing system including a classification device for performing face image analysis.
The method according to claim 1,
Wherein the feature extraction device is located between the feature expression device and the classification device and stores at least one feature selection algorithm and uses the feature selection algorithm selected by the user to reduce the feature data into low dimensional feature data or probability based feature data And a feature shrinking device for correcting the feature of the face image.
The method according to claim 1,
Wherein the face image data includes at least one of horizontal and vertical resolution of the image, image buffer information, image data type, color order information,
Wherein the feature data includes at least one of dimension information, size of each dimension, and data type.
The data input apparatus according to claim 1,
A face database module including at least one database for storing face images used for facial image analysis;
A file storage module for storing the face image in a file format;
An external input module for inputting facial images transmitted from outside in real time;
And an image selection module for selecting a face image input through any one of the face data module, the file storage module, and the external input module, and outputting the selected face image as face image data.
The face detection apparatus according to claim 1,
An algorithm selection module storing the at least one face detection algorithm and selecting a specific face detection algorithm by the user; And
And a face detection engine for detecting a face area from face image data received from the data input device by the selected face detection algorithm.
The apparatus according to claim 1,
A code selection module storing the at least one code generation algorithm and selecting a specific code generation algorithm by a user; And
And a code calculation engine for calculating a code for constructing an outline feature vector from the face image data input from the face detection device by the selected code generation algorithm.
The apparatus according to claim 1,
An algorithm selection module that stores at least one face feature expression algorithm using the regional or global information and selects a specific face feature expression algorithm by the user; And
And a feature calculating module for expressing the face image data as a face feature vector by using the code input from the code generating device by the selected face feature expression algorithm and outputting the feature data.
The apparatus according to claim 2,
And a feature reduction engine for reducing feature data input from the feature expression device to low-dimensional feature data or probability-based feature data using any one of a probability-based feature analysis method and a principal component analysis feature selection algorithm Feature analysis system for face analysis.
2. The apparatus according to claim 1,
A learning database for storing learning data;
A classifier selection module storing at least one learning / classifying algorithm, and wherein a classifying algorithm specific to the user is selected; And
And a classifier engine for comparing the feature data with the learning data stored in the learning database using the selected classification algorithm when the classification algorithm is performed to output a face recognition result. .

Images