KR101491832B1 - Apparatus and method for selecting image - Google Patents

Abstract

Disclosed are a device and a method for selecting an image, capable of: selecting a non-rotated face image in multiple face images by determining whether the face image is rotated to improve face recognition performance when a face is recognized using the face image. The system of the present invention comprises: an input unit to input multiple face images; a detection unit to detect a face box region in each of the face images, to select a face region corresponding to a region of interest (ROI) in the detected face box region, and to detect left and right eyes in the selected face region; a separation unit to separate the selected face region into left and right face regions based on a central vertical line and to symmetrically convert one of the separated left and right face regions based on the central vertical line; a feature extraction unit to extract histogram feature vectors and an average of pixel values of the left and right face regions; a similarity calculation unit to calculate multiple similarity values between the left and right face regions using at least two of the histogram feature vectors, the average of the pixel values, pixels of the face region, and the detected left and right eyes to determine whether the face image is rotated or not; a score calculation unit to calculate a similarity score of the face image with a calculated output value by inputting the similarity values into a Fuzzy logic; and a selection unit to select a non-rotated face image in the face images using the similarity score.

Description

[0001] Apparatus and method for selecting images [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image recognition technology, and more particularly, to an image selection apparatus and method.

Biometrics technology applied to a system for existing user authentication is often a method using inherent biometric feature information that can be identified from a human body. In particular, face recognition technology is one of the most commonly used biometrics technologies for identification of people. It is used for authentication of personal identity through password or official certificate due to recent development of smartphone. However, the facial recognition technology used at this time has a problem in that face recognition performance deteriorates due to a change in the rotation of the facial region detected by a person's head shape and facial pose.

In the present invention, in order to improve the face recognition performance in face recognition using a face image, it is determined whether or not the face image is rotated in a plurality of face images and the non-rotated face images are selected.

According to an aspect of the present invention, an image selection apparatus is disclosed.

An image selection apparatus according to an embodiment of the present invention includes an input unit for receiving a plurality of face images, a face box region detecting unit for detecting a face box region in the face image, and detecting a face region corresponding to a region of interest (ROI) A face detection unit for detecting left and right eyes in the selected face region, a face detection unit for separating the selected face region into left and right face regions based on a central vertical line, and one of the separated left and right face regions as a center vertical line A feature extraction unit for extracting a histogram feature vector and an average of pixel values of the left face region and the right face region, a feature extraction unit for extracting an average of the histogram feature vector, , The pixels of the face region, and the detected left and right eyes, A similarity calculation unit for calculating a plurality of similarity values between right and left face regions, a score calculation unit for calculating a similarity score of the face image with an output value calculated by inputting the plurality of similarity values into fuzzy logic, And a selection unit for selecting a non-rotated face image among the plurality of face images using the selected non-rotated face image.

The detecting unit selects the face region by applying the region of interest from which information that may disturb the determination of symmetry in the face box region is removed.

The feature extraction unit divides the face region into one or more blocks of a predetermined size, extracts a histogram feature vector by applying a local binary pattern (LBP) scheme to each block, and combines the respective histogram feature vectors.

The similarity calculation unit may calculate a similarity score based on a distance between the histogram feature vectors extracted from the right and left face regions, a difference between an average of pixel values extracted from the left and right face regions, a pixel difference between the left and right face regions, And calculates the plurality of similarity values by calculating at least two of the angles formed by the central vertical lines.

The plurality of similarity values are normalized between 0 and 1 to apply to the fuzzy logic.

The smaller the distance between the histogram feature vectors is, the closer the face image is to the non-rotated face image.

The smaller the difference of the average of the pixel values is, the closer the face image is to the non-rotated face image.

The smaller the pixel difference is, the closer the face image is to the non-rotated face image.

As the angle approaches 90 degrees, the face image approaches the non-rotated face image.

According to another aspect of the present invention, an image selection method performed by an image selection device to perform face recognition with a plurality of face images is disclosed.

A method for selecting an image according to an exemplary embodiment of the present invention includes receiving a plurality of face images, detecting a face box area in the face image, detecting a face region corresponding to a region of interest (ROI) Selecting right and left eyes in the selected face area, separating the selected face area into right and left face areas with reference to a central vertical line, selecting one of the separated left and right face areas as a center vertical line Extracting an average of histogram feature vectors and pixel values of the left facial region and the right facial region, calculating an average of the histogram feature vector, the average of the pixel values, A pixel of the face region, and at least two of the detected left and right eyes, Calculating a plurality of similarity values between right and left face regions, calculating the similarity score of the face image with the output value calculated by inputting the plurality of similarity values into fuzzy logic, And selecting the non-rotated face image from among the plurality of face images.

The step of selecting the face region may include selecting the face region by applying the region of interest to which information that may obstruct the determination of symmetry in the face box region is removed.

The extracting of the histogram feature vector and the average of the pixel values may include dividing the face region into one or more predetermined size blocks and applying a local binary pattern (LBP) method to each block to extract a histogram feature vector And combining each of the histogram feature vectors.

The calculating of the plurality of similarity values may include calculating a distance between the histogram feature vectors extracted from the left and right face regions, a difference between an average of pixel values extracted from the left and right face regions, a pixel difference between the left and right face regions, And calculates the plurality of similarity values by calculating at least two of the angles formed by the line segments connecting the left and right eyes and the central vertical line.

The plurality of similarity values are normalized between 0 and 1 to apply to the fuzzy logic.

The smaller the distance between the histogram feature vectors is, the closer the face image is to the non-rotated face image.

The smaller the difference of the average of the pixel values is, the closer the face image is to the non-rotated face image.

The smaller the pixel difference is, the closer the face image is to the non-rotated face image.

As the angle approaches 90 degrees, the face image approaches the non-rotated face image.

The present invention can improve the performance of face recognition in face recognition using a face image by determining whether a face image is rotated in a plurality of face images and selecting a non-rotated face image.

1 schematically illustrates a configuration of an image selecting apparatus;
2 is a flowchart illustrating an image selection method performed by the image selection device of FIG.
Fig. 3 illustrates a detected face box area. Fig.
FIG. 4 illustrates applying a region of interest to a face box area; FIG.
5 is a diagram illustrating a process of determining a symmetry of a face area;
6 is a diagram illustrating calculation of the degree of similarity using detected left and right eyes;
7 is a diagram illustrating a setting diagram according to fuzzy logic;
8 is a conceptual diagram illustrating a process of acquiring a non-rotated face image among a plurality of face images.

While the present invention has been described in connection with certain exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and similarities. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. In addition, numerals used in the description of the present invention are merely an identifier for distinguishing one component from another.

Also, in this specification, when an element is referred to as being "connected" or "connected" with another element, the element may be directly connected or directly connected to the other element, It should be understood that, unless an opposite description is present, it may be connected or connected via another element in the middle.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate a thorough understanding of the present invention, the same reference numerals are used for the same means regardless of the number of the drawings.

1 is a view schematically illustrating a configuration of an image selecting apparatus.

1, an image selection apparatus includes an input unit 10, a detection unit 20, a separation unit 30, a feature extraction unit 40, a similarity calculation unit 50, a score calculation unit 60, 70).

The input unit 10 receives a plurality of face images.

The detecting unit 20 detects a face box area in a face image, selects a face area corresponding to a region of interest (ROI) in the detected face box area, and detects right and left eyes in the selected face area.

Here, the face box area means a rectangular area estimated to be a face among the face images. For example, the detection unit 20 may detect the face box area using the Adaboost face detection method. Referring to FIG. 3, FIG. 3 illustrates a detected face box area. As shown in FIG. 3, the detected face box area can be divided into a non-rotated face box area 210 and rotated face box areas 220 and 230. In this case, the in-plane rotation means that the face is rotated clockwise or counterclockwise around the nose of the face instead of turning the face to the left or right or up and down.

Referring to FIG. 4, FIG. 4 illustrates applying a region of interest to a face box region. 4, the detection unit 20 detects an area of interest in which the background 330 is included in the detected face box area 310, information that hinders the determination of symmetry such as hair, ear, etc., The face area 320 can be selected.

Then, the detection unit 20 can detect left and right eyes by applying Adaboost eye detection in the selected face region. Adaboost detection method is an algorithm used to find various body parts such as face, nose, eye, hand, foot, ear.

The separating unit 30 separates the face region into left and right face regions with reference to the central vertical line, and symmetrically converts one of the separated left and right face regions with reference to the central vertical line. For example, referring to FIG. 4, the separator 30 separates the face region 320 into a left face region 340 and a right face region 360 with reference to a central vertical line Can be separated. 5, FIG. 5 is a view illustrating a process of determining the symmetry of a face region. 5, the separating unit 30 can make the left face region 340 and the right face region 360 similar by symmetrically changing the right face region 350 with respect to the central vertical line.

The feature extraction unit 40 extracts a histogram feature vector of each face area and an average of pixel values. That is, the feature extraction unit 40 divides the face region into one or more blocks of a predetermined size, extracts a histogram feature vector by applying an LBP (Local Binary Pattern) method to each block, and combines each histogram feature vector do. Then, the feature extraction unit 40 may extract the pixel values of the face region or the brightness values of the pixels of the face region, and calculate an average from the extracted values.

The similarity calculation unit 50 calculates a similarity value between the left and right face regions using the detected feature to determine whether or not the face image rotates. That is, the similarity calculation unit 50 calculates the similarity between the histogram feature vector extracted from the right and left face regions, the average of the pixel values extracted from the left and right face regions, the pixel difference between the left and right face regions, And the angle formed by the central vertical line, and calculates a plurality of similarity values. Here, the calculated similarity values are normalized from 0 to 1 in order to apply to the fuzzy logic. At this time, the normalization of the similarity value is performed using the maximum value or the minimum value of the similarity value. For example, the smaller the similarity value is, the smaller the difference between the left and right face regions is. Therefore, the closer the similarity value is, the closer the face image can be to the non-rotated face image. Thus, the normalization of the similarity value can be performed based on the minimum value of the similarity value. Accordingly, as the normalized similarity value approaches 0, the facial image can be judged to be a non-rotated facial image, and conversely, the closer the normalized similarity value to 1, the more the facial image can be judged to be the rotated facial image.

As shown in FIG. 5, the similarity degree calculating unit 50 may calculate the distance between the histogram feature vectors extracted from the left and right face regions as the similarity value. For example, referring back to FIG. 3, since the rotated face box regions 220 and 230 are difficult to find the similarities of the left and right face regions, the distances between the histogram feature vectors must be large and the non-rotated face box regions 210 ) Are similar to the left and right face regions, the distance between the histogram feature vectors must be small. Here, the distance between the histogram feature vectors can be calculated using Chi-square distance, Bhattacharyya distance, and the like.

The similarity calculation unit 50 may calculate the difference of the average of the pixel values as the similarity value by comparing the average of the pixel values extracted from the left and right face regions. The smaller the difference, the closer the face image can be to the non-rotated face image.

Then, the similarity calculating unit 50 may calculate the pixel difference as the similarity value by comparing the pixel differences of the left and right face regions. The smaller the difference, the closer the face image can be to the non-rotated face image.

6, the degree-of-similarity calculating unit 50 may calculate an angle 510 formed by a line segment connecting the detected right and left eyes and a central vertical line as a similarity value. Here, FIG. 6 is a diagram illustrating the calculation of the degree of similarity using the detected left and right eyes. Referring to FIG. 6, the closer the angle 510 is to 90 degrees, the closer the face image is to the non-rotated face image. In this case, the similarity value calculated by the angle 510 may be normalized to 0 to 1 after being converted into a 90-degree-θ format in order to unify the input value format of the fuzzy logic. Here,? Is an angle (510) formed by a line segment connecting the right and left eyes and a central vertical line.

The score calculation unit 60 calculates the similarity score of each face image using fuzzy logic. In other words, the score calculation unit 40 calculates a similarity score of each face image with the output value calculated by inputting a plurality of similarity values into the fuzzy logic. Here, the similarity score is determined using a non-deterministic probability set based on the set range.

7 is a diagram illustrating a setting diagram according to fuzzy logic. Hereinafter, the setting range and the setting degree according to the fuzzy logic will be described with reference to FIG.

The setting chart shows the setting range of the input and output. For example, in the case of setting as shown in FIG. 7, the input is set to "2", which is composed of "small" and "large" Step configuration diagram. If input 1 is 0.2 and input 2 is 0.8, then input 1 is "small, 0.8", "large, 0.2" and assuming that it has a setting range of "small" 2 has a setting of "large, 0.8", "small, 0.2". 0.8 "," large "," 0.8 ", {" small, 0.8 "," small ", 0.2"}, {"large, 0.2" "Large, 0.8"}, {"large, 0.2", "small, 0.2"}.

The score calculating unit 60 can calculate the similarity score of the corresponding face image according to a preset fuzzy logic table based on the lower set degree among a plurality of inputs. For example, the fuzzy logic table can be represented as shown in Table 1 below.

Input 1 Input 2 Input 3 Input 4 Print littleness littleness littleness littleness littleness greatness littleness greatness littleness littleness greatness usually greatness littleness littleness littleness greatness usually greatness littleness usually greatness greatness greatness littleness littleness littleness littleness greatness usually greatness littleness usually greatness greatness greatness littleness littleness usually greatness greatness greatness littleness greatness greatness greatness

According to Table 1, when all the inputs 1 to 4 are "small ", the output can be designated as" small ".

For example, if multiple inputs are applied to the fuzzy logic, the IF-THEN rule can be used. Here, the IF-THEN rule specifies the output range in consideration of the relationship between the inputs, and can be divided into an AND operation and an OR operation. The AND operation is to perform an operation such as " if input 1 is X and input 2 is Y, the range of the output value is Z ", and the OR operation is performed in such a manner that "when input 1 is X or input 2 is Y, Quot; is Z ". In this specification, an AND operation is applied. For example, when four inputs are used, the number of input bundles is sixteen, so that the output value can be determined accordingly. In the determination of the output value, in each bundle, a MIN method of selecting a smaller value among the four input values is used. Finally, any of the methods of center of gravity, first of maximum, last of maximum, middle of maximum, or mean of maxima may be used as a non-fuzzy method to deduce the final result from the 16 input sets . The similarity score can be calculated by being defuzzified in the selected method.

The selecting unit 50 selects a non-rotated face image among a plurality of face images using the similarity score of each face image. For example, FIG. 8 is a conceptual diagram illustrating a process of acquiring a non-rotated face image among a plurality of face images. Referring to FIG. 8, the selector 50 may calculate a weight value of each face image using the similarity score of each face image, and may calculate a weight value using the following equation.

[Equation 1]

Here, x represents an arbitrary xth face image in M face images.

8, if it is assumed that the weight of the first face image is 0.5, the weight of the second face image is 0.3, and the weight of the third face image is 0.1, the weight calculated on the matching score according to the face recognition result And the non-rotated face image can be selected.

2 is a flowchart illustrating an image selection method performed by the image selection device of FIG.

In step S211, the image selection device receives a plurality of face images.

In step S212, the image selection device detects a face box area in the face image.

In step S213, the image selection device selects a face area corresponding to a region of interest (ROI) in the detected face box area.

In step S214, the image selecting device detects right and left eyes in the selected face area.

In step S215, the image selecting device separates the face area into left and right face areas with respect to the central vertical line.

In step S216, the image selection device performs a symmetric conversion on one of the separated left and right face regions with reference to the central vertical line.

In step S217, the image selection device extracts the histogram feature vector of each face area and the average of the pixel values. That is, the image selection apparatus divides the face region into one or more blocks of a predetermined size, applies a local binary pattern (LBP) method to each block, extracts the histogram feature vectors, and combines the respective histogram feature vectors. The image selection device can extract the pixel values of the face area or the brightness values of the pixels of the face area, and calculate an average from the extracted values.

In step S218, the image selection device calculates the similarity value between the left and right face regions using the detected feature to determine whether or not the face image is rotated. That is, the image selecting apparatus includes a histogram feature vector extracted from the left and right face regions, a difference between the average of the pixel values extracted from the left and right face regions, a pixel difference between the left and right face regions, a line segment connecting the detected left and right eyes, And calculates a plurality of similarity values. Here, the calculated similarity values are normalized from 0 to 1 in order to apply to the fuzzy logic. At this time, the normalization of the similarity value is performed using the maximum value or the minimum value of the similarity value. For example, the smaller the similarity value is, the smaller the difference between the left and right face regions is. Therefore, the closer the similarity value is, the closer the face image can be to the non-rotated face image. Thus, the normalization of the similarity value can be performed based on the minimum value of the similarity value. Accordingly, as the normalized similarity value approaches 0, the facial image can be judged to be a non-rotated facial image, and conversely, the closer the normalized similarity value to 1, the more the facial image can be judged to be the rotated facial image.

In step S219, the image selection device calculates the similarity score of each face image using fuzzy logic. That is, the image selection device inputs a plurality of similarity values to the fuzzy logic and calculates the similarity score of each face image with the calculated output value. Here, the similarity score is determined using a non-deterministic probability set based on the set range.

In step S220, the image selecting apparatus selects a non-rotated face image among a plurality of face images using the similarity score of each face image.

Meanwhile, the image selection method according to an exemplary embodiment of the present invention may be implemented in the form of a program command that can be executed through a variety of means for electronically processing information, and may be recorded in a storage medium. The storage medium may include program instructions, data files, data structures, and the like, alone or in combination.

Program instructions to be recorded on the storage medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of software. Examples of storage media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, magneto-optical media and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The above-mentioned medium may also be a transmission medium such as a light or metal wire, wave guide, etc., including a carrier wave for transmitting a signal designating a program command, a data structure and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as devices for processing information electronically using an interpreter or the like, for example, a high-level language code that can be executed by a computer.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

10: Input unit
20:
30:
40: Feature extraction unit
50:
60: score calculation unit
70:

Claims (18)

An input unit for inputting a plurality of face images;
A detecting unit that detects a face box area in the face image, selects a face area corresponding to a region of interest (ROI) in the detected face box area, and detects right and left eyes in the selected face area;
A separator for separating the selected face area into left and right face areas with respect to a center vertical line and symmetrically converting one of the separated left and right face areas with respect to a central vertical line;
A feature extraction unit for extracting an average of histogram feature vectors and pixel values of the left face region and the right face region;
A plurality of similarity values between the left and right face regions are calculated using at least two of the histogram feature vector, the average of the pixel values, the pixels of the face region, and the detected left and right eyes for the presence / absence of rotation of the face image A similarity calculation unit;
A score calculation unit for calculating the similarity score of the face image by inputting the plurality of similarity values into the fuzzy logic and calculating the output value; And
And a selecting unit for selecting a non-rotated face image among the plurality of face images using the similarity score.
The method according to claim 1,
Wherein the detection unit selects the face region by applying the ROI from which information that may obstruct the determination of symmetry in the face box region is removed.
The method according to claim 1,
The feature extraction unit may divide the face region into at least one predetermined size block, apply a local binary pattern (LBP) scheme to each block, extract a histogram feature vector, and combine the respective histogram feature vectors. .
The method according to claim 1,
The similarity calculation unit may calculate a similarity score based on a distance between the histogram feature vectors extracted from the right and left face regions, a difference between an average of pixel values extracted from the left and right face regions, a pixel difference between the left and right face regions, And calculates the plurality of similarity values by calculating at least two of angles formed by the central vertical lines.
5. The method of claim 4,
Wherein the plurality of similarity values are normalized between 0 and 1 for application to the fuzzy logic.
5. The method of claim 4,
Wherein the face image is closer to the non-rotated face image as the distance between the histogram feature vectors becomes smaller.
5. The method of claim 4,
And the face image is closer to the non-rotated face image as the difference of the average of the pixel values is smaller.
5. The method of claim 4,
Wherein the smaller the pixel difference is, the closer the face image is to the non-rotated face image.
5. The method of claim 4,
Wherein the face image is closer to the non-rotated face image as the angle approaches 90 degrees.
An image selection method performed by an image selection device to perform face recognition with a plurality of face images,
Receiving a plurality of face images;
Detecting a face box area in the face image;
Selecting a face region corresponding to a region of interest (ROI) in the detected face box area;
Detecting left and right eyes in the selected face region;
Separating the selected face region into left and right face regions based on a central vertical line;
Symmetrically transforming one of the separated left and right face regions based on a central vertical line;
Extracting an average of histogram feature vectors and pixel values of the left face region and the right face region;
A plurality of similarity values between the left and right face regions are calculated using at least two of the histogram feature vector, the average of the pixel values, the pixels of the face region, and the detected left and right eyes for the presence / absence of rotation of the face image step;
Inputting the plurality of similarity values into fuzzy logic and calculating a similarity score of the face image using the calculated output value; And
And selecting the non-rotated face image from among the plurality of face images using the similarity score.
11. The method of claim 10,
The step of selecting the face region comprises:
And selecting the face region by applying the region of interest from which information that may interfere with the determination of symmetry in the face box region is removed.
11. The method of claim 10,
Wherein the step of extracting the average of the histogram feature vector and the pixel values comprises:
Dividing the face region into one or more blocks of predetermined size; And
Extracting a histogram feature vector by applying an LBP (Local Binary Pattern) method to each block, and combining each histogram feature vector.
11. The method of claim 10,
The step of calculating a plurality of similarity values includes:
A distance between the histogram feature vectors extracted from the left and right face regions, an average difference between pixel values extracted from the right and left face regions, a pixel difference between the right and left face regions, and a line segment connecting the detected left and right eyes, And calculating the plurality of similarity values by calculating at least two of the degrees of similarity.
14. The method of claim 13,
Wherein the plurality of similarity values are normalized between 0 and 1 for application to the fuzzy logic.
14. The method of claim 13,
Wherein the face image is closer to the non-rotated face image as the distance between the histogram feature vectors becomes smaller.
14. The method of claim 13,
Wherein the face image is closer to the non-rotated face image as the difference of the average of the pixel values is smaller.
14. The method of claim 13,
Wherein the smaller the pixel difference is, the closer the face image is to the non-rotated face image.
14. The method of claim 13,
Wherein the face image is closer to the non-rotated face image as the angle approaches 90 degrees.

Images