KR20180080081A - Method and system for robust face dectection in wild environment based on cnn - Google Patents

Abstract

Provided is an improved method for detecting a face based on a convolutional neural network (CNN) and a system thereof. The face can be accurately and quickly detected by using a fully convolutional network (FCN) of a multi-scale sharing a partial neural network even in a wide environment where a face pose is changed and the face is hidden. Accordingly, the present invention can reduce operation complexity.

Description

FIELD OF THE INVENTION [0001] The present invention relates to a face detection method and system robust to a CNN-based wild environment,

The present invention relates to a method for performing face detection in a wild environment where a pose change and occlusion of a face occurs, and a face detection technique based on a Convolutional Neural Network (CNN).

2. Description of the Related Art [0002] With the advent of various applications using face information, a practical face detection method is becoming more and more popular. The face recognition system is used to protect the privacy of individuals in a security system and a surveillance environment that allows a person to enter and exit. In addition, face recognition is used in the field of interpreting human emotion from the external appearance change by analyzing the face change for the face area. As the area of the application using the face information is expanded and the number of the application is increased, a practical face detection method capable of accurately extracting the face area in various environments is actively researched.

The Viola-Jones method proposed in the 2000s is the first model to show the practical possibility of face detection. The Haar-like feature information is efficiently extracted using the integral image technique and the final face area is selected using the proposed Adaboost serial-connected classifier. However, since this method uses simple feature information, face detection performance is greatly degraded in environments such as face posture change or occlusion. In order to solve this problem, a deformable part model (DPM) has been proposed. This method defines a face area as a combination of geometric positional relationships of facial components. Even if a part of the facial component is lost, the facial region can be determined, so that it exhibits a strong characteristic against attitude change or occlusion. However, determining the degree of matching of a part model to a large number of windows extracted from a sliding window method as well as a primary process of the possibility of existence of each face component involves a great deal of complexity. In addition, in order to learn such a part model, a large-scale database including an accurate label of each part is required.

Recently, a learning-based convolutional neural network (CNN) method has achieved great results in various computer vision fields. CNN's face detection method has made great progress in detection performance, but the increased complexity of the system makes it questionable. The number of windows that can be extracted from an image of 320 × 240 amounts to one billion. For a large number of patches, feature information is extracted based on CNN, and classified into non-face and face regions. This indicates a trade off relationship between face detection performance and system complexity. In addition, unnecessary operations are included by repeatedly performing the convolution operation on the intersecting area between adjacent windows, and the input and output of the fully-connected layer of the concurrent neural network are fixed All input data passing through the neural network is subjected to a process of resizing input data to a fixed size, thereby increasing the computational complexity of the system.

One embodiment excludes the process of resizing input data to a fixed size by using a fully concatenated network (FCN) without an Fully-Connected layer at the input, It is possible to provide a face detection method with reduced complexity.

One embodiment of the present invention provides a face detection method for accurately detecting a face region by adding a classfication process for determining whether a face is included or not and a regression process through a face boundary regression process have.

In one embodiment, the complexity is lowered by performing a composite product operation using a common feature map of a plurality of layers, and the pooling layers have strides of different sizes to thereby detect faces of various sizes. Can be provided.

A face detection method according to an embodiment of the present invention includes a plurality of heat maps representing a face component included in an image, the plurality of heat maps being generated by applying different composite products or pooling methods to the images Extracting a plurality of face candidate regions different from each other; And detecting a face region included in the image based on the plurality of different face candidate regions.

Wherein the step of extracting the plurality of different face candidate regions comprises the step of extracting the image through a first layer comprising at least one first composite product layer performing a composite product and at least one first pooling layer performing a pooling, Maps into maps; And converting the feature maps into heat maps through each of a plurality of second layers including at least one second composite product layer and at least one second pooling layer to extract the plurality of different face candidate regions, Wherein the plurality of second layers may use the feature maps in common to convert the feature maps into the heat maps.

Further, the step of converting the feature maps into heat maps comprises successively performing a composite product and a pooling operation on each of the plurality of second layers to convert feature maps into heat maps, A layer included in one of the plurality of second layers and a layer corresponding to the operation order of the layer included in the other second layer may have strides of different sizes.

The step of detecting the face region may include classifying the presence or absence of a face by determining whether the face region exists in the face candidate region; And a step of regression to a precise face candidate region based on the classification and the face candidate regions.

In addition, the classifying step may include a step of presenting a probability 1 if a face area is included and a step of presenting a probability 0 if the face area is not included.

In addition, the step of regressing to the precise face candidate region may include: providing position information of the face region if the face region is classified as the face region in the classifying step; And a step of assigning a label to the user to ignore the message.

The step of detecting the face region may include detecting the face region through at least one of a plurality of composite product layers, at least one pooling layer, and at least one Fully-Connected Layer (FCL) .

Wherein the extracting of the face candidate regions comprises: learning for distinguishing a face region and a non-face region based on a neural network model that distinguishes an object region and an object region; And learning face candidate region extraction using an image database including at least one facial landmark.

The step of detecting the face region may be learned by using the face candidate regions as a database through a negative sample mining technique.

A machine learning based face detection system in accordance with an embodiment of the present invention includes a plurality of heat maps representing a face component included in an image, the plurality of heat maps applying different synthesis products and pooling schemes A proposal unit for extracting a plurality of different face candidate regions from each other and proposing to the detection unit; And a detection unit for detecting a face region included in the image based on the plurality of different face candidate regions.

The proposal section includes a first layer for transforming the image into feature maps through at least one first convolution layer performing a convolution and at least one first pulling layer performing a pulling; And a plurality of second layers converting the feature maps into heat maps through at least one second convolution layer and at least one second pooling layer to extract the plurality of different face candidate regions, The plurality of second layers may commonly use the feature maps to convert the feature maps into the heat maps.

The proposing unit consecutively performs a composite product and a pooling operation on each of the plurality of second layers to convert feature maps into heat maps, and further includes a layer including one of the plurality of second layers, , And a layer corresponding to the operation order of the layer included in the second layer of the other layer may have strides of different sizes.

Wherein the detecting unit comprises: a classifying unit for classifying the face candidate by determining whether or not the face exists in the face candidate region; And a regression unit for regression of the classification and the face candidate regions based on the face candidate regions.

Furthermore, the classifier may present a probability of 1 if the face region is included, and a probability of 0 if the face region is not included.

Further, the regression unit may provide the position information of the face region if the classification unit classifies the face region, and may label the position information of the face region to be ignored if the classification region is classified as not having the face region.

The detecting unit may include at least one collection product layer, at least one pooling layer, and at least one Fully-Connected layer (FCL) to detect the face area.

One embodiment excludes the process of resizing input data to a fixed size by using a fully concatenated network (FCN) without an Fully-Connected layer at the input, The complexity can be lowered.

In one embodiment, the facial region can be precisely detected by adding a classfication process for determining whether a face is included or not and a regression process using a face boundary regression method.

In an embodiment, a plurality of layers may be optimized to reduce complexity by performing a composite product operation using a common feature map, and pooling layers may have strides of different sizes to detect faces of various sizes.

1 is a view for explaining a face detection process according to an embodiment of the present invention.
2 is a diagram for explaining machine learning of a proposed network according to an embodiment of the present invention.
3 is a diagram for explaining a proposed network according to an embodiment of the present invention.
4 is a diagram for explaining a detection network according to an embodiment of the present invention.
5 is a flowchart of a face detection method according to an embodiment of the present invention.
6 is a block diagram of a face detection system according to an embodiment of the present invention.

Hereinafter, specific embodiments among various embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, this specific embodiment does not limit or limit the invention. The same reference numerals denote the same elements regardless of the reference numerals in the drawings, and a duplicate description will be omitted.

1 is a view for explaining a face detection process according to an embodiment of the present invention.

Referring to FIG. 1, an embodiment of the present invention may provide a network for detecting a face region in a target image 100. The network according to an exemplary embodiment of the present invention is divided into a proposal network 110 and a detection network 120.

When the target image 100 is input to the proposal network 110, the first layer 130 converts the target image 100 into feature maps. A feature map is an image generated through a composite product and a pooling operation on a target image.

The first feature map is input to the plurality of second layers 141, 142, and 143, and each of the plurality of second layers converts the first feature map into the heat maps 150 through different methods. The heat map 150 is a probability map generated by a product multiply and a pulling operation on the feature map.

The probability map is a mapping of each pixel value to a probability value of a face, and is used to grasp a region where a face exists.

The first layer 130 and the plurality of second layers 141, 142, and 143 may include convolutional layers or pooling layers.

The product multiply layer may have a kernel containing learned weights and biases and user defined strides and may perform a composite multiply operation on the target image 100 or the feature map .

The pooling layer can perform a pooling operation with a stride of a size defined by the user. The pooling operation may be max-pooling or average-pooling.

The proposal network 110 extracts 160 regions from the heat maps generated through the plurality of second layers 141, 142, and 143, that is, n face candidate regions 170, (120).

Each of the plurality of second layers 141, 142, and 143 successively performs a composite product operation or a pooling operation. The size of the stride included in the specific layer included in the second layer may be different from the size of the stride included in the second layer and correspond to the operation order of the specific layer.

Since the size of the stride is different for each second layer, it is possible to generate the optimized heat map for the faces of various sizes, so that the face candidate region 170 having high accuracy can be generated, and ultimately, The face detection performance can be improved.

The face candidate region 170 extracted from the proposal network 110 has a high face detection performance per se but the recall rate can be improved by passing through the detection network 120 for the extracted face candidate region 170. [ And can contribute to higher face detection performance by reducing false-positive.

When the face candidate region 170 is input to the detection network 120, it is converted into the one-dimensional data 191 through the composite product hierarchy 180 and the complete connection hierarchy 190, We present n values (192) that can classify if the face is included. The detection network 120 may include a pooling layer in addition to the composite product layer 180 and the full connection layer 190 to generate the one-dimensional data 191.

In addition, the detection network 120 regresses the face position to a more accurate face position than the face position determined in the face candidate region 170, thereby presenting the coordinate value 192 for the precise face candidate region. In order to display a box for a precise face candidate area, the coordinate value may be 4n values including x coordinate, y coordinate, width, and height. An algorithm for returning to a precise face candidate region is described in D. Wang, J. Ynag, and Q. Liu, "Hierarchical Convolutional Neural Network for Face Detection," Proceeding of International Conference on Image and Graphics, pp. 373-384, < / RTI > 2015. It may be a face bound regression.

The generated one-dimensional data 191 can represent the face region more accurately than the face candidate region 170 and present a final face region through post processing of 5n values. The post-processing may be Non-Maximum Suppression (NMS).

2 is a diagram for explaining machine learning of the proposed network 110 according to an embodiment of the present invention.

In the proposed network 110, each of the convolution product layers may perform a composite product operation on the target image 100 or the feature map using a kernel including a weight.

In order to create a feature map in which the face area becomes more prominent, a weight value must be determined so that a face component can be revealed. As with the embodiment of the present invention, the in-depth neural network structure includes a large-scale parameter. Therefore, if the initial weight is set to the Gaussian distribution, it is difficult to interpret the information for localizing the facial components using the location information of the facial landmark to which the label is attached. That is, there is a problem that it is difficult to generate a feature map in which face components are highlighted.

To overcome this, through transfer learning, weights that have the property of localizing the components can be used as initial weights for an embodiment of the present invention.

A facial landmark is a point marked on a feature part of a face and can be displayed on the eyes, nose, mouth, ear, and the like. Localization is the process by which facial components are made more visible than others.

As shown in FIG. 2, an embodiment of the present invention provides a method of adjusting the weight of a network 201 having a characteristic of localizing a face component of a cat to a weight of a network 202 having a characteristic of localizing a face component of a person It can be used as an initial weight.

After initializing the weights from the transition learning, we can learn to localize the human face components. A. Krizhevsky, I. Sutskever, and G.E. The structure of AlexNet, which is introduced in Hinton, "Imagenet Classification with Deep Convolutional Neural Networks," Proceedings of Advances in Neural Information Processing Systems, pp. 1097-1105, It is a network with characteristics that distinguish background areas.

With this basic structure, since the weight for distinguishing the face area and the background area can be easily obtained, the network 202 having the characteristic of localizing the face components can be advantageously constructed.

The AlexNet structure may include five concatenation product layers 221, 222, and 225, three pooling layers 231, 232, and 233, and three fully-connected layers. In the last complete connection layer, the position coordinates 250 of the sequentially connected face candidate feature points can be generated. In one embodiment, the position coordinates 250 of the sequentially connected face candidate feature points may include position coordinates of a plurality of facial feature points indicating left eye, right eye, nose, and mouth, respectively.

The weights can be changed by minimizing the loss function by comparing the position coordinates 250 of the sequentially connected face candidate feature points with the position coordinates of the face feature points existing in the actual image. That is, the learning can be performed to grasp the position of the facial feature point existing in the target image through the loss function as described below.

 In one embodiment, a loss function that minimizes the Euclidean distance between the position coordinates 250 of the sequentially connected face candidate feature points and the position coordinates of the facial feature points existing in the real image is defined as follows .

는 미니 배치(mini-batch)의 크기를 의미하며,

은 얼굴 특징점의 총 개수,

은 순차 연결된 얼굴 후보 특징점의 위치 좌표(250),

은 실제 이미지에 존재하는 얼굴 특징점의 위치 좌표이다. 얼굴 특징점의 집합은

의 벡터 형태로 정의될 수 있다.From here,

Refers to the size of the mini-batch,

The total number of facial feature points,

The position coordinates 250 of the sequentially connected face candidate feature points,

Is the position coordinate of the facial feature point existing in the actual image. The set of facial feature points

As shown in FIG.

One embodiment may use 6, 6, 9, and 20 facial feature points to localize the right eye, left eye, nose, and mouth of a total of 41 facial feature points.

, 가속도(momentum)의

에에 대해 매 세대(epoch) 수마다 학습 속도에

의 값을 곱할 수 있다. 완전 연결 계층의 드롭아웃(dropout)의 확률 값은 0.5일 수 있다.One embodiment may use a stochastic gradient descent method to minimize the loss function of equation (1). The Caffe library is available and the initial learning rate is

, Momentum of

The number of epochs per epoch

Can be multiplied. The probability of a dropout in the full connection layer may be 0.5.

3 is a diagram for explaining a proposed network according to an embodiment of the present invention.

As shown in FIG. 3, the weight or bias obtained through the learning disclosed in the embodiment of FIG. 2 may be copied 340 and used as a weight or a bias of each layer of the proposed network.

In one embodiment, the first layer 130 of the proposed network includes two concatenation layers 221 and 222 and a pooling layer 231, and each of the second layers 141, 142, and 143 includes three composites And may include a product layer 223, 224, 225 and a pooling layer 232. In one embodiment, the number of layers of the first and second layers and the number of pooling layers are exemplary according to the structure of AlexNet, so the number may be changed according to the setting of the developer.

As shown in FIG. 3, the proposed network 110 according to one embodiment may not have a last pooling layer 233 and a full connection layer 240, unlike the network shown in FIG. Since the sizes of input and output data must be fixed values to pass through the complete connection layer 240, the size of all input data must be resized to a fixed size. Since the complexity increases in the process of reordering, one embodiment can reduce the complexity of re-sizing the input data by not providing the pooling layer 233 and the complete connection layer 240. [

3, in the proposed network 110 according to an embodiment, the feature maps generated through the first layer 130 may be commonly used in the plurality of second layers 141, 142, and 143 have.

In the composite neural network, features such as simple edges of the target image are extracted in the lower layer, and complex features such as the shape of the object are extracted in the higher layer. Therefore, instead of placing a plurality of layers from a lower layer, a feature map generated in the first layer 130 is sent to a plurality of second layers 141, 142, 143 with only one first layer 130 being present, Unnecessary computational complexity generated in the layer can be reduced.

In one embodiment, the first pooling layer 232, which each of the plurality of second layers 141, 142, and 143 includes, may have strides of different sizes. Even if faces of various sizes are present in the target image 100, by performing a pulling operation with strides of different sizes, it is possible to obtain optimized heat maps for faces of various sizes, thereby improving the face detection performance.

As an example, a second layer including a pooling layer with a small size stripe is suitable for representing a heat map for a small face, and a second layer including a pooling layer with a large size stride includes a hit for a large face It may be appropriate to represent the map. This is an example, and it is possible to obtain an optimized heat map for faces of various sizes by differentiating strides of layers other than the first pooling layer.

을 설정하여 히트맵으로부터 얼굴 영역을 판단할 수 있고 얼굴 후보 영역을 생성할 수 있다.In one embodiment, the last concatenated product layer 225 of each second layer may generate 256 feature maps, and the heat map may be obtained through normalization and scaling. For this heatmap, we distinguish between the facial region and the non-face region.

The face region can be determined from the heat map and the face candidate region can be generated.

4 is a diagram for explaining a detection network according to an embodiment of the present invention.

The input patch 410 received from the proposed network itself has high face detection capability but may perform additional operations through the detection network to increase the recall rate and reduce false- . The face candidate region input to the detection network may be referred to as an input patch 410.

4, in one embodiment, the detection network includes four aggregate product layers 421, 422, 423, and 424, four pooling layers 431, 432, and 433, and one full connection layer 440 ). The detection network can generate one-dimensional data 191 as a result of classification and regression with respect to the input patch 410 and can detect a face area that is more accurate than a boxed face area in the input patch have.

The detection network according to an embodiment can regress to a more accurate face region in the face candidate region and can be regenerated in the face region by D. Wang, J. Ynag, and Q. Liu, "Hierarchical Convolutional Neural Network for Face Detection, International Conference on Image and Graphics, pp. 373-384, 2015, which is incorporated herein by reference in its entirety.

In one embodiment, in addition to the above structure, a classification process for determining whether or not a face exists in the input patch 410 is additionally performed to increase a recall rate and to detect a false-positive ) Can be reduced.

It is possible to present a value 192 for judging whether a face exists in the input patch 410 and to classify the input patch 410. If the input patch 410 has a face, the probability 1 is presented. Can be used.

In one embodiment, accurate face region location information 193 can be presented through Face Bound Regression. The location information may include x coordinate, y coordinate, width and height.

In one embodiment, since the face area is included in the patch 1 411, the probability 1 is presented 451 as a result of the classification process, and the position information 452 of the accurate face area is obtained as a result of the regression process ) May be presented.

Since the face area is not included in the patch 2 412, the probability 0 is presented 261 as a result of the classification process, and since the face area is not classified, the position information does not include the position information of the patch 2 412 A label 462 can be given.

We can learn the weight of the detection network by defining the loss function for Classification and Regression and minimizing the value of the loss function as shown in Equation (4) below.

는 조정 파라미터(parameter)이다. 일 실시예에서, 분류(Classification)에 대한 손실함수는 아래의 수식 (5)와 같이 교차 엔트로피 함수(cross-entropy loss function)일 수 있고, 회귀(Regression)에 대한 손실함수는 아래의 수식 (6)과 같이 정밀한 얼굴 영역의 위치 정보와 실제 얼굴 영역의 위치 정보 간의 유클리디언 거리(Euclidean distance)가 최소가 되도록 설계 할 수 있다.From here,

Is an adjustment parameter. In one embodiment, the loss function for Classification may be a cross-entropy loss function as in Equation (5) below, and the loss function for Regression may be expressed by Equation 6 ) Can be designed so that the Euclidean distance between the position information of the accurate face region and the position information of the actual face region is minimized.

는 미니 배치(mini-batch)의 크기를 의미하며,

는 얼굴 영역의 위치 정보를 정의하는 행렬의 크기,

은 분류 과정에서 얼굴이라고 추정되는 확률 값,

은 목적하는 얼굴 영역인지 얼굴 영역이 아닌지에 대한 라벨이다. 또한,

과

는 각각 정밀한 얼굴 영역의 위치 정보와 이에 대해 가장 근접한 실제 얼굴 위치 정보이다.From here,

Refers to the size of the mini-batch,

The size of the matrix defining the position information of the face area,

Is a probability value estimated to be a face in the classification process,

Is a label of whether it is a desired facial region or a face region. Also,

and

Are respectively the position information of the accurate face region and the actual face position information which is closest thereto.

, 가속도(momentum)의

에 대해 매 세대(epoch) 수마다 학습 속도에

의 값을 곱할 수 있다. 완전 연결 계층의 드롭아웃(dropout)의 확률 값은 0.5일 수 있다.To minimize the loss function of Eq. (4), a stochastic gradient descent method can be used. The Caffe library is available and the initial learning rate is

, Momentum of

The number of epochs per epoch

Can be multiplied. The probability of a dropout in the full connection layer may be 0.5.

Negative sample mining techniques can be used for learning of the detection network. This technique is a technique to learn neural networks which are able to cope with specific situations by extracting small and useful examples that express purpose well, rather than learning synthetic neural networks using many generalized examples. That is, since the extracted face candidate region 170 output through the proposed network shows high face detection performance by itself, the detection network can be learned based on the extracted face candidate region 170 to maximize the performance.

In the face detection method according to an embodiment of the present invention, since the proposed network and the detection network are connected in series, the data to be processed by the detection network has a very high performance and directivity of the proposed network. Most face candidate regions output by the proposed network are likely to be patches with a high degree of similarity with the face.

Therefore, according to the negative example mining technique, it is possible to learn the detection network using the patches including the face region certainly among the face candidate regions generated by the proposal network.

5 is a flowchart of a face detection method according to an embodiment of the present invention.

Referring to FIG. 5, an embodiment may perform a learning process in order to detect a face (510).

In the learning step of the proposed network, a preliminary model that distinguishes an object area and a background area is used as a basic structure, and an initial weight is set through the transition learning, and learning is performed to distinguish the face area and the non-face area . In addition, the method includes a step of learning using a loss function that minimizes an Euclidean distance between a position coordinate 250 of a sequentially connected face candidate feature point and a position coordinate of a facial feature point existing in an actual image can do.

After receiving the face candidate region from the proposed network in the detection network, it can learn to generate a more accurate face region.

The learning network learns by using a hard sample mining technique, using a cross-entropy loss function as a loss function for classification, and returning to a precise face area Regression may be performed using a function that minimizes the Euclidean distance between the position information of the accurate face region and the position information of the actual face region.

When a target image to be subjected to face detection is input (520), a plurality of face candidate regions corresponding to the target image in the proposal network can be extracted (530). The extracting step may include converting the target image to the feature map in the first layer, converting each of the plurality of second layers into the heat map, and converting the heat map to the face candidate region .

In this case, the plurality of second layers can commonly use the feature map converted by the first layer. Also, the pooling layer, which may be included in the plurality of second layers, has strides of different sizes for each of the second layers, so that it is possible to generate an optimized heat map for faces of various sizes. That is, since the face detection performance can be improved even for faces of various sizes, a face detection method robust against a wild environment can be provided.

The detection network can receive a face candidate region from the proposed network, classify whether a face region exists in the face candidate region, and regression to a face region more accurate than the face candidate region (540). Regression means that the position coordinates of the face candidate region received from the proposed network are adjusted to be close to the position coordinates of the actual face region. D. Wang, J. Ynag, and Q. Liu, "Hierarchical Convolutional Neural Network for Face Detection," Proceedings of International Conference on Image and Graphics, pp. 373-384, 2015. The face boundary regression method (Face Bound Regression) can be used.

Precise face candidate regions generated in the detection network can be presented as a final face region through post processing (550). The post-processing may be Non-Maximum Suppression (NMS).

6 is a block diagram of a face detection system according to an embodiment of the present invention.

Referring to FIG. 6, the face detection system may include a proposal unit 610 and a detection unit 640. The proposal unit 610 receiving the target image 100 may extract the face candidate region and propose the same to the detection unit 640. The detection unit 640 may receive the face candidate region and detect a more accurate face candidate region .

The proposal unit 610 may include a first layer unit 620 and a second layer unit 630.

The first layer 620 may include a plurality of convolutional layers and a pooling layer, and may generate a feature map corresponding to the target image.

The plurality of second hierarchical units 630 commonly use the feature maps generated by the first hierarchical unit and each of the plurality of second hierarchical units 630 can generate a heat map corresponding to the feature maps. Each of the plurality of second layer units 630 may include a plurality of concatenated product layers and a pooling layer connected in series, and the size of the stride of the pooling layer included in any one of the second layer units And may be different from the stride size of the pooling layer included in the second layer corresponding to the pooling layer. By varying the size of the stride, it is possible to generate optimized heat maps even if there are various sizes of faces in the target image. In addition to the pulling layer, the sizes of the strides of the convolution layer included in each second layer may be different.

In one embodiment, the proposal unit 610 may use a predecessor model that distinguishes an object area and a background area as a basic structure, sets an initial weight through transition learning, distinguishes a face area and a non-face area Learning can be done. In addition, it is possible to learn using a loss function that minimizes an Euclidean distance between a position coordinate 250 of a sequentially connected face candidate feature point and a position coordinate of a facial feature point existing in an actual image.

The proposal unit 610 can generate a face candidate region from the generated heat map and send it to the detection unit 640. [

In one example, the detector 640 may include a plurality of convolutional layers, a pooling layer, and a full connection layer. The detecting unit 640 can perform an operation on the face candidate region received from the proposing unit 610 according to the layer and can generate one-dimensional data indicating the position coordinates of the face candidate region precisely as a result of the operation.

The detecting unit 640 may include a classifying unit 650 and a regression unit 660. In one embodiment, the classifier 650 and regression unit 660 may be the same network that includes a plurality of convolutional layers and a pooling layer. The classifying unit 650 can determine whether there is a face in the face candidate region. If the face is found, the classifying unit 650 presents the probability 1, and if it is determined that there is no face, the classifying unit 650 can present the probability 0. The regression unit 660 can detect the face region more accurately than the face candidate region boxed in the input patch. More specifically, if the classifying unit 650 determines that there is a face in the face candidate region, the regression unit 660 can present the position information of the face region, which is refined through Face Bound Regression. The location information may include x coordinate, y coordinate, width and height. On the other hand, if the classifying unit 650 determines that there is no face in the face candidate region, the regression unit 660 may assign a label to the face candidate region to ignore the box-marked position information.

The detection unit can learn to generate a more accurate face region after receiving the face candidate region from the proposal unit.

For example, the detector uses a hard sample mining technique, uses a cross-entropy loss function as a loss function for classification, and regresses to a fine face region A learning function can be used to minimize the Euclidean distance between the position information of the accurate face region and the position information of the actual face region.

If the detection unit 640 generates information on the face candidate region that is refined, the final face region 670 can be presented through post processing. The post-processing may be Non-Maximum Suppression (NMS).

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (16)

A plurality of heat maps representing a face component included in the image, the plurality of heat maps being generated by applying different composite products and pooling schemes to the image; a plurality of different face candidate regions Extracting; And
Detecting a face region included in the image based on the plurality of different face candidate regions
And detecting a face of the face. The method according to claim 1,
Wherein the extracting of the plurality of different face candidate regions comprises:
Transforming the image into feature maps through a first layer comprising at least a first convolution layer performing a composite product and at least one first pooling layer performing a pooling; And
And converting the feature maps into heat maps through each of the plurality of second layers including at least one second composite product layer and at least one second pooling layer to extract the plurality of different face candidate regions step
Lt; / RTI >
Wherein the plurality of second layers commonly use the feature maps to convert the feature maps into the heat maps. 3. The method of claim 2,
The step of converting the feature maps into heat maps comprises:
Each of the plurality of second layers successively performing a composite product and a pooling operation to convert feature maps into heat maps,
Wherein a layer included in one of the plurality of second layers and a layer corresponding to an operation order of the layer included in the other second layer have strides of different sizes, Wherein the second hierarchical layer generates an optimized heat map for faces of different sizes. The method according to claim 1,
The step of detecting the face region comprises:
Classifying the face candidate region with respect to face presence or absence by determining whether or not the face region exists in the face candidate region; And
A step of regression to a precise face candidate region based on the classification and the face candidate regions
And a face detection step of detecting a face of the face. 5. The method of claim 4,
The classifying may include:
A probability 1 if the face region is included, and a probability 0 if the face region is not included. 5. The method of claim 4,
The step of regressing to the precise face candidate region comprises:
And a step of providing a label indicating that the position information of the face area is ignored if the face area is classified in the classifying step and if the face area is not classified, . The method according to claim 1,
The step of detecting the face region comprises:
Wherein the face candidate regions detect a face region through at least one convolution product layer, at least one pulling layer, and at least one Fully-Connected layer (FCL). The method according to claim 1,
Wherein the extracting of the face candidate regions comprises:
Performing learning for distinguishing a face region and a non-face region based on a neural network model for distinguishing an object region and a non-object region; And
Learning face candidate region extraction using an image database including one or more facial landmarks
The face detection method comprising: The method according to claim 1,
The step of detecting the face region comprises:
Wherein the face candidate regions are learned by using the face candidate regions as a database through a negative sample mining technique. A plurality of heat maps representing a face component included in the image, the plurality of heat maps being generated by applying different composite products and pooling schemes to the image; a plurality of different face candidate regions A proposal unit for extracting and proposing to a detection unit; And
A detecting unit for detecting a face region included in the image based on the plurality of different face candidate regions,
Based face detection system. 11. The method of claim 10,
[0027]
A first layer for transforming the image into feature maps through at least one first convolution layer performing a convolution and at least one first pooling layer performing a polling; And
A plurality of second hierarchical layers for converting the feature maps into heat maps through at least one second convolution layer and at least one second pooling layer to extract the plurality of different face candidate regions,
Lt; / RTI >
Wherein the plurality of second layers use the feature maps commonly to convert the feature maps into the heat maps. 11. The method of claim 10,
[0027]
Wherein each of the plurality of second layers sequentially performs a composite product and a pooling operation to convert feature maps into heat maps,
A layer included in one of the plurality of second layers and a layer corresponding to an operation order of the layer included in another one of the second layers has strides of different sizes Machine learning based face detection system. 11. The method of claim 10,
Wherein:
A classifying unit for classifying the presence or absence of a face by determining whether or not a face exists in the face candidate region; And
And a regression unit for regression to a precise face candidate region based on the classification and the face candidate regions,
Based face detection system. 14. The method of claim 13,
Wherein,
A probability 1 if the face region is included, and a probability 0 if the face region is not included. 14. The method of claim 13,
The regression unit,
Wherein if the classifying unit classifies the face region as having a face region, the position information of the face region is presented, and if the classifying unit classifies the face region as not having the face region, a label indicating that the position information of the face region is ignored. 11. The method of claim 10,
Wherein:
Characterized in that said face candidate regions comprise at least one of a composite product layer, at least one pooling layer and at least one Fully-Connected layer (FCL) for detecting face regions. system.

Images