CN108171196B - Face detection method and device - Google Patents
Face detection method and device Download PDFInfo
- Publication number
- CN108171196B CN108171196B CN201810018060.4A CN201810018060A CN108171196B CN 108171196 B CN108171196 B CN 108171196B CN 201810018060 A CN201810018060 A CN 201810018060A CN 108171196 B CN108171196 B CN 108171196B
- Authority
- CN
- China
- Prior art keywords
- face
- area
- moving target
- detection
- face detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Abstract
The invention provides a face detection method, which comprises the following steps: inputting a video image to be detected; detecting a moving target of the video image to obtain a moving target area; carrying out blob clustering on the moving target area to obtain a target detection area; selecting face and non-face sample images, training the cascade convolution neural network, and acquiring a trained face detection model; and carrying out face detection on the target detection area by adopting a face detection model, and outputting a detection result. Compared with the prior art, the method can quickly detect the face in a complex scene, and has high detection accuracy.
Description
Technical Field
The invention relates to image processing, video monitoring and security protection, in particular to a face detection method and a face detection device.
Background
In recent years, face recognition has been a popular direction in pattern recognition due to its wide application and its high market expectation.
The face detection is used as a key link in a face recognition system, occupies most system resources, and the detection performance directly influences the final recognition precision, so that the face detection is very important.
At present, common face detection methods based on videos/images include a template matching method, a skin color model method, a neural network method, an SVM classifier method, an Adaboost classifier method and the like. However, these methods have low detection accuracy and are difficult to adapt to complex environments.
In recent years, with the rapid development of artificial intelligence, deep learning is increasingly introduced into the field of face recognition, which forms high-level features by combining bottom-level features, with less influence from environmental changes. The Convolutional Neural Network (CNN) is a model structure for deep learning, processes image information through correlation operations of a convolutional layer and a sub-sampling layer, has low sensitivity to deformation such as translation, scaling, tilting and rotation, and is more suitable for image face detection in an unconstrained environment.
However, the existing face detection method based on the convolutional neural network is generally implemented by adopting more network layers, and the operation complexity is higher.
In summary, a face detection method with low operation complexity and high detection accuracy needs to be provided.
Disclosure of Invention
In view of the above, the main objective of the present invention is to rapidly implement face detection, and the detection accuracy is high.
To achieve the above object, according to a first aspect of the present invention, there is provided a face detection method, including:
the method comprises the following steps that a video image to be detected is input;
secondly, detecting a moving target of the video image to obtain a moving target area;
thirdly, carrying out blob clustering processing on the moving target area to obtain a target detection area;
selecting face and non-face sample images, training the cascade convolution neural network, and acquiring a trained face detection model;
and fifthly, carrying out face detection on the target detection area by adopting a face detection model, and outputting a detection result.
Further, the third step includes:
a small block merging step, namely judging whether two moving target areas are merged or not according to the width, the height and the distance of any two moving target areas in a current frame video image;
filtering isolated small blocks, namely filtering moving target areas belonging to the isolated small blocks in the current frame video image according to the area of the moving target area and the distance between the moving target area and the rest moving target areas;
and an oversize block segmentation step, in the current frame video image, segmenting the moving target area belonging to the oversize block according to the ratio of the area of the moving target area to the area of the video image.
Further, the fourth step includes:
a training sample selection step, namely selecting an annotated face image as a positive sample image, selecting an annotated non-face image as a negative sample image, and selecting a part of face sample images;
a first-stage convolutional neural network training step, wherein a positive sample image, a part of face sample image and a negative sample image are adopted to train a first-stage convolutional neural network, a trained first-stage face detection model is obtained, and a difficult case negative sample image is mined by using the first-stage face detection model;
a second-stage convolutional neural network training step, wherein the second-stage convolutional neural network is trained by adopting the positive sample image, part of the face sample image and the difficult-case negative sample image output in the first-stage convolutional neural network training step to obtain a trained second-stage face detection model, and the difficult-case negative sample image is mined by utilizing the second-stage face detection model;
and a third-stage convolutional neural network training step, namely training the third-stage convolutional neural network by adopting the positive sample image, part of the face sample image and the difficult negative sample image output in the second-stage convolutional neural network training step to obtain a trained third-stage face detection model.
The difficult negative sample image is a negative sample image which is falsely detected by the face detection model.
Further, the fifth step includes:
inputting a target detection area to be detected;
a multi-scale scaling step, namely carrying out multi-scale scaling on the target detection area;
a first-stage face detection model detection step, namely, adopting a first-stage face detection model to respectively carry out face detection on target detection areas of each scale, if a face is detected, acquiring a relative position in the target detection area according to the detected face position, marking the relative position as a first face candidate frame, and turning to a second-stage face detection model detection step, otherwise, turning to a face detection result output step;
a second-level face detection model detection step, namely, carrying out face detection on the first face candidate frame by adopting a second-level face detection model, if a face is detected, acquiring a relative position in a target detection area according to the detected face position, marking the relative position as the second face candidate frame, and turning to a third-level face detection model detection step, otherwise, turning to a face detection result output step;
a third-level face detection model detection step, adopting a third-level face detection model to perform face detection on the second face candidate frame, if a face is detected, acquiring a relative position in a target detection area according to the detected face position, and marking the relative position as a face frame;
a human face detection result output step, namely outputting the position of a human face frame if the human face frame exists in the target detection area; and if the target detection area does not have the face frame, outputting no face.
Further, a face detection method according to the present invention may further include: and a sixth step of tracking the face detection result by adopting an optical flow method and outputting a face tracking result.
According to another aspect of the present invention, there is provided a face detection apparatus, including:
the video image input module is used for inputting a video image to be detected;
the moving target detection module is used for detecting a moving target of the video image to acquire a moving target area;
the blob cluster processing module is used for performing blob cluster processing on the moving target area to obtain a target detection area;
the cascade convolution neural network training module is used for selecting face and non-face sample images, training the cascade convolution neural network and obtaining a trained face detection model;
and the face detection module is used for carrying out face detection on the target detection area by adopting a face detection model and outputting a detection result.
Further, the blob processing module comprises:
the small block merging module is used for judging whether two moving target areas are merged or not according to the width, the height and the distance of any two moving target areas in the current frame video image;
the isolated small block filtering module is used for filtering the moving target area belonging to the isolated small block in the current frame video image according to the area of the moving target area and the distance between the moving target area and the rest moving target areas;
and the oversized block segmentation module is used for segmenting the moving target area belonging to the oversized block in the current frame video image according to the ratio of the area of the moving target area to the area of the video image.
Further, the cascaded convolutional neural network training module comprises:
the training sample selecting module is used for selecting the marked face image as a positive sample image, selecting the marked non-face image as a negative sample image and selecting a part of face sample image;
the first-stage convolutional neural network training module is used for training the first-stage convolutional neural network by adopting the positive sample image, part of the face sample image and the negative sample image, acquiring a trained first-stage face detection model, and mining a difficult case negative sample image by utilizing the first-stage face detection model;
the second-level convolutional neural network training module is used for training the second-level convolutional neural network by adopting the positive sample image, part of the face sample image and the difficult negative sample image output by the first-level convolutional neural network training module to obtain a trained second-level face detection model, and mining the difficult negative sample image by utilizing the second-level face detection model;
and the three-level convolutional neural network training module is used for training the third-level convolutional neural network by adopting the positive sample image, part of the face sample image and the difficult negative sample image output by the two-level convolutional neural network training module to obtain a trained third-level face detection model.
Further, the face detection module comprises:
the to-be-detected area input module is used for inputting a to-be-detected target detection area;
the multi-scale scaling module is used for carrying out multi-scale scaling on the target detection area;
the first-stage face detection model detection module is used for respectively carrying out face detection on the target detection area of each scale by adopting a first-stage face detection model, if a face is detected, acquiring the relative position in the target detection area according to the detected face position, marking the relative position as a first face candidate frame, and switching to the second-stage face detection model detection module, otherwise, switching to the face detection result output module;
the second-level face detection model detection module is used for detecting the face of the first face candidate frame by adopting a second-level face detection model, if the face is detected, the relative position in the target detection area is obtained according to the detected face position, the second face candidate frame is marked and is switched to the third-level face detection model detection module, and if the face is not detected, the third-level face detection model detection module is switched to the face detection result output module;
the third-level face detection model detection module is used for detecting the face of the second face candidate frame by adopting a third-level face detection model, and if the face is detected, the relative position in the target detection area is obtained according to the detected face position and is marked as a face frame;
the human face detection result output module is used for outputting the position of the human face frame if the human face frame exists in the target detection area; and if the target detection area does not have the face frame, outputting no face.
Further, a face detection apparatus according to the present invention may further include: and the face tracking module is used for tracking the face detection result by adopting an optical flow method and outputting a face tracking result.
Compared with the prior face detection technology, the face detection method and the face detection device of the invention adopt the cascade convolution neural network to detect the face on one hand, and effectively improve the accuracy of the face detection through the training of part of face samples and the mining of difficult negative samples; on the other hand, before the face detection of the cascade convolution neural network is carried out, a target detection area is obtained through moving target detection and blob processing, and then the face detection is only carried out in the target detection area, so that the calculation amount is greatly reduced.
Drawings
Fig. 1 shows a flow chart of a first embodiment of a face detection method according to the invention.
Fig. 2 shows a flow chart of a second embodiment of a face detection method according to the invention.
Fig. 3 shows a block diagram of a first embodiment of a face detection apparatus according to the invention.
Fig. 4 shows a block diagram of a second embodiment of a face detection apparatus according to the invention.
Detailed Description
To further clarify the structure, features and other objects of the present invention, a detailed description of the preferred embodiments will be given below with reference to the accompanying drawings, which are provided for illustration of the technical solution of the present invention and are not intended to limit the present invention.
Fig. 1 shows a flow chart of a first embodiment of a face detection method according to the invention. As shown in fig. 1, a face detection method according to the present invention includes:
a first step S1 of inputting a video image to be detected;
a second step S2, detecting a moving target of the video image to obtain a moving target area;
a third step of S3, carrying out blob clustering processing on the moving target area to obtain a target detection area;
a fourth step S4, selecting face and non-face sample images, training the cascade convolution neural network, and obtaining a trained face detection model;
and a fifth step S5, carrying out face detection on the target detection area by adopting a face detection model, and outputting a detection result.
The second step S2 can be implemented by existing moving object detection methods, including but not limited to: static difference method, gaussian background modeling method, CodeBook background modeling method, and VIBE method.
In the embodiment, in the second step S2, a mixed gaussian background modeling method, such as a moving object detection method in the document "entropy theory-based modified mixed gaussian model moving object detection algorithm, liuwanarmy, lie," computer application research, 2015,32(7): 2226-.
Further, the third step S3 includes:
a small block merging step S31, in the current frame video image, judging whether the two moving target areas are merged according to the width, height and distance of any two moving target areas;
an isolated small block filtering step S32, in the current frame video image, according to the area of the moving target area and the distance between the moving target area and the rest of the moving target areas, filtering the moving target areas belonging to the isolated small blocks;
and an excessively large block segmentation step S33 of segmenting the moving target region belonging to the excessively large block in the current frame video image according to the ratio of the area of the moving target region to the area of the video image.
Further, the small lump merging step S31 includes: for any two moving object regions B in the current frame video imageiAnd BjCalculating a moving target region BiWidth WB ofiAnd height HBiCalculating a moving target region BjWidth WB ofjAnd height HBjSeparately calculating moving object regions BiAnd BjSpacing Xd in the x and y directionsij、XdijIf at the same time satisfy
And
the moving object region BiAnd BjAnd combining the two areas into a moving target area, wherein i, j belongs to {1, 2, …, BNum1}, i is not equal to j, BNum1 is the number of the moving target areas in the current frame video image, and W and H are the width and the height of the current frame video image.
Further, the isolated small lump filtering step S32 includes: for the current frame video image, the area of each moving target area is respectively calculated, and the maximum value SB of the area of the moving target area is obtainedmaxFor any one moving target region BpIf it is satisfied
The moving object region B is consideredpFor isolating small lumps, moving target area BpFiltering, wherein p is the {1, 2, …, BNum2}, BNum2 is the number of moving object regions in the video image of the current frame, SBpAs a moving target region BpThe area of (a).
Further, the excessively large blob dividing step S33 includes: for any moving object region B in the current frame video imageqCalculating a moving target region BqArea SB ofqRatio to the area S of the current frame video image
If it is not
The moving object region B is consideredqThe moving target area B is an oversize lump and is divided by adopting a foreground density distribution methodqAnd (2) segmenting into one or more moving object regions, wherein q belongs to {1, 2, …, BNum3}, and BNum3 is the moving object region in the video image of the current frameAnd (4) the number.
Further, the foreground density distribution segmentation method in the oversized blob segmentation step S33 includes: for moving target areas B considered as excessively large blobsqBuild a width of
Has a height of
In the moving target area B, the sliding window is adoptedqSequentially sliding once, counting the number value of the foreground points in each sliding window area, and obtaining the maximum value WFNum of the number value of the foreground points in the sliding window areamaxThe value of the foreground point is WFNummaxAs a divided moving object region Bq1(ii) a Moving object region BqInner removing moving object area Bq1The other parts are used as the segmentation subareas, the area of each segmentation subarea is respectively calculated, if the area of a certain segmentation subarea is equal to the area of the motion target area Bq1Area ratio of less than
Then filter the partition subarea, if the area of a partition subarea and the moving object area Bq1The ratio of the areas is located in
In between, the sub-region is divided into sub-regions and the moving object region Bq1Merging if the area of a certain segmented sub-region is associated with the moving object region Bq1Area ratio is greater than
This segmented sub-region is taken as the segmented moving object region.
The T is1The value range of (1) is 2-8, T2The value range of (1) is 3-8, T3The value range of (1) is 2-8, T4The value range of (A) is 2-8,the T is5The value range of (1) is 8-12, T6The value range of (a) is 2-6. Further, said T1The value range of (1) is 2-4, T2The value range of (1) is 4-6, T3The value range of (1) is 2-4, T4Has a value range of 2 to 4, T5The value range of (1) is 9-11, T6The value range of (A) is 3-5.
Further, the fourth step S4 includes:
a training sample selection step S41, selecting an annotated face image as a positive sample image, selecting an annotated non-face image as a negative sample image, and selecting a part of face sample images;
a first-stage convolutional neural network training step S42, wherein the first-stage convolutional neural network is trained by adopting the positive sample image, part of the face sample image and the negative sample image to obtain a trained first-stage face detection model, and a difficult-case negative sample image is mined by utilizing the first-stage face detection model;
a second-stage convolutional neural network training step S43, wherein the second-stage convolutional neural network is trained by adopting the positive sample image, part of the face sample image and the difficult-case negative sample image output by the first-stage convolutional neural network training step S42 to obtain a trained second-stage face detection model, and the difficult-case negative sample image is mined by utilizing the second-stage face detection model;
and a third-level convolutional neural network training step S44, wherein the third-level convolutional neural network is trained by adopting the positive sample image, part of the face sample image and the difficult negative sample image output by the second-level convolutional neural network training step S43 to obtain a trained third-level face detection model.
The marked face image comprises an image of only selecting a face under the conditions of different scenes, different illumination, different postures, different skin colors and the like, the marked non-face image comprises an image without a face under the conditions of different scenes, different illumination and the like, and the marked part of the face image is an image containing a face sample image.
Further, the labeling the non-face image comprises: the body part containing the skin color characteristic, the background similar to the skin color, and other area images.
The difficult negative sample image is a negative sample image which is falsely detected by the face detection model. In the embodiment, if a negative sample image labeled with a non-human face is detected by a first-level human face detection model or a second-level human face detection model and then output as a human face, the negative sample image is regarded as a difficult negative sample image.
Further, the first-stage Convolutional Neural Network (CNN), the second-stage Convolutional Neural Network, and the third-stage Convolutional Neural Network are existing Convolutional Neural networks, and may include but are not limited to: general CNN, RCNN, SPPNET, Fast-RCNN, YOLO, SSD, etc., or combinations of the above.
In a first embodiment, the first stage convolutional neural network comprises: 4 convolutional layers, 1 pooling layer; the second stage convolutional neural network comprises: 3 convolutional layers, 3 pooling layers, 3 full-link layers; the third-stage convolutional neural network comprises: 4 convolution layers, 3 pooling layers, 3 full-link layers, wherein the last full-link layer output is 3, represents to output 3 classification results, includes: human faces, non-human faces, and partial human faces.
In a second embodiment, the first stage convolutional neural network, the second stage convolutional neural network, and the third stage convolutional neural network adopt a neural network structure in the paper of "application of convolutional neural network in face detection research, liuxiuqing, university of shanxi, 2015", where the last full-connection layer output is 3, which indicates that 3 classification results are output, and includes: human faces, non-human faces, and partial human faces.
Further, the fifth step S5 includes:
a to-be-detected region input step S51 of inputting a target detection region to be detected;
a multi-scale scaling step S52 of multi-scale scaling the target detection region;
a first-level face detection model detection step S53, adopting a first-level face detection model to respectively perform face detection on the target detection area of each scale, if a face is detected, acquiring the relative position in the target detection area according to the detected face position, marking the relative position as a first face candidate frame, and turning to a second-level face detection model detection step S54, otherwise, turning to a face detection result output step S56;
a second-level face detection model detection step S54, adopting a second-level face detection model to perform face detection on the first face candidate frame, if a face is detected, acquiring a relative position in a target detection area according to the detected face position, marking the relative position as the second face candidate frame, and turning to a third-level face detection model detection step S55, otherwise, turning to a face detection result output step S56;
a third-level face detection model detection step S55, adopting a third-level face detection model to perform face detection on the second face candidate frame, and if a face is detected, acquiring a relative position in a target detection area according to the detected face position and marking the relative position as a face frame;
a face detection result output step S56, wherein if a face frame exists in the target detection area, the position of the face frame is output; and if the target detection area does not have the face frame, outputting no face.
Fig. 2 shows a flow chart of a second embodiment of a face detection method according to the invention. As shown in fig. 2, a face detection method according to the present invention includes:
a first step S1 of inputting a video image to be detected;
a second step S2, detecting a moving target of the video image to obtain a moving target area;
a third step of S3, carrying out blob clustering processing on the moving target area to obtain a target detection area;
a fourth step S4, selecting face and non-face sample images, training the cascade convolution neural network, and obtaining a trained face detection model;
a fifth step S5, adopting a face detection model to carry out face detection on the target detection area, and outputting a face detection result;
in a sixth step S6, the face detection result is tracked by an optical flow method, and a face tracking result is output.
The sixth step S6 can be implemented by an existing tracking method or algorithm based on an optical flow method.
Further, the sixth step S6 includes:
step S61 is given to the ID of the first frame of face frame, for the video image to be detected, the video image of which the face frame is detected for the first time is taken as a first frame image, and all face frames in the first frame video image are given unique ID numbers;
a current frame face frame ID obtaining step S62, calculating the position of the face frame of the ID number in the current frame by an optical flow method according to the position of the face frame of each ID number in the previous frame, comparing the face frames of the ID numbers corresponding to the previous frame and the current frame by adopting a face comparison method, if the comparison is successful, giving the ID number to the face frame of the current frame corresponding to the ID number position, if the comparison is failed, giving a new ID number to the face frame of the current frame corresponding to the ID number position, and if the current frame does not have the face frame corresponding to the ID number position, adopting a tracking result as the face frame position of the ID number until the face frame of the ID number disappears;
and a face tracking result output step S63, wherein the tracking results of the face frames with all ID numbers in the video to be detected are output.
In an embodiment, the sixth step S6 tracks all face frames of the video image to be detected by using a pyramid optical flow method, such as the method in "video tracking algorithm based on online Boosting and LK optical flows", proceedings of science and technology university in southwest, 2012,27(4):80-84 ".
Fig. 3 is a block diagram showing a first embodiment of a face detection apparatus according to the present invention. As shown in fig. 3, a face detection apparatus according to the present invention includes:
the video image input module 1 is used for inputting a video image to be detected;
the moving target detection module 2 is used for detecting a moving target of the video image to acquire a moving target area;
the blob cluster processing module 3 is used for performing blob cluster processing on the moving target area to obtain a target detection area;
the cascade convolution neural network training module 4 is used for selecting face and non-face sample images, training the cascade convolution neural network and obtaining a trained face detection model;
and the face detection module 5 is used for detecting the face of the target detection area by adopting a face detection model and outputting a detection result.
The moving object detection module 2 may be implemented by existing moving object detection methods, devices or apparatuses, including but not limited to: a moving object detection method, device or apparatus based on static difference, Gaussian background modeling, CodeBook background modeling and VIBE algorithm.
Further, the blob processing module 3 includes:
a small block merging module 31, configured to determine whether two moving target regions are merged according to the widths, heights, and distances of any two moving target regions in a current frame video image;
the isolated small block filtering module 32 is used for filtering the moving target area belonging to the isolated small block in the current frame video image according to the area of the moving target area and the distance between the moving target area and the rest moving target areas;
and an excessively large block segmentation module 33, configured to perform segmentation processing on the moving target region belonging to an excessively large block in the current frame video image according to a ratio of an area of the moving target region to an area of the video image.
Further, the small blob merging module 31 includes: for any two moving object regions B in the video image of the current frameiAnd BjCalculating a moving target region BiWidth WB ofiAnd height HBiCalculating a moving target region BjWidth WB ofjAnd height HBjSeparately calculating moving object regions BiAnd BjSpacing Xd in the x and y directionsij、XdijIf at the same time satisfy
And
the moving object region BiAnd BjAnd combining the two areas into a moving target area, wherein i, j belongs to {1, 2, …, BNum1}, i is not equal to j, BNum1 is the number of the moving target areas in the current frame video image, and W and H are the width and the height of the current frame video image.
Further, the isolated small block filtering module 32 includes: for the current frame video image, the area of each moving target area is respectively calculated, and the maximum value SB of the area of the moving target area is obtainedmaxFor any one moving target region BpIf it is satisfied
The moving object region B is consideredpFor isolating small lumps, moving target area BpFiltering, wherein p is the {1, 2, …, BNum2}, BNum2 is the number of moving object regions in the video image of the current frame, SBpAs a moving target region BpThe area of (a).
Further, the oversized block segmentation module 33 includes: for any moving object region B in the video image of the current frameqCalculating a moving target region BqArea SB ofqRatio to the area S of the current frame video image
If it is not
The moving object region B is consideredqThe moving target area B is an overlarge block and is divided by adopting a foreground density distribution division moduleqAnd (3) segmenting into one or more moving object regions, wherein q belongs to {1, 2, …, BNum3}, and BNum3 is the number of the moving object regions in the video image of the current frame.
Further, the foreground in the oversized blob segmentation module 33The density distribution segmentation module includes: for moving target areas B considered as excessively large massesqBuild a width of
Has a height of
In the moving target area B, the sliding window is adoptedqSequentially sliding once, counting the number value of the foreground points in each sliding window area, and obtaining the maximum value WFNum of the number value of the foreground points in the sliding window areamaxThe value of the foreground point is WFNummaxAs a divided moving object region Bq1(ii) a Moving object region BqInner removing moving object area Bq1The other parts are used as the segmentation subareas, the area of each segmentation subarea is respectively calculated, if the area of a certain segmentation subarea is equal to the area of the motion target area Bq1Area ratio of less than
Then filter the partition subarea, if the area of a partition subarea and the moving object area Bq1The ratio of the areas is located in
In between, the sub-region is divided into sub-regions and the moving object region Bq1Merging if the area of a certain segmented sub-region is associated with the moving object region Bq1Area ratio is greater than
This segmented sub-region is taken as the segmented moving object region.
The T is1The value range of (1) is 2-8, T2The value range of (1) is 3-8, T3The value range of (1) is 2-8, T4The value range of (1) is 2-8, T5The value range of (1) is 8-12, T6The value range of (a) is 2-6. IntoIn one step, the T1The value range of (1) is 2-4, T2The value range of (1) is 4-6, T3The value range of (1) is 2-4, T4Has a value range of 2 to 4, T5The value range of (1) is 9-11, T6The value range of (A) is 3-5.
Further, the cascaded convolutional neural network training module 4 includes:
a training sample selection module 41, configured to select an annotated face image as a positive sample image, select an annotated non-face image as a negative sample image, and select a part of face sample images;
the primary convolutional neural network training module 42 is configured to train the primary convolutional neural network by using the positive sample image, the partial face sample image and the negative sample image, obtain a trained primary face detection model, and mine a difficult-case negative sample image by using the primary face detection model;
the second-level convolutional neural network training module 43 is configured to train the second-level convolutional neural network by using the positive sample image, part of the face sample image and the difficult-to-sample negative sample image output by the first-level convolutional neural network training module 42, obtain a trained second-level face detection model, and mine the difficult-to-sample negative sample image by using the second-level face detection model;
and the third-level convolutional neural network training module 44 is configured to train the third-level convolutional neural network by using the positive sample image, part of the face sample image and the difficult-case negative sample image output by the second-level convolutional neural network training module 43, so as to obtain a trained third-level face detection model.
The marked face image comprises an image of only selecting a face under the conditions of different scenes, different illumination, different postures, different skin colors and the like, the marked non-face image comprises an image without a face under the conditions of different scenes, different illumination and the like, and the marked part of the face image is an image containing a face sample image.
Further, the labeling the non-face image comprises: the body part containing the skin color characteristic, the background similar to the skin color, and other area images.
The difficult negative sample image is a negative sample image which is falsely detected by the face detection model. In the embodiment, if a negative sample image labeled with a non-human face is detected by a first-level human face detection model or a second-level human face detection model and then output as a human face, the negative sample image is regarded as a difficult negative sample image.
Further, the first-stage Convolutional Neural Network (CNN), the second-stage Convolutional Neural Network, and the third-stage Convolutional Neural Network are existing Convolutional Neural networks, and may include but are not limited to: general CNN, RCNN, SPPNET, Fast-RCNN, YOLO, SSD, etc., or combinations of the above.
In an embodiment, the first stage convolutional neural network comprises: 4 convolutional layers, 1 pooling layer; the second stage convolutional neural network comprises: 3 convolutional layers, 3 pooling layers, 3 full-link layers; the third-stage convolutional neural network comprises: 4 convolutional layers, 3 pooling layers, 3 full-link layers.
Further, the face detection module 5 includes:
a to-be-detected region input module 51 for inputting a target detection region to be detected;
a multi-scale scaling module 52, configured to perform multi-scale scaling on the target detection area;
a first-stage face detection model detection module 53, configured to perform face detection on target detection areas of each scale by using a first-stage face detection model, if a face is detected, obtain a relative position in the target detection area according to the detected face position, mark the relative position as a first face candidate frame, and switch to a second-stage face detection model detection module 54, otherwise switch to a face detection result output module 56;
the second-level face detection model detection module 54 performs face detection on the first face candidate frame by using a second-level face detection model, and if a face is detected, acquires a relative position in a target detection area according to the detected face position, marks the relative position as the second face candidate frame, and switches to the third-level face detection model detection module 55, otherwise, switches to the face detection result output module 56;
the third-level face detection model detection module 55 performs face detection on the second face candidate frame by using a third-level face detection model, and if a face is detected, acquires a relative position in a target detection area according to the detected face position and marks the relative position as a face frame;
a face detection result output module 56, which outputs the position of the face frame if the target detection area has the face frame; and if the target detection area does not have the face frame, outputting no face.
Fig. 4 is a block diagram showing a second embodiment of a face detection apparatus according to the present invention. As shown in fig. 4, a face detection apparatus according to the present invention includes:
the video image input module 1 is used for inputting a video image to be detected;
the moving target detection module 2 is used for detecting a moving target of the video image to acquire a moving target area;
the blob cluster processing module 3 is used for performing blob cluster processing on the moving target area to obtain a target detection area;
the cascade convolution neural network training module 4 is used for selecting face and non-face sample images, training the cascade convolution neural network and obtaining a trained face detection model;
the face detection module 5 is used for detecting the face of the target detection area by adopting a face detection model and outputting a detection result;
and the face tracking module 6 is used for tracking the face detection result by adopting an optical flow method and outputting a face tracking result.
The face tracking module 6 can be implemented by an existing tracking method, device or apparatus based on an optical flow method.
Further, the face tracking module 6 includes:
a first frame face frame ID assigning module 61, configured to assign a unique ID number to all face frames in a first frame video image, by using, as the first frame image, a video image in which a face frame is detected for the first time for a video image to be detected;
a current frame face frame ID obtaining module 62, configured to calculate, according to a position of a face frame of each ID number in a previous frame, a position of the face frame of the ID number in the current frame by an optical flow method, and compare, by using a face comparison method, face frames of the previous frame and the current frame corresponding to the ID number, if the comparison is successful, the ID number is assigned to the face frame of the current frame corresponding to the ID number position, if the comparison is failed, a new ID number is assigned to the face frame of the current frame corresponding to the ID number position, and if the current frame does not have a face frame corresponding to the ID number position, a tracking result is used as the face frame position of the ID number until the face frame of the ID number disappears;
and the face tracking result output module 63 is used for outputting the tracking results of all the face frames with the ID numbers in the video to be detected.
Compared with the prior face detection technology, the face detection method and the face detection device of the invention adopt the cascade convolution neural network to detect the face on one hand, and effectively improve the accuracy of the face detection through the training of part of face samples and the mining of difficult negative samples; on the other hand, before the face detection of the cascade convolution neural network is carried out, a target detection area is obtained through moving target detection and blob processing, and then the face detection is only carried out in the target detection area, so that the calculation amount is greatly reduced.
While the foregoing is directed to the preferred embodiment of the present invention, and is not intended to limit the scope of the invention, it will be understood that the invention is not limited to the embodiments described herein, which are described to assist those skilled in the art in practicing the invention. Further modifications and improvements may readily occur to those skilled in the art without departing from the spirit and scope of the invention, and it is intended that the invention be limited only by the terms and scope of the appended claims, as including all alternatives and equivalents which may be included within the spirit and scope of the invention as defined by the appended claims.
Claims (12)
1. A face detection method is characterized by comprising the following steps:
the method comprises the following steps that a video image to be detected is input;
secondly, detecting a moving target of the video image to obtain a moving target area;
thirdly, carrying out blob clustering processing on the moving target area to obtain a target detection area;
selecting face and non-face sample images, training the cascade convolution neural network, and acquiring a trained face detection model;
a fifth step of adopting a face detection model to carry out face detection on the target detection area and outputting a detection result;
the third step includes:
a small block merging step, namely judging whether two moving target areas are merged or not according to the width, the height and the distance of any two moving target areas in a current frame video image;
filtering isolated small blocks, namely filtering moving target areas belonging to the isolated small blocks in the current frame video image according to the area of the moving target area and the distance between the moving target area and the rest moving target areas;
and an oversize block segmentation step, in the current frame video image, segmenting the moving target area belonging to the oversize block according to the ratio of the area of the moving target area to the area of the video image.
2. The method of claim 1, wherein the blob merging step comprises: for any two moving object regions B in the current frame video imageiAnd BjCalculating a moving target region BiWidth WB ofiAnd height HBiCalculating a moving target region BjWidth WB ofjAnd height HBjSeparately calculating moving object regions BiAnd BjSpacing Xd in the x and y directionsij、XdijIf at the same time satisfy
And
the moving object region BiAnd BjMerging the two video images into a moving target area, wherein i, j belongs to {1, 2, …, BNum1}, i is not equal to j, BNum1 is the number of the moving target areas in the current frame video image, and W and H are the width and the height of the current frame video image;
the T is1The value range of (1) is 2-8.
3. The method of claim 1, wherein the isolated blob filtering step comprises: for the current frame video image, the area of each moving target area is respectively calculated, and the maximum value SB of the area of the moving target area is obtainedmaxFor any one moving target region BpIf it is satisfied
The moving object region B is consideredpFor isolating small lumps, moving target area BpFiltering, wherein p is the {1, 2, …, BNum2}, BNum2 is the number of moving object regions in the video image of the current frame, SBpAs a moving target region BpThe area of (d);
the T is2The value range of (a) is 3-8.
4. The method of claim 1, wherein the oversized blob segmenting step comprises: for any moving object region B in the current frame video imageqCalculating a moving target region BqArea SB ofqRatio to the area S of the current frame video image
If it is not
The moving object region B is consideredqThe moving target area B is an oversize lump and is divided by adopting a foreground density distribution methodqDividing the video image into one or more moving target areas, wherein q belongs to {1, 2, …, BNum3}, and BNum3 is the number of the moving target areas in the video image of the current frame;
the T is3The value range of (1) is 2-8.
5. The method of claim 4, wherein the foreground density distribution segmentation method comprises: for moving target areas B considered as excessively large blobsqBuild a width of
Has a height of
In the moving target area B, the sliding window is adoptedqSequentially sliding once, counting the number value of the foreground points in each sliding window area, and obtaining the maximum value WFNum of the number value of the foreground points in the sliding window areamaxThe value of the foreground point is WFNummaxAs a divided moving object region Bq1(ii) a Moving object region BqInner removing moving object area Bq1The other parts are used as the segmentation subareas, the area of each segmentation subarea is respectively calculated, if the area of a certain segmentation subarea is equal to the area of the motion target area Bq1Area ratio of less than
Then filter the partition subarea, if the area of a partition subarea and the moving object area Bq1The ratio of the areas is located in
In between, the sub-region is divided into sub-regions and the moving object region Bq1Combining, e.g.Area of some divided sub-region and moving target region Bq1Area ratio is greater than
Taking the divided sub-region as a divided moving target region;
the T is4The value range of (1) is 2-8, T5The value range of (1) is 8-12, T6The value range of (a) is 2-6.
6. The method of claim 1, wherein the fourth step comprises:
a training sample selection step, namely selecting an annotated face image as a positive sample image, selecting an annotated non-face image as a negative sample image, and selecting a part of face sample images;
a first-stage convolutional neural network training step, wherein a positive sample image, a part of face sample image and a negative sample image are adopted to train a first-stage convolutional neural network, a trained first-stage face detection model is obtained, and a difficult case negative sample image is mined by using the first-stage face detection model;
a second-stage convolutional neural network training step, wherein the second-stage convolutional neural network is trained by adopting the positive sample image, part of the face sample image and the difficult-case negative sample image output in the first-stage convolutional neural network training step to obtain a trained second-stage face detection model, and the difficult-case negative sample image is mined by utilizing the second-stage face detection model;
a third-stage convolutional neural network training step, wherein a third-stage convolutional neural network is trained by adopting a positive sample image, a part of face sample image and a difficult negative sample image output in the second-stage convolutional neural network training step to obtain a trained third-stage face detection model;
the difficult negative sample image is a negative sample image which is falsely detected by the face detection model.
7. The method of claim 1, wherein the fifth step comprises:
inputting a target detection area to be detected;
a multi-scale scaling step, namely carrying out multi-scale scaling on the target detection area;
a first-stage face detection model detection step, namely, adopting a first-stage face detection model to respectively carry out face detection on target detection areas of each scale, if a face is detected, acquiring a relative position in the target detection area according to the detected face position, marking the relative position as a first face candidate frame, and turning to a second-stage face detection model detection step, otherwise, turning to a face detection result output step;
a second-level face detection model detection step, namely, carrying out face detection on the first face candidate frame by adopting a second-level face detection model, if a face is detected, acquiring a relative position in a target detection area according to the detected face position, marking the relative position as the second face candidate frame, and turning to a third-level face detection model detection step, otherwise, turning to a face detection result output step;
a third-level face detection model detection step, adopting a third-level face detection model to perform face detection on the second face candidate frame, if a face is detected, acquiring a relative position in a target detection area according to the detected face position, and marking the relative position as a face frame;
a human face detection result output step, namely outputting the position of a human face frame if the human face frame exists in the target detection area; and if the target detection area does not have the face frame, outputting no face.
8. The method of claim 1, wherein the method comprises: and a sixth step of tracking the face detection result by adopting an optical flow method and outputting a face tracking result.
9. A face detection apparatus, comprising:
the video image input module is used for inputting a video image to be detected;
the moving target detection module is used for detecting a moving target of the video image to acquire a moving target area;
the blob cluster processing module is used for performing blob cluster processing on the moving target area to obtain a target detection area;
the cascade convolution neural network training module is used for selecting face and non-face sample images, training the cascade convolution neural network and obtaining a trained face detection model;
the face detection module is used for detecting the face of the target detection area by adopting a face detection model and outputting a detection result;
the blob processing module comprises:
the small block merging module is used for judging whether two moving target areas are merged or not according to the width, the height and the distance of any two moving target areas in the current frame video image;
the isolated small block filtering module is used for filtering the moving target area belonging to the isolated small block in the current frame video image according to the area of the moving target area and the distance between the moving target area and the rest moving target areas;
and the oversized block segmentation module is used for segmenting the moving target area belonging to the oversized block in the current frame video image according to the ratio of the area of the moving target area to the area of the video image.
10. The apparatus of claim 9, wherein the cascaded convolutional neural network training module comprises:
the training sample selecting module is used for selecting the marked face image as a positive sample image, selecting the marked non-face image as a negative sample image and selecting a part of face sample image;
the first-stage convolutional neural network training module is used for training the first-stage convolutional neural network by adopting the positive sample image, part of the face sample image and the negative sample image, acquiring a trained first-stage face detection model, and mining a difficult case negative sample image by utilizing the first-stage face detection model;
the second-level convolutional neural network training module is used for training the second-level convolutional neural network by adopting the positive sample image, part of the face sample image and the difficult negative sample image output by the first-level convolutional neural network training module to obtain a trained second-level face detection model, and mining the difficult negative sample image by utilizing the second-level face detection model;
the three-level convolutional neural network training module is used for training the third-level convolutional neural network by adopting the positive sample image, part of the face sample image and the difficult negative sample image output by the two-level convolutional neural network training module to obtain a trained third-level face detection model;
the difficult negative sample image is a negative sample image which is falsely detected by the face detection model.
11. The apparatus of claim 9, wherein the face detection module comprises:
the to-be-detected area input module is used for inputting a to-be-detected target detection area;
the multi-scale scaling module is used for carrying out multi-scale scaling on the target detection area;
the first-stage face detection model detection module is used for respectively carrying out face detection on the target detection area of each scale by adopting a first-stage face detection model, if a face is detected, acquiring the relative position in the target detection area according to the detected face position, marking the relative position as a first face candidate frame, and switching to the second-stage face detection model detection module, otherwise, switching to the face detection result output module;
the second-level face detection model detection module is used for detecting the face of the first face candidate frame by adopting a second-level face detection model, if the face is detected, the relative position in the target detection area is obtained according to the detected face position, the second face candidate frame is marked and is switched to the third-level face detection model detection module, and if the face is not detected, the third-level face detection model detection module is switched to the face detection result output module;
the third-level face detection model detection module is used for detecting the face of the second face candidate frame by adopting a third-level face detection model, and if the face is detected, the relative position in the target detection area is obtained according to the detected face position and is marked as a face frame;
the human face detection result output module is used for outputting the position of the human face frame if the human face frame exists in the target detection area; and if the target detection area does not have the face frame, outputting no face.
12. The apparatus of claim 9, wherein the apparatus comprises: and the face tracking module is used for tracking the face detection result by adopting an optical flow method and outputting a face tracking result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810018060.4A CN108171196B (en) | 2018-01-09 | 2018-01-09 | Face detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810018060.4A CN108171196B (en) | 2018-01-09 | 2018-01-09 | Face detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108171196A CN108171196A (en) | 2018-06-15 |
| CN108171196B true CN108171196B (en) | 2021-10-26 |
Family
ID=62517837
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810018060.4A Active CN108171196B (en) | 2018-01-09 | 2018-01-09 | Face detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108171196B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109063593A (en) * | 2018-07-13 | 2018-12-21 | 北京智芯原动科技有限公司 | A kind of face tracking method and device |
| CN110738079A (en) * | 2018-07-19 | 2020-01-31 | 杭州海康威视数字技术股份有限公司 | Method and device for detecting abnormal number of front row personnel of motor vehicle and computer equipment |
| CN109271877A (en) * | 2018-08-24 | 2019-01-25 | 北京智芯原动科技有限公司 | A kind of human figure identification method and device |
| CN109359555A (en) * | 2018-09-21 | 2019-02-19 | 江苏安凰领御科技有限公司 | A kind of high-precision human face quick detection method |
| CN109740486B (en) * | 2018-12-27 | 2021-07-27 | 北京铁路信号有限公司 | Method and system for identifying number of human beings contained in image |
| CN111753598A (en) * | 2019-03-29 | 2020-10-09 | 中国联合网络通信集团有限公司 | Face detection method and device |
| CN110490073A (en) * | 2019-07-15 | 2019-11-22 | 浙江省北大信息技术高等研究院 | Object detection method, device, equipment and storage medium |
| CN110555867B (en) * | 2019-09-05 | 2023-07-07 | 杭州智爱时刻科技有限公司 | Multi-target object tracking method integrating object capturing and identifying technology |
| CN111695522B (en) * | 2020-06-15 | 2022-10-18 | 重庆邮电大学 | In-plane rotation invariant face detection method and device and storage medium |
| CN111931598A (en) * | 2020-07-20 | 2020-11-13 | 湖北美和易思教育科技有限公司 | Intelligent classroom real-time analysis method and system based on face recognition |
| CN112580449B (en) * | 2020-12-06 | 2022-10-21 | 江苏集萃未来城市应用技术研究所有限公司 | Method for judging abnormal behaviors of people on intelligent construction site |
| CN113780262A (en) * | 2021-08-28 | 2021-12-10 | 特斯联科技集团有限公司 | Intelligent illumination compensation method, device, equipment and system for face recognition |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7903870B1 (en) * | 2006-02-24 | 2011-03-08 | Texas Instruments Incorporated | Digital camera and method |
| CN104021570A (en) * | 2014-06-04 | 2014-09-03 | 电子科技大学 | Method for directional cross-border detection and mixing line detection in video |
| CN104301669A (en) * | 2014-09-12 | 2015-01-21 | 重庆大学 | Suspicious target detection tracking and recognition method based on dual-camera cooperation |
| CN104573613A (en) * | 2013-10-16 | 2015-04-29 | 深圳市捷顺科技实业股份有限公司 | Video security anti-smashing method and device based on blob tracking |
| CN107239736A (en) * | 2017-04-28 | 2017-10-10 | 北京智慧眼科技股份有限公司 | Method for detecting human face and detection means based on multitask concatenated convolutional neutral net |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8744122B2 (en) * | 2008-10-22 | 2014-06-03 | Sri International | System and method for object detection from a moving platform |
-
2018
- 2018-01-09 CN CN201810018060.4A patent/CN108171196B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7903870B1 (en) * | 2006-02-24 | 2011-03-08 | Texas Instruments Incorporated | Digital camera and method |
| CN104573613A (en) * | 2013-10-16 | 2015-04-29 | 深圳市捷顺科技实业股份有限公司 | Video security anti-smashing method and device based on blob tracking |
| CN104021570A (en) * | 2014-06-04 | 2014-09-03 | 电子科技大学 | Method for directional cross-border detection and mixing line detection in video |
| CN104301669A (en) * | 2014-09-12 | 2015-01-21 | 重庆大学 | Suspicious target detection tracking and recognition method based on dual-camera cooperation |
| CN107239736A (en) * | 2017-04-28 | 2017-10-10 | 北京智慧眼科技股份有限公司 | Method for detecting human face and detection means based on multitask concatenated convolutional neutral net |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108171196A (en) | 2018-06-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108171196B (en) | Face detection method and device | |
| Rao et al. | Selfie video based continuous Indian sign language recognition system | |
| US11062123B2 (en) | Method, terminal, and storage medium for tracking facial critical area | |
| CN107563345B (en) | Human body behavior analysis method based on space-time significance region detection | |
| Zhang et al. | Learning semantic scene models by object classification and trajectory clustering | |
| Han et al. | Density-based multifeature background subtraction with support vector machine | |
| Li et al. | Adaptive deep convolutional neural networks for scene-specific object detection | |
| Liu et al. | Crowd counting via cross-stage refinement networks | |
| WO2009109127A1 (en) | Real-time body segmentation system | |
| KR20160096460A (en) | Recognition system based on deep learning including a plurality of classfier and control method thereof | |
| Zhang et al. | License plate localization in unconstrained scenes using a two-stage CNN-RNN | |
| Vignesh et al. | Abnormal event detection on BMTT-PETS 2017 surveillance challenge | |
| Luo et al. | Traffic analytics with low-frame-rate videos | |
| Rong et al. | Scene text recognition in multiple frames based on text tracking | |
| Liu et al. | Smoke-detection framework for high-definition video using fused spatial-and frequency-domain features | |
| Nag et al. | A new unified method for detecting text from marathon runners and sports players in video (PR-D-19-01078R2) | |
| Roy et al. | Foreground segmentation using adaptive 3 phase background model | |
| Lin et al. | Pedestrian detection by exemplar-guided contrastive learning | |
| Lee et al. | License plate detection via information maximization | |
| Silva et al. | Online weighted one-class ensemble for feature selection in background/foreground separation | |
| Dewan et al. | Detection of object in motion using improvised background subtraction algorithm | |
| Dziri et al. | Late fusion of multiple convolutional layers for pedestrian detection | |
| Peng et al. | Semi-Supervised Bolt Anomaly Detection Based on Local Feature Reconstruction | |
| JP5241687B2 (en) | Object detection apparatus and object detection program | |
| Wu et al. | Adaptive pixel-block based background subtraction using low-rank and block-sparse matrix decomposition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |