CN108269250A - Method and apparatus based on convolutional neural networks assessment quality of human face image - Google Patents

Method and apparatus based on convolutional neural networks assessment quality of human face image Download PDF

Info

Publication number
CN108269250A
CN108269250A CN201711439458.7A CN201711439458A CN108269250A CN 108269250 A CN108269250 A CN 108269250A CN 201711439458 A CN201711439458 A CN 201711439458A CN 108269250 A CN108269250 A CN 108269250A
Authority
CN
China
Prior art keywords
image
quality
image data
facial
facial image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711439458.7A
Other languages
Chinese (zh)
Inventor
向少雄
贺波涛
吴迪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Fiberhome Digtal Technology Co Ltd
Original Assignee
Wuhan Fiberhome Digtal Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Fiberhome Digtal Technology Co Ltd filed Critical Wuhan Fiberhome Digtal Technology Co Ltd
Priority to CN201711439458.7A priority Critical patent/CN108269250A/en
Publication of CN108269250A publication Critical patent/CN108269250A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The present invention provides a kind of method and apparatus based on convolutional neural networks assessment quality of human face image, method includes：Multiple facial images in monitor video are acquired, build human face data collection, human face data is concentrated comprising the corresponding preliminary quality value of every facial image；The image data of every facial image is calculated, training sample set is generated according to every image data and preliminary quality value, image data includes key point bianry image, gray-scale map and edge strength figure；Training sample set is trained based on convolutional neural networks, obtains Environmental Evaluation Model；By Face image synthesis image data to be identified, image data is subjected to face quality evaluation by Environmental Evaluation Model, obtains the mass value of facial image to be identified.Scheme is by judging that frame that quality of human face image is best in video identifies work for follow-up human face analysis.The present invention can optimize the effect of quality evaluation and filtering, greatly promote case and investigate and prosecute efficiency.

Description

Method and apparatus based on convolutional neural networks assessment quality of human face image
Technical field
The present invention relates to image identification technical field more particularly to based on convolutional neural networks assessment quality of human face image Method and apparatus.
Background technology
With science and technology fast development, intelligent Video Surveillance Technology extensive use in police criminal detection business, by from regarding Frequency finds and lock suspected target in monitoring become the important means of technique of criminal investigation.
In practical applications, it after staff has locked suspect, needs to identify that determining suspect's identity is believed by static state Breath is deployed to ensure effective monitoring and control of illegal activities by Dynamic Recognition implementation to be facilitated to arrest, however, the angle of face in images to be recognized, rotation, illumination, Resolution ratio, noise the factors such as block and all can generate significant impact to recognition result.How to be selected from a large amount of facial images at present In the still no technology of facial image that meets the requirements can realize.
Invention content
The embodiment of the present invention proposes a kind of method based on convolutional neural networks assessment quality of human face image, method packet It includes：
Multiple facial images in monitor video are acquired, calculate the corresponding preliminary quality value of every facial image；
The image data of every facial image is calculated, according to every image data institute corresponding with described image data Preliminary quality value generation training sample set is stated, described image data include key point bianry image, gray-scale map and edge strength figure；
The training sample set is trained based on convolutional neural networks, obtains Environmental Evaluation Model；
By the image data of Face image synthesis to be identified, face quality evaluation is carried out by the Environmental Evaluation Model, Obtain the mass value of the facial image to be identified.
Wherein, the preliminary quality value is specially that each quality assessment parameter the sum of is multiplied with assessment weight, the quality Assessment parameter includes facial angle, clarity, blocks and brightness.
Specifically, calculating the image data of every facial image, specifically include,
The Keypoint detector of dlib is used to facial image, the key point of the facial image is obtained, generates the people The key point bianry image of face image；
Gray processing is carried out to the facial image to handle to obtain the gray-scale map of facial image；
Edge strength is asked for the gray-scale map using laplacian operators to respond and generate edge strength figure；
By the key point bianry image, the facial image gray-scale map and the edge strength figure, three as image A channel data generates image data, and described image data are as training sample.
Convolutional neural networks are trained the training sample set by convolutional calculation formula in the present invention, the convolution of use Calculating formula is
Wherein, xiAnd yjIt is that i-th of input feature vector figure and j-th export characteristic pattern, k respectivelyijIt is i-th input feature vector Convolution kernel between figure and j-th of output characteristic pattern, * represent convolution, BNγ,β(x) BatchNormalization, r tables are represented Show the regional area of shared weights；
The cross entropy loss function is L(i)(k)=y(i)logp(i)+(1-y(i))(1-logp(i)), wherein, y(i)It represents The corresponding label with quality evaluation score value of i-th of sample, p(i)Represent the quality matter that the sample is exported by convolutional neural networks Amount evaluation score value.
In a first aspect, an embodiment of the present invention provides a kind of dresses based on convolutional neural networks assessment quality of human face image It puts, described device includes：
Collecting unit, for acquiring multiple facial images in monitor video；
Computing unit connects the collecting unit, for calculating the corresponding preliminary quality value of every facial image, and calculates The image data of every facial image, according to every image data preliminary quality value corresponding with described image data Training sample set is generated, described image data include key point bianry image, gray-scale map and edge strength figure；
Training unit connects the computing unit, and the training sample set is trained based on convolutional neural networks, is obtained To Environmental Evaluation Model；
Evaluation unit connects the training unit and the collecting unit, for by the figure of Face image synthesis to be identified As data, face quality evaluation is carried out by the Environmental Evaluation Model that the training unit obtains, obtains the face to be identified The mass value of image.
Wherein, the preliminary quality value that the computing unit obtains is specially that each quality assessment parameter is multiplied with assessment weight The sum of, the quality assessment parameter includes facial angle, clarity, blocks and brightness.
Specifically, the computing unit includes the first computing unit and the second computing unit；
First computing unit is used to calculate the corresponding preliminary quality value of every facial image；
Second computing unit is used to calculate the image data of every facial image, specifically includes,
The Keypoint detector of dlib is used to facial image, the key point of the facial image is obtained, generates the people The key point bianry image of face image；
Gray processing is carried out to the facial image to handle to obtain the gray-scale map of facial image；
Edge strength is asked for the gray-scale map using laplacian operators to respond and generate edge strength figure；
By the key point bianry image, the facial image gray-scale map and the edge strength figure, three as image A channel data generates image data, and described image data are as training sample.
Wherein, using being trained to the training sample set, convolutional calculation formula is the training unit
Wherein, xiAnd yjIt is that i-th of input feature vector figure and j-th export characteristic pattern, k respectivelyijIt is i-th input feature vector Convolution kernel between figure and j-th of output characteristic pattern, * represent convolution, BNγ,β(x) BatchNormalization, r tables are represented Show the regional area of shared weights；
The cross entropy loss function is L(i)(k)=y(i)logp(i)+(1-y(i))(1-logp(i)), wherein, y(i)It represents The corresponding label with quality evaluation score value of i-th of sample, p(i)Represent the quality matter that the sample is exported by convolutional neural networks Amount evaluation score value.
It has the beneficial effect that：
By carrying out quality evaluation to facial image, be conducive to filter out the facial image of high quality from monitor video, Low-quality input sample can be effectively filtered when screening facial image, effectively promote the precision of follow-up identification work.
Description of the drawings
Specific embodiments of the present invention are described below with reference to accompanying drawings, wherein：
Fig. 1 shows the method flow diagram based on convolutional neural networks assessment quality of human face image in the embodiment of the present invention；
Fig. 2 shows the method flow diagrams based on convolutional neural networks assessment quality of human face image in the embodiment of the present invention；
Fig. 3 a show the schematic diagram of the corresponding key point bianry image of face sample in the embodiment of the present invention；
Fig. 3 b show the schematic diagram of the corresponding edge strength image of face sample in the embodiment of the present invention；
Fig. 3 c show the schematic diagram of 3 channel datas of single sample in training set in the embodiment of the present invention；
Fig. 4 shows the structure of convolutional neural networks in the embodiment of the present invention；
Fig. 5 shows the structure drawing of device based on convolutional neural networks assessment quality of human face image in the embodiment of the present invention.
Specific embodiment
In order to which technical scheme of the present invention and advantage is more clearly understood, below in conjunction with attached drawing to the exemplary of the present invention Embodiment is described in more detail, it is clear that described embodiment be only the present invention part of the embodiment rather than The exhaustion of all embodiments.And in the absence of conflict, the feature in the embodiment and embodiment in this explanation can be mutual It is combined.
Embodiment one
Fig. 1 shows the method provided in an embodiment of the present invention based on convolutional neural networks assessment quality of human face image, institute The method of stating includes：
Step 101：Multiple facial images in monitor video are acquired, calculate the corresponding preliminary quality of every facial image Value；
Step 102：The image data of every facial image is calculated, according to every image data and preliminary quality value generation instruction Practice sample set, wherein, image data includes key point bianry image, gray-scale map and edge strength figure；
Step 103：Training sample set is trained based on convolutional neural networks, obtains Environmental Evaluation Model；
Step 104：By Face image synthesis image data to be identified, by image data by Environmental Evaluation Model into pedestrian Face quality evaluation obtains the mass value of facial image to be identified.
This programme by facial image by being converted to the figure being made of key point binary map, gray-scale map and edge strength figure As data, and then judge that frame that quality of human face image is best in video identifies work for follow-up human face analysis, it can be excellent Change the effect of quality evaluation and filtering, greatly promote case and investigate and prosecute efficiency.
In practical application, this programme is by providing the side for carrying out quality evaluation to the picture comprising face to be identified Case enhances the robustness of evaluation, so as to promote the accuracy of follow-up recognition of face work.
With reference to Fig. 2, the detailed reality of method of this programme based on convolutional neural networks assessment quality of human face image is provided Existing mode：
Step 201：Monitor video is obtained, acquires facial image in monitor video, establishes the face for including preliminary quality value Data set；
Wherein, preliminary quality value is evaluated to obtain according to human face analysis requirement to facial image progress preliminary quality；It should In step, quality assessment parameter includes facial angle, clarity, blocks and brightness, by these quality assessment parameters with it is corresponding Facial image is associated storage, and whether the standard of quality evaluation is is conducive to recognition of face, for example, face rotation angle is more than 30 degree or when can not observe entire positive face, the picture quality is with regard to relatively low；Such as facial image clarity is relatively low again, then give compared with Low quality evaluation score value.
In practical application, the quality assessment parameter of above-mentioned facial image can be normalized, it will according to face It asks, quality of human face image is marked by weighted mean method, the preliminary quality value of facial image is obtained by calculation.
In 0~1 section, score can be weighted average preliminary quality Distribution value by the score value to each Primary Reference index It obtains, for example, required according to follow-up human face analysis, give facial angle, block with higher weight.
It the sum of is multiplied specifically, preliminary quality value is specially each quality assessment parameter with assessment weight, the quality is commented Estimate parameter to include facial angle, clarity, block and brightness.
Step 202：Training dataset is established according to human face data collection, training data concentrates each sample to be corresponded to by the sample Key point bianry image, gray-scale map, edge strength figure form and with preliminary quality value associated storage.
Specifically, the method for the bianry image of generation key point is：The Keypoint detector of dlib is used to facial image, The key point of face is obtained, the bianry image of key point is generated, as described in Fig. 3 a, employs 68 key points of face；
Gray processing is carried out to facial image to handle to obtain the gray-scale map of facial image；
Edge strength is asked for the gray-scale map using laplacian operators to respond and generate edge strength figure；
Specifically, the method for generation edge strength figure is：Side is asked for facial image gray-scale map with laplacian operators Edge intensity response, and edge strength figure is generated, wherein, the discrete form of laplacian operators can be expressed as：
The convolution mask of the laplacian operators is：Response diagram as described in Fig. 3 b；
Using above-mentioned key point bianry image, gray-scale map and edge strength figure as the RGB triple channel facial images of image, and With the preliminary quality value associated storage in step 201, training sample is generated, as described in Fig. 3 c.
Multiple facial images can be handled, and then composing training sample set in the step.
Step 203：Convolutional neural networks operation is carried out by the training sample set obtained to above-mentioned steps 202, obtains matter Measure evaluation model；
Specifically, the purpose of the step is the quality evaluation score value and preliminary quality evaluation point that convolutional neural networks is allowed to export The standard of value is unified, and convolutional neural networks is allowed gradually to learn preliminary quality evaluation method；Preliminary quality evaluation score value is according to master Reference index is wanted to provide, can also the subjective feeling of reference man give a mark, trained purpose is that convolutional neural networks is allowed to learn Impression of the people to image quality evaluation.
Fig. 4 is the structure for showing convolutional neural networks in the present invention, including：Connected input layer 401, one or more strings Convolution pond unit 402, dropout layers 403, full articulamentum 404 and the recurrence layer 405 of connection；Wherein, convolution pond unit 402 include：Connected convolutional layer 4021, BatchNorm layers 4022, active coating 4023 and mean value pond layer 4024.
Input layer 401 is configured to input training sample (for first convolutional layer pond unit) or characteristic pattern (preceding a roll The characteristic pattern of product pond unit output), convolution pond unit 402 is shown as N number of in figure, and in practical application, convolution pondization is single Member 402 is preferably 5, and the convolution operation in each convolution pond unit 402 is represented by：
Wherein xiAnd yjIt is i-th of input feature vector figure and j-th of output characteristic pattern respectively.kijIt is i-th of input feature vector figure With the convolution kernel between j-th of output characteristic pattern, * represents convolution, BNγ,β(x) BatchNormalization is represented.Herein, will ReLU nonlinear functions y=max (0, x) is for neuron.Weights in the higher convolutional layer of ConvNets are that part is shared , r represents the regional area of shared weights.
Can be maximum pond after each convolutional layer, maximum pond is formulated into：
Wherein i-th output characteristic pattern yiIn each neuron in i-th of input feature vector figure xiIn the non-overlapping offices of s × s Portion region upper storage reservoir.
Return predicted value of the layer 405 using logistic regression output mass fraction.
The loss function that the present invention is included is cross entropy loss function, in the training stage, passes through cross entropy loss function Cross entropy of the preliminary quality evaluation label of input with input picture after pulleying and the transformation of neural network abovementioned layers is calculated, and Model parameter is updated by backpropagation and gradient descent method, so repeatedly, successive optimization reduces network output and quality evaluation Difference between label, the training process are to optimize the process for minimizing cross entropy loss function by gradient descent method.
The loss function used in the present invention for：
L(i)(k)=y(i)logp(i)+(1-y(i))(1-logp(i))
Wherein, y(i)Represent the corresponding label with quality evaluation score value of i-th of sample, p(i)Represent that the sample passes through convolution The quality quality evaluation score value of neural network output.
Step 204：By the image data of Face image synthesis to be identified, face quality is carried out by Environmental Evaluation Model and is commented Valency obtains the mass value of the facial image to be identified.
During specific identification, calculate facial image to be identified by key point binary map, gray-scale map and edge strength figure, obtain Face quality evaluation is carried out to the image data of facial image, and then by the Evaluation Model on Quality that training obtains, the model It is the evaluation score as facial image to export result, and evaluation score is higher to illustrate that the image is more conducive to subsequent identification work Make.
In practical applications, the evaluation result of K frame videos is counted, selects the highest frame image of score for subsequently identifying Work.
Method provided by the invention based on convolutional neural networks assessment quality of human face image, by being carried out to facial image Quality evaluation is conducive to filter out the facial image of high quality from monitor video, can be when screening facial image, effective mistake Low-quality input sample is filtered, effectively promotes the precision of follow-up identification work.
Embodiment two
Referring to Fig. 5, an embodiment of the present invention provides a kind of device based on convolutional neural networks assessment quality of human face image, Described device includes：
Collecting unit 501, for acquiring multiple facial images in monitor video；
Computing unit 502 connects collecting unit 501, for calculating the corresponding preliminary quality value of every facial image, and counts The image data of every facial image is calculated, according to every image data preliminary quality corresponding with described image data Value generation training sample set, described image data include key point bianry image, gray-scale map and edge strength figure；
Training unit 503 is connected computing unit 502, the training sample set is trained based on convolutional neural networks, Obtain Environmental Evaluation Model；
Evaluation unit 504 connects training unit 503 and collecting unit 501, for by the figure of Face image synthesis to be identified As data, face quality evaluation is carried out by the Environmental Evaluation Model that the training unit obtains, obtains the face to be identified The mass value of image.
Wherein, the preliminary quality value that computing unit 502 obtains is specially that each quality assessment parameter is multiplied with assessment weight The sum of, the quality assessment parameter includes facial angle, clarity, blocks and brightness.
Wherein, computing unit 502 includes the first computing unit and the second computing unit；
First computing unit is used to calculate the corresponding preliminary quality value of every facial image；
Second computing unit is used to calculate the image data of every facial image, specifically includes,
The Keypoint detector of dlib is used to facial image, the key point of the facial image is obtained, generates the people The key point bianry image of face image；
Gray processing is carried out to the facial image to handle to obtain the gray-scale map of facial image；
Edge strength is asked for the gray-scale map using laplacian operators to respond and generate edge strength figure；
By the key point bianry image, the facial image gray-scale map and the edge strength figure, three as image A channel data generates image data, and described image data are as training sample.
Further, using being trained to the training sample set, convolutional calculation formula is training unit
Wherein, xiAnd yjIt is that i-th of input feature vector figure and j-th export characteristic pattern, k respectivelyijIt is i-th input feature vector Convolution kernel between figure and j-th of output characteristic pattern, * represent convolution, BNγ,β(x) represent that BatchNormalization, r are represented The regional area of shared weights；
The cross entropy loss function is L(i)(k)=y(i)logp(i)+(1-y(i))(1-logp(i)), wherein, y(i)It represents The corresponding label with quality evaluation score value of i-th of sample, p(i)Represent the quality matter that the sample is exported by convolutional neural networks Amount evaluation score value.
Device provided by the invention by carrying out quality evaluation to facial image, is conducive to filter out from monitor video The facial image of high quality can effectively filter low-quality input sample when screening facial image, effectively promote follow-up knowledge The precision not worked.
For convenience of description, each section of apparatus above is divided into various modules with function or unit describes respectively.Certainly, Each module or the function of unit can be realized in same or multiple softwares or hardware when implementing the present invention.
It should be understood by those skilled in the art that, the embodiment of the present invention can be provided as method, system or computer program Product.Therefore, the reality in terms of complete hardware embodiment, complete software embodiment or combination software and hardware can be used in the present invention Apply the form of example.Moreover, the computer for wherein including computer usable program code in one or more can be used in the present invention The computer program production that usable storage medium is implemented on (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) The form of product.
The present invention be with reference to according to the method for the embodiment of the present invention, the flow of equipment (system) and computer program product Figure and/or block diagram describe.It should be understood that it can be realized by computer program instructions each in flowchart and/or the block diagram The combination of flow and/or box in flow and/or box and flowchart and/or the block diagram.These computers can be provided Program instruction is to the processor of all-purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices To generate a machine so that the instruction performed by computer or the processor of other programmable data processing devices generates use In the dress of function that realization is specified in one flow of flow chart or multiple flows and/or one box of block diagram or multiple boxes It puts.
These computer program instructions, which may also be stored in, can guide computer or other programmable data processing devices with spy Determine in the computer-readable memory that mode works so that the instruction generation being stored in the computer-readable memory includes referring to Enable the manufacture of device, the command device realize in one flow of flow chart or multiple flows and/or one box of block diagram or The function of being specified in multiple boxes.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that counted Series of operation steps are performed on calculation machine or other programmable devices to generate computer implemented processing, so as in computer or The instruction offer performed on other programmable devices is used to implement in one flow of flow chart or multiple flows and/or block diagram one The step of function of being specified in a box or multiple boxes.
Although preferred embodiments of the present invention have been described, but those skilled in the art once know basic creation Property concept, then additional changes and modifications may be made to these embodiments.So appended claims be intended to be construed to include it is excellent It selects embodiment and falls into all change and modification of the scope of the invention.

Claims (8)

  1. A kind of 1. method based on convolutional neural networks assessment quality of human face image, which is characterized in that the method includes：
    Multiple facial images in monitor video are acquired, calculate the corresponding preliminary quality value of every facial image；
    The image data of every facial image is calculated, it is corresponding with described image data described first according to every image data Mass value generation training sample set is walked, described image data include key point bianry image, gray-scale map and edge strength figure；
    The training sample set is trained based on convolutional neural networks, obtains Environmental Evaluation Model；
    By the image data of Face image synthesis to be identified, face quality evaluation is carried out by the Environmental Evaluation Model, is obtained The mass value of the facial image to be identified.
  2. 2. the method as described in claim 1, which is characterized in that the preliminary quality value be specially each quality assessment parameter with Assessment weight the sum of is multiplied, and the quality assessment parameter includes facial angle, clarity, blocks and brightness.
  3. 3. the method as described in claim 1, which is characterized in that the image data for calculating every facial image, tool Body includes,
    The Keypoint detector of dlib is used to facial image, the key point of the facial image is obtained, generates the face figure The key point bianry image of picture；
    Gray processing is carried out to the facial image to handle to obtain the gray-scale map of facial image；
    Edge strength is asked for the gray-scale map using laplacian operators to respond and generate edge strength figure；
    By the key point bianry image, the facial image gray-scale map and the edge strength figure, three as image are logical Track data generates image data, and described image data are as training sample.
  4. 4. method as described in any one of claims 1-3, which is characterized in that the convolutional neural networks pass through convolutional calculation formula pair The training sample set is trained, and the convolutional calculation formula is
    Wherein, xiAnd yjIt is that i-th of input feature vector figure and j-th export characteristic pattern, k respectivelyijIt is i-th of input feature vector figure and jth Convolution kernel between a output characteristic pattern, * represent convolution, BNγ,β(x) represent that BatchNormalization, r represent shared power The regional area of value；
    The cross entropy loss function is L(i)(k)=y(i)log p(i)+(1-y(i))(1-log p(i)), wherein, y(i)Represent i-th The corresponding label with quality evaluation score value of a sample, p(i)Represent the quality quality that the sample is exported by convolutional neural networks Evaluate score value.
  5. 5. a kind of device based on convolutional neural networks assessment quality of human face image, which is characterized in that described device includes：
    Collecting unit, for acquiring multiple facial images in monitor video；
    Computing unit connects the collecting unit, for calculating the corresponding preliminary quality value of every facial image, and described in calculating The image data of every facial image is generated according to every image data preliminary quality value corresponding with described image data Training sample set, described image data include key point bianry image, gray-scale map and edge strength figure；
    Training unit connects the computing unit, and the training sample set is trained based on convolutional neural networks, obtains matter Measure evaluation model；
    Evaluation unit connects the training unit and the collecting unit, for by the picture number of Face image synthesis to be identified According to the Environmental Evaluation Model progress face quality evaluation obtained by the training unit obtains the facial image to be identified Mass value.
  6. 6. device as claimed in claim 5, which is characterized in that the preliminary quality value that the computing unit obtains is specially each Quality assessment parameter the sum of is multiplied with assessment weight, and the quality assessment parameter includes facial angle, clarity, blocks and bright Degree.
  7. 7. device as claimed in claim 5, which is characterized in that the computing unit includes the first computing unit and second and calculates Unit；
    First computing unit is used to calculate the corresponding preliminary quality value of every facial image；
    Second computing unit is used to calculate the image data of every facial image, specifically includes,
    The Keypoint detector of dlib is used to facial image, the key point of the facial image is obtained, generates the face figure The key point bianry image of picture；
    Gray processing is carried out to the facial image to handle to obtain the gray-scale map of facial image；
    Edge strength is asked for the gray-scale map using laplacian operators to respond and generate edge strength figure；
    By the key point bianry image, the facial image gray-scale map and the edge strength figure, three as image are logical Track data generates image data, and described image data are as training sample.
  8. 8. the device as described in claim 5-7 is any, which is characterized in that the training unit is utilized to the training sample set It is trained, the convolutional calculation formula is
    Wherein, xiAnd yjIt is that i-th of input feature vector figure and j-th export characteristic pattern, k respectivelyijIt is i-th of input feature vector figure and jth Convolution kernel between a output characteristic pattern, * represent convolution, BNγ,β(x) represent that BatchNormalization, r represent shared power The regional area of value；
    The cross entropy loss function is L(i)(k)=y(i)log p(i)+(1-y(i))(1-log p(i)), wherein, y(i)Represent i-th The corresponding label with quality evaluation score value of a sample, p(i)Represent the quality quality that the sample is exported by convolutional neural networks Evaluate score value.
CN201711439458.7A 2017-12-27 2017-12-27 Method and apparatus based on convolutional neural networks assessment quality of human face image Pending CN108269250A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711439458.7A CN108269250A (en) 2017-12-27 2017-12-27 Method and apparatus based on convolutional neural networks assessment quality of human face image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711439458.7A CN108269250A (en) 2017-12-27 2017-12-27 Method and apparatus based on convolutional neural networks assessment quality of human face image

Publications (1)

Publication Number Publication Date
CN108269250A true CN108269250A (en) 2018-07-10

Family

ID=62772661

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711439458.7A Pending CN108269250A (en) 2017-12-27 2017-12-27 Method and apparatus based on convolutional neural networks assessment quality of human face image

Country Status (1)

Country Link
CN (1) CN108269250A (en)

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109101646A (en) * 2018-08-21 2018-12-28 北京深瞐科技有限公司 Data processing method, device, system and computer-readable medium
CN109345522A (en) * 2018-09-25 2019-02-15 北京市商汤科技开发有限公司 A kind of picture quality screening technique and device, equipment and storage medium
CN109615620A (en) * 2018-11-30 2019-04-12 腾讯科技（深圳）有限公司 The recognition methods of compression of images degree, device, equipment and computer readable storage medium
CN109784230A (en) * 2018-12-29 2019-05-21 中国科学院重庆绿色智能技术研究院 A kind of facial video image quality optimization method, system and equipment
CN109871780A (en) * 2019-01-28 2019-06-11 中国科学院重庆绿色智能技术研究院 A kind of face quality decision method, system and face identification method, system
CN109918885A (en) * 2019-03-12 2019-06-21 苏州宏裕千智能设备科技有限公司 A kind of onboard system multi-user access method and device
CN109948564A (en) * 2019-03-25 2019-06-28 四川川大智胜软件股份有限公司 It is a kind of based on have supervision deep learning quality of human face image classification and appraisal procedure
CN109977815A (en) * 2019-03-13 2019-07-05 上海商汤智能科技有限公司 Image quality evaluating method and device, electronic equipment, storage medium
CN109993150A (en) * 2019-04-15 2019-07-09 北京字节跳动网络技术有限公司 The method and apparatus at age for identification
CN110084130A (en) * 2019-04-03 2019-08-02 深圳鲲云信息科技有限公司 Face screening technique, device, equipment and storage medium based on multiple target tracking
CN110674925A (en) * 2019-08-29 2020-01-10 厦门大学 No-reference VR video quality evaluation method based on 3D convolutional neural network
CN110751043A (en) * 2019-09-19 2020-02-04 平安科技（深圳）有限公司 Face recognition method and device based on face visibility and storage medium
CN110807757A (en) * 2019-08-14 2020-02-18 腾讯科技（深圳）有限公司 Image quality evaluation method and device based on artificial intelligence and computer equipment
CN110837750A (en) * 2018-08-15 2020-02-25 华为技术有限公司 Human face quality evaluation method and device
WO2020038254A1 (en) * 2018-08-23 2020-02-27 杭州海康威视数字技术股份有限公司 Image processing method and apparatus for target recognition
CN110879981A (en) * 2019-11-14 2020-03-13 深圳市华付信息技术有限公司 Method and device for evaluating quality of key points of human face, computer equipment and storage medium
CN110956615A (en) * 2019-11-15 2020-04-03 北京金山云网络技术有限公司 Image quality evaluation model training method and device, electronic equipment and storage medium
CN111160219A (en) * 2019-12-26 2020-05-15 深圳云天励飞技术有限公司 Object integrity evaluation method and device, electronic equipment and storage medium
CN111199197A (en) * 2019-12-26 2020-05-26 深圳市优必选科技股份有限公司 Image extraction method and processing equipment for face recognition
CN111199186A (en) * 2019-12-03 2020-05-26 恒大智慧科技有限公司 Image quality scoring model training method, device, equipment and storage medium
CN111241925A (en) * 2019-12-30 2020-06-05 新大陆数字技术股份有限公司 Face quality evaluation method, system, electronic equipment and readable storage medium
CN111640099A (en) * 2020-05-29 2020-09-08 北京金山云网络技术有限公司 Method and device for determining image quality, electronic equipment and storage medium
CN111695522A (en) * 2020-06-15 2020-09-22 重庆邮电大学 In-plane rotation invariant face detection method and device and storage medium
CN111738179A (en) * 2020-06-28 2020-10-02 湖南国科微电子股份有限公司 Method, device, equipment and medium for evaluating quality of face image
CN111862040A (en) * 2020-07-20 2020-10-30 中移（杭州）信息技术有限公司 Portrait picture quality evaluation method, device, equipment and storage medium
CN112347849A (en) * 2020-09-29 2021-02-09 咪咕视讯科技有限公司 Video conference processing method, electronic device and storage medium
CN112529845A (en) * 2020-11-24 2021-03-19 浙江大华技术股份有限公司 Image quality value determination method, image quality value determination device, storage medium, and electronic device
CN113129252A (en) * 2019-12-30 2021-07-16 Tcl集团股份有限公司 Image scoring method and electronic equipment
CN109522950B (en) * 2018-11-09 2022-04-22 网易传媒科技（北京）有限公司 Image scoring model training method and device and image scoring method and device
CN109815465B (en) * 2018-12-19 2023-11-17 平安科技（深圳）有限公司 Deep learning-based poster generation method and device and computer equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106127170A (en) * 2016-07-01 2016-11-16 重庆中科云丛科技有限公司 A kind of merge the training method of key feature points, recognition methods and system
CN106897748A (en) * 2017-03-02 2017-06-27 上海极链网络科技有限公司 Face method for evaluating quality and system based on deep layer convolutional neural networks
CN106951840A (en) * 2017-03-09 2017-07-14 北京工业大学 A kind of facial feature points detection method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106127170A (en) * 2016-07-01 2016-11-16 重庆中科云丛科技有限公司 A kind of merge the training method of key feature points, recognition methods and system
CN106897748A (en) * 2017-03-02 2017-06-27 上海极链网络科技有限公司 Face method for evaluating quality and system based on deep layer convolutional neural networks
CN106951840A (en) * 2017-03-09 2017-07-14 北京工业大学 A kind of facial feature points detection method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
VIGNESH S等: "Face image quality assessment for face selection in surveillance video using convolutional neural networks", 《 2015 IEEE GLOBAL CONFERENCE ON SIGNAL AND INFORMATION PROCESSING (GLOBALSIP)》 *

Cited By (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110837750A (en) * 2018-08-15 2020-02-25 华为技术有限公司 Human face quality evaluation method and device
CN110837750B (en) * 2018-08-15 2023-11-03 华为技术有限公司 Face quality evaluation method and device
CN109101646A (en) * 2018-08-21 2018-12-28 北京深瞐科技有限公司 Data processing method, device, system and computer-readable medium
CN109101646B (en) * 2018-08-21 2020-12-18 北京深瞐科技有限公司 Data processing method, device, system and computer readable medium
WO2020038254A1 (en) * 2018-08-23 2020-02-27 杭州海康威视数字技术股份有限公司 Image processing method and apparatus for target recognition
US11487966B2 (en) 2018-08-23 2022-11-01 Hangzhou Hikvision Digital Technology Co., Ltd. Image processing method and apparatus for target recognition
CN109345522A (en) * 2018-09-25 2019-02-15 北京市商汤科技开发有限公司 A kind of picture quality screening technique and device, equipment and storage medium
CN109522950B (en) * 2018-11-09 2022-04-22 网易传媒科技（北京）有限公司 Image scoring model training method and device and image scoring method and device
CN109615620A (en) * 2018-11-30 2019-04-12 腾讯科技（深圳）有限公司 The recognition methods of compression of images degree, device, equipment and computer readable storage medium
CN109615620B (en) * 2018-11-30 2021-01-08 腾讯科技（深圳）有限公司 Image compression degree identification method, device, equipment and computer readable storage medium
CN109815465B (en) * 2018-12-19 2023-11-17 平安科技（深圳）有限公司 Deep learning-based poster generation method and device and computer equipment
CN109784230A (en) * 2018-12-29 2019-05-21 中国科学院重庆绿色智能技术研究院 A kind of facial video image quality optimization method, system and equipment
CN109871780B (en) * 2019-01-28 2023-02-10 中国科学院重庆绿色智能技术研究院 Face quality judgment method and system and face identification method and system
CN109871780A (en) * 2019-01-28 2019-06-11 中国科学院重庆绿色智能技术研究院 A kind of face quality decision method, system and face identification method, system
CN109918885A (en) * 2019-03-12 2019-06-21 苏州宏裕千智能设备科技有限公司 A kind of onboard system multi-user access method and device
CN109977815A (en) * 2019-03-13 2019-07-05 上海商汤智能科技有限公司 Image quality evaluating method and device, electronic equipment, storage medium
CN109948564A (en) * 2019-03-25 2019-06-28 四川川大智胜软件股份有限公司 It is a kind of based on have supervision deep learning quality of human face image classification and appraisal procedure
CN110084130A (en) * 2019-04-03 2019-08-02 深圳鲲云信息科技有限公司 Face screening technique, device, equipment and storage medium based on multiple target tracking
CN109993150A (en) * 2019-04-15 2019-07-09 北京字节跳动网络技术有限公司 The method and apparatus at age for identification
CN110807757A (en) * 2019-08-14 2020-02-18 腾讯科技（深圳）有限公司 Image quality evaluation method and device based on artificial intelligence and computer equipment
CN110807757B (en) * 2019-08-14 2023-07-25 腾讯科技（深圳）有限公司 Image quality evaluation method and device based on artificial intelligence and computer equipment
CN110674925A (en) * 2019-08-29 2020-01-10 厦门大学 No-reference VR video quality evaluation method based on 3D convolutional neural network
CN110674925B (en) * 2019-08-29 2023-04-18 厦门大学 No-reference VR video quality evaluation method based on 3D convolutional neural network
CN110751043A (en) * 2019-09-19 2020-02-04 平安科技（深圳）有限公司 Face recognition method and device based on face visibility and storage medium
CN110751043B (en) * 2019-09-19 2023-08-22 平安科技（深圳）有限公司 Face recognition method and device based on face visibility and storage medium
CN110879981A (en) * 2019-11-14 2020-03-13 深圳市华付信息技术有限公司 Method and device for evaluating quality of key points of human face, computer equipment and storage medium
CN110956615B (en) * 2019-11-15 2023-04-07 北京金山云网络技术有限公司 Image quality evaluation model training method and device, electronic equipment and storage medium
CN110956615A (en) * 2019-11-15 2020-04-03 北京金山云网络技术有限公司 Image quality evaluation model training method and device, electronic equipment and storage medium
CN111199186A (en) * 2019-12-03 2020-05-26 恒大智慧科技有限公司 Image quality scoring model training method, device, equipment and storage medium
CN111199197A (en) * 2019-12-26 2020-05-26 深圳市优必选科技股份有限公司 Image extraction method and processing equipment for face recognition
CN111199197B (en) * 2019-12-26 2024-01-02 深圳市优必选科技股份有限公司 Image extraction method and processing equipment for face recognition
CN111160219B (en) * 2019-12-26 2022-04-26 深圳云天励飞技术股份有限公司 Object integrity evaluation method and device, electronic equipment and storage medium
CN111160219A (en) * 2019-12-26 2020-05-15 深圳云天励飞技术有限公司 Object integrity evaluation method and device, electronic equipment and storage medium
CN111241925A (en) * 2019-12-30 2020-06-05 新大陆数字技术股份有限公司 Face quality evaluation method, system, electronic equipment and readable storage medium
CN113129252A (en) * 2019-12-30 2021-07-16 Tcl集团股份有限公司 Image scoring method and electronic equipment
CN111241925B (en) * 2019-12-30 2023-08-18 新大陆数字技术股份有限公司 Face quality assessment method, system, electronic equipment and readable storage medium
CN111640099A (en) * 2020-05-29 2020-09-08 北京金山云网络技术有限公司 Method and device for determining image quality, electronic equipment and storage medium
CN111695522B (en) * 2020-06-15 2022-10-18 重庆邮电大学 In-plane rotation invariant face detection method and device and storage medium
CN111695522A (en) * 2020-06-15 2020-09-22 重庆邮电大学 In-plane rotation invariant face detection method and device and storage medium
CN111738179A (en) * 2020-06-28 2020-10-02 湖南国科微电子股份有限公司 Method, device, equipment and medium for evaluating quality of face image
CN111862040A (en) * 2020-07-20 2020-10-30 中移（杭州）信息技术有限公司 Portrait picture quality evaluation method, device, equipment and storage medium
CN111862040B (en) * 2020-07-20 2023-10-31 中移（杭州）信息技术有限公司 Portrait picture quality evaluation method, device, equipment and storage medium
CN112347849A (en) * 2020-09-29 2021-02-09 咪咕视讯科技有限公司 Video conference processing method, electronic device and storage medium
CN112347849B (en) * 2020-09-29 2024-03-26 咪咕视讯科技有限公司 Video conference processing method, electronic equipment and storage medium
CN112529845A (en) * 2020-11-24 2021-03-19 浙江大华技术股份有限公司 Image quality value determination method, image quality value determination device, storage medium, and electronic device

Similar Documents

Publication Publication Date Title
CN108269250A (en) Method and apparatus based on convolutional neural networks assessment quality of human face image
CN108596277B (en) Vehicle identity recognition method and device and storage medium
US10860879B2 (en) Deep convolutional neural networks for crack detection from image data
Zhu et al. Modified densenet for automatic fabric defect detection with edge computing for minimizing latency
CN106295601B (en) A kind of improved Safe belt detection method
Salimi et al. Visual-based trash detection and classification system for smart trash bin robot
CN108875600A (en) A kind of information of vehicles detection and tracking method, apparatus and computer storage medium based on YOLO
US7136524B1 (en) Robust perceptual color identification
CN109978918A (en) A kind of trajectory track method, apparatus and storage medium
CN104700078B (en) A kind of robot scene recognition methods based on scale invariant feature extreme learning machine
CN104484658A (en) Face gender recognition method and device based on multi-channel convolution neural network
CN107341505B (en) Scene classification method based on image significance and Object Bank
CN106709528A (en) Method and device of vehicle reidentification based on multiple objective function deep learning
CN112613454A (en) Electric power infrastructure construction site violation identification method and system
CN105303163B (en) A kind of method and detection device of target detection
CN109815945A (en) A kind of respiratory tract inspection result interpreting system and method based on image recognition
CN104751186A (en) Iris image quality classification method based on BP (back propagation) network and wavelet transformation
CN107886110A (en) Method for detecting human face, device and electronic equipment
Lian et al. Automatic visual inspection for printed circuit board via novel Mask R-CNN in smart city applications
CN107369086A (en) A kind of identity card stamp system and method
CN106682604B (en) Blurred image detection method based on deep learning
Laptev et al. Visualization system for fire detection in the video sequences
CN108694398A (en) A kind of image analysis method and device
Mahmood et al. A new hand gesture recognition system using artificial neural network
CN113902919A (en) Target detection method and system based on double-flow network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20180710

RJ01 Rejection of invention patent application after publication