CN109359541A - A kind of sketch face identification method based on depth migration study - Google Patents

A kind of sketch face identification method based on depth migration study Download PDF

Info

Publication number
CN109359541A
CN109359541A CN201811084907.5A CN201811084907A CN109359541A CN 109359541 A CN109359541 A CN 109359541A CN 201811084907 A CN201811084907 A CN 201811084907A CN 109359541 A CN109359541 A CN 109359541A
Authority
CN
China
Prior art keywords
image
sketch
training
human face
softmax
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811084907.5A
Other languages
Chinese (zh)
Inventor
宋建新
王欣欣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing Post and Telecommunication University
Nanjing University of Posts and Telecommunications
Original Assignee
Nanjing Post and Telecommunication University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing Post and Telecommunication University filed Critical Nanjing Post and Telecommunication University
Priority to CN201811084907.5A priority Critical patent/CN109359541A/en
Publication of CN109359541A publication Critical patent/CN109359541A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches

Abstract

The invention discloses a kind of sketch face identification methods based on depth migration study, comprising the following steps: step 1: establishes depth convolutional neural networks model；Step 2: image in CUFSF sketch image library is pre-processed；Step 3: the LFW image library comprising extensive natural human face photo is pre-processed, as initial training sample training network, obtains training pattern；Training pattern: being moved to the network for being used for sketch images human face photo by step 4, obtains pre-training model；Step 5: establish by reference picture, positive sample image, negative sample image construction triple image；Step 6: using triple image group as the input of pre-training model, loss function is minimized using back-propagation algorithm, training obtains final target training pattern；Step 7: the target training pattern obtained with test set testing procedure six carries out the recognition of face of sketch image.This method has the advantages that the recognition accuracy of sketch facial image is high.

Description

A kind of sketch face identification method based on depth migration study
Technical field
The present invention relates to image recognitions and depth learning technology field, are related to a kind of sketch people based on depth migration study Face recognition method.
Background technique
One important application of recognition of face is exactly to assist enforcement.Realize the automatic of Ministry of Public Security's mug shot database Retrieval, can reduce rapidly the range of suspect.However in most cases, the photo of suspect can not obtain.Do not supervising Under control or the unsharp scene of monitored picture, optimal substitute is namely based on the description of eye witness, and by portrait, expert is drawn Sketch images.Therefore, with sketch images come the individual in Auto-matching picture data library.But the facial detail of suspect's sketch is often Be inaccurate, human face five-sense-organ feature relative position reliability it is not high, while there is a possibility that being exaggerated in five features.Therefore, The main challenge of suspect's sketch recognition of face is that the mode drawn leads to the difference between sketch and photo there are facial detail Different, face the relative positions difference etc. different with difference, the data source of appearance.
There are two main classes for current sketch recognition methods: first is that will first carry out again after human face sketch and photograph image conversion With identification；Second is that not needing the conversion of sketch and photo, match cognization is directly carried out.
The conversion of human face sketch and photograph image.It is the image under both of which because of sketch and photo itself, the two has one Fixed difference is identified again after photo is first converted to pseudo- sketch.Once the image from different modalities is synthesized to same A mode, so that it may which identification is solved the problems, such as using traditional face identification method.But often composograph is than identifying more Complicated task；In addition, due to sketch images on facial contour and facial characteristics there are certain deformation and exaggeration, and eye Position and Hp position etc. may be inaccurate, even if being synthesized to same mode, the accuracy rate of identification still with common natural person The result of face identification differs greatly.
The direct matching of human face sketch and photograph image.It is straight to sketch and photo face in sketch face recognition process It connects and extracts identifiable feature progress match cognization, and without conversion between the two.Some basic Feature Descriptors include SIFT, Gabor, Hog and LBP, also some is the improvement based on previous methods, these use the feature conduct of engineer Method with measurement is not only time-consuming, but also generalization ability is weaker.In addition application is mostly mentioned there are also the feature based on deep learning Method is taken, such as: stacking-type is from coding, depth confidence network and neural network.But it is limited image data to tend not to Enough samples are provided for training, easily cause over-fitting.Thus need to invent it is a kind of can be effective under the premise of not over-fitting Realize the sketch face knowledge that directly matches and the recognition accuracy of sketch facial image high of the sketch facial image with photograph image Other method.
Summary of the invention
The object of the present invention is to provide one kind can effectively realize sketch facial image and photo under the premise of not over-fitting The direct matching of image, and the high sketch recognition of face based on depth migration study of recognition accuracy of sketch facial image Method.
To achieve the above object, present invention employs following technical solutions: described a kind of based on depth migration study Sketch face identification method, comprising the following steps:
Step 1: the AlexNet depth convolutional neural networks model for being used for feature extraction is established；
Step 2: to the face in CUFSF (CUHK Face Sketch FERET Database) human face sketch image library Sketch image and its natural human face photo are pre-processed, and the image library is by facial image to constituting, and described image is to by same The natural human face photo of people and its corresponding sketch images human face photo composition, and image is to the choosing with positive sample image later It takes, pretreated specific step is as follows:
Step (21): color image is changed into single pass gray level image；
Step (22): detecting face using Viola-Jones facial feature detection device and extracts the coordinate of eyes and nose；
Step (23): expanding cut out areas on the basis of conventional face cuts out, so that the entirely hairline of face after cutting Line, neck and ear are visible；
Step (24): by the face image standardization after cutting to the template area of a predefined resolution ratio；
Step (25): the image in pretreated facial image database is divided into training set and test set, training set is used In the depth convolutional neural networks model that training is established by step 1, test set is used for assessment models；
Step 3: to the LFW database comprising extensive natural human face photo, being pre-processed also according to step 2, Using the image in the pretreated facial image database as initial training sample training network, training pattern is obtained；
Step 4: the training pattern that step 3 is obtained moves to the network for being used for sketch images human face photo, is based on The pre-training model of sketch images human face photo training；
Step 5: it is established based on the training set in step 2 by reference picture, positive sample image, negative sample image construction Triple image, the method is as follows: randomly select a sketch image and be used as referring to image, due to sketch facial image and natural person Face photo is that occur in the form of image pair, so by unique corresponding natural human face photo as positive sample image；By reference Image with natural face photo comparison, traverses the natural human face photo in training set, is ranked up by similarity, takes and join respectively Examine image not and be same people and the highest natural human face photo of similarity as negative sample image；
Step 6: the input for the pre-training model that triple image group is obtained as step 4 is calculated using backpropagation Method minimizes loss function, and training obtains the final target training pattern based on depth convolutional neural networks；
Step 7: the target training pattern that the test set testing procedure six obtained with step 2 obtains carries out sketch image Recognition of face.
Further, a kind of sketch face identification method based on depth migration study above-mentioned, in which: in step 1 In, the structure of depth convolutional neural networks model includes: five layers of convolutional layer, two layers of full articulamentum and one layer based on softmax The Nonlinear Classification layer of method.
Further, a kind of sketch face identification method based on depth migration study above-mentioned, in which: in step 4 In, training pattern is moved to the method for being used for the network of sketch images human face photo are as follows: keep convolutional layer parameter constant, will connect entirely It connects layer parameter and parameter initialization is carried out with xavier algorithm, parameter is existed in an uniform manner using xavier algorithmIn the range of initialize so that each layer of output variance is equal as far as possible, wherein layer where parameter is defeated Entering dimension is n, and output dimension is m, thus obtains the pre-training model based on the training of sketch images human face photo.
Further, a kind of sketch face identification method based on depth migration study above-mentioned, in which: in step 5 In, INNegative sample image chooses the negative sample photo for meeting following condition:
Wherein,Indicate i-th of sample in all negative sample set, F (IR)、It is I respectivelyR,In convolution The output of the full articulamentum of the last one in convolutional neural networks, IRFor referring to image, INBe negative sample image.
Further, a kind of sketch face identification method based on depth migration study above-mentioned, in which: in step 6 In, the training of target training pattern the following steps are included:
Step (61): successively using triple image as the input of model, sample characteristics are obtained by propagated forward algorithm Vector calculates loss function；It is jumped if meeting the condition of convergence and executes step (63), it is no to then follow the steps (62)；
Step (62): loss function is calculated to the gradient of each parameter by back-propagation algorithm, uses gradient descent method Each layer parameter for updating convolutional neural networks, continues to execute step (61) later；
Step (63): after meeting the condition of convergence in step (61), judging whether to reach frequency of training, if reaching training time Number thens follow the steps (64), and otherwise frequency of training adds 1, jumps and executes step (61)；
Step (64): target training pattern of the output based on depth convolutional neural networks.
Further, a kind of sketch face identification method based on depth migration study above-mentioned, in which: in step (61) In, loss function is specifically expressed as follows:
Ltotal=Ltirplet+Lpairs+Lsoftmax
Wherein, Ltirplet、LpairsAnd LsoftmaxIt respectively indicates as follows:
Triple loss LtirpletIt is a kind of a kind of loss function that neural network is trained using triple；Network Output by f (I) ∈ RdIt indicates, input picture I is embedded into d dimension Euclidean space by it；Triple loss LtirpletIt is fixed Justice are as follows:
Wherein, F (IR)、F(IP)、F(IN) it is I respectivelyR,IP,INThe last one full articulamentum in convolutional neural networks Output, IRFor referring to image, IPBe positive sample image, INBe negative sample image, and m is defined in Euclidean space between positive negative sample The boundary of minimum rate, all triple images of T ' expression；
The LpairsIt is given image IRDescription and its positive sample IPDescription between Euclidean distance it is flat Fang He, the LpairsIs defined as:
Wherein, F (IR)、F(IP) it is I respectivelyR,IPThe output of the last one full articulamentum, I in convolutional neural networksRFor Referring to image, IPBe positive sample image, all triple images of T ' expression；
The LsoftmaxIt is the softmax loss of given softmax output, the LsoftmaxIs defined as:
Wherein,It is the probability that i-th of sample belongs to k-th of classification in softmax,What is indicated is in softmax I-th of sample belongs to the true probability of k-th of classification, and N indicates classification number.
Further, a kind of sketch face identification method based on depth migration study above-mentioned, in which: in step (62) In, the design parameter update method of convolutional neural networks parameter isIn order to preferably use learning rate α The speed of control parameter, the setting method of α are as follows:
α*For the learning rate used when each round optimization, α is the initial learning rate being previously set, and decay_rate is decaying Coefficient, global_step are the number of iterations, and decay_steps is the rate of decay.
Further, a kind of sketch face identification method based on depth migration study above-mentioned, in which: in step 7 In, the specific test method is as follows for target training pattern:
Step (71): the triple image constituted based on test set is input to target training pattern, obtains test set The feature vector library of image；
Step (72): softmax classifier is used, matched natural person is found from the sketch image of test set Face image；Wherein softmax is defined as:
Wherein, pikIndicate sketch image xiBelong to the probability of k-th of natural human face photo, wiWith wjEqual presentation class device mould Shape parameter, T indicate that transposition, m are the number of sketch image in test set；
Step (73): by pikAccording to being ranked up from big to small, and if xiIt is the natural human face photo appearance of the same person In preceding k classification, then successfully identifies, be denoted as yi=1, otherwise yi=0；
Step (74): the discrimination for obtaining model isM is the number of sketch image in test set.
Through the implementation of the above technical solution, the beneficial effects of the present invention are: the present invention has abandoned traditional direct sketch Image recognition of face utilizes the nature for being equally face using the conclusion formula transfer learning strategy indicated with deep learning Human face photo learns the initial model in source domain, then by the model parameter learnt of initial model be transferred to aiming field into Row matching sketch-photograph image, while the input of model is suitable to choosing in facial image in sketch-photo in aiming field Triple image quickly obtains the model of fit of training data；This method can accelerate the convergence rate of model, moreover it is possible to have compared with High generalization ability, while can also have very high recognition accuracy, the realization of this method is directly to utilize existing recognition of face Frame, in the good model of extensive human face photo pre-training, using sketch image to the triple data formed as The sample characteristics under heterogeneous mode are extracted in input, so that representative feature can also be extracted in the case where small sample amount by realizing, Simultaneously in the case where model second training also can obtain identification sketch image classifier, in addition training neural network when It waits, not only considers the relative distance between positive sample pair and negative sample pair, but also absolute between positive sample pair by minimizing Distance is also obviously improved the identification accuracy of sample while accelerating the convergence rate of model；This method is with less Training needed for data, the direct matching of sketch facial image and photograph image can be effectively realized under the premise of not over-fitting, And the recognition accuracy of sketch facial image is high, this also indicates that the transfer learning method for executing Cross-modality identification Validity.
Detailed description of the invention
Fig. 1 is a kind of flow diagram of sketch face identification method based on depth migration study of the present invention.
Fig. 2 is convolutional neural networks block diagram.
Fig. 3 is the pretreatment process figure of facial image.
Fig. 4 is the facial image after conventional pretreatment.
Fig. 5 is the sketch image after pretreatment.
Fig. 6 is the Optimizing Flow schematic diagram of deep neural network.
Fig. 7 is influence schematic diagram of the loss function to triple image.
Specific embodiment
The present invention will be further explained below with reference to the attached drawings and specific examples, and following embodiment is only used for clearer Ground illustrates technical solution of the present invention, and not intended to limit the protection scope of the present invention.
The present invention provides a kind of sketch face identification method based on depth migration study, with reference to the accompanying drawings and examples The invention will be further described.The deep learning network for initially setting up face physioprints for identification, with comprising extensive The library LFW of facial image is trained, and obtains the initial model that can obtain face generic features, as shown in figure 1 shown in [101]； Then, using transfer learning strategy, original model parameter is moved to the deep learning network for being used for human face sketch image, is such as schemed In 1 shown in [105]；The training of object module is carried out by [102] [103] [104] [106] of Fig. 1 again；[107] step is realized Identification to sketch facial image, specific implementation step are as follows:
Step 1: the AlexNet depth convolutional neural networks model for being used for feature extraction is established, specific network structure is such as Shown in Fig. 2；Wherein, the structure of depth convolutional neural networks model includes: five layers of convolutional layer, two layers of full articulamentum and one layer Nonlinear Classification layer based on softmax method；Entire depth convolutional neural networks build and optimization process passes through What TensorFlow was realized；
Step 2: to the sketch in CUFSF (CUHK Face Sketch FERET Database) human face sketch image library Facial image and its natural human face photo are pre-processed, and the facial image database is by image to constituting, and described image is to by same The natural human face photo of people and its corresponding sketch images human face photo composition；Pretreated specific step is as follows, referring to Fig. 3 institute Show:
Step (21): color image is changed into single pass gray level image by [301]；
Step (22): the coordinate that [302] and [303] complete face key point extracts, and uses Viola-Jones facial characteristics Detector detection face and the coordinate for extracting eyes and nose；
Step (23): [304] realize the cutting of face, expand cut out areas on the basis of conventional face cuts out, so that The hair line, neck and ear of entire face are visible after cutting；
Step (24): [305] carry out the scale calibration of face, and the face image standardization after cutting is predetermined to one The template area of adopted resolution ratio；
Step (25): the image in pretreated facial image database is divided into training set and test set by [306], and really It protects the natural human face photo that sketch image cannot be corresponding to separate, the depth convolution that training set is established for training by step 1 Neural network model, test set are used for assessment models；
Compared to traditional facial pretreatment, why increasing the cut out areas of face is the witness because of scene of a crime It can remember the details of more external facial features rather than face, so in sketch recognition of face, it is more external special Sign helps to improve precision, and the image after specific pretreatment is as shown in Figure 4, Figure 5；
Step 3: to the LFW database comprising extensive natural human face photo, being pre-processed also according to step 2, Using the image in the pretreated image library as initial training sample training network, training pattern is obtained；
Step 4: the training pattern that step 3 is obtained moves to the network for being used for sketch images human face photo, is based on The pre-training model of sketch images human face photo training；
Wherein, training pattern is moved to the method for being used for the network of sketch images human face photo are as follows: keep convolution layer parameter It is constant, layer parameter will be connected entirely with xavier algorithm and carry out parameter initialization, and use xavier algorithm by parameter with uniform side Formula existsIn the range of initialize so that each layer of output variance is equal as far as possible, wherein layer where parameter Input dimension be n, output dimension is m, thus obtains the pre-training model trained based on sketch images human face photo；
Step 5: it is established based on the training set in step 2 by reference picture, positive sample image, negative sample image construction Triple image, wherein using randomly select sketch image as referring to image；Since sketch image and natural face are shone Piece is that occur in the form of image pair, so obtaining unique corresponding natural human face photo as positive sample image；It will be referring to figure As with natural face photo comparison, traversing the natural human face photo in training set respectively, being ranked up by similarity, take and refer to Image be not same people and the highest natural human face photo of similarity as negative sample image；
Wherein, INNegative sample image chooses the negative sample photo for meeting following condition:
Wherein,Indicate i-th of sample in all negative sample set, F (IR)、It is I respectivelyR,In convolution The output of the full articulamentum of the last one in neural network, IRFor referring to image, INBe negative sample image；
Step 6: the input for the pre-training model that triple image group is obtained as step 4 is calculated using backpropagation Method minimizes loss function, and training obtains the final target training pattern based on depth convolutional neural networks；
Wherein, the training of target training pattern includes the following steps (as shown in Figure 6):
Step (61): [601] obtain sample by propagated forward algorithm successively using triple image as the input of model Feature vector calculates loss function；It is jumped if meeting the condition of convergence and executes step (63), it is no to then follow the steps (62)；
Wherein, loss function is specifically expressed as follows:
Ltotal=Ltirplet+Lpairs+Lsoftmax
As shown in fig. 7, [701] and [702] respectively indicate the distance between training front and back sample image, that is, use the loss Function can reduce IRWith IPThe distance between, and increase IRWith INThe distance between；In order to identify the identity of people, in addition add Softmax loss function, each identity attribute final classification is n different identity by it；Wherein, Ltirplet、LpairsWith LsoftmaxIt respectively indicates as follows:
Triple loss LtirpletIt is a kind of a kind of loss function that neural network is trained using triple；Network Output by f (I) ∈ RdIt indicates, input picture I is embedded into d dimension Euclidean space by it；Triple loss LtirpletIt is fixed Justice are as follows:
Wherein, F (IR)、F(IP)、F(IN) it is I respectivelyR,IP,INThe last one full articulamentum in convolutional neural networks Output, IRFor referring to image, IPBe positive sample image, INBe negative sample image, and m is defined in Euclidean space between positive negative sample The boundary of minimum rate, all triple images of T ' expression；
The LpairsIt is given image IRDescription and its positive sample IPDescription between Euclidean distance it is flat Fang He, the LpairsIs defined as:
Wherein, F (IR)、F(IP) it is I respectivelyR,IPThe output of the last one full articulamentum, I in convolutional neural networksRFor Referring to image, IPBe positive sample image, all triple images of T ' expression；
The LsoftmaxIt is the softmax loss of given softmax output, the LsoftmaxIs defined as:
Wherein,It is the probability that i-th of sample belongs to k-th of classification in softmax,What is indicated is in softmax I-th of sample belongs to the true probability of k-th of classification, and N indicates classification number；
Step (62): [602] calculate loss function to the gradient of each parameter, using under gradient by back-propagation algorithm Drop method updates each layer parameter of convolutional neural networks, continues to execute step (61) later；
Wherein, the design parameter update method of convolutional neural networks parameter isIn order to preferably make With the speed of learning rate α control parameter, the setting method of α are as follows:
α*For the learning rate used when each round optimization, α is the initial learning rate being previously set, and decay_rate is decaying Coefficient, global_step are the number of iterations, and decay_steps is the rate of decay, can be first using larger by this function Learning rate quickly to obtain one and solve than preferably, then as iteration continue gradually reduce learning rate so that model is more Add stabilization；
Step (63): after meeting the condition of convergence in step (61), [603] judge whether to reach frequency of training, if reaching Frequency of training thens follow the steps (64), and otherwise frequency of training adds 1, jumps and executes step (61)；
Step (64): [604] export the target training pattern based on depth convolutional neural networks.
Step 7: the target training pattern that test set testing procedure six obtains is obtained with step 2, carries out sketch image Recognition of face；
Wherein, the specific test method is as follows for target training pattern:
Step (71): the triple image constituted based on test set is input to target training pattern, obtains test set The feature vector library of image；
Step (72): softmax classifier is used, matched natural person is found from the sketch image of test set Face image；Wherein softmax is defined as:
Wherein, pikIndicate sketch image xiBelong to the probability of k-th of natural human face photo, wiWith wjEqual presentation class device mould Shape parameter, T indicate that transposition, m are the number of sketch image in test set；
Step (73): by pikAccording to being ranked up from big to small, and if xiIt is the natural human face photo appearance of the same person In preceding k classification, then successfully identifies, be denoted as yi=1, otherwise yi=0；
Step (74): the discrimination for obtaining model isM is the number of sketch image in test set.
The invention has the advantages that the present invention has abandoned traditional direct sketch image recognition of face, using has depth Learn the conclusion formula transfer learning strategy indicated, learns the introductory die in source domain using being equally the natural human face photo of face Then the model parameter of initial model learnt is transferred to aiming field and carries out matching sketch-photograph image, while target by type The input of model is quickly to obtain training data to suitable triple image is chosen in facial image in sketch-photo in domain Model of fit；This method can accelerate the convergence rate of model, moreover it is possible to generalization ability with higher, while can also have very high Recognition accuracy, the realization of this method is directly using existing recognition of face frame, with extensive human face photo In the good model of pre-training, it is special that the sample under heterogeneous mode is extracted as input to the triple data formed using sketch image Sign, so that representative feature can also be extracted in the case where small sample amount by realizing, while in the case where model second training Also the classifier of identification sketch image can be obtained, in addition when training neural network, not only considers positive sample pair and negative sample This relative distance between, but also the absolute distance between positive sample pair will be minimized, in the convergence rate for accelerating model While also the identification accuracy of sample is obviously improved；This method is with data needed for less training, in not over-fitting Under the premise of can effectively realize the direct matching of sketch facial image and photograph image, and it is accurate to reach higher identification Rate, this also indicates that the validity of the transfer learning method for executing Cross-modality identification.

Claims (8)

1. a kind of sketch face identification method based on depth migration study, it is characterised in that: the following steps are included:
Step 1: the AlexNet depth convolutional neural networks model for being used for feature extraction is established；
Step 2: to the human face sketch in CUFSF (CUHK Face Sketch FERET Database) human face sketch image library Image and its natural human face photo are pre-processed, and the image library is by facial image to constituting, and described image is to by same people's Natural human face photo and its corresponding sketch images human face photo composition, pretreated specific step is as follows:
Step (21): color image is changed into single pass gray level image；
Step (22): detecting face using Viola-Jones facial feature detection device and extracts the coordinate of eyes and nose；
Step (23): expanding cut out areas on the basis of conventional face cuts out, so that entirely hair line, the neck of face after cutting Portion and ear are visible；
Step (24): by the face image standardization after cutting to the template area of a predefined resolution ratio；
Step (25): the image in pretreated facial image database is divided into training set and test set, training set is for instructing Practice the depth convolutional neural networks model established by step 1, test set is used for assessment models；
Step 3: it to the LFW database comprising extensive natural human face photo, is pre-processed also according to step 2, by this Image in pretreated facial image database obtains training pattern as initial training sample training network；
Step 4: the training pattern that step 3 is obtained moves to the network for being used for sketch images human face photo, obtains based on sketch Draw the pre-training model of human face photo training；
Step 5: based in step 2 training set establish by reference picture, positive sample image, negative sample image construction ternary Group image, the method is as follows: randomly select a sketch facial image and be used as referring to image, due to sketch facial image and natural person Face photo is that occur in the form of image pair, so by unique corresponding natural human face photo as positive sample image, by reference Image with natural face photo comparison, traverses the natural human face photo in training set, is ranked up by similarity, takes and join respectively Examine image not and be same people and the highest natural human face photo of similarity as negative sample image；
Step 6: the input for the pre-training model that triple image is obtained as step 4 utilizes back-propagation algorithm minimum Change loss function, training obtains the final target training pattern based on depth convolutional neural networks；
Step 7: the target training pattern that the test set testing procedure six obtained with step 2 obtains carries out the people of sketch image Face identification.
2. a kind of sketch face identification method based on depth migration study according to claim 1, it is characterised in that: In step 1, the structure of depth convolutional neural networks model includes: that five layers of convolutional layer, two layers of full articulamentum and one layer are based on The Nonlinear Classification layer of softmax method.
3. a kind of sketch face identification method based on depth migration study according to claim 1, it is characterised in that: In step 4, training pattern is moved to the method for being used for the network of sketch images human face photo are as follows: convolutional layer parameter constant is kept, Layer parameter being connected entirely, parameter initialization being carried out with xavier algorithm, parameter will be existed in an uniform manner using xavier algorithmIn the range of initialize so that each layer of output variance is equal as far as possible, wherein layer where parameter is defeated Entering dimension is n, and output dimension is m, thus obtains the pre-training model based on the training of sketch images human face photo.
4. a kind of sketch face identification method based on depth migration study according to claim 1, it is characterised in that: In step 5, INNegative sample image chooses the negative sample photo for meeting following condition:
Wherein,Indicate i-th of sample in all negative sample set, F (IR)、It is I respectivelyR,In convolutional Neural The output of the full articulamentum of the last one in network, IRFor referring to image, INBe negative sample image.
5. a kind of sketch face identification method based on depth migration study according to claim 1, it is characterised in that: In step 6, the training of target training pattern the following steps are included:
Step (61): successively using triple image as the input of model, obtaining sampling feature vectors by propagated forward algorithm, Calculate loss function；It is jumped if meeting the condition of convergence and executes step (63), it is no to then follow the steps (62)；
Step (62): loss function is calculated to the gradient of each parameter by back-propagation algorithm, is updated using gradient descent method Each layer parameter of convolutional neural networks continues to execute step (61) later；
Step (63): after meeting the condition of convergence in step (61), judge whether to reach frequency of training, if reaching frequency of training It executes step (64), otherwise frequency of training adds 1, jumps and executes step (61)；
Step (64): target training pattern of the output based on depth convolutional neural networks.
6. a kind of sketch face identification method based on depth migration study according to claim 5, it is characterised in that: In step (61), loss function is specifically expressed as follows:
Ltotal=Ltirplet+Lpairs+Lsoftmax
Wherein, Ltirplet、LpaitsAnd LsoftmaxIt respectively indicates as follows:
Triple loss LtirpletIt is a kind of a kind of loss function that neural network is trained using triple；Network it is defeated Out by f (I) ∈ RdIt indicates, input picture I is embedded into d dimension Euclidean space by it；Triple loss LtirpletDefinition Are as follows:
Wherein, F (IR)、F(IP)、F(IN) it is I respectivelyR,IP,IThe output of N the last one full articulamentum in convolutional neural networks, IRFor referring to image, IPBe positive sample image, INBe negative sample image, and m defines in Euclidean space minimum ratio between positive negative sample The boundary of rate, all triple images of T ' expression；
The LpairsIt is given image IRDescription and its positive sample IPDescription son between Euclidean distance quadratic sum, The LpairsIs defined as:
Wherein, F (IR)、F(IP) it is I respectivelyR,IPThe output of the last one full articulamentum, I in convolutional neural networksRFor reference Image, IPBe positive sample image, all triple images of T ' expression；
The LsoftmaxIt is the softmax loss of given softmax output, the LsoftmaxIs defined as:
Wherein,It is the probability that i-th of sample belongs to k-th of classification in softmax,What is indicated is i-th in softmax Sample belongs to the true probability of k-th of classification, and N indicates classification number.
7. a kind of sketch face identification method based on depth migration study according to claim 5, it is characterised in that: In step (62), the design parameter update method of convolutional neural networks parameter isIn order to preferably use The speed of learning rate α control parameter, the setting method of α are as follows:
α*For the learning rate used when each round optimization, α is the initial learning rate being previously set, and decay_rate is attenuation coefficient, Global_step is the number of iterations, and decay_steps is the rate of decay.
8. a kind of sketch face identification method based on depth migration study according to claim 1, it is characterised in that: In step 7, the specific test method is as follows for target training pattern:
Step (71): the triple image constituted based on test set is input to target training pattern, obtains test set image Feature vector library；
Step (72): using softmax classifier, and matched natural face figure is found from the sketch image of test set Picture；Wherein softmax is defined as:
Wherein, pikIndicate sketch image xiBelong to the probability of k-th of natural human face photo, wiWith wjEqual presentation class device model ginseng Number, T indicate that transposition, m are the number of sketch image in test set；
Step (73): by pikAccording to being ranked up from big to small, and if xiIt is that the natural human face photo of the same person appears in preceding k It in a classification, then successfully identifies, is denoted as yi=1, otherwise yi=0；
Step (74): the discrimination for obtaining model isM is the number of sketch image in test set.
CN201811084907.5A 2018-09-17 2018-09-17 A kind of sketch face identification method based on depth migration study Pending CN109359541A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811084907.5A CN109359541A (en) 2018-09-17 2018-09-17 A kind of sketch face identification method based on depth migration study

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811084907.5A CN109359541A (en) 2018-09-17 2018-09-17 A kind of sketch face identification method based on depth migration study

Publications (1)

Publication Number Publication Date
CN109359541A true CN109359541A (en) 2019-02-19

Family

ID=65350952

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811084907.5A Pending CN109359541A (en) 2018-09-17 2018-09-17 A kind of sketch face identification method based on depth migration study

Country Status (1)

Country Link
CN (1) CN109359541A (en)

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109902733A (en) * 2019-02-22 2019-06-18 北京三快在线科技有限公司 The method, apparatus and storage medium of typing Item Information
CN109919934A (en) * 2019-03-11 2019-06-21 重庆邮电大学 A kind of liquid crystal display panel defect inspection method based on the study of multi-source domain depth migration
CN109947960A (en) * 2019-03-08 2019-06-28 南京信息工程大学 The more attribute Combined estimator model building methods of face based on depth convolution
CN110023989A (en) * 2017-03-29 2019-07-16 华为技术有限公司 A kind of generation method and device of sketch image
CN110097033A (en) * 2019-05-15 2019-08-06 成都电科智达科技有限公司 A kind of single sample face recognition method expanded based on feature
CN110148120A (en) * 2019-05-09 2019-08-20 四川省农业科学院农业信息与农村经济研究所 A kind of disease intelligent identification Method and system based on CNN and transfer learning
CN110210486A (en) * 2019-05-15 2019-09-06 西安电子科技大学 A kind of generation confrontation transfer learning method based on sketch markup information
CN110443162A (en) * 2019-07-19 2019-11-12 南京邮电大学 A kind of two-part training method for disguised face identification
CN110458647A (en) * 2019-07-05 2019-11-15 深圳壹账通智能科技有限公司 Product method for pushing, device, computer equipment and storage medium
CN110503154A (en) * 2019-08-27 2019-11-26 携程计算机技术（上海）有限公司 Method, system, electronic equipment and the storage medium of image classification
CN110532880A (en) * 2019-07-29 2019-12-03 深圳大学 Screening sample and expression recognition method, neural network, equipment and storage medium
CN110580726A (en) * 2019-08-21 2019-12-17 中山大学 Dynamic convolution network-based face sketch generation model and method in natural scene
CN110852292A (en) * 2019-11-18 2020-02-28 南京邮电大学 Sketch face recognition method based on cross-modal multi-task depth measurement learning
CN111738157A (en) * 2020-06-23 2020-10-02 平安科技（深圳）有限公司 Method and device for constructing data set of facial action units and computer equipment
WO2020228515A1 (en) * 2019-05-10 2020-11-19 北京京东尚科信息技术有限公司 Fake face recognition method, apparatus and computer-readable storage medium
WO2021036059A1 (en) * 2019-08-29 2021-03-04 深圳云天励飞技术有限公司 Image conversion model training method, heterogeneous face recognition method, device and apparatus
CN112633288A (en) * 2020-12-29 2021-04-09 杭州电子科技大学 Face sketch generation method based on drawing stroke guidance
CN112799511A (en) * 2021-01-28 2021-05-14 Oppo广东移动通信有限公司 Training method and device for graphic code generation model, electronic equipment and storage medium
CN113129208A (en) * 2019-12-31 2021-07-16 深圳云天励飞技术有限公司 Human face image generation method based on sketch and related product
CN113642481A (en) * 2021-08-17 2021-11-12 百度在线网络技术（北京）有限公司 Recognition method, training method, device, electronic equipment and storage medium
CN113722528A (en) * 2021-08-03 2021-11-30 南京邮电大学 Method and system for rapidly retrieving photos facing sketch
CN113743439A (en) * 2020-11-13 2021-12-03 北京沃东天骏信息技术有限公司 Target detection method, device and storage medium
CN115396831A (en) * 2021-05-08 2022-11-25 中国移动通信集团浙江有限公司 Interaction model generation method, device, equipment and storage medium
CN117830083A (en) * 2024-03-05 2024-04-05 昆明理工大学 Method and device for generating face sketch-to-face photo
CN117830083B (en) * 2024-03-05 2024-05-03 昆明理工大学 Method and device for generating face sketch-to-face photo

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030095701A1 (en) * 2001-11-19 2003-05-22 Heung-Yeung Shum Automatic sketch generation
JP2011060289A (en) * 2009-09-08 2011-03-24 Xiaogang Wang Face image synthesis method and system
CN103902991A (en) * 2014-04-24 2014-07-02 西安电子科技大学 Face recognition method based on forensic sketches
CN106096538A (en) * 2016-06-08 2016-11-09 中国科学院自动化研究所 Face identification method based on sequencing neural network model and device
CN106845330A (en) * 2016-11-17 2017-06-13 北京品恩科技股份有限公司 A kind of training method of the two-dimension human face identification model based on depth convolutional neural networks
CN106951840A (en) * 2017-03-09 2017-07-14 北京工业大学 A kind of facial feature points detection method
CN107945244A (en) * 2017-12-29 2018-04-20 哈尔滨拓思科技有限公司 A kind of simple picture generation method based on human face photo
CN108009528A (en) * 2017-12-26 2018-05-08 广州广电运通金融电子股份有限公司 Face authentication method, device, computer equipment and storage medium based on Triplet Loss
CN108154133A (en) * 2018-01-10 2018-06-12 西安电子科技大学 Human face portrait based on asymmetric combination learning-photo array method
CN108429753A (en) * 2018-03-16 2018-08-21 重庆邮电大学 A kind of matched industrial network DDoS intrusion detection methods of swift nature

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030095701A1 (en) * 2001-11-19 2003-05-22 Heung-Yeung Shum Automatic sketch generation
JP2011060289A (en) * 2009-09-08 2011-03-24 Xiaogang Wang Face image synthesis method and system
CN103902991A (en) * 2014-04-24 2014-07-02 西安电子科技大学 Face recognition method based on forensic sketches
CN106096538A (en) * 2016-06-08 2016-11-09 中国科学院自动化研究所 Face identification method based on sequencing neural network model and device
CN106845330A (en) * 2016-11-17 2017-06-13 北京品恩科技股份有限公司 A kind of training method of the two-dimension human face identification model based on depth convolutional neural networks
CN106951840A (en) * 2017-03-09 2017-07-14 北京工业大学 A kind of facial feature points detection method
CN108009528A (en) * 2017-12-26 2018-05-08 广州广电运通金融电子股份有限公司 Face authentication method, device, computer equipment and storage medium based on Triplet Loss
CN107945244A (en) * 2017-12-29 2018-04-20 哈尔滨拓思科技有限公司 A kind of simple picture generation method based on human face photo
CN108154133A (en) * 2018-01-10 2018-06-12 西安电子科技大学 Human face portrait based on asymmetric combination learning-photo array method
CN108429753A (en) * 2018-03-16 2018-08-21 重庆邮电大学 A kind of matched industrial network DDoS intrusion detection methods of swift nature

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
BONG-NAM KANG等: "Deep Convolutional Neural Network using Triplets of Faces, Deep Ensemble, and Score-level Fusion for Face Recognition", 《2017 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS》 *
CHRISTIAN GALEA等: "Matching Software-Generated Sketches to Face Photographs With a Very Deep CNN, Morphed Faces, and Transfer Learning", 《IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY》 *
刘霄翔: "异质人脸识别理论与方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110023989A (en) * 2017-03-29 2019-07-16 华为技术有限公司 A kind of generation method and device of sketch image
CN110023989B (en) * 2017-03-29 2021-06-01 华为技术有限公司 Sketch image generation method and device
CN109902733A (en) * 2019-02-22 2019-06-18 北京三快在线科技有限公司 The method, apparatus and storage medium of typing Item Information
CN109947960A (en) * 2019-03-08 2019-06-28 南京信息工程大学 The more attribute Combined estimator model building methods of face based on depth convolution
CN109947960B (en) * 2019-03-08 2024-01-23 南京信息工程大学 Face multi-attribute joint estimation model construction method based on depth convolution
CN109919934A (en) * 2019-03-11 2019-06-21 重庆邮电大学 A kind of liquid crystal display panel defect inspection method based on the study of multi-source domain depth migration
CN110148120A (en) * 2019-05-09 2019-08-20 四川省农业科学院农业信息与农村经济研究所 A kind of disease intelligent identification Method and system based on CNN and transfer learning
WO2020228515A1 (en) * 2019-05-10 2020-11-19 北京京东尚科信息技术有限公司 Fake face recognition method, apparatus and computer-readable storage medium
CN110097033A (en) * 2019-05-15 2019-08-06 成都电科智达科技有限公司 A kind of single sample face recognition method expanded based on feature
CN110097033B (en) * 2019-05-15 2023-04-07 成都电科智达科技有限公司 Single-sample face recognition method based on feature expansion
CN110210486A (en) * 2019-05-15 2019-09-06 西安电子科技大学 A kind of generation confrontation transfer learning method based on sketch markup information
CN110210486B (en) * 2019-05-15 2021-01-01 西安电子科技大学 Sketch annotation information-based generation countermeasure transfer learning method
CN110458647A (en) * 2019-07-05 2019-11-15 深圳壹账通智能科技有限公司 Product method for pushing, device, computer equipment and storage medium
CN110443162A (en) * 2019-07-19 2019-11-12 南京邮电大学 A kind of two-part training method for disguised face identification
CN110443162B (en) * 2019-07-19 2022-08-30 南京邮电大学 Two-stage training method for disguised face recognition
CN110532880A (en) * 2019-07-29 2019-12-03 深圳大学 Screening sample and expression recognition method, neural network, equipment and storage medium
CN110532880B (en) * 2019-07-29 2022-11-22 深圳大学 Sample screening and expression recognition method, neural network, device and storage medium
CN110580726A (en) * 2019-08-21 2019-12-17 中山大学 Dynamic convolution network-based face sketch generation model and method in natural scene
CN110580726B (en) * 2019-08-21 2022-10-04 中山大学 Dynamic convolution network-based face sketch generation model and method in natural scene
CN110503154A (en) * 2019-08-27 2019-11-26 携程计算机技术（上海）有限公司 Method, system, electronic equipment and the storage medium of image classification
WO2021036059A1 (en) * 2019-08-29 2021-03-04 深圳云天励飞技术有限公司 Image conversion model training method, heterogeneous face recognition method, device and apparatus
CN110852292A (en) * 2019-11-18 2020-02-28 南京邮电大学 Sketch face recognition method based on cross-modal multi-task depth measurement learning
CN113129208B (en) * 2019-12-31 2024-03-26 深圳云天励飞技术有限公司 Sketch-based face image generation method and related products
CN113129208A (en) * 2019-12-31 2021-07-16 深圳云天励飞技术有限公司 Human face image generation method based on sketch and related product
CN111738157A (en) * 2020-06-23 2020-10-02 平安科技（深圳）有限公司 Method and device for constructing data set of facial action units and computer equipment
WO2021259012A1 (en) * 2020-06-23 2021-12-30 平安科技（深圳）有限公司 Method and apparatus for constructing facial action unit data set, and computer device
CN111738157B (en) * 2020-06-23 2023-07-21 平安科技（深圳）有限公司 Face action unit data set construction method and device and computer equipment
CN113743439A (en) * 2020-11-13 2021-12-03 北京沃东天骏信息技术有限公司 Target detection method, device and storage medium
CN112633288B (en) * 2020-12-29 2024-02-13 杭州电子科技大学 Face sketch generation method based on painting brush touch guidance
CN112633288A (en) * 2020-12-29 2021-04-09 杭州电子科技大学 Face sketch generation method based on drawing stroke guidance
CN112799511A (en) * 2021-01-28 2021-05-14 Oppo广东移动通信有限公司 Training method and device for graphic code generation model, electronic equipment and storage medium
CN115396831A (en) * 2021-05-08 2022-11-25 中国移动通信集团浙江有限公司 Interaction model generation method, device, equipment and storage medium
CN113722528A (en) * 2021-08-03 2021-11-30 南京邮电大学 Method and system for rapidly retrieving photos facing sketch
CN113642481A (en) * 2021-08-17 2021-11-12 百度在线网络技术（北京）有限公司 Recognition method, training method, device, electronic equipment and storage medium
CN117830083A (en) * 2024-03-05 2024-04-05 昆明理工大学 Method and device for generating face sketch-to-face photo
CN117830083B (en) * 2024-03-05 2024-05-03 昆明理工大学 Method and device for generating face sketch-to-face photo

Similar Documents

Publication Publication Date Title
CN109359541A (en) A kind of sketch face identification method based on depth migration study
US11195051B2 (en) Method for person re-identification based on deep model with multi-loss fusion training strategy
CN108615010B (en) Facial expression recognition method based on parallel convolution neural network feature map fusion
CN108537136B (en) Pedestrian re-identification method based on attitude normalization image generation
CN109711281B (en) Pedestrian re-recognition and feature recognition fusion method based on deep learning
CN111553193B (en) Visual SLAM closed-loop detection method based on lightweight deep neural network
CN109815826B (en) Method and device for generating face attribute model
CN107194341B (en) Face recognition method and system based on fusion of Maxout multi-convolution neural network
Cheng et al. Exploiting effective facial patches for robust gender recognition
CN109033938A (en) A kind of face identification method based on ga s safety degree Fusion Features
CN109409297B (en) Identity recognition method based on dual-channel convolutional neural network
CN108268859A (en) A kind of facial expression recognizing method based on deep learning
CN109002755B (en) Age estimation model construction method and estimation method based on face image
CN104504362A (en) Face detection method based on convolutional neural network
CN106570491A (en) Robot intelligent interaction method and intelligent robot
JP2022532177A (en) Forged face recognition methods, devices, and non-temporary computer-readable storage media
CN112364791B (en) Pedestrian re-identification method and system based on generation of confrontation network
CN112052772A (en) Face shielding detection algorithm
CN107220598A (en) Iris Texture Classification based on deep learning feature and Fisher Vector encoding models
CN114662497A (en) False news detection method based on cooperative neural network
Zhu et al. A novel simple visual tracking algorithm based on hashing and deep learning
CN110751005B (en) Pedestrian detection method integrating depth perception features and kernel extreme learning machine
CN113221655B (en) Face spoofing detection method based on feature space constraint
CN112949510A (en) Human detection method based on fast R-CNN thermal infrared image
CN112381047A (en) Method for enhancing and identifying facial expression image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190219