CN110059652A - Face image processing process, device and storage medium - Google Patents

Face image processing process, device and storage medium Download PDF

Info

Publication number
CN110059652A
CN110059652A CN201910335080.9A CN201910335080A CN110059652A CN 110059652 A CN110059652 A CN 110059652A CN 201910335080 A CN201910335080 A CN 201910335080A CN 110059652 A CN110059652 A CN 110059652A
Authority
CN
China
Prior art keywords
facial image
feature
model
image
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910335080.9A
Other languages
Chinese (zh)
Other versions
CN110059652B (en
Inventor
邰颖
殷希
李绍欣
李季檩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201910335080.9A priority Critical patent/CN110059652B/en
Publication of CN110059652A publication Critical patent/CN110059652A/en
Application granted granted Critical
Publication of CN110059652B publication Critical patent/CN110059652B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)

Abstract

The embodiment of the invention discloses a kind of face image processing process, device and storage mediums, belong to technical field of image processing.This method comprises: obtaining first sample facial image and corresponding second sample facial image；Based on having trained the fisrt feature completed to extract model, feature extraction is carried out to first sample facial image, obtains the first sample identity characteristic of first sample facial image；Model is extracted based on current second feature, feature extraction is carried out to the second sample facial image, obtains the second sample identity feature of the second sample facial image；According to the error between first sample identity characteristic and the second sample identity feature, the model parameter that second feature extracts model is adjusted.Second feature can be made to extract model learning to the relationship between facial image and corresponding identity characteristic, more accurate Feature Selection Model can be trained, the identity characteristic for arriving second feature extraction model extraction is more accurate, improves accuracy rate.

Description

Face image processing process, device and storage medium
Technical field
The present embodiments relate to technical field of image processing, in particular to a kind of face image processing process, device and Storage medium.
Background technique
With image processing techniques development and intelligent equipment it is widely available, in Face datection, recognition of face, face Facial image can be handled in the several scenes such as Expression analysis.But the if resolution ratio of collected facial image Lower, picture quality is poor, and the treatment effect that will lead to facial image is bad.Therefore, how to the facial image of low resolution It carries out being treated as urgent problem to be solved.
In order to promote picture quality, it will usually obtain high-resolution facial image according to the facial image of low resolution. Referring to Fig. 1, the relevant technologies can first train residual error network model, residual error network model study to low-resolution image and high-resolution Difference between rate image can obtain corresponding high-definition picture according to low-resolution image.Therefore, when getting low point When the first facial image of resolution, the second facial image is obtained based on the residual error network model and first facial image, this Two facial images are high-resolution facial image corresponding with the first facial image, the picture quality of second facial image Higher than the picture quality of first facial image, second facial image can be handled in subsequent process, to promote place Manage effect.
Above scheme does not consider facial image in the corresponding high-resolution human face image of acquisition low-resolution face image In can identify the identity characteristic of face identity, may influence whether subsequent face image processing effect.Therefore, it needs to propose A kind of face image processing process considering identity characteristic.
Summary of the invention
The embodiment of the invention provides a kind of face image processing process, device and storage mediums, can solve related skill Art there are the problem of.The technical solution is as follows:
On the one hand, a kind of face image processing process is provided, which comprises
It obtains first sample facial image and corresponding second sample facial image, the first sample facial image belongs to First kind facial image, the second sample facial image belong to the second class facial image, point of the first kind facial image Resolution is higher than the resolution ratio of the second class facial image；
Based on having trained the fisrt feature completed to extract model, feature extraction is carried out to the first sample facial image, Obtain the first sample identity characteristic of the first sample facial image, the fisrt feature extracts model for extracting described the The identity characteristic of a kind of facial image；
Model is extracted based on current second feature, feature extraction is carried out to the second sample facial image, obtains institute The second sample identity feature of the second sample facial image is stated, the second feature extracts model for extracting the second class people The identity characteristic of face image；
According to the error between the first sample identity characteristic and the second sample identity feature, adjustment described second The model parameter of Feature Selection Model, so that based on having trained the second feature completed to extract model to the second class people The identity characteristic that face image is extracted extracts model to the second class facial image corresponding first with based on the fisrt feature The error convergence between identity characteristic that class facial image extracts.
Optionally, the method also includes:
Target facial image is obtained, the target facial image belongs to the second class facial image；
Model is extracted based on the second feature, feature extraction is carried out to the target facial image, obtains the target The target identities feature of facial image.
Optionally, described that model is extracted based on the second feature, feature extraction is carried out to the target facial image, is obtained To after the target identities feature of the target facial image, the method also includes:
According to the target identities characteristic query identity database, the corresponding identity mark of the target identities feature is obtained Know, the identity database includes at least one identity characteristic and corresponding identity.
Optionally, it is described based on trained complete fisrt feature extract model, to the first sample facial image into Row feature extraction, before obtaining the first sample identity characteristic of the first sample facial image, the method also includes:
Obtain the third sample identity feature of third sample facial image and the third sample facial image, the third Facial image belongs to the first kind facial image；
Model is extracted based on the current fisrt feature, feature extraction is carried out to the third sample facial image, is obtained To the 4th sample identity feature of the third sample facial image；
According to the error between the third sample identity feature and the 4th sample identity feature, adjustment described first The model parameter of Feature Selection Model, so that based on having trained the fisrt feature completed to extract model to the first kind people The identity characteristic that face image is extracted, the error convergence between the actual identity feature of the first kind facial image.
Optionally, it is described based on trained complete fisrt feature extract model, to the first sample facial image into Row feature extraction, after obtaining the first sample identity characteristic of the first sample facial image, the method also includes:
Model is extracted based on current third feature, feature extraction is carried out to the first sample facial image, obtains institute The non-identity characteristic of first sample of first sample facial image is stated, the third feature extracts model for extracting the first kind The non-identity characteristic of facial image；
The first sample identity characteristic and the non-identity characteristic of the first sample are combined, the first image spy is obtained Sign generates the corresponding first specified facial image of the first image feature based on current Face image synthesis model, described Face image synthesis model is used to generate the first kind facial image corresponding with described image feature according to characteristics of image；
Obtain the first image error between the first sample facial image and the first specified facial image；
According to the first image error, adjust the third feature extract model model parameter and the facial image The model parameter of model is generated, so that based on having trained the third feature completed to extract model to the first kind face figure The identity spy that model extracts the first kind facial image is extracted as the non-identity characteristic extracted, and based on the fisrt feature Characteristics of image after sign combination, after being input to the Face image synthesis model for train completion, obtained specified face figure As being restrained with the image error between the corresponding first kind facial image.
Optionally, the error according between the first sample identity characteristic and the second sample identity feature, Adjust the model parameter that the second feature extracts model, comprising:
Based on having trained the third feature completed to extract model, feature is carried out to the first sample facial image and is mentioned It takes, obtains the non-identity characteristic of the second sample of the first sample facial image；
The second sample identity feature and the non-identity characteristic of the second sample are combined, the second image spy is obtained Sign generates corresponding second nominator of second characteristics of image based on the Face image synthesis model completed has been trained Face image；
Obtain the second image error between the first sample facial image and the second specified facial image；
According to second image error and the first sample identity characteristic and the second sample identity feature it Between error, the model parameter that the second feature extracts model is adjusted, so that based on the second feature for train completion Extract the identity characteristic that extracts to the second class facial image of model, with based on fisrt feature extraction model to described the The error convergence between identity characteristic that the corresponding first kind facial image of two class facial images extracts, and the second class face The corresponding first kind facial image of image and specifying based on the Face image synthesis model generation for having trained completion Image error convergence between facial image.
Optionally, described that model is extracted based on the second feature, feature extraction is carried out to the target facial image, is obtained To after the target identities feature of the target facial image, the method also includes:
It by the target identities feature and presets non-identity characteristic and is combined, obtain target image characteristics；
Based on the Face image synthesis model that training is completed, the corresponding specified face figure of the target image characteristics is obtained Picture, the Face image synthesis model are used to generate the first kind face corresponding with described image feature according to characteristics of image Image.
Optionally, described according to the first image error, adjust the third feature extract model model parameter and The model parameter of the Face image synthesis model, comprising:
Based on current discrimination model, differentiation processing is carried out to the first sample facial image, obtains the first differentiation knot Fruit carries out differentiation processing to the described first specified facial image, obtains the second differentiation as a result, the discrimination model is appointed for determining One facial image is the probability of original facial image；
It has trained the fisrt feature completed to extract model based on described, the described first specified facial image has been carried out special Sign is extracted, and the first specified identity characteristic is obtained；
Obtain the first identity characteristic error between the described first specified identity characteristic and the first sample identity characteristic；
Differentiate that result and the first default error differentiated between result, described second differentiate result and the according to described first The two default errors and the first identity characteristic error differentiated between result, adjust the third feature and extract model The model parameter of model parameter, the model parameter of the Face image synthesis model and the discrimination model, so that based on having instructed Practice the error convergence that the discrimination model completed carries out the differentiation result obtained after differentiation processing, and is based on the fisrt feature Extract identity characteristic that model extracts the first kind facial image with based on fisrt feature extraction model to described the The error convergence between identity characteristic that the corresponding specified facial image of a kind of facial image extracts.
Optionally, the error according between the first sample identity characteristic and the second sample identity feature, Adjust the model parameter that the second feature extracts model, comprising:
Based on having trained the third feature completed to extract model, feature is carried out to the first sample facial image and is mentioned It takes, obtains the non-identity characteristic of the second sample of the first sample facial image；
The second sample identity feature and the non-identity characteristic of the second sample are combined, the second image spy is obtained Sign generates corresponding second nominator of second characteristics of image based on the Face image synthesis model completed has been trained Face image obtains the second image error between the first sample facial image and the second specified facial image；
Based on the discrimination model completed has been trained, differentiation processing is carried out to the described second specified facial image, is obtained Third differentiates result；
Based on having trained the fisrt feature completed to extract model, feature is carried out to the described second specified facial image and is mentioned Take, obtain the second specified identity characteristic of the described second specified facial image, obtain the second specified identity characteristic with it is described Second Identity of Local error between first sample identity characteristic；
According to the error between the first sample identity characteristic and the second sample identity feature, second image Error, the third differentiate that result and the second default error differentiated between result and the Second Identity of Local are missed Difference adjusts the model parameter that the second feature extracts model, so that based on having trained the second feature completed to extract mould The identity characteristic that type extracts the second class facial image extracts model to the second class people with based on the fisrt feature The error convergence between identity characteristic that the corresponding first kind facial image of face image extracts, extracts mould based on the fisrt feature Type has trained the second feature completed to extract model, third feature extraction model and the Face image synthesis mould Image error convergence between the specified facial image that type obtains and the corresponding second class facial image, is based on the differentiation Model carries out the error convergence of the differentiation result obtained after differentiation processing, and extracts model to described the based on the fisrt feature The identity characteristic that a kind of facial image extracts, it is corresponding to the second class facial image with model is extracted based on the fisrt feature Specified facial image extract identity characteristic between error convergence.
On the other hand, a kind of face image processing process is provided, which comprises
It obtains second feature and extracts model, the second feature is extracted model and is adjusted so as to according to fisrt feature extraction model It arrives, and adjusting target is to extract the identity characteristic that model extracts the second class facial image based on the second feature, is intended to It is special that the identity that model extracts the corresponding first kind facial image of the second class facial image is extracted based on the fisrt feature Sign, the high resolution of the first kind facial image is in the resolution ratio of the second class facial image；
Model is extracted based on the second feature, the target facial image for belonging to the second class facial image is carried out special Sign is extracted, and the target identities feature of the target facial image is obtained.
Optionally, described that model is extracted based on the second feature, to the target person for belonging to the second class facial image Face image carries out feature extraction, after obtaining the target identities feature of the target facial image, the method also includes:
It by the target identities feature and presets non-identity characteristic and is combined, obtain target image characteristics；
Based on Face image synthesis model, the corresponding specified facial image of the target image characteristics is obtained, it is described specified Facial image belongs to the first kind facial image；
The Face image synthesis model extracts model according to the fisrt feature and third feature is extracted model and is adjusted so as to Arrive, and adjust target be based on the Face image synthesis model be characteristics of image generate specified facial image, with the figure It is restrained as feature is corresponding, belongs to the image error between any facial image of the first kind facial image；Described image Feature is non-identity characteristic that model extracts any facial image to be extracted based on the third feature and based on described the Characteristics of image after the identity characteristic combination that one Feature Selection Model extracts any facial image.
Optionally, described that model is extracted based on second feature, to the target face figure for belonging to the second class facial image As carrying out feature extraction, after obtaining the target identities feature of the target facial image, the method also includes:
According to the target identities characteristic query identity database, the corresponding identity mark of the target identities feature is obtained Know, the identity database includes at least one identity characteristic and corresponding identity.
On the other hand, a kind of face image processing device is provided, described device includes:
First sample facial image obtains module, for obtaining first sample facial image and corresponding second sample face Image, the first sample facial image belong to first kind facial image, and the second sample facial image belongs to the second class people Face image, the high resolution of the first kind facial image is in the resolution ratio of the second class facial image；
Fisrt feature extraction module, for based on trained complete fisrt feature extract model, to the first sample Facial image carries out feature extraction, obtains the first sample identity characteristic of the first sample facial image, the fisrt feature Extract the identity characteristic that model is used to extract the first kind facial image；
Second feature extraction module, for extracting model based on current second feature, to the second sample face figure As carrying out feature extraction, the second sample identity feature of the second sample facial image is obtained, the second feature extracts mould Type is used to extract the identity characteristic of the second class facial image；
The first adjustment module, for according between the first sample identity characteristic and the second sample identity feature Error adjusts the model parameter that the second feature extracts model, so that based on having trained the second feature completed to extract The identity characteristic that model extracts the second class facial image extracts model to second class with based on the fisrt feature The error convergence between identity characteristic that the corresponding first kind facial image of facial image extracts.
Optionally, described device further include:
Target image obtains module, and for obtaining target facial image, the target facial image belongs to second class Facial image；
Target's feature-extraction module carries out the target facial image for extracting model based on the second feature Feature extraction obtains the target identities feature of the target facial image.
Optionally, described device further include:
Enquiry module, for obtaining the target identities feature according to the target identities characteristic query identity database Corresponding identity, the identity database include at least one identity characteristic and corresponding identity.
Optionally, described device further include:
Second sample facial image obtains module, for obtaining third sample facial image and the third sample face figure The third sample identity feature of picture, the third facial image belong to the first kind facial image；
Third feature extraction module, for extracting model based on the current fisrt feature, to the third sample people Face image carries out feature extraction, obtains the 4th sample identity feature of the third sample facial image；
Second adjustment module, for according between the third sample identity feature and the 4th sample identity feature Error adjusts the model parameter that the fisrt feature extracts model, so that based on having trained the fisrt feature completed to extract Between the identity characteristic that model extracts the first kind facial image, and the actual identity feature of the first kind facial image Error convergence.
Optionally, described device further include:
Fourth feature extraction module, for extracting model based on current third feature, to the first sample face figure As carrying out feature extraction, the non-identity characteristic of first sample of the first sample facial image is obtained, the third feature is extracted Model is used to extract the non-identity characteristic of the first kind facial image；
Specified image collection module, for by the first sample identity characteristic and the non-identity characteristic of the first sample into Row combination, obtains the first characteristics of image, and based on current Face image synthesis model, it is corresponding to generate the first image feature First specified facial image, the Face image synthesis model are used to be generated according to characteristics of image corresponding with described image feature The first kind facial image；
Image error obtains module, for obtain the first sample facial image and the first specified facial image it Between the first image error；
Third adjusts module, for adjusting the model that the third feature extracts model according to the first image error The model parameter of parameter and the Face image synthesis model, so that based on having trained the third feature completed to extract model Model is extracted to the first kind people to the non-identity characteristic that the first kind facial image extracts, and based on the fisrt feature Characteristics of image after the identity characteristic combination that face image is extracted, is input to the Face image synthesis model trained and completed Afterwards, the image error convergence between the specified facial image obtained and the corresponding first kind facial image.
Optionally, the first adjustment module, comprising:
Feature extraction unit, the third feature for being completed based on training extracts model, to the first sample people Face image carries out feature extraction, obtains the non-identity characteristic of the second sample of the first sample facial image；
Image acquisition unit, for the second sample identity feature and the non-identity characteristic of the second sample to be carried out group It closes, obtains the second characteristics of image, based on the Face image synthesis model that training is completed, generate second characteristics of image pair The specified facial image of second answered；
Image error acquiring unit, for obtain the first sample facial image and the second specified facial image it Between the second image error；
Adjustment unit, for according to second image error and the first sample identity characteristic and second sample Error between this identity characteristic adjusts the model parameter that the second feature extracts model, so that based on completion has been trained The second feature extracts the identity characteristic that model extracts the second class facial image, extracts with based on the fisrt feature The error convergence between identity characteristic that model extracts the corresponding first kind facial image of the second class facial image, and institute State the corresponding first kind facial image of the second class facial image and based on the Face image synthesis mould for having trained completion Image error convergence between the specified facial image that type generates.
Optionally, described device further include:
Feature combination module obtains target for by the target identities feature and presetting non-identity characteristic and being combined Characteristics of image；
Image generation module, the Face image synthesis model for being completed based on training, obtains the target image characteristics Corresponding specified facial image, the Face image synthesis model are used to be generated according to characteristics of image corresponding with described image feature The first kind facial image.
Optionally, the third adjusts module, comprising:
Judgement unit, for carrying out differentiation processing to the first sample facial image, obtaining based on current discrimination model To the first differentiation as a result, carrying out differentiation processing to the described first specified facial image, the second differentiation is obtained as a result, the differentiation mould Type is for determining that any facial image is the probability of original facial image；
Feature extraction unit, the fisrt feature for being completed based on the training are extracted model, referred to described first Determine facial image and carry out feature extraction, obtains the first specified identity characteristic；
Characteristic error acquiring unit, for obtain the described first specified identity characteristic and the first sample identity characteristic it Between the first identity characteristic error；
Adjustment unit, for differentiating result and the first default error differentiated between result, described the according to described first Two differentiate result and the second default error and the first identity characteristic error differentiated between result, adjust the third The model of the model parameter of Feature Selection Model, the model parameter of the Face image synthesis model and the discrimination model is joined Number, so that based on the error convergence for having trained the discrimination model completed to carry out the differentiation result obtained after differentiation processing, and Identity characteristic that model extracts the first kind facial image is extracted based on the fisrt feature and is based on the fisrt feature Extract the error convergence between the identity characteristic that model specified facial image corresponding to the first kind facial image extracts.
Optionally, the first adjustment module, comprising:
Feature extraction unit, the third feature for being completed based on training extracts model, to the first sample people Face image carries out feature extraction, obtains the non-identity characteristic of the second sample of the first sample facial image；
Image acquisition unit, for the second sample identity feature and the non-identity characteristic of the second sample to be carried out group It closes, obtains the second characteristics of image, based on the Face image synthesis model that training is completed, generate second characteristics of image pair The specified facial image of second answered；
Image error acquiring unit, for obtain the first sample facial image and the second specified facial image it Between the second image error；
Judgement unit, the discrimination model for being completed based on training, sentences the described second specified facial image Other places reason obtains third and differentiates result；
Characteristic error acquiring unit, the fisrt feature for being completed based on training are extracted model, referred to described second Determine facial image and carry out feature extraction, obtain the second specified identity characteristic of the described second specified facial image, obtains described the Second Identity of Local error between two specified identity characteristics and the first sample identity characteristic；
Adjustment unit, for according to the mistake between the first sample identity characteristic and the second sample identity feature Poor, described second image error, the third differentiate result and the second default error differentiated between result and described Second Identity of Local error adjusts the model parameter that the second feature extracts model, so that based on having trained described in completion Second feature extracts the identity characteristic that model extracts the second class facial image, extracts model with based on the fisrt feature The error convergence between identity characteristic that the corresponding first kind facial image of the second class facial image is extracted, based on described Fisrt feature extracts model, the second feature completed train to extract model, third feature extraction model and described Image error between the specified facial image that Face image synthesis model obtains and the corresponding second class facial image is received It holds back, the error convergence based on the differentiation result that the discrimination model obtain after differentiation processing, and is based on the fisrt feature Extract the identity characteristic that extracts to the first kind facial image of model, with based on fisrt feature extraction model to described the The error convergence between identity characteristic that the corresponding specified facial image of two class facial images extracts.
On the other hand, a kind of face image processing device is provided, described device includes:
Second feature obtains module, extracts model for obtaining second feature, and the second feature extracts model according to the One Feature Selection Model adjusts to obtain, and adjusting target is to extract model based on the second feature to mention the second class facial image The identity characteristic taken is intended to extract model to the corresponding first kind people of the second class facial image based on the fisrt feature The identity characteristic that face image is extracted, the high resolution of the first kind facial image is in the resolution of the second class facial image Rate；
Target's feature-extraction module, for extracting model based on the second feature, to belonging to the second class face figure The target facial image of picture carries out feature extraction, obtains the target identities feature of the target facial image.
Optionally, described device further include:
Composite module obtains target image for by the target identities feature and presetting non-identity characteristic and being combined Feature；
It is corresponding to obtain the target image characteristics for being based on Face image synthesis model for specified image collection module Specified facial image, the specified facial image belong to the first kind facial image；
The Face image synthesis model extracts model according to the fisrt feature and third feature is extracted model and is adjusted so as to Arrive, and adjust target be based on the Face image synthesis model be characteristics of image generate specified facial image, with the figure It is restrained as feature is corresponding, belongs to the image error between any facial image of the first kind facial image；Described image Feature is non-identity characteristic that model extracts any facial image to be extracted based on the third feature and based on described the Characteristics of image after the identity characteristic combination that one Feature Selection Model extracts any facial image.
Optionally, described device further include:
Enquiry module, for obtaining the target identities feature according to the target identities characteristic query identity database Corresponding identity, the identity database include at least one identity characteristic and corresponding identity.
On the other hand, a kind of face image processing device is provided, described device includes processor and memory, described to deposit At least one instruction, at least a Duan Chengxu, code set or instruction set, described instruction, described program, the generation are stored in reservoir Code collection or described instruction collection are loaded by the processor and are executed to realize as performed in the face image processing process Operation.
In another aspect, providing a kind of computer readable storage medium, it is stored in the computer readable storage medium At least one instruction, at least a Duan Chengxu, code set or instruction set, described instruction, described program, the code set or the finger Collection is enabled to be loaded by processor and had to realize such as operation performed in the face image processing process.
Method, apparatus provided in an embodiment of the present invention and storage medium extract model to fisrt feature and are trained, be based on It has trained the fisrt feature completed to extract model, has obtained the first sample identity characteristic of first sample facial image, based on current Second feature extract model, obtain the second sample identity feature of the second sample facial image, it is special according to first sample identity Error between sign and the second sample identity feature, adjustment second feature extract the model parameter of model.The embodiment of the present invention mentions A kind of method for having supplied identity characteristic for extracting the second class facial image, after the completion of fisrt feature extracts model training, according to Mutual corresponding first kind facial image and the second class facial image, by first kind facial image and corresponding second class face figure As being used as sample pair, the common training process for participating in second feature and extracting model can make second feature extract model learning and arrive Relationship between facial image and corresponding identity characteristic can train more accurate Feature Selection Model, make the second spy It is more accurate that sign extracts the identity characteristic that arrives of model extraction, improves accuracy rate, convenient for the subsequent identity characteristic according to extraction into Row face image processing.
Also, the embodiment of the present invention obtains target facial image based on having trained the second feature completed to extract model Target identities feature obtains the corresponding identity of target identities feature, can efficiently use people by query identity database For identifying the identity characteristic of face identity in face image, the identity characteristic extracted is applied to more with face identification etc. In kind scene, the application range of facial image is expanded.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment Attached drawing is briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the embodiment of the present invention Example, for those of ordinary skill in the art, without creative efforts, can also obtain according to these attached drawings Obtain other attached drawings.
Fig. 1 is a kind of schematic diagram of the face image processing process provided in the related technology；
Fig. 2 is a kind of schematic diagram of implementation environment provided in an embodiment of the present invention；
Fig. 3 is a kind of flow chart of face image processing process provided in an embodiment of the present invention；
Fig. 4 is a kind of schematic diagram of network architecture provided in an embodiment of the present invention；
Fig. 5 is the network architecture schematic diagram that a kind of fisrt feature provided in an embodiment of the present invention extracts model；
Fig. 6 is the flow chart of another face image processing process provided in an embodiment of the present invention；
Fig. 7 is the schematic diagram of another network architecture provided in an embodiment of the present invention；
Fig. 8 is the flow chart of another face image processing process provided in an embodiment of the present invention；
Fig. 9 is the schematic diagram of another network architecture provided in an embodiment of the present invention；
Figure 10 is a kind of structural schematic diagram of face image processing device provided in an embodiment of the present invention；
Figure 11 is the structural schematic diagram of another face image processing device provided in an embodiment of the present invention；
Figure 12 is a kind of structural schematic diagram of terminal provided in an embodiment of the present invention；
Figure 13 is a kind of structural schematic diagram of server provided in an embodiment of the present invention.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with attached drawing to the present invention Embodiment is described in further detail.
Before to the embodiment of the present invention carrying out that explanation is explained in detail, first to the present embodiments relate to the concepts arrived Carry out description below:
1, first kind facial image and the second class facial image:
According to the difference of the resolution ratio of facial image, facial image is divided into first kind facial image and the second class face Image.Wherein, the high resolution of first kind facial image, then can be by first kind face in the resolution ratio of the second class facial image Image is known as high-resolution human face image, and the second class facial image is known as low-resolution face image.
The picture quality of first kind facial image is higher than the picture quality of the second class facial image.In contrast, the first kind Facial image is relatively clear, and the image information for including is more, and the second class facial image is more fuzzy, the image information for including compared with It is few.
2, identity characteristic: refer to the feature that there is distinctive to the face identity for including in facial image, for describing people The identity information for the face for including in face image.
3, non-identity characteristic: refer to other features in facial image in addition to identity characteristic, for describing facial image In other information in addition to the identity information of face, such as illumination, posture, human face expression information.
4, the present embodiments relate to the model arrived is as shown in table 1 below:
Table 1
5, facial image and original facial image are specified:
Specified facial image is the facial image generated by Face image synthesis model according to characteristics of image, and original face Image is not the facial image generated by Face image synthesis model according to characteristics of image.For example, original facial image can be The facial image that terminal is shot by camera.
Fig. 2 is a kind of schematic diagram of implementation environment provided in an embodiment of the present invention, which includes: 201 He of terminal Server 202 passes through network connection between terminal 201 and server 202.
Wherein, terminal 201 can be a plurality of types of equipment such as mobile phone, computer, tablet computer, and server 202 can be with It is a server, or the server cluster consisted of several servers or a cloud computing service center.
The embodiment of the invention provides a kind of face image processing process, are obtained by training for extracting the second class face The Feature Selection Model of the identity characteristic of image can carry out feature extraction to the second class facial image, obtain that this can be described The identity characteristic for the face identity information for including in second class facial image, the subsequent identity characteristic that can will be extracted are applied to In the scene for carrying out image procossing according to the face identity of facial image.
In a kind of possible implementation, this method is applied in server 202.
Wherein, facial image is input in server 202 after being acquired by operator, or is sent by terminal 201 It is obtained to server 202, or by server 202 using other modes, server 202 is carried out based on the facial image got Training obtains this feature and extracts model.It is subsequent this feature extraction model to be issued in terminal 201, make for terminal 201 With.
In alternatively possible implementation, this method is applied in terminal 201.
Wherein, facial image can be shot by terminal 201 and be obtained, or be obtained by terminal 201 from the downloading of server 202, Or it is obtained by terminal 201 by using other modes.Terminal 201 is trained based on the facial image got, is somebody's turn to do Feature Selection Model, the subsequent identity characteristic that model extraction facial image can be extracted based on this feature.
Fig. 3 is a kind of flow chart of face image processing process provided in an embodiment of the present invention, the face image processing side Method is first extracted model to fisrt feature and is trained, and according to having trained the fisrt feature completed to extract model, mentions to second feature Modulus type is trained.
The executing subject of the embodiment of the present invention is processing equipment, which can be terminal shown in Fig. 2 or service Device, referring to Fig. 3, this method comprises:
301, model is extracted to fisrt feature to be trained.
The image information of facial image may include multiple types, such as the position of the identity information of face, Lighting information, face Confidence breath etc., these information can be divided into identity information and non-identity information, correspondingly, the characteristics of image of facial image can wrap Include identity characteristic and non-identity characteristic, wherein identity characteristic is for being described identity information, rather than identity characteristic for pair Non- identity information is described.
Since facial image may include first kind facial image and the second class facial image, point of first kind facial image Resolution is higher than the resolution ratio of the second class facial image, then the picture quality of first kind facial image is higher than the second class facial image Picture quality, the image information of first kind facial image are more than the image information of the second class facial image.
About the mode for dividing first kind facial image and the second class facial image, in a kind of mode in the cards, First default resolution ratio and the second default resolution ratio are set, and the first default resolution ratio is not less than the second default resolution ratio, will differentiate Rate is higher than the facial image of the first default resolution ratio as first kind facial image, by resolution ratio lower than the second default resolution The facial image of rate is as the second class facial image.
The image information of different types of facial image is different, and the feature of extraction is also different.Therefore, the embodiment of the present invention is Identity characteristic is accurately extracted, is respectively trained out different Feature Selection Models.
Referring to fig. 4, it is trained firstly, extracting model to fisrt feature, which extracts model for extracting the The identity characteristic of a kind of facial image.Training process includes: that processing equipment obtains current fisrt feature and extracts model, and current Fisrt feature extract model can for initialization fisrt feature extract model, or by one or many training, But the fisrt feature that also training is not completed extracts model.The third sample facial image for belonging to first kind facial image is obtained, with And the third sample identity feature of third sample facial image, the third sample identity feature are the reality of third sample facial image Border identity characteristic, extracts model based on current fisrt feature, carries out feature extraction to third sample facial image, obtains third 4th sample identity feature of sample facial image.According to the mistake between third sample identity feature and the 4th sample identity feature Difference, adjustment fisrt feature extract the model parameter of model, so as to extract model to the first kind based on the fisrt feature that training is completed The identity characteristic that facial image extracts, the error convergence between the actual identity feature of the first kind facial image.
For any one model, error convergence refers to, by once being adjusted to model, makes based on adjusted The error obtained when model is handled is smaller compared with the error obtained when being handled based on the model before adjustment.Pass through one Secondary or multiple adjustment, the error of model are gradually reduced, until error convergence can be recognized when the error of model is less than preset threshold It is met the requirements for model at this time, model training is completed.
About the mode for obtaining third sample facial image, in a kind of possible implementation, the third sample face figure As that can shoot to obtain by processing equipment, processing equipment perhaps is sent to or by processing equipment using other by other equipment Mode acquires.
Wherein, fisrt feature extracts model when extracting the identity characteristic of first kind facial image, and extracted identity is special There may be error between sign and the actual identity feature of first kind facial image, the error is smaller, indicates that fisrt feature mentions The extraction effect of modulus type is better, and the error is bigger, indicates that the extraction effect of fisrt feature extraction model is poorer.
In above-mentioned training process, third sample identity feature is the actual identity feature of third sample facial image, and the Four sample identity features are to extract after model carries out feature extraction to third sample facial image to obtain based on current fisrt feature The identity characteristic arrived.Therefore the error between third sample identity feature and the 4th sample identity feature can reflect fisrt feature The current extraction effect of model is extracted, according to the error between third sample identity feature and the 4th sample identity feature, adjustment Fisrt feature extracts the model parameter of model, mentions so as to extract model based on fisrt feature adjusted to first kind facial image The error of the identity characteristic taken reduces, and extracts model to achieve the effect that optimize fisrt feature.
The embodiment of the present invention be only for according to third sample facial image and its corresponding third sample identity feature, The process for extracting model to training fisrt feature is illustrated.And in order to improve the accuracy rate that fisrt feature extracts model, it can be with Multiple sample facial images and its corresponding sample identity feature are obtained, model repeatedly is extracted to fisrt feature and is trained.And The size of the multiple sample facial images got may be the same or different.And fisrt feature extracts the training mesh of model It is designated as: extracting the error convergence for the identity characteristic that model extracts first kind facial image, based on fisrt feature to guarantee to extract To identity characteristic the identity information of the face for including in the first kind facial image can be described as precisely as possible.
One or many training are carried out using above-mentioned training method, and according to above-mentioned training objective, until fisrt feature mentions Until modulus type restrains, the fisrt feature of completion is trained to extract model.
In a kind of possible implementation, model is extracted for fisrt feature, first-loss function is set, be based on first-loss Function extracts model to fisrt feature and is trained, and the output valve of the first-loss function is by extracted first kind facial image The error of identity characteristic determine, and the error correlation, thus fisrt feature extract the training objective of model can be with Are as follows: the output valve of first-loss function restrains, to guarantee that fisrt feature extracts the identity that model extracts any facial image Error convergence between feature and the actual identity feature of the facial image.
Wherein, which can be Softmax loss function (cross entropy loss function), or can also be Unknown losses function.
After the completion of fisrt feature extracts model training, any first kind facial image can be input to and train completion Fisrt feature extracts model, extracts model based on the fisrt feature, carries out feature extraction to the first kind facial image, is somebody's turn to do The identity characteristic of first kind facial image.
In the embodiment of the present invention, after the completion of fisrt feature extracts model training, also model can be extracted according to fisrt feature, It extracts model to second feature to be trained, which extracts the identity spy that model is used to extract the second class facial image Sign.Training process step 402-405 as described below.
302, first sample facial image and corresponding second sample facial image are obtained.
Firstly, processing equipment obtains first sample facial image and the second sample corresponding with the first sample facial image This facial image.
Wherein, first sample facial image is to belong to first kind facial image, and the second sample facial image belongs to the second class Facial image, the high resolution of first kind facial image is in the resolution ratio of the second class facial image.First sample facial image and Second sample facial image corresponds to each other, i.e., first sample facial image and the second sample facial image are respectively same face The corresponding low-resolution image of image and high-definition picture.
About the mode for obtaining first sample facial image and the second sample facial image, in a kind of possible implementation In, which can be shot to obtain by processing equipment, perhaps by other equipment be sent to processing equipment or It is obtained by processing equipment using other modes.The second sample facial image is by adopt to the first sample facial image Sample obtains.The mode of down-sampling can be random scale down-sampling or other modes.
It should be noted that needing to comprehensively consider first sample facial image in the training process of second feature extraction model Identity characteristic and the second sample facial image identity characteristic, if first sample facial image and the second sample facial image Size it is different, training effect may be will affect.Therefore, in a kind of mode in the cards, processing equipment setting unification Pre-set dimension, when getting first sample facial image and the second sample facial image, to first sample facial image and Two sample facial images zoom in and out processing, so that the size of treated sample facial image is pre-set dimension.Alternatively, place When reason equipment gets first sample facial image and the second sample facial image, place is zoomed in and out to the second sample facial image Reason avoids so that the size of treated the second sample facial image is identical as the size of first sample facial image due to people The size of face image is different and bring is interfered.
Wherein, the mode of scaling processing can be interpolation processing mode or other scalable manners.For example, processing equipment uses Bicubic interpolation (Bicubic Interpolation) processing mode, zooms to 128*128's for the facial image of any size Size.
303, based on having trained the fisrt feature completed to extract model, feature extraction is carried out to first sample facial image, Obtain the first sample identity characteristic of first sample facial image.
First sample facial image is input to by processing equipment has trained the fisrt feature completed to extract in model, by first Feature Selection Model carries out feature extraction to first sample facial image, exports the identity characteristic of first sample facial image, i.e., For first sample identity characteristic.
304, model is extracted based on current second feature, feature extraction is carried out to the second sample facial image, obtain the Second sample identity feature of two sample facial images.
Processing equipment obtains current second feature and extracts model, and it can be initial that current second feature, which extracts model, The second feature of change extracts model, or the second feature completed is not trained by one or many training but also to extract Model.Second sample facial image is input to current second feature and extracted in model by processing equipment, by this current second Feature Selection Model carries out feature extraction to the second sample facial image, exports the identity characteristic of the second sample facial image, i.e., Second sample identity feature.
305, it according to the error between first sample identity characteristic and the second sample identity feature, adjusts second feature and extracts The model parameter of model.
Wherein, second feature extracts model when extracting the identity characteristic of the second class facial image, and extracted identity is special There may be error between sign and the actual identity feature of the second class facial image, the error is smaller, indicates that second feature mentions The extraction effect of modulus type is better, and the error is bigger, indicates that the extraction effect of second feature extraction model is poorer.
In above-mentioned training process, first sample identity characteristic is the identity characteristic of first sample facial image, and the second sample This identity characteristic is to extract after model carries out feature extraction to the second sample facial image to obtain based on current second feature Identity characteristic.Since first sample facial image is corresponded to each other with the second sample facial image, although the two image information slightly has Difference, but the identity information of included face should be identical, then identity characteristic should also be as identical, therefore, first sample identity Feature can regard the actual identity feature of the second sample facial image as.
Therefore, the error between first sample identity characteristic and the second sample identity feature can reflect second feature extraction The current extraction effect of model, according to the error between first sample identity characteristic and the second sample identity feature, adjustment second The model parameter of Feature Selection Model, so as to extract what model extracted the second class facial image based on second feature adjusted The error of identity characteristic reduces, and extracts model to achieve the effect that optimize second feature.
The embodiment of the present invention is only for according to first sample facial image and the second sample facial image, to training the The process of two Feature Selection Models is illustrated.And in order to improve the accuracy rate that second feature extracts model, available multiple groups Sample facial image, every group includes a first kind facial image and its a corresponding second class facial image, and the two Size may be the same or different.According to multiple groups sample facial image, model repeatedly is extracted to second feature and is trained. And second feature extracts the training objective of model are as follows: extracts the identity that model extracts the second class facial image based on second feature The error convergence of feature, i.e., it is special based on having trained the second feature completed to extract the identity that model extracts the second class facial image Sign extracts the identity characteristic that model extracts the corresponding first kind facial image of the second class facial image with based on fisrt feature Between error convergence, to guarantee that the identity characteristic that extracts can describe to wrap in the second class facial image as precisely as possible The identity information of the face contained.
One or many training are carried out using above-mentioned training method, and according to above-mentioned training objective, until second feature mentions Until modulus type restrains, the second feature of completion is trained to extract model.
In a kind of possible implementation, model is extracted for second feature, the second loss function is set, based on the second loss Function extracts model to second feature and is trained, and the output valve of second loss function is by extracting model based on second feature The identity characteristic of the second class facial image extracted extracts the extracted first kind facial image of model with based on fisrt feature Difference between identity characteristic determines, and the difference correlation, thus second feature extract the training objective of model can With are as follows: the output valve of the second loss function restrains, to guarantee that second feature extracts the identity characteristic that model extraction arrives and the first spy Sign extracts the difference convergence between the identity characteristic that model extraction arrives.
Wherein, which can be Softmax loss function, or can also be unknown losses function.
Since the image information that the second class facial image includes is less, if directly being instructed according to the second class facial image Practice, it is not accurate enough to will lead to trained Feature Selection Model.And in the embodiment of the present invention, it is contemplated that first kind facial image includes Image information it is more, extracted identity characteristic is more accurate, thus training second feature extract model during, will First kind facial image and corresponding second class facial image are as sample pair, the common training for participating in second feature and extracting model Process can train more accurate Feature Selection Model, and the identity characteristic for arriving second feature extraction model extraction is more Accurately.
306, target facial image is obtained.
307, based on having trained the second feature completed to extract model, feature extraction is carried out to target facial image, is obtained The target identities feature of target facial image.
After the completion of second feature extracts model training, any second class facial image can be input to and train completion Second feature extracts model, extracts model based on the second feature, carries out feature extraction to the second class facial image, is somebody's turn to do The identity characteristic of second class facial image.
For the embodiment of the present invention by taking target facial image as an example, processing equipment obtains the target person for belonging to the second class facial image Face image, the target facial image can be shot to obtain perhaps to download from internet by processing equipment and obtained or by other Equipment is sent to processing equipment.Later, which is input to and the second feature completed has been trained to extract in model, Model being extracted based on second feature, feature extraction being carried out to target facial image, the target identities for obtaining target facial image are special Sign, the target identities feature can be used to the identity information for the face for including in description target facial image.
308, according to target identities characteristic query identity database, the corresponding identity of target identities feature is obtained.
Processing equipment obtains identity database, according to target identities characteristic query identity database, obtains and the target body The identity of part characteristic matching.
Wherein, identity database includes at least one identity characteristic and corresponding identity, and the identity is for true Fixed unique natural person, can for ID card No., telephone number or other be used to determine the mark of natural person's identity.
About the process for determining identity, in a kind of mode in the cards, by target identities feature and identity number It is compared according to the identity characteristic in library, the similarity between target identities feature is less than to the identity characteristic pair of preset threshold The identity answered is determined as the corresponding identity of target identities feature.
For example, the cosine similarity in target identities feature and identity database between identity characteristic is obtained, by cosine phase It is less than the corresponding identity of identity characteristic of preset threshold like degree, is determined as the corresponding identity of target identities feature.
It should be noted that above-mentioned steps 406-408 is optional step, the process for carrying out identification is said It is bright, and in another embodiment, after the completion of second feature extracts model training, model extraction is extracted based on the second feature and is arrived After the identity characteristic of second class facial image, the processing of other modes, such as human face expression can also be carried out according to the identity characteristic Analysis, facial image recovery etc..
Method provided in an embodiment of the present invention is extracted model to fisrt feature and is trained, based on train the of completion One Feature Selection Model obtains the first sample identity characteristic of first sample facial image, is extracted based on current second feature Model obtains the second sample identity feature of the second sample facial image, according to first sample identity characteristic and the second sample body Error between part feature, adjustment second feature extract the model parameter of model.The is extracted the embodiment of the invention provides a kind of The method of the identity characteristic of two class facial images, after the completion of fisrt feature extracts model training, according to mutual corresponding first Class facial image and the second class facial image, using first kind facial image and corresponding second class facial image as sample pair, The common training process for participating in second feature and extracting model, can make second feature extract model learning to facial image with it is corresponding Identity characteristic between relationship, more accurate Feature Selection Model can be trained, make second feature extract model extraction The identity characteristic arrived is more accurate, improves accuracy rate, carries out face image processing according to the identity characteristic of extraction convenient for subsequent.
Also, the embodiment of the present invention obtains target facial image based on having trained the second feature completed to extract model Target identities feature obtains the corresponding identity of target identities feature, can efficiently use people by query identity database For identifying the identity characteristic of face identity in face image, the identity characteristic extracted is applied to more with face identification etc. In kind scene, the application range of facial image is expanded.
Fig. 5 is a kind of schematic diagram of network architecture provided in an embodiment of the present invention, extracts model as high score using fisrt feature For resolution identity characteristic extracts model 501, second feature extracts model as low resolution identity characteristic extraction model 502.Then The model parameter process that adjustment low resolution identity characteristic extracts model 502 may include steps of:
1, training high-resolution identity characteristic extracts model 501.
2, multiple groups sample facial image is obtained, includes mutual corresponding high-resolution human face figure in every group of sample facial image Picture and low-resolution face image.
3, it is directed to every group of sample facial image, high-resolution human face image is input to high-resolution identity characteristic and extracts mould Type 501 obtains the first sample identity characteristic of high-resolution human face image；Low-resolution face image is input to low resolution Identity characteristic extracts model 502, obtains the second sample identity feature of low-resolution face image.
4, first sample identity characteristic and the second sample identity feature are compared, adjustment low resolution identity characteristic extracts model 502 model parameter.
5, according to multiple groups sample facial image, above-mentioned steps 3-4 is repeated, model is extracted to low resolution identity characteristic 502 are repeatedly adjusted, and the low resolution identity characteristic after being trained extracts model 502.
Fig. 6 is a kind of flow chart of face image processing process provided in an embodiment of the present invention, the face image processing side Method, according to having trained the fisrt feature completed to extract model, proposes third feature after the completion of fisrt feature extracts model training Modulus type and Face image synthesis model are trained, and extract model in third feature and Face image synthesis model training is completed Afterwards, model and Face image synthesis model are extracted according to third feature, model is extracted to second feature and is trained.
The executing subject of the embodiment of the present invention is processing equipment, which can be terminal shown in Fig. 2 or service Device, referring to Fig. 6, this method comprises:
601, first sample facial image and corresponding second sample facial image are obtained.
602, based on having trained the fisrt feature completed to extract model, feature extraction is carried out to first sample facial image, Obtain the first sample identity characteristic of first sample facial image.
Step 601-602 is similar with step 302-303 in above-described embodiment, and details are not described herein.
603, model is extracted based on current third feature, feature extraction is carried out to first sample facial image, is somebody's turn to do The non-identity characteristic of the first sample of first sample facial image.
It further include in addition to identity information due to not only including the identity information for identifying face identity in facial image Non- identity information, which includes Lighting information, expression information, posture information etc..In order to obtain for describing people The non-identity characteristic of the non-identity information of face image, processing equipment obtains current third feature and extracts model, by first sample Facial image is input to the third feature and extracts in model, extracts model by the third feature and carries out to first sample facial image Feature extraction exports the non-identity characteristic of first sample facial image, the i.e. non-identity characteristic of first sample.The non-body of the first sample Part feature is used to describe the non-identity information of first sample facial image.
Wherein, current third feature, which extracts model, can extract model for the third feature of initialization, or pass through One or many training but the third feature extraction model that also training is completed.
604, first sample identity characteristic and the non-identity characteristic of first sample are combined, obtain the first characteristics of image.
The embodiment of the present invention can also train the face figure for generating corresponding specified facial image according to characteristics of image As generating model.For same facial image, identity characteristic is only used for describing the identity information of the facial image, non-identity characteristic It is only used for describing the non-identity information of the facial image, only comprehensively considers identity characteristic and non-identity characteristic, can just access The accurate image information of facial image, to generate accurate facial image.
Therefore, in the first sample identity characteristic and the non-identity characteristic of first sample for getting first sample facial image Afterwards, in order to generate corresponding facial image, processing equipment carries out first sample identity characteristic and the non-identity characteristic of first sample Combination, obtains the first characteristics of image.First characteristics of image includes first sample identity characteristic and the non-identity characteristic of first sample, The image information of first sample facial image can be described.
In a kind of mode in the cards, combination be splice and combine mode, i.e., by first sample identity characteristic with The non-identity characteristic of first sample is spliced, and spliced feature is the first characteristics of image.
For example, first sample identity characteristic is the characteristic that dimension is 128 dimensions, the non-identity characteristic of first sample is dimension For the characteristic of 128 dimensions, after the two is spliced, the first characteristics of image that dimension is 256 dimensions is obtained.
605, based on current Face image synthesis model, the corresponding first specified face figure of the first characteristics of image is generated Picture.
Processing equipment obtains current Face image synthesis model, and the first characteristics of image is input to the Face image synthesis In model, the first characteristics of image is handled by the Face image synthesis model, the first characteristics of image of output is corresponding specified Facial image, i.e., the first specified facial image.The first specified facial image belongs to first kind facial image.
Wherein, current Face image synthesis model can be the Face image synthesis model of initialization, or warp It crosses one or many training but does not train the Face image synthesis model completed also.
606, the first image error between first sample facial image and the first specified facial image is obtained.
607, according to the first image error, model parameter and Face image synthesis model that third feature extracts model are adjusted Model parameter.
Since Face image synthesis model is when generating facial image, the specified facial image and practical facial image of generation Between may have image error.Therefore, processing equipment obtain first sample facial image and the first specified facial image it Between image error, i.e. the first image error.First image error is specified for describing first sample facial image and first Difference between facial image.
The image error is smaller, indicates that the accuracy rate of Face image synthesis model is higher, and the image error is bigger, indicates people The accuracy rate that face image generates model is lower.
The effect of third feature extraction model is the non-identity characteristic in order to obtain first kind facial image, and facial image is raw Effect at model be in order to generate facial image true enough, that is, the specified facial image generated will as far as possible with it is corresponding Original facial image is close.Since the first image error can reflect the current accuracy rate of Face image synthesis model, root According to the first image error, adjusts third feature and extract the model parameter of model and the model parameter of Face image synthesis model, with Make based on the non-identity characteristic for having trained the third feature completed extraction model to extract first kind facial image, and is based on first Characteristics of image after the identity characteristic combination that Feature Selection Model extracts first kind facial image, is input to and has trained completion After Face image synthesis model, the obtained image error between specified facial image and corresponding first kind facial image is received It holds back.The image error is smaller, indicates that third feature extracts model and the training effect of Face image synthesis model is better, the image Error is bigger, indicates that third feature extracts model and the training effect of Face image synthesis model is poorer.
In a kind of possible implementation, model is extracted for third feature, third loss function is set, it is raw for facial image At model, the 4th loss function is set, the output valve of the third loss function by extracted first kind facial image non-identity The error of feature determines that, with the error correlation, the output valve of the 4th loss function is by Face image synthesis model The image error between specified facial image and original facial image generated determines, with the image error correlation. Therefore, the training objective that third feature extracts model and Face image synthesis model can be with are as follows: the output valve of third loss function Convergence, and the 4th loss function output valve convergence, with guarantee based on train complete third feature extraction model to any Image after the non-identity characteristic that target facial image extracts and the identity characteristic combination based on fisrt feature extraction model extraction Feature after being input to the Face image synthesis model trained and completed, obtains specified facial image and corresponding target face figure Image error convergence as between.
Wherein, the third loss function and the 4th loss function can be Softmax loss function, or can also be Unknown losses function.
One or many training are carried out using above-mentioned training method, and according to above-mentioned training objective, until third feature mentions Until modulus type and Face image synthesis model are restrained, training is completed, and the third feature of completion has been trained to extract model With Face image synthesis model.
After the completion of third feature extracts model training, any first kind facial image can be input to and train completion Third feature extracts model, extracts model based on the third feature, carries out feature extraction to the first kind facial image, is somebody's turn to do The non-identity characteristic of first kind facial image.
After the completion of Face image synthesis model training, any image feature can be input to the face figure trained and completed As generating model, it is based on the Face image synthesis model, generates the corresponding specified facial image of the characteristics of image.
608, model is extracted based on current second feature, feature extraction is carried out to the second sample facial image, is somebody's turn to do Second sample identity feature of the second sample facial image.
Step 608 is similar with the step 304 in above-described embodiment, and details are not described herein.
609, based on having trained the third feature completed to extract model, feature extraction is carried out to first sample facial image, Obtain the non-identity characteristic of the second sample of the first sample facial image.
The accurate non-identity of first kind facial image can be extracted by having trained the third feature completed to extract model Feature, first sample facial image is input to by processing equipment has trained the third feature completed to extract in model, by third spy Sign extracts model and carries out feature extraction to first sample facial image, obtains the non-identity characteristic of first sample facial image, i.e., The non-identity characteristic of second sample.
Since the accuracy rate for having trained the third feature completed to extract model is extracted higher than the third feature that training is completed Model, and the non-identity characteristic of first sample is to extract model extraction based on the third feature that training is not completed to obtain, the second sample This non-identity characteristic has been obtained based on train the third feature completed to extract model extraction, therefore the non-identity spy of the second sample It is more accurate to levy identity characteristic more non-than first sample.
610, the second sample identity feature and the non-identity characteristic of the second sample are combined, obtain the second characteristics of image, Based on the Face image synthesis model completed has been trained, the corresponding second specified facial image of the second characteristics of image is generated.
Since first sample facial image is corresponded to each other with the second sample facial image, the image information of the two should Unanimously, the identity characteristic and non-identity characteristic for being included should be identical.However, compared with first sample facial image, the second sample The resolution ratio of this facial image is lower, may lack parts of images information.
Therefore, facial image is more accurately specified in order to generate according to the characteristics of image of the second sample facial image, place Equipment is managed using the non-identity characteristic of the second sample as the non-identity characteristic of the second sample facial image, by the second sample identity feature It is combined with the non-identity characteristic of the second sample, obtains the second characteristics of image.Second characteristics of image includes the second sample identity Feature and the non-identity characteristic of the second sample, can describe the image information of the second sample facial image.
After getting the second characteristics of image, which is input to the face figure trained and completed by processing equipment As generating in model, by having trained the Face image synthesis model completed to handle the second characteristics of image, the second figure is exported As the corresponding specified facial image of feature, i.e., the second specified facial image.The second specified facial image belongs to first kind face Image.
Since the second specified facial image is the facial image generated according to the image information of the second sample facial image, and Second sample facial image is therefore the second class facial image can be generated by Face image synthesis model than the second sample The higher first kind facial image of the resolution ratio of facial image, realizes and restores high-resolution human out by low-resolution face image The effect of face image.
611, the second image error between first sample facial image and the second specified facial image is obtained, according to second Error between image error and first sample identity characteristic and the second sample identity feature, adjustment second feature extract model Model parameter.
When second feature extracts model also not training completion, the second sample that model extraction arrives is extracted based on second feature Identity characteristic possibly can not accurate description the second sample facial image identity information, therefore, by the second sample identity feature and The second characteristics of image that the non-identity characteristic of second sample combines also can not accurate description the second sample facial image, lead to base Not accurate enough in the second specified facial image that the second characteristics of image is got, there are images between the second sample facial image Error.
It is missed for this purpose, processing equipment obtains the second image between the second sample facial image and the second specified facial image Difference, according to the error between the second image error and first sample identity characteristic and the second sample identity feature, adjustment second The model parameter of Feature Selection Model, so as to be mentioned based on having trained the second feature completed to extract model to the second class facial image The identity characteristic taken extracts what model extracted the corresponding first kind facial image of the second class facial image with based on fisrt feature Error convergence between identity characteristic, to guarantee that the identity characteristic extracted can describe the second class face as precisely as possible The identity information for the face for including in image, and the corresponding first kind facial image of the second class facial image with based on having trained At Face image synthesis model generate specified facial image between image error convergence.
One or many training are carried out using above-mentioned training method, and according to above-mentioned training objective, until second feature mentions Until modulus type restrains, the second feature of completion is trained to extract model.
The process that training second feature extracts model is similar with step 405 in above-described embodiment, and details are not described herein.
612, target facial image is obtained.
Wherein, target facial image belongs to the second class facial image.
613, based on having trained the second feature completed to extract model, feature extraction is carried out to target facial image, is obtained The target identities feature of target facial image.
Step 612-613 is similar with step 306-307 in above-described embodiment, and details are not described herein.
614, it by target identities feature and presets non-identity characteristic and is combined, obtain target image characteristics.
Since Face image synthesis model is used for according to the characteristics of image for including identity characteristic and non-identity characteristic, generation pair The the second class facial image answered, therefore, after obtaining target identities feature, non-identity characteristic is preset in processing equipment acquisition, by this It presets non-identity characteristic and target identities feature is combined, obtain target image characteristics, the target image characteristics are for describing Target facial image.
Wherein, this is preset non-identity characteristic and can be determined according to the non-identity characteristic of general facial image, as general Facial image non-identity characteristic.Since the image information of the second class facial image is less, the dimension of non-identity characteristic is insufficient, Therefore by way of introducing and presetting non-identity characteristic, characteristic dimension can be increased, obtain the characteristics of image that dimension is met the requirements, It is subsequent to generate first kind facial image according to the characteristics of image.
For example, target identities feature is the vector of M dimension, the null vector that non-identity characteristic is a N-dimensional is preset, will be obtained The target identities feature got is combined with non-identity characteristic is preset, and obtains target image characteristics, the target image characteristics Dimension is the sum of M and N, and M and N are positive integer.
615, based on the Face image synthesis model completed has been trained, the corresponding specified face figure of target image characteristics is obtained Picture.
After getting target image characteristics, target image characteristics are input to the face figure trained and completed by processing equipment As generating in model, by having trained the Face image synthesis model completed to handle target image characteristics, the target is exported The corresponding specified facial image of characteristics of image.The specified facial image is the corresponding first kind facial image of target facial image.
The picture quality of target facial image is poor, and resolution ratio is lower, and method provided in an embodiment of the present invention being capable of root Going out corresponding first kind facial image according to the second class face image restoration, the picture quality of the first kind facial image is higher, point Resolution is higher, and expressed information is more, and user can view more information by the first kind facial image.
For example, being more clear when getting the lower old photo comprising face of a resolution ratio in order to view Photo, method provided in an embodiment of the present invention can be used, obtain the corresponding identity characteristic of the old photo, based on having trained At Face image synthesis model, the corresponding facial image of the identity characteristic is generated, to obtain the corresponding high score of the old photo The photo of resolution.
Method provided in an embodiment of the present invention extracts model based on current third feature, obtains the non-identity of first sample Feature will obtain the first characteristics of image after first sample identity characteristic and the non-identity characteristic combination of first sample, based on current Face image synthesis model generates the first specified facial image, according to first sample facial image and the first specified facial image Between the first image error, adjustment third feature extract model model parameter and Face image synthesis model model ginseng Number.Based on having trained the third feature completed to extract model, the non-identity characteristic of the second sample is obtained, based on the people for having trained completion Face image generates model, generates the corresponding second specified facial image of the second characteristics of image, according to first sample identity characteristic, the The second image error between two sample identity features and first sample facial image and the second specified facial image, adjustment the The model parameter of two Feature Selection Models.The embodiment of the present invention extracts model and Face image synthesis mould by setting third feature Type, available non-identity characteristic and specified facial image, according between specified facial image and corresponding original facial image Difference be trained, can make second feature extract model learning to subject between facial image and corresponding identity characteristic more True relationship further increases the performance that second feature extracts model, so that based on having trained the second feature completed to extract mould Type can obtain the more accurate identity characteristic of the second class facial image, improve the second class facial image extracted The Stability and veracity of identity characteristic.
Also, the target image that the embodiment of the present invention obtains target identities feature and presets after non-identity characteristic combination is special Sign obtains the corresponding specified facial image of target image characteristics, Neng Gouji based on the Face image synthesis model completed has been trained The corresponding first kind facial image of any second class facial image is generated in Face image synthesis model, is realized by the second class people Face image restores the effect of first kind facial image out, improves the picture quality of facial image, can get comprising more Other processing operations are checked to facial image or carried out to the facial image of information convenient for subsequent.
The schematic diagram of Fig. 7 another network architecture provided in an embodiment of the present invention, extracts model as high score using fisrt feature Resolution identity characteristic extracts model 501, second feature extracts model and extracts model 502, third spy for low resolution identity characteristic It is that the non-identity characteristic of high-resolution extracts model 503, Face image synthesis model is that Image restoration 504 is that sign, which extracts model, Example.The process for then adjusting the model parameter that low resolution identity characteristic extracts model 502 includes the following steps:
1, training high-resolution identity characteristic extracts model 501.
2, multiple groups sample facial image is obtained, includes mutual corresponding high-resolution human face figure in every group of sample facial image Picture and low-resolution face image.
3, it is directed to every group of sample facial image, high-resolution human face image is input to high-resolution identity characteristic and extracts mould Type 501 obtains the first sample identity characteristic of high-resolution human face image；High-resolution human face image is input to high-resolution Non- identity characteristic extracts model 503, obtains the non-identity characteristic of first sample of high-resolution human face image.
4, first sample identity characteristic and the non-identity characteristic of first sample are combined, it is special obtains first sample image Sign, is input to Image restoration 504 for the first sample characteristics of image, obtains the first specified facial image.
5, the first specified facial image and high-resolution human face image are compared, error between the two is obtained, according to the mistake Difference, the non-identity characteristic of adjustment high-resolution extract the model parameter of model 503 and the model parameter of Image restoration 504.
6, according to multiple groups sample facial image, above-mentioned steps 3-5 is repeated, mould is extracted to the non-identity characteristic of high-resolution Type 503 and Image restoration 504 are repeatedly adjusted, and the non-identity characteristic of the high-resolution of completion has been trained to extract model 503 and Image restoration 504.
7, it is directed to every group of sample facial image, low-resolution face image is input to low resolution identity characteristic and extracts mould Type 502 obtains the second sample identity feature of low-resolution face image.
8, the second sample identity feature and the non-identity characteristic of first sample are combined, obtain the second sample image spy Sign, is input to Image restoration 504 for the second sample image feature, obtains the second specified facial image.
9, high-resolution human face image and the second specified facial image are compared, error between the two is obtained, according to the mistake Difference and the error between first sample identity characteristic and the second sample identity feature, adjustment low resolution identity characteristic extract mould The model parameter of type 502.
10, according to multiple groups sample facial image, above-mentioned steps 7-9 is repeated, mould is extracted to low resolution identity characteristic Type 502 is repeatedly adjusted, and the low resolution identity characteristic of completion has been trained to extract model 502.
Fig. 8 is a kind of flow chart of face image processing process provided in an embodiment of the present invention, the face image processing side Method, according to having trained the fisrt feature completed to extract model, proposes third feature after the completion of fisrt feature extracts model training Modulus type, Face image synthesis model and discrimination model are trained, and extract model, Face image synthesis model in third feature After the completion of discrimination model training, model, Face image synthesis model and discrimination model are extracted according to third feature, to the second spy Sign is extracted model and is trained.
The executing subject of the embodiment of the present invention is processing equipment, which can obtain service for terminal shown in Fig. 2 Device, referring to Fig. 8, this method comprises:
801, first sample facial image and corresponding second sample facial image are obtained.
802, based on having trained the fisrt feature completed to extract model, feature extraction is carried out to first sample facial image, Obtain the first sample identity characteristic of first sample facial image.
803, model is extracted based on current third feature, feature extraction is carried out to first sample facial image, is somebody's turn to do The non-identity characteristic of the first sample of first sample facial image.
804, first sample identity characteristic and the non-identity characteristic of first sample are combined, obtain the first characteristics of image, Based on current Face image synthesis model, the corresponding first specified facial image of the first characteristics of image is generated.
Step 801-804 is similar with the step 601-605 in above-described embodiment, and details are not described herein.
805, based on current discrimination model, differentiation processing is carried out to first sample facial image, obtains the first differentiation knot Fruit carries out differentiation processing to the first specified facial image, obtains the second differentiation result.
After generating the first specified facial image based on Face image synthesis model, it can determine that this refers to based on discrimination model Determine the probability that facial image is original facial image.It is subsequent Face image synthesis model to be trained according to the probability, And then improve the treatment effect of Face image synthesis model.
Processing equipment obtains current discrimination model, and first sample facial image is input in discrimination model, and being based on should Discrimination model carries out differentiation processing to first sample facial image, and output first sample facial image is first sample face figure The probability of picture, i.e., first differentiates result.First specified facial image is input in discrimination model, the discrimination model is based on, it is right First specified facial image carries out differentiation processing, exports the probability that the first specified facial image is first sample facial image, I.e. second differentiates result.
The differentiation result of discrimination model output can indicate the face from Face image synthesis mode input to discrimination model Difference between image and original facial image, the difference is smaller, indicates that the treatment effect of Face image synthesis model is better, should Difference is bigger, indicates that the treatment effect of Face image synthesis model is poorer.
In a kind of possible implementation, differentiate that result can be the first result or second as a result, wherein differentiating that result is It is not original facial image that the expression of first result, which is input to the facial image of discrimination model, differentiates that result is that the second result indicates defeated Entering to the facial image of discrimination model is original facial image.
For example, the discrimination model is two classifiers, the output result of two classifier includes 0 and 1, wherein 0 indicates The facial image for being input to the discrimination model is original facial image, and 1 indicates that the facial image for being input to the discrimination model is not Original facial image.
In alternatively possible implementation, any value of the result between the first numerical value and second value is differentiated, the Two numerical value are greater than the first numerical value.For the numerical value closer to the first numerical value, it is original that expression, which is input to the facial image of discrimination model, The probability of facial image is smaller, differentiates result close to second value, it is original that expression, which is input to the facial image of discrimination model, The probability of facial image is larger.Wherein, the first numerical value can be 0, and second value can be 1.
806, based on having trained the fisrt feature completed to extract model, feature extraction is carried out to the first specified facial image, Obtain the first specified identity characteristic.
The first specified facial image is input to and has trained after getting the first specified facial image by processing equipment At fisrt feature extract model, model is extracted by fisrt feature, feature extraction is carried out to the first specified facial image, output should The identity characteristic of first specified facial image, i.e., the first specified identity characteristic.
807, the first identity characteristic error between the first specified identity characteristic and first sample identity characteristic is obtained.
Since Face image synthesis model is when generating facial image, the specified facial image and practical facial image of generation Between may have image error.Therefore, between the first specified facial image and first sample facial image there is also error, The the first specified identity characteristic obtained according to the first specified facial image and the first sample obtained according to first sample facial image There is also errors between this identity characteristic.
Therefore, processing equipment obtains the identity characteristic between the first specified identity characteristic and first sample identity characteristic and misses Difference, i.e. the first identity characteristic error.The first identity characteristic error is for describing the first specified identity characteristic and first sample body Difference between part feature.Wherein, the first identity characteristic error is bigger, indicates the first specified identity characteristic and first sample identity Difference between feature is bigger, and the first identity characteristic error is smaller, indicates that the first specified identity characteristic and first sample identity are special Difference between sign is smaller.
808, differentiate that result and the first default error differentiated between result, the second differentiation result and second are pre- according to first If differentiating the error and the first identity characteristic error between result, adjustment third feature extracts the model parameter of model, face Image generates the model parameter of model and the model parameter of discrimination model.
Wherein, the first default facial image for differentiating that result expression is input to discrimination model is original facial image, rather than The specified facial image that Face image synthesis model generates.The second default face figure for differentiating result expression and being input to discrimination model Specified facial image as being the generation of Face image synthesis model, rather than original facial image.First sample facial image is original Beginning facial image, therefore the practical differentiation result of first sample facial image is the first default differentiation as a result, the first specified face Image is the facial image of Face image synthesis, therefore the practical differentiation result of the first specified facial image is the second default differentiation As a result.
After getting the first identity characteristic error, processing equipment differentiates that result is default with first according to first obtained Differentiate that the error between result, second differentiate result and the second default error and the first identity characteristic differentiated between result Error, adjustment third feature extract model parameter, the model parameter of Face image synthesis model and the mould of discrimination model of model Shape parameter, so that based on having trained the discrimination model completed to carry out the differentiation knot obtained after differentiation processing to first kind facial image The error convergence of fruit, based on having trained the discrimination model completed to carry out the error convergence of the differentiation result obtained after differentiation processing, And model is extracted based on fisrt feature, mould is extracted with based on the fisrt feature to the identity characteristic that the first kind facial image extracts The error convergence between identity characteristic that type specified facial image corresponding to the first kind facial image extracts.
In a kind of possible implementation, model is extracted for third feature, third loss function is set, it is raw for facial image The 4th loss function is set at model, the 5th loss function, third loss function and the 4th loss function are set for discrimination model Similar with the step 707 in above-described embodiment, details are not described herein, and the output valve of the 5th loss function is by discrimination model Differentiate that the error of result determines, the error correlation with the differentiation result.Therefore, third feature extracts model, face The adjustment target that image generates model and discrimination model can be with are as follows: the output valve of third loss function restrains, the 4th loss function Output valve convergence, and the 5th loss function output valve convergence.The third feature completed based on training extracts model to any Image after the non-identity characteristic that target facial image extracts and the identity characteristic combination based on fisrt feature extraction model extraction Feature, after being input to the Face image synthesis model trained and completed, obtain the identity characteristic of specified facial image with it is corresponding Error convergence between the identity characteristic of target facial image, and the error convergence of the differentiation result of discrimination model output.
809, based on having trained the third feature completed to extract model, feature extraction is carried out to first sample facial image, The non-identity characteristic of the second sample of the first sample facial image is obtained, model is extracted based on current second feature, to second Sample facial image carries out feature extraction, obtains the second sample identity feature of the second sample facial image.
Step 809 is similar with step 608-609 in above-described embodiment, and details are not described herein.
810, the second sample identity feature and the non-identity characteristic of the second sample are combined, obtain the second characteristics of image, Based on the Face image synthesis model completed has been trained, the corresponding second specified facial image of second characteristics of image is generated, is obtained Take the second image error between first sample facial image and the second specified facial image.
Step 810 is similar with step 610-611 in above-described embodiment, and details are not described herein.
811, based on the discrimination model completed has been trained, differentiation processing is carried out to the second specified facial image, obtains third Differentiate result.
The more accurate differentiation of the second specified facial image is as a result, the second specified facial image is input in order to obtain Training complete discrimination model, based on train completion discrimination model differentiation processing is carried out to the second specified facial image, it is defeated The differentiation of the second specified facial image is as a result, i.e. third differentiates result out.The third differentiates the result is that the second specified facial image It is the probability of first sample facial image.
Due to the identity characteristic in the characteristics of image for generating the second specified facial image, be based on trained complete Third feature extracts what model extraction arrived, and the second specified facial image is based on the Face image synthesis model life for having trained completion At, and third differentiates that therefore, which differentiates the result is that based on having trained the discrimination model differentiation completed to obtain after handling As a result accuracy is higher.
812, based on having trained the fisrt feature completed to extract model, feature extraction is carried out to the second specified facial image, The second specified identity characteristic of the second specified facial image is obtained, the second specified identity characteristic and first sample identity characteristic are obtained Between Second Identity of Local error.
After getting the second specified facial image, the second specified facial image is input to the fisrt feature trained and completed It extracts in model, feature extraction is carried out to the second specified facial image based on having trained the fisrt feature completed to extract model, it is defeated The identity characteristic of second specified facial image out, i.e., the second specified identity characteristic.
Processing equipment obtains the identity error between the second specified identity characteristic and first sample identity characteristic, i.e., and second Identity characteristic error, the Second Identity of Local error is for describing between the second specified identity characteristic and first sample identity characteristic Difference.Wherein, Second Identity of Local error is bigger, indicates between the second specified identity characteristic and first sample identity characteristic Difference is bigger, and Second Identity of Local error is smaller, indicates the difference between the second specified identity characteristic and first sample identity characteristic It is different smaller.
813, according to the error between first sample identity characteristic and the second sample identity feature, the second image error, Three differentiate that result and the second default error and Second Identity of Local error differentiated between result, adjustment second feature are extracted The model parameter of model.
Wherein, since the second specified facial image is the facial image that people's face image generates that model generates, second refers to The practical differentiation result for determining facial image is the second default differentiation result.
After getting Second Identity of Local error, processing equipment is according to the first sample identity characteristic obtained and second Error, the second image error, third between sample identity feature differentiate result and the second default error differentiated between result, And Second Identity of Local error, adjustment second feature extract the model parameter of model, so that based on second completed has been trained The identity characteristic that Feature Selection Model extracts the second class facial image extracts model to the second class face with based on fisrt feature The error convergence between identity characteristic that the corresponding first kind facial image of image extracts, to guarantee the identity characteristic energy extracted The identity information of enough faces for describing to include in the second class facial image as precisely as possible extracts mould based on fisrt feature The second feature that type, training are completed extracts model, third feature extracts the nominator that model and Face image synthesis model obtain Image error convergence between face image and corresponding second class facial image, obtains after carrying out differentiation processing based on discrimination model Differentiation result error convergence, and the identity characteristic that extracts to first kind facial image of model is extracted based on fisrt feature, with The mistake between the identity characteristic that model specified facial image corresponding to the second class facial image extracts is extracted based on fisrt feature Difference convergence.
One or many training are carried out using above-mentioned training method, and according to above-mentioned training objective, until second feature mentions Until modulus type restrains, the second feature of completion is trained to extract model.
The process being trained to second feature extraction model is similar with step 405 in above-described embodiment, no longer superfluous herein It states.
It should be noted that after the completion of discrimination model training, can be applied to any face figure in the embodiment of the present invention It seem in the no scene differentiated for original facial image.For example, only needing to differentiate that any facial image belongs to according to figure In the scene of the facial image or original facial image that are generated as feature, the facial image can be carried out based on discrimination model Differentiation processing obtains differentiating result.
Also, the Face image synthesis model in the embodiment of the present invention and discrimination model together constitute a confrontation and generate Model, Face image synthesis model generates facial image as true as possible to confuse discrimination model, and discrimination model is then to defeated The facial image entered is differentiated judge whether the facial image is original facial image.It, can be with by the common training of the two Mutually study, it is common to fight, to promote the performance of the two simultaneously.
Method provided in an embodiment of the present invention obtains the first specified facial image, based on current discrimination model, obtains the One differentiates that result and second differentiates as a result, obtaining the first identity between the first specified identity characteristic and first sample identity characteristic Characteristic error.According to the error between the first differentiation result and the default differentiation result of first sample facial image, the second differentiation As a result the error between the default differentiation result of the first specified facial image and the first identity characteristic error adjust third Model parameter, the model parameter of Face image synthesis model and the model parameter of discrimination model of Feature Selection Model.Later, it obtains The second image error between the second characteristics of image and corresponding second specified facial image is taken, based on the differentiation for having trained completion Model obtains third and differentiates as a result, based on train the fisrt feature completed to extract model the second specified identity characteristic of acquisition and the Two identity characteristic errors, according between first sample identity characteristic and the second sample identity feature error, the second image error, Third differentiates the error and Second Identity of Local error between result and the default differentiation result of the second specified facial image, Adjust the model parameter that second feature extracts model.The embodiment of the present invention can make discrimination model by setting discrimination model It is trained together with extracting model and Face image synthesis model with third feature, promotes the performance of each model, and then according to training Third feature afterwards extracts model, Face image synthesis model and discrimination model, extracts model to second feature and is trained, can Further to promote the performance that second feature extracts model, improves and the second class people that model extraction arrives is extracted based on second feature The accuracy of the identity characteristic of face image.
It should be noted that each model that above-described embodiment is related to can be optimized using gradient descent method, Optimization process is, gradually complete using the parameter of the continuous iteration of optimization algorithm more new model during being trained to model The training of pairs of model.Alternatively, can also be optimized using other algorithms to model, such as Adam (adaptive moment Estimation, adaptability moments estimation) algorithm.Since training process can be related to a large amount of sample or characteristic, be The efficiency for improving training pattern, can be trained, such as small lot form or small batch style in the form of batch.
It should be noted that each model involved in above-described embodiment can be instructed using default training algorithm Practice, which can be a variety of calculations such as convolutional neural networks algorithm, Recognition with Recurrent Neural Network algorithm, decision Tree algorithms Method.
Fig. 9 is the schematic diagram of another network architecture provided in an embodiment of the present invention, and extracting model with fisrt feature is height Resolution ratio identity characteristic extracts model 501, second feature extracts model and extracts model 502, third for low resolution identity characteristic Feature Selection Model is that the non-identity characteristic of high-resolution extracts model 503, Face image synthesis model is Image restoration 504 For, it further include discrimination model 505.It then adjusts low resolution identity characteristic and extracts the process of model parameter of model 502 and include Following steps:
1, training high-resolution identity characteristic extracts model 501.
2, multiple groups sample facial image is obtained, includes mutual corresponding high-resolution human face figure in every group of sample facial image Picture and low-resolution face image.
3, it is directed to every group of sample facial image, high-resolution human face image is input to high-resolution identity characteristic and extracts mould Type 501 obtains the first sample identity characteristic of high-resolution human face image；High-resolution human face image is input to high-resolution Non- identity characteristic extracts model 503, obtains the non-identity characteristic of first sample of high-resolution human face image.
4, first sample identity characteristic and the non-identity characteristic of first sample are combined, it is special obtains first sample image Sign, is input to Image restoration 504 for the first sample characteristics of image, obtains the first specified facial image.
5, the first specified facial image is input to high-resolution identity characteristic to extract in model 501, it is specified obtains first Identity characteristic；First specified facial image is input in discrimination model 505, the first differentiation result is obtained；By high-resolution human Face image is input in discrimination model 505, obtains the second differentiation result.
6, first sample identity characteristic and the first specified identity characteristic are compared, error between the two is obtained, according to the mistake Difference, first differentiates error, the second differentiation result and first between result and the default differentiation result of high-resolution human face image Error between the default differentiation result of specified facial image, the non-identity characteristic of adjustment high-resolution extract the model of model 503 The model parameter of parameter, the model parameter of Image restoration 504 and discrimination model 505.
7, according to multiple groups sample facial image, above-mentioned steps 3-6 is repeated, mould is extracted to the non-identity characteristic of high-resolution Type 503, Image restoration 504 and discrimination model 505 are repeatedly adjusted, and the non-identity of high-resolution for obtaining training completion is special Sign extracts model 503, Image restoration 504 and discrimination model 505.
8, it is directed to every group of sample facial image, low-resolution face image is input to low resolution identity characteristic and extracts mould Type 502 obtains the second sample identity feature of low-resolution face image.
9, high-resolution human face image is input to the non-identity characteristic of high-resolution to extract in model 503, obtains the second sample This non-identity characteristic, the second sample identity feature and the non-identity characteristic of the second sample are combined, the second sample image is obtained Feature.
10, the second sample image feature is input to Image restoration 504, obtains the second specified facial image；It will Second specified facial image is input to discrimination model 505, obtains third and differentiates as a result, the second specified facial image is input to height Resolution ratio identity characteristic extracts in model 501, obtains the second specified identity characteristic.
11, comparison the second specified facial image and high-resolution human face image obtains error 1 between the two, compares the Two specified identity characteristics and first sample identity characteristic obtain error 2 between the two, according to error 1, error 2, first sample Error between identity characteristic and the second sample identity feature and third differentiate the default of result and the second specified facial image Differentiate the error between result, adjustment low resolution identity characteristic extracts the model parameter of model 502.
12, according to multiple groups sample facial image, above-mentioned steps 8-11 is repeated, mould is extracted to low resolution identity characteristic Type 502 is repeatedly adjusted, and the low resolution identity characteristic for obtaining training completion extracts model 502.
Method provided in an embodiment of the present invention can also be applied to identification scene, identification check scene, facial image It restores in the several scenes such as scene.
In a kind of possible implementation, method provided in an embodiment of the present invention is applied in identification scene.For example, Company personnel is checked card by brush face in scene on and off duty, and company carries out man face image acquiring to all employees in advance, using this The method that inventive embodiments provide obtains the identity characteristic of each employee, and the identity characteristic of all employees is saved to identity data In library.When brush face equipment collects the facial image of any employee, using method provided in an embodiment of the present invention, face is obtained The identity characteristic of image, the identity characteristic in identity characteristic and identity database that will acquire compares, so that it is determined that being Which employee checks card success.
Figure 10 is a kind of structural schematic diagram of face image processing device provided in an embodiment of the present invention.It, should referring to Figure 10 Device includes:
First sample facial image obtains module 1001, and first sample facial image is obtained in above-described embodiment for executing And the step of corresponding second sample facial image；
Fisrt feature extraction module 1002, for executing in above-described embodiment based on the fisrt feature extraction for having trained completion Model carries out feature extraction to first sample facial image, obtains the first sample identity characteristic of first sample facial image Step；
Second feature extraction module 1003 extracts model based on current second feature for executing in above-described embodiment, The step of is carried out by feature extraction, obtains the second sample identity feature of the second sample facial image for second sample facial image；
The first adjustment module 1004, for executing in above-described embodiment according to first sample identity characteristic and the second sample body The step of error between part feature, adjustment second feature extracts the model parameter of model.
Optionally, device further include:
Target image obtains module, and target facial image, target facial image category are obtained in above-described embodiment for executing In the second class facial image the step of；
Target's feature-extraction module extracts model based on second feature for executing in above-described embodiment, to target face The step of image carries out feature extraction, obtains the target identities feature of target facial image.
Optionally, device further include:
Enquiry module obtains target for executing in above-described embodiment according to target identities characteristic query identity database The step of identity characteristic corresponding identity.
Optionally, device further include:
Second sample facial image obtains module, and third sample facial image and the are obtained in above-described embodiment for executing The step of third sample identity feature of three sample facial images；
Third feature extraction module extracts model based on current fisrt feature for executing in above-described embodiment, to the The step of three sample facial images carry out feature extraction, obtain the 4th sample identity feature of third sample facial image；
Second adjustment module, it is special according to third sample identity feature and the 4th sample identity in above-described embodiment for executing The step of error between sign, adjustment fisrt feature extracts the model parameter of model.
Optionally, device further include:
Fourth feature extraction module extracts model based on current third feature for executing in above-described embodiment, to the The step of this same facial image carries out feature extraction, obtains the first sample non-identity characteristic of first sample facial image；
Specified image collection module, for executing first sample identity characteristic and the non-body of first sample in above-described embodiment Part feature is combined, and obtains the first characteristics of image, based on current Face image synthesis model, generates the first characteristics of image pair The step of the first specified facial image answered；
Image error obtains module, and first sample facial image and the first nominator are obtained in above-described embodiment for executing The step of the first image error between face image；
Third adjusts module, for executing in above-described embodiment according to the first image error, adjusts third feature and extracts mould The step of model parameter of type and the model parameter of Face image synthesis model.
Optionally, the first adjustment module 1004, comprising:
Feature extraction unit is right for executing based on the third feature extraction model for having trained completion in above-described embodiment The step of first sample facial image carries out feature extraction, obtains the second sample non-identity characteristic of first sample facial image；
Image acquisition unit, it is in above-described embodiment that the second sample identity feature and the non-identity of the second sample is special for executing Sign is combined, and obtains the second characteristics of image, based on the Face image synthesis model completed has been trained, generates the second characteristics of image The step of corresponding second specified facial image；
Image error acquiring unit obtains first sample facial image and the second nominator for executing in above-described embodiment The step of the second image error between face image；
Adjustment unit, for executing in above-described embodiment according to the second image error and first sample identity characteristic and The step of error between two sample identity features, adjustment second feature extracts the model parameter of model.
Optionally, device further include:
Feature combination module, for executing target identities feature in above-described embodiment and presetting non-identity characteristic and carry out group The step of closing, obtaining target image characteristics；
Image generation module is obtained for executing in above-described embodiment based on the Face image synthesis model completed has been trained The step of taking target image characteristics corresponding specified facial image.
Optionally, third adjusts module, comprising:
Judgement unit, for executing in above-described embodiment based on current discrimination model, to first sample facial image into Row differentiation processing, obtains the first differentiation as a result, carrying out differentiation processing to the first specified facial image, obtains the second differentiation result Step；
Feature extraction unit is right for executing based on the fisrt feature extraction model for having trained completion in above-described embodiment The step of first specified facial image carries out feature extraction, obtains the first specified identity characteristic；
Characteristic error acquiring unit obtains the first specified identity characteristic and first sample body for executing in above-described embodiment The step of the first identity characteristic error between part feature；
Adjustment unit differentiates between result and the first default differentiation result for executing in above-described embodiment according to first Error, second differentiate that result and the second default error and the first identity characteristic error differentiated between result, adjustment third are special The step of levying the model parameter of the model parameter for extracting model, the model parameter of Face image synthesis model and discrimination model.
Optionally, the first adjustment module 1004, comprising:
Feature extraction unit is right for executing based on the third feature extraction model for having trained completion in above-described embodiment The step of first sample facial image carries out feature extraction, obtains the second sample non-identity characteristic of first sample facial image；
Image acquisition unit, it is in above-described embodiment that the second sample identity feature and the non-identity of the second sample is special for executing Sign is combined, and obtains the second characteristics of image, based on the Face image synthesis model completed has been trained, generates the second characteristics of image The step of corresponding second specified facial image；
Image error acquiring unit obtains first sample facial image and the second nominator for executing in above-described embodiment The step of the second image error between face image；
Judgement unit, for executing based on the discrimination model completed has been trained in above-described embodiment, to the second specified face Image carries out differentiation processing, obtains the step of third differentiates result；
Characteristic error acquiring unit, for executing in above-described embodiment based on the fisrt feature extraction mould for having trained completion Type carries out feature extraction to the second specified facial image, obtains the second specified identity characteristic of the second specified facial image, obtain The step of Second Identity of Local error between second specified identity characteristic and first sample identity characteristic；
Adjustment unit, for execute in above-described embodiment according to first sample identity characteristic and the second sample identity feature it Between error, the second image error, third differentiate that result and the second default error differentiated between result and the second identity are special Error is levied, the step of second feature extracts the model parameter of model is adjusted.
Figure 11 is a kind of structural schematic diagram of face image processing device provided in an embodiment of the present invention.It, should referring to Figure 11 Device includes:
Second feature obtains module 1101, and the step of second feature extracts model is obtained in above-described embodiment for executing；
Target's feature-extraction module 1102 extracts model based on second feature for executing in above-described embodiment, to belonging to The target facial image of second class facial image carries out feature extraction, obtains the step of the target identities feature of target facial image Suddenly.
Optionally, device further include:
Composite module, for executing target identities feature in above-described embodiment and presetting non-identity characteristic and be combined, The step of obtaining target image characteristics；
Specified image collection module obtains target figure for executing in above-described embodiment based on Face image synthesis model The step of specified facial image corresponding as feature.
Optionally, device further include:
Enquiry module obtains target for executing in above-described embodiment according to target identities characteristic query identity database The step of identity characteristic corresponding identity.
It should be understood that face image processing device provided by the above embodiment is when handling facial image, only more than The division progress of each functional module is stated for example, can according to need and in practical application by above-mentioned function distribution by difference Functional module complete, i.e., the internal structure of processing equipment is divided into different functional modules, it is described above complete to complete Portion or partial function.In addition, face image processing device provided by the above embodiment and face image processing process embodiment Belong to same design, specific implementation process is detailed in embodiment of the method, and which is not described herein again.
Figure 12 shows the structural block diagram of the terminal 1200 of an illustrative embodiment of the invention offer.The terminal 1200 can To be portable mobile termianl, such as: smart phone, tablet computer, MP3 player (MovingPicture Experts Group Audio Layer III, dynamic image expert's compression standard audio level 3), MP4 (Moving Picture Experts Group Audio Layer IV, dynamic image expert's compression standard audio level 4) player, laptop, Desktop computer, headset equipment or any other intelligent terminal.Terminal 1200 is also possible to referred to as user equipment, portable terminal Other titles such as end, laptop terminal, terminal console.
In general, terminal 1200 includes: processor 1201 and memory 1202.
Processor 1201 may include one or more processing cores, such as 4 core processors, 8 core processors etc..Place Reason device 1201 can use DSP (Digital Signal Processing, Digital Signal Processing), FPGA (Field- Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, may be programmed Logic array) at least one of example, in hardware realize.Processor 1201 also may include primary processor and coprocessor, master Processor is the processor for being handled data in the awake state, also referred to as CPU (Central Processing Unit, central processing unit)；Coprocessor is the low power processor for being handled data in the standby state.? In some embodiments, processor 1201 can be integrated with GPU (Graphics Processing Unit, image processor), GPU is used to be responsible for the rendering and drafting of content to be shown needed for display screen.In some embodiments, processor 1201 can also be wrapped AI (Artificial Intelligence, artificial intelligence) processor is included, the AI processor is for handling related machine learning Calculating operation.
Memory 1202 may include one or more computer readable storage mediums, which can To be non-transient.Memory 1202 may also include high-speed random access memory and nonvolatile memory, such as one Or multiple disk storage equipments, flash memory device.In some embodiments, the non-transient computer in memory 1202 can Storage medium is read for storing at least one instruction, at least one instruction by processor 1201 for being had to realize this Shen Please in embodiment of the method provide face image processing process.
In some embodiments, terminal 1200 is also optional includes: peripheral device interface 1203 and at least one periphery are set It is standby.It can be connected by bus or signal wire between processor 1201, memory 1202 and peripheral device interface 1203.It is each outer Peripheral equipment can be connected by bus, signal wire or circuit board with peripheral device interface 1203.Specifically, peripheral equipment includes: In radio circuit 1204, touch display screen 1205, camera 1206, voicefrequency circuit 1207, positioning component 1208 and power supply 1209 At least one.
Peripheral device interface 1203 can be used for I/O (Input/Output, input/output) is relevant outside at least one Peripheral equipment is connected to processor 1201 and memory 1202.In some embodiments, processor 1201, memory 1202 and periphery Equipment interface 1203 is integrated on same chip or circuit board；In some other embodiments, processor 1201, memory 1202 and peripheral device interface 1203 in any one or two can be realized on individual chip or circuit board, this implementation Example is not limited this.
Radio circuit 1204 is for receiving and emitting RF (Radio Frequency, radio frequency) signal, also referred to as electromagnetic signal. Radio circuit 1204 is communicated by electromagnetic signal with communication network and other communication equipments.Radio circuit 1204 is by telecommunications Number being converted to electromagnetic signal is sent, alternatively, the electromagnetic signal received is converted to electric signal.Optionally, radio circuit 1204 include: antenna system, RF transceiver, one or more amplifiers, tuner, oscillator, digital signal processor, volume solution Code chipset, user identity module card etc..Radio circuit 1204 can by least one wireless communication protocol come with it is other Terminal is communicated.The wireless communication protocol includes but is not limited to: Metropolitan Area Network (MAN), each third generation mobile communication network (2G, 3G, 4G and 8G), WLAN and/or WiFi (Wireless Fidelity, Wireless Fidelity) network.In some embodiments, radio frequency electrical Road 1204 can also include NFC (Near Field Communication, wireless near field communication) related circuit, the application This is not limited.
Display screen 1205 is for showing UI (User Interface, user interface).The UI may include figure, text, Icon, video and its their any combination.When display screen 1205 is touch display screen, display screen 1205 also there is acquisition to exist The ability of the touch signal on the surface or surface of display screen 1205.The touch signal can be used as control signal and be input to place Reason device 1201 is handled.At this point, display screen 1205 can be also used for providing virtual push button and/or dummy keyboard, it is also referred to as soft to press Button and/or soft keyboard.In some embodiments, display screen 1205 can be one, and the front panel of terminal 1200 is arranged；Another In a little embodiments, display screen 1205 can be at least two, be separately positioned on the different surfaces of terminal 1200 or in foldover design； In still other embodiments, display screen 1205 can be flexible display screen, is arranged on the curved surface of terminal 1200 or folds On face.Even, display screen 1205 can also be arranged to non-rectangle irregular figure, namely abnormity screen.Display screen 1205 can be with Using LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, Organic Light Emitting Diode) etc. materials preparation.
CCD camera assembly 1206 is for acquiring image or video.Optionally, CCD camera assembly 1206 includes front camera And rear camera.In general, the front panel of terminal is arranged in front camera, the back side of terminal is arranged in rear camera.? In some embodiments, rear camera at least two is that main camera, depth of field camera, wide-angle camera, focal length are taken the photograph respectively As any one in head, to realize that main camera and the fusion of depth of field camera realize background blurring function, main camera and wide Pan-shot and VR (Virtual Reality, virtual reality) shooting function or other fusions are realized in camera fusion in angle Shooting function.In some embodiments, CCD camera assembly 1206 can also include flash lamp.Flash lamp can be monochromatic temperature flash of light Lamp is also possible to double-colored temperature flash lamp.Double-colored temperature flash lamp refers to the combination of warm light flash lamp and cold light flash lamp, can be used for Light compensation under different-colour.
Voicefrequency circuit 1207 may include microphone and loudspeaker.Microphone is used to acquire the sound wave of user and environment, and It converts sound waves into electric signal and is input to processor 1201 and handled, or be input to radio circuit 1204 to realize that voice is logical Letter.For stereo acquisition or the purpose of noise reduction, microphone can be separately positioned on the different parts of terminal 1200 to be multiple. Microphone can also be array microphone or omnidirectional's acquisition type microphone.Loudspeaker is then used to that processor 1201 or radio frequency will to be come from The electric signal of circuit 1204 is converted to sound wave.Loudspeaker can be traditional wafer speaker, be also possible to piezoelectric ceramics loudspeaking Device.When loudspeaker is piezoelectric ceramic loudspeaker, the audible sound wave of the mankind can be not only converted electrical signals to, can also be incited somebody to action Electric signal is converted to the sound wave that the mankind do not hear to carry out the purposes such as ranging.In some embodiments, voicefrequency circuit 1207 may be used also To include earphone jack.
Positioning component 1208 is used for the current geographic position of positioning terminal 1200, to realize navigation or LBS (Location Based Service, location based service).Positioning component 1208 can be the GPS (Global based on the U.S. Positioning System, global positioning system), the dipper system of China, Russia Gray receive this system or European Union The positioning component of Galileo system.
Power supply 1209 is used to be powered for the various components in terminal 1200.Power supply 1209 can be alternating current, direct current Electricity, disposable battery or rechargeable battery.When power supply 1209 includes rechargeable battery, which can support wired Charging or wireless charging.The rechargeable battery can be also used for supporting fast charge technology.
In some embodiments, terminal 1200 further includes having one or more sensors 1120.One or more sensing Device 1120 includes but is not limited to: acceleration transducer 1211, gyro sensor 1212, pressure sensor 1213, fingerprint sensing Device 1214, optical sensor 1215 and proximity sensor 1216.
Acceleration transducer 1211 can detecte the acceleration in three reference axis of the coordinate system established with terminal 1200 Size.For example, acceleration transducer 1211 can be used for detecting component of the acceleration of gravity in three reference axis.Processor The 1201 acceleration of gravity signals that can be acquired according to acceleration transducer 1211, control touch display screen 1205 with transverse views Or longitudinal view carries out the display of user interface.Acceleration transducer 1211 can be also used for game or the exercise data of user Acquisition.
Gyro sensor 1212 can detecte body direction and the rotational angle of terminal 1200, gyro sensor 1212 Acquisition user can be cooperateed with to act the 3D of terminal 1200 with acceleration transducer 1211.Processor 1201 is according to gyro sensors The data that device 1212 acquires, following function may be implemented: action induction (for example changing UI according to the tilt operation of user) is clapped Image stabilization, game control and inertial navigation when taking the photograph.
The lower layer of side frame and/or touch display screen 1205 in terminal 1200 can be set in pressure sensor 1213.When When the side frame of terminal 1200 is arranged in pressure sensor 1213, user can detecte to the gripping signal of terminal 1200, by Reason device 1201 carries out right-hand man's identification or prompt operation according to the gripping signal that pressure sensor 1213 acquires.Work as pressure sensor 1213 when being arranged in the lower layer of touch display screen 1205, is grasped by processor 1201 according to pressure of the user to touch display screen 1205 Make, realization controls the operability control on the interface UI.Operability control include button control, scroll bar control, At least one of icon control, menu control.
Fingerprint sensor 1214 is used to acquire the fingerprint of user, is collected by processor 1201 according to fingerprint sensor 1214 Fingerprint recognition user identity, alternatively, by fingerprint sensor 1214 according to the identity of collected fingerprint recognition user.Knowing Not Chu user identity be trusted identity when, by processor 1201 authorize the user have relevant sensitive operation, sensitivity grasp Make to include solving lock screen, checking encryption information, downloading software, payment and change setting etc..Fingerprint sensor 1214 can be set Set the front, the back side or side of terminal 1200.When being provided with physical button or manufacturer Logo in terminal 1200, fingerprint sensor 1214 can integrate with physical button or manufacturer's mark.
Optical sensor 1215 is for acquiring ambient light intensity.In one embodiment, processor 1201 can be according to light The ambient light intensity that sensor 1215 acquires is learned, the display brightness of touch display screen 1205 is controlled.Specifically, work as ambient light intensity When higher, the display brightness of touch display screen 1205 is turned up；When ambient light intensity is lower, the aobvious of touch display screen 1205 is turned down Show brightness.In another embodiment, the ambient light intensity that processor 1201 can also be acquired according to optical sensor 1215, is moved The acquisition parameters of state adjustment CCD camera assembly 1206.
Proximity sensor 1216, also referred to as range sensor are generally arranged at the front panel of terminal 1200.Proximity sensor 1216 for acquiring the distance between the front of user Yu terminal 1200.In one embodiment, when proximity sensor 1216 is examined When measuring the distance between the front of user and terminal 1200 and gradually becoming smaller, by processor 1201 control touch display screen 1205 from Bright screen state is switched to breath screen state；When proximity sensor 1216 detect the distance between front of user and terminal 1200 by When gradual change is big, touch display screen 1205 is controlled by processor 1201 and is switched to bright screen state from breath screen state.
It, can be with it will be understood by those skilled in the art that the restriction of the not structure paired terminal 1200 of structure shown in Figure 12 Including than illustrating more or fewer components, perhaps combining certain components or being arranged using different components.
Figure 13 is a kind of structural schematic diagram of server provided in an embodiment of the present invention, the server 1300 can because of configuration or Performance is different and generates bigger difference, may include one or more processors (central processing Units, CPU) 1301 and one or more memory 1302, wherein at least one is stored in the memory 1302 Item instruction, at least one instruction are loaded by the processor 1301 and are executed to realize that above-mentioned each embodiment of the method provides Method.Certainly, which can also have the components such as wired or wireless network interface, keyboard and input/output interface, To carry out input and output, which can also include other for realizing the component of functions of the equipments, and this will not be repeated here.
Server 1300 can be used for executing step performed by processing equipment in above-mentioned face image processing process.
The embodiment of the invention also provides a kind of face image processing device, which includes processor and memory, is deposited At least one instruction, at least a Duan Chengxu, code set or instruction set, instruction, program, code set or instruction set are stored in reservoir Loaded by processor and had operation performed in the face image processing process to realize above-described embodiment.
The embodiment of the invention also provides a kind of computer readable storage medium, stored in the computer readable storage medium Have at least one instruction, at least a Duan Chengxu, code set or instruction set, the instruction, the program, the code set or the instruction set by Processor loads and has operation performed in the face image processing process to realize above-described embodiment.
Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable In storage medium, storage medium mentioned above can be read-only memory, disk or CD etc..
The foregoing is merely the preferred embodiments of the embodiment of the present invention, are not intended to limit the invention embodiment, all at this Within the spirit and principle of inventive embodiments, any modification, equivalent replacement, improvement and so on be should be included in of the invention Within protection scope.

Claims (15)

1. a kind of face image processing process, which is characterized in that the described method includes:
It obtains first sample facial image and corresponding second sample facial image, the first sample facial image belongs to first Class facial image, the second sample facial image belong to the second class facial image, the resolution ratio of the first kind facial image Higher than the resolution ratio of the second class facial image；
Based on having trained the fisrt feature completed to extract model, feature extraction is carried out to the first sample facial image, is obtained The first sample identity characteristic of the first sample facial image, the fisrt feature extract model for extracting the first kind The identity characteristic of facial image；
Model is extracted based on current second feature, feature extraction is carried out to the second sample facial image, obtains described the Second sample identity feature of two sample facial images, the second feature extract model for extracting the second class face figure The identity characteristic of picture；
According to the error between the first sample identity characteristic and the second sample identity feature, the second feature is adjusted The model parameter of model is extracted, so that based on having trained the second feature completed to extract model to the second class face figure As the identity characteristic extracted, model is extracted to the corresponding first kind people of the second class facial image with based on the fisrt feature The error convergence between identity characteristic that face image is extracted.
2. the method according to claim 1, wherein the method also includes:
Target facial image is obtained, the target facial image belongs to the second class facial image；
Model is extracted based on the second feature, feature extraction is carried out to the target facial image, obtains the target face The target identities feature of image.
3. according to the method described in claim 2, it is characterized in that, described extract model based on the second feature, to described Target facial image carries out feature extraction, and after obtaining the target identities feature of the target facial image, the method is also wrapped It includes:
According to the target identities characteristic query identity database, the corresponding identity of the target identities feature, institute are obtained Stating identity database includes at least one identity characteristic and corresponding identity.
4. the method according to claim 1, wherein described based on the fisrt feature extraction mould for having trained completion Type carries out feature extraction to the first sample facial image, obtains the first sample identity of the first sample facial image Before feature, the method also includes:
Obtain the third sample identity feature of third sample facial image and the third sample facial image, the third face Image belongs to the first kind facial image；
Model is extracted based on the current fisrt feature, feature extraction is carried out to the third sample facial image, obtains institute State the 4th sample identity feature of third sample facial image；
According to the error between the third sample identity feature and the 4th sample identity feature, the fisrt feature is adjusted The model parameter of model is extracted, so that based on having trained the fisrt feature completed to extract model to the first kind face figure As the identity characteristic extracted, the error convergence between the actual identity feature of the first kind facial image.
5. the method according to claim 1, wherein described based on the fisrt feature extraction mould for having trained completion Type carries out feature extraction to the first sample facial image, obtains the first sample identity of the first sample facial image After feature, the method also includes:
Model is extracted based on current third feature, feature extraction is carried out to the first sample facial image, obtains described the The non-identity characteristic of first sample of this same facial image, the third feature extract model for extracting the first kind face The non-identity characteristic of image；
The first sample identity characteristic and the non-identity characteristic of the first sample are combined, the first characteristics of image is obtained, Based on current Face image synthesis model, the corresponding first specified facial image of the first image feature, the people are generated Face image generates model and is used to generate the first kind facial image corresponding with described image feature according to characteristics of image；
Obtain the first image error between the first sample facial image and the first specified facial image；
According to the first image error, adjust the third feature extract model model parameter and the Face image synthesis The model parameter of model, so as to be mentioned based on having trained the third feature completed to extract model to the first kind facial image The non-identity characteristic taken, and the identity characteristic group that model extracts the first kind facial image is extracted based on the fisrt feature Characteristics of image after conjunction, after being input to the Face image synthesis model trained and completed, obtained specified facial image with Image error convergence between the corresponding first kind facial image.
6. according to the method described in claim 5, it is characterized in that, described according to the first sample identity characteristic and described the Error between two sample identity features adjusts the model parameter that the second feature extracts model, comprising:
Based on having trained the third feature completed to extract model, feature extraction is carried out to the first sample facial image, Obtain the non-identity characteristic of the second sample of the first sample facial image；
The second sample identity feature and the non-identity characteristic of the second sample are combined, the second characteristics of image is obtained, Based on the Face image synthesis model completed has been trained, the corresponding second specified face figure of second characteristics of image is generated Picture；
Obtain the second image error between the first sample facial image and the second specified facial image；
According between second image error and the first sample identity characteristic and the second sample identity feature Error adjusts the model parameter that the second feature extracts model, so that based on having trained the second feature completed to extract The identity characteristic that model extracts the second class facial image extracts model to second class with based on the fisrt feature The error convergence between identity characteristic that the corresponding first kind facial image of facial image extracts, and the second class facial image The corresponding first kind facial image and the specified face based on the Face image synthesis model generation for having trained completion Image error convergence between image.
7. according to the method described in claim 2, it is characterized in that, described extract model based on the second feature, to described Target facial image carries out feature extraction, and after obtaining the target identities feature of the target facial image, the method is also wrapped It includes:
It by the target identities feature and presets non-identity characteristic and is combined, obtain target image characteristics；
Based on the Face image synthesis model completed has been trained, the corresponding specified facial image of the target image characteristics is obtained, The Face image synthesis model is used to generate the first kind face figure corresponding with described image feature according to characteristics of image Picture.
8. according to the method described in claim 5, it is characterized in that, described according to the first image error, described the is adjusted The model parameter of the model parameter of three Feature Selection Models and the Face image synthesis model, comprising:
Based on current discrimination model, differentiation processing is carried out to the first sample facial image, obtains the first differentiation as a result, right The first specified facial image carries out differentiation processing, obtains the second differentiation as a result, the discrimination model is for determining any people Face image is the probability of original facial image；
It has trained the fisrt feature completed to extract model based on described, feature is carried out to the described first specified facial image and is mentioned It takes, obtains the first specified identity characteristic；
Obtain the first identity characteristic error between the described first specified identity characteristic and the first sample identity characteristic；
Differentiate that result and the first default error differentiated between result, the second differentiation result and second are pre- according to described first If differentiating the error and the first identity characteristic error between result, the model that the third feature extracts model is adjusted The model parameter of parameter, the model parameter of the Face image synthesis model and the discrimination model, so that based on having trained At the discrimination model carry out the error convergence of the differentiation result obtained after differentiation processing, and extracted based on the fisrt feature Model extracts model to the first kind with based on the fisrt feature to the identity characteristic that the first kind facial image extracts The error convergence between identity characteristic that the corresponding specified facial image of facial image extracts.
9. according to the method described in claim 8, it is characterized in that, described according to the first sample identity characteristic and described the Error between two sample identity features adjusts the model parameter that the second feature extracts model, comprising:
Based on having trained the third feature completed to extract model, feature extraction is carried out to the first sample facial image, Obtain the non-identity characteristic of the second sample of the first sample facial image；
The second sample identity feature and the non-identity characteristic of the second sample are combined, the second characteristics of image is obtained, Based on the Face image synthesis model completed has been trained, the corresponding second specified face figure of second characteristics of image is generated Picture obtains the second image error between the first sample facial image and the second specified facial image；
Based on the discrimination model completed has been trained, differentiation processing is carried out to the described second specified facial image, obtains third Differentiate result；
Based on having trained the fisrt feature completed to extract model, feature extraction is carried out to the described second specified facial image, The second specified identity characteristic of the described second specified facial image is obtained, the second specified identity characteristic and described first is obtained Second Identity of Local error between sample identity feature；
It is missed according to the error between the first sample identity characteristic and the second sample identity feature, second image Poor, the described third differentiates result and the second default error and the Second Identity of Local error differentiated between result, The model parameter that the second feature extracts model is adjusted, so that based on having trained the second feature completed to extract model pair The identity characteristic that the second class facial image extracts extracts model to the second class face figure with based on the fisrt feature As corresponding first kind facial image extract identity characteristic between error convergence, based on the fisrt feature extract model, It has trained the second feature completed to extract model, third feature extraction model and the Face image synthesis model to obtain Image error convergence between the specified facial image arrived and the corresponding second class facial image, is based on the discrimination model The error convergence of differentiation result obtained after differentiation processing is carried out, and model is extracted to the first kind based on the fisrt feature The identity characteristic that facial image extracts extracts model to the corresponding finger of the second class facial image with based on the fisrt feature Determine the error convergence between the identity characteristic of facial image extraction.
10. a kind of face image processing process, which is characterized in that the described method includes:
It obtaining second feature and extracts model, the second feature is extracted model and is obtained according to fisrt feature extraction model training, and Training objective is to extract the identity characteristic that model extracts the second class facial image based on the second feature, is intended to based on institute It states fisrt feature and extracts the identity characteristic that model extracts the corresponding first kind facial image of the second class facial image, it is described The high resolution of first kind facial image is in the resolution ratio of the second class facial image；
Model is extracted based on the second feature, feature is carried out to the target facial image for belonging to the second class facial image and is mentioned It takes, obtains the target identities feature of the target facial image.
11. according to the method described in claim 10, it is characterized in that, described extract model based on the second feature, to category Feature extraction is carried out in the target facial image of the second class facial image, obtains the target identities of the target facial image After feature, the method also includes:
It by the target identities feature and presets non-identity characteristic and is combined, obtain target image characteristics；
Based on Face image synthesis model, the corresponding specified facial image of the target image characteristics, the specified face are obtained Image belongs to the first kind facial image；
The Face image synthesis model extracts model according to the fisrt feature and third feature is extracted model training and obtained, and Training objective be based on the Face image synthesis model be characteristics of image generate specified facial image, with described image feature Image error convergence between any facial image that is corresponding, belonging to the first kind facial image；Described image feature is Model is extracted to the non-identity characteristic of any facial image extraction based on the third feature and is based on the fisrt feature Characteristics of image after extracting the identity characteristic combination that model extracts any facial image.
12. a kind of face image processing device, which is characterized in that described device includes:
First sample facial image obtains module, for obtaining first sample facial image and corresponding second sample face figure Picture, the first sample facial image belong to first kind facial image, and the second sample facial image belongs to the second class face Image, the high resolution of the first kind facial image is in the resolution ratio of the second class facial image；
Fisrt feature extraction module, for based on trained complete fisrt feature extract model, to the first sample face Image carries out feature extraction, obtains the first sample identity characteristic of the first sample facial image, and the fisrt feature is extracted Model is used to extract the identity characteristic of the first kind facial image；
Second feature extraction module, for extracting model based on current second feature, to the second sample facial image into Row feature extraction, obtains the second sample identity feature of the second sample facial image, and the second feature is extracted model and used In the identity characteristic for extracting the second class facial image；
The first adjustment module, for according to the mistake between the first sample identity characteristic and the second sample identity feature Difference adjusts the model parameter that the second feature extracts model, so that based on having trained the second feature completed to extract mould The identity characteristic that type extracts the second class facial image extracts model to the second class people with based on the fisrt feature The error convergence between identity characteristic that the corresponding first kind facial image of face image extracts.
13. a kind of face image processing device, which is characterized in that described device includes:
Second feature obtains module, extracts model for obtaining second feature, the second feature extracts model according to the first spy Sign is extracted model training and is obtained, and training objective is to extract what model extracted the second class facial image based on the second feature Identity characteristic is intended to extract model to the corresponding first kind face figure of the second class facial image based on the fisrt feature As the identity characteristic extracted, the high resolution of the first kind facial image is in the resolution ratio of the second class facial image；
Target's feature-extraction module, for extracting model based on the second feature, to belonging to the second class facial image Target facial image carries out feature extraction, obtains the target identities feature of the target facial image.
14. a kind of face image processing device, which is characterized in that described device includes processor and memory, the memory In be stored at least one instruction, at least a Duan Chengxu, code set or instruction set, described instruction, described program, the code set Or described instruction collection is loaded as the processor and is executed to realize the face as described in claim 1 to 9 any claim Performed operation in image processing method；Or face figure of the realization as described in any one of claim 10 to 12 claim As operation performed in processing method.
15. a kind of computer readable storage medium, which is characterized in that be stored at least one in the computer readable storage medium Item instruction, at least a Duan Chengxu, code set or instruction set, described instruction, described program, the code set or described instruction collection by Processor is loaded and is executed to realize in the face image processing process as described in claim 1 to 9 any claim and hold Capable operation；Or realize as claim 10 to 12 any one claim as described in face image processing process in performed by Operation.
CN201910335080.9A 2019-04-24 2019-04-24 Face image processing method, device and storage medium Active CN110059652B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910335080.9A CN110059652B (en) 2019-04-24 2019-04-24 Face image processing method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910335080.9A CN110059652B (en) 2019-04-24 2019-04-24 Face image processing method, device and storage medium

Publications (2)

Publication Number Publication Date
CN110059652A true CN110059652A (en) 2019-07-26
CN110059652B CN110059652B (en) 2023-07-25

Family

ID=67320530

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910335080.9A Active CN110059652B (en) 2019-04-24 2019-04-24 Face image processing method, device and storage medium

Country Status (1)

Country Link
CN (1) CN110059652B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111160095A (en) * 2019-11-26 2020-05-15 华东师范大学 Unbiased face feature extraction and classification method and system based on depth self-encoder network
CN111209878A (en) * 2020-01-10 2020-05-29 公安部户政管理研究中心 Cross-age face recognition method and device
CN111582200A (en) * 2020-05-12 2020-08-25 北京邮电大学 Human body posture estimation method and device, electronic equipment and medium
CN111651623A (en) * 2020-06-11 2020-09-11 腾讯科技（深圳）有限公司 Method, device and equipment for constructing high-precision facial expression library and storage medium
CN111928947A (en) * 2020-07-22 2020-11-13 广州朗国电子科技有限公司 Forehead temperature measuring method and device based on low-precision face thermometer and thermometer
WO2021147199A1 (en) * 2020-01-21 2021-07-29 北京市商汤科技开发有限公司 Network training method and apparatus, and image processing method and apparatus
CN113409207A (en) * 2021-06-15 2021-09-17 广州光锥元信息科技有限公司 Method and device for improving definition of face image
CN114648531A (en) * 2022-05-20 2022-06-21 领伟创新智能系统(浙江)有限公司 Solar panel surface dust identification method based on color channel brightness distribution

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105164696A (en) * 2013-05-03 2015-12-16 诺基亚技术有限公司 A method and technical equipment for people identification
CN107633218A (en) * 2017-09-08 2018-01-26 百度在线网络技术（北京）有限公司 Method and apparatus for generating image
WO2018054283A1 (en) * 2016-09-23 2018-03-29 北京眼神科技有限公司 Face model training method and device, and face authentication method and device
US20180330511A1 (en) * 2017-05-11 2018-11-15 Kla-Tencor Corporation Learning based approach for aligning images acquired with different modalities
CN108921782A (en) * 2018-05-17 2018-11-30 腾讯科技（深圳）有限公司 A kind of image processing method, device and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105164696A (en) * 2013-05-03 2015-12-16 诺基亚技术有限公司 A method and technical equipment for people identification
WO2018054283A1 (en) * 2016-09-23 2018-03-29 北京眼神科技有限公司 Face model training method and device, and face authentication method and device
US20180330511A1 (en) * 2017-05-11 2018-11-15 Kla-Tencor Corporation Learning based approach for aligning images acquired with different modalities
CN107633218A (en) * 2017-09-08 2018-01-26 百度在线网络技术（北京）有限公司 Method and apparatus for generating image
CN108921782A (en) * 2018-05-17 2018-11-30 腾讯科技（深圳）有限公司 A kind of image processing method, device and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
马祥;齐春;: "全局重建和位置块残差补偿的人脸图像超分辨率算法", 西安交通大学学报 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111160095A (en) * 2019-11-26 2020-05-15 华东师范大学 Unbiased face feature extraction and classification method and system based on depth self-encoder network
CN111160095B (en) * 2019-11-26 2023-04-25 华东师范大学 Unbiased face feature extraction and classification method and system based on depth self-encoder network
CN111209878A (en) * 2020-01-10 2020-05-29 公安部户政管理研究中心 Cross-age face recognition method and device
JP2022521372A (en) * 2020-01-21 2022-04-07 ベイジン センスタイム テクノロジー ディベロップメント カンパニー リミテッド Network training methods and equipment, image processing methods and equipment
WO2021147199A1 (en) * 2020-01-21 2021-07-29 北京市商汤科技开发有限公司 Network training method and apparatus, and image processing method and apparatus
CN111582200A (en) * 2020-05-12 2020-08-25 北京邮电大学 Human body posture estimation method and device, electronic equipment and medium
CN111582200B (en) * 2020-05-12 2023-11-21 北京邮电大学 Human body posture estimation method, device, electronic equipment and medium
CN111651623A (en) * 2020-06-11 2020-09-11 腾讯科技（深圳）有限公司 Method, device and equipment for constructing high-precision facial expression library and storage medium
CN111651623B (en) * 2020-06-11 2023-08-22 腾讯科技（深圳）有限公司 Method, device, equipment and storage medium for constructing high-precision facial expression library
CN111928947A (en) * 2020-07-22 2020-11-13 广州朗国电子科技有限公司 Forehead temperature measuring method and device based on low-precision face thermometer and thermometer
CN113409207A (en) * 2021-06-15 2021-09-17 广州光锥元信息科技有限公司 Method and device for improving definition of face image
CN113409207B (en) * 2021-06-15 2023-12-08 广州光锥元信息科技有限公司 Face image definition improving method and device
CN114648531A (en) * 2022-05-20 2022-06-21 领伟创新智能系统(浙江)有限公司 Solar panel surface dust identification method based on color channel brightness distribution

Also Published As

Publication number Publication date
CN110059652B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN110059652A (en) Face image processing process, device and storage medium
CN111079576B (en) Living body detection method, living body detection device, living body detection equipment and storage medium
CN110059661A (en) Action identification method, man-machine interaction method, device and storage medium
CN110136136A (en) Scene Segmentation, device, computer equipment and storage medium
CN109034102A (en) Human face in-vivo detection method, device, equipment and storage medium
CN110121118A (en) Video clip localization method, device, computer equipment and storage medium
CN110083791B (en) Target group detection method and device, computer equipment and storage medium
CN109299315A (en) Multimedia resource classification method, device, computer equipment and storage medium
CN110210571A (en) Image-recognizing method, device, computer equipment and computer readable storage medium
CN110110787A (en) Location acquiring method, device, computer equipment and the storage medium of target
CN108594997A (en) Gesture framework construction method, apparatus, equipment and storage medium
CN109360210A (en) Image partition method, device, computer equipment and storage medium
CN110110145A (en) Document creation method and device are described
CN110059685A (en) Word area detection method, apparatus and storage medium
CN110222789A (en) Image-recognizing method and storage medium
CN110222551A (en) Method, apparatus, electronic equipment and the storage medium of identification maneuver classification
CN109978936A (en) Parallax picture capturing method, device, storage medium and equipment
CN109815150A (en) Application testing method, device, electronic equipment and storage medium
CN110288518A (en) Image processing method, device, terminal and storage medium
CN109285178A (en) Image partition method, device and storage medium
CN110135336A (en) Training method, device and the storage medium of pedestrian's generation model
CN109522863A (en) Ear's critical point detection method, apparatus and storage medium
CN110166786A (en) Virtual objects transfer method and device
CN110163160A (en) Face identification method, device, equipment and storage medium
CN109840584A (en) Convolutional neural networks model, data processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant