CN109978754A - Image processing method, device, storage medium and electronic equipment - Google Patents
Image processing method, device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN109978754A CN109978754A CN201711466358.3A CN201711466358A CN109978754A CN 109978754 A CN109978754 A CN 109978754A CN 201711466358 A CN201711466358 A CN 201711466358A CN 109978754 A CN109978754 A CN 109978754A
- Authority
- CN
- China
- Prior art keywords
- image
- face
- original image
- facial image
- target facial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 230000001815 facial effect Effects 0.000 claims abstract description 135
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 45
- 230000006870 function Effects 0.000 claims description 53
- 238000005286 illumination Methods 0.000 claims description 27
- 238000012549 training Methods 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 25
- 230000009466 transformation Effects 0.000 claims description 24
- 238000000034 method Methods 0.000 claims description 21
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 19
- 238000012937 correction Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 9
- 230000004927 fusion Effects 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 230000001537 neural effect Effects 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 9
- 230000000694 effects Effects 0.000 abstract description 7
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 10
- 239000011159 matrix material Substances 0.000 description 9
- 230000036544 posture Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 4
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 239000003607 modifier Substances 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention discloses a kind of image processing method, device, storage medium and electronic equipments.The image processing method, by the first location information for obtaining face key point in original image；Corresponding target facial image is matched from default face database according to first location information, and target facial image is aligned with the face in original image；Based on the human face region in trained convolutional neural networks model and original image, target facial image is modified；Revised target facial image is merged with original image.The program can preferably keep certain feature invariants of original image by depth learning technology, while can desalinate image mosaic trace, promote image synthetic effect.
Description
Technical field
The present invention relates to technical field of image processing more particularly to a kind of image processing method, device, storage medium and electricity
Sub- equipment.
Background technique
Existing electronic equipment generally have take pictures, camera function.With intelligent electronic device and computer vision technique
High speed development, user for the demand of the camera of intelligent electronic device be not solely restricted to it is traditional take pictures, photograph, and more
Tend to image processing function, as the technologies such as the U.S. face of intelligence, Style Transfer are popularized by more and more intelligent electronic devices more.
Summary of the invention
The embodiment of the present invention provides a kind of image processing method, device, storage medium and electronic equipment, can desalinate image
Splice trace, promotes image synthetic effect.
In a first aspect, the embodiment of the present invention provides a kind of image processing method, it is applied to electronic equipment, comprising:
Obtain the first location information of face key point in original image；
Match corresponding target facial image from default face database according to first location information, and by target face
Image is aligned with the face in original image；
Based on the human face region in trained convolutional neural networks model and original image, target facial image is carried out
Amendment；
Revised target facial image is merged with original image.
Second aspect, the embodiment of the invention provides a kind of image processing apparatus, are applied to electronic equipment, comprising:
Position acquisition module, for obtaining the first location information of face key point in original image；
Alignment module, for matching corresponding target face figure from default face database according to first location information
Picture, and target facial image is aligned with the face in original image；
Correction module, for based on the human face region in trained convolutional neural networks model and original image, to mesh
Mark facial image is modified；
Fusion Module, for merging revised target facial image with original image.
The third aspect is stored with a plurality of finger the embodiment of the invention also provides a kind of storage medium in the storage medium
It enables, described instruction is suitable for being loaded by processor to execute above-mentioned image processing method.
Fourth aspect, the embodiment of the invention also provides a kind of electronic equipment, including processor, memory, the processing
Device and the memory are electrically connected, and the memory is for storing instruction and data；Processor is for executing above-mentioned image
Processing method.
The embodiment of the invention discloses a kind of image processing method, device, storage medium and electronic equipments.The image procossing
Method, by the first location information for obtaining face key point in original image；According to first location information from default face number
According to matching corresponding target facial image in library, and target facial image is aligned with the face in original image；Based on training
Human face region in good convolutional neural networks model and original image, is modified target facial image；It will be revised
Target facial image is merged with original image.The program can preferably keep certain features of original image by depth learning technology not
Become, while image mosaic trace can be desalinated, promotes image synthetic effect.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those skilled in the art, without creative efforts, it can also be obtained according to these attached drawings other attached
Figure.
Fig. 1 is the scene framework schematic diagram that electronic equipment provided in an embodiment of the present invention realizes deep learning.
Fig. 2 is a kind of flow diagram of image processing method provided in an embodiment of the present invention.
Fig. 3 is a kind of application scenario diagram of image processing method provided in an embodiment of the present invention.
Fig. 4 is the partial structural diagram of convolutional neural networks provided in an embodiment of the present invention.
Fig. 5 is another application scenario diagram of image processing method provided in an embodiment of the present invention.
Fig. 6 is another flow diagram of image processing apparatus provided in an embodiment of the present invention.
Fig. 7 is a kind of structural schematic diagram of image processing apparatus provided in an embodiment of the present invention.
Fig. 8 is another structural schematic diagram of image processing apparatus provided in an embodiment of the present invention.
Fig. 9 is another structural schematic diagram of image processing apparatus provided in an embodiment of the present invention.
Figure 10 is the yet another construction schematic diagram of image processing apparatus provided in an embodiment of the present invention.
Figure 11 is a kind of structural schematic diagram of electronic equipment provided in an embodiment of the present invention.
Figure 12 is another structural schematic diagram of electronic equipment provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those skilled in the art's every other implementation obtained without creative efforts
Example, shall fall within the protection scope of the present invention.
The embodiment of the present invention provides a kind of image processing method, device, storage medium and electronic equipment.Below will respectively into
Row is described in detail.
Referring to Fig. 1, Fig. 1 is the schematic diagram of a scenario that electronic equipment provided in an embodiment of the present invention realizes deep learning.
When user is handled image by the image processing function in electronic equipment, the recordable processing of electronic equipment
Inputoutput data in the process.Wherein, may include statistics of data acquisition system in electronic equipment with adjusted with feedback it is pre-
Examining system.Electronic equipment can obtain a large amount of image classification result data of user by data collection system, make corresponding system
Meter, and the characteristics of image of image is extracted, the characteristics of image extracted is analyzed and processed based on machine deep learning, to volume
The loss function of product neural network carries out parameter training.In input picture, electronic equipment passes through predictive system image
Classification results.After user makes final choice behavior, final result of the forecasting system according to user behavior, reversed feedback
Adjust the weight of each weight term.After multiple iteration corrigendum, so that the weight of each weight term of the forecasting system
Final convergence forms the database that study obtains.
Electronic equipment can be mobile terminal, such as mobile phone, tablet computer, or traditional PC (Personal
Computer, PC) etc., the embodiment of the present invention is to this without limiting.
In one embodiment, a kind of image processing method is provided, as shown in Fig. 2, process can be such that
101, the first location information of face key point in original image is obtained.
It include at least one face in the original image in the embodiment of the present application.Wherein, which specifically can be with
It is electronic equipment by camera acquired image, is also possible to the storage of electronic equipment from server or other external equipments
Image is directly acquired to obtain in area.
If electronic equipment can be through preposition equipment head certainly by carrying camera collection image, acquired image
Clap obtained image, be also possible to by rear camera the collected image with face.In some embodiments, should
Camera can be digital camera, can also be simulation camera.The mould that digital camera can generate image capture device
Quasi- picture signal is converted into digital signal, and then is stored in computer.The picture signal that simulation camera captures must
Figure pattern must be converted analog signals by specific picture catching card, and just may switch to computer after being compressed
Upper utilization.Digital camera can directly capture image, then be passed in computer by string, parallel port or USB interface.This Shen
Please be in embodiment, electronic equipment generally uses digital camera, is set so that picture collected is converted into data in real time in electronics
Real-time display on standby display interface (i.e. the preview pane of camera).
In the embodiment of the present application, needs to carry out Face datection to original image first, human face region is determined, then again from people
The detection of face key point is carried out in face region, to obtain the location information of face key point.That is, in some embodiments,
Step " first location information for obtaining face key point in original image " may include following below scheme:
Extract the characteristics of image of original image；
The human face region in original image is determined according to described image feature；
Face critical point detection is carried out to human face region, to obtain the first location information of face key point.
Specifically, the detection of face key point can be carried out using Dlib, Dlib is the library C++ an of machine learning, packet
The common algorithm of many machine learning is contained.With reference to Fig. 3, Fig. 3 is the calibration result schematic diagram of detected face key point.
Wherein, chin is demarcated by 17 face key points, and left and right eyebrow is respectively demarcated by 5 face key points, and nose is by 9
Face key point is demarcated, and right and left eyes 6 face key points of each freedom are demarcated, and mouth is demarcated by 20 face key points
It forms, it is total to have 68 face key points.
102, corresponding target facial image is matched from default face database according to first location information, and by target
Facial image is aligned with the face in original image.
Specifically, since original image may be influenced when shooting by shooting angle, leading to image, deformation occurs, sternly
The identification of image is influenced again.Therefore, certain measure need to be taken to convert image, it is carried out it is a degree of correction with
Facilitate identification and the registration of machine.Therefore, in the embodiment of the present application, need to construct a face database in advance, storage is used
In the image for the reference face difference posture for carrying out identity replacement, it can specifically make the photo for obtaining different angle, so as to original
Image can be matched, operation of changing face using the reference facial image in face database as target identities.
In the embodiment of the present application, mode that target facial image is aligned with the face in original image can there are many,
For example, target facial image can be aligned with the face in original image by the way of affine transformation.That is, in some realities
It applies in example, step " matches corresponding target facial image according to first location information, and by target from default face database
Facial image is aligned with the face in original image " may include following below scheme:
Obtain the second location information of the face key point of multiple sample facial images in default face database；
First location information is matched with second location information；
The maximum sample facial image of matching degree is chosen from sample facial image, as target facial image；
Target facial image is mapped to the human face region of original image by affine transformation so that target facial image with
Face alignment in original image.
Specifically, by the second location information progress of sample facial image in first location information and face database
Match, matching degree is bigger, then it represents that posture, the size of two images are closer, become so as to reduce in subsequent affine conversion process
Change the difficulty in computation of matrix.In order to promote the matching degree of facial image Yu sample facial image, available a large amount of different postures
Facial image, to increase the density of sample facial image deflection angle in face database, between reducing between deflection angle
Every value.
Wherein, affine transformation, also known as affine maps, refer in geometry, and a vector space carries out once linear transformation
And a translation is connected, it is transformed to another vector space.Affine transformation has: translation, rotation, scaling, beveling etc..It is imitated
Penetrate transformation, it is necessary to first obtain transformation matrix.Obtain transformation matrix, it is necessary to the information such as characteristic point coordinate, angle are first obtained, it is such as several
The methods of what matching, bolb can obtain characteristic point coordinate, angle information.
In the specific implementation, first have to obtain the coordinate of key point, angle information (the i.e. second position in target facial image
Information), affine transformation matrix is then calculated according to the second location information of acquisition and first location information.According to being calculated
Transformation matrix to target facial image carry out affine transformation, target facial image is mapped to the face location of original image.
103, based on the human face region in trained convolutional neural networks model and original image, to target facial image
It is modified.
In some embodiments, in obtaining original image before the first location information of face key point, this method is also
It may comprise steps of:
Construct convolutional neural networks；
The image of multiple angles of face in original image is obtained as training sample；
Parameter training is carried out to constructed convolutional neural networks based on training sample, with Suitable content loss function, light
Convolutional neural networks model according to the parameter setting of loss function and smooth loss function, after being trained.
With reference to Fig. 4, in the embodiment of the present application, constructed convolutional neural networks are the multiple dimensioned frame for having branch
Structure, these branches execute operation in different sampled versions according to the difference of the size of institute's input test image.Small size
Up-sampling is 2 times of sizes to image automatically after convolution, then carries out channel with large-sized image again and connects.Each is in this way
Branch have the convolution module of zero padding, also and then linear amendment (linear rectification) thereafter.These branches
Arest neighbors up-sampling (nearest-neighbor upsampling) by one times of difference and the cascade along channel axis again
(concatenation along the channel axis) combines.
In specific training process, training sample is inputted first, parameter initialization is executed, after convolution sum sampling process
Full articulamentum is reached, and carries out affine transformation and parameter and calculates, treated that image is obtained by logistic regression analysis for output
The weight of each loss function, by judging whether to meet expectation artificially come the parameter setting of each loss function of continuous feedback modifiers.
In some embodiments, " right based on the human face region in trained convolutional neural networks model and original image
Target facial image is modified " it may comprise steps of:
Content characteristic, illumination spy are extracted from the human face region of original image based on trained convolutional neural networks model
Sign and smooth features；
According to content characteristic, illumination feature and smooth features, in content loss function, illumination loss function and smooth damage
Under the constraint for losing function, adjusting parameter is generated；
Target facial image is modified according to adjusting parameter.
In the embodiment of the present application, face can be exchanged the problem of being described as a kind of Style Transfer and realized.And style is moved
The target of shifting is the style by an image rendering at another image.Based on this, by the posture of face in original image and
Expression designs a kind of convolutional neural networks generation hi-vision validity that can allow as style as content, target facial image
As a result loss function.The loss function of image is based on the characteristic pattern in one trained neural network.
Aiming at the problem that style loss function, arest neighbors method, i.e. the image mesh of some position in original image can be used
Most like segment is replaced in marking on a map.Region of search is limited according to the face key point extracted in face.I.e. pair
Some part of face in original image only carries out similar area searching near some part in mesh image.
In some embodiments, it is desirable to multiple images of target face, i.e. multiple style images.In similar area searching
When, loss is limited on image-region, but can be scanned on the section of multiple image zooming-outs, in this way, can be with
Guarantee to reappear diversified expression.
In the present embodiment, in order to keep illumination during changing face to remain unchanged, need to punish the transformation in illumination.
And in order to extract illumination variation, algorithm has trained a convolutional neural networks classifier for illumination.For two in addition to illumination
Other outer all constant images, classifier judge whether this pair of of image has occurred light change, and use is from this network
Obtained characteristic pattern carries out the calculating of illumination loss.
104, revised target facial image is merged with original image.
In the embodiment of the present application, mode that revised target facial image is merged with original image can there are many.
One kind is the algorithm based on region, refers to using the relationship of gray scale between two images the parameter of changes in coordinates between determining image,
Including pixel matching algorithm space-based and based on the algorithm of frequency domain.Another kind of is the algorithm based on merging features, is
The transformation between image is calculated using the obvious characteristic (point, line, edge, profile, angle point) in image.Third class is based on most
The splicing of big mutual information, by splicing operation from transform of spatial domain to small domain wave, carrying out wavelet reconstruction can be obtained complete image.
It in some embodiments, will be revised after being aligned target facial image with the face in original image
Before target facial image is merged with original image, can with the following steps are included:
Edge Feature Points detection is carried out to the human face region in original image, and obtains the third place letter of Edge Feature Points
Breath；
Image dividing processing is carried out to original image according to the third place information, human face region is removed, by residual graph
As region is as background image.
Specifically, pattern is cut along the position of detected Edge Feature Points, since Edge Feature Points are face side
Edge characteristic point therefore may finally be separated from original image by human face region, and background image finally can be obtained.
Then step " merging revised target facial image with original image " may include following below scheme:
Face exposure mask is generated according to the third place information；
Using the face exposure mask, revised target facial image is merged with the background image in original image.
Wherein, the relative position information that the location information of Edge Feature Points can be mutual for these Edge Feature Points.
A closed pattern is constituted based on these Edge Feature Points, using the region other than the closed pattern as face exposure mask
(segmentation mask).By the aid of face exposure mask in mapping on the original image the target facial image of human face region,
And be aligned with the human face region of original image, it will not shown by the region that the face exposure mask blocks in target facial image,
And the region being blocked is not shown then.For example, with reference to Fig. 5, wherein a is target facial image, and b is original image, c be based on
The face exposure mask that human face region in original image b generates, d are the image that target image a is obtained after affine transformation, final defeated
Blending image e after changing face out.
When by the target facial image that replaces with that treated of facial image in target picture, it can lead to based on graph cut skill
Art, by treated, target facial image is merged with target picture, coverage goal picture Central Plains somebody's face image, to realize
By the target facial image that replaces with that treated of facial image in target picture.Wherein, graph cut technology can preferably disappear
Except the boundary of target facial image and target picture, so that picture is more natural and not lofty, realize seamless spliced.
From the foregoing, it will be observed that passing through the first location information for obtaining face key point in original image；According to first location information
Match corresponding target facial image from default face database, and by the face pair in target facial image and original image
Together；Based on the human face region in trained convolutional neural networks model and original image, target facial image is modified；
Revised target facial image is merged with original image.The program can preferably keep original image by depth learning technology
Certain feature invariants, while image mosaic trace can be desalinated, promote image synthetic effect.
In one embodiment, another image processing method is also provided, as shown in fig. 6, process can be such that
201, convolutional neural networks are constructed.
In the embodiment of the present application, in the embodiment of the present application, constructed convolutional neural networks are one with branch
Multiple dimensioned framework, these branches execute fortune in different sampled versions according to the difference of the size of institute's input test image
It calculates.Up-sampling is 2 times of sizes to the image of small size automatically after convolution, then carries out channel company with large-sized image again
It connects.There is the convolution module of zero padding in each such branch, thereafter also and then linear amendment.These branches pass through difference again
One times of arest neighbors up-samples and gets up along the cascading of channel axis.
202, parameter training is carried out to constructed convolutional neural networks based on training sample, letter is lost with Suitable content
The parameter setting of number, illumination loss function and smooth loss function, the convolutional neural networks model after being trained.
It in the embodiment of the present application, need to be to volume in order to guarantee that certain features after subsequent change face in original image will not lose
Loss function in product neural network carries out parameter training.
Specifically, face can be exchanged the problem of being described as a kind of Style Transfer and realized.And the target of Style Transfer is
By an image rendering at the style of another image.Based on this, using the posture of face in original image and expression as in
Hold, target facial image designs a kind of loss that the convolutional neural networks can be allowed to generate hi-vision validity result as style
Function.
In some embodiments, the image of multiple angles of available same face is as training sample.Specifically instructing
During white silk, input training sample, execution parameter initialization reach full articulamentum after convolution sum sampling process first, and
It carries out affine transformation and parameter to calculate, output treated image obtains the power of each loss function by logistic regression analysis
Weight, by judging whether to meet expectation artificially come the parameter setting of each loss function of continuous feedback modifiers.
203, face critical point detection obtains the first location information and target person of face key point in original image
The second location information of the face key point of face image.
It include at least one face in the original image in the embodiment of the present application.Wherein, which specifically can be with
It is electronic equipment by camera acquired image, is also possible to the storage of electronic equipment from server or other external equipments
Image is directly acquired to obtain in area.And target facial image is then the reference man for carrying out identity replacement to face in original image
Face.
Therefore, in the embodiment of the present application, need to construct a face database in advance, store for carrying out identity replacement
Reference face difference posture image, can specifically make obtain different angle photo, so that original image can be with face number
According to the reference facial image in library as target identities, matched, operation of changing face.
When it is implemented, needing to carry out Face datection to original image first, human face region is determined, then again from face area
The detection of face key point is carried out in domain, to obtain the location information of face key point.
204, affine transformation matrix is calculated according to first location information and second location information, it will based on affine transformation matrix
Target facial image is mapped to the human face region of original image.
Since original image may be influenced when shooting by shooting angle, leading to image, deformation occurs, seriously affects
To the identification of image.Therefore, certain measure need to be taken to convert image, carries out a degree of correction to it to facilitate machine
The identification of device and registration.
Specifically, first location information is matched with two location informations, and sets up original image and target face
The one-to-one matching relationship of face key point in image.Then by face key point to the affine transformation for finding out two sub-pictures
Matrix, and target facial image is mapped to based on affine transformation matrix the human face region of original image, so that by target face
Image is aligned with the face in original image.
205, based on the human face region in trained convolutional neural networks model and original image, to target facial image
It is modified.
Specifically, based on trained convolutional neural networks model from the human face region of original image extract content characteristic,
Illumination feature and smooth features；According to content characteristic, illumination feature and smooth features, lost in content loss function, illumination
Under the constraint of function and smooth loss function, corresponding adjusting parameter is generated, target facial image is carried out according to adjusting parameter
Amendment.Joined by trained convolutional neural networks model tune, preferably keeps expression, the colour of skin, light of face in original image
According to equal feature invariants, so that the face after changing face is more natural.
206, Edge Feature Points detection is carried out to the human face region in original image, and obtains the third position of Edge Feature Points
Confidence breath.
Likewise, needing to carry out original image Face datection first, then it is based on relevant edge algorithm, calculating gets face
The third place information of edges of regions characteristic point.
207, image dividing processing is carried out to original image according to the third place information, human face region is removed, will be remained
Remaining image-region is as background image.
Specifically, pattern is cut along the position of detected Edge Feature Points, since Edge Feature Points are face side
Edge characteristic point therefore may finally be separated from original image by human face region, and background image finally can be obtained.
208, face exposure mask is generated according to the third place information, and utilizes the face exposure mask, by revised target face
Image is merged with the background image in original image.
Wherein, the relative position information that the location information of Edge Feature Points can be mutual for these Edge Feature Points.
A closed pattern is constituted based on these Edge Feature Points, using the region other than the closed pattern as face exposure mask.Face is covered
Film aid is aligned in mapping on the original image the target facial image of human face region with the human face region of original image,
It will not not shown then by the region that the region that the face exposure mask blocks shows, and is blocked in target facial image.
When by the target facial image that replaces with that treated of facial image in target picture, it can lead to based on graph cut skill
Art, by treated, target facial image is merged with target picture, coverage goal picture Central Plains somebody's face image, to realize
By the target facial image that replaces with that treated of facial image in target picture.Wherein, graph cut technology can preferably disappear
Except the boundary of target facial image and target picture, so that picture is more natural and not lofty, realize seamless spliced.
From the foregoing, it will be observed that image processing method provided in an embodiment of the present invention, by obtaining face key point in original image
First location information；Corresponding target facial image is matched from default face database according to first location information, and will
Target facial image is aligned with the face in original image；Based in trained convolutional neural networks model and original image
Human face region is modified target facial image；Revised target facial image is merged with original image.The program can
The expression, the colour of skin, illumination invariant of original image can be preferably kept by depth learning technology, while can desalinate image mosaic trace
Mark promotes image synthetic effect.
In still another embodiment of the process, a kind of image processing apparatus is also provided, the image processing apparatus can with software or
The form of hardware is integrated in the electronic device, which can specifically include mobile phone, tablet computer, laptop etc. and set
It is standby.As shown in fig. 7, the image processing apparatus 30 may include position acquisition module 31, alignment module 32, correction module 33, with
And Fusion Module 34, in which:
Position acquisition module 31, for obtaining the first location information of face key point in original image；
Alignment module 32, for matching corresponding target face figure from default face database according to first location information
Picture, and target facial image is aligned with the face in original image；
Correction module 33 is right for based on the human face region in trained convolutional neural networks model and original image
Target facial image is modified；
Fusion Module 34, for merging revised target facial image with original image.
In some embodiments, with reference to Fig. 8, which can also include:
Module 35 is constructed, before the first location information of face key point in obtaining original image, constructs convolution
Neural network；
Sample acquisition module 36, for obtaining the image of multiple angles of face in original image as training sample；
Training module 37, for carrying out parameter training to constructed convolutional neural networks based on training sample, with adjustment
The parameter setting of content loss function, illumination loss function and smooth loss function, the convolutional neural networks mould after being trained
Type.
In some embodiments, with reference to Fig. 9, correction module 33 may include:
Extracting sub-module 331, for being mentioned based on trained convolutional neural networks model from the human face region of original image
Take content characteristic, illumination feature and smooth features；
Submodule 332 is generated, is used for according to content characteristic, illumination feature and smooth features, in content loss function, light
According under the constraint of loss function and smooth loss function, adjusting parameter is generated；
Submodule 333 is corrected, for being modified according to adjusting parameter to target facial image.
In some embodiments, with reference to Figure 10, correction module 32 may include:
Acquisition submodule 321, for obtaining the face key point of multiple sample facial images in default face database
Second location information；
Matched sub-block 322, for matching first location information with second location information；
Submodule 323 is chosen, for choosing the maximum sample facial image of matching degree from sample facial image, as mesh
Mark facial image；
Mapping submodule 324, for target facial image to be mapped to the human face region of original image by affine transformation,
So that target facial image is aligned with the face in original image.
From the foregoing, it will be observed that image processing apparatus provided in an embodiment of the present invention, by obtaining face key point in original image
First location information；Corresponding target facial image is matched from default face database according to first location information, and will
Target facial image is aligned with the face in original image；Based in trained convolutional neural networks model and original image
Human face region is modified target facial image；Revised target facial image is merged with original image.The program can
Certain feature invariants of original image are preferably kept by depth learning technology, while can desalinate image mosaic trace, promote image
Synthetic effect.
A kind of electronic equipment is also provided in still another embodiment of the process, which can be smart phone, plate
Apparatus such as computer.As shown in figure 11, electronic equipment 400 includes processor 401, memory 402.Wherein, processor 401 and storage
Device 402 is electrically connected.
Processor 401 is the control centre of electronic equipment 400, utilizes various interfaces and the entire electronic equipment of connection
Various pieces by the application program of operation or load store in memory 402, and are called and are stored in memory 402
Data, execute electronic equipment various functions and processing data, thus to electronic equipment carry out integral monitoring.
In the present embodiment, processor 401 in electronic equipment 400 can according to following step, by one or one with
On the corresponding instruction of process of application program be loaded into memory 402, and be stored in memory by processor 401 to run
Application program in 402, to realize various functions:
Obtain the first location information of face key point in original image；
Match corresponding target facial image from default face database according to first location information, and by target face
Image is aligned with the face in original image；
Based on the human face region in trained convolutional neural networks model and original image, target facial image is carried out
Amendment；
Revised target facial image is merged with original image.
In some embodiments, in obtaining original image before the first location information of face key point, processor 401
For executing following steps:
Construct convolutional neural networks；
The image of multiple angles of face in original image is obtained as training sample；
Parameter training is carried out to constructed convolutional neural networks based on training sample, with Suitable content loss function, light
Convolutional neural networks model according to the parameter setting of loss function and smooth loss function, after being trained.
In some embodiments, processor 401 is further used for executing following steps:
Content characteristic, illumination spy are extracted from the human face region of original image based on trained convolutional neural networks model
Sign and smooth features；
According to content characteristic, illumination feature and smooth features, in content loss function, illumination loss function and smooth damage
Under the constraint for losing function, adjusting parameter is generated；
Target facial image is modified according to adjusting parameter.
In some embodiments, processor 401 is further used for executing following steps:
Obtain the second location information of the face key point of multiple sample facial images in default face database；
First location information is matched with second location information；
The maximum sample facial image of matching degree is chosen from sample facial image, as target facial image；
Target facial image is mapped to the human face region of original image by affine transformation so that target facial image with
Face alignment in original image.
It in some embodiments, will be revised after being aligned target facial image with the face in original image
Before target facial image is merged with original image, processor 401 also executes following steps:
Edge Feature Points detection is carried out to the human face region in original image, and obtains the third place letter of Edge Feature Points
Breath；
Image dividing processing is carried out to original image according to the third place information, human face region is removed, by residual graph
As region is as background image.
In some embodiments, processor 401 is further used for executing following steps:
Face exposure mask is generated according to the third place information；
Using face exposure mask, revised target facial image is merged with the background image in original image.
Memory 402 can be used for storing application program and data.Including in the application program that memory 402 stores can be
The instruction executed in processor.Application program can form various functional modules.Processor 401 is stored in memory by operation
402 application program, thereby executing various function application and data processing.
In some embodiments, as shown in figure 12, electronic equipment 400 further include: display screen 403, is penetrated at control circuit 404
Frequency circuit 405, input unit 406, voicefrequency circuit 407, sensor 408 and power supply 409.Wherein, processor 401 respectively with it is aobvious
Display screen 403, control circuit 404, radio circuit 405, input unit 406, voicefrequency circuit 407, sensor 408 and power supply 409
It is electrically connected.
Display screen 403 can be used for showing information input by user or be supplied to user information and electronic equipment it is each
Kind graphical user interface, these graphical user interface can be made of image, text, icon, video and any combination thereof.Its
In, which can be used as the screen in the embodiment of the present invention, for showing information.
Control circuit 404 and display screen 403 are electrically connected, and show information for controlling display screen 403.
Radio circuit 405 is used for transceiving radio frequency signal, to build by wireless communication with the network equipment or other electronic equipments
Vertical wireless telecommunications, the receiving and transmitting signal between the network equipment or other electronic equipments.
Input unit 406 can be used for receiving number, character information or the user's characteristic information (such as fingerprint) of input, and
Generate keyboard related with user setting and function control, mouse, operating stick, optics or trackball signal input.Wherein,
Input unit 406 may include fingerprint recognition mould group.
Voicefrequency circuit 407 can provide the audio interface between user and electronic equipment by loudspeaker, microphone.
Sensor 408 is for acquiring external environmental information.Sensor 408 may include ambient light sensor, acceleration
Sensor, optical sensor, motion sensor and other sensors.
All parts of the power supply 409 for electron equipment 400 are powered.In some embodiments, power supply 409 can pass through
Power-supply management system and processor 401 are logically contiguous, to realize management charging, electric discharge, Yi Jigong by power-supply management system
The functions such as consumption management.
Camera 410 can make digital camera for acquiring extraneous picture, or simulation camera.Some
In embodiment, collected extraneous picture can be converted into data and be sent to processor 401 to execute image procossing by camera 410
Operation.
Although being not shown in Figure 12, electronic equipment 400 can also be including bluetooth module etc., and details are not described herein.
From the foregoing, it will be observed that electronic equipment provided in an embodiment of the present invention, by obtain face key point in original image the
One location information；Match corresponding target facial image from default face database according to first location information, and by target
Facial image is aligned with the face in original image；Based on the face in trained convolutional neural networks model and original image
Region is modified target facial image；Revised target facial image is merged with original image.The program can pass through
Depth learning technology preferably keeps certain feature invariants of original image, while can desalinate image mosaic trace, promotes image synthesis
Effect.
A kind of storage medium is also provided in further embodiment of this invention, a plurality of instruction is stored in the storage medium, this refers to
It enables and is suitable for the step of being loaded by processor to execute any of the above-described image processing method.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of above-described embodiment is can
It is completed with instructing relevant hardware by program, which can be stored in a computer readable storage medium, storage
Medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random
Access Memory), disk or CD etc..
Term " one " and " described " and similar word have been used during describing idea of the invention (especially
In the appended claims), it should be construed to not only cover odd number by these terms but also cover plural number.In addition, unless herein
In be otherwise noted, otherwise herein narration numberical range when referred to merely by quick method and belong to the every of relevant range
A independent value, and each independent value is incorporated into this specification, just as these values have individually carried out statement one herein
Sample.In addition, unless otherwise stated herein or context has specific opposite prompt, otherwise institute described herein is methodical
Step can be executed by any appropriate order.Change of the invention is not limited to the step of description sequence.Unless in addition
Advocate, is otherwise all only using any and all example or exemplary language presented herein (for example, " such as ")
Idea of the invention is better described, and not the range of idea of the invention limited.Spirit and model are not being departed from
In the case where enclosing, those skilled in the art becomes readily apparent that a variety of modifications and adaptation.
Be provided for the embodiments of the invention above a kind of image processing method, device, storage medium and electronic equipment into
It has gone and has been discussed in detail, a specific example illustrates the principle and implementation of the invention for program used herein, above
The explanation of embodiment is merely used to help understand method and its core concept of the invention；Meanwhile for those skilled in the art
Member, according to the thought of the present invention, there will be changes in specific embodiment and range of applications, in conclusion this
Description should not be construed as limiting the invention.
Claims (12)
1. a kind of image processing method is applied to electronic equipment characterized by comprising
Obtain the first location information of face key point in original image；
Match corresponding target facial image from default face database according to first location information, and by target facial image
It is aligned with the face in original image；
Based on the human face region in trained convolutional neural networks model and original image, target facial image is repaired
Just；
Revised target facial image is merged with original image.
2. image processing method as described in claim 1, which is characterized in that the of face key point in obtaining original image
Before one location information, the method also includes:
Construct convolutional neural networks；
The image of multiple angles of face in original image is obtained as training sample；
Parameter training is carried out to constructed convolutional neural networks based on the training sample, with Suitable content loss function, light
Convolutional neural networks model according to the parameter setting of loss function and smooth loss function, after being trained.
3. image processing method as claimed in claim 2, which is characterized in that based on trained convolutional neural networks model and
Human face region in original image, the step of being modified to target facial image, comprising:
Based on trained convolutional neural networks model from the human face region of original image extract content characteristic, illumination feature and
Smooth features；
According to the content characteristic, illumination feature and smooth features, in content loss function, illumination loss function and smooth damage
Under the constraint for losing function, adjusting parameter is generated；
Target facial image is modified according to the adjusting parameter.
4. image processing method as described in claim 1, which is characterized in that according to the positional information from default human face data
Corresponding target facial image, and the step of target facial image is aligned with the face in original image are matched in library, comprising:
Obtain the second location information of the face key point of multiple sample facial images in default face database；
First location information is matched with second location information；
The maximum sample facial image of matching degree is chosen from sample facial image, as target facial image；
Target facial image is mapped to the human face region of original image by affine transformation so that target facial image with it is original
Face alignment in image.
5. image processing method according to any one of claims 1-4, which is characterized in that by target facial image with it is original
After face alignment in image, before revised target facial image is merged with original image, the method also includes:
Edge Feature Points detection is carried out to the human face region in original image, and obtains the third place information of Edge Feature Points；
Image dividing processing is carried out to original image according to the third place information, human face region is removed, by residual image area
Domain is as background image.
6. image processing method as claimed in claim 5 is applied to electronic equipment, which is characterized in that by revised target
The step of facial image is merged with original image, comprising:
Face exposure mask is generated according to the third place information；
Using the face exposure mask, revised target facial image is merged with the background image in original image.
7. a kind of image processing apparatus, which is characterized in that described device includes:
Position acquisition module, for obtaining the first location information of face key point in original image；
Alignment module, for corresponding target facial image to be matched from default face database according to first location information, and
Target facial image is aligned with the face in original image；
Correction module, for based on the human face region in trained convolutional neural networks model and original image, to target person
Face image is modified；
Fusion Module, for merging revised target facial image with original image.
8. image processing apparatus as claimed in claim 7, which is characterized in that described device further include:
Module is constructed, before the first location information of face key point in obtaining original image, constructs convolutional Neural net
Network；
Sample acquisition module, for obtaining the image of multiple angles of face in original image as training sample；
Training module, for carrying out parameter training to constructed convolutional neural networks based on the training sample, in adjustment
Hold the parameter setting of loss function, illumination loss function and smooth loss function, the convolutional neural networks mould after being trained
Type.
9. image processing apparatus as claimed in claim 8, which is characterized in that the correction module includes:
Extracting sub-module, it is special for extracting content from the human face region of original image based on trained convolutional neural networks model
Sign, illumination feature and smooth features；
Submodule is generated, for being damaged in content loss function, illumination according to the content characteristic, illumination feature and smooth features
Under the constraint for losing function and smooth loss function, adjusting parameter is generated；
Submodule is corrected, for being modified according to the adjusting parameter to target facial image.
10. image processing apparatus as claimed in claim 7, which is characterized in that the alignment module includes:
Acquisition submodule, for obtaining the second position of the face key point of multiple sample facial images in default face database
Information；
Matched sub-block, for matching first location information with second location information；
Submodule is chosen, for choosing the maximum sample facial image of matching degree from sample facial image, as target face
Image；
Mapping submodule, for target facial image to be mapped to the human face region of original image by affine transformation, so that mesh
Mark facial image is aligned with the face in original image.
11. a kind of storage medium, which is characterized in that be stored with a plurality of instruction in the storage medium, described instruction be suitable for by
Device load is managed to execute such as image processing method of any of claims 1-6.
12. a kind of electronic equipment, which is characterized in that including processor and memory, the processor and the memory are electrical
Connection, the memory is for storing instruction and data；The processor is for executing as described in any one of claim 1-6
Image processing method.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711466358.3A CN109978754A (en) | 2017-12-28 | 2017-12-28 | Image processing method, device, storage medium and electronic equipment |
| PCT/CN2018/115470 WO2019128508A1 (en) | 2017-12-28 | 2018-11-14 | Method and apparatus for processing image, storage medium, and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711466358.3A CN109978754A (en) | 2017-12-28 | 2017-12-28 | Image processing method, device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109978754A true CN109978754A (en) | 2019-07-05 |
Family
ID=67066431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711466358.3A Pending CN109978754A (en) | 2017-12-28 | 2017-12-28 | Image processing method, device, storage medium and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109978754A (en) |
| WO (1) | WO2019128508A1 (en) |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110458781A (en) * | 2019-08-14 | 2019-11-15 | 北京百度网讯科技有限公司 | Method and apparatus for handling image |
| CN110602403A (en) * | 2019-09-23 | 2019-12-20 | 华为技术有限公司 | Method for taking pictures under dark light and electronic equipment |
| CN110796593A (en) * | 2019-10-15 | 2020-02-14 | 腾讯科技（深圳）有限公司 | Image processing method, device, medium and electronic equipment based on artificial intelligence |
| CN110838084A (en) * | 2019-09-24 | 2020-02-25 | 咪咕文化科技有限公司 | Image style transfer method and device, electronic equipment and storage medium |
| CN110879983A (en) * | 2019-11-18 | 2020-03-13 | 讯飞幻境(北京)科技有限公司 | Face feature key point extraction method and face image synthesis method |
| CN111127378A (en) * | 2019-12-23 | 2020-05-08 | Oppo广东移动通信有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN111209823A (en) * | 2019-12-30 | 2020-05-29 | 南京华图信息技术有限公司 | Infrared human face alignment method |
| CN111476709A (en) * | 2020-04-09 | 2020-07-31 | 广州华多网络科技有限公司 | Face image processing method and device and electronic equipment |
| CN111553865A (en) * | 2020-04-30 | 2020-08-18 | 深圳市商汤科技有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN111553864A (en) * | 2020-04-30 | 2020-08-18 | 深圳市商汤科技有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN111598818A (en) * | 2020-04-17 | 2020-08-28 | 北京百度网讯科技有限公司 | Face fusion model training method and device and electronic equipment |
| CN111709878A (en) * | 2020-06-17 | 2020-09-25 | 北京百度网讯科技有限公司 | Face super-resolution implementation method and device, electronic equipment and storage medium |
| CN111741214A (en) * | 2020-05-13 | 2020-10-02 | 北京迈格威科技有限公司 | Image processing method and device and electronic equipment |
| CN111783519A (en) * | 2020-05-15 | 2020-10-16 | 北京迈格威科技有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111797753A (en) * | 2020-06-29 | 2020-10-20 | 北京灵汐科技有限公司 | Training method, device, equipment and medium of image driving model, and image generation method, device and medium |
| CN112069995A (en) * | 2020-09-04 | 2020-12-11 | 西安西图之光智能科技有限公司 | Method, system and storage medium for extracting dense features of face in depth domain |
| CN112258622A (en) * | 2020-10-26 | 2021-01-22 | 北京字跳网络技术有限公司 | Image processing method, image processing device, readable medium and electronic equipment |
| CN112288665A (en) * | 2020-09-30 | 2021-01-29 | 北京大米科技有限公司 | Image fusion method and device, storage medium and electronic equipment |
| CN112381749A (en) * | 2020-11-24 | 2021-02-19 | 维沃移动通信有限公司 | Image processing method, image processing device and electronic equipment |
| CN112750071A (en) * | 2020-11-04 | 2021-05-04 | 上海序言泽网络科技有限公司 | User-defined expression making method and system |
| CN112766215A (en) * | 2021-01-29 | 2021-05-07 | 北京字跳网络技术有限公司 | Face fusion method and device, electronic equipment and storage medium |
| CN112926369A (en) * | 2019-12-06 | 2021-06-08 | 中兴通讯股份有限公司 | Face image processing method and device, computer equipment and medium |
| CN113128304A (en) * | 2019-12-31 | 2021-07-16 | 深圳云天励飞技术有限公司 | Image processing method and electronic equipment |
| CN113469903A (en) * | 2021-06-11 | 2021-10-01 | 维沃移动通信有限公司 | Image processing method and device, electronic equipment and readable storage medium |
| CN113569789A (en) * | 2019-07-30 | 2021-10-29 | 北京市商汤科技开发有限公司 | Image processing method and device, processor, electronic device and storage medium |
| CN113658125A (en) * | 2021-08-11 | 2021-11-16 | 全芯智造技术有限公司 | Method, device and storage medium for evaluating layout hot spot |
| CN113673278A (en) * | 2020-05-13 | 2021-11-19 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN114387649A (en) * | 2022-01-11 | 2022-04-22 | 北京百度网讯科技有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN115082298A (en) * | 2022-07-15 | 2022-09-20 | 北京百度网讯科技有限公司 | Image generation method, image generation device, electronic device, and storage medium |
| CN115861042A (en) * | 2023-02-08 | 2023-03-28 | 荣耀终端有限公司 | Image processing method, electronic device and medium |
| CN116758124A (en) * | 2023-06-16 | 2023-09-15 | 北京代码空间科技有限公司 | 3D model correction method and terminal equipment |
| CN111754396B (en) * | 2020-07-27 | 2024-01-09 | 腾讯科技（深圳）有限公司 | Face image processing method, device, computer equipment and storage medium |
Families Citing this family (58)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110321849B (en) * | 2019-07-05 | 2023-12-22 | 腾讯科技（深圳）有限公司 | Image data processing method, device and computer readable storage medium |
| CN110517185B (en) * | 2019-07-23 | 2024-02-09 | 北京达佳互联信息技术有限公司 | Image processing method, device, electronic equipment and storage medium |
| CN112330526B (en) * | 2019-08-05 | 2024-02-09 | 深圳Tcl新技术有限公司 | Training method of face conversion model, storage medium and terminal equipment |
| CN111008927B (en) * | 2019-08-07 | 2023-10-31 | 深圳华侨城文化旅游科技集团有限公司 | Face replacement method, storage medium and terminal equipment |
| CN112419328B (en) * | 2019-08-22 | 2023-08-04 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN110517214B (en) * | 2019-08-28 | 2022-04-12 | 北京百度网讯科技有限公司 | Method and apparatus for generating image |
| CN110826395B (en) * | 2019-09-18 | 2023-10-31 | 平安科技（深圳）有限公司 | Face rotation model generation method and device, computer equipment and storage medium |
| CN110766645B (en) * | 2019-10-24 | 2023-03-10 | 西安电子科技大学 | Target person recurrence map generation method based on person identification and segmentation |
| CN111008935B (en) * | 2019-11-01 | 2023-12-12 | 北京迈格威科技有限公司 | Face image enhancement method, device, system and storage medium |
| CN111028142B (en) * | 2019-11-25 | 2024-02-13 | 泰康保险集团股份有限公司 | Image processing method, device and storage medium |
| CN112991191B (en) * | 2019-12-13 | 2024-09-10 | 北京金山云网络技术有限公司 | Face image enhancement method and device and electronic equipment |
| CN111160350B (en) * | 2019-12-23 | 2023-05-16 | Oppo广东移动通信有限公司 | Portrait segmentation method, model training method, device, medium and electronic equipment |
| CN111080747B (en) * | 2019-12-26 | 2023-04-07 | 维沃移动通信有限公司 | Face image processing method and electronic equipment |
| CN111179285B (en) * | 2019-12-31 | 2023-06-20 | 珠海方图智能科技有限公司 | Image processing method, system and storage medium |
| CN113128277A (en) * | 2019-12-31 | 2021-07-16 | Tcl集团股份有限公司 | Generation method of face key point detection model and related equipment |
| CN113129208B (en) * | 2019-12-31 | 2024-03-26 | 深圳云天励飞技术有限公司 | Sketch-based face image generation method and related products |
| CN111445437A (en) * | 2020-02-25 | 2020-07-24 | 杭州火烧云科技有限公司 | Method, system and equipment for processing image by skin processing model constructed based on convolutional neural network |
| CN113313786B (en) * | 2020-02-27 | 2024-06-11 | 深圳云天励飞技术有限公司 | Portrait picture coloring method and device and terminal equipment |
| CN111402181A (en) * | 2020-03-13 | 2020-07-10 | 北京奇艺世纪科技有限公司 | Image fusion method and device and computer readable storage medium |
| CN111666974B (en) * | 2020-04-29 | 2024-07-16 | 平安科技（深圳）有限公司 | Image matching method, device, computer equipment and storage medium |
| CN113628122A (en) * | 2020-05-09 | 2021-11-09 | 阿里巴巴集团控股有限公司 | Image processing method, model training method, device and equipment |
| CN111583159B (en) * | 2020-05-29 | 2024-01-05 | 北京金山云网络技术有限公司 | Image complement method and device and electronic equipment |
| CN111833240B (en) * | 2020-06-03 | 2023-07-25 | 北京百度网讯科技有限公司 | Face image conversion method and device, electronic equipment and storage medium |
| CN111814566A (en) * | 2020-06-11 | 2020-10-23 | 北京三快在线科技有限公司 | Image editing method, image editing device, electronic equipment and storage medium |
| CN111914630A (en) * | 2020-06-19 | 2020-11-10 | 北京百度网讯科技有限公司 | Method, apparatus, device and storage medium for generating training data for face recognition |
| CN111739046A (en) * | 2020-06-19 | 2020-10-02 | 百度在线网络技术（北京）有限公司 | Method, apparatus, device and medium for model update and image detection |
| CN111783608B (en) * | 2020-06-24 | 2024-03-19 | 南京烽火星空通信发展有限公司 | Face-changing video detection method |
| CN111768356A (en) * | 2020-06-28 | 2020-10-13 | 北京百度网讯科技有限公司 | Face image fusion method and device, electronic equipment and storage medium |
| CN113870094A (en) * | 2020-06-30 | 2021-12-31 | 北京达佳互联信息技术有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN111784611B (en) * | 2020-07-03 | 2023-11-03 | 厦门美图之家科技有限公司 | Portrait whitening method, device, electronic equipment and readable storage medium |
| CN111797797B (en) * | 2020-07-13 | 2023-09-15 | 深圳大学 | Face image processing method, terminal and storage medium based on grid deformation optimization |
| CN111815534B (en) * | 2020-07-14 | 2023-12-19 | 厦门美图之家科技有限公司 | Real-time skin makeup migration method, device, electronic equipment and readable storage medium |
| CN113971822A (en) * | 2020-07-22 | 2022-01-25 | 武汉Tcl集团工业研究院有限公司 | Face detection method, intelligent terminal and storage medium |
| CN112001262B (en) * | 2020-07-28 | 2022-07-29 | 山东师范大学 | Method for generating accessory capable of influencing face authentication |
| CN112001859B (en) * | 2020-08-10 | 2024-04-16 | 深思考人工智能科技(上海)有限公司 | Face image restoration method and system |
| CN112037143A (en) * | 2020-08-27 | 2020-12-04 | 腾讯音乐娱乐科技（深圳）有限公司 | Image processing method and device |
| CN112116565B (en) * | 2020-09-03 | 2023-12-05 | 深圳大学 | Method, apparatus and storage medium for generating countersamples for falsifying a flip image |
| CN112070021B (en) * | 2020-09-09 | 2024-08-13 | 深圳数联天下智能科技有限公司 | Ranging method, ranging system, equipment and storage medium based on face detection |
| CN112257504B (en) * | 2020-09-16 | 2024-09-06 | 深圳数联天下智能科技有限公司 | Face recognition method, face recognition model training method and related device |
| CN112102198A (en) * | 2020-09-17 | 2020-12-18 | 广州虎牙科技有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN112102200B (en) * | 2020-09-21 | 2024-05-07 | 腾讯科技（深圳）有限公司 | Image complement model initialization method, training method and image complement method |
| CN112150489A (en) * | 2020-09-25 | 2020-12-29 | 北京百度网讯科技有限公司 | Image style conversion method and device, electronic equipment and storage medium |
| CN112287772B (en) * | 2020-10-10 | 2023-02-10 | 深圳市中达瑞和科技有限公司 | Fingerprint trace detection method, fingerprint detection device and computer readable storage medium |
| CN112241716B (en) * | 2020-10-23 | 2023-06-20 | 北京百度网讯科技有限公司 | Training sample generation method and device |
| CN112330533A (en) * | 2020-11-13 | 2021-02-05 | 北京字跳网络技术有限公司 | Mixed blood face image generation method, model training method, device and equipment |
| CN112288657B (en) * | 2020-11-16 | 2024-09-06 | 北京小米松果电子有限公司 | Image processing method, image processing apparatus, and storage medium |
| CN112561816A (en) * | 2020-12-10 | 2021-03-26 | 厦门美图之家科技有限公司 | Image processing method, image processing device, electronic equipment and readable storage medium |
| CN112541477B (en) | 2020-12-24 | 2024-05-31 | 北京百度网讯科技有限公司 | Expression pack generation method and device, electronic equipment and storage medium |
| CN112733667B (en) * | 2020-12-30 | 2024-06-28 | 平安科技（深圳）有限公司 | Face alignment method and device based on face recognition |
| CN113095134B (en) * | 2021-03-08 | 2024-03-29 | 北京达佳互联信息技术有限公司 | Facial expression extraction model generation method and device and facial image generation method and device |
| CN113469041A (en) * | 2021-06-30 | 2021-10-01 | 北京市商汤科技开发有限公司 | Image processing method and device, computer equipment and storage medium |
| CN113706428B (en) * | 2021-07-02 | 2024-01-05 | 杭州海康威视数字技术股份有限公司 | Image generation method and device |
| CN113705466B (en) * | 2021-08-30 | 2024-02-09 | 浙江中正智能科技有限公司 | Face five sense organ shielding detection method for shielding scene, especially under high imitation shielding |
| CN113743371B (en) * | 2021-09-22 | 2024-07-19 | 京东方科技集团股份有限公司 | Fingerprint identification method and fingerprint identification device |
| CN113920005B (en) * | 2021-09-29 | 2024-04-19 | 杭州海马体摄影有限公司 | Method for constructing single face skin difference picture pair |
| CN114373033B (en) * | 2022-01-10 | 2024-08-20 | 腾讯科技（深圳）有限公司 | Image processing method, apparatus, device, storage medium, and computer program |
| CN115908119B (en) * | 2023-01-05 | 2023-06-06 | 广州佰锐网络科技有限公司 | Face image beautifying processing method and system based on artificial intelligence |
| CN116862757B (en) * | 2023-05-19 | 2024-08-02 | 上海任意门科技有限公司 | Method, device, electronic equipment and medium for controlling face stylization degree |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123749A (en) * | 2014-07-23 | 2014-10-29 | 邢小月 | Picture processing method and system |
| CN105095856A (en) * | 2015-06-26 | 2015-11-25 | 上海交通大学 | Method for recognizing human face with shielding based on mask layer |
| US20160086304A1 (en) * | 2014-09-22 | 2016-03-24 | Ming Chuan University | Method for estimating a 3d vector angle from a 2d face image, method for creating face replacement database, and method for replacing face image |
| CN106023063A (en) * | 2016-05-09 | 2016-10-12 | 西安北升信息科技有限公司 | Video transplantation face changing method |
| CN107330408A (en) * | 2017-06-30 | 2017-11-07 | 北京金山安全软件有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN107507216A (en) * | 2017-08-17 | 2017-12-22 | 北京觅己科技有限公司 | The replacement method of regional area, device and storage medium in image |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657974A (en) * | 2013-11-25 | 2015-05-27 | 腾讯科技（上海）有限公司 | Image processing method and device |
| CN105741229B (en) * | 2016-02-01 | 2019-01-08 | 成都通甲优博科技有限责任公司 | The method for realizing facial image rapid fusion |
| CN106412547B (en) * | 2016-08-29 | 2019-01-22 | 厦门美图之家科技有限公司 | A kind of image white balance method based on convolutional neural networks, device and calculate equipment |
| CN106815566B (en) * | 2016-12-29 | 2021-04-16 | 天津中科智能识别产业技术研究院有限公司 | Face retrieval method based on multitask convolutional neural network |
-
2017
- 2017-12-28 CN CN201711466358.3A patent/CN109978754A/en active Pending
-
2018
- 2018-11-14 WO PCT/CN2018/115470 patent/WO2019128508A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104123749A (en) * | 2014-07-23 | 2014-10-29 | 邢小月 | Picture processing method and system |
| US20160086304A1 (en) * | 2014-09-22 | 2016-03-24 | Ming Chuan University | Method for estimating a 3d vector angle from a 2d face image, method for creating face replacement database, and method for replacing face image |
| CN105095856A (en) * | 2015-06-26 | 2015-11-25 | 上海交通大学 | Method for recognizing human face with shielding based on mask layer |
| CN106023063A (en) * | 2016-05-09 | 2016-10-12 | 西安北升信息科技有限公司 | Video transplantation face changing method |
| CN107330408A (en) * | 2017-06-30 | 2017-11-07 | 北京金山安全软件有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN107507216A (en) * | 2017-08-17 | 2017-12-22 | 北京觅己科技有限公司 | The replacement method of regional area, device and storage medium in image |
Non-Patent Citations (1)
| Title |
|---|
| KORSHUNOVA,ET AL: "Fast Face-swap Using Convolutional Neural", 《IEEE 2017 INTERNATIONAL CONFERENCE ON COMPUTER VISION》 * |
Cited By (50)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113569789B (en) * | 2019-07-30 | 2024-04-16 | 北京市商汤科技开发有限公司 | Image processing method and device, processor, electronic equipment and storage medium |
| CN113569789A (en) * | 2019-07-30 | 2021-10-29 | 北京市商汤科技开发有限公司 | Image processing method and device, processor, electronic device and storage medium |
| TWI753327B (en) * | 2019-07-30 | 2022-01-21 | 大陸商北京市商湯科技開發有限公司 | Image processing method, processor, electronic device and computer-readable storage medium |
| CN110458781A (en) * | 2019-08-14 | 2019-11-15 | 北京百度网讯科技有限公司 | Method and apparatus for handling image |
| CN110602403A (en) * | 2019-09-23 | 2019-12-20 | 华为技术有限公司 | Method for taking pictures under dark light and electronic equipment |
| CN110838084A (en) * | 2019-09-24 | 2020-02-25 | 咪咕文化科技有限公司 | Image style transfer method and device, electronic equipment and storage medium |
| CN110838084B (en) * | 2019-09-24 | 2023-10-17 | 咪咕文化科技有限公司 | Method and device for transferring style of image, electronic equipment and storage medium |
| CN110796593A (en) * | 2019-10-15 | 2020-02-14 | 腾讯科技（深圳）有限公司 | Image processing method, device, medium and electronic equipment based on artificial intelligence |
| CN110879983A (en) * | 2019-11-18 | 2020-03-13 | 讯飞幻境(北京)科技有限公司 | Face feature key point extraction method and face image synthesis method |
| CN112926369A (en) * | 2019-12-06 | 2021-06-08 | 中兴通讯股份有限公司 | Face image processing method and device, computer equipment and medium |
| WO2021129642A1 (en) * | 2019-12-23 | 2021-07-01 | Oppo广东移动通信有限公司 | Image processing method, apparatus, computer device, and storage medium |
| CN111127378A (en) * | 2019-12-23 | 2020-05-08 | Oppo广东移动通信有限公司 | Image processing method, image processing device, computer equipment and storage medium |
| CN111209823A (en) * | 2019-12-30 | 2020-05-29 | 南京华图信息技术有限公司 | Infrared human face alignment method |
| CN113128304A (en) * | 2019-12-31 | 2021-07-16 | 深圳云天励飞技术有限公司 | Image processing method and electronic equipment |
| CN113128304B (en) * | 2019-12-31 | 2024-01-05 | 深圳云天励飞技术有限公司 | Image processing method and electronic equipment |
| CN111476709A (en) * | 2020-04-09 | 2020-07-31 | 广州华多网络科技有限公司 | Face image processing method and device and electronic equipment |
| CN111476709B (en) * | 2020-04-09 | 2023-04-07 | 广州方硅信息技术有限公司 | Face image processing method and device and electronic equipment |
| CN111598818A (en) * | 2020-04-17 | 2020-08-28 | 北京百度网讯科技有限公司 | Face fusion model training method and device and electronic equipment |
| US11830288B2 (en) | 2020-04-17 | 2023-11-28 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Method and apparatus for training face fusion model and electronic device |
| CN111598818B (en) * | 2020-04-17 | 2023-04-28 | 北京百度网讯科技有限公司 | Training method and device for face fusion model and electronic equipment |
| CN111553865B (en) * | 2020-04-30 | 2023-08-22 | 深圳市商汤科技有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN111553864B (en) * | 2020-04-30 | 2023-11-28 | 深圳市商汤科技有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN111553864A (en) * | 2020-04-30 | 2020-08-18 | 深圳市商汤科技有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN111553865A (en) * | 2020-04-30 | 2020-08-18 | 深圳市商汤科技有限公司 | Image restoration method and device, electronic equipment and storage medium |
| CN113673278A (en) * | 2020-05-13 | 2021-11-19 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN111741214A (en) * | 2020-05-13 | 2020-10-02 | 北京迈格威科技有限公司 | Image processing method and device and electronic equipment |
| CN111783519A (en) * | 2020-05-15 | 2020-10-16 | 北京迈格威科技有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN111709878A (en) * | 2020-06-17 | 2020-09-25 | 北京百度网讯科技有限公司 | Face super-resolution implementation method and device, electronic equipment and storage medium |
| US11710215B2 (en) | 2020-06-17 | 2023-07-25 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Face super-resolution realization method and apparatus, electronic device and storage medium |
| CN111797753B (en) * | 2020-06-29 | 2024-02-27 | 北京灵汐科技有限公司 | Training of image driving model, image generation method, device, equipment and medium |
| CN111797753A (en) * | 2020-06-29 | 2020-10-20 | 北京灵汐科技有限公司 | Training method, device, equipment and medium of image driving model, and image generation method, device and medium |
| CN111754396B (en) * | 2020-07-27 | 2024-01-09 | 腾讯科技（深圳）有限公司 | Face image processing method, device, computer equipment and storage medium |
| CN112069995B (en) * | 2020-09-04 | 2024-02-27 | 西安西图之光智能科技有限公司 | Depth domain face dense feature extraction method, system and storage medium |
| CN112069995A (en) * | 2020-09-04 | 2020-12-11 | 西安西图之光智能科技有限公司 | Method, system and storage medium for extracting dense features of face in depth domain |
| CN112288665B (en) * | 2020-09-30 | 2024-05-07 | 北京大米科技有限公司 | Image fusion method and device, storage medium and electronic equipment |
| CN112288665A (en) * | 2020-09-30 | 2021-01-29 | 北京大米科技有限公司 | Image fusion method and device, storage medium and electronic equipment |
| CN112258622A (en) * | 2020-10-26 | 2021-01-22 | 北京字跳网络技术有限公司 | Image processing method, image processing device, readable medium and electronic equipment |
| CN112750071B (en) * | 2020-11-04 | 2023-11-24 | 上海序言泽网络科技有限公司 | User-defined expression making method and system |
| CN112750071A (en) * | 2020-11-04 | 2021-05-04 | 上海序言泽网络科技有限公司 | User-defined expression making method and system |
| CN112381749B (en) * | 2020-11-24 | 2024-08-23 | 维沃移动通信有限公司 | Image processing method, image processing device and electronic equipment |
| CN112381749A (en) * | 2020-11-24 | 2021-02-19 | 维沃移动通信有限公司 | Image processing method, image processing device and electronic equipment |
| CN112766215A (en) * | 2021-01-29 | 2021-05-07 | 北京字跳网络技术有限公司 | Face fusion method and device, electronic equipment and storage medium |
| WO2022258013A1 (en) * | 2021-06-11 | 2022-12-15 | 维沃移动通信有限公司 | Image processing method and apparatus, electronic device and readable storage medium |
| CN113469903A (en) * | 2021-06-11 | 2021-10-01 | 维沃移动通信有限公司 | Image processing method and device, electronic equipment and readable storage medium |
| CN113658125A (en) * | 2021-08-11 | 2021-11-16 | 全芯智造技术有限公司 | Method, device and storage medium for evaluating layout hot spot |
| CN113658125B (en) * | 2021-08-11 | 2024-02-23 | 全芯智造技术有限公司 | Method, device and storage medium for evaluating layout hot spot |
| CN114387649A (en) * | 2022-01-11 | 2022-04-22 | 北京百度网讯科技有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN115082298A (en) * | 2022-07-15 | 2022-09-20 | 北京百度网讯科技有限公司 | Image generation method, image generation device, electronic device, and storage medium |
| CN115861042A (en) * | 2023-02-08 | 2023-03-28 | 荣耀终端有限公司 | Image processing method, electronic device and medium |
| CN116758124A (en) * | 2023-06-16 | 2023-09-15 | 北京代码空间科技有限公司 | 3D model correction method and terminal equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2019128508A1 (en) | 2019-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109978754A (en) | Image processing method, device, storage medium and electronic equipment | |
| CN108229369B (en) | Image shooting method and device, storage medium and electronic equipment | |
| WO2021036059A1 (en) | Image conversion model training method, heterogeneous face recognition method, device and apparatus | |
| WO2020103700A1 (en) | Image recognition method based on micro facial expressions, apparatus and related device | |
| WO2021052375A1 (en) | Target image generation method, apparatus, server and storage medium | |
| Liu et al. | Real-time robust vision-based hand gesture recognition using stereo images | |
| CN103731583B (en) | Intelligent synthetic, print processing method is used for taking pictures | |
| CN113822977A (en) | Image rendering method, device, equipment and storage medium | |
| CN109657533A (en) | Pedestrian recognition methods and Related product again | |
| CN108198130A (en) | Image processing method, device, storage medium and electronic equipment | |
| CN111739027B (en) | Image processing method, device, equipment and readable storage medium | |
| Vazquez-Fernandez et al. | Built-in face recognition for smart photo sharing in mobile devices | |
| CN112036331A (en) | Training method, device and equipment of living body detection model and storage medium | |
| CN113298158B (en) | Data detection method, device, equipment and storage medium | |
| WO2022120843A1 (en) | Three-dimensional human body reconstruction method and apparatus, and computer device and storage medium | |
| CN112446322B (en) | Eyeball characteristic detection method, device, equipment and computer readable storage medium | |
| CN112561973A (en) | Method and device for training image registration model and electronic equipment | |
| CN116048244B (en) | Gaze point estimation method and related equipment | |
| CN115115552B (en) | Image correction model training method, image correction device and computer equipment | |
| Shah et al. | Efficient portable camera based text to speech converter for blind person | |
| CN114881893B (en) | Image processing method, device, equipment and computer readable storage medium | |
| Yu et al. | A video-based facial motion tracking and expression recognition system | |
| CN111666976A (en) | Feature fusion method and device based on attribute information and storage medium | |
| CN110991325A (en) | Model training method, image recognition method and related device | |
| CN115967823A (en) | Video cover generation method and device, electronic equipment and readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190705 |
|
| RJ01 | Rejection of invention patent application after publication |