CN113343878A - High-fidelity face privacy protection method and system based on generation countermeasure network - Google Patents

High-fidelity face privacy protection method and system based on generation countermeasure network Download PDF

Info

Publication number
CN113343878A
CN113343878A CN202110681374.4A CN202110681374A CN113343878A CN 113343878 A CN113343878 A CN 113343878A CN 202110681374 A CN202110681374 A CN 202110681374A CN 113343878 A CN113343878 A CN 113343878A
Authority
CN
China
Prior art keywords
face
image
identity
fidelity
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110681374.4A
Other languages
Chinese (zh)
Inventor
杨辉华
张隆昊
李灵巧
许亦博
李忠明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing University of Posts and Telecommunications
Original Assignee
Beijing University of Posts and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing University of Posts and Telecommunications filed Critical Beijing University of Posts and Telecommunications
Priority to CN202110681374.4A priority Critical patent/CN113343878A/en
Publication of CN113343878A publication Critical patent/CN113343878A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2132—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on discrimination criteria, e.g. discriminant analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Bioethics (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention provides a high-fidelity face privacy protection method and system based on a generation countermeasure network, wherein the method comprises the following steps: carrying out face recognition on a source face image to obtain multi-scale identity characteristics of a source face; at least obtaining a target face boundary box and face key point information based on an input target face image; extracting specific key points related to postures and/or expressions in the obtained face key points, and obtaining a face key point connecting line graph based on the extracted specific key points; obtaining a fidelity face privacy protection image based on a generation countermeasure network GAN, comprising: the method comprises the steps of taking a target face image and multi-scale identity characteristics of a source face as input, and generating a fidelity face privacy protection image with the identity of the source face and the attributes of the target face by utilizing an Unet neural network structure; and (3) taking the fidelity face privacy protection image and the face key point connecting line graph as common input to judge the recognition result of the fidelity face privacy protection image, and optimizing the Unet neural network structure based on the recognition result.

Description

High-fidelity face privacy protection method and system based on generation countermeasure network
Technical Field
The invention relates to the technical field of computer vision, in particular to a high-fidelity face privacy protection method and system based on a generation countermeasure network.
Background
At present, the face synthesis and replacement technology is receiving more and more attention as a new computer vision technology, and has great application value in the aspects of entertainment, virtual reality, privacy protection, video chat and other vision technologies. The current mainstream face synthesis and replacement technology model is complex, has high requirements on hardware and has long reasoning time. In addition, the generated face of each frame cannot well keep the attributes (such as posture, expression, skin color, illumination, makeup and the like) of the target face of the frame, so that the problems of discontinuity, instability and the like in a time domain occur, the effect of the generated face on video application is poor, the distortion of the image of the target face is often serious under the condition of meeting privacy protection, and the good privacy protection effect under the high-fidelity state of the face is difficult to achieve at present.
Therefore, how to realize good privacy protection of the target face in a high fidelity state is a problem to be solved.
Disclosure of Invention
In view of the above, the present invention provides a high fidelity face privacy protection method and system based on generation countermeasure network, so as to eliminate or improve one or more defects existing in the prior art.
According to one aspect of the invention, a high-fidelity face privacy protection method based on a generation countermeasure network is provided, and the method comprises the following steps:
a source face identity coding step, which is used for carrying out face recognition on a source face image and acquiring the multi-scale identity characteristics of a source face;
a face and key point detection step, which is used for at least obtaining a target face boundary box and face key point information based on an input target face image;
a key point connecting line graph obtaining step, which is used for extracting specific key points related to gestures and/or expressions in the obtained face key points and obtaining the face key point connecting line graph based on the extracted specific key points;
obtaining a fidelity face privacy protection image based on a generation countermeasure network GAN, comprising:
an image generation step, which is used for synthesizing a fidelity face privacy protection image with the identity of the source face and the attribute of the target face by using a Unet neural network structure by taking the multi-scale identity characteristics of the target face image and the source face as input;
and a judging step, namely judging the recognition result of the fidelity face privacy protection image by taking the fidelity face privacy protection image and the face key point connecting line graph as common input, and optimizing the Unet neural network structure based on the recognition result.
In some embodiments of the invention, the identity encoding step comprises: acquiring multi-scale identity characteristics of a source face by adopting a k-shot strategy; the method for acquiring the multi-scale identity characteristics of the source face by adopting the k-shot strategy comprises the following steps: acquiring k identity characteristics based on k source face images, and averaging the acquired k identity characteristics to obtain final identity characteristics; and obtaining the multi-scale identity characteristics of the source face based on the final identity characteristics.
In some embodiments of the invention, the image generating step comprises: an identity migration step, namely obtaining a first self-adaptive output characteristic of a source face image through example normalization processing, layer normalization processing and deep learning based on multi-scale identity characteristics of the source face image; a gesture expression control step, namely obtaining a second self-adaptive output characteristic of the target face image based on example normalization processing, layer normalization processing and deep learning based on a face key point connecting line graph of the target face image; and generating a fidelity face privacy protection image with the identity of the source face and the attributes of the target person by utilizing a Unet neural network structure based on the first adaptive output characteristics and the first adaptive output characteristics.
In some embodiments of the invention, the first adaptive output characteristic conforms to the following equation:
Figure BDA0003122726470000021
the second adaptive output characteristic conforms to the following equation:
Figure BDA0003122726470000022
wherein the content of the first and second substances,
Figure BDA0003122726470000023
Figure BDA0003122726470000024
M∈RH×W×C；
Figure BDA0003122726470000025
Figure BDA0003122726470000026
Figure BDA0003122726470000027
Figure BDA0003122726470000031
Figure BDA0003122726470000032
wherein M is the characteristic of the input image, and R represents a real number domain; h and W represent the height and width of the characteristic diagram respectively, and C represents the number of characteristic channels; mu.sINAnd σINIs the mean and standard deviation for the example normalization,
Figure BDA0003122726470000033
and
Figure BDA0003122726470000034
respectively represent the mean and variance, μ, of each instance in the c-th channelLNAnd σLNIs the mean and standard deviation for layer normalization; gamma rayIDAnd betaIDIs a parameter learned by multi-scale identity features; gamma rayPEAnd betaPEIs a line graph B passing through the key pointstThe parameters of the parameters that are learned are,
Figure BDA0003122726470000035
is MINThe value of the c-th channel position (x, y),
Figure BDA0003122726470000036
is MLNThe value of the c-th channel position (x, y),
Figure BDA0003122726470000037
is the average of LN over the c-th channel.
In some embodiments of the invention, the method further comprises: determining a face area based on the obtained target face bounding box, and calculating an affine transformation matrix to align the face; a fusion step, namely adjusting the generated fidelity face privacy protection image by using a mirror image Sigmoid mask, and fusing the synthesized fidelity face privacy protection image with a target image, so that the outer edge pixels of the generated face mainly use the pixels of the target face, and the inner part of the generated face keeps the pixels of the generated face; and pasting the adjusted fidelity face privacy protection image back to the corresponding position of the target image or the video by utilizing the affine transformation matrix.
In some embodiments of the invention, the method further comprises: the discrimination module optimizes the Unet neural network using a countering loss function, a perceptual loss function, an identity loss function, and a reconstruction loss function.
In some embodiments of the invention, the penalty function satisfies:
Figure BDA0003122726470000038
Figure BDA0003122726470000039
the identity loss function is expressed as:
Figure BDA00031227264700000310
the reconstruction loss function is expressed as:
Figure BDA00031227264700000311
the perceptual loss function is expressed as:
Figure BDA00031227264700000312
Figure BDA00031227264700000313
Figure BDA00031227264700000314
Figure BDA00031227264700000315
wherein G represents the result of the generator, DiI-th discrimination result representing the discriminator, n representing the number of discrimination results, FtRepresenting the image of the target face, FsRepresenting the source face image, BtRepresentative face gateA connection line graph of the key points is shown,
Figure BDA00031227264700000316
representing the desired value, Y representing the generated face image,1represents weight value, which can be set to 0.1, H and W represent height and width of the feature map, respectively, C represents number of feature channels, FsAnd YsAre respectively FtAnd Y in s layers, m representing the number of layers, Cs、HsAnd WsFG representing the number, width and height of channels of the s-layer, respectivelysAnd YGsIs FsAnd YsThe matrix of (a) is a matrix of,
Figure BDA0003122726470000041
represents FtThe value at the ith row and kth column of the Gram matrix of features at the s-th layer,
Figure BDA0003122726470000042
represents FtThe value of the Gram matrix of features at the s-th layer at the j-th row and k-th column,
Figure BDA0003122726470000043
the value of the ith row and the kth column of the Gram matrix representing the characteristics of Y at the s-th layer,
Figure BDA0003122726470000044
the j row and k column values of the Gram matrix representing the characteristics of Y at the s layer.
In some embodiments of the present invention, the source face image or the target face image comprises a still image or a moving image, the moving image comprising a video frame; the attributes of the target face comprise at least one of the following attributes: pose, expression, skin color, lighting and makeup of the target face; the method further comprises the following steps: and inputting the synthesized fidelity human face privacy protection images with different resolutions into a plurality of discrimination modules with the same network structure, and optimizing the Unet neural network structure of the generation module based on the discrimination result.
According to another aspect of the present invention, there is also provided a high fidelity face privacy protection system based on generation of a countermeasure network, the system comprising a processor and a memory, the memory having stored therein computer instructions for executing the computer instructions stored in the memory, the apparatus implementing the steps of the method as described above when the computer instructions are executed by the processor.
In a further aspect of the invention, a computer-readable storage medium is also provided, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method as set forth above.
The high-fidelity face privacy protection method and system based on the generation countermeasure network can well synthesize the whole face area, hair and background, so that the face area, the hair and the background can be easily fused into a target frame.
It will be appreciated by those skilled in the art that the objects and advantages that can be achieved with the present invention are not limited to the specific details set forth above, and that these and other objects that can be achieved with the present invention will be more clearly understood from the detailed description that follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:
fig. 1 is a schematic flow chart of a high-fidelity face privacy protection method based on a generation countermeasure network in an embodiment of the present invention.
Fig. 2 is a schematic diagram of a system module structure of a high-fidelity face privacy protection method in an embodiment of the present invention.
Fig. 3 is a schematic block diagram of an identity migration module according to an embodiment of the present invention.
Fig. 4 is a schematic structural diagram of a gesture expression control module according to an embodiment of the present invention.
Fig. 5 is a schematic flow chart of a high-fidelity face privacy protection method in another embodiment of the present invention.
Fig. 6a, 6b and 6c are schematic diagrams of a mirror sigmoid mask and 2D and 3D visualizations thereof, respectively, according to an embodiment of the invention.
FIG. 7 is a graph showing the effect of the method of the present invention compared with the prior art method.
Fig. 8 is a schematic implementation flow diagram of a high-fidelity face privacy protection method in another embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
It should be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and/or processing steps closely related to the scheme according to the present invention are shown in the drawings, and other details not so relevant to the present invention are omitted.
It should be emphasized that the term "comprises/comprising" when used herein, is taken to specify the presence of stated features, elements, steps or components, but does not preclude the presence or addition of one or more other features, elements, steps or components.
In order to obtain a high-fidelity image which can fully protect the face privacy, the invention provides a high-fidelity face privacy protection method based on a generation countermeasure network.
Fig. 1 is a schematic flow chart of a high-fidelity face privacy protection method based on a generation countermeasure network in an embodiment of the present invention. As shown in fig. 1, the method comprises the steps of:
and a source face identity coding step S110, wherein the source face image is subjected to face recognition to obtain the multi-scale identity characteristics of the source face.
The module for realizing the step in the method can be called as an identity coding module, the identity coding module carries out face recognition on a source face image by utilizing a pre-trained face recognition network, and multi-scale identity features are extracted from a source face.
As an example, the face recognition network employed is the currently popular ArcFace. The multi-scale identity feature may be an identity feature of multiple resolution scales.
And a human face and key point detection step S120, wherein at least a target human face boundary box and human face key point information are obtained based on the input target human face image.
The module for realizing the step in the method can be called a face and key point detection module, more specifically, the face and key point detection module can adopt a pre-trained current mainstream SFD algorithm to detect the face key points, and can also use a current mainstream 2D-FAN algorithm to detect the face boundary box. The models have small parameter quantity, short reasoning time and lower requirement on video memory.
And a key point connecting line graph obtaining step S130 of extracting specific key points related to the pose and/or expression from the target face key points obtained in step S120, and obtaining a face key point connecting line graph based on the extracted specific key points.
For example, the face key point wiring diagram B can be obtained by wiring specific key points related to the pose and/or expressiontThe face keypoint wiring graph is also associated with pose and/or expression.
Since the gesture and expression can be considered to be mainly described by the opening and closing of the eyes, the position of the eyeballs, the direction of the nose bridge, and the mouth shape, the key points selected by the embodiment of the present invention, which are closely related to the gesture and expression, include, for example: the eyes, the eyeballs, the mouth, the nose bridge and other fewer key points are selected, and the influence on the identity of the face can be reduced as much as possible.
In step S140, a fidelity face privacy protection image is obtained based on a generated countermeasure network (GAN).
The generation of the countermeasure network GAN is a neural network learning model in which a game is continuously played by a generator G (generator) and a discriminator d (discriminator), and the generator G learns the distribution of data. During the training process, the generator G aims to generate as many real pictures as possible to trick the discriminator D. The goal of the discriminator D is to discriminate as much as possible between the false image generated by the generator G and the true image. Thus, the generator G and the discriminator D form a dynamic game process, and finally, the result of the game of the two parties is that the false samples generated by the generator are almost not different from the true samples, so that the discriminator cannot distinguish the true samples from the false samples, the generator and the discriminator reach balance at the moment, and the whole training process is finished.
Based on the principle of generating a countermeasure network, in the embodiment of the present invention, the step S140 of obtaining a privacy-preserving image of a fidelity face based on GAN includes an image generating step S141 and a discriminating step S142.
The image generation step S141 can be realized by a generator (generation module) for generating a countermeasure network, in which the target face image F is usedtAnd source face FsThe multi-scale identity characteristics are used as input, a Unet neural network structure is used for generating (synthesizing) a fidelity face image, the fidelity face image has the identity of a source face and the attribute of a target face, so that the target face is replaced by the fidelity face, the target face can be protected, and the generated fidelity face can also be called as a fidelity face privacy protection image.
The Unet neural network is a semantic segmentation network obtained by improving on the basis of a Full Convolution Network (FCN), and the network structure of the Unet neural network comprises two symmetrical parts: the former part network uses convolution and pooling downsampling of 3x3, and can grasp the context information (namely the relation between pixels) in the image; the latter part of the network is substantially symmetrical to the front, using 3x3 convolution and upsampling for output image segmentation purposes. Feature fusion is also used in the Unet neural network, and features of a down-sampling part in the front and features of an up-sampling part in the back are fused to obtain more accurate context information, so that a better segmentation effect is achieved. The Unet neural network model is simple, light and compact, and can reduce the complexity of face generation compared with the existing model for face replacement.
The attributes of the target face include at least one of the following attributes: pose, expression, skin tone, lighting, and makeup of the target face.
In the determination step S142, the fidelity face privacy protection image synthesized in step S141 and the face key point wiring diagram are used as common inputs to determine the recognition result of the fidelity face privacy protection image, and the Unet neural network structure is optimized based on the recognition result.
The determination step S142 may be implemented by a determiner (determination module) that generates an antagonistic network, and in this step, the determination module may implement perception of the pose and/or expression of the target face by splicing the fidelity face privacy protection image generated in step S141 and the face key point connecting line graph as input. That is, the discrimination module will note whether the poses and/or expressions, such as the positions of the five sense organs, etc., of the synthesized face are consistent with those of the target face in addition to the authenticity of the image generated by the generation module, thereby realizing explicit supervision of the poses and expressions of the face.
After synthesizing the fidelity face privacy protection image based on a generated confrontation network (GAN), the synthesized face image may replace the target face in the target image.
In some embodiments of the present invention, in order to facilitate the replacement of the face, a face region clipping step may be further included before the step S140 is performed. The step can cut out a face area based on the face and a target face boundary box obtained by the key point detection module, and the subsequent image generation step can synthesize a face image based on the cut-out face area.
The face and key point detection module is further used for detecting the source face bounding box to cut out the face area of the source face image, so that the image generation module carries out face image synthesis based on the cut out source face area and the target face area.
In addition, in practical application, the target face image may have a certain inclination and cannot be aligned well with the source face, and at this time, in order to improve the fidelity of the synthesized face, the present invention performs face alignment on the cut target face region by using the affine transformation matrix AM before the generation step is performed.
The high-fidelity face privacy protection method based on the generation countermeasure network, disclosed by the invention, has the advantages that the generated face not only retains the attributes including the posture, the expression and the like of the target face with high fidelity, but also has the identity of the source face, the target face in a video or a picture is replaced by the face generated by the method, the realization is simple, and the privacy information of the target face can be effectively protected.
In this embodiment of the present invention, the source face identity encoding step S110 may further include: and acquiring the multi-scale identity characteristics of the source face by adopting a k-shot strategy.
The step of obtaining the multi-scale identity characteristics of the source face by adopting the k-shot strategy can comprise the following steps:
(1) k identity features are obtained based on k source face images, and the obtained k identity features are averaged to obtain the final identity feature.
The k source face images may be static source face images or source face video frames. K source images or source video frames are simultaneously input into an identity coding module to obtain k identity characteristics, and then the final identity characteristics can be obtained in an averaging mode.
(2) And obtaining the multi-scale identity characteristics of the source face based on the final identity characteristics.
Here, the multi-scale identity feature may refer to identity features of different resolution scales. According to the final identity feature, the resolution can be reduced in a grading way to obtain the multi-scale identity features with different resolutions.
The identity characteristics of the source face are obtained by adopting a k-shot strategy averaging mode, so that the accuracy and the comprehensiveness of identity coding can be further improved. The k-shot strategy can effectively avoid interference and defects caused by certain extreme conditions in a single image or a single video frame to identity extraction, and can improve the face changing effect of a static image or a video by utilizing richer face identity information.
In addition, in the embodiment of the present invention, in order to further improve the accuracy of face conversion and ensure high fidelity of the synthesized face, the present invention also restricts the pose and/or expression of the generated face while ensuring accurate identity conversion, and for this, the image generating step S141 may further include an identity migration step and a pose expression control step that employ an adaptive normalization technique.
IN the identity migration step, not only the Instance Normalization (IN) processing is performed on the source face image, but also the Layer Normalization (LN) processing related to the number of feature channels is performed on the source face image, and the multi-scale identity features based on the source face are input together for learning, so as to obtain the corrected source face identity feature output. The module of the generation module of the present invention that implements the identity migration step may be referred to as an identity migration module.
In the gesture expression control step, not only the example normalization processing is carried out on the target face image, but also the layer normalization processing related to the number of the characteristic channels is carried out on the target face image, and the target face image is input together and is learned based on the face key point connecting line graph, so that the target face characteristic corrected based on the gesture and the expression of the face is obtained. The module for realizing the gesture expression control step in the generation module can be called as a gesture expression control module.
In the identity migration module and the gesture expression control module, making M belong to RH×W×CAnd representing the characteristics of the input image, wherein H and W respectively represent the height and width of the characteristic diagram, C represents the number of characteristic channels, and R is a real number domain. As shown IN FIGS. 3 and 4, M is first obtained by passing M through an example normalization (IN) layer and a Layer Normalization (LN) layer, respectivelyINAnd MLN：
Figure BDA0003122726470000081
Figure BDA0003122726470000082
Figure BDA0003122726470000091
Figure BDA0003122726470000092
Figure BDA0003122726470000093
Figure BDA0003122726470000094
Figure BDA0003122726470000095
The case normalization means that the mean and variance are calculated for each channel of each case only in the (H, W) dimension, and the layer normalization means that the mean and variance are calculated for each case (each graph) in the (C, H, W) dimension.
As in the above formula, μINAnd σINExpressed are mean and standard deviation of IN, μLNAnd σLNThe mean and standard deviation of the LN are indicated,
Figure BDA0003122726470000096
and
Figure BDA0003122726470000097
respectively representing the mean and variance of each instance in the c-th channel,
Figure BDA0003122726470000098
is MINThe value of the c-th channel position (x, y),
Figure BDA0003122726470000099
is MLNThe value of the c-th channel position (x, y),
Figure BDA00031227264700000910
is the average of LN over the c-th channel. Subsequently, two outputs M are output using a learnable parameter ρINAnd MLNAre combined and madeAdaptive normalization is achieved with the parameters γ and β:
Figure BDA00031227264700000911
more specifically, for the identity migration module, the resulting output characteristics are:
Figure BDA00031227264700000912
for the gesture expression control module, the obtained output characteristics are as follows:
Figure BDA00031227264700000913
gamma for identity transfer moduleIDAnd betaIDIs learned through multi-scale identity characteristics, and the gamma of the gesture expression control modulePEAnd betaPEIs a line graph B passing through the key pointstAnd (4) learning. Fig. 3 shows an exemplary process of performing example normalization processing and layer normalization processing on input features (source face images) in the identity migration module, and then combining ID features (corresponding identity features in the source face images) and ID vectors (ID resolution scales) to obtain modified source face feature outputs, and fig. 4 shows a connecting line graph B of combining key points after performing example normalization processing and layer normalization processing on input features (target face images) in the gesture expression control moduletAn exemplary process for deriving a modified target face feature output is illustrated in figures 3 and 4,
Figure BDA00031227264700000914
represents the addition of the related elements;
Figure BDA00031227264700000915
which means that the elements involved are multiplied by each other,
Figure BDA00031227264700000916
representation mergingOperation, GAP is global average pooling, and Linear represents the connection layer.
In the embodiment of the invention, as described above, the example normalization is channel-by-channel calculation, the correlation between the channels is ignored, the layer normalization does not ignore the correlation between the channels, the identity migration module can realize accurate identity conversion by introducing the example normalization and the layer normalization simultaneously, and the pose and expression control module corrects the pose and expression of the synthesized face by using the face key point information so as to make the pose and expression of the synthesized face consistent with the target face. In the embodiment of the invention, the problem of asymmetry of the face image synthesized in the generating step can be greatly improved or eliminated by introducing layer normalization processing in the generating step.
Fig. 2 is a schematic diagram of a system module structure for implementing the high-fidelity face privacy protection method in an embodiment of the present invention. In fig. 2, each ID block corresponds to the processing of the identity migration module shown in fig. 3, ID blocks of different levels correspond to different ID resolution feature scales, each PE block corresponds to the processing of the gesture expression control module shown in fig. 4, and PE blocks of different levels correspond to the five sense organ key point connecting line graphs of different resolutions. The identity migration module is applied at a higher level in the generation module, and the gestural expression control module is applied at a lower level. It can be considered that identity features contain more abstract and richer semantic information and therefore should be used more at a higher feature level (low resolution feature map) to shape identities; and the face key points and the connecting line graph thereof should be used at a lower feature level (a high-resolution feature graph) to correct the outline details.
As shown in FIG. 2, the discrimination module may perceive the pose and/or expression by connecting the face key points to a line graph BtAnd the face image synthesized by the generation module is used as common input to judge whether the synthesized image is a synthesized image or an original image, and the Unet neural network structure is optimized based on the recognition result to weakly monitor the posture and expression of the generated face.
Further, in the embodiment of the present invention, the determination module may adopt a multi-scale structural design. The synthesized face images with different scales (different resolutions) are input into a plurality of discrimination modules with the same network structure, and the Unet neural network structure of the generation module is optimized based on each discrimination result. The discrimination module for inputting the large-scale image has a larger receptive field and a more global visual angle, so that the generation module can be guided to generate an image with global consistency; on the other hand, the discrimination module inputting the small-scale image can be used for guiding the generation module to generate better image details.
Furthermore, the discrimination module can also utilize the multi-scale features to assist in retaining attributes of the target face, such as skin color, illumination, makeup, occlusion and the like.
More specifically, the discrimination module utilizes a perceptual loss function to preserve the attributes of the target face without affecting identity conversion, such as: skin color, lighting, makeup, and shading. The perception loss function comprises an occlusion perception part and a style perception part, and attributes such as skin color, illumination, makeup, occlusion and the like of the target face are retained in an auxiliary mode by utilizing the multi-scale features of the discrimination module.
In an embodiment of the present invention, the perceptual loss function is defined as:
Figure BDA0003122726470000101
Figure BDA0003122726470000102
Figure BDA0003122726470000111
wherein H and W represent the height and width of the feature map, respectively, C represents the number of feature channels, and FsAnd YsAre respectively FtAnd Y in s layers, m representing the number of layers, Cs、HsAnd WsFG representing the number, width and height of channels of the s-layer, respectivelysAnd YGsIs FsAnd YsA Gram (Gram) matrix of (a),
Figure BDA0003122726470000112
represents FtThe value at the ith row and kth column of the gram matrix of features at the s-th layer,
Figure BDA0003122726470000113
represents FtThe value of the characteristic of the gram matrix at the s-th layer at the j-th row and k-th column,
Figure BDA0003122726470000114
the value of the ith row and the kth column of the gram matrix representing the feature of Y at the s-th layer,
Figure BDA0003122726470000115
the value of the jth row and kth column of the gram matrix representing the characteristics of Y at the s-th layer.
In addition, in the preferred embodiment of the present invention, the discrimination module further uses a countermeasure loss function to implement a "binary minuscule gaming", where the countermeasure loss function is defined as:
Figure BDA0003122726470000116
Figure BDA0003122726470000117
wherein G represents the result of the generator, DiI-th discrimination result representing a discriminator, n representing the number of discrimination results, FtRepresenting the image of the target face, FsRepresenting the source face image, BtRepresents a five sense organs key point connecting line graph, and E represents the expectation value.
In addition, the discrimination module also accurately converts the generated face identity into a source face identity using an identity loss function, the identity loss function formula being:
Figure BDA0003122726470000118
wherein cos (·, ·) represents the cosine similarity of two identity features; IDEnc (Y)) Identity features representing the synthetic face Y generated by the generating module, IDEnc (F)s) Representing the identity of the source face, a synthetic face Y and a source face FsThe identity features of (1) are extracted by using an identity encoder (identity encoding module).
In addition, the discrimination module also utilizes a reconstruction loss function to help preserve hair and a portion of facial occlusions of the target face. The reconstruction loss function is defined as the square of the euclidean distance of the target face from the generated face:
Figure BDA0003122726470000119
wherein the content of the first and second substances,
Figure BDA00031227264700001110
for reconstructing the loss value, λ1The weight value may be set to 0.1.
In the embodiment of the invention, the discrimination module can keep various attributes of the target face without influencing identity conversion by introducing various loss functions such as countermeasure loss, identity loss, reconstruction loss, perception loss and the like.
In addition, in order to eliminate an obvious boundary between an outer edge and an inner edge of a synthesized face after face replacement, the method of the embodiment of the invention further comprises an image fusion step, namely, a mirror image Sigmoid mask is further utilized to adjust the synthesized fidelity face privacy protection image, so that the outer edge pixels of the generated face mainly use the pixels of the target face, and the pixels of the generated face are reserved inside the generated face.
More specifically, the mask formula employed is as follows:
Figure BDA0003122726470000121
where x denotes the abscissa of the synthesized face pixel, y denotes the ordinate of the synthesized face pixel, mid denotes the half size of the image, α is used to control the slope of the function, and as an example, α may be set to 0.08, and θ denotes a variation factor, and as an example, θ may be set to 0.7, but the present invention is not limited thereto.
After the generated face pixels are processed by using the mirror image Sigmoid mask, the generated face pixels can be directly attached to the corresponding positions of the target image or the video frame, or the synthesized face pixels can be directly attached to the corresponding positions of the target image or the video frame after being transformed by using the affine transformation matrix AM of the target face calculated in the prior art. The visual effect of the masking process is shown in fig. 6. Fig. 6a, 6b, 6c show a mirror Sigmoid mask, a 2D visualization thereof and a 3D visualization thereof, respectively. After the images are fused, the outer edge and the inner edge of the synthesized face are in smooth transition, and no obvious boundary appears.
Fig. 5 is a schematic flow chart of a high-fidelity face privacy protection method in another embodiment of the present invention, as shown in fig. 5, the method in this embodiment includes the following steps:
step S1, the face and key point detection module obtains a face bounding box and face key points of a target image (a still image or a video frame), cuts out a target face region, obtains target face position information and target face key point information, and further calculates an affine transformation matrix AM using an OpenCV function to align the target face.
Step S2, respectively connecting the eye, mouth and nose bridge points in the key points of the target face to obtain a target face key point connecting line graph Bt。
And step S3, carrying out face identity feature coding on the identified source face image to acquire the multi-scale identity feature of the source face.
The multi-scale identity features of the source face can be extracted from k source images or video frames through a k-shot strategy, for example.
And step S4, sending the multi-scale identity characteristics of the target face, the key point connecting line graph and the source face into a generating module to obtain a synthesized face image.
And step S5, the synthesized face image and the face key point connecting line graph are used as common input to judge whether the fidelity face privacy protection image is a synthesized image or an original image, and the Unet neural network structure is optimized based on the recognition result.
This step is present only during the training phase and not during the actual application phase.
And step S6, adjusting the pixel value of the generated face by using a mirror image sigmoid mask, and fusing the synthesized face with the target video frame, so that the outer edge pixels of the generated face mainly use the pixels of the target face, and the inner part of the generated face is reserved.
In step S7, the affine transformation matrix AM of the target face calculated in step S1 may be used to directly paste the processed generated face back to the corresponding position of the target image or video frame.
Fig. 8 shows a schematic flow chart of an implementation of the corresponding method. As shown in fig. 8, a target face attribute feature and a face key point connecting line graph are obtained based on a target frame, and face pixel alignment is performed by calculating an affine transformation matrix AM. And then obtaining final face identity characteristics by averaging based on a plurality of source face video frames, synthesizing a high-fidelity face by using a generated confrontation network based on the obtained face identity characteristics, target face attribute characteristics and face key point connecting line graphs, and further improving the continuity and consistency of the face-changed images by image fusion.
The high-fidelity face privacy protection method based on the generation countermeasure network can well synthesize the whole face area, hair and background, so that the face area, the hair and the background can be easily fused into a target frame.
In order to verify the effect of the invention, the face changing effect of the invention and other open-source face synthesis and replacement methods is compared on a FaceForensics + + data set. The effect pairs are shown in fig. 7. Both FaceSwap and deepfaces suffer from blending inconsistency (blending inconsistency) because they follow a strategy of first synthesizing an internal face region and then blending it into a target face. However, the present invention can synthesize the whole face region, hair, and background well, and thus can be easily fused into a target frame. In addition, existing methods do not preserve attributes (e.g., pose, expression, skin tone, lighting, makeup, etc.) in the target face. Furthermore, generating inconsistencies in the pose and expression of the face and the target face reduces temporal continuity of the results, while deviations in light, skin tone and makeup reduce temporal stability and realism of the results. The invention can well reserve the attribute of the human face so as to obtain higher fidelity. Table 1 shows the quantitative comparison of the present invention with other state-of-the-art methods (Nirkin, IP-GAN, faceShifter, DeepFakes, faceSwap and FS-GAN). The following general criteria were used for quantitative comparison: identity retrieval (id), Domain Invariant Perceptual Distance (DIPD), pose distance (pos), expression distance (exp), Structural Similarity (SSIM), mean endpoint error (AEE), and Flow Warping Error (FWE).
Table 1. quantitative comparison with other face exchange methods (heel @ indicates the better the number, and ↓ indicates the better the smaller the number).
Figure BDA0003122726470000131
Figure BDA0003122726470000141
As is apparent from fig. 7 and table 1, the present invention is superior to other existing methods in terms of identity conversion, attribute maintenance, authenticity, and time continuity and stability.
In accordance with the foregoing method, there is also provided a high fidelity face privacy protection system based on generation of a countermeasure network, the system comprising a processor and a memory, the memory having stored therein computer instructions for execution by the processor, the system implementing the steps of the foregoing method when the computer instructions are executed by the processor.
As another embodiment, the high fidelity face privacy protection system based on generation of a countermeasure network of the present invention may comprise:
a target face input unit, wherein the target face image information is input in the form of video or still picture;
a source face input unit, wherein the source face image information is input in the form of video or still picture;
the high-fidelity face privacy protection unit is used for inputting target face information and source face information, synthesizing a face according to a high-fidelity face privacy protection method based on a generation countermeasure network, keeping all attributes of the target face with high fidelity and having the identity of the source face, and replacing the target face in a video or a picture by using the generated face so as to protect the privacy information of the target face. The high-fidelity face privacy protection unit may comprise:
the source face identity coding module is used for executing the source face identity coding step;
the face and key point detection module is used for executing the face and key point detection step;
the key point wiring diagram acquisition module is used for executing the key point wiring diagram acquisition step;
an image generation module for performing the image generation step;
and the judging module is used for executing the judging step.
In addition, the high-fidelity face privacy protection unit can further comprise a fusion module used for executing a fusion step, adjusting the generated fidelity face privacy protection image by the mirror image Sigmoid mask, and fusing the synthesized face with the target video frame.
Software implementing embodiments of the present invention may be disposed in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of tangible storage medium known in the art.
Accordingly, the present disclosure also relates to a storage medium as above, on which a computer program code may be stored, which when executed may implement various embodiments of the method of the present invention.
It should be noted that the exemplary embodiments of the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be performed in an order different from the order in the embodiments, or may be performed simultaneously.
Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments in the present invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A high-fidelity face privacy protection method based on a generation countermeasure network is characterized by comprising the following steps:
a source face identity coding step, which is used for carrying out face recognition on a source face image and acquiring the multi-scale identity characteristics of a source face;
a face and key point detection step, which is used for at least obtaining a target face boundary box and face key point information based on an input target face image;
a key point connecting line graph obtaining step, which is used for extracting specific key points related to gestures and/or expressions in the obtained face key points and obtaining the face key point connecting line graph based on the extracted specific key points;
obtaining a fidelity face privacy protection image based on a generation countermeasure network GAN, comprising:
an image generation step, which is used for synthesizing a fidelity face privacy protection image with the identity of the source face and the attribute of the target face by using a Unet neural network structure by taking the multi-scale identity characteristics of the target face image and the source face as input;
and a judging step, namely judging the recognition result of the fidelity face privacy protection image by taking the fidelity face privacy protection image and the face key point connecting line graph as common input, and optimizing the Unet neural network structure based on the recognition result.
2. The method of claim 1, wherein the identity encoding step comprises: acquiring multi-scale identity characteristics of a source face by adopting a k-shot strategy; the method for acquiring the multi-scale identity characteristics of the source face by adopting the k-shot strategy comprises the following steps:
acquiring k identity characteristics based on k source face images, and averaging the acquired k identity characteristics to obtain final identity characteristics;
and obtaining the multi-scale identity characteristics of the source face based on the final identity characteristics.
3. The method of claim 1, wherein the image generating step comprises:
an identity migration step, namely obtaining a first self-adaptive output characteristic of a source face image through example normalization processing, layer normalization processing and deep learning based on multi-scale identity characteristics of the source face image;
a gesture expression control step, namely obtaining a second self-adaptive output characteristic of the target face image based on example normalization processing, layer normalization processing and deep learning based on a face key point connecting line graph of the target face image;
and generating a fidelity face privacy protection image with the identity of the source face and the attributes of the target person by utilizing a Unet neural network structure based on the first adaptive output characteristics and the first adaptive output characteristics.
4. The method of claim 3,
the first adaptive output characteristic conforms to the following equation:
Figure FDA0003122726460000021
the second adaptive output characteristic conforms to the following equation:
Figure FDA0003122726460000022
wherein the content of the first and second substances,
Figure FDA0003122726460000023
Figure FDA0003122726460000024
M∈RH×W×C；
Figure FDA0003122726460000025
Figure FDA0003122726460000026
Figure FDA0003122726460000027
Figure FDA0003122726460000028
Figure FDA0003122726460000029
wherein M is the characteristic of the input image, and R represents a real number domain; h and W represent the height and width of the characteristic diagram respectively, and C represents the number of characteristic channels; mu.sINAnd σINIs for exampleThe normalized mean and standard deviation of the mean and standard deviation,
Figure FDA00031227264600000210
and
Figure FDA00031227264600000211
respectively represent the mean and variance, μ, of each instance in the c-th channelLNAnd σLNIs the mean and standard deviation for layer normalization; gamma rayIDAnd betaIDIs a parameter learned by multi-scale identity features; gamma rayPEAnd betaPEIs a line graph B passing through the key pointstThe parameters of the parameters that are learned are,
Figure FDA00031227264600000212
is MINThe value of the c-th channel position (x, y),
Figure FDA00031227264600000213
is MLNThe value of the c-th channel position (x, y),
Figure FDA00031227264600000214
is the average of LN over the c-th channel.
5. The method of claim 1, further comprising:
determining a face area based on the obtained target face bounding box, and calculating an affine transformation matrix to align the face;
a fusion step, namely adjusting the generated fidelity face privacy protection image by using a mirror image Sigmoid mask, and fusing the synthesized fidelity face privacy protection image with a target image, so that the outer edge pixels of the generated face mainly use the pixels of the target face, and the inner part of the generated face keeps the pixels of the generated face;
and pasting the adjusted fidelity face privacy protection image back to the corresponding position of the target image or the video by utilizing the affine transformation matrix.
6. The method of claim 1, further comprising:
the discrimination module optimizes the Unet neural network using a countering loss function, a perceptual loss function, an identity loss function, and a reconstruction loss function.
7. The method of claim 6,
the opposition loss function satisfies:
Figure FDA0003122726460000031
Figure FDA0003122726460000032
the identity loss function is expressed as:
Figure FDA0003122726460000033
the reconstruction loss function is expressed as:
Figure FDA0003122726460000034
the perceptual loss function is expressed as:
Figure FDA0003122726460000035
Figure FDA0003122726460000036
Figure FDA0003122726460000037
Figure FDA0003122726460000038
wherein G represents the result of the generator, DiI-th discrimination result representing the discriminator, n representing the number of discrimination results, FtRepresenting the image of the target face, FsRepresenting the source face image, BtRepresenting a connecting line graph of key points of the human face,
Figure FDA0003122726460000039
representing the desired value, Y representing the generated face image, lambda1Representing weight values, H and W respectively representing height and width of the feature map, C representing number of feature channels, FsAnd YsAre respectively FtAnd Y in s layers, m representing the number of layers, Cs、HsAnd WsFG representing the number, width and height of channels of the s-layer, respectivelysAnd YGsIs FsAnd YsThe matrix of the gram of (a) is,
Figure FDA00031227264600000310
represents FtThe value at the ith row and kth column of the gram matrix of features at the s-th layer,
Figure FDA00031227264600000311
represents FtThe value of the characteristic of the gram matrix at the s-th layer at the j-th row and k-th column,
Figure FDA00031227264600000312
the value of the ith row and the kth column of the gram matrix representing the feature of Y at the s-th layer,
Figure FDA00031227264600000313
the value of the jth row and kth column of the gram matrix representing the characteristics of Y at the s-th layer.
8. The method of claim 1, wherein the source or target facial image comprises a still image or a moving image, the moving image comprising a video frame;
the attributes of the target face comprise at least one of the following attributes: pose, expression, skin color, lighting and makeup of the target face;
the method further comprises the following steps: and inputting the synthesized fidelity human face privacy protection images with different resolutions into a plurality of discrimination modules with the same network structure, and optimizing the Unet neural network structure of the generation module based on the discrimination result.
9. A high fidelity face privacy protection system based on a generative confrontation network, the system comprising a processor and a memory, wherein the memory has stored therein computer instructions for execution by the processor, the apparatus implementing the steps of the method of any one of claims 1 to 8 when the computer instructions are executed by the processor.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.
CN202110681374.4A 2021-06-18 2021-06-18 High-fidelity face privacy protection method and system based on generation countermeasure network Pending CN113343878A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110681374.4A CN113343878A (en) 2021-06-18 2021-06-18 High-fidelity face privacy protection method and system based on generation countermeasure network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110681374.4A CN113343878A (en) 2021-06-18 2021-06-18 High-fidelity face privacy protection method and system based on generation countermeasure network

Publications (1)

Publication Number Publication Date
CN113343878A true CN113343878A (en) 2021-09-03

Family

ID=77477793

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110681374.4A Pending CN113343878A (en) 2021-06-18 2021-06-18 High-fidelity face privacy protection method and system based on generation countermeasure network

Country Status (1)

Country Link
CN (1) CN113343878A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113919998A (en) * 2021-10-14 2022-01-11 天翼数字生活科技有限公司 Image anonymization method based on semantic and attitude map guidance
CN114581612A (en) * 2022-04-28 2022-06-03 南京理工大学 High-fidelity face reproduction method represented by mixed actions
CN114842003A (en) * 2022-07-04 2022-08-02 杭州健培科技有限公司 Medical image follow-up target pairing method, device and application
CN115272534A (en) * 2022-07-29 2022-11-01 中国电信股份有限公司 Face image protection method, protection device, electronic equipment and readable storage medium
WO2023040679A1 (en) * 2021-09-16 2023-03-23 百果园技术(新加坡)有限公司 Fusion method and apparatus for facial images, and device and storage medium
CN117196937A (en) * 2023-09-08 2023-12-08 天翼爱音乐文化科技有限公司 Video face changing method, device and storage medium based on face recognition model

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190286950A1 (en) * 2018-03-16 2019-09-19 Ebay Inc. Generating a digital image using a generative adversarial network
CN111242837A (en) * 2020-01-03 2020-06-05 杭州电子科技大学 Face anonymous privacy protection method based on generation of countermeasure network
CN111553235A (en) * 2020-04-22 2020-08-18 支付宝(杭州)信息技术有限公司 Network training method for protecting privacy, identity recognition method and device
CN111815509A (en) * 2020-09-02 2020-10-23 北京邮电大学 Image style conversion and model training method and device
CN112084962A (en) * 2020-09-11 2020-12-15 贵州大学 Face privacy protection method based on generation type countermeasure network
CN112215927A (en) * 2020-09-18 2021-01-12 腾讯科技（深圳）有限公司 Method, device, equipment and medium for synthesizing face video

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190286950A1 (en) * 2018-03-16 2019-09-19 Ebay Inc. Generating a digital image using a generative adversarial network
CN111242837A (en) * 2020-01-03 2020-06-05 杭州电子科技大学 Face anonymous privacy protection method based on generation of countermeasure network
CN111553235A (en) * 2020-04-22 2020-08-18 支付宝(杭州)信息技术有限公司 Network training method for protecting privacy, identity recognition method and device
CN111815509A (en) * 2020-09-02 2020-10-23 北京邮电大学 Image style conversion and model training method and device
CN112084962A (en) * 2020-09-11 2020-12-15 贵州大学 Face privacy protection method based on generation type countermeasure network
CN112215927A (en) * 2020-09-18 2021-01-12 腾讯科技（深圳）有限公司 Method, device, equipment and medium for synthesizing face video

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LONGHAO ZHANG 等: "AP-GAN: Improving Attribute Preservation in Video Face Swapping", 《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY(EARLY ACCESS)》 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023040679A1 (en) * 2021-09-16 2023-03-23 百果园技术(新加坡)有限公司 Fusion method and apparatus for facial images, and device and storage medium
CN113919998A (en) * 2021-10-14 2022-01-11 天翼数字生活科技有限公司 Image anonymization method based on semantic and attitude map guidance
WO2023060918A1 (en) * 2021-10-14 2023-04-20 天翼数字生活科技有限公司 Image anonymization method based on guidance of semantic and pose graphs
CN113919998B (en) * 2021-10-14 2024-05-14 天翼数字生活科技有限公司 Picture anonymizing method based on semantic and gesture graph guidance
CN114581612A (en) * 2022-04-28 2022-06-03 南京理工大学 High-fidelity face reproduction method represented by mixed actions
CN114581612B (en) * 2022-04-28 2022-08-02 南京理工大学 High-fidelity face reproduction method represented by mixed actions
CN114842003A (en) * 2022-07-04 2022-08-02 杭州健培科技有限公司 Medical image follow-up target pairing method, device and application
CN114842003B (en) * 2022-07-04 2022-11-01 杭州健培科技有限公司 Medical image follow-up target pairing method, device and application
CN115272534A (en) * 2022-07-29 2022-11-01 中国电信股份有限公司 Face image protection method, protection device, electronic equipment and readable storage medium
CN115272534B (en) * 2022-07-29 2024-02-02 中国电信股份有限公司 Face image protection method, protection device, electronic equipment and readable storage medium
CN117196937A (en) * 2023-09-08 2023-12-08 天翼爱音乐文化科技有限公司 Video face changing method, device and storage medium based on face recognition model
CN117196937B (en) * 2023-09-08 2024-05-14 天翼爱音乐文化科技有限公司 Video face changing method, device and storage medium based on face recognition model

Similar Documents

Publication Publication Date Title
CN111489287B (en) Image conversion method, device, computer equipment and storage medium
CN113343878A (en) High-fidelity face privacy protection method and system based on generation countermeasure network
JP7476428B2 (en) Image line of sight correction method, device, electronic device, computer-readable storage medium, and computer program
CN110363116B (en) Irregular human face correction method, system and medium based on GLD-GAN
He et al. Photo-realistic monocular gaze redirection using generative adversarial networks
WO2022095721A1 (en) Parameter estimation model training method and apparatus, and device and storage medium
WO2022156626A1 (en) Image sight correction method and apparatus, electronic device, computer-readable storage medium, and computer program product
CN112733795B (en) Method, device and equipment for correcting sight of face image and storage medium
CN110381268B (en) Method, device, storage medium and electronic equipment for generating video
CN111445410A (en) Texture enhancement method, device and equipment based on texture image and storage medium
WO2019032604A1 (en) Adjusting a digital representation of a head region
CN113807265B (en) Diversified human face image synthesis method and system
CN111931908B (en) Face image automatic generation method based on face contour
CN115689869A (en) Video makeup migration method and system
CN112101320A (en) Model training method, image generation method, device, equipment and storage medium
CN117095128A (en) Priori-free multi-view human body clothes editing method
GB2612881A (en) Techniques for re-aging faces in images and video frames
CN113887329A (en) Head posture positioning and detecting method and application and system thereof
WO2024104144A1 (en) Image synthesis method and apparatus, storage medium, and electrical device
CN111275778B (en) Face simple drawing generation method and device
CN112825188A (en) Occlusion face completion algorithm for generating confrontation network based on deep convolution
US20230319223A1 (en) Method and system for deep learning based face swapping with multiple encoders
CN114120391B (en) Multi-pose face recognition system and method thereof
CN115393471A (en) Image processing method and device and electronic equipment
Koumparoulis et al. Audio-assisted image inpainting for talking faces

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20210903

WD01 Invention patent application deemed withdrawn after publication