CN116452453A - CNN-based face contour automatic smoothing method, system and storage medium - Google Patents

CNN-based face contour automatic smoothing method, system and storage medium Download PDF

Info

Publication number
CN116452453A
CN116452453A CN202310429189.5A CN202310429189A CN116452453A CN 116452453 A CN116452453 A CN 116452453A CN 202310429189 A CN202310429189 A CN 202310429189A CN 116452453 A CN116452453 A CN 116452453A
Authority
CN
China
Prior art keywords
face
image
smoothing
contour
cnn
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310429189.5A
Other languages
Chinese (zh)
Inventor
尚伟艺
刘洛麒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen Meitu Technology Co Ltd
Original Assignee
Xiamen Meitu Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen Meitu Technology Co Ltd filed Critical Xiamen Meitu Technology Co Ltd
Priority to CN202310429189.5A priority Critical patent/CN116452453A/en
Publication of CN116452453A publication Critical patent/CN116452453A/en
Pending legal-status Critical Current

Links

Classifications

    • G06T5/70—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06T5/77—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The invention discloses a face contour automatic smoothing method, a system and a storage medium based on CNN, wherein the method comprises the following steps: obtaining a second face image of the image to be smoothed and a face contour mask image, multiplying the second face image and the face contour mask image, calculating to obtain a region of interest of a face contour part, inputting the region of interest into a face contour smoothing generating network obtained through pre-training, and generating a face contour smoothing light flow diagram; the face contour smoothing light flow graph is amplified, then acting on the first face image to obtain a face contour pre-smoothing result; and carrying out affine transformation processing on the facial contour smoothing result to obtain the facial contour smoothing result. According to the invention, the position of the face point is not required to be relied on, and the region of interest of the face contour part obtained through calculation is only required to be input into the face contour smoothing generation network obtained through pre-training, so that the face contours of different scenes and different defects can be intelligently and accurately smoothed, and the automatic contour adjustment requirement of a user is met.

Description

CNN-based face contour automatic smoothing method, system and storage medium
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a CNN-based automatic facial contour smoothing method, and an image processing apparatus, a device, and a computer-readable storage medium using the same.
Background
Colloquially speaking, "the beauty is not on the skin at the bones", and what is said is that the beauty is not on the five sense organs but on the skeletal layout of the facial contours. The facial contours with excellent visual perception are often determined by contour lines. The outline of the face is what we often say, each person's face is different, and the attractive face is not completely unified. The outline of the face can bring visual feeling to the vision of people, and smooth arcs can give the face a sense of convergence, so that the face is compact and full, young and active. The visual perception of the facial contour is mainly influenced by frontal bones, temples, eyebrow bows, zygomatic bows, mandibular angles and chin, and the outer contour determines whether the overall shape of the face is smooth or not, and if the overall connection is natural, the advanced sense of the face is naturally displayed.
From the perspective of mass aesthetic, whether the face is a goose egg face or a melon seed face, or a round face or a square face, the outline and the line of the face are smooth, and the basic condition of the aesthetic feeling can be revealed. However, in reality, not every person's face is smooth, and many beauty-seeking persons face various face problems such as protruding cheekbones, recessed faces, etc., and improvement of the aesthetic feeling of the face contour is urgently required. Therefore, the user also expects smooth and even facial lines when taking pictures, so as to highlight the state of younger and full faces.
However, the beautifying software on the market generally adopts a traditional image algorithm, and analyzes the defects of the facial contours through the positions of the face points according to the detection result of the face points, so that the faces of different parts are adjusted with different forces. Therefore, the error detection of the individual face points can cause that the face shape after photographing by the user can not realize a smooth effect and is even more uneven; and the user needs to manually adjust the contour to achieve the effect of contour smoothing. Especially in extreme cases (such as the case of face shielding and spectacle perspective facial deformity), the corresponding position is more severely distorted after the facial contour is adjusted according to the face points, and the user is required to manually adjust the facial contour.
Therefore, the facial contour smoothing method in the prior art cannot meet the automatic contour adjustment requirement of the user, and the adjustment effect is poor.
Disclosure of Invention
The invention mainly aims to provide a CNN-based face contour automatic smoothing method, a CNN-based face contour automatic smoothing system and a storage medium, and aims to solve the technical problems that an existing CNN-based face contour automatic smoothing method cannot meet automatic contour adjustment requirements of users and adjustment effects are poor.
To achieve the above object, the present invention provides a CNN-based face contour automatic smoothing method, which includes the steps of: acquiring a second face image of the image to be smoothed and a face outline mask image, and performing multiplication operation on the second face image and the face outline mask image to obtain a region of interest of a face outline part through calculation; inputting a region of interest of a face contour part as an input image into a face contour smoothing generation network obtained by pre-training so as to generate a face contour smoothing light flow graph; amplifying the smooth optical flow diagram of the face outline to obtain an amplified optical flow diagram; the amplified optical flow diagram acts on the first face image to obtain a face contour pre-smoothing result; and carrying out affine transformation processing on the facial contour smoothing result, and recovering to the same size as the image to be smoothed to obtain the facial contour smoothing result.
Optionally, acquiring a second face image and a face contour mask map of the image to be smoothed specifically includes the following steps: obtaining an image to be smoothed, carrying out face detection and face alignment treatment on the image to be smoothed to obtain a first face point set P of the image to be smoothed, calculating an external rectangle of the first face point set P, and expanding the external rectangle to obtain a cutting rectangle of a face; carrying out affine transformation processing on the cut rectangle of the face to obtain a cut rectangle affine transformation matrix of the face, cutting out a face image to obtain a first face image F, and simultaneously converting a first face point set P into coordinates of the first face image F to obtain a second face point set FP; based on the second face point set FP, a face image is cut out, and scaled to a preset size, so that a second face image fi and a third face point set FP are obtained; and obtaining a facial contour mask map fm based on the facial contour points in the third facial point set fp.
Optionally, the face contour smoothing generates training input images and target output images in the network training stage by the following acquisition modes: collecting an original data set, and acquiring a second face image fi and a third face point fp of the original data set; performing offset processing on the positions of certain points in the third face point set fp, and obtaining a third face image fi' with unsmooth face contours through a triangular mesh-based remapping method; taking the third face image fi' as a training input image of the face contour smoothing generating network; according to the third face image fi' and the second face image fi, calculating to obtain a target light flow graph f through an optical flow algorithm target And smoothing it as a face contour to generate a target output image of the network.
Optionally, an occlusion data gain is added in a training stage of the face contour smoothing generation network, specifically, adding an occlusion object to the training input image and the target output image.
Optionally, the face contour smoothing light flow graph is subjected to amplification treatment, specifically, the face contour smoothing light flow graph is amplified to an original resolution, and the original resolution is consistent with the resolution of the image to be smoothed.
Optionally, the face contour smoothing generating network adopts a coding-decoding network structure, and the network connects the feature map of the encoder and the feature map of the corresponding size of the decoder by using a connection mode of combining the feature maps so as to multiplex the feature map of the encoder and extract the bottom layer feature information.
Optionally, the face contour smoothing generating network adopts a learning mode of predicting the light flow diagram, and finally the network outputs a 3-channel light flow diagram, wherein all third channels are assigned 255, and the first channel and the second channel respectively represent an x-direction offset value and a y-direction offset value of the pixel point at the position.
Optionally, face contour smoothing generates a total loss L for the network training phase Total ＝αL 1 +βL Perc ；
Wherein alpha and beta are L respectively 1 Loss and L Perc Lose corresponding weight, L 1 Loss and L Perc Loss ofThe corresponding calculation formula is as follows;
wherein W is the width of the preset size, H is the height of the preset size,different loss map weight factors representing different positions, < ->To output an image for a target, G (fi input *fm) x,y An output image representing a face contour smoothing generation network;
wherein phi is j A feature map representing the output of the last convolutional layer through the jth module of the VGG16 network.
Corresponding to the CNN-based face contour automatic smoothing method, the present invention provides a CNN-based face contour automatic smoothing system, which includes: the computing module is used for acquiring a second face image of the image to be smoothed and a face outline mask image, multiplying the second face image and the face outline mask image, and computing to obtain a region of interest of a face outline part; the facial contour smoothing generation network module is used for acquiring a region of interest of a facial contour part as an input image and inputting the region of interest into a facial contour smoothing generation network obtained through pre-training so as to generate a facial contour smoothing light flow graph; the light flow diagram recovery module is used for amplifying the smooth light flow diagram of the face outline to obtain an amplified light flow diagram; the face contour pre-smoothing module is used for acting the amplified optical flow image on the first face image to obtain a face contour pre-smoothing result; and the affine transformation module is used for carrying out affine transformation processing on the facial contour smoothing result, recovering to the same size as the image to be smoothed, and obtaining the facial contour smoothing result.
In addition, to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a CNN-based face contour automatic smoothing program which, when executed by a processor, implements the steps of the CNN-based face contour automatic smoothing method as described above.
The beneficial effects of the invention are as follows:
(1) Compared with the prior art, the algorithm of the traditional image needs to predict the face shape defect through the accurate face point position and then carry out the beautifying operation; according to the invention, the position of the face point is not required to be relied on, and the region of interest of the face contour part obtained through calculation is only required to be input into the face contour smoothing generation network obtained through pre-training, so that the face contours of different scenes and different defects can be intelligently and accurately smoothed, and the automatic contour adjustment requirement of a user is met;
(2) According to the invention, the occlusion object is added to the training input image and the target output image, so that the robustness of the model is improved, and a natural and accurate face contour smoothing effect can be achieved under extreme conditions, such as the condition that the face is occluded and the face is deformed under the perspective of glasses;
(3) The invention can directly act on the low-resolution optical flow prediction result on the high-resolution image by learning the optical flow diagram, thereby reducing the operation amount and ensuring the definition of the output face contour smoothing result.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:
fig. 1 is a flow chart diagram of the CNN-based face contour automatic smoothing method of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, the CNN-based face contour automatic smoothing method of the present invention includes the steps of: acquiring a second face image of the image to be smoothed and a face outline mask image, and performing multiplication operation on the second face image and the face outline mask image to obtain a region of interest of a face outline part through calculation; inputting a region of interest of a face contour part as an input image into a face contour smoothing generation network obtained by pre-training so as to generate a face contour smoothing light flow graph; amplifying the smooth optical flow diagram of the face outline to obtain an amplified optical flow diagram; the amplified optical flow diagram acts on the first face image to obtain a face contour pre-smoothing result; and carrying out affine transformation processing on the facial contour smoothing result, and recovering to the same size as the image to be smoothed to obtain the facial contour smoothing result.
Compared with the prior art, the algorithm of the traditional image needs to predict the face shape defect through the accurate face point position and then carry out the beautifying operation; according to the invention, the position of the face point is not required to be relied on, and the region of interest of the face contour part obtained through calculation is only required to be input into the face contour smoothing generation network obtained through pre-training, so that the face contours of different scenes and different defects can be intelligently and accurately smoothed, and the automatic contour adjustment requirement of a user is met.
In this embodiment, a second face image and a face contour mask map of an image to be smoothed are obtained, and specifically include the following steps: obtaining an image to be smoothed, carrying out face detection and face alignment treatment on the image to be smoothed to obtain a first face point set P of the image to be smoothed, calculating an external rectangle of the first face point set P, and expanding the external rectangle to obtain a cutting rectangle of a face; carrying out affine transformation processing on the cut rectangle of the face to obtain a cut rectangle affine transformation matrix of the face, cutting out a face image to obtain a first face image F, and simultaneously converting a first face point set P into coordinates of the first face image F to obtain a second face point set FP; based on the second face point set FP, a face image is cut out, and scaled to a preset size, so that a second face image fi and a third face point set FP are obtained; and obtaining a facial contour mask map fm based on the facial contour points in the third facial point set fp.
In this embodiment, outward expansion means that the top, bottom, left and right of the circumscribed rectangle can be expanded outward to a certain extent, so that the edge of the attached image caused by direct clipping is avoided.
In this embodiment, the magnified optical flow map is directly applied to the first face image that is not scaled, ensuring image sharpness.
In this embodiment, based on a rectangular affine transformation matrix of a face, affine transformation processing is performed on a face contour smoothing result, an inverse matrix of the face contour smoothing result is calculated, and the face contour smoothing result is obtained after the face contour smoothing result is restored to the same size as an image to be smoothed.
In this embodiment, the face contour smoothing generates the training input image and the target output image in the network training stage in the following manner: collecting an original data set, and acquiring a second face image fi and a third face point fp of the original data set; performing offset processing on the positions of certain points in the third face point set fp, and obtaining a third face image fi' with unsmooth face contours through a triangular mesh-based remapping method; taking the third face image fi' as a training input image of the face contour smoothing generating network; according to the third face image fi' and the second face image fi, calculating to obtain a target light flow graph f through an optical flow algorithm target And smoothing it as a face contour to generate a target output image of the network.
The positions of certain points in the third face point set fp are offset-processed, and it is preferable that the mandible be widened, the cheekbone be protruded, and the like.
In this embodiment, the occlusion data gain is added in the training stage of the face contour smoothing generation network, specifically, an occlusion object is added to the training input image and the target output image, and the occlusion object is added, preferably, glasses mapping is performed, so that environmental interference is avoided.
According to the invention, the occlusion object is added to the training input image and the target output image, the robustness of the model is improved, and a natural and accurate face contour smoothing effect can be achieved under extreme conditions, such as the condition that the face is occluded and the face is deformed under the perspective of glasses.
In this embodiment, the face contour smoothing light flow graph is enlarged, specifically, the face contour smoothing light flow graph is enlarged to an original resolution, and the original resolution is consistent with the resolution of the image to be smoothed.
In this embodiment, the face contour smoothing generation network adopts an encoding-decoding network structure, and the network connects the feature map of the encoder and the feature map of the corresponding size of the decoder by using a connection mode of combining the feature maps, so as to multiplex the feature map of the encoder and extract the bottom layer feature information.
In this embodiment, the face contour smoothing generating network adopts a learning mode of predicting the optical flow diagram, and finally outputs a 3-channel optical flow diagram, where all third channels are assigned 255, and the first and second channels represent the x-direction offset value and the y-direction offset value of the pixel point at the position respectively.
The invention can directly act on the low-resolution optical flow prediction result on the high-resolution image by learning the optical flow diagram, thereby reducing the operation amount and ensuring the definition of the output face contour smoothing result.
In the present embodiment, face contour smoothing generates a total loss L of the network training phase Total ＝αL 1 +βL Perc ；
Wherein alpha and beta are L respectively 1 Loss and L Perc The corresponding weight is lost.
In this embodiment, to ensure the important supervision of the temple, zygomatic arch, mandibular angle areas, weights associated with different sites are added as supervision. The specific operation is as follows: obtaining a facial contour mask map fm by utilizing facial contour points in the third face point set fp, and simultaneously, according to the position of the third face point set fp, connecting a temple,The weight of the contour part of the cheek arch, the mandibular angle, and the position near the chin was set to 1.0, the weight of the other contour parts was set to 0.5, and the facial contour mask map with weight was denoted as fm. L (L) 1 The calculation formula of (2) is as follows:
wherein W is the width of the preset size, H is the height of the preset size,different loss map weight factors representing different positions, < ->For outputting an image for a target, G represents a face contour smoothing generation network, G (fi input *fm) x,y An output image representing the face contour smoothing generation network. In the present embodiment, L Perc ＝L Perc/j The specific calculation formula is as follows:
wherein phi is j A feature map representing the output of the last convolutional layer through the jth module of the VGG16 network. The VGG16 network is a common network used in calculating the perceived loss and is used for participating in calculation after outputting a characteristic diagram.
Preferably, the invention is applicable to an Adam optimization solver, the training initial learning rate is 0.0002, the iteration number is 200K, and the actual training adjustment parameters are alpha=1 and beta=0.5.
The embodiment of the invention also provides a CNN-based face contour automatic smoothing system, which comprises: the computing module is used for acquiring a second face image of the image to be smoothed and a face outline mask image, multiplying the second face image and the face outline mask image, and computing to obtain a region of interest of a face outline part; the facial contour smoothing generation network module is used for acquiring a region of interest of a facial contour part as an input image and inputting the region of interest into a facial contour smoothing generation network obtained through pre-training so as to generate a facial contour smoothing light flow graph; the light flow diagram recovery module is used for amplifying the smooth light flow diagram of the face outline to obtain an amplified light flow diagram; the face contour pre-smoothing module is used for acting the amplified optical flow image on the first face image to obtain a face contour pre-smoothing result; and the affine transformation module is used for carrying out affine transformation processing on the facial contour smoothing result, recovering to the same size as the image to be smoothed, and obtaining the facial contour smoothing result.
Embodiments of the present invention also provide a computer readable storage medium having at least one instruction stored therein, the instruction being loaded and executed by a processor to implement the CNN-based facial contour automatic smoothing method shown in fig. 1. The computer readable storage medium may be a read-only memory, a magnetic disk or optical disk, etc.
It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other. For the device embodiments, the apparatus embodiments, and the storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and reference should be made to the description of the method embodiments for relevant points.
Also, herein, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
While the foregoing description illustrates and describes the preferred embodiments of the present invention, it is to be understood that the invention is not limited to the forms disclosed herein, but is not to be construed as limited to other embodiments, but is capable of use in various other combinations, modifications and environments and is capable of changes or modifications within the scope of the inventive concept, either as described above or as a matter of skill or knowledge in the relevant art. And that modifications and variations which do not depart from the spirit and scope of the invention are intended to be within the scope of the appended claims.

Claims (10)

1. The CNN-based facial contour automatic smoothing method is characterized by comprising the following steps of:
acquiring a second face image of the image to be smoothed and a face outline mask image, and performing multiplication operation on the second face image and the face outline mask image to obtain a region of interest of a face outline part through calculation;
inputting a region of interest of a face contour part as an input image into a face contour smoothing generation network obtained by pre-training so as to generate a face contour smoothing light flow graph;
amplifying the smooth optical flow diagram of the face outline to obtain an amplified optical flow diagram;
the amplified optical flow diagram acts on the first face image to obtain a face contour pre-smoothing result;
and carrying out affine transformation processing on the facial contour smoothing result, and recovering to the same size as the image to be smoothed to obtain the facial contour smoothing result.
2. The CNN-based face contour automatic smoothing method according to claim 1, wherein: the method for acquiring the second face image and the face outline mask image of the image to be smoothed specifically comprises the following steps:
obtaining an image to be smoothed, carrying out face detection and face alignment treatment on the image to be smoothed to obtain a first face point set P of the image to be smoothed, calculating an external rectangle of the first face point set P, and expanding the external rectangle to obtain a cutting rectangle of a face;
carrying out affine transformation processing on the cut rectangle of the face to obtain a cut rectangle affine transformation matrix of the face, cutting out a face image to obtain a first face image F, and simultaneously converting a first face point set P into coordinates of the first face image F to obtain a second face point set FP;
based on the second face point set FP, a face image is cut out, and scaled to a preset size, so that a second face image fi and a third face point set FP are obtained;
and obtaining a facial contour mask map fm based on the facial contour points in the third facial point set fp.
3. The CNN-based face contour automatic smoothing method according to claim 2, wherein: the training input image and the target output image of the face contour smoothing generating network training stage are obtained as follows:
collecting an original data set, and acquiring a second face image fi and a third face point fp of the original data set;
performing offset processing on the positions of certain points in the third face point set fp, and obtaining a third face image fi' with unsmooth face contours through a triangular mesh-based remapping method;
taking the third face image fi' as a training input image of the face contour smoothing generating network;
according to the third face image fi' and the second face image fi, calculating to obtain a target light flow graph f through an optical flow algorithm target And smoothing it as a face contour to generate a target output image of the network.
4. The CNN-based face contour automatic smoothing method according to claim 1, wherein: and adding an occlusion data gain in a training stage of the facial contour smoothing generation network, and specifically adding an occlusion object to the training input image and the target output image.
5. The CNN-based face contour automatic smoothing method according to claim 1, wherein: and amplifying the facial contour smoothing light flow graph, namely amplifying the facial contour smoothing light flow graph to an original resolution, wherein the original resolution is consistent with the resolution of the image to be smoothed.
6. The CNN-based face contour automatic smoothing method according to claim 1, wherein: the face contour smoothing generating network adopts a coding-decoding network structure, and the network connects the characteristic diagram of the encoder and the characteristic diagram of the corresponding size of the decoder by using a connection mode of combining the characteristic diagrams so as to multiplex the characteristic diagram of the encoder and extract bottom characteristic information.
7. The CNN-based face contour automatic smoothing method according to claim 6, wherein: the face contour smoothing generation network adopts a learning mode of a predicted light flow graph, and finally outputs a 3-channel light flow graph, wherein all third channels are assigned 255, and the first channel and the second channel respectively represent an x-direction offset value and a y-direction offset value of the pixel point at the position.
8. The CNN-based face contour automatic smoothing method according to claim 1, wherein: face contour smoothing generates total loss L in network training phase Total ＝L 1 +L Perc ；
Wherein alpha and beta are L respectively 1 Loss and L Perc Lose corresponding weight, L 1 Loss and L Perc The calculation formula corresponding to the loss is as follows;
wherein W is the width of the preset size, H is the height of the preset size,different loss map weight factors representing different positions, < ->To output an image for a target, G (fi input *fm) x,y An output image representing a face contour smoothing generation network;
wherein phi is j A feature map representing the output of the last convolutional layer through the jth module of the VGG16 network.
9. A CNN-based automatic facial contour smoothing system, comprising:
the computing module is used for acquiring a second face image of the image to be smoothed and a face outline mask image, multiplying the second face image and the face outline mask image, and computing to obtain a region of interest of a face outline part;
the facial contour smoothing generation network module is used for acquiring a region of interest of a facial contour part as an input image and inputting the region of interest into a facial contour smoothing generation network obtained through pre-training so as to generate a facial contour smoothing light flow graph;
the light flow diagram recovery module is used for amplifying the smooth light flow diagram of the face outline to obtain an amplified light flow diagram;
the face contour pre-smoothing module is used for acting the amplified optical flow image on the first face image to obtain a face contour pre-smoothing result;
and the affine transformation module is used for carrying out affine transformation processing on the facial contour smoothing result, recovering to the same size as the image to be smoothed, and obtaining the facial contour smoothing result.
10. A computer-readable storage medium, characterized in that it has stored thereon a CNN-based facial contour automatic smoothing program, which when executed by a processor, implements the steps of the CNN-based facial contour automatic smoothing method according to any one of claims 1 to 8.
CN202310429189.5A 2023-04-20 2023-04-20 CNN-based face contour automatic smoothing method, system and storage medium Pending CN116452453A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310429189.5A CN116452453A (en) 2023-04-20 2023-04-20 CNN-based face contour automatic smoothing method, system and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310429189.5A CN116452453A (en) 2023-04-20 2023-04-20 CNN-based face contour automatic smoothing method, system and storage medium

Publications (1)

Publication Number Publication Date
CN116452453A true CN116452453A (en) 2023-07-18

Family

ID=87128398

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310429189.5A Pending CN116452453A (en) 2023-04-20 2023-04-20 CNN-based face contour automatic smoothing method, system and storage medium

Country Status (1)

Country Link
CN (1) CN116452453A (en)

Similar Documents

Publication Publication Date Title
Zhou et al. Parametric reshaping of human bodies in images
US9477878B2 (en) Rigid stabilization of facial expressions
US6031539A (en) Facial image method and apparatus for semi-automatically mapping a face on to a wireframe topology
DeCarlo et al. An anthropometric face model using variational techniques
KR100571115B1 (en) Method and system using a data-driven model for monocular face tracking
US8390628B2 (en) Facial animation using motion capture data
US20130287294A1 (en) Methods for Generating Personalized 3D Models Using 2D Images and Generic 3D Models, and Related Personalized 3D Model Generating System
CN103279936B (en) Human face fake photo based on portrait is synthesized and modification method automatically
EP0907144A2 (en) Method for extracting a three-dimensional model from a sequence of images
DE112016004437T5 (en) Head-mounted display with facial expression detection capability
CN108376421B (en) Method for generating human face three-dimensional model based on shadow recovery shape method
CN110853119B (en) Reference picture-based makeup transfer method with robustness
CN110264396B (en) Video face replacement method, system and computer readable storage medium
CN106709886A (en) Automatic image retouching method and device
CN112257657A (en) Face image fusion method and device, storage medium and electronic equipment
Archer Craniofacial reconstruction using hierarchical B-spline interpolation
Suetens et al. Statistically deformable face models for cranio-facial reconstruction
CN101021943A (en) Image regulating method and system
WO2023273247A1 (en) Face image processing method and device, computer readable storage medium, terminal
JP2005020338A (en) Method, apparatus and program for detecting abnormal shadow
Parthasarathy et al. Spatiotemporal visualization of the tongue surface using ultrasound and kriging (SURFACES)
CN116452453A (en) CNN-based face contour automatic smoothing method, system and storage medium
CN116311474A (en) Face image face filling method, system and storage medium
CN116630599A (en) Method for generating post-orthodontic predicted pictures
WO2013021150A1 (en) Fisheye video correction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination