CN114627216A - Human face shape cartoon method, device, equipment and storage medium - Google Patents
Human face shape cartoon method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114627216A CN114627216A CN202210307587.5A CN202210307587A CN114627216A CN 114627216 A CN114627216 A CN 114627216A CN 202210307587 A CN202210307587 A CN 202210307587A CN 114627216 A CN114627216 A CN 114627216A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- shape
- initial
- cartoon
- human face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
- G06T17/205—Re-meshing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/44—Morphing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2021—Shape modification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Architecture (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application relates to a human face shape cartoon method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring a preset three-dimensional initial cartoon shape; acquiring an input face image; inputting the face image into a three-dimensional deformable model to obtain a predicted identity coefficient and a predicted expression coefficient; carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape; and deforming the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the human face image. The method and the device are used for solving the technical problem that in the prior art, the personalized three-dimensional cartoon image of the user cannot be generated through the picture.
Description
Technical Field
The application relates to the technical field of artificial intelligence, in particular to a human face shape cartoon method, device, equipment and storage medium.
Background
At present, functions such as a special effect camera and a shooting prop become standard configuration of video application programs, a user only needs to upload a photo or a video to automatically synthesize the personalized face special effect in the cartoon style, but the personalized face special effect in the cartoon style synthesized at present is only limited in the cartoon of a two-dimensional image, and the personalized three-dimensional cartoon image of the user cannot be generated through the picture.
Disclosure of Invention
The application provides a human face shape cartoon method, a human face shape cartoon device, human face shape cartoon equipment and a storage medium, which are used for solving the technical problem that in the prior art, a user personalized three-dimensional cartoon image cannot be generated through pictures.
In a first aspect, the present application provides a human face shape cartoon method, including:
acquiring a preset three-dimensional initial cartoon shape;
acquiring an input face image;
inputting the face image into a three-dimensional deformable model to obtain a predicted identity coefficient and a predicted expression coefficient;
carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape;
and deforming the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the human face image.
Optionally, the deforming the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the face image includes:
acquiring the corresponding relation between the three-dimensional initial cartoon shape and the triangular patch between the three-dimensional initial cartoon shape and the initial three-dimensional shape;
calculating a deformation gradient between the initial three-dimensional shape and the reconstructed three-dimensional shape;
and transferring the deformation gradient to the three-dimensional initial cartoon shape according to the corresponding relation to obtain the target three-dimensional cartoon shape.
Optionally, the obtaining of the correspondence between the three-dimensional initial cartoon shape and the triangular patches between the three-dimensional initial cartoon shape includes:
aligning the three-dimensional initial cartoon shape and the initial three-dimensional shape;
obtaining a pre-calibrated key point pair between the initial three-dimensional shape and the three-dimensional initial cartoon shape;
and taking the key point pairs as initial point pairs, and searching the corresponding relation between the three-dimensional initial cartoon shape and the triangular patch between the initial three-dimensional shapes by using an iterative closest point algorithm.
Optionally, the obtaining of the pre-calibrated key point pair between the initial three-dimensional shape and the three-dimensional initial cartoon shape includes:
acquiring the positions of a first number of key point pairs marked on the three-dimensional initial cartoon shape and the initial three-dimensional shape;
training a preset interpolation model by using the positions of the first number of key points to obtain a trained interpolation model;
acquiring the positions of a second number of key points marked on the initial three-dimensional shape;
inputting the positions of the second number of key points into the trained interpolation model to obtain the positions of the key points on the three-dimensional initial cartoon shape corresponding to each key point in the second number of key points, so as to obtain a second number of key point pairs;
and taking the key point pairs of the first quantity and the key point pairs of the second quantity as the key point pairs calibrated in advance.
Optionally, the obtaining of the preset three-dimensional initial cartoon shape includes:
acquiring an input target card ventilation grid;
and acquiring the three-dimensional initial cartoon shape corresponding to the target card ventilating grid.
Optionally, the obtaining the input target card air grid includes:
acquiring an input selection operation;
and determining the target card ventilation lattice selected by the selection operation from a plurality of preset cartoon styles.
In a second aspect, the present application provides a human face shape cartoon apparatus, including:
the first acquisition module is used for acquiring a preset three-dimensional initial cartoon shape;
the second acquisition module is used for acquiring the input face image;
the reconstruction module is used for inputting the face image into a three-dimensional deformable model to obtain a predicted identity coefficient and a predicted expression coefficient; carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape;
and the processing module is used for carrying out deformation processing on the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the human face image.
Optionally, the processing module is configured to obtain a correspondence between the three-dimensional initial cartoon shape and a triangular patch between the three-dimensional initial cartoon shape and the initial three-dimensional shape; calculating a deformation gradient between the initial three-dimensional shape and the reconstructed three-dimensional shape; and transferring the deformation gradient to the three-dimensional initial cartoon shape according to the corresponding relation to obtain the target three-dimensional cartoon shape.
In a third aspect, the present application provides an electronic device, comprising: the system comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus; the memory for storing a computer program; the processor is configured to execute the program stored in the memory, and implement the human face shape cartoon method of the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor implements the human face shape cartoon method of the first aspect.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: the method provided by the embodiment of the application acquires an input face image; inputting the face image into a three-dimensional deformable model to obtain a predicted identity coefficient and an expression coefficient; carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape; and based on the reconstructed three-dimensional shape, carrying out deformation processing on the three-dimensional initial cartoon shape to obtain a target three-dimensional cartoon shape corresponding to the human face image. The identity coefficient and the expression coefficient which are obtained by predicting the human face image input into the three-dimensional deformable model are used for three-dimensional reconstruction, so that the reconstructed three-dimensional shape is closer to the user, the generated cartoon image is closer to the user, and the individuation degree of the cartoon image is improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
Fig. 1 is a schematic flow chart of a human face shape cartoon method according to an embodiment of the present disclosure;
fig. 2 is a schematic view of an overall processing flow of a human face shape cartoon method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a method for labeling key points on an initial three-dimensional shape and a three-dimensional initial cartoon shape according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a human face shape cartoon device provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
The term "and/or" herein is merely an association relationship describing an associated object, and means that there may be three relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
The embodiment of the application provides a human face shape cartoon method, which is characterized in that a three-dimensional initial cartoon shape is subjected to deformation processing through a reconstructed three-dimensional shape obtained by three-dimensional reconstruction of a human face image to obtain a target three-dimensional cartoon shape corresponding to the human face image, and a three-dimensional cartoon image of a user can be generated through a picture, so that the cartoon image is closer to the user, and the personalization degree of the cartoon image is improved.
As shown in fig. 1, an embodiment of the present application provides a human face shape cartoon method, which specifically includes the following steps:
101, acquiring a preset three-dimensional initial cartoon shape;
in specific implementation, the three-dimensional cartoon shape of the presently disclosed virtualized cartoon image can be collected as the three-dimensional initial cartoon shape, and the virtual image can be reproduced by shooting a video, so that the three-dimensional initial cartoon shape is obtained.
In a specific implementation, as shown in fig. 2, a plurality of three-dimensional initial cartoon shapes with cartoon styles may be preset, wherein each card air grid corresponds to at least one three-dimensional cartoon shape. In specific implementation, the target card air grid input by the user can be obtained, and the three-dimensional initial cartoon shape corresponding to the target card air grid is obtained. Specifically, input selection operation is acquired; and determining the target card ventilation lattice selected by the selection operation from a plurality of preset cartoon styles. For example: presetting a plurality of three-dimensional initial cartoon shapes with cartoon styles in a database, enabling a user to select a target card ventilation lattice which the user wants, and selecting the three-dimensional cartoon shape as the three-dimensional initial cartoon shape if the cartoon style only has one three-dimensional cartoon shape; if the cartoon style corresponds to at least two three-dimensional cartoon shapes, one of the at least two three-dimensional cartoon shapes can be randomly selected and selected as the three-dimensional cartoon initial shape, or the selection operation of the user is obtained, and the three-dimensional cartoon shape selected by the selection operation is used as the three-dimensional cartoon initial shape.
102, acquiring an input face image;
in a specific implementation, the face image may be one or more face images, or may be a face image of a certain frame or several frames extracted from a video. Specifically, the information can be obtained by photographing or shooting a video by a user.
in this embodiment of the application, the three-dimensional deformable Model may be a three-dimensional basele Face Model, and the input Face image is reconstructed by using the three-dimensional Basele Face Model (BFM) to obtain a reconstructed three-dimensional shape. The BFM coefficient of the face image mainly comprises: shape coefficient and attitude coefficient; wherein, the identity coefficient and the expression coefficient are shape coefficients.
104, deforming the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape;
specifically, the input face image is input into a coefficient regression neural network of a three-dimensional Barceler face model, and the coefficient of BFM (bidirectional Forwarding model) is predicted, including a shape coefficient and a posture coefficient (c)i,ce,ct,p,γ)∈R257Wherein c isi∈R80Representing an identity coefficient; c. Ce∈R64Representing an expression coefficient; c. Ct∈R80Representing texture map coefficients; p is epsilon of R6Representing a head pose; gamma epsilon R27Representing the illumination coefficient; wherein, ci∈R80And ce∈R64Are all shape factors.
Then, the geometric shape (i.e. the reconstructed three-dimensional shape) S of the face image is reconstructed by using the parameters, which is described by the formula:
wherein the content of the first and second substances,
is the initial three-dimensional shape corresponding to BFM, BidA PCA (principal component analysis) base that is the identity of the BFM model; b isexpThe PCA base is the expression of the BFM model.
In the embodiment of the application, the emphasis is to make the three-dimensional cartoon image and the face of the user closer from the angle of the shape. Therefore, during reconstruction, reconstruction is mainly performed through the identity coefficient and the expression coefficient, and other coefficients are mainly used for rendering.
In the embodiment of the application, the emphasis is to make the three-dimensional cartoon image and the face of the user closer from the angle of the shape. Therefore, during reconstruction, reconstruction is mainly performed through the identity coefficient and the expression coefficient, and other coefficients are mainly used for rendering.
And 105, deforming the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the human face image.
In a specific embodiment, the deformation difference between the initial three-dimensional shape and the reconstructed three-dimensional shape can be calculated, and the deformation difference is transferred to the three-dimensional initial cartoon shape by using a deformation transfer mode to obtain the target three-dimensional cartoon shape.
Specifically, acquiring a corresponding relation between three-dimensional initial cartoon shapes and triangular patches between the initial three-dimensional shapes; calculating a deformation gradient between the initial three-dimensional shape and the reconstructed three-dimensional shape; and transferring the deformation gradient to the three-dimensional initial cartoon shape according to the corresponding relation to obtain the target three-dimensional cartoon shape.
The corresponding relation between the triangular patches is calculated off-line in advance during specific implementation, and the specific method comprises the following steps: aligning the three-dimensional initial cartoon shape with the initial three-dimensional shape; obtaining a pre-calibrated key point pair between the initial three-dimensional shape and the three-dimensional initial cartoon shape; and taking the key point pairs as initial point pairs, and searching the corresponding relation between the three-dimensional initial cartoon shape and the triangular patch between the initial three-dimensional shapes by using an iterative closest point algorithm.
For the key point pairs which are calibrated in advance, the positions of a first number of key point pairs marked on the three-dimensional initial cartoon shape and the initial three-dimensional shape can be obtained; training a preset interpolation model by using the positions of the first number of key points to obtain a trained interpolation model; acquiring the positions of the second number of key points marked on the initial three-dimensional shape; inputting the positions of the second number of key points into the trained interpolation model to obtain the positions of the key points on the three-dimensional initial cartoon shape corresponding to each key point in the second number of key points, so as to obtain a second number of key point pairs; and taking the key point pairs of the first quantity and the key point pairs of the second quantity as pre-calibrated key point pairs.
In this embodiment of the present application, the first number of key point pairs may be a part of key point pairs that are manually calibrated, and the interpolation model is trained using the part of key point pairs that are manually calibrated, for example: and an RBF (radial basis function) interpolation model, then labeling another part of key points (a second number of key points) on the initial three-dimensional shape, then substituting the other part of key points into the trained interpolation model to obtain corresponding key points on the three-dimensional initial cartoon shape, at the moment, obtaining another part of key point pairs, and combining the two parts of key points to be used as the initial point pair for searching the corresponding relation of the triangular patch.
The method for marking the key points of the mobile terminal by the aid of the manual marking is not a good choice, errors are easy to occur, and the situation that the marked points do not correspond to each other may occur.
In a specific implementation, the first number may be 20, and the second number may be 22, that is, 42 sets of key point pairs are obtained.
In the embodiment of the application, an input face image is obtained; inputting the face image into a three-dimensional deformable model to obtain a predicted identity coefficient and an expression coefficient; carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape; and based on the reconstructed three-dimensional shape, carrying out deformation processing on the three-dimensional initial cartoon shape to obtain a target three-dimensional cartoon shape corresponding to the human face image. The identity coefficient and the expression coefficient which are obtained by predicting the human face image input into the three-dimensional deformable model are used for three-dimensional reconstruction, so that the reconstructed three-dimensional shape is closer to the user, the generated cartoon image is closer to the user, and the individuation degree of the cartoon image is improved.
In order to further understand the human face shape cartoon method provided in the embodiment of the present application, the embodiment of the present application is further described below with reference to fig. 2.
As shown in fig. 2, the process mainly comprises three parts: the system comprises a user image input and cartoon style selection module, a three-dimensional head model reconstruction module and a three-dimensional cartoon model deformation module.
1. User image input and cartoon style selection module: the user uploads a clear face image of the user and selects a card ventilating grid which the user wants, wherein each card ventilating grid corresponds to a preset three-dimensional initial cartoon shape
2. The three-dimensional head model reconstruction module reconstructs a real three-dimensional shape with user characteristics through a three-dimensional head reconstruction technology according to the face image uploaded by the user, namely reconstructs the three-dimensional shape;
in a specific implementation, a three-dimensional deformable Model (3D deformable Model,3DMM) may be utilized, for example: and (3) selecting a three-dimensional Basel Face Model (BFM) to reconstruct the three-dimensional shape of the input image. Firstly, inputting a human face image input by a user into a coefficient regression neural network of a three-dimensional Barceler human face model, and predicting the coefficient of BFM (bidirectional Forwarding model) comprising a shape coefficient and a posture coefficient (c)i,ce,ct,p,γ)∈R257Wherein c isi∈R80Representing an identity coefficient; c. Ce∈R64Representing an expression coefficient; c. Ct∈R80Representing texture map coefficients; p is epsilon of R6Representing a head pose; gamma epsilon R27Representing the illumination coefficient; wherein, ci∈R80And ce∈R64Is a form factor. Then, the geometric shape (i.e. the reconstructed three-dimensional shape) S of the face image is reconstructed by using the parameters, which is described by the formula:
wherein the content of the first and second substances,
is the initial three-dimensional shape corresponding to BFM, BidA PCA (principal component analysis) base that is the identity of the BFM model; b isexpThe PCA base is an expression of the BFM model.
3. And the three-dimensional cartoon model deformation module is used for transferring the deformation from the initial three-dimensional shape to the reconstructed three-dimensional shape to the three-dimensional initial cartoon shape selected by the user in a deformation transfer mode.
Now that a reconstructed three-dimensional shape with the characteristics of the user's face has been generated, the initial three-dimensional shape from the BFM will be
The process of deforming to a reconstructed three-dimensional shape S can be viewed as being applied to the original three-dimensional shape
And (5) a personalized process. Then only the personalized deformation needs to be migrated to the three-dimensional original cartoon shape.
As shown in FIG. 3, the initial three-dimensional shape of the BFM is first required
With the three-dimensional initial cartoon shape
And (3) carrying out key point registration calibration, wherein 42 groups of corresponding points of the face are selected for carrying out the standard, and the points are not well selected by simple manual calibration, so that errors are easy to occur, and the problem that the labeled points do not correspond exists, so that 20 groups of key points are manually labeled firstly, and then the rest 22 groups of key points are automatically labeled in a heuristic search mode.
Wherein, the automatic labeling process is as follows: first, 20 sets of keypoint training based on manual labelingA RBF interpolation model, followed by an initial three-dimensional shape
And labeling 22 key points (the left face shown in fig. 3), and substituting the coordinates of the 22 key points into the RBF interpolation model to obtain the positions of the 22 key points on the three-dimensional initial cartoon shape (the right face shown in fig. 3).
Next, the personalized deformation in the initial three-dimensional shape is transferred to the cartoon shape by means of deformation transfer, and the deformation transfer is performed by means of deformation gradient in the embodiment of the application.
The deformation migration process mainly comprises the following steps:
1. preliminarily combining three-dimensional initial cartoon shapes
With the original three-dimensional shape
Alignment is performed, i.e. the two shapes are in similar spatial positions;
2. searching for a three-dimensional initial cartoon shape by using the calibrated key point pairs as initial point pairs
With the original three-dimensional shape
The correspondence relationship N between the triangular patches:
N＝{(s1,t1),(s2,t2),...,(s|M|,t|M|)}
wherein(s)i,ti) Representing an initial three-dimensional shape
Upper triangular patch siAnd
upper triangular patch tiShould have similar deformations, establishing such correspondence helps to shape the original three-dimensional shape
Migration of the deformation to the reconstructed three-dimensional shape S to the cartoon three-dimensional basic shape
In the above, a method for establishing a corresponding relationship based on an iterative closest point algorithm is introduced below; m represents the logarithm of the triangular patch.
(1) Using the calibrated key point pairs as deformation control points to obtain the initial three-dimensional shape
To three-dimensional initial cartoon shape
Deforming to obtain a shape ScAt this time ScHaving and original three-dimensional shape
Identical topology (i.e., two shapes having the same number of points, number of triangle faces, and connectivity), but shape and
close.
(2) Shape ScAnd three-dimensional original cartoon shapes
The triangular patches with similar centroid distances are taken as one item in the corresponding relation N, and it is worth explaining that ScMay correspond to one of the triangular patches
A plurality ofTriangular patches and vice versa.
3. Computing an initial three-dimensional shape
The deformation gradient of the three-dimensional shape S is reconstructed, and then the deformation gradient is transferred to the three-dimensional initial cartoon shape according to the established corresponding relation between the point pairs
Obtain the personalized cartoon three-dimensional shape Scartoon。
The process of deformation migration is briefly described as follows:
(1) constructing a deformation representation (i.e., a deformation gradient): setting an initial three-dimensional shape
Ith triangular patch f of (2)iFrom v1,v2,v3Three-vertex composition, and in order to represent the spatial variation of a triangular patch, an additional point v needs to be introduced4As an aid, therefore fi＝[v1,v2,v3,v4]. Due to the topology of the reconstructed three-dimensional shape S and the original three-dimensional shape
Are identical, so that a corresponding triangular patch can be found on the reconstructed three-dimensional shape S in the same way
fiTo
Can be transformed by an affine matrix QiAnd an offset d, expressed as:
is provided with
The above formula may become:
thus, d is eliminated, and the above formula is represented by a matrix:
wherein:
V＝[v2-v1 v3-v1 v4-v1]
thus, it is possible to provide
Affine matrix QiIndicates the ith triangular patch fiTo
The deformation gradient of (a).
(2) Deformation migration: according to the pre-established corresponding relation N between the triangular patches, the method can be used for solving the problem that the existing method is not suitable for the conventional method
Find a and fiCorresponding dough sheet
At this time, it is desired to obtain
Deformation gradient of
And hope that
And QiAs close as possible. Meanwhile, since one vertex may be shared by a plurality of triangular patches, the deformation of adjacent triangular patches should satisfy the following relationship:
wherein
And
is the respective affine matrix of two adjacent triangular patches, dmAnd dnIs the respective offset of two adjacent triangular patches,
a vertex shared by two adjacent triangular patches.
Therefore, the final goal is to reduce the gap between two sets of affine matrices as much as possible according to the corresponding relationship N on the premise of ensuring vertex sharing, and then can be represented by a constraint optimization problem:
and (3) constraint:
wherein, F represents a 2 norm; p represents other patches sharing the ith point;
an index in the initial three-dimensional shape representing the jth group of the M triangular groups of panels;
represents the j-th panel group in the M triangular panel groups and the index in the three-dimensional initial cartoon shape. For example: the jth panel group consists of the 2 nd triangular panel in the initial three-dimensional shape and the 4 th triangular panel in the three-dimensional initial cartoon shape, and then
Is the number of 2, and the number of the second,
is 4.
Then, the optimization problem is solved, and the deformed target three-dimensional cartoon shape can be obtained.
It should be noted that, the correspondence between the triangular patches is calculated offline in advance and preset in the system, and in actual application, only step 3 is performed.
Based on the same concept, the embodiment of the application provides a human face shape cartoon-making device, and specific implementation of the device can be referred to the description of the embodiment part of the method, and repeated parts are not described again. As shown in fig. 4, the apparatus mainly includes:
a first obtaining module 401, configured to obtain a preset three-dimensional initial cartoon shape;
a second obtaining module 402, configured to obtain an input face image;
a reconstruction module 403, configured to input the face image into a three-dimensional deformable model, so as to obtain a predicted identity coefficient and an expression coefficient; carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by utilizing the identity coefficient and the expression coefficient to obtain the reconstructed three-dimensional shape;
and the processing module 404 is configured to perform deformation processing on the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the face image.
In a specific embodiment, the processing module 404 is configured to obtain a correspondence between the three-dimensional initial cartoon shape and a triangular patch of the initial three-dimensional shape; calculating a deformation gradient between the initial three-dimensional shape and the reconstructed three-dimensional shape; and transferring the deformation gradient to the three-dimensional initial cartoon shape according to the corresponding relation to obtain the target three-dimensional cartoon shape.
In a particular embodiment, a processing module 404 for aligning the three-dimensional initial cartoon shape and the initial three-dimensional shape; obtaining a pre-calibrated key point pair between the initial three-dimensional shape and the three-dimensional initial cartoon shape; and taking the key point pairs as initial point pairs, and searching the corresponding relation between the three-dimensional initial cartoon shape and the triangular patch between the initial three-dimensional shapes by using an iterative closest point algorithm.
In a specific embodiment, the processing module 404 is configured to obtain positions of a first number of key point pairs marked on the three-dimensional initial cartoon shape and the initial three-dimensional shape; training a preset interpolation model by using the positions of the first number of key points to obtain a trained interpolation model; acquiring the positions of a second number of key points marked on the initial three-dimensional shape; inputting the positions of the second number of key points into the trained interpolation model to obtain the positions of the key points on the three-dimensional initial cartoon shape corresponding to each key point in the second number of key points, so as to obtain a second number of key point pairs; and taking the key point pairs of the first quantity and the key point pairs of the second quantity as the pre-calibrated key point pairs.
In a specific embodiment, the first obtaining module 401 is configured to obtain an input target card air grid; and acquiring the three-dimensional initial cartoon shape corresponding to the target card ventilating grid.
In a specific embodiment, the first obtaining module 401 is configured to obtain an input selection operation; and determining the target card ventilation lattice selected by the selection operation from a plurality of preset cartoon styles.
Based on the same concept, an embodiment of the present application further provides an electronic device, as shown in fig. 5, the electronic device mainly includes: a processor 501, a memory 502 and a communication bus 503, wherein the processor 501 and the memory 502 communicate with each other through the communication bus 503. The memory 502 stores a program executable by the processor 501, and the processor 501 executes the program stored in the memory 502, so as to implement the following steps:
acquiring a preset three-dimensional initial cartoon shape;
acquiring an input face image;
inputting the face image into a three-dimensional deformable model to obtain a predicted identity coefficient and a predicted expression coefficient;
carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape;
and deforming the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the human face image.
The communication bus 503 mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus 503 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 5, but this is not intended to represent only one bus or type of bus.
The Memory 502 may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Alternatively, the memory may be at least one memory device located remotely from the aforementioned processor 501.
The Processor 501 may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc., and may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.
In yet another embodiment of the present application, there is also provided a computer-readable storage medium having stored therein a computer program, which, when run on a computer, causes the computer to execute a face shape cartoonizing method described in the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The available media may be magnetic media (e.g., floppy disks, hard disks, tapes, etc.), optical media (e.g., DVDs), or semiconductor media (e.g., solid state drives), among others.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The foregoing are merely exemplary embodiments of the present invention, which enable those skilled in the art to understand or practice the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A human face shape cartoon method is characterized by comprising the following steps:
acquiring a preset three-dimensional initial cartoon shape;
acquiring an input face image;
inputting the face image into a three-dimensional deformable model to obtain a predicted identity coefficient and a predicted expression coefficient;
carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape;
and deforming the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the human face image.
2. The human face shape cartoon method of claim 1, wherein the deforming the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the human face image comprises:
acquiring the corresponding relation between the three-dimensional initial cartoon shape and the triangular patch between the three-dimensional initial cartoon shape and the initial three-dimensional shape;
calculating a deformation gradient between the initial three-dimensional shape and the reconstructed three-dimensional shape;
and transferring the deformation gradient to the three-dimensional initial cartoon shape according to the corresponding relation to obtain the target three-dimensional cartoon shape.
3. The method of claim 2, wherein the obtaining of the correspondence between the three-dimensional initial cartoon shape and the triangular patches of the initial three-dimensional shape comprises:
aligning the three-dimensional initial cartoon shape and the initial three-dimensional shape;
obtaining a pre-calibrated key point pair between the initial three-dimensional shape and the three-dimensional initial cartoon shape;
and taking the key point pair as an initial point pair, and searching the corresponding relation between the three-dimensional initial cartoon shape and the triangular patch between the three-dimensional initial cartoon shape and the initial three-dimensional shape by using an iterative closest point algorithm.
4. The human face shape cartoonification method of claim 3, wherein the obtaining of the pre-calibrated key point pairs between the initial three-dimensional shape and the three-dimensional initial cartoon shape comprises:
acquiring the positions of a first number of key point pairs marked on the three-dimensional initial cartoon shape and the initial three-dimensional shape;
training a preset interpolation model by using the positions of the first number of key points to obtain a trained interpolation model;
acquiring the positions of a second number of key points marked on the initial three-dimensional shape;
inputting the positions of the second number of key points into the trained interpolation model to obtain the positions of the key points on the three-dimensional initial cartoon shape corresponding to each key point in the second number of key points, so as to obtain a second number of key point pairs;
and taking the key point pairs of the first quantity and the key point pairs of the second quantity as the pre-calibrated key point pairs.
5. The human face shape cartoon method of claim 1, wherein the obtaining of the preset three-dimensional initial cartoon shape comprises:
acquiring an input target card ventilation grid;
and acquiring the three-dimensional initial cartoon shape corresponding to the target card ventilating grid.
6. The human face shape cartoonizing method of claim 1, wherein the obtaining the input target card ventilation check comprises:
acquiring an input selection operation;
and determining the target card ventilation lattice selected by the selection operation from a plurality of preset cartoon styles.
7. A human face shape cartoon device is characterized by comprising:
the first acquisition module is used for acquiring a preset three-dimensional initial cartoon shape;
the second acquisition module is used for acquiring the input face image;
the reconstruction module is used for inputting the face image into a three-dimensional deformable model to obtain a predicted identity coefficient and a predicted expression coefficient; carrying out deformation processing on the initial three-dimensional shape corresponding to the three-dimensional deformable model by using the identity coefficient and the expression coefficient to obtain a reconstructed three-dimensional shape;
and the processing module is used for carrying out deformation processing on the three-dimensional initial cartoon shape based on the reconstructed three-dimensional shape to obtain a target three-dimensional cartoon shape corresponding to the human face image.
8. The human face shape cartoonification device according to claim 7, wherein the processing module is configured to obtain a correspondence between the three-dimensional initial cartoon shape and a triangular patch of the initial three-dimensional shape; calculating a deformation gradient between the initial three-dimensional shape and the reconstructed three-dimensional shape; and transferring the deformation gradient to the three-dimensional initial cartoon shape according to the corresponding relation to obtain the target three-dimensional cartoon shape.
9. An electronic device, comprising: the system comprises a processor, a memory and a communication bus, wherein the processor and the memory are communicated with each other through the communication bus; the memory for storing a computer program; the processor is used for executing the program stored in the memory and realizing the human face shape cartoon method of any one of claims 1 to 6.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the face shape cartoonizing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210307587.5A CN114627216A (en) | 2022-03-25 | 2022-03-25 | Human face shape cartoon method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210307587.5A CN114627216A (en) | 2022-03-25 | 2022-03-25 | Human face shape cartoon method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114627216A true CN114627216A (en) | 2022-06-14 |
Family
ID=81903264
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210307587.5A Pending CN114627216A (en) | 2022-03-25 | 2022-03-25 | Human face shape cartoon method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114627216A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115359171A (en) * | 2022-10-21 | 2022-11-18 | 北京百度网讯科技有限公司 | Virtual image processing method and device, electronic equipment and storage medium |
-
2022
- 2022-03-25 CN CN202210307587.5A patent/CN114627216A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115359171A (en) * | 2022-10-21 | 2022-11-18 | 北京百度网讯科技有限公司 | Virtual image processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110399825B (en) | Facial expression migration method and device, storage medium and computer equipment | |
| US20150097827A1 (en) | Target Region Fill Utilizing Transformations | |
| US10699453B2 (en) | Digital media environment for style-aware patching in a digital image | |
| US20150117784A1 (en) | Image foreground detection | |
| US20150116350A1 (en) | Combined composition and change-based models for image cropping | |
| WO2020087564A1 (en) | Three-dimensional object reconstruction method, computer device and storage medium | |
| CA2995857A1 (en) | Edge-aware bilateral image processing | |
| EP3876110A1 (en) | Method, device and apparatus for recognizing, categorizing and searching for garment, and storage medium | |
| CN113689578B (en) | Human body data set generation method and device | |
| US20220327769A1 (en) | Neural light transport | |
| WO2022143354A1 (en) | Face generation method and apparatus for virtual object, and device and readable storage medium | |
| JP2023514107A (en) | 3D reconstruction method, device, equipment and storage medium | |
| CN114925748A (en) | Model training and modal information prediction method, related device, equipment and medium | |
| CN114627216A (en) | Human face shape cartoon method, device, equipment and storage medium | |
| CN112287730A (en) | Gesture recognition method, device, system, storage medium and equipment | |
| Hwang et al. | Probabilistic moving least squares with spatial constraints for nonlinear color transfer between images | |
| CN112733901A (en) | Structured action classification method and device based on federal learning and block chain | |
| Yang et al. | Efficient global color correction for large-scale multiple-view images in three-dimensional reconstruction | |
| CN111507288A (en) | Image detection method, image detection device, computer equipment and storage medium | |
| CN113469091B (en) | Face recognition method, training method, electronic device and storage medium | |
| CN110956131A (en) | Single-target tracking method, device and system | |
| WO2024093763A1 (en) | Panoramic image processing method and apparatus, computer device, medium and program product | |
| CN116881886A (en) | Identity recognition method, identity recognition device, computer equipment and storage medium | |
| Li et al. | Color correction based on point clouds alignment in the logarithmic RGB space | |
| US9853982B2 (en) | Image-based group profiles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |