CN112581518A - Eyeball registration method, device, server and medium based on three-dimensional cartoon model - Google Patents
Eyeball registration method, device, server and medium based on three-dimensional cartoon model Download PDFInfo
- Publication number
- CN112581518A CN112581518A CN202011565476.1A CN202011565476A CN112581518A CN 112581518 A CN112581518 A CN 112581518A CN 202011565476 A CN202011565476 A CN 202011565476A CN 112581518 A CN112581518 A CN 112581518A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- eyeball
- human face
- template
- cartoon model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 210000005252 bulbus oculi Anatomy 0.000 title claims abstract description 270
- 238000000034 method Methods 0.000 title claims abstract description 68
- 210000000744 eyelid Anatomy 0.000 claims abstract description 67
- 238000013519 translation Methods 0.000 claims abstract description 61
- 210000001508 eye Anatomy 0.000 claims description 33
- 230000008921 facial expression Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 abstract description 11
- 210000004279 orbit Anatomy 0.000 description 20
- 230000008569 process Effects 0.000 description 20
- 238000010586 diagram Methods 0.000 description 12
- 239000000203 mixture Substances 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 230000014509 gene expression Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2021—Shape modification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Architecture (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses an eyeball registration method, an eyeball registration device, a server and a medium based on a three-dimensional cartoon model. Wherein, the method comprises the following steps: deforming the three-dimensional cartoon grid template according to the face image to be customized to obtain a customized three-dimensional face cartoon model; calculating scaling and registration parameters of a preset three-dimensional eyeball template in the three-dimensional human face cartoon model according to orbital deformation information in the three-dimensional human face cartoon model; and calculating the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model by taking the fact that no interpenetration exists between the three-dimensional eyeball template which is zoomed according to the zoom registration parameters and eyelid grids in the three-dimensional human face cartoon model as a target. According to the technical scheme provided by the embodiment of the invention, the calculation cost of eyeball registration in the three-dimensional cartoon model is greatly reduced, the complexity of eyeball registration in the three-dimensional cartoon model is reduced, and the high efficiency of eyeball registration in the three-dimensional cartoon model is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of three-dimensional model reconstruction, in particular to an eyeball registration method, device, server and medium based on a three-dimensional cartoon model.
Background
With the rapid development of computer technology, Augmented Reality (AR) products have been widely applied to people's daily life, and virtual content in AR products needs to be seamlessly embedded into a real image, thereby giving a user a feeling of real existence of the virtual content; at this time, in order to realize diversified interaction of the user in the AR products, a scheme for customizing the corresponding three-dimensional cartoon image according to the actual image of the user is specially designed.
However, when customizing three-dimensional cartoon images of different users, the size, the position and the like of eyeballs in the three-dimensional cartoon models need to be adjusted adaptively according to the deformation conditions of the models in the process of customizing the three-dimensional cartoon models, a large number of model points are sampled in the universal three-dimensional grid models at the moment, and then the three-dimensional cartoon images and the specific deformation of the eyeballs in the models are analyzed synchronously by calculating the closest point conditions of the model points, so that the registration of the eyeballs is synchronously realized in the customizing process of the three-dimensional cartoon models; at this time, when the general three-dimensional grid model is used for analyzing the specific deformation of the eyeball, the deformation condition of the model is also considered, so that the existing eyeball registration algorithm is complex, the great calculation overhead exists, and the eyeball registration efficiency in the three-dimensional cartoon model is influenced.
Disclosure of Invention
The embodiment of the invention provides an eyeball registration method, device, server and medium based on a three-dimensional cartoon model, which reduces the calculation overhead of eyeball registration in the three-dimensional cartoon model, reduces the complexity of eyeball registration in the three-dimensional cartoon model and improves the high efficiency of eyeball registration in the three-dimensional cartoon model.
In a first aspect, an embodiment of the present invention provides an eyeball registration method based on a three-dimensional cartoon model, where the method includes:
deforming the three-dimensional cartoon grid template according to the face image to be customized to obtain a customized three-dimensional face cartoon model;
calculating scaling and registration parameters of a preset three-dimensional eyeball template in the three-dimensional human face cartoon model according to orbital deformation information in the three-dimensional human face cartoon model;
and calculating the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model by taking the fact that no interpenetration exists between the three-dimensional eyeball template which is zoomed according to the zoom registration parameters and eyelid grids in the three-dimensional human face cartoon model as a target.
In a second aspect, an embodiment of the present invention provides an eyeball registration apparatus based on a three-dimensional cartoon model, where the apparatus includes:
the cartoon model customizing module is used for deforming the three-dimensional cartoon grid template according to the human face image to be customized to obtain a customized three-dimensional human face cartoon model;
the scaling and registration determining module is used for calculating scaling and registration parameters of a preset three-dimensional eyeball template in the three-dimensional face cartoon model according to orbital deformation information in the three-dimensional face cartoon model;
and the translation registration determining module is used for calculating the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model by taking the fact that no interpenetration exists between the three-dimensional eyeball template which is scaled according to the scaling registration parameters and eyelid grids in the three-dimensional human face cartoon model as a target.
In a third aspect, an embodiment of the present invention provides a server, where the server includes:
one or more processors;
storage means for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors implement the three-dimensional cartoon model-based eyeball registration method according to any embodiment of the invention.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the three-dimensional cartoon model-based eyeball registration method according to any embodiment of the present invention.
The embodiment of the invention provides an eyeball registration method, a device, a server and a medium based on a three-dimensional cartoon model, which comprises the steps of firstly deforming a three-dimensional cartoon grid template which is uniformly set according to a human face image to be customized to obtain the customized three-dimensional cartoon model, then calculating a zoom registration parameter of the preset three-dimensional eyeball template in the three-dimensional cartoon model, which is suitable for the deformation of the eye socket, according to eye socket deformation information in the three-dimensional cartoon model, further calculating a translation registration parameter of the three-dimensional eyeball template in the three-dimensional cartoon model by taking the three-dimensional eyeball template which is zoomed according to the zoom registration parameter as a target without interpenetration between the three-dimensional eyeball template and an eyelid grid in the three-dimensional cartoon model, thereby avoiding the situation that the eyeball in the three-dimensional cartoon model is not suitable and the situation that a large number of model points in the universal three-dimensional cartoon model are deformed, the calculation cost of eyeball registration in the three-dimensional cartoon model is greatly reduced, the eyeball registration in the three-dimensional cartoon model can be realized by directly carrying out zooming and translation operations on the three-dimensional eyeball template in the three-dimensional cartoon model, the complexity of eyeball registration in the three-dimensional cartoon model is reduced, and the high efficiency of eyeball registration in the three-dimensional cartoon model is improved.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
fig. 1A is a flowchart of an eyeball registration method based on a three-dimensional cartoon model according to a first embodiment of the present invention;
fig. 1B is a schematic diagram illustrating an eyeball registration process based on a three-dimensional cartoon model according to an embodiment of the present invention;
fig. 2A is a flowchart of an eyeball registration method based on a three-dimensional cartoon model according to a second embodiment of the present invention;
fig. 2B is a schematic diagram illustrating an eyeball registration process based on a three-dimensional cartoon model according to a second embodiment of the present invention;
fig. 2C is a schematic diagram of eyelid edge feature points in the three-dimensional human face cartoon model in the method provided by the second embodiment of the present invention;
fig. 2D is a schematic diagram of each feature point in an eyelid mesh in the three-dimensional human face cartoon model in the method provided by the second embodiment of the present invention;
fig. 3A is a flowchart of an eyeball registration method based on a three-dimensional cartoon model according to a third embodiment of the present invention;
fig. 3B is a schematic diagram of a pupil center point in the three-dimensional eyeball template in the method provided by the third embodiment of the invention;
fig. 3C is a schematic diagram of the nearest feature point of the pupil center point on the three-dimensional cartoon grid template in the three-dimensional eyeball template in the method provided by the third embodiment of the present invention;
fig. 3D is a schematic diagram illustrating a translation registration process of the three-dimensional eyeball template in the method according to the third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an eyeball registration apparatus based on a three-dimensional cartoon model according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a server according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures. In addition, the embodiments and features of the embodiments in the present invention may be combined with each other without conflict.
Example one
Fig. 1A is a flowchart of an eyeball registration method based on a three-dimensional cartoon model according to an embodiment of the present invention, and this embodiment is applicable to any case of registering sizes and positions of eyeballs in a customized three-dimensional cartoon image of a user. The eyeball registration method based on the three-dimensional cartoon model provided by the embodiment of the invention can be executed by the eyeball registration device based on the three-dimensional cartoon model provided by the embodiment of the invention, the device can be realized in a software and/or hardware mode, and is integrated in a server executing the method, and the server can be a background server with three-dimensional image reconstruction capability.
Specifically, referring to fig. 1A, the method may include the steps of:
s110, deforming the three-dimensional cartoon grid template according to the face image to be customized to obtain a customized three-dimensional face cartoon model.
Specifically, in order to increase diversified interactive playing methods of users in AR products, a function of customizing a corresponding three-dimensional cartoon image according to the actual image of the user is specially designed in the AR products, at the moment, in order to avoid the adoption of a large number of model points in a universal three-dimensional grid model, the problems of overlarge calculation cost and complex customized algorithm when eyeball registration is synchronously realized in the three-dimensional cartoon model customizing process, meanwhile, eyeballs needing to be registered in the three-dimensional cartoon image customizing process are considered, the three-dimensional cartoon image customizing process and the eyeball registration process in a face image are separated and sequentially realized, the three-dimensional cartoon image is firstly reconstructed according to the actual image of the user in the face image, the appearance, the outline, the expression and the like of the user, and then the eyeballs are registered in the reconstructed three-dimensional face cartoon model, thereby obtaining the final three-dimensional human face cartoon image.
The face image is acquired image data containing a face picture, and the embodiment is mainly applied to three-dimensional cartoon image customization of the face image acquired in the AR product, so that the current face image can be acquired through image collectors such as a camera and the like configured in the AR product. Meanwhile, in order to ensure the reality of the three-dimensional human face cartoon image, the embodiment presets a three-dimensional cartoon grid template with a standard cartoon human face image, wherein the three-dimensional cartoon grid template is a three-dimensional cartoon model which is created in advance according to the standard human face cartoon image and added with grid distribution instructions; in the process of creating the three-dimensional cartoon grid template, the three-dimensional cartoon grid template with different accuracies or different grid shapes can be created according to different selected model pixel resolutions and grid shapes, that is, the grid distribution density or the grid distribution shapes in the created three-dimensional cartoon grid template are different, for example, the three-dimensional cartoon grid template can be composed of quadrilateral grids or triangular grids. In this embodiment, before customizing a corresponding three-dimensional cartoon image according to a certain face image, a created three-dimensional cartoon grid template is pre-stored, and then, the same three-dimensional cartoon grid template may be referred to and respectively matched with different face images to perform three-dimensional cartoon image customization on each face image, so as to avoid the limitation of the face pose or image in different face images.
Optionally, before the eyeball in the three-dimensional cartoon image is registered, in this embodiment, a face image to be referred to when the three-dimensional cartoon image is customized is collected through an AR product, and a three-dimensional cartoon grid template which is pre-stored and supports each user to accurately customize the three-dimensional cartoon image is collected, and then, by analyzing information such as the pose position of each feature point in the face image, the same feature point in the three-dimensional cartoon grid template is correspondingly deformed, so that the cartoon pose in the deformed three-dimensional cartoon grid template can be approximately the same as the face pose in the face image, and the deformed three-dimensional cartoon grid template is taken as the three-dimensional face cartoon model customized according to the face image in this embodiment, at this time, for different face images, the three-dimensional cartoon grid template can be referred to for corresponding deformation, and obtaining a customized three-dimensional human face cartoon model under each human face image, wherein the customized three-dimensional human face cartoon model and the three-dimensional cartoon grid template have the same grid vertexes, so that the registration condition of the eyeball can be accurately analyzed according to the deformation condition of the specific vertex on the eyelid part.
And S120, calculating scaling and registration parameters of the preset three-dimensional eyeball template in the three-dimensional human face cartoon model according to orbital deformation information in the three-dimensional human face cartoon model.
Optionally, considering that an eyeball to be registered in the process of customizing the three-dimensional cartoon image belongs to a relatively regular three-dimensional model, the three-dimensional cartoon image customizing process and the eyeball registering process in the face image are separated and sequentially realized, at this time, the eyeball in the three-dimensional face cartoon model is accurately registered, a three-dimensional eyeball template with standard eyeball specification is preset in the embodiment, and after the three-dimensional face cartoon model is customized according to the face pose in the face image, the three-dimensional eyeball template can be adjusted adaptively, so that the adjusted three-dimensional eyeball template can be completely suitable for the eye region in the customized three-dimensional face cartoon model.
When the three-dimensional cartoon face model is customized by deforming the three-dimensional cartoon grid template according to the face image, the size, the depth and the like of the eye socket in the customized three-dimensional cartoon face model can be correspondingly changed, so that the size and the position of the eyeball can be correspondingly changed, the change of the size of the eyeball can be represented by the zoom registration parameter of the eyeball, and the change of the position of the eyeball can be represented by the translation registration parameter of the eyeball.
In this embodiment, since the preset three-dimensional eyeball template can be completely fitted into the orbit of the preset three-dimensional cartoon grid template, and the three-dimensional eyeball model after the eyeball registration also needs to be completely fitted into the orbit of the customized three-dimensional human face cartoon model, as shown in fig. 1B, for the eyeball registration under any one of the left eye and the right eye in the three-dimensional cartoon grid template, the zoom registration parameters of the three-dimensional eyeball template under each eye in the three-dimensional human face cartoon model can be calculated by analyzing the original size of the orbit under the eye in the three-dimensional cartoon grid template and the difference between the actual size of the orbit under the eye in the deformed customized three-dimensional human face cartoon model, and then the zoom registration parameters are adopted to perform corresponding zoom operations on the three-dimensional eyeball template under different eyes, so that the three-dimensional eyeball model obtained after the zoom can be completely fitted into the orbit of the corresponding eye in the three-dimensional human face cartoon model Therefore, the accuracy of eyeball size registration in the customized three-dimensional human face cartoon model is ensured.
And S130, calculating the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model by taking the fact that no interpenetration exists between the three-dimensional eyeball template which is zoomed according to the zoom registration parameters and eyelid grids in the three-dimensional human face cartoon model as a target.
Optionally, when the three-dimensional human face cartoon model is customized, the orbit depth in the three-dimensional cartoon grid model may be deformed according to the eye pose of the human face image, and at this time, if only the three-dimensional eyeball template is zoomed, the zoomed three-dimensional eyeball template may protrude from the orbit in the three-dimensional cartoon grid model, so in order to ensure the adaptability of the zoomed three-dimensional eyeball template to the orbit depth in the three-dimensional cartoon grid model, in this embodiment, the zoomed three-dimensional eyeball template further needs to be subjected to position translation, so that an eyelid in the customized three-dimensional cartoon human face model can completely wrap the three-dimensional eyeball model obtained after registration, and the fidelity of eyeball registration in the three-dimensional human face cartoon model is ensured.
Specifically, in this embodiment, for the eyeball registration in each eye of the three-dimensional face cartoon model, whether the three-dimensional eyeball model obtained by registration is completely matched with the orbit depth of the three-dimensional cartoon grid model can be determined by analyzing whether there is an interpenetration between the scaled three-dimensional eyeball template and the eyelid grid under the eye in the three-dimensional face cartoon model, at this time, firstly, in the customization process of the three-dimensional cartoon grid model, the eyelid grid customized under each eye to satisfy various eye opening or eye closing conditions is synchronously found out, at this time, if the scaling registration parameters of the three-dimensional eyeball template in the three-dimensional face cartoon model are used to scale the three-dimensional eyeball template, then, through analyzing whether there is an interpenetration between the scaled three-dimensional eyeball template and each eyelid grid in the three-dimensional cartoon model, and the specific interpenetration condition, therefore, the translation registration parameters which can ensure that no interpenetration exists between the zoomed three-dimensional eyeball template and the eyelid grids in the three-dimensional human face cartoon model are determined.
In addition, after determining the scaling registration parameters and the translation registration parameters of the three-dimensional eyeball template in the three-dimensional face cartoon model, the embodiment also uses the scaling registration parameters and the translation registration parameters to sequentially register the three-dimensional eyeball template in the three-dimensional face cartoon model; specifically, for eyeball registration under each eye in the three-dimensional human face cartoon model, firstly, the zoom registration parameters calculated for the three-dimensional eyeball template under the eye in the three-dimensional human face cartoon model are adopted, the three-dimensional eyeball template is zoomed in the eye of the three-dimensional human face cartoon model, then, the translation registration parameters under the eye are adopted again, and the zoomed three-dimensional eyeball template is continuously translated in the eye of the three-dimensional human face cartoon model correspondingly, so that accurate eyeball registration in each eye of the three-dimensional human face cartoon model is realized, and the condition that interpenetration exists between the three-dimensional eyeball model after registration and eyelid grids in the three-dimensional human face cartoon model is avoided.
According to the technical scheme provided by the embodiment, firstly, a three-dimensional cartoon grid template which is uniformly set is deformed according to a human face image to be customized to obtain a customized three-dimensional cartoon model, then, scaling and registration parameters of the preset three-dimensional eyeball template in the three-dimensional cartoon model and adapted to the deformation of the eyepit are calculated according to the deformation information of the eyepit in the three-dimensional cartoon model, so that the three-dimensional eyeball template which is scaled according to the scaling and registration parameters does not penetrate into an eyelid grid in the three-dimensional cartoon model as a target, and translation and registration parameters of the three-dimensional eyeball template in the three-dimensional cartoon model are calculated, thereby avoiding the condition that the eyeball in the three-dimensional cartoon model is not adapted, avoiding the deformation condition of a large number of model points in the universal three-dimensional grid model, and greatly reducing the calculation cost of eyeball registration in the three-dimensional cartoon model, the eyeball registration in the three-dimensional human face cartoon model can be realized by directly carrying out zooming and translation operations on the three-dimensional eyeball template in the three-dimensional human face cartoon model, the complexity of eyeball registration in the three-dimensional cartoon model is reduced, and the high efficiency of eyeball registration in the three-dimensional cartoon model is improved.
Example two
Fig. 2A is a flowchart of an eyeball registration method based on a three-dimensional cartoon model according to a second embodiment of the present invention, and fig. 2B is a schematic diagram of a principle of an eyeball registration process based on a three-dimensional cartoon model according to a second embodiment of the present invention. The embodiment is optimized on the basis of the embodiment. Specifically, as shown in fig. 2A, the present embodiment mainly explains a specific deformation process of the three-dimensional cartoon grid template and a specific calculation process of the registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model in detail.
Optionally, as shown in fig. 2A, the present embodiment may include the following steps:
s210, sequentially deforming the three-dimensional cartoon grid template according to the human face appearance and the human face expression in the human face image to be customized to obtain a customized three-dimensional human face cartoon model.
Optionally, when customizing the corresponding three-dimensional cartoon image according to the human face image, the cartoon image of the human face appearance and the human face expression is required to be capable of matching with the human face image, in order to ensure the fidelity of the three-dimensional human face cartoon model, as shown in fig. 2B, the face image can be analyzed by analyzing the poses of the respective feature points representing the face appearance in the face image, and the poses of the respective feature points representing the facial expression, then, the three-dimensional cartoon grid template is subjected to shape deformation according to the shape and the posture of the human face so as to ensure the matching between the cartoon shape in the three-dimensional human face cartoon model and the human face image, then further carrying out deformation adjustment under the facial expression on the deformed three-dimensional cartoon grid template according to the facial expression posture, therefore, the customized three-dimensional human face cartoon model is obtained, and the reality of the human face appearance and the human face expression in the three-dimensional human face cartoon model is ensured.
For example, in this embodiment, a three-dimensional cartoon modeling may be performed on the shape of a human face by using deformation among feature points in a human face image, and in order to ensure the reality of a human face expression in the three-dimensional cartoon model, a hybrid deformation (blend shape) model may be used, where the model includes a smile blend shape, an eye-closed blend shape (eyeblend _ L), an eye-closed blend shape (eyeblend) and the like, and at this time, each blend shape mesh is correspondingly deformed according to the human face expression, so that a three-dimensional cartoon model of the human face conforming to an actual human face image is obtained.
And S220, extracting eyelid edge feature points in the three-dimensional human face cartoon model, and calculating corresponding orbit deformation information according to the deformation information of each eyelid edge feature point.
In this embodiment, the orbit deformation information referred to when the scaling registration parameter of the three-dimensional eyeball template in the three-dimensional face cartoon model is calculated is mainly used for judging the orbit size in each eye of the customized three-dimensional face cartoon model, and in this embodiment, each eyelid edge feature point capable of accurately representing the orbit position can be extracted from the eye region where the left eye or the right eye in the three-dimensional face cartoon model is located, and then the initial position of each eyelid edge feature point in the three-dimensional cartoon grid template before deformation and the deformation position in the deformed three-dimensional cartoon face model are analyzed, and the deformation condition between the initial position and the deformation position of each eyelid edge feature point is determined, so that the deformation information of each eyelid edge feature point is determined, and the corresponding orbit deformation information is calculated.
Illustratively, as shown in fig. 2C, if there are 4 feature points v of the eyelid edge at a certain eyelid location of the three-dimensional human cartoon modell,i、vr,i、vu,iAnd vb,iAnd the position coordinate before each eyelid edge feature point is deformed is represented by P, and the position coordinate after deformation is represented by Q, wherein i represents the left eye or the right eye in the three-dimensional human face cartoon model, and then the scaling registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model can be as follows:
and S230, calculating scaling and registration parameters of the preset three-dimensional eyeball template in the three-dimensional face cartoon model according to the eye socket deformation information in the three-dimensional face cartoon model.
S240, eyelid grids in different eyelid states in the three-dimensional human face cartoon model are determined, and feature points in each eyelid grid are extracted.
Optionally, in the customization process of the three-dimensional human face cartoon model, in order to meet the reality of eyelid opening and closing under various eye expressions, corresponding eyelid grids are customized specially and synchronously according to the eyelid states under various eye expressions, so as to ensure the reality of the three-dimensional human face cartoon model under various eye expressions; meanwhile, after the three-dimensional eyeball template is subsequently zoomed according to the zooming registration parameters, whether interpenetration exists between the zoomed three-dimensional eyeball template and an eyelid mesh in the three-dimensional human face cartoon model can be judged by analyzing whether the distance between an eyeball center point in the zoomed three-dimensional eyeball template and each feature point in the eyelid mesh exists or not, so that as shown in fig. 2D, a large number of mesh vertexes are firstly extracted from each eyelid mesh in the three-dimensional human face cartoon model under different eyelid states to serve as feature points in the eyelid mesh, and whether interpenetration exists between the zoomed three-dimensional eyeball template and the eyelid mesh in the three-dimensional human face cartoon model is subsequently analyzed according to the method.
And S250, calculating the eyeball center point and the eyeball radius of the three-dimensional eyeball template after scaling according to the scaling and registration parameters.
In this embodiment, in order to accurately analyze whether there is an interpenetration between the scaled three-dimensional eyeball template and an eyelid grid in the three-dimensional human face cartoon model, the radius of the scaled three-dimensional eyeball template needs to be correspondingly calculated after the scaling operation is performed on the three-dimensional eyeball template according to the scaling registration parameters; and analyzing the associated characteristic points between the three-dimensional eyeball template and the three-dimensional cartoon grid template before deformation, obtaining a deformation position after corresponding deformation, correspondingly calculating the eyeball center point of the three-dimensional eyeball template relative to the three-dimensional human face cartoon model after the three-dimensional eyeball template is zoomed according to the deformed position of the associated characteristic points and the eyeball radius of the three-dimensional eyeball template after the three-dimensional eyeball template is zoomed, and subsequently judging whether interpenetration exists between the zoomed three-dimensional eyeball template and the eyelid grid in the three-dimensional human face cartoon model by analyzing whether the distance between each characteristic point in the three-dimensional human face cartoon model and the eyeball center point exceeds the zoomed eyeball radius.
And S260, aiming at each feature point on an eyelid grid in the three-dimensional human face cartoon model, calculating a sub-translation parameter corresponding to the feature point according to the distance between the feature point and the center point of the eyeball and the difference between the radius of the eyeball.
Specifically, after each feature point is extracted from each eyelid mesh in the three-dimensional human face cartoon model, the distance between the feature point and the zoomed eyeball center point of the three-dimensional eyeball template can be respectively calculated for each feature point, if the distance is smaller than the zoomed eyeball radius of the three-dimensional eyeball template, the feature point in the eyelid mesh in the three-dimensional human face cartoon model is inserted into the zoomed three-dimensional eyeball template, so that the zoomed three-dimensional eyeball template is displayed in the eye socket of the three-dimensional human face cartoon model in a protruding manner, at this time, the zoomed three-dimensional eyeball template needs to be translated for a certain distance towards the inside of the eye frame of the three-dimensional human face cartoon model, so that the feature point on the eyelid mesh in the three-dimensional human face cartoon model cannot be inserted into the three-dimensional eyeball model, and at this time, according to the distance between the feature point and the eyeball center point, the distance is different relative to the zoomed eyeball radius, the corresponding sub-translation parameters when the characteristic points meet the requirement of not penetrating in the three-dimensional eyeball model can be calculated; at this time, the sub-translation parameters corresponding to each feature point in each eyelid grid can be calculated according to the above steps, and then a translation registration parameter which can enable all feature points not to be inserted in the three-dimensional eyeball model is selected from the sub-translation parameters corresponding to each feature point.
And S270, comparing the sub-translation parameters corresponding to each feature point on the eyelid grids in the three-dimensional human face cartoon model, and determining the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model.
Optionally, after calculating the sub-translation parameter corresponding to each feature point in each eyelid grid in the three-dimensional human face cartoon model, since each sub-translation parameter represents the translation distance of the three-dimensional eyeball template, the largest sub-translation parameter can be selected by comparing the sub-translation parameters corresponding to each feature point, so that all feature points are not inserted into the three-dimensional eyeball model, and then the selected largest sub-translation parameter is used as the translation registration parameter of the three-dimensional eyeball template in the three-dimensional human face cartoon model in this embodiment, so as to ensure the fidelity of eyeball registration in the three-dimensional human face cartoon model.
According to the technical scheme provided by the embodiment, the sub-translation parameters of each feature point meeting the requirement of the registered three-dimensional eyeball model when the feature point does not penetrate through are calculated by utilizing the distance between each feature point on the eyelid grid in the three-dimensional human face cartoon model and the zoomed eyeball center point relative to the distance difference between the zoomed eyeball radiuses, then the sub-translation parameters corresponding to each feature point are compared, the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model are determined, the translation accuracy of the three-dimensional eyeball template in the three-dimensional human face cartoon model is ensured, and the reality of eyeball registration in the three-dimensional human face cartoon model is improved.
EXAMPLE III
Fig. 3A is a flowchart of an eyeball registration method based on a three-dimensional cartoon model according to a third embodiment of the present invention. The embodiment is optimized on the basis of the embodiment. Specifically, as shown in fig. 3A, the present embodiment mainly explains the specific calculation process and the translational registration process of the eyeball center point and the eyeball radius after the three-dimensional eyeball template is zoomed in detail.
Optionally, as shown in fig. 3A, the present embodiment may include the following steps:
s310, deforming the three-dimensional cartoon grid template according to the face image to be customized to obtain a customized three-dimensional face cartoon model.
And S320, calculating scaling and registration parameters of the preset three-dimensional eyeball template in the three-dimensional face cartoon model according to the eye socket deformation information in the three-dimensional face cartoon model.
And S330, calculating the eyeball radius of the three-dimensional eyeball template after being scaled according to the scaling and registration parameters according to the original eyeball radius and the scaling and registration parameters of the three-dimensional eyeball template.
Optionally, when the three-dimensional eyeball template is zoomed according to the zoom registration parameter, the eyeball radius is also zoomed correspondingly, at this time, a corresponding pupil center point may be extracted from the preset three-dimensional eyeball template, as shown in fig. 3B, since the eyeball center point under the three-dimensional eyeball template may be the origin of coordinates, by analyzing the position of the pupil center point, the original eyeball radius of the three-dimensional eyeball template may be calculated, and then the original eyeball radius of the three-dimensional eyeball template is zoomed correspondingly according to the zoom registration parameter, so that the eyeball radius of the three-dimensional eyeball template after being zoomed according to the zoom registration parameter may be obtained.
Illustratively, if the scaled registration parameter is siThe eyeball radius of the three-dimensional eyeball template after scaling according to the scaling registration parameters can be as follows:
wherein the content of the first and second substances,
original eyeball radius of three-dimensional eyeball template, EiAs a coordinate system of a three-dimensional eyeball template, ciIs the pupil center of the three-dimensional eyeball template.
And S340, calculating the eyeball center point of the three-dimensional eyeball template after scaling according to the scaling and registration parameters according to the nearest feature point position and the eyeball radius of the pupil center point in the three-dimensional eyeball template in the three-dimensional human face cartoon model.
Optionally, because the translation of the three-dimensional eyeball template is set facing the three-dimensional face cartoon model, when analyzing the distance between each feature point in the eyelid grid in the three-dimensional face cartoon model and the zoomed eyeball center point, the position of the eyeball center point relative to the three-dimensional face cartoon model also needs to be calculated, so that the present embodiment may approximate the actual position of the pupil center point relative to the three-dimensional face cartoon model by analyzing the deformation position of the pupil center point of the three-dimensional eyeball template at the nearest feature point in the three-dimensional face cartoon model, and further calculate the eyeball center point relative to the three-dimensional face cartoon model after the three-dimensional eyeball template is zoomed according to the zoom registration parameters according to the distance difference between the nearest feature point position of the pupil center point in the three-dimensional face cartoon model and the eyeball radius.
For example, the specific calculation manner for the position of the nearest feature point of the pupil center point in the three-dimensional human face cartoon model may be: determining the original nearest characteristic point of a pupil center point in the three-dimensional eyeball template in the three-dimensional face grid template; and determining the position of the nearest feature point of the pupil center point in the three-dimensional human face cartoon model based on the deformation position of the original nearest feature point in the three-dimensional human face cartoon model.
Specifically, as shown in fig. 3C, according to the left eye and the right eye in the three-dimensional cartoon grid template, the original nearest feature point position of the pupil center point in the three-dimensional eyeball template under any eyelid in the three-dimensional cartoon grid template before deformation can be calculated as: bi= barycentric, triangle }; wherein barycentric is the barycentric coordinate of the original nearest feature point, triangle is each grid vertex of a certain grid in which the original nearest feature point is located in the three-dimensional cartoon grid template, and then after the three-dimensional cartoon grid template is deformed to obtain a customized three-dimensional human face cartoon model, the position of the deformed nearest feature point in the deformed three-dimensional human face cartoon model can be calculated to be t'iBarycentricgq (t.triangle); the barycentric is the barycentric coordinate of an original nearest feature point of a pupil center point in the three-dimensional cartoon grid template before deformation, the t.triangle is the grid vertex of the pupil center point in the grid where the original nearest feature point in the three-dimensional cartoon grid template before deformation is located, and Q represents the specific deformation coordinate of the three-dimensional cartoon grid template. Then, the nearest feature point of the pupil center point in the three-dimensional human face cartoon model can be approximately used as the pupil center point, and the eyeball center point of the three-dimensional eyeball template relative to the three-dimensional human face cartoon model after being zoomed according to the zoom registration parameter can be calculated by calculating the difference between the position of the nearest feature point and the eyeball radius, and the eyeball center point of the three-dimensional eyeball template after being zoomed according to the zoom registration parameter can be the eyeball center point of the three-dimensional eyeball template after being zoomed according to the
S350, aiming at each feature point on the eyelid grid in the three-dimensional human face cartoon model, taking the distance between the eyeball center point after the three-dimensional eyeball template is translated and the feature point as a target, and calculating the sub-translation parameter corresponding to the feature point, wherein the distance is equal to the eyeball radius.
Illustratively, the method is used for each feature point p on each eyelid grid in the three-dimensional human face cartoon modeliIf the feature points are required not to be inserted into the scaled three-dimensional eyeball templateIn this case, the zoomed three-dimensional eyeball template can be exactly tangent to the feature point, that is, as shown in fig. 3D, the distance between the eyeball center point obtained by translating the zoomed three-dimensional eyeball template for a certain distance and the feature point is exactly equal to the radius of the zoomed eyeball, and at this time, if the sub-translation parameter corresponding to the feature point is set to x, the eyeball center point of the zoomed three-dimensional eyeball template after translation is set to be the eyeball center point
At this time, it can be determined
That is to say
Finishing to obtain:
the sub-translation parameter x corresponding to the characteristic point can be calculated; the sub-translation parameters corresponding to each feature point can be obtained according to the steps, and the maximum sub-translation parameter is obtained by comparing the sub-translation parameters
Thereby calculating the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model as
And the scaled registration parameter is siAnd subsequently, zooming and translating the three-dimensional eyeball template directly and sequentially according to the zooming registration parameters and the translating registration parameters, so that accurate eyeball registration in the three-dimensional human face cartoon model can be realized.
And S360, comparing the sub-translation parameters corresponding to each feature point on the eyelid grids in the three-dimensional human face cartoon model, and determining the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model.
According to the technical scheme provided by the embodiment, firstly, a three-dimensional cartoon grid template which is uniformly set is deformed according to a human face image to be customized to obtain a customized three-dimensional cartoon model, then, scaling and registration parameters of the preset three-dimensional eyeball template in the three-dimensional cartoon model and adapted to the deformation of the eyepit are calculated according to the deformation information of the eyepit in the three-dimensional cartoon model, so that the three-dimensional eyeball template which is scaled according to the scaling and registration parameters does not penetrate into an eyelid grid in the three-dimensional cartoon model as a target, and translation and registration parameters of the three-dimensional eyeball template in the three-dimensional cartoon model are calculated, thereby avoiding the condition that the eyeball in the three-dimensional cartoon model is not adapted, avoiding the deformation condition of a large number of model points in the universal three-dimensional grid model, and greatly reducing the calculation cost of eyeball registration in the three-dimensional cartoon model, the eyeball registration in the three-dimensional human face cartoon model can be realized by directly carrying out zooming and translation operations on the three-dimensional eyeball template in the three-dimensional human face cartoon model, the complexity of eyeball registration in the three-dimensional cartoon model is reduced, and the high efficiency of eyeball registration in the three-dimensional cartoon model is improved.
Example four
Fig. 4 is a schematic structural diagram of an eyeball registration apparatus based on a three-dimensional cartoon model according to a fourth embodiment of the present invention, specifically, as shown in fig. 4, the apparatus may include:
the cartoon model customizing module 410 is used for deforming the three-dimensional cartoon grid template according to the human face image to be customized to obtain a customized three-dimensional human face cartoon model;
a scaling and registration determining module 420, configured to calculate scaling and registration parameters of a preset three-dimensional eyeball template in the three-dimensional face cartoon model according to orbital deformation information in the three-dimensional face cartoon model;
a translation registration determining module 430, configured to calculate translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model, with a goal that no interpenetration exists between the three-dimensional eyeball template scaled according to the scaling registration parameters and eyelid meshes in the three-dimensional human face cartoon model.
According to the technical scheme provided by the embodiment, firstly, a three-dimensional cartoon grid template which is uniformly set is deformed according to a human face image to be customized to obtain a customized three-dimensional cartoon model, then, scaling and registration parameters of the preset three-dimensional eyeball template in the three-dimensional cartoon model and adapted to the deformation of the eyepit are calculated according to the deformation information of the eyepit in the three-dimensional cartoon model, so that the three-dimensional eyeball template which is scaled according to the scaling and registration parameters does not penetrate into an eyelid grid in the three-dimensional cartoon model as a target, and translation and registration parameters of the three-dimensional eyeball template in the three-dimensional cartoon model are calculated, thereby avoiding the condition that the eyeball in the three-dimensional cartoon model is not adapted, avoiding the deformation condition of a large number of model points in the universal three-dimensional grid model, and greatly reducing the calculation cost of eyeball registration in the three-dimensional cartoon model, the eyeball registration in the three-dimensional human face cartoon model can be realized by directly carrying out zooming and translation operations on the three-dimensional eyeball template in the three-dimensional human face cartoon model, the complexity of eyeball registration in the three-dimensional cartoon model is reduced, and the high efficiency of eyeball registration in the three-dimensional cartoon model is improved.
The eyeball registration device based on the three-dimensional cartoon model provided by the embodiment can be applied to the eyeball registration method based on the three-dimensional cartoon model provided by any embodiment, and has corresponding functions and beneficial effects.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a server according to a fifth embodiment of the present invention, and as shown in fig. 5, the server includes a processor 50, a storage device 51, and a communication device 52; the number of the processors 50 in the server may be one or more, and one processor 50 is taken as an example in fig. 5; the processor 50, the storage device 51 and the communication device 52 in the server may be connected by a bus or other means, and the bus connection is taken as an example in fig. 5.
The server provided by the embodiment can be used for executing the eyeball registration method based on the three-dimensional cartoon model provided by any embodiment, and has corresponding functions and beneficial effects.
EXAMPLE six
The sixth embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, can implement the eyeball registration method based on the three-dimensional cartoon model in any of the above embodiments. The method specifically comprises the following steps:
deforming the three-dimensional cartoon grid template according to the face image to be customized to obtain a customized three-dimensional face cartoon model;
calculating scaling and registration parameters of a preset three-dimensional eyeball template in the three-dimensional human face cartoon model according to orbital deformation information in the three-dimensional human face cartoon model;
and calculating the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model by taking the fact that no interpenetration exists between the three-dimensional eyeball template which is zoomed according to the zoom registration parameters and eyelid grids in the three-dimensional human face cartoon model as a target.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in the three-dimensional cartoon model based eyeball registration method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the eyeball registration apparatus based on the three-dimensional cartoon model, the included units and modules are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (12)
1. An eyeball registration method based on a three-dimensional cartoon model is characterized by comprising the following steps:
deforming the three-dimensional cartoon grid template according to the face image to be customized to obtain a customized three-dimensional face cartoon model;
calculating scaling and registration parameters of a preset three-dimensional eyeball template in the three-dimensional human face cartoon model according to orbital deformation information in the three-dimensional human face cartoon model;
and calculating the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model by taking the fact that no interpenetration exists between the three-dimensional eyeball template which is zoomed according to the zoom registration parameters and eyelid grids in the three-dimensional human face cartoon model as a target.
2. The method of claim 1, wherein calculating the translational registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model by aiming at the fact that no interpenetration exists between the three-dimensional eyeball template scaled according to the scaling registration parameters and eyelid grids in the three-dimensional human face cartoon model comprises:
calculating the eyeball center point and the eyeball radius of the three-dimensional eyeball template after scaling according to the scaling and registration parameters;
aiming at each feature point on an eyelid grid in the three-dimensional human face cartoon model, calculating a sub-translation parameter corresponding to the feature point according to the distance between the feature point and the eyeball center point and the difference between the feature point and the eyeball radius;
and comparing the sub-translation parameters corresponding to each feature point on the eyelid grid in the three-dimensional human face cartoon model, and determining the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model.
3. The method according to claim 2, wherein the calculating the eyeball center point and the eyeball radius of the three-dimensional eyeball template after being scaled according to the scaling and registration parameters comprises:
calculating the eyeball radius of the three-dimensional eyeball template after scaling according to the scaling and registration parameters according to the original eyeball radius of the three-dimensional eyeball template and the scaling and registration parameters;
and calculating the eyeball center point of the three-dimensional eyeball template after scaling according to the scaling and registration parameters according to the nearest feature point position of the pupil center point in the three-dimensional human face cartoon model and the eyeball radius.
4. The method according to claim 3, further comprising, before calculating the eyeball center point of the three-dimensional eyeball template after being scaled according to the scaling and registration parameters according to the nearest feature point position of the pupil center point in the three-dimensional human face cartoon model and the eyeball radius, the method further comprising:
determining an original nearest feature point of a pupil center point in the three-dimensional eyeball template in the three-dimensional face grid template;
and determining the position of the nearest feature point of the pupil center point in the three-dimensional human face cartoon model based on the deformation position of the original nearest feature point in the three-dimensional human face cartoon model.
5. The method of claim 2, wherein after the three-dimensional cartoon grid template is deformed according to the human face image to be customized to obtain the customized three-dimensional human face cartoon model, the method further comprises:
and determining eyelid grids in different eyelid states in the three-dimensional human face cartoon model, and extracting feature points in each eyelid grid.
6. The method according to claim 2, wherein the calculating, for each feature point on the eyelid grid in the three-dimensional human face cartoon model, the sub-translation parameter corresponding to the feature point according to the distance between the feature point and the eyeball center point and the difference between the eyeball radius comprises:
aiming at each feature point on an eyelid grid in the three-dimensional human face cartoon model, calculating a sub-translation parameter corresponding to the feature point by taking the distance between the eyeball center point after the three-dimensional eyeball template is translated and the feature point as a target, wherein the distance is equal to the eyeball radius.
7. The method according to any one of claims 1-6, further comprising, before calculating the scaled registration parameters of the preset three-dimensional eyeball template in the three-dimensional face cartoon model according to the orbital deformation information in the three-dimensional face cartoon model:
and extracting eyelid edge feature points in the three-dimensional human face cartoon model, and calculating corresponding orbit deformation information according to the deformation information of each eyelid edge feature point.
8. The method according to any one of claims 1 to 6, wherein the deforming the three-dimensional cartoon grid template according to the human face image to be customized to obtain the customized three-dimensional human face cartoon model comprises:
and sequentially deforming the three-dimensional cartoon grid template according to the human face appearance and the human face expression in the human face image to be customized to obtain a customized three-dimensional human face cartoon model.
9. The method according to any one of claims 1-6, further comprising, after calculating translational registration parameters of the three-dimensional eyeball template within the three-dimensional human face cartoon model:
and in the three-dimensional human face cartoon model, adopting the scaling registration parameters and the translation registration parameters to sequentially register the three-dimensional eyeball template.
10. An eyeball registration device based on a three-dimensional cartoon model is characterized by comprising:
the cartoon model customizing module is used for deforming the three-dimensional cartoon grid template according to the human face image to be customized to obtain a customized three-dimensional human face cartoon model;
the scaling and registration determining module is used for calculating scaling and registration parameters of a preset three-dimensional eyeball template in the three-dimensional face cartoon model according to orbital deformation information in the three-dimensional face cartoon model;
and the translation registration determining module is used for calculating the translation registration parameters of the three-dimensional eyeball template in the three-dimensional human face cartoon model by taking the fact that no interpenetration exists between the three-dimensional eyeball template which is scaled according to the scaling registration parameters and eyelid grids in the three-dimensional human face cartoon model as a target.
11. A server, characterized in that the server comprises:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the three-dimensional cartoon model based eyeball registration method of any one of claims 1-9.
12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the method for eye registration based on a three-dimensional cartoon model according to one of claims 1 to 9.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011565476.1A CN112581518A (en) | 2020-12-25 | 2020-12-25 | Eyeball registration method, device, server and medium based on three-dimensional cartoon model |
| PCT/CN2021/140798 WO2022135518A1 (en) | 2020-12-25 | 2021-12-23 | Eyeball registration method and apparatus based on three-dimensional cartoon model, and server and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011565476.1A CN112581518A (en) | 2020-12-25 | 2020-12-25 | Eyeball registration method, device, server and medium based on three-dimensional cartoon model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112581518A true CN112581518A (en) | 2021-03-30 |
Family
ID=75140730
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011565476.1A Pending CN112581518A (en) | 2020-12-25 | 2020-12-25 | Eyeball registration method, device, server and medium based on three-dimensional cartoon model |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112581518A (en) |
| WO (1) | WO2022135518A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113516755A (en) * | 2021-04-21 | 2021-10-19 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| WO2022135518A1 (en) * | 2020-12-25 | 2022-06-30 | 百果园技术(新加坡)有限公司 | Eyeball registration method and apparatus based on three-dimensional cartoon model, and server and medium |
| CN116824036A (en) * | 2023-04-19 | 2023-09-29 | 南京医科大学 | Eyeball three-dimensional modeling method and system based on three-dimensional imaging |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116188640B (en) * | 2022-12-09 | 2023-09-08 | 北京百度网讯科技有限公司 | Three-dimensional virtual image generation method, device, equipment and medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20020014844A (en) * | 2000-07-18 | 2002-02-27 | 최창석 | Three dimensional face modeling method |
| KR101757642B1 (en) * | 2016-07-20 | 2017-07-13 | (주)레벨소프트 | Apparatus and method for 3d face modeling |
| CN107330371A (en) * | 2017-06-02 | 2017-11-07 | 深圳奥比中光科技有限公司 | Acquisition methods, device and the storage device of the countenance of 3D facial models |
| CN109191508A (en) * | 2018-09-29 | 2019-01-11 | 深圳阜时科技有限公司 | A kind of simulation beauty device, simulation lift face method and apparatus |
| CN109191552A (en) * | 2018-08-16 | 2019-01-11 | Oppo广东移动通信有限公司 | Threedimensional model processing method, device, electronic equipment and storage medium |
| CN110585592A (en) * | 2019-07-31 | 2019-12-20 | 毕宏生 | Personalized electronic acupuncture device and generation method and generation device thereof |
| CN111028343A (en) * | 2019-12-16 | 2020-04-17 | 腾讯科技（深圳）有限公司 | Three-dimensional face model generation method, device, equipment and medium |
| CN111382618A (en) * | 2018-12-28 | 2020-07-07 | 广州市百果园信息技术有限公司 | Illumination detection method, device, equipment and storage medium for face image |
| CN111882627A (en) * | 2020-07-20 | 2020-11-03 | 广州市百果园信息技术有限公司 | Image processing method, video processing method, device, equipment and storage medium |
| CN112085835A (en) * | 2020-08-31 | 2020-12-15 | 腾讯科技（深圳）有限公司 | Three-dimensional cartoon face generation method and device, electronic equipment and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8437514B2 (en) * | 2007-10-02 | 2013-05-07 | Microsoft Corporation | Cartoon face generation |
| CN102262788A (en) * | 2010-05-24 | 2011-11-30 | 上海一格信息科技有限公司 | Method and device for processing interactive makeup information data of personal three-dimensional (3D) image |
| CN102157010A (en) * | 2011-05-25 | 2011-08-17 | 上海大学 | Method for realizing three-dimensional facial animation based on layered modeling and multi-body driving |
| CN104715447B (en) * | 2015-03-02 | 2019-08-30 | 百度在线网络技术（北京）有限公司 | Image composition method and device |
| CN108717719A (en) * | 2018-05-23 | 2018-10-30 | 腾讯科技（深圳）有限公司 | Generation method, device and the computer storage media of cartoon human face image |
| CN112581518A (en) * | 2020-12-25 | 2021-03-30 | 百果园技术(新加坡)有限公司 | Eyeball registration method, device, server and medium based on three-dimensional cartoon model |
-
2020
- 2020-12-25 CN CN202011565476.1A patent/CN112581518A/en active Pending
-
2021
- 2021-12-23 WO PCT/CN2021/140798 patent/WO2022135518A1/en active Application Filing
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20020014844A (en) * | 2000-07-18 | 2002-02-27 | 최창석 | Three dimensional face modeling method |
| KR101757642B1 (en) * | 2016-07-20 | 2017-07-13 | (주)레벨소프트 | Apparatus and method for 3d face modeling |
| CN107330371A (en) * | 2017-06-02 | 2017-11-07 | 深圳奥比中光科技有限公司 | Acquisition methods, device and the storage device of the countenance of 3D facial models |
| CN109191552A (en) * | 2018-08-16 | 2019-01-11 | Oppo广东移动通信有限公司 | Threedimensional model processing method, device, electronic equipment and storage medium |
| CN109191508A (en) * | 2018-09-29 | 2019-01-11 | 深圳阜时科技有限公司 | A kind of simulation beauty device, simulation lift face method and apparatus |
| CN111382618A (en) * | 2018-12-28 | 2020-07-07 | 广州市百果园信息技术有限公司 | Illumination detection method, device, equipment and storage medium for face image |
| CN110585592A (en) * | 2019-07-31 | 2019-12-20 | 毕宏生 | Personalized electronic acupuncture device and generation method and generation device thereof |
| CN111028343A (en) * | 2019-12-16 | 2020-04-17 | 腾讯科技（深圳）有限公司 | Three-dimensional face model generation method, device, equipment and medium |
| CN111882627A (en) * | 2020-07-20 | 2020-11-03 | 广州市百果园信息技术有限公司 | Image processing method, video processing method, device, equipment and storage medium |
| CN112085835A (en) * | 2020-08-31 | 2020-12-15 | 腾讯科技（深圳）有限公司 | Three-dimensional cartoon face generation method and device, electronic equipment and storage medium |
Non-Patent Citations (4)
| Title |
|---|
| PARAMA BAGCHI 等: "A method for nose-tip based 3D face registration using maximum intensity algorithm", 《ARXIV》, 30 September 2013 (2013-09-30), pages 1 - 5 * |
| TAKAMASA UTSU 等: "Remote Corneal Imaging by Integrating a 3D Face Mo del and an Eyeball Model", 《ETRA ’19》, 25 June 2019 (2019-06-25), pages 1 - 5, XP058753346, DOI: 10.1145/3314111.3319817 * |
| 刘俭: "基于形变模型的三维人脸建模关键技术研究", 《中国博士学位论文全文数据库 信息科技辑》, no. 1, 15 January 2019 (2019-01-15), pages 138 - 197 * |
| 梁伯均: "基于立体视觉的高精度三维人脸重建", 《万方学位论文》, 31 May 2012 (2012-05-31), pages 1 - 81 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022135518A1 (en) * | 2020-12-25 | 2022-06-30 | 百果园技术(新加坡)有限公司 | Eyeball registration method and apparatus based on three-dimensional cartoon model, and server and medium |
| CN113516755A (en) * | 2021-04-21 | 2021-10-19 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN113516755B (en) * | 2021-04-21 | 2024-05-17 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN116824036A (en) * | 2023-04-19 | 2023-09-29 | 南京医科大学 | Eyeball three-dimensional modeling method and system based on three-dimensional imaging |
| CN116824036B (en) * | 2023-04-19 | 2024-05-17 | 南京医科大学 | Eyeball three-dimensional modeling method and system based on three-dimensional imaging |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022135518A1 (en) | 2022-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111598998B (en) | Three-dimensional virtual model reconstruction method, three-dimensional virtual model reconstruction device, computer equipment and storage medium | |
| WO2020207270A1 (en) | Three-dimensional face reconstruction method, system and apparatus, and storage medium | |
| CN112581518A (en) | Eyeball registration method, device, server and medium based on three-dimensional cartoon model | |
| EP3323249B1 (en) | Three dimensional content generating apparatus and three dimensional content generating method thereof | |
| CN111710036B (en) | Method, device, equipment and storage medium for constructing three-dimensional face model | |
| WO2022095721A1 (en) | Parameter estimation model training method and apparatus, and device and storage medium | |
| EP3992919B1 (en) | Three-dimensional facial model generation method and apparatus, device, and medium | |
| CN113628327B (en) | Head three-dimensional reconstruction method and device | |
| CN107507217B (en) | Method and device for making certificate photo and storage medium | |
| CN107610202B (en) | Face image replacement method, device and storage medium | |
| CN107564080B (en) | Face image replacement system | |
| WO2022143645A1 (en) | Three-dimensional face reconstruction method and apparatus, device, and storage medium | |
| CN112419170A (en) | Method for training occlusion detection model and method for beautifying face image | |
| CN111652123B (en) | Image processing and image synthesizing method, device and storage medium | |
| CN111008935B (en) | Face image enhancement method, device, system and storage medium | |
| JP2022550948A (en) | 3D face model generation method, device, computer device and computer program | |
| CN114821675B (en) | Object processing method and system and processor | |
| WO2019075656A1 (en) | Image processing method and device, terminal, and storage medium | |
| WO2023066120A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| KR20230085931A (en) | Method and system for extracting color from face images | |
| Widanagamaachchi et al. | 3D face reconstruction from 2D images | |
| CN114373043A (en) | Head three-dimensional reconstruction method and equipment | |
| CN116681579A (en) | Real-time video face replacement method, medium and system | |
| US20220277586A1 (en) | Modeling method, device, and system for three-dimensional head model, and storage medium | |
| CN112561784B (en) | Image synthesis method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |