CN117274528A - Method and device for acquiring three-dimensional grid data, electronic equipment and readable storage medium - Google Patents
Method and device for acquiring three-dimensional grid data, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN117274528A CN117274528A CN202311118745.3A CN202311118745A CN117274528A CN 117274528 A CN117274528 A CN 117274528A CN 202311118745 A CN202311118745 A CN 202311118745A CN 117274528 A CN117274528 A CN 117274528A
- Authority
- CN
- China
- Prior art keywords
- target
- initial
- dimensional grid
- grid data
- expression base
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 230000014509 gene expression Effects 0.000 claims abstract description 246
- 238000013508 migration Methods 0.000 claims abstract description 39
- 230000005012 migration Effects 0.000 claims abstract description 39
- 238000012545 processing Methods 0.000 claims description 34
- 238000004590 computer program Methods 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 230000003190 augmentative effect Effects 0.000 claims description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 14
- 230000008921 facial expression Effects 0.000 description 12
- 238000006243 chemical reaction Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000004397 blinking Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2021—Shape modification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Geometry (AREA)
- Architecture (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The disclosure provides a method and a device for acquiring three-dimensional grid data, electronic equipment and a readable storage medium, relates to the technical field of artificial intelligence, and in particular relates to the technical fields of computer vision, virtual reality, large models and the like, and can be applied to scenes such as content generation of artificial intelligence. The method for acquiring the three-dimensional grid data comprises the following steps: acquiring a source expression base and initial three-dimensional grid data corresponding to a target character, wherein the initial three-dimensional grid data and the source expression base correspond to different technical specifications; performing deformation migration on the source expression base according to the initial three-dimensional grid data to obtain a target expression base corresponding to the target character, wherein the target expression base and the source expression base correspond to the same technical specification; and obtaining target three-dimensional grid data corresponding to the target person according to the initial three-dimensional grid data and the target expression base, wherein the target three-dimensional grid data and the source expression base correspond to the same technical specifications.
Description
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, virtual reality, large models and the like, and can be applied to scenes such as content generation of artificial intelligence. Provided are a method, a device, an electronic device and a readable storage medium for acquiring three-dimensional grid data.
Background
With the rapid development of artificial intelligence technology, more and more specifications for making expression data are presented. But before new specifications appear, there is also three-dimensional grid data acquired using the original specifications. Under the new technical specifications, three-dimensional grid data corresponding to the original technical specifications cannot be directly used for producing expression data conforming to the new technical specifications, so that the utilization rate of the three-dimensional grid data is low.
Disclosure of Invention
According to a first aspect of the present disclosure, there is provided a method for acquiring three-dimensional mesh data, including: acquiring a source expression base and initial three-dimensional grid data corresponding to a target character, wherein the initial three-dimensional grid data and the source expression base correspond to different technical specifications; performing deformation migration on the source expression base according to the initial three-dimensional grid data to obtain a target expression base corresponding to the target character, wherein the target expression base and the source expression base correspond to the same technical specification; and obtaining target three-dimensional grid data corresponding to the target person according to the initial three-dimensional grid data and the target expression base, wherein the target three-dimensional grid data and the source expression base correspond to the same technical specifications.
According to a second aspect of the present disclosure, there is provided an acquisition apparatus of three-dimensional mesh data, including: the system comprises an acquisition unit, a target character generation unit and a target character generation unit, wherein the acquisition unit is used for acquiring a source expression base and initial three-dimensional grid data corresponding to the target character, and the initial three-dimensional grid data and the source expression base correspond to different technical specifications; the migration unit is used for carrying out deformation migration on the source expression group according to the initial three-dimensional grid data to obtain a target expression group corresponding to the target character, wherein the target expression group and the source expression group correspond to the same technical specification; and the processing unit is used for obtaining target three-dimensional grid data corresponding to the target person according to the initial three-dimensional grid data and the target expression group, wherein the target three-dimensional grid data and the source expression group correspond to the same technical specifications.
According to a third aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method as described above.
According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method as described above.
According to the technical scheme, the initial three-dimensional grid data can be converted into the target three-dimensional grid data which accords with the specific technical specification, so that the purpose of more effectively utilizing the existing initial three-dimensional grid data which does not accord with the specific technical specification is achieved, the cost when the target three-dimensional grid data which accords with the specific technical specification is acquired is reduced, and the utilization rate of the initial three-dimensional grid data and the acquisition efficiency of the target three-dimensional grid data are improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure;
FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure;
FIG. 3 is a schematic diagram according to a third embodiment of the present disclosure;
FIG. 4 is a schematic diagram according to a fourth embodiment of the present disclosure;
FIG. 5 is a schematic diagram according to a fifth embodiment of the present disclosure;
fig. 6 is a block diagram of an electronic device for implementing a method of acquiring three-dimensional mesh data according to an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram according to a first embodiment of the present disclosure. As shown in fig. 1, the method for acquiring three-dimensional grid data in this embodiment specifically includes the following steps:
S101, acquiring a source expression group and initial three-dimensional grid data corresponding to a target character, wherein the initial three-dimensional grid data and the source expression group correspond to different technical specifications;
s102, performing deformation migration on the source expression group according to the initial three-dimensional grid data to obtain a target expression group corresponding to the target character, wherein the target expression group and the source expression group correspond to the same technical specification;
and S103, obtaining target three-dimensional grid data corresponding to the target person according to the initial three-dimensional grid data and the target expression group, wherein the target three-dimensional grid data and the source expression group correspond to the same technical specifications.
According to the method for acquiring the three-dimensional grid data, after acquiring the source expression base corresponding to different technical specifications and the initial three-dimensional grid data corresponding to the target character, firstly, performing deformation migration on the source expression base according to the initial three-dimensional grid data to obtain the target expression base corresponding to the target character, and then, according to the initial three-dimensional grid data and the target expression base, obtaining the target three-dimensional grid data corresponding to the target character, wherein the obtained target three-dimensional grid data corresponds to the same technical specifications as the source expression base.
In this embodiment, the source expression group acquired in S101 is executed, and the corresponding technical specification may be an ARKit (augmented reality toolkit) technical specification; the ARKit technical specification is a technical specification for capturing facial expressions of people in an ARKit development platform, and a large number of facial expression capturing technologies of people use the technical specification because the technical specification has good expression decoupling characteristics; it can be understood that the technical specification corresponding to the source expression group may be other technical specifications, which is not limited in this embodiment.
In the embodiment, when S101 is executed, a plurality of source expression groups may be obtained, for example, when the technical specification is an ARKit technical specification, 52 expression groups based on the ARKit may be obtained as source expression groups; different source expression groups correspond to different facial expressions of the face, for example, the facial expression corresponding to the source expression group 1 is a blinking of the left eye, the facial expression corresponding to the source expression group 2 is a visual lower part of the left eye, the facial expression corresponding to the source expression group 3 is a mouth-folding, and the like.
The initial three-dimensional grid data obtained in S101 is header three-dimensional grid data or face three-dimensional grid data, where the initial three-dimensional grid data may be one image frame or may be a video including multiple image frames; the three-dimensional mesh data is composed of a plurality of meshes (meshes), each of which is a triangle or a quadrangle, and each of which may include three vertexes (vertexes) as an example, so that the initial three-dimensional mesh data in this embodiment is composed of different vertexes, which is a set of vertexes.
In the embodiment, when S101 is executed to acquire initial three-dimensional mesh data corresponding to the target person, the implementation manner may be: acquiring the identification information of the target person, wherein the acquired identification information can be the name, the number and the like of the person; three-dimensional mesh data corresponding to the identification information is acquired from a three-dimensional mesh data set containing initial three-dimensional mesh data corresponding to different persons as initial three-dimensional mesh data corresponding to the target person.
In this embodiment, the target person may be one or more than one target person; that is, in this embodiment, the technical specification conversion may be performed on the initial three-dimensional mesh data corresponding to the specific person in the three-dimensional mesh data set, or may be performed on the initial three-dimensional mesh data corresponding to all the persons in the three-dimensional mesh data set.
It may be understood that the technical specifications corresponding to the three-dimensional grid data in the three-dimensional grid data set in this embodiment are different from the technical specifications corresponding to the source expression base, and different initial three-dimensional grid data corresponding to the target person may also correspond to different technical specifications; for ease of understanding, the technical specification corresponding to the source expression group may be referred to as a target technical specification, and the technical specification corresponding to the initial three-dimensional mesh data may be referred to as a source technical specification, which is not the same as the target technical specification.
Preferably, the technical specification corresponding to the three-dimensional mesh data in this embodiment may be a technical specification existing before the technical specification corresponding to the source expression base appears, for example, a three-dimensional deformation model (3 DMM) technical specification existing before the ARKit technical specification.
After executing S101 to obtain a source expression base and initial three-dimensional grid data corresponding to a target character, executing S102 to perform deformation migration on the source expression base according to the initial three-dimensional grid data to obtain a target expression base corresponding to the target character; the obtained target expression base corresponds to the same technical specification as the source expression base.
Because the face corresponding to the source expression base is different from the face corresponding to the initial three-dimensional grid data, the face corresponding to the source expression base needs to be converted into the face corresponding to the initial three-dimensional grid data, and therefore accuracy of the target three-dimensional grid data obtained according to the target expression base in the follow-up process is improved.
In the embodiment, when performing S102 to perform deformation migration on the source expression group according to the initial three-dimensional grid data to obtain the target expression group corresponding to the target character, the implementation manner may be as follows: carrying out re-topology processing on the source topology structure of the source expression group according to the target topology structure of the initial three-dimensional grid data to obtain an initial expression group, wherein the obtained topology structure of the initial expression group is the target topology structure; and performing deformation migration on the initial expression base according to the initial three-dimensional grid data to obtain a target expression base corresponding to the target character.
In this embodiment, the topology is used to characterize the structure of three-dimensional mesh data or expression base; the re-topology processing refers to changing the topology structure of the source expression base, so that the topology structure of the initial expression base obtained through the re-topology processing is identical to the topology structure of the three-dimensional grid data.
That is, since the topology of the initial three-dimensional mesh data may be varied according to different head or face models, and the topology of the initial three-dimensional mesh data may not be the same as the topology of the source expression base, it is necessary to convert the topology of the source expression base into the topology of the initial three-dimensional mesh data, thereby improving accuracy of deformation migration.
In the implementation of S102, the present embodiment may use the existing heavy topology tool (e.g. wrap4 d) to perform topology conversion; in the present embodiment, when S102 is executed, the target expression group corresponding to the target person may be obtained using the existing deformation migration (Deformation Transfer, DT) algorithm.
The present embodiment may further include the following when S102 is executed: under the condition that the target topological structure of the initial three-dimensional grid data is inconsistent with the source topological structure of the source expression group, carrying out re-topology processing on the source topological structure of the source expression group according to the target topological structure of the initial three-dimensional grid data to obtain the initial expression group.
That is, the embodiment can also compare the target topology structure with the source topology structure before the re-topology processing, and further execute the re-topology processing only under the condition that the target topology structure and the source topology structure are inconsistent, thereby improving the accuracy of the re-topology processing and avoiding the waste of computing resources.
In addition, if the embodiment executes S102 to determine that the target topology structure of the initial three-dimensional grid data is consistent with the source topology structure of the source expression group, the source expression group is directly used as the initial expression group, and then the operation of deformation migration is executed.
After executing S102 to obtain a target expression group corresponding to a target person, executing S103 to obtain target three-dimensional grid data corresponding to the target person according to the initial three-dimensional grid data and the target expression group; the obtained target three-dimensional grid data and the source expression base correspond to the same technical specifications.
Specifically, in the embodiment, when executing S103 to obtain the target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression group, the implementation manner may be as follows: acquiring camera parameters according to the size of the model in the initial three-dimensional grid data; according to the model pose in the initial three-dimensional grid data, obtaining model pose parameters, wherein the model pose reflects whether the model is askew, low, raised, normal and the like; according to the model expression of the initial three-dimensional grid data, obtaining expression base weight parameters corresponding to target expression bases, wherein under the condition that the number of the target expression bases is multiple, the embodiment can respectively obtain the expression base weight parameters corresponding to different target expression bases; and obtaining target three-dimensional grid data corresponding to the target character according to the camera parameters, the model pose parameters, the target expression base and the expression base parameters corresponding to the target expression base.
For example, if the target expression group includes expression group 1, expression group 2 and expression group 3, if the expression group weight parameter corresponding to expression group 1 is determined to be 0.2, the expression group weight parameter corresponding to expression group 2 is determined to be 0.3, and the expression group weight parameter corresponding to expression group 3 is determined to be 0.5 according to the model expression, then the embodiment may obtain the target three-dimensional grid data according to the calculation result, the camera parameter and the model pose parameter after obtaining the calculation result of (expression group 1×0.2+expression group 2×0.3+expression group 3×0.5) when executing S103.
It can be understood that, if the initial three-dimensional grid data is a video including a plurality of image frames, in the embodiment, when executing S103, corresponding camera parameters, model pose parameters and expression base weight parameters may be respectively obtained for the three-dimensional grid data corresponding to each image frame in the initial three-dimensional grid data, so as to obtain target three-dimensional grid data corresponding to the image frame, and then the target three-dimensional grid data corresponding to all the image frames may be formed into a video.
That is, in this embodiment, camera parameters, model pose parameters and expression base weight parameters are obtained according to the initial three-dimensional mesh data, and then the target expression base corresponding to the target person is combined to obtain the target three-dimensional mesh data corresponding to the target person, so that the conversion of technical specifications is realized, the target three-dimensional mesh data and the source expression base correspond to the same technical specifications, and the application range of the target three-dimensional mesh data is improved.
The present embodiment may further include the following after performing S103 to obtain the target three-dimensional mesh data corresponding to the target person: the target texture map is obtained, and the texture map corresponding to the target scene can be used as the target texture map in the embodiment; and obtaining the virtual image corresponding to the target person according to the target texture mapping and the target three-dimensional grid data corresponding to the target person.
That is, the embodiment may further obtain the avatar corresponding to the target character by combining the obtained target texture map after obtaining the target three-dimensional mesh data, so that the obtained avatar also conforms to the technical specification corresponding to the source expression group, thereby further improving the application range of the avatar.
Fig. 2 is a schematic diagram according to a second embodiment of the present disclosure. A flowchart of the present embodiment when acquiring target three-dimensional mesh data is shown in fig. 2: s201, acquiring a three-dimensional grid data set and a source expression base based on ARkit, wherein the three-dimensional grid data set comprises initial three-dimensional grid data corresponding to different characters; s202, carrying out heavy topology processing, namely converting a topological structure of a source expression group based on ARkit into a topological structure of a three-dimensional grid data set to obtain an initial expression group based on ARkit, which accords with the topological structure of the three-dimensional grid data set; s203, performing deformation migration processing, namely performing deformation migration on the initial expression base based on the ARkit according to the initial three-dimensional grid data corresponding to the target character to obtain a target expression base based on the ARkit corresponding to the target character; and S204, fitting and resolving, namely obtaining target three-dimensional grid data corresponding to the target person based on the ARkit according to the initial three-dimensional grid data corresponding to the target person and the target expression base based on the ARkit corresponding to the target person.
Fig. 3 is a schematic diagram according to a third embodiment of the present disclosure. A flow chart of this embodiment when fitting solutions are performed is shown in fig. 3: s301, initializing all parameters including camera parameters, model pose parameters and expression base weight parameters; s302, optimizing camera parameters and model pose parameters according to the initial three-dimensional grid data, namely, the camera parameters and model pose parameters after initialization correspond to the model size and model pose in the initial three-dimensional grid data; s303, optimizing expression base weight parameters according to the initial three-dimensional grid data so that the expression base weight parameters are aligned with the model expressions in the initial three-dimensional grid data; s304, fine tuning and optimizing the optimized camera parameters, model pose parameters and expression base weight parameters, wherein fine tuning and optimizing can be performed manually; and S305, obtaining target three-dimensional grid data corresponding to the target person according to the parameters after fine adjustment and optimization and the target expression group.
Fig. 4 is a schematic diagram according to a fourth embodiment of the present disclosure. FIG. 4 is a schematic diagram of initial three-dimensional mesh data and target three-dimensional mesh data, in which meshes have been filled in order to enhance display; as can be seen from fig. 4, after the conversion of the technical specification of the three-dimensional grid data is completed, the expression of the target person does not change greatly, and the conversion effect is good.
Fig. 5 is a schematic diagram according to a fifth embodiment of the present disclosure. As shown in fig. 5, the apparatus 500 for acquiring three-dimensional mesh data of the present embodiment includes:
an obtaining unit 501, configured to obtain a source expression group and initial three-dimensional grid data corresponding to a target person, where the initial three-dimensional grid data corresponds to different technical specifications with respect to the source expression group;
the migration unit 502 is configured to perform deformation migration on the source expression group according to the initial three-dimensional grid data, so as to obtain a target expression group corresponding to the target character, where the target expression group corresponds to the same technical specification as the source expression group;
the processing unit 503 is configured to obtain, according to the initial three-dimensional mesh data and the target expression group, target three-dimensional mesh data corresponding to the target person, where the target three-dimensional mesh data corresponds to the same technical specification as the source expression group.
The source expression group acquired by the acquiring unit 501 may be an ARKit technical specification; the ARKit technical specification is a technical specification for capturing facial expressions of people in an ARKit development platform, and a large number of facial expression capturing technologies of people use the technical specification because the technical specification has good expression decoupling characteristics; it can be understood that the technical specification corresponding to the source expression group may be other technical specifications, which is not limited in this embodiment.
The acquiring unit 501 may acquire a plurality of source expression groups, for example, when the technical specification is an ARKit technical specification, 52 expression groups based on the ARKit may be acquired as source expression groups; different source expression groups correspond to different facial expressions of the face, for example, the facial expression corresponding to the source expression group 1 is a blinking of the left eye, the facial expression corresponding to the source expression group 2 is a visual lower part of the left eye, the facial expression corresponding to the source expression group 3 is a mouth-folding, and the like.
The initial three-dimensional grid data acquired by the acquiring unit 501 is three-dimensional grid data of a human head or three-dimensional grid data of a human face, and the initial three-dimensional grid data can be one image frame or a video containing a plurality of image frames; the three-dimensional mesh data is composed of a plurality of meshes (meshes), each of which is a triangle or a quadrangle, and each of which may include three vertexes (vertexes) as an example, so that the initial three-dimensional mesh data in this embodiment is composed of different vertexes, which is a set of vertexes.
The obtaining unit 501 may adopt the following implementation manners when obtaining the initial three-dimensional mesh data corresponding to the target person: acquiring identification information of a target person; three-dimensional mesh data corresponding to the identification information is acquired from a three-dimensional mesh data set containing initial three-dimensional mesh data corresponding to different persons as initial three-dimensional mesh data corresponding to the target person.
In this embodiment, the target person may be one or more than one target person; that is, the acquisition unit 501 may perform the conversion of the technical specification on the initial three-dimensional mesh data of the three-dimensional mesh data set corresponding to the specific person, or may perform the conversion of the technical specification on the initial three-dimensional mesh data of all the persons in the three-dimensional mesh data set.
It may be understood that the technical specifications corresponding to the three-dimensional grid data in the three-dimensional grid data set in this embodiment are different from the technical specifications corresponding to the source expression base, and different initial three-dimensional grid data corresponding to the target person may also correspond to different technical specifications; for ease of understanding, the technical specification corresponding to the source expression group may be referred to as a target technical specification, and the technical specification corresponding to the initial three-dimensional mesh data may be referred to as a source technical specification, which is not the same as the target technical specification.
Preferably, the technical specification corresponding to the three-dimensional mesh data in this embodiment may be a technical specification existing before the technical specification corresponding to the source expression base appears, for example, a three-dimensional deformation model (3 DMM) technical specification existing before the ARKit technical specification.
In this embodiment, after the obtaining unit 501 obtains the source expression base and the initial three-dimensional grid data corresponding to the target character, the migration unit 502 performs deformation migration on the source expression base according to the initial three-dimensional grid data to obtain the target expression base corresponding to the target character; the obtained target expression base corresponds to the same technical specification as the source expression base.
Because the face corresponding to the source expression base is different from the face corresponding to the initial three-dimensional grid data, the face corresponding to the source expression base needs to be converted into the face corresponding to the initial three-dimensional grid data, and therefore accuracy of the target three-dimensional grid data obtained according to the target expression base in the follow-up process is improved.
The migration unit 502 performs deformation migration on the source expression base according to the initial three-dimensional grid data, and when obtaining a target expression base corresponding to the target character, the implementation manner may be as follows: carrying out re-topology processing on the source topology structure of the source expression group according to the target topology structure of the initial three-dimensional grid data to obtain an initial expression group, wherein the obtained topology structure of the initial expression group is the target topology structure; and performing deformation migration on the initial expression base according to the initial three-dimensional grid data to obtain a target expression base corresponding to the target character.
In this embodiment, the topology is used to characterize the structure of three-dimensional mesh data or expression base; the re-topology processing refers to changing the topology structure of the source expression base, so that the topology structure of the initial expression base obtained through the re-topology processing is identical to the topology structure of the three-dimensional grid data.
That is, since the topology of the initial three-dimensional mesh data may be varied according to different head or face models, and the topology of the initial three-dimensional mesh data may not be the same as the topology of the source expression base, it is necessary to convert the topology of the source expression base into the topology of the initial three-dimensional mesh data, thereby improving accuracy of deformation migration.
The migration unit 502 may use existing heavy topology tools (e.g., wrap4 d) for topology conversion; the migration unit 502 may obtain the target expression group corresponding to the target character using an existing deformation migration (Deformation Transfer, DT) algorithm.
The migration unit 502 may also contain the following: under the condition that the target topological structure of the initial three-dimensional grid data is inconsistent with the source topological structure of the source expression group, carrying out re-topology processing on the source topological structure of the source expression group according to the target topological structure of the initial three-dimensional grid data to obtain the initial expression group.
That is, the migration unit 502 may compare the target topology structure with the source topology structure before the re-topology processing, and further execute the re-topology processing only if the target topology structure and the source topology structure are inconsistent, thereby improving accuracy of the re-topology processing and avoiding waste of computing resources.
In addition, if the migration unit 502 determines that the target topology structure of the initial three-dimensional grid data is consistent with the source topology structure of the source expression group, the source expression group is directly used as the initial expression group, and then the deformation migration operation is performed.
In this embodiment, after the migration unit 502 obtains the target expression group corresponding to the target person, the processing unit 503 obtains the target three-dimensional grid data corresponding to the target person according to the initial three-dimensional grid data and the target expression group; the obtained target three-dimensional grid data and the source expression base correspond to the same technical specifications.
Specifically, when the processing unit 503 obtains the target three-dimensional mesh data corresponding to the target person according to the initial three-dimensional mesh data and the target expression group, the implementation manner may be as follows: acquiring camera parameters according to the size of the model in the initial three-dimensional grid data; obtaining model pose parameters according to the model pose in the initial three-dimensional grid data; according to the model expression of the initial three-dimensional grid data, obtaining expression base weight parameters corresponding to target expression bases, wherein under the condition that the number of the target expression bases is multiple, the embodiment can respectively obtain the expression base weight parameters corresponding to different target expression bases; and obtaining target three-dimensional grid data corresponding to the target character according to the camera parameters, the model pose parameters, the target expression base and the expression base parameters corresponding to the target expression base.
It may be appreciated that, if the initial three-dimensional grid data is a video including a plurality of image frames, the processing unit 503 may obtain, for the three-dimensional grid data corresponding to each image frame in the initial three-dimensional grid data, a corresponding camera parameter, a model pose parameter, and an expression base weight parameter, so as to obtain target three-dimensional grid data corresponding to the image frame.
That is, the processing unit 503 obtains the camera parameters, the model pose parameters and the expression base weight parameters according to the initial three-dimensional grid data, and further combines the target expression base corresponding to the target person to obtain the target three-dimensional grid data corresponding to the target person, thereby realizing the conversion of the technical specifications, enabling the target three-dimensional grid data to correspond to the same technical specifications as the source expression base, and further improving the application range of the target three-dimensional grid data.
The apparatus 500 for acquiring three-dimensional mesh data of the present embodiment may further include a generating unit 504 configured to perform: obtaining a target texture map; and obtaining the virtual image corresponding to the target person according to the target texture mapping and the target three-dimensional grid data corresponding to the target person.
That is, after the target three-dimensional mesh data is acquired, the generating unit 504 may combine the acquired target texture map to obtain an avatar corresponding to the target character, so that the obtained avatar also conforms to the technical specification corresponding to the source expression group, thereby enhancing the application range of the avatar.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
As shown in fig. 6, a block diagram of an electronic device of a method of acquiring three-dimensional mesh data according to an embodiment of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the device 600 may also be stored. The computing unit 601, ROM602, and RAM603 are connected to each other by a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Various components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, mouse, etc.; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 601 performs the respective methods and processes described above, for example, the acquisition method of three-dimensional mesh data. For example, in some embodiments, the method of acquiring three-dimensional grid data may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 608.
In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM602 and/or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the above-described three-dimensional mesh data acquisition method may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the method of acquiring three-dimensional mesh data in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here can be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable three-dimensional grid data acquisition device such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a presentation device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for presenting information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.
Claims (17)
1. A method for acquiring three-dimensional grid data comprises the following steps:
acquiring a source expression base and initial three-dimensional grid data corresponding to a target character, wherein the initial three-dimensional grid data and the source expression base correspond to different technical specifications;
performing deformation migration on the source expression base according to the initial three-dimensional grid data to obtain a target expression base corresponding to the target character, wherein the target expression base and the source expression base correspond to the same technical specification;
and obtaining target three-dimensional grid data corresponding to the target person according to the initial three-dimensional grid data and the target expression base, wherein the target three-dimensional grid data and the source expression base correspond to the same technical specifications.
2. The method of claim 1, wherein the obtaining a source expression group comprises:
and obtaining an expression base based on an augmented reality toolkit ARkit as the source expression base.
3. The method of claim 1, wherein the deforming migration of the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target character comprises:
carrying out re-topology processing on the source topology structure of the source expression group according to the target topology structure of the initial three-dimensional grid data to obtain an initial expression group, wherein the topology structure of the initial expression group is the target topology structure;
And performing deformation migration on the initial expression base according to the initial three-dimensional grid data to obtain the target expression base corresponding to the target character.
4. The method of claim 3, wherein the re-topology processing the source topology of the source expression base according to the target topology of the initial three-dimensional mesh data, to obtain an initial expression base comprises:
and under the condition that the target topological structure is inconsistent with the source topological structure, carrying out re-topology processing on the source topological structure according to the target topological structure to obtain the initial expression group.
5. The method of claim 1, wherein the obtaining target three-dimensional mesh data corresponding to the target person from the initial three-dimensional mesh data and the target expression base comprises:
acquiring camera parameters according to the model size in the initial three-dimensional grid data;
obtaining model pose parameters according to the model pose in the initial three-dimensional grid data;
according to the model expression of the initial three-dimensional grid data, obtaining expression base weight parameters corresponding to the target expression base;
and obtaining the target three-dimensional grid data corresponding to the target character according to the camera parameters, the model pose parameters, the target expression base and the expression base parameters corresponding to the target expression base.
6. The method of claim 1, wherein the acquiring initial three-dimensional mesh data corresponding to the target person comprises:
acquiring the identification information of the target person;
and acquiring three-dimensional grid data corresponding to the identification information from the three-dimensional grid data set as the initial three-dimensional grid data corresponding to the target person.
7. The method of claim 1, further comprising,
obtaining a target texture map;
and obtaining the virtual image corresponding to the target person according to the target texture mapping and the target three-dimensional grid data corresponding to the target person.
8. An acquisition apparatus of three-dimensional mesh data, comprising:
the system comprises an acquisition unit, a target character generation unit and a target character generation unit, wherein the acquisition unit is used for acquiring a source expression base and initial three-dimensional grid data corresponding to the target character, and the initial three-dimensional grid data and the source expression base correspond to different technical specifications;
the migration unit is used for carrying out deformation migration on the source expression group according to the initial three-dimensional grid data to obtain a target expression group corresponding to the target character, wherein the target expression group and the source expression group correspond to the same technical specification;
and the processing unit is used for obtaining target three-dimensional grid data corresponding to the target person according to the initial three-dimensional grid data and the target expression group, wherein the target three-dimensional grid data and the source expression group correspond to the same technical specifications.
9. The apparatus of claim 8, wherein the obtaining unit, when obtaining the source expression base, specifically performs:
and obtaining an expression base based on an augmented reality toolkit ARkit as the source expression base.
10. The apparatus of claim 8, wherein the migration unit, when performing deformation migration on the source expression base according to the initial three-dimensional mesh data to obtain a target expression base corresponding to the target person, specifically performs:
carrying out re-topology processing on the source topology structure of the source expression group according to the target topology structure of the initial three-dimensional grid data to obtain an initial expression group, wherein the topology structure of the initial expression group is the target topology structure;
and performing deformation migration on the initial expression base according to the initial three-dimensional grid data to obtain the target expression base corresponding to the target character.
11. The apparatus of claim 10, wherein the migration unit performs, when performing a re-topology process on the source topology of the source expression base according to the target topology of the initial three-dimensional mesh data to obtain an initial expression base, specific execution:
and under the condition that the target topological structure is inconsistent with the source topological structure, carrying out re-topology processing on the source topological structure according to the target topological structure to obtain the initial expression group.
12. The apparatus of claim 8, wherein the processing unit, when obtaining target three-dimensional mesh data corresponding to the target person from the initial three-dimensional mesh data and the target expression base, specifically performs:
acquiring camera parameters according to the model size in the initial three-dimensional grid data;
obtaining model pose parameters according to the model pose in the initial three-dimensional grid data;
according to the model expression of the initial three-dimensional grid data, obtaining expression base weight parameters corresponding to the target expression base;
and obtaining the target three-dimensional grid data corresponding to the target character according to the camera parameters, the model pose parameters, the target expression base and the expression base parameters corresponding to the target expression base.
13. The apparatus of claim 8, wherein the acquisition unit, when acquiring the initial three-dimensional mesh data corresponding to the target person, specifically performs:
acquiring the identification information of the target person;
and acquiring three-dimensional grid data corresponding to the identification information from the three-dimensional grid data set as the initial three-dimensional grid data corresponding to the target person.
14. The apparatus of claim 8, further comprising a generation unit to perform:
obtaining a target texture map;
and obtaining the virtual image corresponding to the target person according to the target texture mapping and the target three-dimensional grid data corresponding to the target person.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311118745.3A CN117274528A (en) | 2023-08-31 | 2023-08-31 | Method and device for acquiring three-dimensional grid data, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311118745.3A CN117274528A (en) | 2023-08-31 | 2023-08-31 | Method and device for acquiring three-dimensional grid data, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117274528A true CN117274528A (en) | 2023-12-22 |
Family
ID=89218683
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311118745.3A Pending CN117274528A (en) | 2023-08-31 | 2023-08-31 | Method and device for acquiring three-dimensional grid data, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117274528A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114708636A (en) * | 2022-04-01 | 2022-07-05 | 成都市谛视科技有限公司 | Dense face grid expression driving method, device and medium |
| WO2022143354A1 (en) * | 2020-12-29 | 2022-07-07 | 百果园技术(新加坡)有限公司 | Face generation method and apparatus for virtual object, and device and readable storage medium |
| CN116524165A (en) * | 2023-05-29 | 2023-08-01 | 北京百度网讯科技有限公司 | Migration method, migration device, migration equipment and migration storage medium for three-dimensional expression model |
-
2023
- 2023-08-31 CN CN202311118745.3A patent/CN117274528A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022143354A1 (en) * | 2020-12-29 | 2022-07-07 | 百果园技术(新加坡)有限公司 | Face generation method and apparatus for virtual object, and device and readable storage medium |
| CN114708636A (en) * | 2022-04-01 | 2022-07-05 | 成都市谛视科技有限公司 | Dense face grid expression driving method, device and medium |
| CN116524165A (en) * | 2023-05-29 | 2023-08-01 | 北京百度网讯科技有限公司 | Migration method, migration device, migration equipment and migration storage medium for three-dimensional expression model |
Non-Patent Citations (1)
| Title |
|---|
| 叶亚男;迟静;于志平;战玉丽;张彩明;: "基于改进CycleGan模型和区域分割的表情动画合成", 计算机科学, no. 09, 31 December 2020 (2020-12-31) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11823306B2 (en) | Virtual image generation method and apparatus, electronic device and storage medium | |
| CN114820905B (en) | Virtual image generation method and device, electronic equipment and readable storage medium | |
| CN115409933B (en) | Multi-style texture mapping generation method and device | |
| CN113409430B (en) | Drivable three-dimensional character generation method, drivable three-dimensional character generation device, electronic equipment and storage medium | |
| CN114723888B (en) | Three-dimensional hair model generation method, device, equipment, storage medium and product | |
| CN112581573A (en) | Avatar driving method, apparatus, device, medium, and program product | |
| CN115147265A (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| US20220292795A1 (en) | Face image processing method, electronic device, and storage medium | |
| CN114792355B (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| CN114120413A (en) | Model training method, image synthesis method, device, equipment and program product | |
| CN112562043B (en) | Image processing method and device and electronic equipment | |
| CN116524165B (en) | Migration method, migration device, migration equipment and migration storage medium for three-dimensional expression model | |
| CN115423919B (en) | Image rendering method, device, equipment and storage medium | |
| CN114078184B (en) | Data processing method, device, electronic equipment and medium | |
| CN117274528A (en) | Method and device for acquiring three-dimensional grid data, electronic equipment and readable storage medium | |
| CN113870428A (en) | Scene map generation method, related device and computer program product | |
| CN113903071A (en) | Face recognition method and device, electronic equipment and storage medium | |
| CN113887435A (en) | Face image processing method, device, equipment, storage medium and program product | |
| CN113781653A (en) | Object model generation method and device, electronic equipment and storage medium | |
| CN114037814B (en) | Data processing method, device, electronic equipment and medium | |
| CN113538644B (en) | Character dynamic video generation method, device, electronic equipment and storage medium | |
| CN113470131B (en) | Sea surface simulation image generation method and device, electronic equipment and storage medium | |
| CN115953553B (en) | Avatar generation method, apparatus, electronic device, and storage medium | |
| EP4120202A1 (en) | Image processing method and apparatus, and electronic device | |
| CN116882482A (en) | Training of virtual image generation model and virtual image generation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |