CN116228951A - Expression data processing method and device for digital virtual object - Google Patents
Expression data processing method and device for digital virtual object Download PDFInfo
- Publication number
- CN116228951A CN116228951A CN202310452655.1A CN202310452655A CN116228951A CN 116228951 A CN116228951 A CN 116228951A CN 202310452655 A CN202310452655 A CN 202310452655A CN 116228951 A CN116228951 A CN 116228951A
- Authority
- CN
- China
- Prior art keywords
- matrix
- vertex
- expression data
- facial expression
- constructing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Architecture (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Processing Or Creating Images (AREA)
- Image Generation (AREA)
Abstract
The invention relates to an expression data processing method and device of a digital virtual object, wherein the method comprises the following steps: acquiring facial expression data of a digital virtual object in real time; constructing and initializing a vertex diagonal matrix of the facial expression data; constructing and initializing a vertex matrix of the facial expression data; multiplying the initialized vertex diagonal matrix and the vertex matrix to generate a facial expression data matrix, and taking out the optimal value of each expression vertex of the facial expression data from the facial expression data matrix as new facial expression data; the new facial expression data is utilized to update the facial expression data, and the new expression is rendered according to the updated facial expression data, so that the expression of the virtual object can be quickly and massively repaired on line in real time, and the expression is more natural.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to an expression data processing method and device of a digital virtual object.
Background
This section introduces the reader with background information that may be relevant to various aspects of the inventive embodiments and is believed to provide the reader with useful background information to aid the reader in a better understanding of the various aspects of the inventive embodiments. It is to be understood, therefore, that the description in this section is for the purpose of illustration and not an admission that the prior art is prior art.
In recent years, the universe concept is rolled up globally, and with the vigorous development of the universe, large enterprises in China accelerate the track layout, and related content ecologies of different application scenes are energized through the universe. Metauniverse (Metaverse), which is a digital living space constructed by human beings using digital technology, is a virtual world mapped or surpassed by the real world and can interact with the real world, and is provided with a novel social system. A person in reality can have his own personalized avatar in the meta-universe. The manner in which humans interact with virtual worlds is also constantly changing. The face three-dimensional model of the user is generally obtained through a face reconstruction technology and then applied to the fields of games, social contact, film and television and the like, so that entertainment experience of people is improved.
In the existing mode, a face three-dimensional model is made by pinching the face, namely, a large number of parameter sliding rods with adjustable face parts of the person are provided in application, the user can adjust the shape, the position and the details of the facial features of the corresponding digital virtual object by operating the sliding rods, and the method can meet the requirement that the user can accurately operate the features of the person, such as eyes, nose tips, faces and the like, so that personalized digital virtual object creation is realized. But the consistency between the expression of the digital virtual object after face pinching and the standard expression is low, and the expression defect can occur when the expression driving is carried out, so that the expression is not very natural.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides the expression data processing method and the device for the digital virtual object, which can repair the expression of the virtual object on line in real time rapidly and in a large scale, and make the expression more natural.
In a first aspect, the present invention provides an expression data processing method of a digital virtual object, including:
acquiring facial expression data of a digital virtual object in real time;
constructing and initializing a vertex diagonal matrix of the facial expression data;
constructing and initializing a vertex matrix of the facial expression data;
multiplying the initialized vertex diagonal matrix and the vertex matrix to generate a facial expression data matrix, and taking out the optimal value of each expression vertex of the facial expression data from the facial expression data matrix as new facial expression data;
and updating the facial expression data by utilizing the new facial expression data, and rendering a new expression according to the updated facial expression data.
In a second aspect, an embodiment of the present invention further provides an apparatus for processing expression data of a digital virtual object, including:
an information acquisition unit for acquiring facial expression data of the digital virtual object in real time;
a vertex diagonal matrix initializing unit for constructing and initializing a vertex diagonal matrix of the facial expression data;
a second matrix initializing unit for constructing and initializing a vertex matrix of the facial expression data;
the data operation unit is used for carrying out multiplication operation on the initialized vertex diagonal matrix and the vertex matrix to generate a facial expression data matrix, and taking out the optimal value of each expression vertex of the facial expression data from the facial expression data matrix as new facial expression data;
and the rendering unit is used for updating the facial expression data by utilizing the new facial expression data and rendering the new expression according to the updated facial expression data.
Compared with the prior art, the method and the device for processing the expression data of the digital virtual object solve the problem of low efficiency in the conventional application of expression driving restoration of facial expression deformation and the like, and determine the optimal value of each expression vertex of the facial expression data as new facial expression data by carrying out data transformation on the data vertices of the facial expression data, so that the expression of the virtual object can be quickly and massively restored on line in real time according to the new facial expression data, and expression can be more natural.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief description will be given below of the drawings required for the embodiments or the prior art descriptions, and it is obvious that the drawings in the following description are some embodiments of the present invention, and that other drawings may be obtained according to these drawings without the need for inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for processing expression data of a digital virtual object according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an expression data processing apparatus for a digital virtual object according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.
The expression data processing method of the digital virtual object provided by the embodiment of the application can be implemented by various electronic devices, for example, can be implemented by a terminal alone, can be implemented by a cloud server alone, and can be implemented by the terminal and the cloud server in a cooperative manner. For example, the cloud server alone executes the expression data processing method of the digital virtual object provided in the embodiment of the present application, or the terminal sends an expression data processing request for the virtual object to the cloud server, and the cloud server executes the expression data processing method of the digital virtual object provided in the embodiment of the present application according to the received expression data processing request. The embodiments of the present application may be applied to various scenarios including, but not limited to, cloud technology, artificial intelligence, intelligent transportation, assisted driving, and the like.
As shown in fig. 1, the expression data processing method for the digital virtual object provided by the embodiment of the application can repair the expression of the virtual object on line in real time, rapidly and in a large scale, so that the expression is more natural. The method specifically comprises the following steps:
s1, acquiring facial expression data of a digital virtual object in real time;
in an embodiment of the present invention, the facial expression data includes: expression vertex data and adjacent triangle data, wherein the expression vertex data comprises moving vertex data, fixed vertex data, unlabeled other vertex data and the like. Expression vertices are vertices in the face model mesh that relate to expressions when the digital virtual object is modeled, and changes in expression vertex data reflect changes in facial expressions of the digital virtual object. In order to fine-adjust the repairing expression, the relatively large vertex of the data change amplitude in the animation change process is marked as a moving vertex; vertices with relatively small data change amplitude in the animation change process are marked as fixed vertices. The facial expression data may be entered into the illusion engine by other software, such as by external DCC software (e.g., maya, etc.).
S2, constructing and initializing a vertex diagonal matrix of the facial expression data;
s3, constructing and initializing a vertex matrix of the facial expression data;
s4, multiplying the initialized vertex diagonal matrix and the vertex matrix to generate a facial expression data matrix, and taking out the optimal value of each expression vertex of the facial expression data from the facial expression data matrix as new facial expression data;
in the embodiment of the invention, the element values of each position in the facial expression data matrix are coordinate values of corresponding coordinates of the corresponding expression vertex in the three-dimensional coordinate system, the coordinate values of each coordinate form the position coordinate of the expression vertex in the three-dimensional coordinate system, and the position coordinate of each vertex can be obtained from the facial expression data matrix, and the position coordinate is the optimal value of the position coordinate of the corresponding expression vertex (namely the optimal value of the expression vertex). The initialized vertex diagonal matrix and the vertex matrix can be subjected to distributed matrix multiplication operation through a multithreading technology, so that the optimal value of the position coordinates of each expression vertex is obtained in Euclidean space and used as new facial expression data.
And S5, updating the facial expression data by utilizing the new facial expression data, and rendering a new expression according to the updated facial expression data.
In the embodiment of the invention, new facial expression data (namely, the optimal value of the position coordinates of each expression vertex) is required to be used for updating old facial expression data, and the new facial expression data can be updated through the rendering thread of the illusion engine UE, so that the GPU renders new expression according to the updated facial expression data.
In summary, compared with the prior art, the method, the device and the storage medium for processing the expression data of the digital virtual object provided by the embodiment of the invention determine the optimal value of each expression vertex of the facial expression data as new facial expression data by carrying out data transformation on the data vertex of the facial expression data, and render the expression according to the new facial expression data, so that the expression of the virtual object can be quickly and massively repaired on line in real time, and expression is more natural.
On the basis of the foregoing method embodiment, the constructing and initializing the vertex diagonal matrix of the facial expression data may include:
constructing and assigning a vertex diagonal matrix with the size of M multiplied by M, wherein M is the number of expression vertices in the facial expression data;
adding and expanding at the tail of the vertex diagonal matrix according to the number value of the fixed vertexes in the facial expression data to form a second matrix and assigning values;
adding and expanding at the tail of the second matrix according to the quantity value of the moving vertexes in the facial expression data to form a third matrix and assigning values;
and performing transposition operation on the third matrix to finish the initialization of the vertex diagonal matrix.
In the embodiment of the invention, the size of the vertex diagonal matrix is m×m, the j (j is a positive integer, and 1 is greater than or equal to i is greater than or equal to M) th row (j is a positive integer, and 1 is greater than or equal to j is greater than or equal to M) th column corresponds to the i th expression vertex and the j th expression vertex, and the element value of the j th column position of the i th row is determined according to the i th expression vertex and the j th expression vertex (simply understood can be that all expression vertices are respectively arranged in a row and a column according to the sequence number, so that the j th column position of the i th row corresponds to the i th expression vertex and the j th expression vertex).
The second matrix is obtained by adding expansion at the tail end of the vertex diagonal matrix, and the size of the second matrix can be M× (M+F) or (M+F) ×M, wherein F is the number of fixed vertices. If the size of the second matrix is m× (m+f), the position of the ith row and the kth column of the second matrix (k is a positive integer, and m+1+.ltoreq.k+.m+f) corresponds to the ith expression vertex and the kth-M fixed vertices, and the element value of the position of the kth row and the kth column of the ith row is determined according to the ith expression vertex and the kth-M fixed vertices (it is simply understood that all the expression vertices are respectively arranged in a row and a column by sequence number, and all the fixed vertices are arranged in sequence after the expression vertex row, so that the position of the kth row corresponds to the ith expression vertex and the kth-M fixed vertices, and the element value of the position of the jth row and the jth column of the vertex diagonal matrix is the same as the element value of the position of the ith row and the jth column of the vertex diagonal matrix); and if the size of the second matrix is (m+f) ×m, the position of the kth row and the jth column of the second matrix corresponds to the kth-M fixed vertices and the jth expression vertex, and the element value of the position of the kth row and the jth column is determined according to the kth-M fixed vertices and the jth expression vertex (it is simply understood that all expression vertices may be respectively arranged in a row and a column by sequence number and all the fixed vertices may be arranged in sequence after the expression vertex column, so that the position of the kth row and the jth column corresponds to the kth-M fixed vertices and the jth expression vertex, and the element value of the position of the jth row and the element value of the position of the jth column are the same as the element value of the position of the ith row and the jth column of the vertex diagonal matrix).
The third matrix is obtained by adding and expanding at the end of the second matrix, the size of the third matrix can be M× (M+F+N) or (M+F+N) ×M, and the specific form needs to be determined according to the second matrix: the size of the third matrix is m× (m+f+n) if the size of the second matrix is m× (m+f) or (m+f+n) x M if the size of the second matrix is (m+f) x M, where N is the number of moving vertices. If the size of the third matrix is m× (m+f+n), the ith row of the third matrix is the column (i is a positive integer, and m+f+1+.ltoreq.l+.m+f+n) corresponding to the ith expression vertex and the ith-M-F moving vertex, the element value of the ith row and the ith column is determined according to the ith expression vertex and the ith-M-F moving vertex (it is simply understood that all expression vertices are respectively arranged in a row and a column by sequence, and all the fixed vertices are arranged in sequence after the expression vertex row, and all the moving vertices are arranged in sequence after the fixed vertex row, so that the element value of the ith row and the ith column corresponds to the ith expression vertex and the ith-M-F moving vertex, and the element value of the ith row and the ith column is the same as the element value of the ith row and the ith column of the second matrix; and if the size of the third matrix is (m+f+n) ×m, the position of the jth column of the first row of the third matrix corresponds to the first-M-F movable vertex and the jth expression vertex, and the element value of the position of the jth column of the first row is determined according to the first-M-F movable vertex and the jth expression vertex (it is simply understood that all the expression vertices may be respectively arranged in a row and a column by sequence, and all the fixed vertices are arranged in sequence after the expression vertex column, and all the movable vertices are arranged in sequence after the fixed vertex column, so that the element value of the position of the jth column of the first row corresponds to the first-M-F movable vertex and the jth expression vertex, and the element value of the position of the jth column of the first row is the same as the element value of the position of the jth column of the second matrix, and the element value of the position of the jth column of the kth row is the same as the element value of the position of the kth column of the second row of the second matrix).
On the basis of the foregoing method embodiment, the constructing and assigning a vertex diagonal matrix with a size of mxm may include:
constructing a diagonal matrix with the size of M multiplied by M;
setting the element value of the diagonal position in the diagonal matrix to be 1, determining the expression vertex corresponding to each position except the diagonal position in the diagonal matrix, and determining the element value of the position according to the number of the adjacent side triangles to which the adjacent side determined by the expression vertex belongs.
In the embodiment of the present invention, the element value of the diagonal position in the diagonal matrix may be set to 1, and the element values of other positions may be the product of the number of adjacent edge triangles to which the adjacent edge formed by the expression vertices corresponding to the other positions belongs and a preset threshold (the preset threshold may be adjusted according to the expression rendering image quality and/or the computing efficiency).
On the basis of the foregoing method embodiment, the forming a second matrix by additionally expanding the number of the fixed vertices in the facial expression data at the end of the vertex diagonal matrix and assigning values may include:
adding and expanding at the tail end of the vertex diagonal matrix according to the quantity value of the fixed vertexes in the facial expression data to form a second matrix;
and for each newly added position in the second matrix, if the two expression vertexes corresponding to the position are the same, setting the element value of the position to be 1, or if the two expression vertexes corresponding to the position are different, setting the element value of the position to be 0, wherein the newly added position is a position of the second matrix which is increased compared with the vertex diagonal matrix.
In the embodiment of the present invention, the element value of the newly added position in the second matrix may be determined according to the expression vertex corresponding to the position: if the expression vertices corresponding to the position are the same, the element value of the position can be 1, otherwise, the element value of the position is 0.
On the basis of the foregoing method embodiment, the additionally expanding the end of the second matrix according to the number value of the moving vertices in the facial expression data to form a third matrix and assigning may include:
adding and expanding at the tail of the second matrix according to the quantity value of the moving vertexes in the facial expression data to form a third matrix;
and for each newly added position in the third matrix, if the two expression vertexes corresponding to the position are the same, setting the element value of the position to be 1, or if the two expression vertexes corresponding to the position are different, setting the element value of the position to be 0, wherein the newly added position is a position of the third matrix which is increased compared with the second matrix.
In the embodiment of the present invention, the element value of the newly added position in the third matrix may be determined according to the expression vertex corresponding to the position: if the expression vertices corresponding to the position are the same, the element value of the position can be 1, otherwise, the element value of the position is 0.
On the basis of the foregoing method embodiment, the constructing and initializing the vertex matrix of the facial expression data may include:
constructing a vertex matrix with a preset size according to the quantity of expression vertices in the facial expression data and assigning values;
adding and expanding the end of the vertex matrix according to the number value of the fixed vertexes in the facial expression data to form a fourth matrix and assigning values;
and adding and expanding at the tail of the fourth matrix according to the quantity value of the moving vertexes in the facial expression data to form a fifth matrix, and assigning values to finish vertex matrix initialization.
In the embodiment of the present invention, the vertex matrix may have a size of mx 3 or 3×m, and the specific size corresponds to the third matrix, that is, the transpose of the third matrix may be guaranteed to be multiplied by the fifth matrix, and the following description will be given by taking the vertex matrix having a size of mx 3 as an example (the vertex matrix having a size of 3×m is a transpose matrix of the vertex matrix having a size of mx 3, and the operation of the vertex matrix having a size of 3×m is identical to the operation of the vertex matrix having a size of mx 3, which is not repeated herein). The position of the ith (i is a positive integer and 1 is less than or equal to i is less than or equal to M) row p (p is a positive integer and 1 is less than or equal to p is less than or equal to 3) column of the vertex matrix with the size of M multiplied by 3 corresponds to the ith expression vertex, and the element value of the position of the ith row p is determined according to the position coordinate of the ith expression vertex.
The fourth matrix may have a size of (m+f) ×3, and the position of the kth row and the kth column of the fourth matrix corresponds to the kth-M fixed vertices, the element value of the position of the kth row and the kth column is determined according to the position coordinates of the kth-M fixed vertices, and the element value of the position of the ith row and the kth column is the same as the element value of the position of the ith row and the kth column in the vertex matrix.
The fifth matrix may have a size of (m+f+n) ×3, and the position of the p-th column of the first row of the fifth matrix corresponds to the first-M-F moving vertex, the element value of the position of the p-th column of the first row is determined according to the position coordinates of the first-M-F moving vertex, the element value of the position of the p-th column of the i row is the same as the element value of the position of the p-th column of the i row in the fourth matrix, and the element value of the position of the p-th column of the k row is the same as the element value of the position of the p-th column of the k row in the fourth matrix.
If the size of the vertex matrix is 3×m, the size of the fourth matrix is 3× (m+f), and the size of the fifth matrix is 3× (m+f+n).
On the basis of the foregoing method embodiment, the constructing and assigning a vertex matrix of a predetermined size according to the number of expression vertices in the facial expression data may include:
constructing a matrix with the size of M multiplied by 3 or 3 multiplied by M, wherein M is the number of expression vertices in the facial expression data;
for each position in the matrix with the size of M multiplied by 3 or 3 multiplied by M, determining the element value of the position according to the position coordinate value corresponding to the expression vertex corresponding to the position.
In the embodiment of the present invention, taking a vertex matrix with a size of mx 3 as an example, an element value at a position of an ith row and a 1 st column in the vertex matrix may be a product of a first coordinate axis coordinate of an ith expression vertex and a preset threshold, an element value at a position of an ith row and a 2 nd column in the vertex matrix may be a product of a second coordinate axis coordinate of the ith expression vertex and a preset threshold, and an element value at a position of an ith row and a 3 rd column in the vertex matrix may be a product of a third coordinate axis coordinate of the ith expression vertex and a preset threshold, where the preset threshold may be adjusted according to expression rendering quality and/or calculation efficiency. The first, second and third coordinate axes are three of the x, y and z axes.
On the basis of the foregoing method embodiment, the forming a fourth matrix by additionally expanding the number of the fixed vertices in the facial expression data at the end of the vertex matrix and assigning values may include:
adding and expanding the end of the vertex matrix according to the number value of the fixed vertexes in the facial expression data to form a fourth matrix;
and for each position in the fourth matrix, determining the element value of the position according to the position coordinate value corresponding to the fixed vertex corresponding to the position.
In the embodiment of the present invention, taking the fourth matrix with the size of (m+f) x 3 (the fourth matrix with the size of 3× (m+f) x 3 is a transposed matrix of the fourth matrix with the size of (m+f) x 3, the operation of the fourth matrix with the size of 3× (m+f) x 3 is consistent with the operation of the fourth matrix with the size of (m+f) x 3, which will not be described herein again), the element values at the position of the kth row and 1 st column in the fourth matrix may be the product of the first coordinate axis coordinates of the kth-M fixed vertices and the preset threshold, the element values at the position of the kth row and 2 nd column in the fourth matrix may be the product of the second coordinate axis coordinates of the kth-M fixed vertices and the preset threshold, and the element values at the position of the kth row and 3 rd column in the fourth matrix may be the product of the third coordinate axes of the kth-M fixed vertices and the preset threshold.
On the basis of the foregoing method embodiment, the additionally expanding the number of moving vertices in the facial expression data at the end of the fourth matrix to form a fifth matrix and assigning values may include:
adding and expanding the tail of the fourth matrix according to the quantity value of the moving vertexes in the facial expression data to form a fifth matrix;
and for each position in the fifth matrix, determining the element value of the position according to the position coordinate value corresponding to the mobile vertex corresponding to the position.
In the embodiment of the present invention, taking the fifth matrix with the size of (m+f+n) x 3 (the fifth matrix with the size of 3× (m+f+n) x 3 is a transposed matrix of the fifth matrix with the size of (m+f+n), the operation of the fifth matrix with the size of 3× (m+f+n) x 3 is consistent with the operation of the fifth matrix with the size of (m+f+n) ×3, which will not be described herein again), the element value of the position of the first row and the first column in the fifth matrix may be the product of the first coordinate axis coordinate of the first moving vertex and the preset threshold value, the element value of the position of the first row and the second column in the fifth matrix may be the product of the second coordinate axis coordinate of the first moving vertex and the preset threshold value, and the element value of the position of the first row and the first column in the fifth matrix may be the product of the third coordinate of the first row and the third coordinate of the first moving vertex and the second coordinate of the first row and the second column in the fifth matrix and the preset threshold value.
In order to further embody the advantages of the expression data processing method of the digital virtual object provided by the present invention, the present invention further provides an apparatus for processing expression data using the digital virtual object, where the apparatus may be a processor, a chip or a chip system in an electronic device, or be a module in an electronic device, as shown in fig. 2, and the apparatus includes: an information acquisition unit 1 for acquiring facial expression data of a digital virtual object in real time; a vertex diagonal matrix initializing unit 2 for constructing and initializing a vertex diagonal matrix of the facial expression data; a vertex matrix initializing unit 3 for constructing and initializing a vertex matrix of the facial expression data; a data operation unit 4, configured to multiply the initialized vertex diagonal matrix and the initialized vertex matrix to generate a facial expression data matrix, and extract an optimal value of each expression vertex of the facial expression data from the facial expression data matrix as new facial expression data; and a rendering unit 5 for updating the facial expression data with the new facial expression data, and rendering the new expression according to the updated facial expression data.
The implementation process of the expression data processing device for the digital virtual object provided in the embodiment of the present disclosure is consistent with the expression data processing method for the digital virtual object provided in the embodiment of the present application, and the effect that can be achieved is the same as the expression data processing method for the digital virtual object provided in the embodiment of the present application, which is not described herein again.
The embodiment of the invention also provides an electronic device, as shown in fig. 3, which includes: one or more processors; a memory for storing one or more programs; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the steps of the expression data processing method for a digital virtual object described above in the embodiments of the present specification.
Embodiments of the present specification provide a computer readable storage medium having stored thereon computer programs which, when executed by a processor, perform the steps of the expression data processing method of the digital virtual object described above in the embodiments of the present specification.
The foregoing description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, since it is intended that all modifications, equivalents, improvements, etc. that fall within the spirit and scope of the invention.
Claims (10)
1. A method for processing expression data of a digital virtual object, comprising:
acquiring facial expression data of a digital virtual object in real time;
constructing and initializing a vertex diagonal matrix of the facial expression data;
constructing and initializing a vertex matrix of the facial expression data;
multiplying the initialized vertex diagonal matrix and the vertex matrix to generate a facial expression data matrix, and taking out the optimal value of each expression vertex of the facial expression data from the facial expression data matrix as new facial expression data;
and updating the facial expression data by utilizing the new facial expression data, and rendering a new expression according to the updated facial expression data.
2. The method of claim 1, wherein the constructing and initializing a vertex diagonal matrix of facial expression data comprises:
constructing and assigning a vertex diagonal matrix with the size of M multiplied by M, wherein M is the number of expression vertices in the facial expression data;
adding and expanding at the tail of the vertex diagonal matrix according to the number value of the fixed vertexes in the facial expression data to form a second matrix and assigning values;
adding and expanding at the tail of the second matrix according to the quantity value of the moving vertexes in the facial expression data to form a third matrix and assigning values;
and performing transposition operation on the third matrix to finish the initialization of the vertex diagonal matrix.
3. The method of claim 2, wherein constructing and assigning a vertex diagonal matrix of size mxm comprises:
constructing a diagonal matrix with the size of M multiplied by M;
setting the element value of the diagonal position in the diagonal matrix to be 1, determining the expression vertex corresponding to each position except the diagonal position in the diagonal matrix, and determining the element value of the position according to the number of the adjacent side triangles to which the adjacent side determined by the expression vertex belongs.
4. The method of claim 2, wherein the additionally expanding at the end of the vertex diagonal matrix to form a second matrix according to the number of fixed vertices in the facial expression data and assigning values comprises:
adding and expanding at the tail end of the vertex diagonal matrix according to the quantity value of the fixed vertexes in the facial expression data to form a second matrix;
and for each newly added position in the second matrix, if the two expression vertexes corresponding to the position are the same, setting the element value of the position to be 1, or if the two expression vertexes corresponding to the position are different, setting the element value of the position to be 0, wherein the newly added position is a position of the second matrix which is increased compared with the vertex diagonal matrix.
5. The method of claim 2, wherein the additionally expanding at the end of the second matrix according to the number of moving vertices in the facial expression data to form a third matrix and assigning values includes:
adding and expanding at the tail of the second matrix according to the quantity value of the moving vertexes in the facial expression data to form a third matrix;
and for each newly added position in the third matrix, if the two expression vertexes corresponding to the position are the same, setting the element value of the position to be 1, or if the two expression vertexes corresponding to the position are different, setting the element value of the position to be 0, wherein the newly added position is a position of the third matrix which is increased compared with the second matrix.
6. The method of claim 1, wherein the constructing and initializing a vertex matrix of facial expression data comprises:
constructing a vertex matrix with a preset size according to the quantity of expression vertices in the facial expression data and assigning values;
adding and expanding the end of the vertex matrix according to the number value of the fixed vertexes in the facial expression data to form a fourth matrix and assigning values;
and adding and expanding at the tail of the fourth matrix according to the quantity value of the moving vertexes in the facial expression data to form a fifth matrix, and assigning values to finish vertex matrix initialization.
7. The method of claim 6, wherein constructing and assigning a vertex matrix of a predetermined size according to the number of expression vertices in the facial expression data comprises:
constructing a matrix with the size of M multiplied by 3 or 3 multiplied by M, wherein M is the number of expression vertices in the facial expression data;
for each position in the matrix with the size of M multiplied by 3 or 3 multiplied by M, determining the element value of the position according to the position coordinate value corresponding to the expression vertex corresponding to the position.
8. The method of claim 6, wherein the additionally expanding at the end of the vertex matrix by the number of fixed vertices in the facial expression data to form a fourth matrix and assigning values comprises:
adding and expanding the end of the vertex matrix according to the number value of the fixed vertexes in the facial expression data to form a fourth matrix;
and for each position in the fourth matrix, determining the element value of the position according to the position coordinate value corresponding to the fixed vertex corresponding to the position.
9. The method of claim 6, wherein the additionally expanding at the end of the fourth matrix by the number of moving vertices in the facial expression data to form a fifth matrix and assigning values comprises:
adding and expanding the tail of the fourth matrix according to the quantity value of the moving vertexes in the facial expression data to form a fifth matrix;
and for each position in the fifth matrix, determining the element value of the position according to the position coordinate value corresponding to the mobile vertex corresponding to the position.
10. An expression data processing apparatus of a digital virtual object, comprising:
an information acquisition unit for acquiring facial expression data of the digital virtual object in real time;
a vertex diagonal matrix initializing unit for constructing and initializing a vertex diagonal matrix of the facial expression data;
a vertex matrix initializing unit for constructing and initializing a vertex matrix of the facial expression data;
the data operation unit is used for carrying out multiplication operation on the initialized vertex diagonal matrix and the vertex matrix to generate a facial expression data matrix, and taking out the optimal value of each expression vertex of the facial expression data from the facial expression data matrix as new facial expression data;
and the rendering unit is used for updating the facial expression data by utilizing the new facial expression data and rendering the new expression according to the updated facial expression data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310452655.1A CN116228951B (en) | 2023-04-25 | 2023-04-25 | Expression data processing method and device for digital virtual object |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310452655.1A CN116228951B (en) | 2023-04-25 | 2023-04-25 | Expression data processing method and device for digital virtual object |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116228951A true CN116228951A (en) | 2023-06-06 |
| CN116228951B CN116228951B (en) | 2023-07-14 |
Family
ID=86569778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310452655.1A Active CN116228951B (en) | 2023-04-25 | 2023-04-25 | Expression data processing method and device for digital virtual object |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116228951B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116485964A (en) * | 2023-06-21 | 2023-07-25 | 海马云（天津）信息技术有限公司 | Expression processing method, device and storage medium of digital virtual object |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110075372A (en) * | 2009-12-28 | 2011-07-06 | 성결대학교 산학협력단 | Generating method for exaggerated 3d facial expressions with personal styles |
| KR20140040416A (en) * | 2012-09-26 | 2014-04-03 | 에스케이플래닛 주식회사 | Apparatus for 3d object creation and thereof method |
| US20140362091A1 (en) * | 2013-06-07 | 2014-12-11 | Ecole Polytechnique Federale De Lausanne | Online modeling for real-time facial animation |
| CN112116699A (en) * | 2020-08-14 | 2020-12-22 | 浙江工商大学 | Real-time real-person virtual trial sending method based on 3D face tracking |
| WO2021012590A1 (en) * | 2019-07-22 | 2021-01-28 | 广州华多网络科技有限公司 | Facial expression shift method, apparatus, storage medium, and computer device |
-
2023
- 2023-04-25 CN CN202310452655.1A patent/CN116228951B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20110075372A (en) * | 2009-12-28 | 2011-07-06 | 성결대학교 산학협력단 | Generating method for exaggerated 3d facial expressions with personal styles |
| KR20140040416A (en) * | 2012-09-26 | 2014-04-03 | 에스케이플래닛 주식회사 | Apparatus for 3d object creation and thereof method |
| US20140362091A1 (en) * | 2013-06-07 | 2014-12-11 | Ecole Polytechnique Federale De Lausanne | Online modeling for real-time facial animation |
| CN105900144A (en) * | 2013-06-07 | 2016-08-24 | 费斯史福特股份公司 | Online modeling for real-time facial animation |
| WO2021012590A1 (en) * | 2019-07-22 | 2021-01-28 | 广州华多网络科技有限公司 | Facial expression shift method, apparatus, storage medium, and computer device |
| CN112116699A (en) * | 2020-08-14 | 2020-12-22 | 浙江工商大学 | Real-time real-person virtual trial sending method based on 3D face tracking |
Non-Patent Citations (3)
| Title |
|---|
| ANURAG RANJAN，ETC: "Generating 3D Faces using Convolutional Mesh Autoencoders", 《PROCEEDINGS OF THE EUROPEAN CONFERENCE ON COMPUTER VISION (ECCV), 2018》, pages 704 - 720 * |
| 李茜: "基于超图的人脸表情驱动", 《中国优秀硕士学位论文全文数据库 信息科技辑》, pages 138 - 1370 * |
| 罗江林;王青青;朱妹丽;: "改进的拉普拉斯算法在动画表情控制中的应用", 科技资讯, no. 09, pages 27 - 28 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116485964A (en) * | 2023-06-21 | 2023-07-25 | 海马云（天津）信息技术有限公司 | Expression processing method, device and storage medium of digital virtual object |
| CN116485964B (en) * | 2023-06-21 | 2023-10-13 | 海马云（天津）信息技术有限公司 | Expression processing method, device and storage medium of digital virtual object |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116228951B (en) | 2023-07-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106710003B (en) | OpenG L ES-based three-dimensional photographing method and system | |
| CN113838176B (en) | Model training method, three-dimensional face image generation method and three-dimensional face image generation equipment | |
| US11514638B2 (en) | 3D asset generation from 2D images | |
| Chen et al. | A survey on 3d gaussian splatting | |
| CN116228951B (en) | Expression data processing method and device for digital virtual object | |
| CN113658309A (en) | Three-dimensional reconstruction method, device, equipment and storage medium | |
| CN112530005B (en) | Three-dimensional model linear structure recognition and automatic restoration method | |
| CN115018992B (en) | Method and device for generating hair style model, electronic equipment and storage medium | |
| CN116977522A (en) | Rendering method and device of three-dimensional model, computer equipment and storage medium | |
| US20180276870A1 (en) | System and method for mass-animating characters in animated sequences | |
| US9056247B2 (en) | System and method for efficient character animation | |
| Chen et al. | Synthesizing cloth wrinkles by CNN‐based geometry image superresolution | |
| CN116385667B (en) | Reconstruction method of three-dimensional model, training method and device of texture reconstruction model | |
| Peng et al. | A real-time system for crowd rendering: parallel lod and texture-preserving approach on gpu | |
| JP2000268188A (en) | Device and method for drawing three-dimensional graphics to perform occlusion culling | |
| Chen et al. | Skeleton-driven surface deformation through lattices for real-time character animation | |
| CN107818578B (en) | Rapid face model reconstruction algorithm and system based on registration method | |
| CN115908651A (en) | Synchronous updating method for three-dimensional human body model and skeleton and electronic equipment | |
| Pawaskar et al. | Expression transfer: A system to build 3d blend shapes for facial animation | |
| CN116485964B (en) | Expression processing method, device and storage medium of digital virtual object | |
| CN117557699B (en) | Animation data generation method, device, computer equipment and storage medium | |
| Wang | Construction of Internet Game Development Environment based on OpenGL and Augmented Reality | |
| Yueh et al. | Conformal surface morphing with applications on facial expressions | |
| CN116993905B (en) | Three-dimensional pipeline reconstruction method and system based on B/S architecture | |
| US20240054728A1 (en) | Mesh Model Transformation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |