CN113538219A - Three-dimensional face data transmission and receiving method, equipment and computer readable storage medium - Google Patents
Three-dimensional face data transmission and receiving method, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN113538219A CN113538219A CN202110802622.6A CN202110802622A CN113538219A CN 113538219 A CN113538219 A CN 113538219A CN 202110802622 A CN202110802622 A CN 202110802622A CN 113538219 A CN113538219 A CN 113538219A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- dimensional image
- dimensional face
- face
- face model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 83
- 230000005540 biological transmission Effects 0.000 title claims abstract description 36
- 238000013507 mapping Methods 0.000 claims description 24
- 238000009877 rendering Methods 0.000 claims description 24
- 238000011084 recovery Methods 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 4
- 235000014676 Phragmites communis Nutrition 0.000 claims description 2
- 238000004590 computer program Methods 0.000 claims description 2
- 230000001815 facial effect Effects 0.000 claims 1
- 230000009467 reduction Effects 0.000 abstract description 4
- 230000006835 compression Effects 0.000 description 12
- 238000007906 compression Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000013499 data model Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 230000004044 response Effects 0.000 description 5
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006837 decompression Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000037303 wrinkles Effects 0.000 description 1
Images
Classifications
-
- G06T3/08—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
- G06T17/205—Re-meshing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/001—Model-based coding, e.g. wire frame
Abstract
The present disclosure relates to a transmission and reception method of face data, a computing device, and a computer-readable storage medium. There is provided a transmission method of three-dimensional face data implemented by a computing device, including: receiving a three-dimensional face model representing three-dimensional face information; converting the three-dimensional face model into a two-dimensional image; compressing the two-dimensional image; and transmitting the compressed two-dimensional image to other devices. The scheme of the disclosure can realize reduction of data transmission amount by optimizing the data format, thereby improving transmission efficiency.
Description
Technical Field
The present disclosure relates to the field of imaging, and more particularly, to a method, apparatus, and computer-readable storage medium for transmitting and receiving three-dimensional face data.
Background
With the development of science and technology, the application of three-dimensional face data is more and more extensive. These applications include, but are not limited to, three-dimensional gaming, face payment, face security, and the like.
Compared with two-dimensional face data, the three-dimensional face data contains richer face depth and geometric topological information, and is more robust to changes of factors such as expressions, postures, illumination, shielding and the like. However, the three-dimensional face data also has a problem that a large amount of bandwidth is consumed during transmission due to the abundant information contained.
Disclosure of Invention
In view of this, aspects of the disclosed embodiments provide transmission and reception schemes for three-dimensional face data. According to the embodiment of the disclosure, large-scale three-dimensional face data can be converted into a two-dimensional image, so that a compression method which can be generally applied to the two-dimensional image can be used during transmission, thereby effectively reducing the data transmission amount and improving the transmission efficiency.
In a first aspect, the present disclosure provides a method for transmitting three-dimensional face data implemented by a computing device, including: receiving a three-dimensional face model representing three-dimensional face information; converting the three-dimensional face model into a two-dimensional image; compressing the two-dimensional image; and transmitting the compressed two-dimensional image to other devices.
In some embodiments, converting the three-dimensional face model to a two-dimensional image includes rasterizing the three-dimensional face model into a two-dimensional image by conformal mapping.
In some embodiments, the three-dimensional face model represents a three-dimensional curved face surface using a triangular mesh, and rasterizing the three-dimensional face model into a two-dimensional image by conformal mapping comprises: converting the human face three-dimensional curved surface into a plane rectangle; and representing the planar rectangle using a two-dimensional image.
In some embodiments, converting the three-dimensional curved face surface into a planar rectangle comprises: selecting 4 vertexes on the boundary of the three-dimensional curved surface of the human face, wherein the 4 vertexes are distributed relatively uniformly; and conformally mapping the three-dimensional curved surface of the human face to a plane rectangle by taking the 4 vertexes as corner points.
In some embodiments, conformally mapping the three-dimensional curved face surface onto a planar rectangle comprises: and mapping the human face three-dimensional curved surface onto a plane rectangle by using a discrete surface Regoji stream method.
In some embodiments, mapping the face three-dimensional surface onto a planar rectangle using a discrete surface reed stream method comprises: setting target curvatures based on the plane rectangle, wherein the target curvatures of four corner points are
The target curvatures of the other vertexes are 0; and constructing and optimizing the discrete entropy energy to obtain a flatness measure of the planar rectangle.
In some embodiments, the method further comprises sampling surface attributes of the three-dimensional surface of the human face on the plane rectangle to form a surface attribute map, wherein the surface attributes comprise vertex coordinates.
In some embodiments, wherein the surface properties further comprise at least one of: color, texture coordinates, and normal.
In a second aspect, the present disclosure provides a method of receiving three-dimensional face data implemented by a computing device, comprising: receiving a compressed two-dimensional image representing a three-dimensional face model, wherein the three-dimensional face model is used for representing a three-dimensional face curved surface; decompressing the compressed two-dimensional image; and rendering the three-dimensional face curved surface by using the decompressed two-dimensional image.
In some embodiments, rendering the three-dimensional face surface using the decompressed two-dimensional image comprises: carrying out data recovery on the three-dimensional face model by using the decompressed two-dimensional image; correspondingly associating the attribute information of the two-dimensional image with the three-dimensional face model; and rendering by using the three-dimensional face model.
In some embodiments, the data recovery of the three-dimensional face model using the decompressed two-dimensional image comprises: converting the two-dimensional image into a planar standard annulus, wherein the two-dimensional image represents a planar standard circle; conformally mapping the planar standard annulus onto a topological annulus; and generating the three-dimensional face model by using the topological girdle band.
In some embodiments, the data recovery of the three-dimensional face model using the decompressed two-dimensional image comprises: and converting the two-dimensional image into a three-dimensional face model, wherein the two-dimensional image represents a plane rectangle, the color information of the two-dimensional image represents the geometric coordinate information of the vertexes of the three-dimensional face model, and the relationship between the pixels of the two-dimensional image represents the relationship between the vertexes of the three-dimensional face model.
In a third aspect, the present disclosure provides a computing device comprising a processor and a memory having stored therein computer program code which, when executed, causes the processor to perform a method as defined in any of the embodiments of the first aspect of the present disclosure or to perform a method as defined in any of the embodiments of the second aspect of the present disclosure.
In a fourth aspect, the present disclosure provides a computer readable storage medium having stored thereon computer readable instructions which, when executed by one or more processors, implement a method as in any one of the embodiments of the first aspect of the present disclosure, or implement a method as in any one of the embodiments of the second aspect of the present disclosure.
With the transmission and reception method of three-dimensional face data, the computing device, and the computer-readable storage medium provided as above, the scheme of the present disclosure can use a compression coding method generally applied to a two-dimensional image at the time of transmission by optimizing a data format, for example, rasterizing the three-dimensional face data into a two-dimensional image, thereby enabling a reduction in data transmission amount, thereby improving transmission efficiency. In some embodiments, by separating data types, such as color and texture coordinates in three-dimensional data, further reduction in data transfer volume may be achieved.
Drawings
The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. In the drawings, several embodiments of the disclosure are illustrated by way of example and not by way of limitation, and like or corresponding reference numerals indicate like or corresponding parts and in which:
FIG. 1 illustrates an exemplary system in which aspects of transmitting and/or receiving three-dimensional face data according to embodiments of the present disclosure may be implemented;
FIG. 2 illustrates an exemplary three-dimensional face model;
FIG. 3 illustrates an exemplary mesh representation of the three-dimensional face model of FIG. 2;
FIG. 4 illustrates an example flow diagram of a method of transmission of three-dimensional face data implemented by a computing device in accordance with an embodiment of this disclosure;
FIG. 5 illustrates an example flow diagram of a rasterization method in accordance with one embodiment of the present disclosure;
FIG. 6 illustrates a three-dimensional face model two-dimensional image generated in accordance with an embodiment of the present disclosure; and
fig. 7 illustrates an example flowchart of a receiving method of three-dimensional face data implemented by a computing device according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.
It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, description, and drawings of the present disclosure are used to distinguish between different objects and are not used to describe a particular order. The terms "comprises" and "comprising," when used in the specification and claims of this disclosure, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the disclosure is for the purpose of describing particular embodiments only, and is not intended to be limiting of the disclosure. As used in the specification and claims of this disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the specification and claims of this disclosure refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.
As used in this specification and claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
Fig. 1 illustrates an exemplary system 100 in which aspects of transmitting and/or receiving three-dimensional face data according to embodiments of the present disclosure may be implemented.
As shown in FIG. 1, system 100 includes a server 110 and a client 120. The server 110 may be any computing device for providing three-dimensional face data, and may include, for example and without limitation, a three-dimensional game server, a multimedia information server, and the like. The client 120 may be any computing device that needs to present a three-dimensional face, and may include, for example and without limitation, a tablet computer, a smart terminal, a PC device, an internet of things terminal, a mobile phone, a visual terminal, and the like.
There may be client-based rendering and server-based rendering, depending on whether the rendering occurs on the client side or the server side. In the embodiment of the disclosure, the method is mainly applied to rendering scenes based on the client. Therefore, the following description is primarily directed to this scenario.
In a client-based rendering scenario, the client 120 may send simple request information to the server 110. The server 110 transmits three-dimensional face data, for example, information describing a three-dimensional face surface, to the client 120 in response to a request of the client 120. After receiving the three-dimensional face data, the client 120 may perform rendering by using the three-dimensional face data. Three-dimensional rendering can be generally classified into two types: real-time rendering (e.g., three-dimensional gaming, etc. applications) and offline rendering (e.g., animation movies, etc. applications).
Since three-dimensional data needs to express more abundant information, the data volume of the three-dimensional data is generally large. Large scale geometry data transmission in the network between the server 110 and the client 120 consumes a large amount of bandwidth and is therefore expected to improve.
Although a scenario in which the embodiments of the present disclosure may be applied is described above with reference to the exemplary system 100 including the server 110 and the client 120, the transmission and reception method of three-dimensional face data provided by the embodiments of the present disclosure is not limited to this application scenario. For example, the transmission and reception of three-dimensional face data may also occur between various components within the computing device to reduce the computing resources required for storage and transmission.
The three-dimensional face data represents a three-dimensional data model of a three-dimensional face. In general, three-dimensional data models may be modeled based on the faces and/or volumes of an object. Based on the modeling of the body, not only the surface characteristics of the object but also the internal characteristics of the object can be given. The surface-based structuring gives only the surface features of the object. The combination of volume-based and surface-based modelling is referred to as geometric model modelling.
Fig. 2 illustrates an exemplary three-dimensional face model. FIG. 3 illustrates an exemplary mesh representation of the three-dimensional face model of FIG. 2.
Three-dimensional surfaces are typically represented as polygonal (e.g., triangular) meshes and texture images. The triangular mesh in fig. 3 represents the geometric topology information of the curved surface, and the texture image (not shown) gives information of the color and material of the curved surface, and the like. The texture image can store more information, for example, each pixel can record a lot of information such as color, normal vector, material, metal, background light, scattering, highlight, transparency, geometric height, geometric displacement, etc. for describing the details of the object surface. The process of mapping the triangular mesh onto a planar region may be referred to as surface parameterization, and the process of fitting the texture image onto the surface may be referred to as texture mapping. For example, normal vector mapping is often used to add geometric details, particularly wrinkles, to the surface of a human face.
In order to increase the transmission speed of the three-dimensional face data, in various embodiments of the present disclosure, a compression coding method generally applied to a two-dimensional image may be used in transmission by rasterizing the three-dimensional face data into a two-dimensional image, whereby a reduction in data transmission amount may be achieved, thereby increasing transmission efficiency.
Fig. 4 illustrates an example flow diagram of a method 400 of transmission of three-dimensional face data implemented by a computing device in accordance with an embodiment of this disclosure. In this embodiment, the computing device may be, for example, but not limited to, the server 110 shown in FIG. 1.
As shown in fig. 4, the method 400 begins with step S410 of receiving a three-dimensional face model that characterizes a three-dimensional face surface. It should be understood that the three-dimensional face model described herein represents a digitized representation of a three-dimensional face, which can be read, analyzed, and processed by a computer to achieve desired results based on the results of such analysis and processing.
In some embodiments, a three-dimensional face model may be constructed based on face information acquired by a three-dimensional data acquisition system. The three-dimensional data acquisition system is mainly used for acquiring three-dimensional data of a target (such as a human face) so as to construct a three-dimensional data model. According to the imaging principle of a camera used by a three-dimensional data acquisition system, the method can be mainly divided into three modes: 3D structured light, laser ranging and binocular stereoscopic vision. The 3D structured light passes through the infrared light projector, projects light with certain structural characteristics to a shot object, and is collected by a special infrared camera. The method mainly utilizes the principle of similarity of triangles to calculate, so that the depth information of each point on the image is obtained, and the three-dimensional data is finally obtained. Laser ranging, in short, measures the distance according to the time difference between pulse transmission and pulse reception. Binocular stereo vision is a method for acquiring three-dimensional geometric information of an object from a plurality of images based on the parallax principle. Two digital images of the measured object are simultaneously obtained from different angles by the two cameras, and three-dimensional geometric information of the object is recovered based on a parallax principle, so that depth information of each point on the image is obtained, and three-dimensional data is finally obtained.
The three-dimensional data model may have different representation methods, such as point clouds, voxel meshes, polygon meshes, multi-view maps, and so on. In an embodiment of the present disclosure, a polygonal mesh (e.g., a triangular mesh) is employed to represent a three-dimensional model of a face surface. A polygonal mesh is a set of vertices and polygons representing a polyhedron shape in three-dimensional computer graphics, and is also called an unstructured mesh. These meshes are usually composed of triangles, quadrilaterals or other simple convex polygons, which may simplify the rendering process. However, the mesh may also include objects composed of normal polygons with holes.
In some embodiments, the three-dimensional face acquisition system may be independent of or included in a computing device implementing the method 400, and embodiments of the present disclosure are not limited in this respect.
In other embodiments, the three-dimensional face model may be pre-constructed, such as from an existing three-dimensional face library, or a virtual face model.
Next, in step S420, the three-dimensional face model is converted into a two-dimensional image.
There are many mature techniques for processing two-dimensional images, whether in hardware or software, as opposed to three-dimensional data. Thus, rasterization of the three-dimensional face model into a two-dimensional image may facilitate subsequent processing, such as compression and transmission.
Various ways may be employed to convert the three-dimensional face model into a two-dimensional image, such as various lossy conversions. Considering that the two-dimensional image needs to be restored into three-dimensional face data for other processing, such as rendering, it is necessary to retain the information of the original three-dimensional face data as much as possible. Thus, in some embodiments of the present disclosure, conformal mapping is employed to rasterize the three-dimensional face data into a two-dimensional image. The rasterization method of the conformal mapping manner of the embodiment of the present disclosure will be described in detail later.
Next, in step S430, the converted two-dimensional image is compressed. Since the three-dimensional face data has been converted into a two-dimensional image in step S420, the two-dimensional image may be compressed using a compression method existing in the industry in step S430. For example, the two-dimensional image may be compressed using industry standard algorithms, such as picture compression techniques, to reduce the amount of data to be transmitted. Still image compression standards include, for example, the still image experts group JPEG standard, which mainly includes DCT, vector quantization method, Huffman coding method, wavelet-based compression algorithm, and the like. Those skilled in the art will appreciate that the disclosed embodiments are not limited in this respect.
Finally, in step S440, the compressed two-dimensional image is transmitted to other devices. In this example, the other device may be, for example, but not limited to, client 120 shown in fig. 1. Depending on the communication connection between the computing device and the other device, the compressed two-dimensional image may be transmitted over a corresponding transmission channel. For example, the communication connection may be a wired connection and/or a wireless connection, and embodiments of the present disclosure are not limited in this respect.
As can be seen from the above description, since the three-dimensional face model representing the three-dimensional face is rasterized into the two-dimensional image and compressed using the existing compression technology for the two-dimensional image, the amount of data during transmission can be greatly reduced, thereby improving the transmission efficiency. In a client-based rendering scenario, for example, an increase in transmission efficiency may also correspondingly increase the rendering speed at the client.
FIG. 5 illustrates an example flow diagram of a rasterization method 500 in accordance with one embodiment of the present disclosure. Those skilled in the art will appreciate that the method 500 of fig. 5 may be performed in step S420 of fig. 4 to rasterize the three-dimensional face model into a two-dimensional image. In this rasterization method 500, a discrete curved surface curie Flow (Ricci Flow) method is used for conformal mapping.
As shown in fig. 5, in step S510, the three-dimensional curved surface of the human face is converted into a planar rectangle.
As mentioned previously, the three-dimensional face model may use a triangular mesh to represent the three-dimensional curved surface of the face. For example, a three-dimensional data acquisition system is used to scan a real human face S, and the resulting surface is a topological disc, i.e., a surface with a boundary of genus 0. The continuous surface S is densely sampled with a set of sampling points v0, v2, ·, vn. And (3) triangulating T on the curved surface by taking the sampling points as vertexes, wherein each triangle is an Euclidean triangle formed by opening three vertexes { vi, vj, vk }, so that a discrete curved surface (namely a piecewise linear polyhedral surface) is formed, or a triangular mesh is formed. The discrete curvature Ridge flow method used in the embodiment of the disclosure has no requirement on the quality of triangulation, and can completely process low-quality meshes.
The three-dimensional model of the face is rasterized into a two-dimensional image that may be a variety of shapes, such as a square, rectangle, circle, and the like. In some embodiments, the two-dimensional image may be rectangular, since most images are rectangular, which facilitates applying known compression, decompression, and like processing methods to the two-dimensional image at a later time.
In some embodiments, converting the three-dimensional face surface into a planar rectangle may include selecting 4 vertices on the boundary of the three-dimensional face surface, where the 4 vertices are distributed relatively uniformly; then, the three-dimensional curved surface of the human face is conformally mapped onto the plane rectangle by taking the 4 vertexes as corner points.
In some embodiments, a discrete surface curie Flow (Ricci Flow) method may be used to achieve conformal mapping of a three-dimensional surface of a human face to a flat rectangle.
The information of a surface typically contains topological information and geometric information. The topology information is determined by the genus and the boundary of the surface. Geometric information refers to a metric defined on a surface. As mentioned above, in the field of computer graphics, a triangular mesh is usually used to represent a discrete surface model in a three-dimensional data model, wherein the connection relationship between triangle edges represents the topology of the network model, and the length of each edge represents the discrete measurement of the mesh. The mapping process of a surface may also be referred to as a deformation process, which is actually a process that continuously changes the measurement of the surface itself.
The basic idea of the Ricci Flow method is as follows: starting from the starting measurement and its induced Ricci curvature, the measurement of the surface is continuously changed conformally until the last measurement can derive the predefined target curvature.
By the Ricci Flow method, a conformal mapping function from a three-dimensional curved surface of the human face to a plane rectangle can be calculated. Specifically, since the target curvature is determined by the plane rectangle, the target curvature may be set based on the plane rectangle, for example, the target curvatures of four corner points of the face surface are set to be pi/2, and the target curvatures of other vertices are 0. Then, the discrete entropy energy can be constructed and optimized to obtain a target flatness measure, i.e., a flatness measure of a planar rectangle.
Next, in step S520, the converted planar rectangle is represented using a two-dimensional image.
Through the foregoing steps, the three-dimensional face surface has been mapped onto a planar rectangle, and thus the planar rectangle can be represented using a representation of a two-dimensional image. For example, a color two-dimensional image may represent the image according to different color spaces, such as RGB, HUV color spaces, and so forth. Thus, the planar rectangles representing the three-dimensional face model can be rasterized into a bitmap, e.g., an m × n array of [ x, y, z ] data values. At this time, the color information of the image corresponds to coordinate information of the geometric structure in the three-dimensional face curved surface, which can be represented using, for example, three floating point numbers.
Alternatively or additionally, other surface attributes of the three-dimensional face surface may also be sampled to generate a surface attribute map. These surface attributes may include vertex coordinates. Further, these surface attributes may also include, but are not limited to, color, texture coordinates, normal, and the like, for example. For example, the surface properties of the three-dimensional surface of the face may be sampled on the aforementioned planar rectangle to form a surface property map. Thus, if the normal mapping is required for later rendering, another two-dimensional array can be similarly created, consisting of the normal values [ nx, ny, nz ].
FIG. 6 illustrates a two-dimensional image of a three-dimensional face model generated according to an embodiment of the present disclosure. The left diagram in the figure shows the planar rectangle, i.e. the parametric representation, after conformal mapping of the three-dimensional face model of fig. 2 and 3; the right diagram shows a geometric image representation.
The transmission method of three-dimensional face data according to the embodiment of the present disclosure is described above with reference to fig. 4 to 6. Correspondingly, the disclosure also provides a receiving method of the three-dimensional face data.
Fig. 7 shows an example flow diagram of a method 700 of receiving three-dimensional face data implemented by a computing device according to an embodiment of the disclosure. In this embodiment, the computing device may be, for example, but not limited to, client 120 shown in FIG. 1.
As shown in fig. 7, the method 700 begins in step S710 by receiving a compressed two-dimensional image representing a three-dimensional face model, wherein the three-dimensional face model is used to characterize a three-dimensional face surface. It will be appreciated by those skilled in the art that the received two-dimensional image is generated, for example, by the method described above with reference to fig. 4-6, and will not be described in detail herein.
Next, in step S720, the compressed two-dimensional image is decompressed. Both compression and decompression of two-dimensional images may be performed using techniques that are well established in the art and, therefore, will not be described in detail herein.
Finally, in step S730, the three-dimensional object is rendered by using the decompressed two-dimensional image.
In some embodiments, rendering the three-dimensional object using the decompressed two-dimensional image may include: carrying out data recovery on the three-dimensional face model by using the decompressed two-dimensional image; correspondingly associating the attribute information of the two-dimensional image with the three-dimensional face model; and rendering by using the three-dimensional face model.
Depending on the conversion method of the three-dimensional face model into the two-dimensional image adopted by the transmitting end (for example, the server 110 in fig. 1), a corresponding method can be adopted to perform data recovery on the three-dimensional face model according to the two-dimensional image. Because the conversion methods from the three-dimensional face model to the two-dimensional image are all in one-to-one correspondence, the three-dimensional face model can also be directly and reversely corresponded from the two-dimensional image. At this time, the color information of the two-dimensional image represents the geometric coordinate information of the vertices on the three-dimensional face model, which are all represented using, for example, three floating-point numbers, and thus can be converted back to the original geometry by the reverse operation. The relationship between pixels on the two-dimensional image can represent the relationship between vertexes on the three-dimensional face model.
As can be seen from the above description, since the three-dimensional face model representing the three-dimensional face curved surface is rasterized into a two-dimensional image and compressed using the existing compression technology for the two-dimensional image, the amount of data during transmission can be greatly reduced, thereby improving the transmission efficiency. In a client-based rendering scenario, an increase in transmission efficiency may also correspondingly increase the rendering speed at the client.
Exemplary devices
As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
In some possible embodiments, an apparatus for transmitting and/or receiving three-dimensional data according to embodiments of the present disclosure may include at least one processing unit, and at least one storage unit. Wherein the storage unit stores program code which, when executed by the processing unit, causes the processing unit to perform the steps of the method for transmitting and/or receiving three-dimensional face data according to various exemplary embodiments of the present disclosure described in the above section "exemplary methods" of this specification.
Exemplary program product
In some possible embodiments, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing an apparatus to perform the steps of the method for transmitting and/or receiving three-dimensional face data according to various exemplary embodiments of the disclosure described in the "exemplary methods" section above of this specification, when said program product is run on the apparatus.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device over any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., over the internet using an internet service provider).
It should be noted that although in the above detailed description several units or sub-units of the apparatus are mentioned, this division is only illustrative and not mandatory. Indeed, the features and functions of two or more units described above may be embodied in one unit, in accordance with embodiments of the present disclosure. Conversely, the features and functions of one unit described above may be further divided into embodiments by a plurality of units.
Further, while the operations of the disclosed methods are depicted in the drawings in a particular order, this does not require or imply that these operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
While the spirit and principles of the present disclosure have been described with reference to several particular embodiments, it is to be understood that the present disclosure is not limited to the particular embodiments disclosed, nor is the division of aspects, which is for convenience only as the features in such aspects may not be combined to benefit. The disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (13)
1. A method of three-dimensional face data transmission implemented by a computing device, comprising:
receiving a three-dimensional face model representing three-dimensional face information;
converting the three-dimensional face model into a two-dimensional image;
compressing the two-dimensional image; and
and transmitting the compressed two-dimensional image to other equipment.
2. The method of claim 1, wherein converting the three-dimensional face model to a two-dimensional image comprises rasterizing the three-dimensional face model into a two-dimensional image by conformal mapping.
3. The method of claim 2, wherein the three-dimensional face model represents a three-dimensional curved face surface using a triangular mesh, and rasterizing the three-dimensional face model into a two-dimensional image by conformal mapping comprises:
converting the human face three-dimensional curved surface into a plane rectangle; and
the planar rectangle is represented using a two-dimensional image.
4. The method of claim 3, wherein converting the three-dimensional curved face surface into a planar rectangle comprises:
selecting 4 vertexes on the boundary of the three-dimensional curved surface of the human face, wherein the 4 vertexes are distributed relatively uniformly; and
and conformally mapping the three-dimensional curved surface of the human face onto a plane rectangle by taking the 4 vertexes as corner points.
5. The method of claim 4, wherein conformally mapping the three-dimensional curved face surface onto a planar rectangle comprises:
and mapping the human face three-dimensional curved surface onto a plane rectangle by using a discrete surface Regoji stream method.
6. The method of claim 5, wherein mapping the facial three-dimensional surface onto a planar rectangle using a discrete surface reed stream method comprises:
setting target curvatures based on the plane rectangle, wherein the target curvatures of 4 corner points are pi/2, and the target curvatures of the rest vertexes are 0; and
discrete entropy energies are constructed and optimized to yield a flatness measure of a planar rectangle.
7. The method of any of claims 3-6, further comprising sampling surface attributes of the three-dimensional surface of the human face on the planar rectangle to form a surface attribute map, wherein the surface attributes comprise vertex coordinates.
8. The method of claim 7, wherein the surface properties further comprise at least one of: color, texture coordinates, and normal.
9. A method of receiving three-dimensional face data implemented by a computing device, comprising:
receiving a compressed two-dimensional image representing a three-dimensional face model, wherein the three-dimensional face model is used for representing a three-dimensional face curved surface;
decompressing the compressed two-dimensional image; and
and rendering the three-dimensional face curved surface by using the decompressed two-dimensional image.
10. The method of claim 9, wherein rendering the three-dimensional face surface using the decompressed two-dimensional image comprises:
carrying out data recovery on the three-dimensional face model by using the decompressed two-dimensional image;
correspondingly associating the attribute information of the two-dimensional image with the three-dimensional face model; and
and rendering by using the three-dimensional face model.
11. The method of claim 10, wherein data recovery of the three-dimensional face model using the decompressed two-dimensional image comprises:
and converting the two-dimensional image into a three-dimensional face model, wherein the two-dimensional image represents a plane rectangle, the color information of the two-dimensional image represents the geometric coordinate information of the vertexes of the three-dimensional face model, and the relationship between the pixels of the two-dimensional image represents the relationship between the vertexes of the three-dimensional face model.
12. A computing device comprising a processor and a memory, the memory having stored therein computer program code which, when executed, causes the processor to perform the method of any of claims 1-8 or to perform the method of any of claims 9-11.
13. A computer-readable storage medium having stored thereon computer-readable instructions which, when executed by one or more processors, perform the method of any one of claims 1-8, or perform the method of any one of claims 9-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110802622.6A CN113538219A (en) | 2021-07-15 | 2021-07-15 | Three-dimensional face data transmission and receiving method, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110802622.6A CN113538219A (en) | 2021-07-15 | 2021-07-15 | Three-dimensional face data transmission and receiving method, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113538219A true CN113538219A (en) | 2021-10-22 |
Family
ID=78099610
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110802622.6A Pending CN113538219A (en) | 2021-07-15 | 2021-07-15 | Three-dimensional face data transmission and receiving method, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113538219A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114549284A (en) * | 2022-01-14 | 2022-05-27 | 北京有竹居网络技术有限公司 | Image information processing method and device and electronic equipment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060013505A1 (en) * | 2002-11-06 | 2006-01-19 | Shing-Tung Yau | Analysis of geometric surfaces by comformal structure |
| CN110766808A (en) * | 2019-11-05 | 2020-02-07 | 北京智拓视界科技有限责任公司 | Method and device for processing face data and computer-readable storage medium |
-
2021
- 2021-07-15 CN CN202110802622.6A patent/CN113538219A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060013505A1 (en) * | 2002-11-06 | 2006-01-19 | Shing-Tung Yau | Analysis of geometric surfaces by comformal structure |
| CN110766808A (en) * | 2019-11-05 | 2020-02-07 | 北京智拓视界科技有限责任公司 | Method and device for processing face data and computer-readable storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114549284A (en) * | 2022-01-14 | 2022-05-27 | 北京有竹居网络技术有限公司 | Image information processing method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11562507B2 (en) | Point cloud compression using video encoding with time consistent patches | |
| Fu et al. | Single image 3D object reconstruction based on deep learning: A review | |
| Maglo et al. | 3d mesh compression: Survey, comparisons, and emerging trends | |
| Lee et al. | GROOT: a real-time streaming system of high-fidelity volumetric videos | |
| WO2023138477A1 (en) | Three-dimensional model reconstruction method, image generation method, device and storage medium | |
| JP7371691B2 (en) | Point cloud encoding using homography transformation | |
| CN113313832B (en) | Semantic generation method and device of three-dimensional model, storage medium and electronic equipment | |
| US20050093862A1 (en) | System, method, and program product for extracting a multiresolution quadrilateral-based subdivision surface representation from an arbitrary two-manifold polygon mesh | |
| WO2023093085A1 (en) | Method and apparatus for reconstructing surface of object, and computer storage medium and computer program product | |
| CN115880415A (en) | Three-dimensional reconstruction method and device, electronic equipment and storage medium | |
| CN111127658A (en) | Point cloud reconstruction-based feature-preserving curved surface reconstruction method for triangular mesh curved surface | |
| WO2013029232A1 (en) | Multi-resolution 3d textured mesh coding | |
| Xia et al. | Modeling and compressing 3-D facial expressions using geometry videos | |
| CN113538219A (en) | Three-dimensional face data transmission and receiving method, equipment and computer readable storage medium | |
| US11418769B1 (en) | Viewport adaptive volumetric content streaming and/or rendering | |
| Fei et al. | 3D Gaussian as a New Vision Era: A Survey | |
| Berjón et al. | Objective and subjective evaluation of static 3D mesh compression | |
| US10347034B2 (en) | Out-of-core point rendering with dynamic shapes | |
| Yu et al. | Saliency computation and simplification of point cloud data | |
| Han | Texture image compression algorithm based on self-organizing neural network | |
| Lin et al. | Visual saliency and quality evaluation for 3D point clouds and meshes: An overview | |
| Hubo et al. | Self-similarity based compression of point set surfaces with application to ray tracing | |
| CN115375847B (en) | Material recovery method, three-dimensional model generation method and model training method | |
| Zhang et al. | Toward mobile 3d vision | |
| CN113613011B (en) | Light field image compression method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |