WO2021170127A1 - Method and apparatus for three-dimensional reconstruction of half-length portrait - Google Patents

Abstract

The present application provides a method and apparatus for three-dimensional reconstruction of a half-length portrait, for solving the problems of high complexity and long reconstruction duration. The method comprises: obtaining an image comprising a frontal human face of a half-length portrait of a target person, then unfolding a texture by using a head semantic mask of the frontal human face, so that at least two organs in the unfolded texture are located at preset positions, and then complementing a back texture according to the upfolded texture map comprising the front. A three-dimensional mesh model can further be constructed according to the frontal human face, a preconfigured ear model is used to replace ears in the three-dimensional mesh model, a texture of an ear area is obtained from a replaced three-dimensional mesh model, and is fused to the complemented back texture, so as to obtain the three-dimensional mesh model. A professional modeling device is not required, and the complexity is low. Because the frontal and back textures of a human body have certain correlation, the back texture is complemented according to the frontal texture, so that the constructed three-dimensional model fits the character better, and a better effect is achieved.

Description

Method and device for three-dimensional reconstruction of bust

Cross-references to related applications

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on February 29, 2020, the application number is 202010132592.8, and the application title is "A method and device for three-dimensional reconstruction of bust", the entire content of which is incorporated by reference In this application.

Technical field

The embodiments of the present application relate to the field of image processing technology, and in particular to a method and device for three-dimensional reconstruction of a bust.

Background technique

At present, three-dimensional bust reconstruction technology has a wide range of applications in three-dimensional printing, entertainment, and remote augmented reality (AR) calls. The traditional three-dimensional bust image reconstruction technology can adopt a monocular system or a multi-eye system. When using a monocular system, an infrared depth camera is usually used to scan a person in a circle, and the scanned person needs to remain still during the scanning process. Therefore, this method takes a long time to scan the person, and the reconstruction calculation time is long, and the scanning effect may be unsatisfactory or failure. The multi-eye scanning system is an acquisition system based on multiple viewing angles. Although it has real-time reconstruction capabilities, it is expensive, with large equipment and high complexity, and it is inconvenient to operate.

Summary of the invention

The embodiments of the present application provide a method and device for three-dimensional reconstruction of a bust to solve the problems of long reconstruction time and high complexity.

The specific technical solutions provided by the embodiments of this application are as follows:

In the first aspect, the present application provides a three-dimensional reconstruction method of a bust. The three-dimensional reconstruction method of the bust can be implemented by an electronic device, such as one of the electronic devices or a processing unit. The method may include: obtaining an image to be processed including a bust of the target person, the bust including a frontal face, and then obtaining a first texture expansion image based on the obtained image to be processed, and the first texture expansion image is used to represent the front face of the bust Texture, and at least two organs (such as the nose and ears) of the five senses of the bust in the first texture expansion picture are located at preset positions; after that, expand the first texture according to the front texture of the bust in the first texture expansion picture The backside texture of the bust in the figure is supplemented to obtain a second texture expansion map, which is used to characterize the surface texture of the bust; finally, a three-dimensional model of the bust is obtained according to the second texture expansion map. The reconstruction method provided in this application does not require professional modeling equipment and has low complexity. Since the front and back textures of the human body are related to a certain degree, the back texture is supplemented according to the front texture, so that the constructed three-dimensional model fits the character more closely and achieves better results.

The second texture expansion map is used to characterize the surface texture of the bust, that is, the second texture expansion map describes the omnidirectional texture of the character surface of the bust. The surface texture includes the front texture of the bust and the back texture of the bust. For example, the front of a bust includes a human face, front neck, or front shoulders. The back of the bust can include the back of the head, the back of the neck, or the back of the shoulders.

In a possible design, obtaining the first texture expansion map according to the image to be processed can be implemented in the following ways:

Removing the background in the image to be processed to obtain the front image of the bust;

Performing semantic segmentation on the frontal image to obtain a semantic mask of the head of the frontal image;

Obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the frontal image;

According to the positions of the at least two organs on the head in the head semantic mask and the three-dimensional mesh model, a first texture expansion map is obtained, and the at least two organs in the first texture expansion map are located at preset positions.

Through the above design, the texture expansion map is obtained according to the semantic mask of the head, that is, the texture expansion according to the semantic pixel coordinates can improve the expansion accuracy.

In a possible design, the obtaining the first texture expansion map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model may be implemented in the following manner:

Performing texture expansion on the texture corresponding to the three-dimensional mesh model based on a central axis to obtain a third texture expansion diagram, where the central axis is the connecting line from the top end of the head to the bottom end of the head in the three-dimensional mesh model;

Determining the locations of the at least two organs in the three-dimensional mesh model according to the locations of the at least two organs on the head in the head semantic mask;

The third texture expansion map is adjusted according to the positions of the at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.

In the above design, after the three-dimensional mesh model of the bust is obtained, combined with the front texture of the bust, the texture of the bust can be expanded in a circle to obtain the first texture expansion map. Circumferential expansion is non-Atlas (atlas) texture expansion. Non-atlas expansion helps to improve the continuity of textures and reduce the gaps between textures. In addition, inpainting can be avoided, and the position of the semantic block on the texture map can be relatively fixed, which is convenient for machine learning.

In a possible design, obtaining the three-dimensional model of the bust based on the second texture expansion map includes:

Smoothing the texture seam area in the second texture expansion map to obtain a fourth texture expansion map, obtain a three-dimensional model of the bust based on the fourth texture expansion map, and the texture seam area according to the The three-dimensional mesh model is used to determine the expansion line used when the texture is expanded.

The above design smoothes the stitching area, smoothing the gaps in the stitching area, and optimizing the modeling details.

In a possible design, the at least two organs include ears; the method further includes:

Fusing the pre-configured ear model to the ear region in the three-dimensional mesh model to obtain a fused three-dimensional mesh model;

Obtaining the three-dimensional model of the bust based on the processed second texture expansion map, including:

The texture of the ear region on the fused three-dimensional network model is fused to the ear region located at the preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map, which is based on the fused fourth texture expansion map Obtain a three-dimensional model of the bust.

The above design realizes high-precision local texture and geometry through mesh optimization and texture replacement of the ear part, and optimizes texture completion and mesh reconstruction details.

In a possible design, smoothing the texture stitching area in the second texture expansion image can be achieved in the following manner:

Determining the back texture image of the bust according to the front image of the bust;

Perform weighted fusion processing on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image.

In the above design, due to the correlation between the front and back textures of the human body, the back texture is estimated based on the front image. The estimated back texture is free of gaps, and then weighted with the back texture obtained based on the second texture expansion map , Optimize the smoothing effect of the back gap.

In a possible design, weighted fusion processing is performed on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image, which may be implemented in the following manner:

Performing weighted fusion processing on the second texture expansion image and the back image of the bust according to a set rule to obtain the fourth texture expansion image;

The setting rules are:

I ₃ (i,j)=αI ₁ (i,j)+(1-α)I ₂ (map ₁ (i),map ₂ (j));

α=I _alpha (i,j);

Among them, I ₁ represents the second texture expansion image, I ₂ represents the back image of the bust, I ₃ represents the fourth texture expansion image, I _alpha represents the weight, and map ₁ represents in the X-axis direction, the bust’s The pixels in the back image are mapped to the mapping function on the second texture expansion map, map ₂ represents the mapping function on the Y-axis direction, the pixels in the back image of the bust are mapped to the mapping function on the second texture expansion map, i Indicates the coordinate value of the pixel in the X-axis direction, and j represents the coordinate value of the pixel in the X-axis direction.

In a possible design, before obtaining the first texture expansion map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, the method further includes:

Perform at least one of the following processes on the three-dimensional mesh model:

Hole filling processing, mesh uniformization processing or mesh smoothing processing.

In the above design, the three-dimensional mesh model is processed to fill holes to make the three-dimensional mesh model more complete and improve the accuracy of the reconstruction of the three-dimensional model. Performing grid homogenization processing can prevent the obtained 3D mesh model from being too dense or too sparse, which will affect the accuracy of the 3D model reconstruction. Performing mesh smoothing can remove inaccurate meshes in the 3D mesh model, that is, noise points, thereby improving the accuracy and smoothness of the 3D model reconstruction.

In the second aspect, the present application provides a three-dimensional reconstruction device for a bust, including:

An acquiring unit, configured to acquire an image to be processed, the image to be processed includes a bust of a target person, and the bust includes a frontal face;

The reconstruction unit is configured to obtain a first texture expansion map according to the image to be processed, the first texture expansion map being used to characterize the frontal texture of the bust, and the first texture expansion map in the five senses of the bust At least two organs are located at preset positions; according to the front texture of the bust in the first texture expansion view, supplementing the back texture of the bust in the first texture expansion view to obtain a second texture expansion view The second texture expansion map is used to characterize the surface texture of the bust; and the three-dimensional model of the bust is obtained according to the second texture expansion map.

In a possible design, when the reconstruction unit obtains the first texture expansion map according to the image to be processed, it is specifically used for:

Removing the background in the image to be processed to obtain the front image of the bust;

Performing semantic segmentation on the frontal image to obtain a semantic mask of the head of the frontal image;

Obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the frontal image;

According to the positions of the at least two organs on the head in the head semantic mask and the three-dimensional mesh model, a first texture expansion map is obtained, and the at least two organs in the first texture expansion map are located at preset positions.

In a possible design, the reconstruction unit is specifically configured to: when obtaining the first texture expansion map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model:

Performing texture expansion on the texture corresponding to the three-dimensional mesh model based on a central axis to obtain a third texture expansion diagram, where the central axis is the connecting line from the top end of the head to the bottom end of the head in the three-dimensional mesh model;

Determining the locations of the at least two organs in the three-dimensional mesh model according to the locations of the at least two organs on the head in the head semantic mask;

The third texture expansion map is adjusted according to the positions of the at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.

In a possible design, the reconstruction unit is specifically configured to: when obtaining the three-dimensional model of the bust based on the second texture expansion map:

Smoothing the texture seam area in the second texture expansion map to obtain a fourth texture expansion map, obtain a three-dimensional model of the bust based on the fourth texture expansion map, and the texture seam area according to the The three-dimensional mesh model is used to determine the expansion line used when the texture is expanded.

In a possible design, the at least two organs include ears; the reconstruction unit is also used to fuse the pre-configured ear model into the three-dimensional mesh model of the ear region after the fusion is obtained. Lattice model

The reconstruction unit is specifically configured to: when obtaining the three-dimensional model of the bust based on the processed second texture expansion map:

The texture of the ear region on the fused three-dimensional network model is fused to the ear region located at the preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map, which is based on the fused fourth texture expansion map Obtain a three-dimensional model of the bust.

In a possible design, the reconstruction unit is specifically configured to: when smoothing the texture stitch line region in the second texture expansion map:

Determining the back texture image of the bust according to the front image of the bust;

Perform weighted fusion processing on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image.

In a possible design, when the reconstruction unit performs weighted fusion processing on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image, it is specifically configured to:

Performing weighted fusion processing on the second texture expansion image and the back image of the bust according to a set rule to obtain the fourth texture expansion image;

The setting rules are:

I ₃ (i,j)=αI ₁ (i,j)+(1-α)I ₂ (map ₁ (i),map ₂ (j));

α=I _alpha (i,j);

Among them, I ₁ represents the second texture expansion image, I ₂ represents the back image of the bust, I ₃ represents the fourth texture expansion image, I _alpha represents the weight, and map ₁ represents in the X-axis direction, the bust’s The pixels in the back image are mapped to the mapping function on the second texture expansion map, map ₂ represents the mapping function on the Y-axis direction, the pixels in the back image of the bust are mapped to the mapping function on the second texture expansion map, i Indicates the coordinate value of the pixel in the X-axis direction, and j represents the coordinate value of the pixel in the X-axis direction.

In a possible design, the reconstruction unit is further configured to: before obtaining the first texture expansion map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model: The three-dimensional mesh model performs at least one of the following processing: hole filling processing, mesh uniformization processing, or mesh smoothing processing.

In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory; wherein the processor is coupled to the memory; wherein the memory is used to store program instructions; the processor is used to read program instructions stored in the memory, To achieve the first aspect and any possible design methods. Optionally, the processor includes an ISP for executing the process of acquiring the image to be processed.

In a possible design, the electronic device further includes a camera; the camera is used to collect images to be processed. The above-mentioned processor is used to control the camera to collect images.

In a fourth aspect, a computer storage medium provided by an embodiment of the present application, the computer storage medium stores program instructions, and when the program instructions run on an electronic device, the electronic device or processor executes the first aspect and any of its possibilities The method of design.

In the fifth aspect, a computer program product provided by an embodiment of the present application, when the computer program product runs on an electronic device, causes the electronic device or processor to execute the first aspect and any possible design method thereof.

The sixth aspect is a chip provided by an embodiment of the present application, which is coupled with a memory in an electronic device, and executes the first aspect and any possible design method thereof. Optionally, the chip includes an ISP for performing the process of acquiring the image to be processed.

In addition, the technical effects brought by the second aspect to the sixth aspect can be referred to the description of the above-mentioned first aspect, which will not be repeated here.

It should be noted that “coupled” in the embodiments of the present application means that two components are directly or indirectly combined with each other.

Description of the drawings

Figure 1 is a schematic diagram of an electronic device in an embodiment of the application;

FIG. 2 is a schematic flowchart of a method for three-dimensional reconstruction of a bust in an embodiment of this application;

Fig. 3 is a schematic diagram of a bust in an embodiment of the application;

FIG. 4 is a schematic flowchart of a method for texture expansion in an embodiment of the application;

FIG. 5 is a schematic diagram of three-dimensional reconstruction of a bust in an embodiment of the application;

Fig. 6 is a schematic diagram of the central axis in an embodiment of the application;

FIG. 7 is a schematic diagram illustrating the expansion of the circumference in an embodiment of the application;

Fig. 8 is a schematic diagram of determining the positions of the nose and ears in an embodiment of the application;

FIG. 9 is a schematic diagram of smoothing the gap in an embodiment of the application;

FIG. 10 is a schematic diagram of a smoothing method used for smoothing the gap in an embodiment of the application;

Figure 11 is a schematic diagram of ear stitching in an embodiment of the application;

FIG. 12 is a schematic diagram of a reconstructed three-dimensional model in an embodiment of this application;

FIG. 13 is a schematic diagram of a three-dimensional virtual video call scene in an embodiment of the application;

FIG. 14 is a schematic diagram of a device 1400 in an embodiment of the application.

Detailed ways

The three-dimensional reconstruction scheme of the bust involved in this application can be applied to application scenarios such as three-dimensional printing, AR calling, and virtual reality (VR) models. The three-dimensional reconstruction method of the bust provided in this application can be applied to electronic equipment. Electronic devices can be personal computers, server computers, client computers, handheld or laptop devices, microprocessor-based system devices, embedded system devices, set-top boxes, programmable consumer electronics, network personal computers, and small computers, large computers Distributed cloud computing technology environment servers including any of the above-mentioned systems, as well as portable terminal devices with functions such as personal digital assistants and/or image processing, such as mobile phones, tablets, and wearable devices with wireless communication functions (such as smart watches) , In-vehicle equipment, etc.

Referring to FIG. 1, the electronic device involved in the embodiment of the present application may include a processor 110 and a memory 120. The processor 110 may include one or more processing units. For example, the processor 110 may include a central processing unit (CPU), an image processing unit (graphics processing unit, GPU), an image signal processor (ISP), and a digital signal processor (digital signal processor). One or more of DSP), or neural processing unit (NPU). Among them, the different processing units can be independent devices, and can also be integrated in one or more chips or circuit boards. Among them, the digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. The neural processor includes, but is not limited to, a neural network processing unit, such as a deep neural network processing unit or a convolutional neural network processing unit. The neural processor can use the neural network model to perform training, calculation, or processing. The neural network model includes, but is not limited to, a deep neural network model or a convolutional neural network model. The above digital signal processor, image processing unit or central processing unit can also use the neural network model to perform training, calculation or processing.

In some embodiments, a memory may also be provided in the processor 110 to temporarily store instructions and data. For example, the memory in the processor 110 may be a cache memory (cache). The memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.

In other embodiments, the processor 110 may further include one or more interfaces. For example, the interface may be a universal serial bus (USB) interface. For another example, the interface can also be an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, and a universal asynchronous transmission/reception transmission. UART (universal asynchronous receiver/transmitter) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, etc. It is understandable that, in the embodiments of the present application, different modules of the electronic device, including the processor, can be connected through interfaces, so that the electronic device can implement different functions. It should be noted that the embodiment of the present application does not limit the connection mode of the interface in the electronic device 100.

In one example, the NPU included in the processor 110 is a neural-network (NN) computing processor. By drawing on the structure of a biological neural network, for example, drawing on the transfer mode between human brain neurons, the input information can be quickly processed. You can also continue to learn by yourself. The three-dimensional reconstruction of the bust of the electronic device 100 can be achieved through the NPU.

In another example, the processor 110 includes an NPU and other processors, and the three-dimensional reconstruction of the bust of the electronic device 100 can be realized through the NPU and other processors. The other processor may be, for example, one or more of a CPU, an image processor (GPU), an image signal processor (ISP), and a digital signal processor (digital signal processor, DSP).

The memory 120 may be used to store computer executable program code, where the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the memory 120. The memory 120 may include a program storage area and a data storage area. Among them, the storage program area can store an operating system, driver software, or at least one application program required by a function (such as a sound playback function, an image playback function, etc.). The data storage area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 100. The memory 120 may include at least one of a power-down volatile memory or a non-power-down volatile memory, such as read only memory (ROM), random access memory (RAM), and dynamic random access memory. (dynamic random access memory, DRAM), embedded multimedia card (eMMC), universal flash storage (UFS), hard disk or magnetic disk, etc.

In a possible implementation manner, the electronic device may further include an image collector 130 for collecting images. The image collector 130 may include a camera, or may further include the aforementioned ISP. ISP is used to process the image data collected by the camera. For example, when taking a picture, the shutter is opened, the light is transmitted to the photosensitive element in the camera through the lens in the camera, the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP, and the ISP processes and converts the relevant data. It is an image visible to the naked eye. ISP can also optimize the image noise, brightness, and chroma algorithm. ISP can also control image exposure or color temperature optimization for the shooting scene. In some embodiments, the ISP can be set in the camera. More commonly, an ISP can exist as a part of a processor, integrated with other types of processing units, such as a CPU, GPU, or DSP, on one or more chips.

The camera is used to capture still images or dynamic video. The object generates an optical image through the lens and is projected to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transfers the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the processor 110, which is processed by other processing units. In some embodiments, the electronic device may include one or more cameras.

In a possible implementation manner, the electronic device may further include a display screen 140, and the display screen 140 is used to display images, videos, and the like. In some embodiments, the electronic device may include one or more display screens. The display screen includes but is not limited to a touch screen. In the embodiment of the present application, the display screen can be used to display a three-dimensional model of a reconstructed back bust of a person.

The term "at least one" referred to in this application refers to one or more than one, that is, one, two, three and more; , Three and more. "Multiple" refers to two or more than two, that is, two, three and more are included. In addition, it should be understood that in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing description, and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating Or imply the order. "And/or" describes the association relationship of the associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the associated objects before and after are in an "or" relationship. "The following at least one item (a)" or similar expressions refers to any combination of these items, including any combination of a single item (a) or a plurality of items (a). For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple .

Refer to FIG. 2, which is a schematic flowchart of a method for three-dimensional reconstruction of a bust according to an embodiment of the present application. The three-dimensional reconstruction method of the bust can be implemented by the electronic device shown in FIG. 2, for example, by one of the electronic devices or a processing unit. As shown in Figure 2, the three-dimensional reconstruction method of the bust mainly includes S201-S204.

S201: Acquire an image to be processed, where the image to be processed includes a bust of a target person, and the bust includes a frontal face.

The acquiring action of S201 can be understood as a receiving operation or a processing operation. For example, the NPU may receive the to-be-processed image sent by other devices, such as the ISP, in S201. For another example, S201 may be executed by an ISP, which generates an image by processing image signals or data collected by a camera. This processing includes, but is not limited to, various types of color calibration, pixel calibration, white balance, or scaling. In the embodiment of the present application, the image to be processed may include a frontal face of a bust.

Exemplarily, the bust may include a frontal face, neck, shoulders, etc., for example, see FIG. 3.

S202: Obtain a first texture expansion map according to the image to be processed, where the first texture expansion map is used to represent the front texture of the bust.

Exemplarily, at least two organs in the five sense organs of the bust in the first texture development image are located at preset positions. The facial features include eyebrows, eyes, ears, nose, and mouth. At least two organs, such as ears and nose, or eyes and nose. The at least two organs are located at preset positions, which may be texture expansion images obtained for different images to be processed, and the positions of the at least two organs are the same. For example, two organs include a nose and an ear, the position of the ear in the texture expansion map of different images to be processed is the same, and the position of the nose in the texture expansion map of different images to be processed is the same. Or the at least two organs located at the preset positions may be texture expansion images obtained for different images to be processed, and the relative positions of the at least two organs are fixed. Taking two ears and a nose as an example, the distance between the two ears and the distance between the nose and the ears can be the same in the texture expansion of different images to be processed.

S203: Supplement the back texture of the bust in the first texture expansion image according to the front texture of the bust in the first texture expansion image to obtain a second texture expansion image. The second texture expansion map is used to characterize the surface texture of the bust.

The second texture expansion map is used to characterize the surface texture of the bust, that is, the second texture expansion map describes the omnidirectional texture of the character surface of the bust. The surface texture may be the texture of the upper body surface of the three-dimensional target person. That is, the surface texture includes a texture that surrounds the surface of the upper body along the axis perpendicular to the ground plane.

The surface texture includes the front texture of the bust and the back texture of the bust. For example, the front of a bust includes a human face, front neck, or front shoulders. The back of the bust can include the back of the head, the back of the neck, or the back of the shoulders.

S204: Obtain a three-dimensional model of the bust according to the second texture expansion map.

In a possible implementation manner, referring to FIG. 4, S202 obtaining the first texture expansion image according to the image to be processed may be implemented by the following S401-S404.

S401: Remove the background in the image to be processed to obtain a front image of the bust.

Exemplarily, the first neural network model may be used to remove the background in the processed image to obtain a frontal image of the bust. The first neural network model is used to segment the foreground and background of the image, and output the foreground image. In the implementation of this application, the foreground image is the front image of the bust.

S402: Perform semantic segmentation on the front image to obtain a head semantic mask of the front image.

Semantic segmentation is the grouping/segmentation of pixels according to the different semantic meanings expressed in the image.

Exemplarily, the semantic segmentation of the front image may use a full convolution network (Fully Convolution Networks, FCN), such as U-net network, SegNet network, DeepLab, RefineNet, or PSPNet.

S403: Obtain a three-dimensional mesh model of the bust according to the semantic mask of the head and the frontal image.

Exemplarily, the head semantic mask and the frontal image may be input to the second neural network model, and the second neural network model is used for human body reconstruction. The second neural network model outputs a three-dimensional (3D) truncated signed distance function (truncated signed distance function, TSDF) volume. Then extract the surface mesh (mesh) of the TSDF body to obtain a three-dimensional mesh model of the bust.

For example, when extracting the surface mesh of a TSDF body, a marching cube algorithm or other three-dimensional isosurface extraction algorithms can be used.

S404: Obtain a first texture expansion map according to the three-dimensional mesh model.

In a possible implementation manner, before obtaining the first texture expansion map according to the three-dimensional mesh model, at least one of the following processing may be performed on the three-dimensional mesh model: hole filling processing, mesh uniformization processing, or mesh processing. Smoothing.

Exemplarily, when performing hole filling processing on a three-dimensional mesh model, a triangular mesh hole filling method based on a radial basis function (radial basis function, RBF) or a hole filling algorithm based on the Poisson equation may be used. When performing grid uniformization on a three-dimensional grid model, grid uniformization algorithms such as point clustering, edge folding, and vertex addition and deletion can be used. When performing mesh smoothing on a three-dimensional mesh model, mesh smoothing methods based on Poisson's equation or discrete Laplace equation can be used.

After the 3D mesh model is filled with holes, the 3D mesh model is made more complete and the accuracy of the 3D model reconstruction is improved. Performing grid homogenization processing can prevent the obtained 3D mesh model from being too dense or too sparse, which will affect the accuracy of the 3D model reconstruction. Performing mesh smoothing can remove inaccurate meshes in the 3D mesh model, that is, noise points, thereby improving the accuracy and smoothness of the 3D model reconstruction.

Exemplarily, when it is determined that the bust includes ears according to the semantics of the head, when the first texture expansion map of the three-dimensional mesh model is obtained, the three-dimensional mesh can be obtained according to the positions of at least two organs on the head in the semantic mask of the head. The first texture expansion map of the grid model makes at least two organs in the first texture expansion map to be located at the preset positions.

Exemplarily, after obtaining the three-dimensional mesh model of the bust, combined with the front texture of the bust, the texture of the bust may be expanded in a circle to obtain the first texture expansion map. Circumferential expansion is non-Atlas (atlas) texture expansion. Non-atlas expansion helps to improve the continuity of textures and reduce the gaps between textures. In addition, inpainting can be avoided, and the position of the semantic block on the texture map can be relatively fixed, which is convenient for machine learning.

As an example, taking the bust shown in FIG. 3 as an example, the front image of the bust shown in (a) in FIG. 5 is obtained after A1 processing. Semantic segmentation of the front image of the bust to obtain the semantic head mask, that is, the semantic mask of the head obtained after A2 processing is shown in Figure 5(b). The head semantic mask and frontal image can be input into the second neural network model. The output TSDF volume is shown in Figure 5 (c). The surface mesh of the TSDF volume is extracted to obtain the three-dimensional mesh model of the bust. As shown in (d) in 5. Then, after the circumferential expansion, the first texture expansion diagram is obtained as shown in (e) of FIG. 5. Further, after S203 processing, a second texture expansion map as shown in (f) in FIG. 5 is obtained, that is, a texture map after texture completion. Finally, the texture map after texture completion and the surface mesh of the three-dimensional mesh model are combined to obtain the final three-dimensional model of the textured bust, as shown in Figure 5 (g).

In a possible implementation manner, S404 obtains the first texture expansion map of the three-dimensional mesh model according to the positions of at least two organs on the head in the head semantic mask, which can be implemented in the following manner:

A1, performing texture expansion on the texture corresponding to the three-dimensional mesh model based on the central axis to obtain a third texture expansion image, the central axis being the connecting line from the top of the head to the bottom of the head in the three-dimensional mesh model. For example, the top of the head can be the highest point on the top of the head. The bottom of the head can be the geometric center of the lowest surface of the three-dimensional network model. Refer to Figure 6, which is a schematic diagram of the central axis of the three-dimensional mesh model.

Exemplarily, the third texture development map can be obtained based on the central axis according to the spatial angle or the circumference of the curved surface of a certain place on the Mesh relative to the central axis. For example, the first rule can be satisfied by the circumferential expansion according to the angle. The first rule can be:

Where (x,y,z) is the spatial coordinates of a point in a three-dimensional grid model, [u,v] represents the pixel coordinates of the point with spatial coordinates (x,y,z) in the texture expansion map, pixel coordinates Take the lower left corner of the image as the origin.

Where W is the width of the expanded texture image, H is the height of the expanded texture image, and cx and cz represent the axis coordinates on the plane of equal Y value.

It should be understood that the third texture expansion map describes the grid texture of the three-dimensional grid model, or a grid texture expansion map without pixel values.

A2. Determine the positions of at least two organs in the three-dimensional mesh model according to the positions of at least two organs on the head in the head semantic mask.

The head semantic mask includes the locations of at least two organs, that is, the coordinates of at least two organs can be determined according to the head semantic mask. Take the two organs, the ear and the nose, for example. In addition, it should be understood that the three-dimensional mesh model is obtained according to the semantic mask of the head. Therefore, there is a mapping relationship between the ears and nose in the semantic mask of the head and the ears and noses in the three-dimensional mesh model. Therefore, the positions of the ears and the nose in the three-dimensional mesh model can be determined according to the positions of the ears and the nose in the semantic mask of the head.

A3: Adjust the third texture expansion map according to the positions of at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.

For example, see Figure 8. Figure 8 (a) shows the ear and nose positions in the semantic mask of the head. Figure 8(b) shows the positions of ears and nose in the three-dimensional mesh model.

One way is to adjust the positions of the ears and/or noses in the third texture expansion map according to the positions of the ears and noses in the 3D mesh model, that is, determine the ears and/or noses in the third texture expansion map according to the positions of the ears and noses in the 3D network model. Or nose position, and adjust the ear and/or nose position so that the adjusted ears and nose are located at the preset position of the third texture expansion map, and the first texture expansion is obtained according to the adjusted third texture expansion map and the front image In the figure, the adjusted ears and nose are located at the preset positions of the first texture expansion image. For example, the distortion mapping process may be performed on the third expanded texture image, so that the ears and the nose in the first expanded texture image are located at preset positions.

Another way is to obtain a color-filled third texture expansion map based on the third texture expansion map and the front image. According to the positions of the ears and the nose in the three-dimensional network model, the positions of the ears and the nose in the third texture expansion map filled with color are determined, and the positions of the ears and/or the nose are adjusted to obtain the first texture expansion map. The ears and the nose are located at the preset positions of the first texture development image.

There is a mapping relationship between the coordinates of each pixel in the third texture expansion image and the coordinates of each point in the three-dimensional grid model, as in the first rule. Since the three-dimensional grid model is obtained based on the front image, there is a mapping relationship between the coordinates of each point in the three-dimensional grid model and the coordinates of the pixel points in the front image. Further, there is a mapping relationship between the coordinates of each pixel in the third texture expansion image and the coordinates of the pixel in the front image. Therefore, the pixel values of the pixels in the front image can be mapped to the third texture expansion map (or to the adjusted third texture expansion map).

The current half-body texture expansion is generally used to expand the front and back separately, that is, expand into the visible part of the front and the invisible part of the back, resulting in poor texture reconstruction of the invisible part of the hair and ears, and the geometric mesh reconstruction of the ears The effect is poor, so it is difficult to meet user needs. Through the above solution, by performing semantically aligned texture expansion on the mesh model, a complete texture expansion image is obtained, which can improve the effect of texture completion in the invisible part; and through the mesh optimization and texture replacement of the ear part, high Accurate local texture and geometry can produce better texture completion and mesh detail reconstruction effects.

It should be noted that the adjusted ears and nose are located at the preset positions of the first texture expansion map. This process provides convenient conditions for the subsequent optimization of the ear part, and can improve the accuracy of the subsequent ear optimization . The optimization of the ear part will be described later, which will not be repeated here.

As a possible implementation manner, for a bust with long hair, the hair may completely cover the ear area. In this case, the subsequent optimization of the ear part may not be performed. Based on this, when performing texture expansion on the corresponding texture of the three-dimensional mesh model, the texture expansion map can be obtained based on the spatial angle or surface circumference of a certain place based on the central axis, and A2 and A3 can no longer be executed. In this scene, the color-filled third texture expansion map can be obtained directly according to the third texture expansion map and the front image, that is, the filled third texture expansion map is used as the first texture expansion map.

As an example, referring to (a) in Figure 7, a certain cross section of the head can be considered as a circle. The black dots between -π and π in (a) in FIG. 7 can be considered as the points on the cross-section of the expansion line used when the texture is expanded. Take the nose at the position of 0π and the two ears at the positions of -π and π respectively after the twisting process, that is, the two ears are symmetrical with the mouth as the center. It can be seen from Figure 7(a) that the nose and the two ears are asymmetrical before being twisted. As shown in (b) in Figure 7, after A3 processing, in the first texture development image, the nose is located at the 0π position. In order to show the location of the ears more intuitively, see (c) in FIG. 7, which is a schematic diagram of the second texture expansion map after S203 processing. In (c) in Figure 7, the nose is located at the position of 0π, and the two ears are located at the positions of -π and π, respectively.

After S203 supplements the back texture of the bust in the first texture expansion image to the back texture of the bust in the first texture expansion image according to the front texture of the bust in the first texture expansion image to obtain the second texture expansion image, if the three-dimensional model of the bust image is obtained according to the second texture expansion image At the time, if the back suture area is processed, there may be gaps in the suture area of the three-dimensional model obtained. As an example, when the three-dimensional model of the bust is obtained based on the second texture expansion map, the texture stitching area in the second texture expansion map may be smoothed first to obtain the fourth texture expansion map, which is based on the fourth texture expansion map The three-dimensional model of the bust is obtained, and the texture stitching area is determined according to the unfolding line used when the three-dimensional mesh model is unfolded.

As an example, when the three-dimensional mesh model is texture-expanded, the expansion line is on the back as an example. Refer to the left image in Figure 9, which is a schematic diagram of similar gaps in the suture area of the three-dimensional model. After the gap is smoothed, it is shown on the right in Figure 9. The gap smoothing operation is to perform fusion processing on the texture stitching area in the second texture expansion map.

When fusing the texture stitching area of the second texture expansion image, you can first determine the back texture image of the bust according to the front image of the bust; then perform the second texture expansion image and the back texture image of the bust The weighted fusion process obtains the fourth texture expansion image.

Exemplarily, the second texture expansion image and the back texture image of the bust may be subjected to weighted fusion processing according to the set rules to obtain the fourth texture expansion image;

The setting rules can be:

I ₃ (i,j)=αI ₁ (i,j)+(1-α)I ₂ (map ₁ (i),map ₂ (j));

α=I _alpha (i,j);

Among them, I ₁ represents the second texture expansion image, I ₂ represents the back image of the bust, I ₃ represents the fourth texture expansion image, I _alpha represents the weight, and map ₁ represents in the X-axis direction, in the back image of the bust The pixel points are mapped to the mapping function on the second texture expansion map, map ₂ represents the mapping function on the Y-axis direction, the pixel points in the back image of the bust are mapped to the mapping function on the second texture expansion map, i represents the pixel point on the X axis The coordinate value in the direction is taken, and j represents the coordinate value of the pixel point in the X-axis direction.

For example, referring to FIG. 10, the back texture image of the bust is determined according to the front image of the bust as shown in FIG. 10(a), and the weight of each pixel can be shown in FIG. 10(b). The second texture expansion diagram is shown in 10 (c). The second texture expansion image ((c) in Fig. 10) and the back texture image of the bust ((a) in Fig. 10) are merged according to (b) in Fig. 10 to obtain (d) in Fig. 10, namely The fourth texture expanded view. The fusion operation is indicated by the plus sign "+" in Figure 10.

The process of optimizing the ear part will be described in detail below. The face of the bust includes the ears, that is, the ears are not covered by the hair. You can determine whether to include ears based on the semantic mask of the head. When optimizing the ear part, the following methods can be used:

B1, first fusion the pre-configured ear model to the ear region in the 3D mesh model to obtain the fused 3D mesh model.

Exemplarily, the ear fusion to the ear region of the three-dimensional mesh model may adopt a Laplacian mesh fusion method.

B2. Fusion the texture of the ear region on the fused 3D network model to the ear region located at the preset position of the fourth texture expansion map to obtain the fused fourth texture expansion map; finally get the fusion fourth texture expansion map Three-dimensional model of the bust.

Exemplarily, the fusion method used when fusing the texture of the ear region on the fused three-dimensional network model to the ear region located at the preset position of the fourth texture expansion map may be a fusion algorithm based on the image Laplacian gradient.

See (a) in Figure 11, which is a schematic diagram of a three-dimensional mesh model. The ear area in the determined three-dimensional mesh model can be seen in (b) of FIG. 11. The geometric dimensions of the ear can be determined according to the size of the three-dimensional mesh model. The size of the three-dimensional mesh model can be preset by the user or a default size can be adopted. Fit the ear model to the ear area in the 3D mesh model, as shown in (c) in Figure 11, and then perform the fusion processing on the 3D mesh model fitted to the ear model, and the resulting fused 3D mesh model can be See (d) in Figure 11. Then, the texture of the ear region on the fused three-dimensional network model is obtained, and the texture of the ear region is fused to the ear region at the preset position in the fourth texture expansion map.

As an example, for example, the acquired image to be processed is shown in FIG. 2, and the reconstructed three-dimensional model obtained through the solution provided in the embodiment of the present application is shown in FIG. 12. (A) in FIG. 12 is a schematic front view of the three-dimensional model, and (b) in FIG. 12 is a schematic back view of the three-dimensional model.

For example, the solution provided in the embodiment of this application is applied to a virtual three-dimensional dialogue. When the local terminal device (referred to as the terminal) receives the user’s trigger and starts a virtual 3D video call, the video stream is obtained through the terminal’s camera. The 3D reconstruction method based on the bust of the single frame image proposed in this patent can be used to reconstruct the 3D Model. The terminal drives the three-dimensional model by acquiring the user's expression in each frame of the video stream, and sends it to the opposite terminal, and the opposite terminal displays the local terminal user's expression simulated by the three-dimensional model. As an example, referring to FIG. 13, creating a three-dimensional model can be performed by a computing cloud. The electronic device sends a single frame image to the computing cloud, and the computing cloud creates a three-dimensional model, and then sends the created three-dimensional model to the terminal.

In order to better explain the above embodiments, the embodiments of the present application also provide an apparatus 1400. As shown in FIG. 14, the apparatus 1400 may specifically include functional modules in an electronic device (for example, the processor 110 in FIG. 1 Components or software modules executed by them), or the apparatus 1400 may be a chip or a chip system, or the apparatus 1400 may be a module in an electronic device, or the like. Illustratively, the apparatus may include an obtaining unit 1401 and a reconstruction unit 1402. The obtaining unit 1401 and the reconstruction unit 1402 respectively execute different steps of the method shown in the embodiment corresponding to FIG. 2 and FIG. 4. For example, the obtaining unit 1401 may be used to obtain the image to be processed in S201, and the reconstruction unit 1402 may be used to perform the process of S202-S204. The specific implementation is as described above and will not be repeated here.

Therefore, it can be considered that the aforementioned acquisition unit 1401 or reconstruction unit 1402 can be implemented by software, hardware, or a combination of software and hardware. When the module is implemented in hardware, the hardware can be CPU, microprocessor, DSP, micro control unit (MCU), artificial intelligence processor, application specific integrated circuit (ASIC), field programmable gate array (field programmable gate array) , FPGA), dedicated digital circuits, hardware accelerators, or any one or any combination of non-integrated discrete devices, which can run the necessary software or do not rely on software to perform the above method flow, and are located in the previous description of Figure 1 Inside the processor 110. When the module is implemented in software, the software exists in the form of computer program instructions and is stored in a memory, such as the memory 120 in FIG. 1, and a processor, such as the processor 110 in FIG. 1, can be used to execute The program instructions are used to realize the above method flow. The processor may include, but is not limited to, at least one of the following: CPU, microprocessor, DSP, microcontroller, or artificial intelligence processor and other computing devices that run software. Each computing device may include one or more A core used to execute software instructions for calculation or processing. The processor can be a single semiconductor chip, or it can be integrated with other circuits to form a semiconductor chip. For example, it can form an SoC (on-chip) with other circuits (such as codec circuits, hardware acceleration circuits, or various bus and interface circuits). System), or it can be integrated into the ASIC as a built-in processor of an ASIC, and the ASIC integrated with the processor can be packaged separately or together with other circuits. In addition to the core used to execute software instructions to perform operations or processing, the processor may further include necessary hardware accelerators, such as FPGAs, PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, this application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

This application is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of this application. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are used to generate It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. The instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the scope of the embodiments of the present application. In this way, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims (20)

1. A method for three-dimensional reconstruction of a bust, which is characterized in that it comprises:

   Acquiring a to-be-processed image, where the to-be-processed image includes a bust of a target person, and the bust includes a frontal face;

   Obtaining a first texture expansion map according to the image to be processed, the first texture expansion map being used to characterize the front texture of the bust;

   Wherein, at least two organs of the five sense organs of the bust in the first texture expansion view are located at preset positions;

   According to the front texture of the bust in the first texture expansion view, the back texture of the bust in the first texture expansion view is supplemented to obtain a second texture expansion view, and the second texture expansion view is used for Characterize the surface texture of the bust;

   Obtain a three-dimensional model of the bust according to the second texture expansion map.
2. The method according to claim 1, wherein obtaining the first texture expansion map according to the image to be processed comprises:

   Removing the background in the image to be processed to obtain the front image of the bust;

   Performing semantic segmentation on the frontal image to obtain a semantic mask of the head of the frontal image;

   Obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the frontal image;

   According to the positions of the at least two organs on the head in the head semantic mask and the three-dimensional mesh model, a first texture expansion map is obtained, and the at least two organs in the first texture expansion map are located at preset positions.
3. The method according to claim 2, wherein the obtaining a first texture expansion map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model comprises:

   Performing texture expansion on the texture corresponding to the three-dimensional mesh model based on a central axis to obtain a third texture expansion diagram, where the central axis is the connecting line from the top end of the head to the bottom end of the head in the three-dimensional mesh model;

   Determining the locations of the at least two organs in the three-dimensional mesh model according to the locations of the at least two organs on the head in the head semantic mask;

   The third texture expansion map is adjusted according to the positions of the at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.
4. The method according to claim 2 or 3, wherein obtaining a three-dimensional model of the bust based on the second texture expansion map comprises:

   Smoothing the texture seam area in the second texture expansion map to obtain a fourth texture expansion map, obtain a three-dimensional model of the bust based on the fourth texture expansion map, and the texture seam area according to the The three-dimensional mesh model is used to determine the expansion line used when the texture is expanded.
5. The method of claim 4, wherein the at least two organs include ears; and the method further comprises:

   Fusing the pre-configured ear model to the ear region in the three-dimensional mesh model to obtain a fused three-dimensional mesh model;

   Obtaining the three-dimensional model of the bust based on the processed second texture expansion map, including:

   The texture of the ear region on the fused three-dimensional network model is fused to the ear region located at the preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map, which is based on the fused fourth texture expansion map Obtain a three-dimensional model of the bust.
6. The method according to claim 4 or 5, wherein the smoothing of the texture stitch line region in the second texture expansion map comprises:

   Determining the back texture image of the bust according to the front image of the bust;

   Perform weighted fusion processing on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image.
7. 7. The method of claim 6, wherein performing weighted fusion processing on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image comprises:

   Performing weighted fusion processing on the second texture expansion image and the back image of the bust according to a set rule to obtain the fourth texture expansion image;

   The setting rules are:

   I ₃ (i,j)=αI ₁ (i,j)+(1-α)I ₂ (map ₁ (i),map ₂ (j));

   α=I _alpha (i,j);

   Among them, I ₁ represents the second texture expansion image, I ₂ represents the back image of the bust, I ₃ represents the fourth texture expansion image, I _alpha represents the weight, and map ₁ represents in the X-axis direction, the bust’s The pixels in the back image are mapped to the mapping function on the second texture expansion map, map ₂ represents the mapping function on the Y-axis direction, the pixels in the back image of the bust are mapped to the mapping function on the second texture expansion map, i Indicates the coordinate value of the pixel in the X-axis direction, and j represents the coordinate value of the pixel in the X-axis direction.
8. The method according to any one of claims 2-7, wherein, before obtaining the first texture expansion map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, The method also includes:

   Perform at least one of the following processes on the three-dimensional mesh model:

   Hole filling processing, mesh uniformization processing or mesh smoothing processing.
9. A three-dimensional reconstruction device for a bust, which is characterized in that it comprises:

   An acquiring unit, configured to acquire an image to be processed, the image to be processed includes a bust of a target person, and the bust includes a frontal face;

   The reconstruction unit is configured to obtain a first texture expansion map according to the image to be processed, and the first texture expansion map is used to characterize the frontal texture of the bust; wherein, the five senses of the bust in the first texture expansion map At least two organs in are located at preset positions; according to the front texture of the bust in the first texture expansion view, the second texture is obtained by supplementing the back texture of the bust in the first texture expansion view An expanded view, where the second texture expanded view is used to characterize the surface texture of the bust; and a three-dimensional model of the bust is obtained according to the second texture expanded view.
10. 9. The device according to claim 9, wherein the reconstruction unit is specifically configured to:

    Removing the background in the image to be processed to obtain the front image of the bust;

    Performing semantic segmentation on the frontal image to obtain a semantic mask of the head of the frontal image;

    Obtaining a three-dimensional mesh model of the bust according to the head semantic mask and the frontal image;

    According to the positions of the at least two organs on the head in the head semantic mask and the three-dimensional mesh model, a first texture expansion map is obtained, and the at least two organs in the first texture expansion map are located at preset positions.
11. The device according to claim 10, wherein the reconstruction unit, when obtaining the first texture expansion map according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model, Specifically used for:

    Performing texture expansion on the texture corresponding to the three-dimensional mesh model based on a central axis to obtain a third texture expansion diagram, where the central axis is the connecting line from the top end of the head to the bottom end of the head in the three-dimensional mesh model;

    Determining the locations of the at least two organs in the three-dimensional mesh model according to the locations of the at least two organs on the head in the head semantic mask;

    The third texture expansion map is adjusted according to the positions of the at least two organs in the three-dimensional mesh model to obtain the first texture expansion map.
12. The device according to claim 10 or 11, wherein the reconstruction unit is specifically configured to: when obtaining the three-dimensional model of the bust based on the second texture expansion map:

    Smoothing the texture seam area in the second texture expansion map to obtain a fourth texture expansion map, obtain a three-dimensional model of the bust based on the fourth texture expansion map, and the texture seam area according to the The three-dimensional mesh model is used to determine the expansion line used when the texture is expanded.
13. The device of claim 12, wherein the at least two organs include ears; and the reconstruction unit is further used to fuse a pre-configured ear model into the ear region in the three-dimensional mesh model to obtain 3D mesh model after fusion;

    The reconstruction unit is specifically configured to: when obtaining the three-dimensional model of the bust based on the processed second texture expansion map:

    The texture of the ear region on the fused three-dimensional network model is fused to the ear region located at the preset position of the fourth texture expansion map to obtain a fused fourth texture expansion map, which is based on the fused fourth texture expansion map Obtain a three-dimensional model of the bust.
14. The device according to claim 12 or 13, wherein the reconstruction unit is specifically configured to:

    Determining the back texture image of the bust according to the front image of the bust;

    Perform weighted fusion processing on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image.
15. The device according to claim 14, wherein the reconstruction unit performs weighted fusion processing on the second texture expansion image and the back texture image of the bust to obtain the fourth texture expansion image, Specifically used for:

    Performing weighted fusion processing on the second texture expansion image and the back image of the bust according to a set rule to obtain the fourth texture expansion image;

    The setting rules are:

    I ₃ (i,j)=αI ₁ (i,j)+(1-α)I ₂ (map ₁ (i),map ₂ (j));

    α=I _alpha (i,j);

    Among them, I ₁ represents the second texture expansion image, I ₂ represents the back image of the bust, I ₃ represents the fourth texture expansion image, I _alpha represents the weight, and map ₁ represents in the X-axis direction, the bust’s The pixels in the back image are mapped to the mapping function on the second texture expansion map, map ₂ represents the mapping function on the Y-axis direction, the pixels in the back image of the bust are mapped to the mapping function on the second texture expansion map, i Indicates the coordinate value of the pixel in the X-axis direction, and j represents the coordinate value of the pixel in the X-axis direction.
16. The device according to any one of claims 10-15, wherein the reconstruction unit obtains the first position according to the positions of at least two organs on the head in the head semantic mask and the three-dimensional mesh model. Before the texture expansion map, it is also used to:

    Perform at least one of the following processes on the three-dimensional mesh model:

    Hole filling processing, mesh uniformization processing or mesh smoothing processing.
17. An electronic device, characterized by comprising a processor and a memory; wherein the processor is coupled with the memory;

    The memory is used to store program instructions;

    The processor is configured to read the program instructions stored in the memory to implement the method according to any one of claims 1 to 8.
18. A computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, when the program instructions run on an electronic device or a processor, the electronic device executes claims 1 to 8 Any of the methods described.
19. A computer program product, characterized in that, when the computer program product runs on an electronic device, the electronic device or the processor is caused to execute the method according to any one of claims 1 to 8.
20. A chip, characterized in that the chip is coupled with a memory in an electronic device, so that the electronic device executes the method according to any one of claims 1 to 8.

Images