CN113963110A - Texture map generation method and device, electronic equipment and storage medium - Google Patents
Texture map generation method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113963110A CN113963110A CN202111182396.2A CN202111182396A CN113963110A CN 113963110 A CN113963110 A CN 113963110A CN 202111182396 A CN202111182396 A CN 202111182396A CN 113963110 A CN113963110 A CN 113963110A
- Authority
- CN
- China
- Prior art keywords
- texture
- target
- map
- rendering
- round
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
Abstract
The disclosure provides a texture map generation method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical field of enhancement/virtual reality, computer vision and deep learning. The scheme is as follows: executing a plurality of rounds of texture base updating processes according to the target texture coefficients, wherein any round of updating process comprises the steps of fusing the texture base of the round with the target texture coefficients to obtain a texture map of the round, rendering by adopting the texture map of the round to obtain a predicted rendering map of the round, and updating the texture base according to the texture map of the round to obtain the texture base of the next round; stopping performing the update process in a case where a difference between the predicted rendering map and a reference rendering map of the target face image is less than a threshold value; and generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round. Therefore, the expression force category of the texture substrate can be more suitable for the target facial image, and the migration performance and the generalization performance of the texture substrate are improved.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to the field of augmented/virtual reality, computer vision, and deep learning technologies, and in particular, to a texture map generation method and apparatus, an electronic device, and a storage medium.
Background
The face contains a lot of important information of an object, and with the rapid development of related fields such as computer vision, computer technology and the like, the face three-dimensional reconstruction technology is continuously updated. The face three-dimensional reconstruction has important research significance in the aspects of military affairs, medical treatment, security protection, virtual reality, game entertainment and the like. In the face three-dimensional reconstruction, in addition to reconstructing the face shape, a corresponding face texture map needs to be generated.
Disclosure of Invention
The disclosure provides a texture map generation method, a texture map generation device, an electronic device and a storage medium.
According to an aspect of the present disclosure, there is provided a texture map generation method, including: extracting a target texture coefficient from the target face image; and executing a plurality of rounds of texture base updating processes according to the target texture coefficients, wherein any round of updating processes comprises the following steps: fusing the texture substrate of the current round with the target texture coefficient to obtain a texture map of the current round, rendering by adopting the texture map of the current round to obtain a predicted rendering map of the current round, and updating the texture substrate according to the texture map of the current round to obtain the texture substrate of the next round; stopping performing the update process if a difference between the predicted rendering and a reference rendering of the target facial image is less than a threshold; and generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round.
According to another aspect of the present disclosure, there is provided a texture map generating apparatus including: an extraction module for extracting a target texture coefficient from a target face image; an updating module, configured to execute a multi-round updating process of the texture base according to the target texture coefficient, where any one round of updating process includes: fusing the texture substrate of the current round with the target texture coefficient to obtain a texture map of the current round, rendering by adopting the texture map of the current round to obtain a predicted rendering map of the current round, and updating the texture substrate according to the texture map of the current round to obtain the texture substrate of the next round; a stopping module for stopping the execution of the updating process in a case where a difference between the predicted rendering and a reference rendering of the target face image is less than a threshold; and the generating module is used for generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of the first aspect of the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of an embodiment of the first aspect of the present disclosure.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure;
FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure;
FIG. 3 is a schematic diagram according to a third embodiment of the present disclosure;
FIG. 4 is a schematic diagram according to a fourth embodiment of the present disclosure;
FIG. 5 is a schematic diagram according to a fifth embodiment of the present disclosure;
FIG. 6 is a schematic diagram according to a sixth embodiment of the present disclosure;
FIG. 7 is a schematic diagram of a texture map generation flow according to an embodiment of the present disclosure;
FIG. 8 is a schematic diagram according to a seventh embodiment of the present disclosure;
FIG. 9 illustrates a schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
The face contains a lot of important information of an object, and with the rapid development of related fields such as computer vision, computer technology and the like, the face three-dimensional reconstruction technology is continuously updated. The face three-dimensional reconstruction has important research significance in the aspects of military affairs, medical treatment, security protection, virtual reality, game entertainment and the like. In the face three-dimensional reconstruction, in addition to reconstructing the face shape, a corresponding face texture map needs to be generated.
In the related art, a multi-dimensional sheet texture map is extracted from a large number of face scan samples in advance as a fixed texture base to generate a texture image. However, this approach limits the expressiveness of the actually reconstructed face to be within the expressive range of the texture base, and once the expressive range of the texture base is exceeded, the texture image cannot be accurately generated.
In order to solve the above problems, the present disclosure provides a texture map generation method, apparatus, electronic device, and storage medium.
Fig. 1 is a schematic diagram according to a first embodiment of the present disclosure. It should be noted that the texture map generation method according to the embodiment of the present disclosure may be applied to a texture map generation apparatus according to the embodiment of the present disclosure, and the apparatus may be configured in an electronic device. The electronic device may be a mobile terminal, for example, a mobile phone, a tablet computer, a personal digital assistant, and other hardware devices with various operating systems.
As shown in fig. 1, the texture map generating method may include the steps of:
In this disclosure, the target facial image may be a facial image of a target object, where the target object may be a human or an animal, the target facial image may be a facial image acquired online, for example, a facial image of the target object may be acquired online through web crawler technology, or the target facial image may also be a facial image acquired offline, or the target facial image may also be a facial image acquired in real time of the target object, or the target facial image may also be a facial image synthesized by a human, and so on, which is not limited in this disclosure.
In the disclosed embodiments, a target texture coefficient may be extracted from a target face image.
As an example, the target face image may be input into the face texture coefficient model, and the target texture coefficient may be extracted.
As another example, the texture feature extraction may be performed on the target face image by a feature extraction algorithm, and the target texture coefficient may be determined based on the extracted texture feature.
In order to improve the mobility and generalization of the texture base and make the expressive force category of the texture base more suitable for the target facial image, in the embodiment of the present disclosure, multiple rounds of updating may be performed on the texture base according to the target texture coefficients. Wherein, can include in the renewal process of arbitrary round: fusing the texture base of the current round with the target texture coefficient to obtain a texture map of the current round, and rendering the texture map of the current round through a rendering technology to obtain a predicted rendering map of the current round; and meanwhile, updating the texture substrate according to the texture map of the current round so as to obtain the texture substrate of the next round.
In step 103, in the case where the difference between the predicted rendering and the reference rendering of the target face image is less than the threshold value, the execution of the update process is stopped.
In the embodiment of the present disclosure, the target facial image may be subjected to facial rendering to obtain a reference rendering map of the target facial image, then, in any round of updating the texture base, the predicted rendering map of the current round may be compared with the reference rendering map of the target facial image, a difference between the predicted rendering map of the current round and the reference rendering map of the target facial image is determined, the difference is compared with a set threshold, and the updating of the texture base is stopped when the difference between the predicted rendering map and the reference rendering map of the target facial image is smaller than the threshold.
And 104, generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round.
It should be noted that, since the expression force category of the texture substrate of the last round is already applied to the target image, for example, the target facial image is an asian facial image, the initial texture substrate is a european facial texture substrate, and after multiple rounds of updating of the texture substrate, the texture substrate is updated to an asian texture substrate.
Further, the texture base of the last round may be obtained, and a face texture map of the target face image may be generated according to the target texture coefficient and the texture base of the last round.
In conclusion, a multi-round texture base updating process is executed by extracting a target texture coefficient from a target face image and according to the target texture coefficient; wherein, an arbitrary round of updating process includes: fusing the texture substrate of the current round with the target texture coefficient to obtain a texture map of the current round, rendering by adopting the texture map of the current round to obtain a predicted rendering map of the current round, and updating the texture substrate according to the texture map of the current round to obtain the texture substrate of the next round; stopping performing the update process in a case where a difference between the predicted rendering map and a reference rendering map of the target face image is less than a threshold value; and generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round. Therefore, the texture substrate is updated according to the facial texture coefficient of the actually used target facial image, so that the expression scope of the texture substrate is more suitable for the target facial image, the mobility and the generalization of the texture substrate are improved, a more accurate texture map can be generated, meanwhile, a large number of sample images are not needed for training the texture substrate, and the research and development cost of the texture substrate is greatly reduced.
In order to clearly illustrate how the texture base is updated according to the texture map of the current round to obtain the texture base of the next round in the above embodiments, the present disclosure further provides a texture map generating method.
Fig. 2 is a schematic diagram according to a second embodiment of the present disclosure.
As shown in fig. 2, the texture map generating method may include:
In order to make the expression force category of the texture base more suitable for the target face image, in the embodiment of the present disclosure, in the process of updating the texture base in any round, the target texture coefficient corresponding to the target face image and the texture base in the round may be fused to obtain the texture map of the round, and then the texture base is updated according to the texture map of the round to obtain the texture base in the next round.
As an example, the texture map of the current round may be input to a convolutional neural network for feature extraction, so as to obtain a texture base corresponding to the texture map of the current round, and the texture base corresponding to the texture map of the current round is used as a texture base of a next round.
As another example, a texture feature extraction algorithm may be used to perform feature extraction on the texture map of the current round to obtain a texture base corresponding to the texture map of the current round, and the texture base corresponding to the texture map of the current round is used as the texture base of the next round.
In step 203, in the case where the difference between the predicted rendering and the reference rendering of the target face image is less than the threshold value, the execution of the update process is stopped.
And step 204, generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round.
It should be noted that the execution processes of step 201 and steps 203 to 204 may be implemented by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this, and are not described again.
In conclusion, feature extraction is performed on the texture map of the current round to obtain a texture substrate corresponding to the texture map of the current round; and taking the texture substrate corresponding to the texture map of the round as the texture substrate of the next round. Therefore, the texture base of the current round is updated by adopting the texture base corresponding to the texture map of the current round obtained by fusing the texture base of the current round and the target texture coefficient, so that the expression force category of the texture base is more suitable for the target facial image, and the mobility and the generalization of the texture base are improved.
In order to clearly illustrate how to stop performing the update process in the case where the difference between the predicted rendering map and the reference rendering map of the target face image is smaller than the threshold in the above embodiments, the present disclosure also proposes a texture map generation method.
Fig. 3 is a schematic diagram according to a third embodiment of the present disclosure.
As shown in fig. 3, the texture map generating method may include:
step 301 extracts a target texture coefficient from a target face image.
In the embodiment of the present disclosure, in the process of updating the texture substrate, the texture map of the current round may be rendered to obtain the predicted rendering map of the current round, and then the loss function value is determined according to the difference between the predicted rendering map and the reference rendering map of the target facial image in combination with the set loss function, and the texture substrate is updated by using the loss function value in a gradient pass-back manner.
In step 304, the updating process is stopped when the loss function value is less than the threshold value.
Further, in the updating process of the texture base, the loss function value determined according to the difference between the prediction rendering map of the current round and the reference rendering map of the target facial image is compared with a set threshold value, and the updating process of the texture base is stopped when the loss function value is smaller than the threshold value.
It should be noted that the execution processes of the steps 301-302 and 305 may be implemented by any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and are not described again.
In summary, the loss function value is determined by the difference between the predicted rendering and the reference rendering of the target facial image; in the case where the loss function value is smaller than the threshold value, the execution of the update process is stopped. Therefore, a large number of sample images are not needed to update the texture substrate, the development cost of the texture substrate is greatly reduced, the expression scope of the texture substrate is more suitable for the target face image, and the mobility and the generalization of the texture substrate are improved.
In order to clearly illustrate how the facial texture map of the target facial image is generated according to the target texture coefficients and the texture basis of the last round in the above embodiments, the present disclosure also proposes a texture map generation method.
Fig. 4 is a schematic diagram according to a fourth embodiment of the present disclosure.
As shown in fig. 4, the texture map generating method may include:
It should be noted that, in the process of performing multiple rounds of updating of the texture base, the texture base of the current round is fused with the target texture coefficient to obtain the texture map of the current round, and as an example, the texture base of the current round and the target texture coefficient may be linearly summed to generate the texture map of the current round.
In step 403, in the case where the difference between the predicted rendering and the reference rendering of the target face image is less than the threshold value, the execution of the update process is stopped.
In embodiments of the present disclosure, the target texture coefficients may be linearly summed with the texture base of the last round to generate a facial texture map corresponding to the target facial image.
It should be noted that the execution process of steps 401-403 may be implemented by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and are not described again.
In summary, the target texture coefficient and the texture basis of the last round are linearly summed to generate the face texture map of the target face image, and thus, the face texture map corresponding to the target face image can be accurately generated.
In order to clearly illustrate how the texture map of the current round is used for rendering in the above embodiment to obtain the predicted rendering map of the current round, the present disclosure further provides a texture map generation method.
Fig. 5 is a schematic diagram according to a fifth embodiment of the present disclosure.
As shown in fig. 5, the texture map generating method may include:
In the embodiment of the present disclosure, a feature extraction algorithm may be used to extract facial features of a target facial image, and facial key points of the target facial image may be determined according to the extracted facial features.
Further, the face shape corresponding to the target face image can be determined based on the position information of the face key points.
And 505, performing face rendering on the face texture map of the current round and the face shape corresponding to the target face image to obtain a predicted rendering map of the current round.
In the embodiment of the present disclosure, a 3D rendering technology may be adopted to perform face rendering on the face texture map of the current round and the face shape corresponding to the target face image, so as to obtain the predicted rendering map of the current round.
In step 506, in the case where the difference between the predicted rendering and the reference rendering of the target face image is less than the threshold value, the execution of the update process is stopped.
It should be noted that the execution processes of steps 501-502 and 506-507 may be implemented by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and are not described again.
In conclusion, facial feature extraction is carried out on the target facial image to obtain facial key points corresponding to the target facial image; determining a face shape corresponding to the target face image according to the key points of the face; and performing face rendering on the face texture image of the current round and the face shape corresponding to the target face image to obtain a predicted rendering image of the current round. Therefore, the predicted rendering map of the current round can be accurately acquired.
In order to accurately obtain a target texture coefficient corresponding to a target face image, as shown in fig. 6, fig. 6 is a schematic diagram according to a sixth embodiment of the present disclosure, in the embodiment of the present disclosure, the target face image may be input into a trained facial texture coefficient model to obtain a target texture coefficient, and therefore, before performing multiple rounds of texture base updating processes according to the target texture coefficient, the embodiment shown in fig. 6 may include the following steps:
In the disclosed embodiment, the target image may be input into an initial facial texture coefficient model, which performs texture coefficient extraction on the target facial image to obtain initial texture coefficients. The facial texture coefficient model may be a convolutional neural network.
In an embodiment of the present disclosure, the initial texture base may be a texture base of a first round in a texture base updating process, and the initial texture base may be a set texture base, where the set texture base may be a texture base of the same type as the target facial image, or a texture base of a different type from the target facial image, or a set texture base is blank, which is not limited in the present disclosure.
In embodiments of the present disclosure, the initial texture base may be linearly summed with the initial texture coefficients to obtain an initial texture map.
Further, 3D rendering is performed on the initial texture map by using a 3D rendering technology, so that an initial predicted rendering map can be obtained.
Further, the initial predicted rendering may be compared to a reference rendering of the target facial image to determine a difference between the initial predicted rendering and the reference rendering of the target facial image, and the facial texture coefficient model may be trained to minimize the difference between the initial predicted rendering and the reference rendering of the target facial image based on the difference between the initial predicted rendering and the reference rendering of the target facial image. That is, after the facial texture coefficient model is stabilized, a plurality of rounds of texture base updating processes are performed, and thus, the expression force category of the texture base after updating can be more suitable for the target facial image.
In this disclosure, the target facial image may be a facial image of a target object, where the target object may be a human or an animal, the target facial image may be a facial image acquired online, for example, a facial image of the target object may be acquired online through web crawler technology, or the target facial image may also be a facial image acquired offline, or the target facial image may also be a facial image acquired in real time of the target object, or the target facial image may also be a facial image synthesized by a human, and so on, which is not limited in this disclosure.
Furthermore, the target face image is input into the trained face texture coefficient model, and the trained face texture coefficient model can accurately extract the target texture coefficient corresponding to the target face image.
In step 608, in the event that the difference between the predicted rendering and the reference rendering of the target face image is less than the threshold, execution of the update process is stopped.
And step 609, generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round.
It should be noted that the execution process of steps 607-609 may be implemented by any one of the embodiments of the present disclosure, which is not limited by the embodiments of the present disclosure and is not repeated herein
In conclusion, the target face image is input into the face texture coefficient model to extract an initial texture coefficient; fusing the initial texture substrate and the initial texture coefficient to obtain an initial texture map; rendering by adopting the initial texture map to obtain an initial prediction rendering map; training the facial texture coefficient model according to the difference between the initial predicted rendering and the reference rendering of the target facial image to minimize the difference between the initial predicted rendering and the reference rendering of the target facial image; acquiring a target face image; and inputting the target face image into the trained face texture coefficient model to extract the target texture coefficient. Therefore, before the updating process of the multi-round texture base is executed, the facial texture model is trained, the target texture coefficient can be accurately obtained, then, the updating process of the multi-round texture base is executed according to the target texture coefficient, the expression scope of the updated texture base can be more suitable for the target facial image, the mobility and the generalization of the texture base are improved, and a more accurate texture map can be generated.
In order to more clearly illustrate the above embodiments, the description will now be made by way of example.
As shown in fig. 7, taking the target face image as a 2D face image as an example, first, the 2D face image may be input into a Convolutional Neural Network (CNN) (trained facial texture coefficient model), obtain a target texture coefficient, linearly sum the target texture coefficient and a fixed texture base to generate a texture map, perform 3D rendering on the texture map to generate a predicted rendering map, compare the predicted rendering map with a reference rendering map of the 2D face image, determine a difference between the predicted rendering map and the reference rendering map of the 2D face image, perform multiple rounds of updating on the texture base by using a gradient pass-back method (backsward) for the difference, and stop performing an updating process of the texture base when the difference between the predicted rendering map and the reference rendering map of the 2D face image is smaller than a threshold. And then, generating a face texture image of the 2D face image according to the target texture coefficient and the texture base of the last round.
According to the texture map generation method, a target texture coefficient is extracted from a target face image, and a multi-round texture base updating process is executed according to the target texture coefficient; wherein, an arbitrary round of updating process includes: fusing the texture substrate of the current round with the target texture coefficient to obtain a texture map of the current round, rendering by adopting the texture map of the current round to obtain a predicted rendering map of the current round, and updating the texture substrate according to the texture map of the current round to obtain the texture substrate of the next round; stopping performing the update process in a case where a difference between the predicted rendering map and a reference rendering map of the target face image is less than a threshold value; and generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round. Therefore, the texture substrate is updated according to the facial texture coefficient of the actually used target facial image, so that the expression scope of the texture substrate is more suitable for the target facial image, the mobility and the generalization of the texture substrate are improved, a more accurate texture map can be generated, meanwhile, a large number of sample images are not needed for training the texture substrate, and the research and development cost of the texture substrate is greatly reduced.
In order to implement the above embodiments, the present disclosure further provides a texture map generating device.
Fig. 8 is a schematic diagram according to a seventh embodiment of the present disclosure, and as shown in fig. 8, the texture map generating apparatus includes: an extraction module 810, an update module 820, a stop module 830, and a generation module 840.
Wherein, the extracting module 810 is configured to extract a target texture coefficient from the target face image; an updating module 820, configured to perform multiple rounds of texture base updating processes according to the target texture coefficients, where any round of the updating processes includes: fusing the texture substrate of the current round with the target texture coefficient to obtain a texture map of the current round, rendering by adopting the texture map of the current round to obtain a predicted rendering map of the current round, and updating the texture substrate according to the texture map of the current round to obtain the texture substrate of the next round; a stopping module 830 for stopping performing the updating process in a case where a difference between the predicted rendering and the reference rendering of the target face image is less than a threshold; a generating module 840, configured to generate a face texture map of the target face image according to the target texture coefficients and the texture basis of the last round.
As a possible implementation manner of the embodiment of the present disclosure, the updating module 820 is configured to: extracting the characteristics of the texture map of the wheel to obtain a texture substrate corresponding to the texture map of the wheel; and taking the texture substrate corresponding to the texture map of the round as the texture substrate of the next round.
As a possible implementation manner of the embodiment of the present disclosure, the stopping module 830 is configured to: determining a loss function value according to the difference between the predicted rendering map and a reference rendering map of the target face image; in the case where the loss function value is smaller than the threshold value, the execution of the update process is stopped.
As a possible implementation manner of the embodiment of the present disclosure, the generating module 840 is configured to: and linearly summing the target texture coefficient and the texture base of the last round to generate a face texture map of the target face image.
As a possible implementation manner of the embodiment of the present disclosure, the updating module 820 is configured to: extracting facial features of the target facial image to obtain facial key points corresponding to the target facial image; determining a face shape corresponding to the target face image according to the key points of the face; and performing face rendering on the face texture map of the current round and the face shape corresponding to the target face image to obtain a predicted rendering map of the current round.
As a possible implementation manner of the embodiment of the present disclosure, the texture map generating apparatus further includes: the device comprises an input module, a fusion module, a rendering module and a training module.
The input module is used for inputting the target face image into the face texture coefficient model so as to extract an initial texture coefficient; the fusion module is used for fusing the initial texture substrate and the initial texture coefficient to obtain an initial texture map; the rendering module is used for rendering by adopting the initial texture map to obtain an initial prediction rendering map; a training module for training the facial texture coefficient model based on a difference between the initial predicted rendering and the reference rendering of the target facial image to minimize the difference between the initial predicted rendering and the reference rendering of the target facial image.
As a possible implementation manner of the embodiment of the present disclosure, the extracting module 810 is configured to: acquiring a target face image; and inputting the target face image into the trained face texture coefficient model to extract the target texture coefficient.
The texture map generation device of the disclosed embodiment performs a multi-round texture base update process by extracting a target texture coefficient from a target face image and according to the target texture coefficient; wherein, an arbitrary round of updating process includes: fusing the texture substrate of the current round with the target texture coefficient to obtain a texture map of the current round, rendering by adopting the texture map of the current round to obtain a predicted rendering map of the current round, and updating the texture substrate according to the texture map of the current round to obtain the texture substrate of the next round; stopping performing the update process in a case where a difference between the predicted rendering map and a reference rendering map of the target face image is less than a threshold value; and generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round. Therefore, the texture substrate is updated according to the facial texture coefficient of the actually used target facial image, so that the expression scope of the texture substrate is more suitable for the target facial image, the mobility and the generalization of the texture substrate are improved, a more accurate texture map can be generated, meanwhile, a large number of sample images are not needed for training the texture substrate, and the research and development cost of the texture substrate is greatly reduced.
In the technical scheme of the disclosure, the collection, storage, use, processing, transmission, provision, disclosure and other processing of the personal information of the related user are all carried out on the premise of obtaining the consent of the user, and all accord with the regulation of related laws and regulations without violating the good custom of the public order.
The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.
FIG. 9 illustrates a schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 9, the apparatus 900 includes a computing unit 901, which can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The calculation unit 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.
A number of components in the device 900 are connected to the I/O interface 905, including: an input unit 906 such as a keyboard, a mouse, and the like; an output unit 907 such as various types of displays, speakers, and the like; a storage unit 908 such as a magnetic disk, optical disk, or the like; and a communication unit 909 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 909 allows the device 900 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.
The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 901 performs the respective methods and processes described above, such as the texture map generation method. For example, in some embodiments, the texture map generation method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 900 via ROM 902 and/or communications unit 909. When the computer program is loaded into RAM 903 and executed by computing unit 901, one or more steps of the texture map generation method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the texture map generation method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (17)
1. A texture map generation method, comprising:
extracting a target texture coefficient from the target face image;
and executing a plurality of rounds of texture base updating processes according to the target texture coefficients, wherein any round of updating processes comprises the following steps: fusing the texture substrate of the current round with the target texture coefficient to obtain a texture map of the current round, rendering by adopting the texture map of the current round to obtain a predicted rendering map of the current round, and updating the texture substrate according to the texture map of the current round to obtain the texture substrate of the next round;
stopping performing the update process if a difference between the predicted rendering and a reference rendering of the target facial image is less than a threshold;
and generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round.
2. The method according to claim 1, wherein the performing texture substrate updating according to the texture map of the current round to obtain the texture substrate of the next round comprises:
performing feature extraction on the texture map of the current round to obtain a texture substrate corresponding to the texture map of the current round;
and taking the texture substrate corresponding to the texture map of the current round as the texture substrate of the next round.
3. The method of claim 1, wherein the stopping the updating process from being performed in the event that the difference between the predicted rendering and the reference rendering of the target facial image is less than a threshold value comprises:
determining a loss function value according to a difference between the predicted rendering map and a reference rendering map of the target facial image;
stopping performing the update procedure if the loss function value is less than a threshold.
4. The method of claim 1, wherein the generating a face texture map of the target face image from the target texture coefficients and a texture base of a last round comprises:
and linearly summing the target texture coefficient and the texture base of the last round to generate a face texture map of the target face image.
5. The method of claim 1, wherein the rendering with the texture map of the current round to obtain the predicted rendering map of the current round comprises:
extracting facial features of the target facial image to obtain facial key points corresponding to the target facial image;
determining a face shape corresponding to the target face image according to the face key points;
and performing face rendering on the face texture map of the current wheel and the face shape corresponding to the target face image to obtain a predicted rendering map of the current wheel.
6. The method according to claim 1, wherein before performing the plurality of rounds of the updating process of the texture base according to the target texture coefficient, the method further comprises:
inputting the target face image into a face texture coefficient model to extract an initial texture coefficient;
fusing an initial texture substrate and the initial texture coefficient to obtain an initial texture map;
rendering by adopting the initial texture map to obtain an initial prediction rendering map;
training the facial texture coefficient model to minimize a difference between the initial predicted rendering and a reference rendering of the target facial image based on the difference between the initial predicted rendering and the reference rendering of the target facial image.
7. The method of claim 6, wherein said extracting a target texture coefficient from a target facial image comprises:
acquiring the target face image;
and inputting the target facial image into a trained facial texture coefficient model to extract the target texture coefficient.
8. A texture map generating apparatus comprising:
an extraction module for extracting a target texture coefficient from a target face image;
an updating module, configured to execute a multi-round updating process of the texture base according to the target texture coefficient, where any one round of updating process includes: fusing the texture substrate of the current round with the target texture coefficient to obtain a texture map of the current round, rendering by adopting the texture map of the current round to obtain a predicted rendering map of the current round, and updating the texture substrate according to the texture map of the current round to obtain the texture substrate of the next round;
a stopping module for stopping the execution of the updating process in a case where a difference between the predicted rendering and a reference rendering of the target face image is less than a threshold;
and the generating module is used for generating a face texture map of the target face image according to the target texture coefficient and the texture base of the last round.
9. The apparatus of claim 8, wherein the update module is to:
performing feature extraction on the texture map of the current round to obtain a texture substrate corresponding to the texture map of the current round;
and taking the texture substrate corresponding to the texture map of the current round as the texture substrate of the next round.
10. The apparatus of claim 8, wherein the stopping module is to:
determining a loss function value according to a difference between the predicted rendering map and a reference rendering map of the target facial image;
stopping performing the update procedure if the loss function value is less than a threshold.
11. The apparatus of claim 8, wherein the generating means is configured to:
and linearly summing the target texture coefficient and the texture base of the last round to generate a face texture map of the target face image.
12. The apparatus of claim 8, wherein the update module is to:
extracting facial features of the target facial image to obtain facial key points corresponding to the target facial image;
determining a face shape corresponding to the target face image according to the face key points;
and performing face rendering on the face texture map of the current wheel and the face shape corresponding to the target face image to obtain a predicted rendering map of the current wheel.
13. The apparatus of claim 8, wherein the apparatus further comprises:
an input module, configured to input the target face image into a facial texture coefficient model to extract an initial texture coefficient;
the fusion module is used for fusing an initial texture substrate and the initial texture coefficient to obtain an initial texture map;
the rendering module is used for rendering by adopting the initial texture map to obtain an initial prediction rendering map;
a training module for training the facial texture coefficient model according to a difference between the initial predicted rendering and the reference rendering of the target facial image to minimize the difference between the initial predicted rendering and the reference rendering of the target facial image.
14. The apparatus of claim 13, wherein the extraction module is to:
acquiring the target face image;
and inputting the target facial image into a trained facial texture coefficient model to extract the target texture coefficient.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program which, when being executed by a processor, carries out the steps of the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111182396.2A CN113963110B (en) | 2021-10-11 | 2021-10-11 | Texture map generation method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111182396.2A CN113963110B (en) | 2021-10-11 | 2021-10-11 | Texture map generation method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113963110A true CN113963110A (en) | 2022-01-21 |
| CN113963110B CN113963110B (en) | 2022-10-25 |
Family
ID=79463794
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111182396.2A Active CN113963110B (en) | 2021-10-11 | 2021-10-11 | Texture map generation method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113963110B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114092673A (en) * | 2021-11-23 | 2022-02-25 | 北京百度网讯科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| WO2023160054A1 (en) * | 2022-02-25 | 2023-08-31 | 腾讯科技（深圳）有限公司 | Image rendering method and apparatus, electronic device, computer-readable storage medium, and computer program product |
| WO2023165198A1 (en) * | 2022-03-03 | 2023-09-07 | 腾讯科技（深圳）有限公司 | Image rendering method and apparatus, electronic device, computer-readable storage medium, and computer program product |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1990015322A1 (en) * | 1989-05-31 | 1990-12-13 | Ishikawajima-Harima Jukogyo Kabushiki Kaisha | Texture measuring method and texture control method and apparatus using the texture measuring method |
| CN111882643A (en) * | 2020-08-10 | 2020-11-03 | 网易（杭州）网络有限公司 | Three-dimensional face construction method and device and electronic equipment |
| CN113240792A (en) * | 2021-04-29 | 2021-08-10 | 浙江大学 | Image fusion generation type face changing method based on face reconstruction |
| CN113313631A (en) * | 2021-06-10 | 2021-08-27 | 北京百度网讯科技有限公司 | Image rendering method and device |
| CN113327278A (en) * | 2021-06-17 | 2021-08-31 | 北京百度网讯科技有限公司 | Three-dimensional face reconstruction method, device, equipment and storage medium |
| CN113469883A (en) * | 2021-06-30 | 2021-10-01 | 完美世界(北京)软件科技发展有限公司 | Rendering method and device of dynamic resolution, electronic equipment and readable storage medium |
-
2021
- 2021-10-11 CN CN202111182396.2A patent/CN113963110B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1990015322A1 (en) * | 1989-05-31 | 1990-12-13 | Ishikawajima-Harima Jukogyo Kabushiki Kaisha | Texture measuring method and texture control method and apparatus using the texture measuring method |
| CN111882643A (en) * | 2020-08-10 | 2020-11-03 | 网易（杭州）网络有限公司 | Three-dimensional face construction method and device and electronic equipment |
| CN113240792A (en) * | 2021-04-29 | 2021-08-10 | 浙江大学 | Image fusion generation type face changing method based on face reconstruction |
| CN113313631A (en) * | 2021-06-10 | 2021-08-27 | 北京百度网讯科技有限公司 | Image rendering method and device |
| CN113327278A (en) * | 2021-06-17 | 2021-08-31 | 北京百度网讯科技有限公司 | Three-dimensional face reconstruction method, device, equipment and storage medium |
| CN113469883A (en) * | 2021-06-30 | 2021-10-01 | 完美世界(北京)软件科技发展有限公司 | Rendering method and device of dynamic resolution, electronic equipment and readable storage medium |
Non-Patent Citations (3)
| Title |
|---|
| JIANG L等: "3D face reconstruction with geometry details from a single image", 《 IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
| 唐勇等: "基于多项式纹理的Lengyel毛发实时绘制", 《计算机工程》 * |
| 李印等: "基于小波变换的PET/CT图像融合算法研究进展", 《中国医学影像技术》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114092673A (en) * | 2021-11-23 | 2022-02-25 | 北京百度网讯科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN114092673B (en) * | 2021-11-23 | 2022-11-04 | 北京百度网讯科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| WO2023160054A1 (en) * | 2022-02-25 | 2023-08-31 | 腾讯科技（深圳）有限公司 | Image rendering method and apparatus, electronic device, computer-readable storage medium, and computer program product |
| WO2023165198A1 (en) * | 2022-03-03 | 2023-09-07 | 腾讯科技（深圳）有限公司 | Image rendering method and apparatus, electronic device, computer-readable storage medium, and computer program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113963110B (en) | 2022-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113963110B (en) | Texture map generation method and device, electronic equipment and storage medium | |
| CN113643412B (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| CN114549710A (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| CN113407850B (en) | Method and device for determining and acquiring virtual image and electronic equipment | |
| CN112580666A (en) | Image feature extraction method, training method, device, electronic equipment and medium | |
| CN114708374A (en) | Virtual image generation method and device, electronic equipment and storage medium | |
| CN114529796A (en) | Model training method, image recognition method, device and electronic equipment | |
| CN112949433B (en) | Method, device and equipment for generating video classification model and storage medium | |
| CN113627361A (en) | Training method and device for face recognition model and computer program product | |
| CN113870399A (en) | Expression driving method and device, electronic equipment and storage medium | |
| CN113177466A (en) | Identity recognition method and device based on face image, electronic equipment and medium | |
| CN113052962A (en) | Model training method, information output method, device, equipment and storage medium | |
| CN112784967B (en) | Information processing method and device and electronic equipment | |
| CN114549785A (en) | Method and device for generating model substrate, electronic equipment and storage medium | |
| CN115393514A (en) | Training method of three-dimensional reconstruction model, three-dimensional reconstruction method, device and equipment | |
| CN113903071A (en) | Face recognition method and device, electronic equipment and storage medium | |
| CN114078097A (en) | Method and device for acquiring image defogging model and electronic equipment | |
| CN114463361A (en) | Network model training method, device, equipment, medium and program product | |
| CN114529649A (en) | Image processing method and device | |
| CN113408304A (en) | Text translation method and device, electronic equipment and storage medium | |
| CN114092616B (en) | Rendering method, rendering device, electronic equipment and storage medium | |
| CN116229214B (en) | Model training method and device and electronic equipment | |
| CN114549926A (en) | Target detection and target detection model training method and device | |
| CN114550251A (en) | Face recognition method and device, electronic equipment and storage medium | |
| CN113361719A (en) | Incremental learning method based on image processing model and image processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |