CN116030185A - Three-dimensional hairline generating method and model training method - Google Patents
Three-dimensional hairline generating method and model training method Download PDFInfo
- Publication number
- CN116030185A CN116030185A CN202211542715.0A CN202211542715A CN116030185A CN 116030185 A CN116030185 A CN 116030185A CN 202211542715 A CN202211542715 A CN 202211542715A CN 116030185 A CN116030185 A CN 116030185A
- Authority
- CN
- China
- Prior art keywords
- hair
- hairstyle
- dimensional
- training
- hidden vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The present disclosure provides a three-dimensional hairstyle generating method and a model training method, which relate to the technical field of artificial intelligence, in particular to the technical fields of augmented reality, virtual reality, computer vision, deep learning, etc., and can be applied to scenes such as meta universe, virtual digital people, etc. The implementation scheme is as follows: obtaining a plurality of first images for a plurality of perspectives of a first object, the first object comprising a hairstyle area comprising a plurality of hair strands; obtaining a plurality of first trend graphs corresponding to the plurality of first images based on the plurality of first images, each of the plurality of first trend graphs indicating a direction of a hair line segment corresponding to each pixel in a region corresponding to a hair style region of a first object in a corresponding one of the plurality of images; obtaining a first hairstyle hidden vector of a hairstyle area of a first object based on the plurality of first trend graphs; and obtaining the three-dimensional hairstyle corresponding to the first object based on the first hairstyle hidden vector.
Description
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of augmented reality, virtual reality, computer vision, deep learning and the like, and can be applied to scenes such as metauniverse, virtual digital man and the like, in particular to a three-dimensional hairline generating method, a training method of a model, a device, electronic equipment, a computer readable storage medium and a computer program product.
Background
Artificial intelligence is the discipline of studying the process of making a computer mimic certain mental processes and intelligent behaviors (e.g., learning, reasoning, thinking, planning, etc.) of a person, both hardware-level and software-level techniques. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge graph technology and the like.
The three-dimensional virtual image has wide application value in social, live broadcast, game and other user scenes. The three-dimensional virtual image generation based on artificial intelligence generates the virtual image through the face image, and the personalized virtual image customized for the user can effectively meet the personalized requirements of the user, and has wide application prospect.
The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, the problems mentioned in this section should not be considered as having been recognized in any prior art unless otherwise indicated.
Disclosure of Invention
The present disclosure provides a three-dimensional hairstyle generation method, a model training method, an apparatus, an electronic device, a computer readable storage medium and a computer program product.
According to an aspect of the present disclosure, there is provided a three-dimensional hairstyle generating method, including: obtaining a plurality of first images for a plurality of perspectives of a first object, the first object comprising a hairstyle area comprising a plurality of hair strands; obtaining a plurality of first trend graphs corresponding to the plurality of first images based on the plurality of first images, each of the plurality of first trend graphs indicating a direction of a hair line segment corresponding to each pixel in a region corresponding to a hair style region of the first object in a respective image of the plurality of images; obtaining a first hair style hidden vector of a hair style area of the first object based on the plurality of first trend graphs; and obtaining a three-dimensional hairstyle corresponding to the first object based on the first hairstyle hidden vector, the three-dimensional hairstyle including hairline data for each of the plurality of hairlines, the hairline data including coordinates for each of a plurality of nodes on the hairline.
According to another aspect of the present disclosure, there is provided a training method of a three-dimensional hairstyle generation model including a feature extraction network and a decoder, the method comprising: obtaining a three-dimensional hair data set comprising coordinates of each of a plurality of nodes on each of a plurality of hair strands from a hairstyle area of a first head mold; based on the three-dimensional hairline dataset, obtaining a plurality of training trend graphs of the first head model corresponding to a plurality of visual angles, wherein each training trend graph in the plurality of training trend graphs corresponds to an image of the first head model at a corresponding visual angle in the plurality of visual angles and indicates the direction of a hairline segment corresponding to each pixel in a hairstyle area in the image; acquiring training hair style hidden vectors corresponding to the three-dimensional hair data set based on the training trend graphs by utilizing the feature extraction network; obtaining a prediction result corresponding to the three-dimensional hair data set based on the training hair style hidden vector by using the decoder, wherein the prediction result comprises coordinates of each of a plurality of nodes on each of the plurality of hair; and adjusting parameters of the three-dimensional hairstyle generation model based on the three-dimensional hairline dataset and the prediction result.
According to another aspect of the present disclosure, there is provided a three-dimensional hairstyle generating device comprising: a first image acquisition unit configured to obtain a plurality of first images of a plurality of perspectives with respect to a first object, the first object including a hair styling area containing a plurality of hair; a first trend graph obtaining unit configured to obtain, based on the plurality of first images, a plurality of first trend graphs corresponding to the plurality of first images, each of the plurality of first trend graphs indicating a direction of a hair line segment corresponding to each pixel in a region corresponding to a hairstyle region of the first object in a corresponding one of the plurality of images; a first hair style hidden vector acquisition unit configured to acquire a first hair style hidden vector of a hair style area of the first object based on the plurality of first trend graphs; and a three-dimensional hair style acquisition unit configured to acquire a three-dimensional hair style corresponding to the first object based on the first hair style hidden vector, the three-dimensional hair style including hair line data for each of the plurality of hair lines, the hair line data including coordinates of each of a plurality of nodes on the hair line.
According to another aspect of the present disclosure, there is provided a training apparatus of a three-dimensional hairstyle generation model including a feature extraction network and a decoder, the apparatus comprising: a three-dimensional hair-line data set acquisition unit configured to acquire a three-dimensional hair-line data set including coordinates of each of a plurality of nodes on each of a plurality of hair lines from a hairstyle area of a first head model; a training trend graph obtaining unit, configured to obtain a plurality of training trend graphs corresponding to a plurality of view angles of the first head model based on the three-dimensional hair dataset, where each training trend graph in the plurality of training trend graphs corresponds to an image of the first head model at a corresponding view angle in the plurality of view angles, and indicates a direction of a hair line segment corresponding to each pixel in a hair style area in the image; a training hair style hidden vector obtaining unit, configured to obtain training hair style hidden vectors corresponding to the three-dimensional hair data set based on the plurality of training trend graphs by using the feature extraction network; a prediction result obtaining unit configured to obtain, based on the training hair style hidden vector, a prediction result corresponding to the three-dimensional hair data set, the prediction result including coordinates of each of a plurality of nodes on each of a plurality of hair strands of the first head model; and a parameter adjustment unit configured to adjust parameters of the three-dimensional hairstyle generation model based on the three-dimensional hairline dataset and the prediction result.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method according to embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method according to an embodiment of the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements a method according to embodiments of the present disclosure.
According to one or more embodiments of the present disclosure, it is achieved that a three-dimensional hairstyle is directly and automatically generated based on a two-dimensional image, the generation efficiency of the three-dimensional hairstyle is improved, and simultaneously, the data processing amount is reduced.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The accompanying drawings illustrate exemplary embodiments and, together with the description, serve to explain exemplary implementations of the embodiments. The illustrated embodiments are for exemplary purposes only and do not limit the scope of the claims. Throughout the drawings, identical reference numerals designate similar, but not necessarily identical, elements.
FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, in accordance with an embodiment of the present disclosure;
FIG. 2 illustrates a flow chart of a three-dimensional hairstyle generation method according to an embodiment of the present disclosure;
FIG. 3 is a flow chart illustrating a process of obtaining a first hair style hidden vector based on a plurality of first trend graphs in a three-dimensional hair style generation method according to an embodiment of the present disclosure;
fig. 4 is a flowchart illustrating a process of obtaining a three-dimensional hairstyle corresponding to a first object based on a first hairstyle hidden vector in a three-dimensional hairstyle generation method according to an embodiment of the present disclosure;
FIG. 5 is a flow chart illustrating a process of modifying a first hair style hidden vector to obtain a target hair style hidden vector in a three-dimensional hair style generation method according to an embodiment of the present disclosure;
FIG. 6 illustrates a flowchart of a training method of a three-dimensional hairstyle generation model in accordance with an embodiment of the present disclosure;
FIG. 7 shows a comparative schematic diagram of a training process and a prediction process of a three-dimensional hairstyle generation model in a training method of a three-dimensional hairstyle generation model according to the present disclosure;
fig. 8 is a block diagram illustrating a structure of a three-dimensional hairstyle generating apparatus according to an embodiment of the present disclosure;
FIG. 9 shows a block diagram of a training device of a three-dimensional hairstyle generation model in accordance with an embodiment of the present disclosure; and
fig. 10 illustrates a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the present disclosure, the use of the terms "first," "second," and the like to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of the elements, unless otherwise indicated, and such terms are merely used to distinguish one element from another element. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, they may also refer to different instances based on the description of the context.
The terminology used in the description of the various illustrated examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, the elements may be one or more if the number of the elements is not specifically limited. Furthermore, the term "and/or" as used in this disclosure encompasses any and all possible combinations of the listed items.
Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented, in accordance with an embodiment of the present disclosure. Referring to fig. 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. Client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.
In an embodiment of the present disclosure, the server 120 may run one or more services or software applications enabling the execution of the three-dimensional hairstyle generation method according to the present disclosure.
In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, for example, provided to users of client devices 101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) model.
In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof that are executable by one or more processors. A user operating client devices 101, 102, 103, 104, 105, and/or 106 may in turn utilize one or more client applications to interact with server 120 to utilize the services provided by these components. It should be appreciated that a variety of different system configurations are possible, which may differ from system 100. Accordingly, FIG. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.
The user may use the client devices 101, 102, 103, 104, 105 and/or 106 to obtain the three-dimensional hairstyle generated in the three-dimensional hairstyle generation method according to the present disclosure. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that the present disclosure may support any number of client devices.
The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture that involves virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain virtual storage devices of the server). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.
The computing units in server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. Server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.
In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and/or event updates received from users of client devices 101, 102, 103, 104, 105, and/or 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and/or 106.
In some implementations, the server 120 may be a server of a distributed system or a server that incorporates a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical host and virtual private server (VPS, virtual Private Server) service.
The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of databases 130 may be used to store information such as audio files and video files. Database 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. Database 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve the databases and data from the databases in response to the commands.
In some embodiments, one or more of databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key value stores, object stores, or conventional stores supported by the file system.
The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.
According to one aspect of the present disclosure, a three-dimensional hairstyle generation method is provided. As shown in fig. 2, the three-dimensional hairstyle generation method 200 includes:
step S210: obtaining a plurality of first images for a plurality of perspectives of a first object, the first object comprising a hairstyle area comprising a plurality of hair strands;
step S220: obtaining a plurality of first trend graphs corresponding to the plurality of first images based on the plurality of first images, each of the plurality of first trend graphs indicating a direction of a hair line segment corresponding to each pixel in a region corresponding to a hair style region of the first object in a respective image of the plurality of images;
step S230: obtaining a first hair style hidden vector of a hair style area of the first object based on the plurality of first trend graphs; and
step S240: and obtaining a three-dimensional hairstyle corresponding to the first object based on the first hairstyle hidden vector, wherein the three-dimensional hairstyle comprises hairline data of each hairline in the plurality of hairlines, and the hairline data comprises coordinates of each node in the plurality of nodes on the hairline.
In the related art, a three-dimensional hairstyle is designed according to images of multiple views, a designer is required to manually construct a hairline, and the three-dimensional hairline is obtained based on the constructed hairline, so that the production cost of the three-dimensional hairline is increased, and the production efficiency is low.
According to the embodiment of the disclosure, the three-dimensional hairstyle can be automatically generated based on the two-dimensional image by obtaining the plurality of first trend graphs based on the plurality of first images of the plurality of view angles of the first object and obtaining the hairstyle hidden vector based on the plurality of first trend graphs, and finally obtaining the three-dimensional hairstyle based on the hairstyle hidden vector, so that the designer is replaced by manually relating to the hairstyle, and the generation efficiency of the three-dimensional hairstyle is improved. Meanwhile, according to the method disclosed by the invention, the method for decoding the hidden vector of the hairstyle obtained by aiming at the image to obtain the three-dimensional hairstyle does not need to construct or grow aiming at a single hairline, and compared with the method for constructing or growing aiming at the single hairline to generate the three-dimensional hairstyle, the method for decoding the hidden vector of the hairstyle can simplify the processing flow and reduce the data processing capacity.
In some embodiments, the first object may be a person, or any three-dimensional object with hair such as a head form.
In some embodiments, in step S210, images of a first object are acquired from a plurality of perspectives by an imaging device to obtain the plurality of first images. Wherein the plurality of viewing angles may be any two or more viewing angles.
For example, the plurality of first images are a plurality of images acquired by the image pickup device during 360 ° rotation around a plane in which the head circumference of the first object is located.
In some embodiments, the plurality of perspectives includes at least four perspectives, front, back, left and right, corresponding to the first object.
The plurality of view angles comprise front, back, left and right view angles of the first object, so that the obtained plurality of first images of the first object comprise information reflecting all hairstyle areas of the first object, and the hairstyle hidden vector obtained based on the plurality of first images can reflect all information of the first object in a three-dimensional space, and therefore the obtained hairstyle hidden vector is accurate, and the three-dimensional hairstyle obtained based on the hairstyle hidden vector can more accurately reflect the hairstyle of the first object. And simultaneously, only the first images of the front, back, left and right view angles of the first object are processed to obtain the three-dimensional hairstyle, and the data processing amount is as small as possible while the three-dimensional hairstyle accurately representing the hairstyle of the first object is obtained.
In some scenes, a designer of a three-dimensional hairstyle performs hairstyle design based on views of four front, rear, left and right views of a first object, and according to an embodiment of the present disclosure, by obtaining a plurality of first images including a plurality of views of four front, rear, left and right views corresponding to the first object, the method according to the present disclosure can be directly applied to an application scene in which the designer of a three-dimensional hairstyle designs a three-dimensional hairstyle.
It should be noted that the three-dimensional hairstyle generating method according to the present disclosure is merely exemplary in a scene where a designer of a three-dimensional hairstyle designs a three-dimensional hairstyle, and may be applied to any scene where a three-dimensional hairstyle needs to be generated, which is not limited herein.
In some embodiments, in step S220, a first trend map corresponding to each of the plurality of first images is obtained by Gabor (Gabor) filtering a hairstyle area of the first image.
In some embodiments, in step S220, each first image is input into an image generation model to generate a corresponding first trend graph, where the image generation model is obtained by training with a training image and a trend graph corresponding to the training image. The training image is an image of a hairstyle area containing a training object, and the trend graph is a trend graph obtained by rendering a three-dimensional hairstyle data set.
In some embodiments, in step S220, a segmentation process for the hair style region is first performed on each of the plurality of first images to obtain a mask image corresponding to the hair style region in the first image, and then a first trend graph corresponding to the first image is obtained based on the mask image, for example, based on a pixel gradient in the mask image, a direction of a hair line segment corresponding to a pixel is extracted, so as to obtain the first trend graph.
The pixels of the three colors of RGB in the trend chart correspond to the direction vectors of XYZ axes in the three-dimensional space, respectively. In the process of obtaining the first trend graph, the direction vector of each point in the hairstyle area in the first image in the XYZ space is normalized to be a value between 0 and 255, so that each pixel in the obtained first trend graph indicates the direction of a hairline segment corresponding to each pixel in the area corresponding to the hairstyle area of the first object in the corresponding first image.
In some embodiments, in step S230, feature extraction is performed for each of the plurality of first trend graphs to obtain an image feature corresponding to each of the first trend graphs; and obtaining the hidden vector of the hairstyle based on a plurality of image features corresponding to the plurality of first trend graphs.
Since each pixel in the first trend graph indicates the direction of the hairline segment corresponding to each pixel in the region corresponding to the hairstyle region of the first object in the corresponding first image, in the process of extracting the features of the first trend graph, the direction information of each hairline in the hairstyle region can be extracted based on the direction indicated by each pixel, and the hairstyle hidden vector can be obtained by encoding the direction information of each hairline.
In some embodiments, the hair style hidden vector includes a hair hidden vector corresponding to each hair of the hair style area. In some embodiments, the hair style hidden vector is a hidden vector obtained by encoding a plurality of hair hidden vectors corresponding to a plurality of hair in a hair style area, wherein in the process of encoding the plurality of hair hidden vectors, information that the plurality of hair are identical in a hidden vector space is extracted to generate the hair style hidden vector.
In some embodiments, as shown in fig. 3, in step S230, obtaining the first hair style hidden vector based on the plurality of first trend graphs includes:
step S310: splicing the plurality of first trend graphs to obtain spliced images; and
step S320: and extracting the characteristics of the spliced image to obtain the first hairstyle hidden vector.
The first trend graphs are spliced to obtain a spliced image, the characteristics of the spliced image are extracted to obtain the hidden vector of the hairstyle, and compared with the process of processing each trend graph respectively, the hidden vector of the hairstyle is obtained, so that the data processing amount is reduced.
In some embodiments, in step S310, the plurality of first trend graphs are spliced by stacking the plurality of first trend graphs in the channel direction.
In some embodiments, in step S320, feature extraction is performed on the stitched image by employing a feature extraction network.
In some embodiments, in step S320, the performing feature extraction on the stitched image to obtain the first hair style hidden vector comprises:
inputting the spliced image into a trained feature extraction network to obtain the first hairstyle hidden vector, wherein the trained feature extraction network is obtained by training a hairstyle generation model formed by the feature extraction network and a decoder and then adopting three-dimensional hairline data, in the process of training the hairstyle generation model, a plurality of training trend graphs corresponding to the plurality of visual angles are obtained based on the three-dimensional hairline data set, feature extraction is carried out on the spliced image of the plurality of training trend graphs by utilizing the feature extraction network to obtain a training hairstyle hidden vector, a prediction result is obtained by inputting the training hairstyle hidden vector into the decoder, and parameters of the hairstyle generation model are adjusted based on loss between the three-dimensional hairline data set and the prediction result.
The feature extraction network is obtained by training the hair style generation model formed by the decoder, and the three-dimensional hair data set is adopted to perform training, so that the feature extraction network can learn and extract spatial information corresponding to the three-dimensional hair data set in the training process, and when the feature extraction is performed based on a plurality of first trend graphs of a plurality of view angles, the information of a hair style region of a first object contained in the plurality of first trend graphs in the three-dimensional space can be extracted, the information of an issue region contained in the obtained hair style hidden vector in the three-dimensional space is consistent with the information of the hair style region represented by the three-dimensional hair data set in the three-dimensional space, and the accuracy of the obtained hair style hidden vector is improved.
In some embodiments, a plurality of training trend graphs are obtained by performing rendering operations corresponding to a plurality of perspectives using a three-dimensional hair dataset.
In some embodiments, the three-dimensional hair data set includes hair data from each of a plurality of hair strands on a preset hair model, the hair data indicating coordinates of each of a plurality of nodes on the hair strand. And, the decoder is based on the prediction result obtained by training the hair style hidden vector, a predicted hair line data set containing hair line data of each of the plurality of hair lines, wherein the hair line data of each hair line indicates coordinates of each of a plurality of nodes on the hair line. Parameters of the hairstyle generation model are adjusted by obtaining losses between the plurality of hair data for the plurality of hairs in the three-dimensional hair dataset and the plurality of hair data for the plurality of hairs in the prediction result.
In some embodiments, after steps S210-S230 are completed, the first hairstyle hidden vector is directly decoded in step S240 to generate a three-dimensional hairstyle corresponding to the first object. And decoding the first hair style hidden vector by adopting a decoder in the three-dimensional hair style generation model.
In some embodiments, as shown in fig. 4, in step S240, obtaining, based on the first hair style hidden vector, a three-dimensional hair style corresponding to the first object includes:
step S410: modifying the first hairstyle hidden vector to obtain a target hairstyle hidden vector; and
step S420: and decoding the target hairstyle hidden vector to obtain the three-dimensional hairstyle.
The target hair style hidden vector for obtaining the three-dimensional hair style is obtained by modifying the first hair style hidden vector, so that a designer can modify the hair style of the first object based on obtaining the first hair style hidden vector in a scene of designing the three-dimensional hair style by the designer, and the design efficiency is improved.
In some embodiments, the target hair style hidden vector is obtained by manually modifying the first hair style hidden vector.
In some embodiments, as shown in fig. 5, in step S410, modifying the first hair style hidden vector to obtain a target hair style hidden vector includes:
step S510: obtaining a plurality of second images of the plurality of perspectives with respect to a second object, the second object comprising a hairstyle area comprising a plurality of hair strands;
step S520: obtaining a plurality of second trend graphs corresponding to the plurality of second images based on the plurality of second images;
Step S530: obtaining a second hair style hidden vector of the hair style area of the second object based on the plurality of second trend graphs; and
step S540: and modifying the first hairstyle hidden vector based on the second hairstyle hidden vector to obtain a target hairstyle hidden vector.
By obtaining a second hair style hidden vector different from the first hair style hidden vector obtained from a plurality of second images of a plurality of perspectives of a second object and modifying the first hair style hidden vector based on the second hair style hidden vector to obtain a target hair style hidden vector, interpolation of the hidden vectors can be achieved, thereby obtaining more types of hair styles different from the hair styles of the first object and the hair styles of the second object.
In some embodiments, the target hair style hidden vector is obtained by obtaining weighting coefficients for each of the first and second hair style hidden vectors and fusing the first and second hair style hidden vectors based on the weighting coefficients.
According to another aspect of the present disclosure, a training method of a three-dimensional hairstyle generation model is also provided. The three-dimensional hairstyle generation model includes a feature extraction network and a decoder. As shown in fig. 6, the training method 600 of the three-dimensional hairstyle generation model includes:
Step S610: obtaining a three-dimensional hair data set comprising coordinates of each of a plurality of nodes on each of a plurality of hair strands from a hairstyle area of a first head mold;
step S620: based on the three-dimensional hairline dataset, obtaining a plurality of training trend graphs of the first head model corresponding to a plurality of visual angles, wherein each training trend graph in the plurality of training trend graphs corresponds to an image of the first head model at a corresponding visual angle in the plurality of visual angles and indicates the direction of a hairline segment corresponding to each pixel in a hairstyle area in the image;
step S630: acquiring training hair style hidden vectors corresponding to the three-dimensional hair data set based on the training trend graphs by utilizing the feature extraction network;
step S640: obtaining a prediction result corresponding to the three-dimensional hair data set based on the training hair style hidden vector by using the decoder, wherein the prediction result comprises coordinates of each of a plurality of nodes on each of the plurality of hair; and
step S650: and adjusting parameters of the three-dimensional hairstyle generation model based on the three-dimensional hairline data set and the prediction result.
The three-dimensional hair data set is adopted to train the three-dimensional generation model, in the training process, a plurality of two-dimensional training trend images are obtained through the three-dimensional hair data set, the loss between the prediction result and the three-dimensional hair data set is obtained by decoding the hair style hidden vector extracted based on the two-dimensional training trend images, and the three-dimensional hair style generation model is adjusted, so that the feature extraction network in the trained three-dimensional hair style generation model can extract the information of the three-dimensional hair style data in the three-dimensional space based on the input two-dimensional trend graph, the information is consistent with the information of the hair style area reflected by the three-dimensional hair data set in the space, the accuracy of the obtained hair style hidden vector is improved, and the accuracy of the obtained hair style hidden vector is high.
Meanwhile, according to the three-dimensional hairstyle generation model disclosed by the invention, the training trend graph obtained by the three-dimensional hairline data set is adopted for training in the training process, and the trend graph can be obtained based on the two-dimensional image for prediction in the prediction process, so that the three-dimensional hairstyle generation process can be realized by only adopting the two-dimensional image (replacing the three-dimensional hairline data) in the prediction process, the three-dimensional hairstyle generation process is simplified, and the end-to-end generation from the image to the three-dimensional hairstyle is realized.
In some embodiments, the plurality of viewing angles includes at least four viewing angles, front, rear, left and right, corresponding to the first head mold.
The plurality of visual angles comprise front, back, left and right visual angles corresponding to the first head model, so that the obtained training trend graphs of the first head model comprise information reflecting all hairstyle areas of the first head model, and the training hairstyle hidden vectors obtained based on the training trend graphs can reflect all the information of the first head model in a three-dimensional space, so that the obtained training hairstyle hidden vectors are accurate. Meanwhile, the three-dimensional hairstyle generating model is trained by utilizing training trend graphs of front, back, left and right visual angles corresponding to the first hairstyle, and in the process of predicting by utilizing the three-dimensional hairstyle generating model, the three-dimensional hairstyle can be generated only by obtaining trend graphs of the four visual angles for prediction, so that the data processing amount in the prediction process is reduced.
As shown in fig. 7, a comparative schematic diagram of a training and prediction process of a three-dimensional hairstyle generation model according to some embodiments of the present disclosure is shown. Wherein, the liquid crystal display device comprises a liquid crystal display device,
in the training process, a four-view trend graph 720 is obtained based on the three-dimensional hairline data set 710, the four-view trend graph 720 is input into the feature extraction network 701 to obtain training hair style hidden vectors, the decoder 702 is adopted to decode the training hair style hidden vectors to obtain a prediction result 730, and parameters of the feature extraction network 701 and the decoder 702 are adjusted based on loss between the prediction result 730 and the three-dimensional hairline data set 710.
In the prediction process, a four-view trend graph 750 corresponding to the four-view image is obtained based on the four-view image 740 about the hairstyle area of the person, a hairstyle hidden vector is obtained by inputting the four-view trend graph 750 to the feature extraction network 701, and the three-dimensional hairstyle 760 is obtained by decoding the hairstyle hidden vector using the decoder 702.
In the process, the input of the training process of the three-dimensional hairstyle generation model is a four-view trend graph obtained based on a three-dimensional hairline data set, the input of the prediction process of the three-dimensional hairstyle generation model is a four-view trend graph obtained based on a four-view two-dimensional image, the three-dimensional hairstyle generation process can be realized by only adopting the two-dimensional image (replacing three-dimensional hairline data) in the prediction process, the three-dimensional hairstyle generation process is simplified, and the end-to-end generation from the image to the three-dimensional hairstyle is realized.
In the four-view image 740 and the three-dimensional hairstyle 760, the image is mosaic-processed based on the description and privacy protection, and this processing is not required in the practical application process. Meanwhile, it should be noted that the face image in the embodiment according to the present disclosure is not a face image for a specific user, and cannot reflect personal information of a specific user, and the face image is from the public dataset.
In some embodiments, the feature extraction network may be based on a residual network (resnet).
In some embodiments, step S620, obtaining a plurality of training trend graphs of the first head model corresponding to a plurality of perspectives based on the three-dimensional hair dataset includes:
and performing rendering operation corresponding to each view angle of the plurality of view angles based on the three-dimensional hairline data set so as to obtain a trend graph corresponding to each view angle of the plurality of view angles.
And obtaining a trend graph by adopting a rendering method, so that the obtained trend graph is accurate.
In some embodiments, obtaining training hair style hidden vectors corresponding to the three-dimensional hair dataset based on the plurality of training trend graphs using the feature extraction network comprises:
splicing the training trend graphs to obtain spliced images; and
inputting the spliced image into the feature extraction network to obtain the training hair style hidden vector.
The training trend graphs are spliced to obtain spliced images, features of the spliced images are extracted to obtain hidden hairstyle vectors, and compared with the process of processing each trend graph respectively, the hidden hairstyle vectors are obtained, so that the data processing capacity is reduced.
In some embodiments, parameters of the three-dimensional hairstyle generation model are adjusted by calculating L1 losses between the prediction results and the three-dimensional hairline dataset.
According to another aspect of the present disclosure, there is also provided a three-dimensional hairstyle generating device. As shown in fig. 8, includes: a first image acquisition unit 810 configured to obtain a plurality of first images of a plurality of perspectives with respect to a first object, the first object including a hair styling area containing a plurality of hair; a first trend graph obtaining unit 820 configured to obtain, based on the plurality of first images, a plurality of first trend graphs corresponding to the plurality of first images, each of the plurality of first trend graphs indicating a direction of a hair line segment corresponding to each pixel in a region corresponding to a hair style region of the first object in a respective one of the plurality of images; a first hair style hidden vector obtaining unit 830 configured to obtain a first hair style hidden vector of a hair style area of the first object based on the plurality of first trend graphs; and a three-dimensional hair style acquisition unit 840 configured to acquire a three-dimensional hair style corresponding to the first object based on the first hair style hidden vector, the three-dimensional hair style including hair piece data for each of the plurality of hair pieces, the hair piece data including coordinates of each of a plurality of nodes on the hair piece.
In some embodiments, the plurality of perspectives includes at least four perspectives, front, back, left, and right, corresponding to the first object.
In some embodiments, the first hair style hidden vector acquisition unit includes: the image stitching unit is configured to stitch the plurality of first trend graphs to obtain stitched images; and a feature extraction unit configured to perform feature extraction on the stitched image to obtain the first hair style hidden vector.
In some embodiments, the feature extraction unit comprises: and an image input unit configured to input the stitched image to a trained feature extraction network to obtain the first hair style hidden vector, wherein the trained feature extraction network is obtained by training a hair style generation model composed of a feature extraction network and a decoder and then using a three-dimensional hair data set, in the process of training the hair style generation model, a plurality of training trend graphs corresponding to the plurality of view angles are obtained based on the three-dimensional hair data set, feature extraction is performed on the stitched image of the plurality of training trend graphs by using the feature extraction network to obtain a training hair style hidden vector, a prediction result is obtained by inputting the training hair style hidden vector to a decoder, and parameters of the hair style generation model are adjusted based on a loss between the three-dimensional hair data set and the prediction result.
In some embodiments, the three-dimensional hair style acquisition unit includes: a modification unit configured to modify the first hair style hidden vector to obtain a target hair style hidden vector; and a decoding unit configured to decode the target hair style hidden vector to obtain the three-dimensional hair style.
In some embodiments, the modification unit comprises: a second image acquisition unit configured to obtain a plurality of second images of the plurality of perspectives with respect to a second object, the second object including a hair styling area including a plurality of hair; a second trend graph acquisition unit configured to acquire a plurality of second trend graphs corresponding to the plurality of second images based on the plurality of second images; a second hair style hidden vector acquisition unit configured to acquire a second hair style hidden vector of a hair style area of the second object based on the plurality of second trend graphs; and a modification subunit configured to modify the first hair style hidden vector based on the second hair style hidden vector to obtain a target hair style hidden vector.
According to another aspect of the present disclosure, there is also provided that the three-dimensional hairstyle generation model includes a feature extraction network and a decoder, as shown in fig. 9, the apparatus 900 includes: a three-dimensional hair-line data set obtaining unit 910 configured to obtain a three-dimensional hair-line data set including coordinates of each of a plurality of nodes on each of a plurality of hair lines from a hairstyle area of a first head model; a training trend graph obtaining unit 920 configured to obtain, based on the three-dimensional hair dataset, a plurality of training trend graphs corresponding to a plurality of viewing angles for the first head model, where each training trend graph in the plurality of training trend graphs corresponds to an image of the first head model at a corresponding viewing angle in the plurality of viewing angles, and indicates a direction of a hair line segment corresponding to each pixel in a hair style area in the image; a training hair style hidden vector obtaining unit 930 configured to obtain training hair style hidden vectors corresponding to the three-dimensional hair data set based on the plurality of training trend graphs using the feature extraction network; a prediction result obtaining unit 940 configured to obtain, with the decoder, a prediction result corresponding to the three-dimensional hair line dataset based on the training hair style hidden vector, the prediction result including coordinates of each of a plurality of nodes on each of the plurality of hair lines; and a parameter adjustment unit 950 configured to adjust parameters of the three-dimensional hairstyle generation model based on the three-dimensional hairline dataset and the prediction result.
In some embodiments, the three-dimensional hair dataset acquisition unit comprises: and a rendering unit configured to perform a rendering operation corresponding to each of the plurality of perspectives based on the three-dimensional hair dataset to obtain a trend graph corresponding to each of the plurality of perspectives.
In some embodiments, the training hairstyle hidden vector acquisition unit includes: the image stitching unit is configured to stitch the training trend graphs to obtain stitched images; and a training hair style hidden vector acquisition subunit configured to input the stitched image to the feature extraction network to obtain the training hair style hidden vector.
In some embodiments, the plurality of viewing angles includes at least four viewing angles, front, rear, left and right, corresponding to the first head mold.
In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order colloquial is not violated.
According to embodiments of the present disclosure, there is also provided an electronic device, a readable storage medium and a computer program product.
Referring to fig. 10, a block diagram of a structure of an electronic device 1000 that may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 10, the electronic device 1000 includes a computing unit 1001 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic apparatus 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.
Various components in the electronic device 1000 are connected to the I/O interface 1005, including: an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. The input unit 1006 may be any type of device capable of inputting information to the electronic device 1000, the input unit 1006 may receive input numeric or character information and generate key signal inputs related to user settings and/or function control of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone, and/or a remote control. The output unit 1007 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video/audio output terminals, vibrators, and/or printers. Storage unit 1008 may include, but is not limited to, magnetic disks, optical disks. Communication unit 1009 allows electronic device 1000 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth (TM) devices, 802.11 devices, wiFi devices, wiMax devices, cellular communication devices, and/or the like.
The computing unit 1001 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1001 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as method 200 or method 600. For example, in some embodiments, the method 200 or the method 600 may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 1000 via the ROM 1002 and/or the communication unit 1009. When the computer program is loaded into RAM 1003 and executed by computing unit 1001, one or more steps of method 200 or method 600 described above may be performed. Alternatively, in other embodiments, computing unit 1001 may be configured to perform method 200 or method 600 in any other suitable manner (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the foregoing methods, systems, and apparatus are merely exemplary embodiments or examples, and that the scope of the present invention is not limited by these embodiments or examples but only by the claims following the grant and their equivalents. Various elements of the embodiments or examples may be omitted or replaced with equivalent elements thereof. Furthermore, the steps may be performed in a different order than described in the present disclosure. Further, various elements of the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after the disclosure.
Claims (23)
1. A method of three-dimensional hairstyle generation comprising:
obtaining a plurality of first images for a plurality of perspectives of a first object, the first object comprising a hairstyle area comprising a plurality of hair strands;
Obtaining a plurality of first trend graphs corresponding to the plurality of first images based on the plurality of first images, each of the plurality of first trend graphs indicating a direction of a hair line segment corresponding to each pixel in a region corresponding to a hair style region of the first object in a respective image of the plurality of images;
obtaining a first hair style hidden vector of a hair style area of the first object based on the plurality of first trend graphs; and
and obtaining a three-dimensional hairstyle corresponding to the first object based on the first hairstyle hidden vector, wherein the three-dimensional hairstyle comprises hairline data of each hairline in the plurality of hairlines, and the hairline data comprises coordinates of each node in the plurality of nodes on the hairline.
2. The method of claim 1, wherein the plurality of perspectives includes at least four perspectives of front, back, left, and right corresponding to the first object.
3. The method of claim 1, wherein the obtaining the first hair style hidden vector based on the plurality of first trend graphs comprises:
splicing the plurality of first trend graphs to obtain spliced images; and
and extracting the characteristics of the spliced image to obtain the first hairstyle hidden vector.
4. The method of claim 3, wherein the performing feature extraction on the stitched image to obtain the first hair style hidden vector comprises:
inputting the spliced image into a trained feature extraction network to obtain the first hairstyle hidden vector, wherein the trained feature extraction network is obtained by training a three-dimensional hairline data set after a hairstyle generating model is formed by the feature extraction network and a decoder, in the process of training the hairstyle generating model, a plurality of training trend graphs corresponding to the plurality of visual angles are obtained based on the three-dimensional hairline data set, feature extraction is performed on the spliced image of the plurality of training trend graphs by utilizing the feature extraction network to obtain a training hairstyle hidden vector, a prediction result is obtained by inputting the training hairstyle hidden vector into the decoder, and parameters of the hairstyle generating model are adjusted based on loss between the three-dimensional hairline data set and the prediction result.
5. The method according to any one of claims 1-4, wherein the obtaining the three-dimensional hairstyle corresponding to the first object based on the first hairstyle hidden vector comprises:
Modifying the first hairstyle hidden vector to obtain a target hairstyle hidden vector; and
and decoding the target hairstyle hidden vector to obtain the three-dimensional hairstyle.
6. The method of claim 5, wherein said modifying the first hair style hidden vector to obtain a target hair style hidden vector comprises:
obtaining a plurality of second images of the plurality of perspectives with respect to a second object, the second object comprising a hairstyle area comprising a plurality of hair strands;
obtaining a plurality of second trend graphs corresponding to the plurality of second images based on the plurality of second images;
obtaining a second hair style hidden vector of the hair style area of the second object based on the plurality of second trend graphs; and
and modifying the first hairstyle hidden vector based on the second hairstyle hidden vector to obtain a target hairstyle hidden vector.
7. A training method of a three-dimensional hairstyle generation model comprising a feature extraction network and a decoder, the method comprising:
obtaining a three-dimensional hair data set comprising coordinates of each of a plurality of nodes on each of a plurality of hair strands from a hairstyle area of a first head mold;
Obtaining a plurality of training trend graphs corresponding to a plurality of visual angles of the first head model based on the three-dimensional hairline data set, wherein each training trend graph in the plurality of training trend graphs corresponds to an image of the first head model at a corresponding visual angle in the plurality of visual angles and indicates the direction of a hairline segment corresponding to each pixel in a hairstyle area in the image;
acquiring training hair style hidden vectors corresponding to the three-dimensional hair data set based on the training trend graphs by utilizing the feature extraction network;
obtaining a prediction result corresponding to the three-dimensional hair data set based on the training hair style hidden vector by using the decoder, wherein the prediction result comprises coordinates of each of a plurality of nodes on each of the plurality of hair; and
and adjusting parameters of the three-dimensional hairstyle generation model based on the three-dimensional hairline data set and the prediction result.
8. The method of claim 7, wherein the obtaining a plurality of training trend graphs for the first head model corresponding to a plurality of perspectives based on the three-dimensional hair dataset comprises:
and performing rendering operation corresponding to each view angle of the plurality of view angles based on the three-dimensional hairline data set so as to obtain a trend graph corresponding to each view angle of the plurality of view angles.
9. The method of claim 7, wherein the obtaining, with the feature extraction network, a training hair style hidden vector corresponding to the three-dimensional hair dataset based on the plurality of training trend graphs comprises:
splicing the training trend graphs to obtain spliced images; and
inputting the spliced image into the feature extraction network to obtain the training hair style hidden vector.
10. The method of claim 7, wherein the plurality of viewing angles includes at least four viewing angles of front, back, left, and right for the first head mold.
11. A three-dimensional hair style generating device comprising:
a first image acquisition unit configured to obtain a plurality of first images of a plurality of perspectives with respect to a first object, the first object including a hair styling area containing a plurality of hair;
a first trend graph obtaining unit configured to obtain, based on the plurality of first images, a plurality of first trend graphs corresponding to the plurality of first images, each of the plurality of first trend graphs indicating a direction of a hair line segment corresponding to each pixel in a region corresponding to a hairstyle region of the first object in a corresponding one of the plurality of images;
A first hair style hidden vector acquisition unit configured to acquire a first hair style hidden vector of a hair style area of the first object based on the plurality of first trend graphs; and
a three-dimensional hair style acquisition unit configured to acquire a three-dimensional hair style corresponding to the first object based on the first hair style hidden vector, the three-dimensional hair style including hair line data for each of the plurality of hair lines, the hair line data including coordinates of each of a plurality of nodes on the hair line.
12. The apparatus of claim 11, wherein the plurality of perspectives comprises at least four perspectives of front, back, left, and right corresponding to the first object.
13. The apparatus of claim 11, wherein the first hair style implicit vector acquisition unit comprises:
the image stitching unit is configured to stitch the plurality of first trend graphs to obtain stitched images; and
and the feature extraction unit is configured to perform feature extraction on the spliced image so as to obtain the first hairstyle hidden vector.
14. The apparatus of claim 13, wherein the feature extraction unit comprises:
and an image input unit configured to input the stitched image to a trained feature extraction network to obtain the first hair style hidden vector, wherein the trained feature extraction network is obtained by training a three-dimensional hair data set after a feature extraction network and a decoder form a hair style generation model, in the process of training the hair style generation model, a plurality of training trend graphs corresponding to the plurality of view angles are obtained based on the three-dimensional hair data set, feature extraction is performed on the stitched image of the plurality of training trend graphs by using the feature extraction network to obtain a training hair style hidden vector, a prediction result is obtained by inputting the training hair style hidden vector to a decoder, and parameters of the hair style generation model are adjusted based on a loss between the three-dimensional hair data set and the prediction result.
15. The apparatus according to any one of claims 11-14, wherein the three-dimensional hairstyle acquisition unit comprises:
a modification unit configured to modify the first hair style hidden vector to obtain a target hair style hidden vector; and
and the decoding unit is configured to decode the target hairstyle hidden vector so as to obtain the three-dimensional hairstyle.
16. The apparatus of claim 15, wherein the modifying unit comprises:
a second image acquisition unit configured to obtain a plurality of second images of the plurality of perspectives with respect to a second object, the second object including a hair styling area including a plurality of hair;
a second trend graph acquisition unit configured to acquire a plurality of second trend graphs corresponding to the plurality of second images based on the plurality of second images;
a second hair style hidden vector acquisition unit configured to acquire a second hair style hidden vector of a hair style area of the second object based on the plurality of second trend graphs; and
and the modification subunit is configured to modify the first hairstyle hidden vector based on the second hairstyle hidden vector to obtain a target hairstyle hidden vector.
17. A training device for a three-dimensional hair style generation model, wherein the three-dimensional hair style generation model includes a feature extraction network and a decoder, the device comprising:
a three-dimensional hair-line data set acquisition unit configured to acquire a three-dimensional hair-line data set including coordinates of each of a plurality of nodes on each of a plurality of hair lines from a hairstyle area of a first head model;
a training trend graph obtaining unit, configured to obtain a plurality of training trend graphs corresponding to a plurality of view angles of the first head model based on the three-dimensional hair dataset, where each training trend graph in the plurality of training trend graphs corresponds to an image of the first head model at a corresponding view angle in the plurality of view angles, and indicates a direction of a hair line segment corresponding to each pixel in a hair style area in the image;
a training hair style hidden vector obtaining unit, configured to obtain training hair style hidden vectors corresponding to the three-dimensional hair data set based on the plurality of training trend graphs by using the feature extraction network;
a prediction result obtaining unit configured to obtain, based on the training hair style hidden vector, a prediction result corresponding to the three-dimensional hair data set, the prediction result including coordinates of each of a plurality of nodes on each of the plurality of hair strands, using the decoder; and
And a parameter adjustment unit configured to adjust parameters of the three-dimensional hairstyle generation model based on the three-dimensional hairline dataset and the prediction result.
18. The apparatus of claim 17, wherein the three-dimensional hair dataset acquisition unit comprises:
and a rendering unit configured to perform a rendering operation corresponding to each of the plurality of perspectives based on the three-dimensional hair dataset to obtain a trend graph corresponding to each of the plurality of perspectives.
19. The apparatus of claim 17, wherein the training hairstyle hidden vector acquisition unit comprises:
the image stitching unit is configured to stitch the training trend graphs to obtain stitched images; and
a training hair style hidden vector acquisition subunit configured to input the stitched image to the feature extraction network to obtain the training hair style hidden vector.
20. The apparatus of claim 17, wherein the plurality of viewing angles comprises at least four viewing angles, front, back, left, and right, corresponding to the first head mold.
21. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the method comprises the steps of
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.
22. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-10.
23. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211542715.0A CN116030185A (en) | 2022-12-02 | 2022-12-02 | Three-dimensional hairline generating method and model training method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211542715.0A CN116030185A (en) | 2022-12-02 | 2022-12-02 | Three-dimensional hairline generating method and model training method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116030185A true CN116030185A (en) | 2023-04-28 |
Family
ID=86071359
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211542715.0A Pending CN116030185A (en) | 2022-12-02 | 2022-12-02 | Three-dimensional hairline generating method and model training method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116030185A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116580212A (en) * | 2023-05-16 | 2023-08-11 | 北京百度网讯科技有限公司 | Image generation method, training method, device and equipment of image generation model |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105405163A (en) * | 2015-12-28 | 2016-03-16 | 北京航空航天大学 | Vivid static-state hair modeling method based on multiple direction fields |
| CN114140603A (en) * | 2021-12-08 | 2022-03-04 | 北京百度网讯科技有限公司 | Training method of virtual image generation model and virtual image generation method |
| CN114187633A (en) * | 2021-12-07 | 2022-03-15 | 北京百度网讯科技有限公司 | Image processing method and device, and training method and device of image generation model |
| CN115311403A (en) * | 2022-08-26 | 2022-11-08 | 北京百度网讯科技有限公司 | Deep learning network training method, virtual image generation method and device |
| CN115409922A (en) * | 2022-08-30 | 2022-11-29 | 北京百度网讯科技有限公司 | Three-dimensional hairstyle generation method and device, electronic equipment and storage medium |
-
2022
- 2022-12-02 CN CN202211542715.0A patent/CN116030185A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105405163A (en) * | 2015-12-28 | 2016-03-16 | 北京航空航天大学 | Vivid static-state hair modeling method based on multiple direction fields |
| CN114187633A (en) * | 2021-12-07 | 2022-03-15 | 北京百度网讯科技有限公司 | Image processing method and device, and training method and device of image generation model |
| CN114140603A (en) * | 2021-12-08 | 2022-03-04 | 北京百度网讯科技有限公司 | Training method of virtual image generation model and virtual image generation method |
| CN115311403A (en) * | 2022-08-26 | 2022-11-08 | 北京百度网讯科技有限公司 | Deep learning network training method, virtual image generation method and device |
| CN115409922A (en) * | 2022-08-30 | 2022-11-29 | 北京百度网讯科技有限公司 | Three-dimensional hairstyle generation method and device, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116580212A (en) * | 2023-05-16 | 2023-08-11 | 北京百度网讯科技有限公司 | Image generation method, training method, device and equipment of image generation model |
| CN116580212B (en) * | 2023-05-16 | 2024-02-06 | 北京百度网讯科技有限公司 | Image generation method, training method, device and equipment of image generation model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113313650B (en) | Image quality enhancement method, device, equipment and medium | |
| CN115409922B (en) | Three-dimensional hairstyle generation method, device, electronic equipment and storage medium | |
| CN116051729B (en) | Three-dimensional content generation method and device and electronic equipment | |
| CN114972958B (en) | Key point detection method, neural network training method, device and equipment | |
| CN115482325B (en) | Picture rendering method, device, system, equipment and medium | |
| CN115239888B (en) | Method, device, electronic equipment and medium for reconstructing three-dimensional face image | |
| CN117274491A (en) | Training method, device, equipment and medium for three-dimensional reconstruction model | |
| CN115170819A (en) | Target identification method and device, electronic equipment and medium | |
| CN116030185A (en) | Three-dimensional hairline generating method and model training method | |
| CN115661375B (en) | Three-dimensional hair style generation method and device, electronic equipment and storage medium | |
| CN116245998B (en) | Rendering map generation method and device, and model training method and device | |
| CN116205819B (en) | Character image generation method, training method and device of deep learning model | |
| CN114119935B (en) | Image processing method and device | |
| CN114120448B (en) | Image processing method and device | |
| CN115601555A (en) | Image processing method and apparatus, device and medium | |
| CN114049472A (en) | Three-dimensional model adjustment method, device, electronic apparatus, and medium | |
| CN114119154A (en) | Virtual makeup method and device | |
| CN116385641B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN115423827B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN114120412B (en) | Image processing method and device | |
| CN115511779B (en) | Image detection method, device, electronic equipment and storage medium | |
| CN115619981B (en) | Three-dimensional hairstyle generation method and model training method | |
| CN117218297A (en) | Human body reconstruction parameter generation method, device, equipment and medium | |
| CN117218499A (en) | Training method of facial expression capturing model, facial expression driving method and device | |
| CN117274575A (en) | Training method of target detection model, target detection method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |