CN117218499A - Training method of facial expression capturing model, facial expression driving method and device - Google Patents
Training method of facial expression capturing model, facial expression driving method and device Download PDFInfo
- Publication number
- CN117218499A CN117218499A CN202311286394.7A CN202311286394A CN117218499A CN 117218499 A CN117218499 A CN 117218499A CN 202311286394 A CN202311286394 A CN 202311286394A CN 117218499 A CN117218499 A CN 117218499A
- Authority
- CN
- China
- Prior art keywords
- face
- sample
- face image
- image
- loss
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008921 facial expression Effects 0.000 title claims abstract description 91
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000012549 training Methods 0.000 title claims abstract description 48
- 230000008451 emotion Effects 0.000 claims abstract description 106
- 238000013136 deep learning model Methods 0.000 claims abstract description 49
- 238000013145 classification model Methods 0.000 claims abstract description 20
- 238000009877 rendering Methods 0.000 claims abstract description 13
- 238000005070 sampling Methods 0.000 claims abstract description 3
- 238000004590 computer program Methods 0.000 claims description 16
- 238000000605 extraction Methods 0.000 claims description 13
- 230000001815 facial effect Effects 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 4
- 238000013507 mapping Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000036651 mood Effects 0.000 claims 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 238000013135 deep learning Methods 0.000 abstract description 3
- 230000003190 augmentative effect Effects 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 14
- 238000004891 communication Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 6
- 239000013598 vector Substances 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000013459 approach Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 239000013604 expression vector Substances 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000010267 cellular communication Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003924 mental process Effects 0.000 description 1
- 230000003278 mimic effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The disclosure provides a training method of a facial expression capturing model, a facial expression driving method and a device, relates to the technical field of artificial intelligence, and particularly relates to the technical fields of computer vision, augmented reality, virtual reality, deep learning and the like, and can be applied to scenes such as metauniverse, digital people and the like. The method comprises the following steps: obtaining a sampling face image; predicting face parameters by using a deep learning model to obtain sample face parameters; generating a sample three-dimensional face model by using the three-dimensional face reconstruction model, and rendering to obtain a predicted face image; respectively carrying out emotion classification on the sample face image and the predicted face image by utilizing a pre-trained emotion classification model to obtain a first emotion type corresponding to the sample face image and a second emotion type corresponding to the predicted face image; calculating an emotion category loss based on the first emotion category and the second emotion category; and adjusting parameters of the deep learning model based at least on the emotion category loss to obtain a facial expression capture model.
Description
Technical Field
The disclosure relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, augmented reality, virtual reality, deep learning and the like, and can be applied to scenes such as metauniverse, digital people and the like, in particular to a training method of a facial expression capturing model, a facial expression driving method, a training device of the facial expression capturing model, a facial expression driving device, electronic equipment, a computer readable storage medium and a computer program product.
Background
Artificial intelligence is the discipline of studying the process of making a computer mimic certain mental processes and intelligent behaviors (e.g., learning, reasoning, thinking, planning, etc.) of a person, both hardware-level and software-level techniques. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge graph technology and the like.
The facial expression capturing technology is one of important technologies in three-dimensional digital people, and the task of the facial expression capturing technology is to acquire facial expressions through a three-dimensional facial reconstruction technology, and transfer the expression of a driving person to the digital people, so that the expression controllability of a virtual person is realized.
The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, the problems mentioned in this section should not be considered as having been recognized in any prior art unless otherwise indicated.
Disclosure of Invention
The present disclosure provides a training method of a facial expression capturing model, a facial expression driving method, a training apparatus of a facial expression capturing model, a facial expression driving apparatus, an electronic device, a computer-readable storage medium, and a computer program product.
According to an aspect of the present disclosure, there is provided a training method of a facial expression capture model, including: obtaining a sampling face image; based on the sample face image, carrying out face parameter prediction by using a deep learning model to obtain sample face parameters; generating a sample three-dimensional face model by utilizing the three-dimensional face reconstruction model based on the sample face parameters, and rendering to obtain a predicted face image; respectively carrying out emotion classification on the sample face image and the predicted face image by utilizing a pre-trained emotion classification model to obtain a first emotion type corresponding to the sample face image and a second emotion type corresponding to the predicted face image; calculating an emotion category loss based on the first emotion category and the second emotion category; and adjusting parameters of the deep learning model based at least on the emotion category loss to obtain a facial expression capture model.
According to an aspect of the present disclosure, there is provided a facial expression driving method including: acquiring a target image; based on the target image, carrying out facial parameter prediction by utilizing a facial expression capturing model to obtain target facial parameters, wherein the facial expression capturing model is obtained by training by utilizing the training method; and driving the facial expression of the target three-dimensional digital person based on the target face parameters.
According to an aspect of the present disclosure, there is provided a training apparatus of a facial expression capture model, including: a first face image acquisition unit configured to acquire a sample face image; the first face parameter prediction unit is configured to predict face parameters by using a deep learning model based on the sample face image to obtain sample face parameters; the reconstruction rendering unit is configured to generate a sample three-dimensional face model by utilizing the three-dimensional face reconstruction model based on the sample face parameters and render to obtain a predicted face image; the emotion classification unit is configured to respectively carry out emotion classification on the sample face image and the predicted face image by utilizing a pre-trained emotion classification model to obtain a first emotion category corresponding to the sample face image and a second emotion category corresponding to the predicted face image; an emotion category loss calculation unit configured to calculate an emotion category loss based on the first emotion category and the second emotion category; and a parameter tuning unit configured to adjust parameters of the deep learning model based at least on the emotion category loss to obtain a facial expression capture model.
According to an aspect of the present disclosure, there is provided a facial expression driving apparatus including: a second face image acquisition unit configured to acquire a target image; the second face parameter prediction unit is configured to predict face parameters based on the target image by using a facial expression capturing model to obtain target face parameters, wherein the facial expression capturing model is obtained by training by using the training device; and a driving unit configured to perform facial expression driving on the target three-dimensional digital person based on the target face parameter.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described above.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the above-described method.
According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the above-described method.
According to one or more embodiments of the present disclosure, by generating a sample face parameter corresponding to a sample face image using a deep learning model to be trained, and generating a sample three-dimensional face model and a corresponding rendering image based on the sample face parameter, further performing emotion classification on the sample face image and the rendering image using a pre-trained emotion classification model, and calculating a loss value using the obtained emotion classification and adjusting parameters of the deep learning model for generating the face parameter, an effect of using emotion classification information as a supervision and improvement facial expression capture model is achieved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The accompanying drawings illustrate exemplary embodiments and, together with the description, serve to explain exemplary implementations of the embodiments. The illustrated embodiments are for exemplary purposes only and do not limit the scope of the claims. Throughout the drawings, identical reference numerals designate similar, but not necessarily identical, elements.
FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, in accordance with an embodiment of the present disclosure;
FIG. 2 illustrates a flowchart of a training method of a facial expression capture model according to an exemplary embodiment of the present disclosure;
FIG. 3 illustrates a flowchart of face parameter prediction for a sample face image using a deep learning model according to an exemplary embodiment of the present disclosure;
FIG. 4 illustrates a flowchart of a training method of a facial expression capture model according to an exemplary embodiment of the present disclosure;
FIG. 5 shows a schematic diagram of a deep learning model and penalty according to an exemplary embodiment of the present disclosure;
fig. 6 illustrates a flowchart of a facial expression driving method according to an exemplary embodiment of the present disclosure;
FIG. 7 illustrates a block diagram of a training device of a facial expression capture model according to an exemplary embodiment of the present disclosure;
fig. 8 illustrates a block diagram of a training device of a facial expression driving apparatus according to an exemplary embodiment of the present disclosure; and
fig. 9 illustrates a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the present disclosure, the use of the terms "first," "second," and the like to describe various elements is not intended to limit the positional relationship, timing relationship, or importance relationship of the elements, unless otherwise indicated, and such terms are merely used to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, they may also refer to different instances based on the description of the context.
The terminology used in the description of the various illustrated examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, the elements may be one or more if the number of the elements is not specifically limited. Furthermore, the term "and/or" as used in this disclosure encompasses any and all possible combinations of the listed items.
In the related art, when training a facial expression capturing model, a model is usually trained by calculating a corresponding loss value using a face parameter prediction result and a face parameter real label output by the model, but such training effect is poor.
In order to solve the problems, the method and the device for generating the facial expression capturing model by using the deep learning model to be trained generate sample face parameters corresponding to sample face images, generate a sample three-dimensional face model and corresponding rendering images based on the sample face parameters, further conduct emotion classification on the sample face images and the rendering images by using the pre-trained emotion classification model, calculate loss values by using the obtained emotion classification and adjust parameters of the deep learning model for generating the face parameters, and achieve the effect of using emotion classification information as supervision and improvement of the facial expression capturing model.
Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented, in accordance with an embodiment of the present disclosure. Referring to fig. 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. Client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.
In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of the methods of the present disclosure.
In some embodiments, server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) network.
In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof that are executable by one or more processors. A user operating client devices 101, 102, 103, 104, 105, and/or 106 may in turn utilize one or more client applications to interact with server 120 to utilize the services provided by these components. It should be appreciated that a variety of different system configurations are possible, which may differ from system 100. Accordingly, FIG. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.
The user may use client devices 101, 102, 103, 104, 105, and/or 106 for human-machine interaction. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that the present disclosure may support any number of client devices.
Client devices 101, 102, 103, 104, 105, and/or 106 may include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptop computers), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and the like. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, linux, or Linux-like operating systems (e.g., GOOGLE Chrome OS); or include various mobile operating systems such as MICROSOFT Windows Mobile OS, iOS, windows Phone, android. Portable handheld devices may include cellular telephones, smart phones, tablet computers, personal Digital Assistants (PDAs), and the like. Wearable devices may include head mounted displays (such as smart glasses) and other devices. The gaming system may include various handheld gaming devices, internet-enabled gaming devices, and the like. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications), short Message Service (SMS) applications, and may use a variety of communication protocols.
Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a number of available protocols, including but not limited to TCP/IP, SNA, IPX, etc. For example only, the one or more networks 110 may be a Local Area Network (LAN), an ethernet-based network, a token ring, a Wide Area Network (WAN), the internet, a virtual network, a Virtual Private Network (VPN), an intranet, an extranet, a Public Switched Telephone Network (PSTN), an infrared network, a wireless network (e.g., bluetooth, WIFI), and/or any combination of these and/or other networks.
The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture that involves virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain virtual storage devices of the server). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.
The computing units in server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. Server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.
In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and/or event updates received from users of client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.
In some implementations, the server 120 may be a server of a distributed system or a server that incorporates a blockchain. The server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical host and virtual private server (VPS, virtual Private Server) service.
The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of databases 130 may be used to store information such as audio files and video files. Database 130 may reside in various locations. For example, the data store used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. Database 130 may be of different types. In some embodiments, the database used by server 120 may be a database, such as a relational database. One or more of these databases may store, update, and retrieve the databases and data from the databases in response to the commands.
In some embodiments, one or more of databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key value stores, object stores, or conventional stores supported by the file system.
The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.
According to an aspect of the present disclosure, a training method of a facial expression capture model is provided. As shown in fig. 2, the training method of the facial expression capturing model includes: step S201, acquiring a sample face image; step S202, based on a sample face image, carrying out face parameter prediction by using a deep learning model to obtain sample face parameters; step S203, generating a sample three-dimensional face model by using the three-dimensional face reconstruction model based on the sample face parameters, and rendering to obtain a predicted face image; step S204, respectively carrying out emotion classification on the sample face image and the predicted face image by utilizing a pre-trained emotion classification model to obtain a first emotion type corresponding to the sample face image and a second emotion type corresponding to the predicted face image; step S205, calculating emotion type loss based on the first emotion type and the second emotion type; and step S206, adjusting parameters of the deep learning model at least based on the emotion category loss to obtain a facial expression capturing model.
The method comprises the steps of generating sample face parameters corresponding to sample face images by using a deep learning model to be trained, generating a sample three-dimensional face model and corresponding rendering images based on the sample face parameters, carrying out emotion classification on the sample face images and the rendering images by using a pre-trained emotion classification model, calculating a loss value by using the obtained emotion classification model, and adjusting parameters of the deep learning model for generating the face parameters, so that the effect of using emotion classification information as a supervision and improvement facial expression capturing model is achieved.
In some embodiments, in step S201, the sample face image may be an image including a face acquired by various means.
In some embodiments, in step S202, the deep learning model may be used to generate sample face parameters corresponding to the input sample face image, and specifically may include a 150-dimensional face shape vector, a 52-dimensional facial expression vector, a 12-dimensional face pose vector, and a 200-dimensional texture vector.
According to some embodiments, the deep learning model may include an image feature extraction backbone network and a face parameter prediction network. As shown in fig. 3, in step S202 in the training method of the facial expression capturing model described above, performing face parameter prediction on a sample face image by using a deep learning model, obtaining sample face parameters may include: step S301, based on the sample face image, extracting image features by utilizing an image feature extraction backbone network to obtain sample face image features; and step S302, based on the characteristics of the sample face image, carrying out face parameter prediction by utilizing a face parameter prediction network to obtain the sample face parameters.
Therefore, by generating the face image characteristics and then predicting the face parameters based on the face image characteristics, an accurate face parameter predicting result can be obtained.
In some embodiments, the image feature extraction backbone network may be, for example, resNet18, but may be other networks including convolutional layers and pooling layers. The face parameter prediction network may be, for example, a network for regression and/or classification.
In some embodiments, in step S203, the three-dimensional face reconstruction model may be, for example, a 3D deformable model (3D Morphable Model,3DMM). In one exemplary embodiment, the FaceVerse model may be used as a three-dimensional face reconstruction model. By inputting sample face parameters (e.g., face shape vectors, face expression vectors, pose vectors, and texture vectors output by the deep learning model) into the faceverse model, a predicted 3D face model, i.e., a sample three-dimensional face model, can be obtained. Further, the sample three-dimensional face model may be rendered (e.g., differentially rendered), resulting in a corresponding predicted face image.
In some embodiments, in step S204, the emotion classification model obtained by training in advance may be used to perform emotion classification on the sample face image and the predicted face image, so as to obtain a first emotion category corresponding to the sample face image and a second emotion category corresponding to the predicted face image.
According to some embodiments, the pre-trained emotion classification model may be pre-trained using video data having faces and corresponding emotion classification tags. The corresponding emotion type label can be obtained by, for example, labeling segments of continuous video frames including faces in video data. In some embodiments, a video frame including a face and an emotion classification label corresponding to the video frame may be used as sample data to train an emotion classification model.
Therefore, by using the mode, a large number of training samples of the emotion classification model can be obtained at low cost, and the emotion classification model which outputs accurate results can be obtained while the training cost is reduced.
In some embodiments, at step S205, a classification penalty between the first emotion category and the second emotion category, i.e., an emotion category penalty, may be calculated. Further, parameters of the deep learning model may be adjusted based on the loss to obtain a trained facial expression capture model at step S206.
Fig. 4 illustrates another embodiment of a training method for a facial expression capture model, according to some embodiments. It is understood that the operations of step S401 to step S403, step S408 to step S410 in fig. 4 are similar to those of step S201 to step S206 in fig. 2, and are not described herein. As shown in fig. 4, the training method may further include: step S404, obtaining face key points of the sample face image and face key points of the predicted face image, and calculating face key point loss. Parameters of the deep learning model may be adjusted based on face keypoint loss and emotion category loss.
Therefore, the human face key point loss between the sample human face image and the predicted human face image based on the three-dimensional human face reconstruction to obtain the sample three-dimensional human face model is used as supervision of the deep learning model for capturing the facial expression, so that the prediction capability of the trained facial expression capturing model is improved.
According to some embodiments, step S201 in the training method of the facial expression capturing model described above, acquiring a sample face image may include: performing key point detection on the original face image to obtain face key points of the original face image; and cutting, size transformation and normalization are carried out on the original face image based on the face key points of the original face image so as to obtain a sample face image and the face key points of the sample face image. In some embodiments, the face key points in the sample face image may be points obtained after operations such as clipping and size transformation are performed on the original face image.
Therefore, through the mode, the part corresponding to the face can be cut out from the originally acquired image, and the image is standardized through means such as size change and normalization, so that the comparability between different images is realized, the model can be trained and predicted more stably, and the generalization capability, the prediction accuracy and the training speed of the model are improved.
According to some embodiments, the training method may further comprise: extracting three-dimensional key points from a sample three-dimensional face model; and mapping the three-dimensional key points to obtain the face key points of the predicted face image. Because the sample three-dimensional face model is already obtained, the face key point information can be accurately obtained through the mode.
According to some embodiments, as shown in fig. 4, the training method may further include: step S405, calculating an image pixel loss based on pixel values of a plurality of pixels in the sample face image and pixel values of a plurality of pixels in the predicted face image. Parameters of the deep learning model may be adjusted based on image pixel loss, face keypoint loss, and emotion category loss.
Therefore, the prediction capability of the trained facial expression capturing model is improved by using the image pixel loss between the sample face image and the predicted face image based on the three-dimensional face reconstruction to obtain the sample three-dimensional face model as the supervision of the deep learning model for facial expression capturing.
In some embodiments, the image pixel loss may employ an L1 loss.
According to some embodiments, the sample face parameters output by the deep learning model may include sample microexpressive coefficients. As shown in fig. 4, the training method may further include: step S406, acquiring a micro-expression label of a sample face image; and step S407, calculating the micro-expression loss based on the sample micro-expression coefficient and the micro-expression label. Parameters of the deep learning model may be adjusted based on microexpressive loss, image pixel loss, face keypoint loss, and emotion category loss.
Therefore, the prediction capability of the trained facial expression capturing model is improved by using the micro-expression loss between the sample face image and the predicted face image based on the three-dimensional face reconstruction to obtain the sample three-dimensional face model as the supervision of the deep learning model for facial expression capturing.
In some embodiments, the microexpressive loss may employ a classification loss. The micro-emoticons may be facial parameters for determining whether there is a mouth motion or change. In one exemplary embodiment, the microexpressive coefficient may be a lingual coefficient indicating whether a human face linguals. Accordingly, the micro-expression tag may be a real value (ground trunk) indicating whether the face spits out the tongue.
It should be understood that the combination of step S406 and step S407, and the combination of step S404 and step S405 in fig. 4 may be alternatively performed, or may be partially or completely performed, which is not limited herein.
In some embodiments, fig. 5 shows a schematic diagram of a deep learning model and penalty according to an exemplary embodiment of the present disclosure. As shown in fig. 5, the deep learning model 502 may receive a sample face image 504 and output sample face parameters 506, and further may input the sample face parameters 506 into a three-dimensional face reconstruction model 508, and render the obtained sample three-dimensional face model, and finally generate a predicted face image 510. The sample face image 504 and the predicted face image 510 may then be subjected to emotion classification using the pre-trained emotion classification model 512, respectively, and emotion class loss 514 may be calculated based on the classification results, and image pixel loss 516 may be calculated using the sample face image 504 and the predicted face image 510. In addition, the micro-expression loss 520 may also be calculated using the micro-expression parameters in the sample face parameters 506 and the micro-expression tags 518 of the pre-acquired sample face image.
According to another aspect of the present disclosure, a facial expression driving method is provided. As shown in fig. 6, the method includes: step S601, obtaining a target image; step S602, based on a target image, predicting facial parameters by using a facial expression capturing model to obtain target facial parameters, wherein the facial expression capturing model is obtained by training by using the training method; and step S603, performing facial expression driving on the target three-dimensional digital person based on the target face parameters.
Thus, by using the training method described above, a facial expression capturing model capable of outputting accurate face parameters is obtained. By using the facial expression capturing model to predict the facial parameters, accurate target facial parameters can be obtained, and further, accurate three-dimensional digital facial expression driving results can be obtained by using the target facial parameters.
According to another aspect of the present disclosure, a training device for a facial expression capture model is provided. As shown in fig. 7, the apparatus 700 includes: a first face image acquisition unit 710 configured to acquire a sample face image; the first face parameter prediction unit 720 is configured to predict face parameters by using the deep learning model based on the sample face image, so as to obtain sample face parameters; a reconstruction rendering unit 730 configured to generate a sample three-dimensional face model using the three-dimensional face reconstruction model based on the sample face parameters, and render a predicted face image; the emotion classification unit 740 is configured to respectively perform emotion classification on the sample face image and the predicted face image by using a pre-trained emotion classification model, so as to obtain a first emotion category corresponding to the sample face image and a second emotion category corresponding to the predicted face image; an emotion category loss calculation unit 750 configured to calculate an emotion category loss based on the first emotion category and the second emotion category; and a parameter tuning unit 760 configured to adjust parameters of the deep learning model based at least on the emotion classification loss to obtain a facial expression capture model.
It will be appreciated that the operations and effects of the units 710-760 in the apparatus 700 may refer to the descriptions of the steps S201-S206 in fig. 2, and are not described herein.
According to some embodiments, the deep learning model may include an image feature extraction backbone network and a face parameter prediction network. The first face parameter prediction unit may include: the image feature extraction subunit is configured to extract image features by utilizing an image feature extraction backbone network based on the sample face image to obtain sample face image features; and a face parameter prediction subunit configured to predict the face parameter by using the face parameter prediction network based on the sample face image feature to obtain the sample face parameter.
According to some embodiments, the pre-trained emotion classification model may be pre-trained using video data having faces and corresponding emotion classification tags.
According to some embodiments, the training device of the facial expression capture model described above may further include: and the face key point loss calculation unit is configured to acquire face key points of the sample face image and face key points of the predicted face image and calculate face key point loss. Parameters of the deep learning model may be adjusted based on face keypoint loss and emotion category loss.
According to some embodiments, the first face image acquisition unit in the training apparatus of the facial expression capture model described above may include: the key point detection unit is configured to detect key points of the original face image so as to obtain face key points of the original face image; and the image processing unit is configured to cut, transform and normalize the original face image based on the face key points of the original face image so as to obtain a sample face image and the face key points of the sample face image.
According to some embodiments, the training device of the facial expression capture model described above may further include: a three-dimensional key point extraction unit configured to extract three-dimensional key points from a sample three-dimensional face model; and the mapping unit is configured to map the three-dimensional key points to obtain the face key points of the predicted face image.
According to some embodiments, the training device of the facial expression capture model described above may further include: an image pixel loss calculation unit configured to calculate an image pixel loss based on pixel values of a plurality of pixels in the sample face image and pixel values of a plurality of pixels in the predicted face image. Parameters of the deep learning model may be adjusted based on image pixel loss, face keypoint loss, and emotion category loss, among others.
According to some embodiments, the sample face parameters may include sample microexpressive coefficients. The training device may further include: the micro-expression label acquisition unit is configured to acquire micro-expression labels of the sample face images; and a microexpressive loss calculation unit configured to calculate microexpressive loss based on the sample microexpressive coefficient and the microexpressive label. Parameters of the deep learning model may be adjusted based on microexpressive loss, image pixel loss, face keypoint loss, and emotion category loss.
According to another aspect of the present disclosure, a facial expression driving apparatus is provided. As shown in fig. 8, the apparatus 800 includes: a second face image acquisition unit 810 configured to acquire a target image; a second face parameter prediction unit 820 configured to perform face parameter prediction using a facial expression capture model based on the target image, the facial expression capture model being trained using the apparatus 700, to obtain target face parameters; and a driving unit 830 configured to perform facial expression driving on the target three-dimensional digital person based on the target face parameter.
It is to be understood that the operations and effects of the units 810-830 in the apparatus 800 may refer to the descriptions of the steps S601-S603 in fig. 6, which are not described herein.
In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order colloquial is not violated.
According to embodiments of the present disclosure, there is also provided an electronic device, a readable storage medium and a computer program product.
Referring to fig. 9, a block diagram of an electronic device 900 that may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile apparatuses, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing apparatuses. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 9, the apparatus 900 includes a computing unit 901 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.
Various components in device 900 are connected to I/O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 may be any type of device capable of inputting information to the device 900, the input unit 906 may receive input numeric or character information and generate key signal inputs related to user settings and/or function control of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone, and/or a remote control. The output unit 907 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video/audio output terminals, vibrators, and/or printers. Storage unit 908 may include, but is not limited to, magnetic disks, optical disks. Communication unit 909 allows device 900 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and/or chipsets, such as bluetooth TM Devices, 802.11 devices, wiFi devices, wiMax devices, cellular communication devices, and/or the like.
The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning network algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the respective methods and processes described above, for example, a training method of a facial expression capturing model and/or facial expression driving. For example, in some embodiments, the training method of the facial expression capture model and/or the facial expression driver may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 900 via the ROM 902 and/or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the training method and/or facial expression driving of the facial expression capture model described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform a training method of the facial expression capture model and/or facial expression driving by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service ("Virtual Private Server" or simply "VPS") are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the foregoing methods, systems, and apparatus are merely exemplary embodiments or examples, and that the scope of the present invention is not limited by these embodiments or examples but only by the claims following the grant and their equivalents. Various elements of the embodiments or examples may be omitted or replaced with equivalent elements thereof. Furthermore, the steps may be performed in a different order than described in the present disclosure. Further, various elements of the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after the disclosure.
Claims (21)
1. A training method of a facial expression capture model, comprising:
Obtaining a sampling face image;
based on the sample face image, carrying out face parameter prediction by using a deep learning model to obtain sample face parameters;
generating a sample three-dimensional face model by utilizing a three-dimensional face reconstruction model based on the sample face parameters, and rendering to obtain a predicted face image;
respectively carrying out emotion classification on the sample face image and the predicted face image by utilizing a pre-trained emotion classification model to obtain a first emotion category corresponding to the sample face image and a second emotion category corresponding to the predicted face image;
calculating a mood category loss based on the first mood category and the second mood category; and
based at least on the emotion category loss, parameters of the deep learning model are adjusted to obtain a facial expression capture model.
2. The method of claim 1, further comprising:
obtaining the face key points of the sample face image and the face key points of the predicted face image, calculating the face key point loss,
wherein parameters of the deep learning model are adjusted based on the face keypoint loss and the emotion category loss.
3. The method of claim 2, wherein acquiring a sample face image comprises:
performing key point detection on an original face image to obtain face key points of the original face image; and
based on the face key points of the original face image, cutting, size transformation and normalization are carried out on the original face image so as to obtain the sample face image and the face key points of the sample face image.
4. The method of claim 2, further comprising:
extracting three-dimensional key points from the sample three-dimensional face model; and
and mapping the three-dimensional key points to obtain the face key points of the predicted face image.
5. The method of claim 2, further comprising:
calculating an image pixel loss based on pixel values of a plurality of pixels in the sample face image and pixel values of a plurality of pixels in the predicted face image,
wherein parameters of the deep learning model are adjusted based on the image pixel loss, face keypoint loss, and emotion category loss.
6. The method of claim 5, wherein the sample face parameters comprise sample microexpressive coefficients, the method further comprising:
Acquiring a micro-expression label of the sample face image; and
calculating a microexpressive loss based on the sample microexpressive coefficient and the microexpressive label,
wherein parameters of the deep learning model are adjusted based on the microexpressive loss, the image pixel loss, the face keypoint loss, and the emotion category loss.
7. The method of any of claims 1-6, wherein the pre-trained emotion classification model is pre-trained using video data having faces and corresponding emotion classification tags.
8. The method of any of claims 1-6, wherein the deep learning model includes an image feature extraction backbone network and a face parameter prediction network, and performing face parameter prediction on the sample face image using the deep learning model to obtain sample face parameters includes:
based on the sample face image, extracting image features by utilizing the image feature extraction backbone network to obtain sample face image features; and
and based on the characteristics of the sample face image, predicting the face parameters by using the face parameter predicting network to obtain the sample face parameters.
9. A facial expression driving method, comprising:
acquiring a target image;
predicting facial parameters based on the target image using a facial expression capture model to obtain target facial parameters, the facial expression capture model being trained using the method of any one of claims 1-8; and
and carrying out facial expression driving on the target three-dimensional digital person based on the target face parameters.
10. A training device of a facial expression capture model, comprising:
a first face image acquisition unit configured to acquire a sample face image;
the first face parameter prediction unit is configured to predict face parameters by using a deep learning model based on the sample face image to obtain sample face parameters;
the reconstruction rendering unit is configured to generate a sample three-dimensional face model by utilizing the three-dimensional face reconstruction model based on the sample face parameters and render a predicted face image;
the emotion classification unit is configured to respectively carry out emotion classification on the sample face image and the predicted face image by utilizing a pre-trained emotion classification model to obtain a first emotion category corresponding to the sample face image and a second emotion category corresponding to the predicted face image;
An emotion category loss calculation unit configured to calculate an emotion category loss based on the first emotion category and the second emotion category; and
and a parameter tuning unit configured to adjust parameters of the deep learning model based at least on the emotion category loss to obtain a facial expression capture model.
11. The apparatus of claim 10, further comprising:
a face key point loss calculation unit configured to acquire face key points of the sample face image and face key points of the predicted face image, and calculate face key point loss,
wherein parameters of the deep learning model are adjusted based on the face keypoint loss and the emotion category loss.
12. The apparatus of claim 11, wherein the first face image acquisition unit comprises:
the key point detection unit is configured to detect key points of an original face image so as to obtain face key points of the original face image; and
and the image processing unit is configured to cut, transform and normalize the original face image based on the face key points of the original face image so as to obtain the sample face image and the face key points of the sample face image.
13. The apparatus of claim 11, further comprising:
a three-dimensional key point extraction unit configured to extract three-dimensional key points from the sample three-dimensional face model; and
and the mapping unit is configured to map the three-dimensional key points to obtain the face key points of the predicted face image.
14. The apparatus of claim 11, further comprising:
an image pixel loss calculation unit configured to calculate an image pixel loss based on pixel values of a plurality of pixels in the sample face image and pixel values of a plurality of pixels in the predicted face image,
wherein parameters of the deep learning model are adjusted based on the image pixel loss, face keypoint loss, and emotion category loss.
15. The apparatus of claim 14, wherein the sample face parameters comprise sample microexpressive coefficients, the apparatus further comprising:
a microexpressive label acquisition unit configured to acquire microexpressive labels of the sample face image; and
a microexpressive loss calculation unit configured to calculate microexpressive loss based on the sample microexpressive coefficient and the microexpressive label,
wherein parameters of the deep learning model are adjusted based on the microexpressive loss, the image pixel loss, the face keypoint loss, and the emotion category loss.
16. The apparatus of any of claims 10-15, wherein the pre-trained emotion classification model is pre-trained using video data having faces and corresponding emotion classification tags.
17. The apparatus of any of claims 10-15, wherein the deep learning model comprises an image feature extraction backbone network and a face parameter prediction network, the first face parameter prediction unit comprising:
the image feature extraction subunit is configured to extract image features by utilizing the image feature extraction backbone network based on the sample face image to obtain sample face image features; and
and the face parameter prediction subunit is configured to predict the face parameter by utilizing the face parameter prediction network based on the sample face image characteristics to obtain the sample face parameter.
18. A facial expression driving apparatus comprising:
a second face image acquisition unit configured to acquire a target image;
a second face parameter prediction unit configured to perform face parameter prediction based on the target image using a facial expression capture model obtained by training using the apparatus according to any one of claims 10 to 17, to obtain a target face parameter; and
And the driving unit is configured to drive the facial expression of the target three-dimensional digital person based on the target face parameters.
19. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the method comprises the steps of
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
20. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-9.
21. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311286394.7A CN117218499A (en) | 2023-09-29 | 2023-09-29 | Training method of facial expression capturing model, facial expression driving method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311286394.7A CN117218499A (en) | 2023-09-29 | 2023-09-29 | Training method of facial expression capturing model, facial expression driving method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117218499A true CN117218499A (en) | 2023-12-12 |
Family
ID=89048068
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311286394.7A Pending CN117218499A (en) | 2023-09-29 | 2023-09-29 | Training method of facial expression capturing model, facial expression driving method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117218499A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255535A (en) * | 2021-05-28 | 2021-08-13 | 广州大学 | Depression identification method based on micro-expression analysis |
| CN115512014A (en) * | 2022-10-08 | 2022-12-23 | 北京世纪好未来教育科技有限公司 | Method for training expression driving generation model, expression driving method and device |
| CN116071494A (en) * | 2022-12-23 | 2023-05-05 | 杭州像衍科技有限公司 | High-fidelity three-dimensional face reconstruction and generation method based on implicit nerve function |
| CN116311483A (en) * | 2023-05-24 | 2023-06-23 | 山东科技大学 | Micro-expression recognition method based on local facial area reconstruction and memory contrast learning |
| CN116385827A (en) * | 2023-03-27 | 2023-07-04 | 中国科学技术大学 | Parameterized face reconstruction model training method and key point tag data generation method |
| CN116434303A (en) * | 2023-03-30 | 2023-07-14 | 华南理工大学 | Facial expression capturing method, device and medium based on multi-scale feature fusion |
| CN116524162A (en) * | 2023-04-11 | 2023-08-01 | 北京百度网讯科技有限公司 | Three-dimensional virtual image migration method, model updating method and related equipment |
-
2023
- 2023-09-29 CN CN202311286394.7A patent/CN117218499A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113255535A (en) * | 2021-05-28 | 2021-08-13 | 广州大学 | Depression identification method based on micro-expression analysis |
| CN115512014A (en) * | 2022-10-08 | 2022-12-23 | 北京世纪好未来教育科技有限公司 | Method for training expression driving generation model, expression driving method and device |
| CN116071494A (en) * | 2022-12-23 | 2023-05-05 | 杭州像衍科技有限公司 | High-fidelity three-dimensional face reconstruction and generation method based on implicit nerve function |
| CN116385827A (en) * | 2023-03-27 | 2023-07-04 | 中国科学技术大学 | Parameterized face reconstruction model training method and key point tag data generation method |
| CN116434303A (en) * | 2023-03-30 | 2023-07-14 | 华南理工大学 | Facial expression capturing method, device and medium based on multi-scale feature fusion |
| CN116524162A (en) * | 2023-04-11 | 2023-08-01 | 北京百度网讯科技有限公司 | Three-dimensional virtual image migration method, model updating method and related equipment |
| CN116311483A (en) * | 2023-05-24 | 2023-06-23 | 山东科技大学 | Micro-expression recognition method based on local facial area reconstruction and memory contrast learning |
Non-Patent Citations (2)
| Title |
|---|
| YANTE LI, ET AL: "Deep Learning for Micro-Expression Recognition: A Survey", 《IEEE TRANSACTIONS ON AFFECTIVE COMPUTING》, 10 October 2022 (2022-10-10), pages 2028 - 2046, XP011928667, DOI: 10.1109/TAFFC.2022.3205170 * |
| 罗予东，等: "结合深度神经网络的网络监控系统微表情识别", 《计算机应用软件》, 12 July 2023 (2023-07-12), pages 172 - 179 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113807440B (en) | Method, apparatus, and medium for processing multimodal data using neural networks | |
| CN114511758A (en) | Image recognition method and device, electronic device and medium | |
| CN112749758B (en) | Image processing method, neural network training method, device, equipment and medium | |
| CN114005178B (en) | Character interaction detection method, neural network, training method, training equipment and training medium thereof | |
| CN115082740B (en) | Target detection model training method, target detection device and electronic equipment | |
| CN114119935B (en) | Image processing method and device | |
| CN114723949A (en) | Three-dimensional scene segmentation method and method for training segmentation model | |
| CN113723305A (en) | Image and video detection method, device, electronic equipment and medium | |
| CN116205819B (en) | Character image generation method, training method and device of deep learning model | |
| CN115761855B (en) | Face key point information generation, neural network training and three-dimensional face reconstruction method | |
| CN115511779B (en) | Image detection method, device, electronic equipment and storage medium | |
| CN116152607A (en) | Target detection method, method and device for training target detection model | |
| CN116030185A (en) | Three-dimensional hairline generating method and model training method | |
| CN114842476A (en) | Watermark detection method and device and model training method and device | |
| CN114998963A (en) | Image detection method and method for training image detection model | |
| CN114494797A (en) | Method and apparatus for training image detection model | |
| CN114429678A (en) | Model training method and device, electronic device and medium | |
| CN117218499A (en) | Training method of facial expression capturing model, facial expression driving method and device | |
| CN114821233B (en) | Training method, device, equipment and medium of target detection model | |
| CN115170536B (en) | Image detection method, training method and device of model | |
| CN115578451B (en) | Image processing method, training method and device of image processing model | |
| CN115131562B (en) | Three-dimensional scene segmentation method, model training method, device and electronic equipment | |
| CN115100431B (en) | Target detection method, neural network, training method, training device and training medium thereof | |
| CN115331077B (en) | Training method of feature extraction model, target classification method, device and equipment | |
| CN115797455B (en) | Target detection method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |