WO2025213835A1 - 一种脸部重建方法、装置、设备、介质、产品 - Google Patents

一种脸部重建方法、装置、设备、介质、产品

Info

Publication number
WO2025213835A1
WO2025213835A1 PCT/CN2024/139769 CN2024139769W WO2025213835A1 WO 2025213835 A1 WO2025213835 A1 WO 2025213835A1 CN 2024139769 W CN2024139769 W CN 2024139769W WO 2025213835 A1 WO2025213835 A1 WO 2025213835A1
Authority
WO
WIPO (PCT)
Prior art keywords
facial
dimensional
target image
result
key point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/139769
Other languages
English (en)
French (fr)
Inventor
葛志鹏
胡天舒
梁爽
张隆昊
唐铭谦
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025213835A1 publication Critical patent/WO2025213835A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation

Definitions

  • the present application relates to the field of data processing technology, and in particular to a face reconstruction method, apparatus, device, medium, and product.
  • these application scenarios may have the following requirements: facial reconstruction for a single image or a video, so that the facial reconstruction results can be used to complete certain tasks later, such as 3D facial model construction tasks, 2D video conversion to 3D video tasks, or audio adjustment tasks in videos.
  • the present application provides a facial reconstruction method, apparatus, device, medium, and product, which are conducive to improving the facial reconstruction effect.
  • the present application provides a face reconstruction method, the method comprising:
  • the three-dimensional facial parameter prediction result is fine-tuned based on the two-dimensional facial key point detection result to obtain a three-dimensional facial reconstruction result corresponding to the target image.
  • the fine-tuning process includes:
  • the three-dimensional face reconstruction result is updated according to the difference representation data between the two-dimensional facial key point mapping result and the two-dimensional facial key point detection result.
  • updating the three-dimensional face reconstruction result based on the difference representation data between the two-dimensional facial key point mapping result and the two-dimensional facial key point detection result includes:
  • the three-dimensional face reconstruction result is updated according to the difference representation data between the two-dimensional facial landmark mapping result and the two-dimensional facial landmark detection result, and the difference representation data between the three-dimensional face reconstruction result and the three-dimensional face parameter prediction result.
  • the constraint strength of the update imposed by the difference representation data between the three-dimensional facial reconstruction result and the three-dimensional facial parameter prediction result is weaker than the constraint strength of the update imposed by the difference representation data between the two-dimensional facial key point mapping result and the two-dimensional facial key point detection result.
  • the target image refers to a frame image in a reference video
  • the reference video includes a frame image previous to the target image
  • the three-dimensional face reconstruction result is updated according to the temporal loss corresponding to the three-dimensional face reconstruction result and/or the temporal loss corresponding to the two-dimensional facial key point mapping result;
  • the temporal loss corresponding to the 3D facial reconstruction result is determined based on motion state representation data between the 3D facial parameter prediction result of the target image and the 3D facial parameter information of the previous frame image, and motion state representation data between the 3D facial reconstruction result corresponding to the target image and the 3D facial parameter information of the previous frame image;
  • the 3D facial parameter information of the previous frame image is determined based on the 3D facial parameter prediction result of the previous frame image and/or the 3D facial reconstruction result corresponding to the previous frame image;
  • the temporal loss corresponding to the two-dimensional facial key point mapping result is determined based on the motion state representation data between the two-dimensional facial key point detection result of the target image and the two-dimensional facial key point detection result of the previous frame image, and the motion state representation data between the two-dimensional facial key point mapping result corresponding to the target image and the two-dimensional facial key point detection result of the previous frame image.
  • updating the three-dimensional face reconstruction result includes:
  • the target image refers to any frame image in a reference video
  • the facial identification parameters in the three-dimensional facial reconstruction result corresponding to the target image are determined based on an average value of the facial identification parameters in the three-dimensional facial parameter prediction results of at least two frames of images in the reference video, where the at least two frames of images include the target image.
  • the target image refers to any frame image in a reference video
  • the method further comprises:
  • a video corresponding to the audio sequence is generated, where the object presented in the video corresponding to the audio sequence is consistent with the object presented in the reference video, and the video corresponding to the audio sequence is used to describe the changes in the facial state of the object under the audio sequence.
  • the present application provides a facial reconstruction device, comprising:
  • an acquisition unit configured to acquire a target image
  • a processing unit configured to perform two-dimensional facial key point detection processing on the target image to obtain a two-dimensional facial key point detection result of the target image, and perform three-dimensional facial parameter prediction processing on the target image to obtain a three-dimensional facial parameter prediction result of the target image;
  • a fine-tuning unit is used to fine-tune the three-dimensional facial parameter prediction result based on the two-dimensional facial key point detection result to obtain a three-dimensional facial reconstruction result corresponding to the target image.
  • the present application provides an electronic device, the device comprising: a processor and a memory;
  • the memory is used to store instructions or computer programs
  • the processor is configured to execute the instructions or computer program in the memory so that the electronic device performs the face reconstruction method provided in this application.
  • the present application provides a computer-readable medium having instructions or a computer program stored therein.
  • the instructions or the computer program are executed on a device, the device executes the face reconstruction method provided by the present application.
  • the present application provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the face reconstruction method provided by the present application.
  • a two-dimensional facial key point detection process is performed on the target image to obtain a two-dimensional facial key point detection result of the target image, so that the two-dimensional facial key point detection result can represent the facial state of the object in the target image in the two-dimensional space
  • a three-dimensional facial parameter prediction process is performed on the target image to obtain a three-dimensional facial parameter prediction result of the target image, so that the three-dimensional facial parameter prediction result can more accurately represent the facial state of the object in the three-dimensional space
  • the three-dimensional facial parameter prediction result is fine-tuned to obtain a three-dimensional facial reconstruction result corresponding to the target image.
  • FIG1 is a flow chart of a face reconstruction method provided in an embodiment of the present application.
  • FIG2 is a schematic diagram of an implementation process of an audio adjustment task in a video provided by an embodiment of the present application
  • FIG3 is a schematic diagram of a face reconstruction process provided in an embodiment of the present application.
  • FIG4 is a schematic structural diagram of a face reconstruction device provided in an embodiment of the present application.
  • FIG5 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.
  • the facial reconstruction method provided by this application includes steps S1-S4 below.
  • Figure 1 is a flow chart of a facial reconstruction method provided by this embodiment of the application.
  • the target image refers to an image that needs to be reconstructed for the face, and this application does not limit the implementation method of the target image. For ease of understanding, two situations are described below.
  • the target image mentioned above may refer to the image involved in the single-image processing task, such as an image specified by the user or an image provided by another task. It should be noted that this application is not limited to single-image processing tasks.
  • the single-image processing task may at least involve face reconstruction processing for a single image.
  • the target image mentioned above may refer to any frame of an image in a reference video, such as the i-th frame, where i is a positive integer, i ⁇ I, I is a positive integer, and I represents the total number of frames in the reference video.
  • the reference video refers to a video involved in a certain video processing task that requires face reconstruction processing, such as the reference video shown in FIG2 or FIG3; and the present application does not limit the implementation method of the reference video.
  • the reference video can be implemented using a single-person voice-over video.
  • the present application does not limit the implementation method of the video processing task.
  • the video processing task at least involves face reconstruction processing for part or all of the images in the video.
  • the video processing task can be implemented using tasks such as video translation, video speech error correction, and partial audio replacement in a video.
  • the target image above may refer to the i-th frame image in the reference video, where i is a positive integer, i ⁇ I, I is a positive integer, and I represents the total number of image frames in the reference video.
  • this application does not limit the method for obtaining the above target image.
  • the two-dimensional facial key point detection results of the target image are used to describe the facial state of the object in the target image within the two-dimensional space, such as the facial expression. Furthermore, this application does not limit the implementation of the two-dimensional facial key point detection results. For example, it can be implemented using any existing or future two-dimensional facial key points, such as two-dimensional landmarks. It should be noted that this application does not limit the implementation of the object. For example, the object can be implemented using an animal or a virtual image.
  • the present application does not limit the implementation method of the two-dimensional facial key point detection processing in S2 above.
  • it can adopt any existing or future method that can perform two-dimensional facial key point detection processing on an image, such as using a pre-built machine learning model with two-dimensional facial key point detection processing function.
  • the three-dimensional facial parameter prediction result of the target image is used to describe the facial state of the object in the target image in the three-dimensional space, such as the expression state.
  • the present application does not limit the implementation method of the three-dimensional facial parameter prediction result of the target image above.
  • it may include facial identification (Identity document, ID) parameters, facial expression parameters and facial posture parameters.
  • the facial identification parameters are used to describe the facial features of the object in the target image, such as facial contour, distribution of facial features and other features, so that the three-dimensional facial model constructed based on the facial identification parameters can represent the facial state of the object in the absence of expression and posture.
  • the facial expression parameters are used to describe the expression state of the object in the target image, so that the three-dimensional facial model constructed based on the facial expression parameters can represent the facial state of the object in the absence of ID and posture; and the present application does not limit the implementation method of the facial expression parameters.
  • the facial expression parameters can be implemented using blendshape coefficients.
  • the facial pose parameters are used to describe the facial pose of the subject in the target image, such as frontal or profile, so that the three-dimensional facial model constructed based on the facial pose parameters can represent the facial state of the subject without an ID and without expression.
  • this application does not limit the implementation of the facial pose parameters.
  • the facial pose parameters may include coefficients such as rotation, translation, and scaling. Therefore, in one possible implementation, the three-dimensional facial parameter prediction result of the target image can be implemented using a three-dimensional facial morphable model (3DMM).
  • 3DMM three-dimensional facial morphable model
  • S3 can specifically be: using a pre-built three-dimensional facial parameter prediction model to perform three-dimensional facial parameter prediction processing on the target image to obtain a three-dimensional facial parameter prediction result of the target image.
  • the three-dimensional facial parameter prediction model is used to perform three-dimensional facial parameter prediction processing on the input data of the three-dimensional facial parameter prediction model; and this application does not limit the implementation of the three-dimensional facial parameter prediction model.
  • it can be implemented using any existing or future model with three-dimensional facial parameter prediction function, such as a machine learning model.
  • this application does not limit the correlation between the execution time of S3 and the execution time of S2.
  • the former may be earlier than the latter.
  • the former is later than the latter.
  • the two are the same.
  • the i-th frame image in the reference video after obtaining the i-th frame image, the i-th frame image can be input into a pre-constructed three-dimensional facial parameter prediction model, so that the three-dimensional facial parameter prediction model can perform three-dimensional facial parameter prediction processing on the i-th frame image, and obtain and output the three-dimensional facial parameter prediction result of the i-th frame image.
  • the three-dimensional facial parameter prediction model has good three-dimensional facial parameter prediction performance, the three-dimensional facial parameter prediction result obtained by using the three-dimensional facial parameter prediction model to predict the i-th frame image can more accurately represent the facial state presented by the object in the i-th frame image in three-dimensional space, so that the three-dimensional facial parameter prediction result can be used as the initial value to better determine the three-dimensional facial reconstruction result for the i-th frame image.
  • i is a positive integer
  • i ⁇ I I
  • I represents the total number of image frames in the reference video.
  • the three-dimensional facial reconstruction result corresponding to the target image refers to the fine-tuning processing result of the three-dimensional facial parameter prediction result of the target image, so that the three-dimensional facial reconstruction result corresponding to the target image can more accurately represent the facial state of the object in the target image in the three-dimensional space.
  • this application does not limit the implementation of the 3D facial reconstruction result corresponding to the target image described above.
  • the implementation of the 3D facial reconstruction result corresponding to the target image described above may be similar to the implementation of the 3D facial parameter prediction result for the target image described above.
  • the 3D facial reconstruction result corresponding to the target image may include facial identification parameters, facial expression parameters, and facial posture parameters.
  • S4 can specifically be: inputting the 2D facial key point detection results of the target image and the 3D facial parameter prediction results of the target image into a pre-built parameter fine-tuning model, so that the parameter fine-tuning model can fine-tune the 3D facial parameter prediction results of the target image based on the 2D facial key point detection results of the target image, and obtain and output the 3D facial reconstruction result corresponding to the target image.
  • the parameter fine-tuning model refers to a pre-built model with 3D facial parameter fine-tuning processing capabilities, such as a machine learning model; and this application does not limit the implementation of the parameter fine-tuning model.
  • the present application also provides a possible implementation of the above S4.
  • the S4 may specifically include the following steps 11 to 14.
  • Step 11 Initialize the 3D facial reconstruction result corresponding to the target image based on the 3D facial parameter prediction result of the target image.
  • the present application does not limit the implementation of the above step 11.
  • it can specifically be: determining the three-dimensional facial parameter prediction result of the target image as the initial value of the three-dimensional facial reconstruction result corresponding to the target image.
  • the above step 11 can specifically be: using the facial identification parameters in the three-dimensional facial parameter prediction result, initializing the facial identification parameters in the three-dimensional facial reconstruction result corresponding to the target image, so that the initial values of the facial identification parameters in the three-dimensional facial reconstruction result are consistent with the facial identification parameters in the three-dimensional facial parameter prediction result; using the facial expression parameters in the three-dimensional facial parameter prediction result, initializing the facial expression parameters in the three-dimensional facial reconstruction result, so that the initial values of the facial expression parameters in the three-dimensional facial reconstruction result are consistent with the facial expression parameters in the three-dimensional facial parameter prediction result; using the facial posture parameters in the three-dimensional facial parameter prediction result, initializing the facial posture parameters in the three-dimensional facial reconstruction result, so that the initial values of the facial posture parameters in the three-dimensional facial reconstruction result are consistent with the facial posture parameters in the three-dimensional facial parameter prediction result.
  • Step 12 Based on the 3D facial reconstruction result corresponding to the target image, a 3D facial model corresponding to the target image is constructed.
  • the three-dimensional face model corresponding to the target image refers to a three-dimensional face model constructed based on the three-dimensional face reconstruction result corresponding to the target image, so that the model can present the facial state of the object in the target image in the three-dimensional space.
  • step 12 can be implemented using any existing or future method that can construct a three-dimensional facial model based on 3DMM.
  • Step 13 Map the three-dimensional facial model corresponding to the target image to the two-dimensional image space to obtain the two-dimensional facial key point mapping result corresponding to the target image.
  • the two-dimensional facial key point mapping result corresponding to the target image is obtained by mapping the three-dimensional facial model corresponding to the target image to the two-dimensional image space, so that the two-dimensional facial key point mapping result can represent the state of the three-dimensional facial model in the two-dimensional space, and thus the two-dimensional facial key point mapping result can, to a certain extent, represent the facial state of the object in the target image in the two-dimensional space.
  • this application does not limit the implementation of step 13 above.
  • it may specifically include: first mapping the 3D facial model corresponding to the target image into a 2D image; then performing 2D facial landmark detection on the 2D image to obtain a 2D facial landmark mapping result corresponding to the target image.
  • this application does not limit the method for obtaining the 2D image; for example, it may be implemented using any existing or future method that can map the 3D facial model back to a 2D image.
  • Step 14 Based on the difference between the 2D facial landmark mapping result corresponding to the target image and the 2D facial landmark detection result of the target image, update the 3D facial reconstruction result corresponding to the target image, and return to step 12 and subsequent steps above until the preset stopping condition is met.
  • the difference characterization data is used to characterize the difference between the two-dimensional facial key point mapping result and the two-dimensional facial key point detection result, so that the difference characterization data can represent the error presented by the three-dimensional facial reconstruction result corresponding to the target image in the current round under the reprojection of the two-dimensional key points, thereby making the difference characterization data able to represent the accuracy of the three-dimensional facial reconstruction result corresponding to the target image in the current round to a certain extent, and the present application does not limit the determination process of the difference characterization data. For example, it can be implemented using an existing or future loss function that can measure the differences between different two-dimensional facial key points.
  • step 14 it may specifically include: updating the 3D facial reconstruction result corresponding to the target image based on the difference representation data between the 2D facial landmark mapping result corresponding to the target image and the 2D facial landmark detection result of the target image, so that the error of the updated 3D facial reconstruction result in the reprojection of the 2D landmarks is less than the error of the 3D facial reconstruction result before the update. Then, based on the updated 3D facial reconstruction result, continuing step 12 and subsequent steps above to initiate the next round of the process, and repeating the process until a preset stopping condition is met.
  • the preset stopping condition refers to a condition that must be met at the end of the iterative cycle, such as a condition that the error of the 3D facial reconstruction result in the reprojection of the 2D landmarks of the current round is less than a preset error threshold.
  • the 3D facial reconstruction result can be iteratively updated by continuously reducing the error of the 3D facial reconstruction result in the reprojection of the 2D landmarks of the target image.
  • the three-dimensional facial parameter prediction results of the target image can relatively accurately represent the facial state of the object in the target image in the three-dimensional space. Therefore, the three-dimensional facial reconstruction results corresponding to the target image can be determined within a variation range not too far from the three-dimensional facial parameter prediction results. This is conducive to improving accuracy and efficiency.
  • step 14 can specifically be: based on the difference representation data between the two-dimensional facial key point mapping result corresponding to the target image and the two-dimensional facial key point detection result of the target image, and the difference representation data between the three-dimensional facial reconstruction result corresponding to the target image and the three-dimensional facial parameter prediction result of the target image, update the three-dimensional facial reconstruction result corresponding to the target image, and return to continue executing step 12 above and its subsequent steps until the preset stop condition is reached.
  • the difference characterization data is used to characterize the difference between the three-dimensional facial reconstruction result and the three-dimensional facial parameter prediction result, so that the difference characterization data can represent the relative distance between the three-dimensional facial reconstruction result corresponding to the target image in the current round and the three-dimensional facial parameter prediction result, so that the difference characterization data can, to a certain extent, represent the degree of change of the three-dimensional facial reconstruction result corresponding to the target image in the current round relative to the three-dimensional facial parameter prediction result, and then, with the help of the difference characterization data, the three-dimensional facial reconstruction result can be constrained to not change drastically relative to the three-dimensional facial parameter prediction result during updating, which is conducive to improving the reconstruction accuracy.
  • the above two types of difference representation data can satisfy the following constraints: the constraint strength of the difference representation data between the three-dimensional facial reconstruction result corresponding to the target image and the three-dimensional facial parameter prediction result of the target image on the update is weaker than the constraint strength of the difference representation data between the two-dimensional facial key point mapping result corresponding to the target image and the two-dimensional facial key point detection result of the target image on the update, so that the subsequent update process based on this constraint can achieve better optimization effect.
  • this application does not limit the implementation method of this constraint. For example, it can control the constraint strength of the two types of difference representation data on the update through a hyperparameter.
  • the update process of the three-dimensional facial reconstruction result not only refers to the influencing factor of "the difference representation data between the two-dimensional facial key point mapping result corresponding to the target image and the two-dimensional facial key point detection result of the target image” above, but also refers to the influencing factor of "the difference representation data between the three-dimensional facial reconstruction result corresponding to the target image and the three-dimensional facial parameter prediction result of the target image” above, so that the gap between the updated three-dimensional facial reconstruction result and the three-dimensional facial parameter prediction result of the target image is smaller, and the error presented by the updated three-dimensional facial reconstruction result in the reprojection of the two-dimensional key points is smaller than the error presented by the three-dimensional facial reconstruction result before the update in the reprojection of the two-dimensional key points.
  • the optimization is carried out along the guidance direction of the two-dimensional facial key point detection result without devi
  • the present application does not limit the implementation of the preset stop condition corresponding to the update process in the above paragraph.
  • the preset stop condition may specifically include: the loss of the 3D facial reconstruction result of the current round is lower than a preset loss threshold.
  • the preset stop condition may specifically include: the rate of change of the loss of the 3D facial reconstruction result of the current round is lower than a preset rate of change threshold.
  • the preset stop condition may specifically include: the number of updates of the 3D facial reconstruction result reaches a preset number threshold.
  • the loss is used to characterize the performance of the 3D facial reconstruction result, such as accuracy and stability; and the loss is determined based on the difference characterization data between the 2D facial key point mapping result corresponding to the target image and the 2D facial key point detection result of the target image, and the difference characterization data between the 3D facial reconstruction result corresponding to the target image and the 3D facial parameter prediction result of the target image.
  • the present application also provides a possible implementation method of the update process of the 3D facial reconstruction result corresponding to the target image mentioned above.
  • the 3D facial reconstruction result corresponding to the target image can be updated based on the timing loss corresponding to the 3D facial reconstruction result and/or the timing loss corresponding to the 2D facial key point mapping result mentioned above, so that the update process of the 3D facial reconstruction result satisfies the timing constraints in the reference video.
  • the arrangement position of the target image in the reference video is adjacent to the arrangement position of the previous frame image in the reference video
  • the arrangement position of the target image in the reference video is later than the arrangement position of the previous frame image in the reference video.
  • the above reference video can be divided into multiple segments, so that each video segment is used to describe the changes in an action, and different video segments are used to describe different actions, so that timing constraints can be performed on each video segment later.
  • the following constraints can be satisfied between the above target image and the previous frame image: the target image and the previous frame image come from the same video segment, the target image's arrangement position in the video segment is adjacent to the previous frame image's arrangement position in the video segment, and the target image's arrangement position in the video segment is later than the previous frame image's arrangement position in the video segment.
  • the temporal loss corresponding to the 3D facial reconstruction result is used to represent the state of the 3D facial reconstruction result in terms of temporal constraints; and the temporal loss corresponding to the 3D facial reconstruction result is determined based on the motion state representation data between the 3D facial parameter prediction result of the target image and the 3D facial parameter information of the previous frame image, and the motion state representation data between the 3D facial reconstruction result corresponding to the target image and the 3D facial parameter information of the previous frame image.
  • the 3D facial parameter information of the previous frame image is used to describe the facial state of the object in the previous frame image in 3D space; and the 3D facial parameter information of the previous frame image is determined based on the 3D facial parameter prediction result of the previous frame image and/or the 3D facial reconstruction result corresponding to the previous frame image, so that the 3D facial parameter information of the previous frame image may include the 3D facial parameter prediction result of the previous frame image and/or the 3D facial reconstruction result corresponding to the previous frame image.
  • the three-dimensional facial parameter prediction result of the previous frame image is obtained by performing a three-dimensional facial parameter prediction process on the previous frame image; and the implementation method of the three-dimensional facial parameter prediction result of the previous frame image is similar to the implementation method of the three-dimensional facial parameter prediction result of the target image described above.
  • the three-dimensional facial reconstruction result corresponding to the previous frame image is obtained by fine-tuning the three-dimensional facial parameter prediction result of the previous frame image based on the two-dimensional facial key point detection result of the previous frame image; and the implementation method of the three-dimensional facial reconstruction result corresponding to the previous frame image is similar to the implementation method of the three-dimensional facial reconstruction result corresponding to the target image described above.
  • the two-dimensional facial key point detection result of the previous frame image is obtained by performing a two-dimensional facial key point detection process on the previous frame image; and the implementation method of the two-dimensional facial key point detection result of the previous frame image is similar to the implementation method of the two-dimensional facial key point detection result of the target image described above.
  • the motion state representation data is used to represent the motion state of the object represented by the three-dimensional facial parameter prediction result relative to the object in the previous frame image in the three-dimensional space, such as speed, acceleration, etc., so that the motion state representation data can, to a certain extent, represent the motion state of the object in the target image relative to the object in the previous frame image in the three-dimensional space, thereby enabling the motion state representation data to, to a certain extent, represent the timing constraints that the three-dimensional facial reconstruction result corresponding to the target image needs to meet; and this application does not limit the implementation method of the motion state representation data, for example, it can include speed and/or acceleration, etc. In addition, this application does not limit the method for obtaining the motion state representation data, for example, it can be implemented using any existing or future method that can measure the motion state between two frames of data.
  • the motion state representation data is used to represent the motion state of the object represented by the three-dimensional facial reconstruction result relative to the object in the previous frame image in three-dimensional space, such as speed, acceleration, etc.; and this application does not limit the implementation method of the motion state representation data; for example, it may include speed and/or acceleration, etc. In addition, this application does not limit the method for obtaining the motion state representation data; for example, it may be implemented using any existing or future method that can measure the motion state between two frames of data.
  • the present application does not limit the method for determining the timing loss corresponding to the above-mentioned three-dimensional facial reconstruction result.
  • it can be specifically as follows: first calculate the difference representation data between the above-mentioned "motion state representation data between the three-dimensional facial reconstruction result corresponding to the target image and the three-dimensional facial parameter information of the previous frame image" and the above-mentioned "motion state representation data between the three-dimensional facial parameter prediction result of the target image and the three-dimensional facial parameter information of the previous frame image", so that the difference representation data can represent the state of the three-dimensional facial reconstruction result in terms of timing constraints, such as the degree of constraint satisfaction or whether the constraints are satisfied; and then determine the timing loss corresponding to the three-dimensional facial reconstruction result based on the difference representation data.
  • the temporal loss corresponding to the two-dimensional facial keypoint mapping result is used to represent the state of the two-dimensional facial keypoint mapping result in terms of temporal constraints; and the temporal loss corresponding to the two-dimensional facial keypoint mapping result is determined based on the motion state representation data between the two-dimensional facial keypoint detection result of the target image and the two-dimensional facial keypoint detection result of the previous frame image, and the motion state representation data between the two-dimensional facial keypoint mapping result corresponding to the target image and the two-dimensional facial keypoint detection result of the previous frame image.
  • the two-dimensional facial keypoint detection result of the previous frame image is used to describe the facial state of the object in the previous frame image in two-dimensional space.
  • the motion state representation data is used to represent the motion state of the object represented by the two-dimensional facial key point detection result relative to the object in the previous frame image in the two-dimensional space, such as speed, acceleration, etc., so that the motion state representation data can, to a certain extent, represent the motion state of the object in the target image relative to the object in the previous frame image in the two-dimensional space, thereby enabling the motion state representation data to, to a certain extent, represent the timing constraints that the two-dimensional facial key point mapping result corresponding to the target image needs to meet; and the present application does not limit the implementation method of the motion state representation data, for example, it can include speed and/or acceleration, etc. In addition, the present application does not limit the method for obtaining the motion state representation data, for example, it can be implemented using any existing or future method that can measure the motion state between two frames of data.
  • the motion state representation data is used to represent the motion state of the object represented by the two-dimensional facial key point mapping result relative to the object in the previous frame image in the two-dimensional space, such as speed, acceleration, etc.; and this application does not limit the implementation method of the motion state representation data, for example, it can include speed and/or acceleration, etc. In addition, this application does not limit the method for obtaining the motion state representation data, for example, it can be implemented using any existing or future method that can measure the motion state between two frames of data.
  • the present application does not limit the method for determining the timing loss corresponding to the above-mentioned two-dimensional facial key point mapping result.
  • it can be specifically as follows: first calculate the difference representation data between the above-mentioned "motion state representation data between the two-dimensional facial key point mapping result corresponding to the target image and the two-dimensional facial key point detection result of the previous frame image" and the above-mentioned "motion state representation data between the two-dimensional facial key point detection result of the target image and the two-dimensional facial key point detection result of the previous frame image", so that the difference representation data can represent the state of the two-dimensional facial key point mapping result in terms of timing constraints, such as the degree of constraint satisfaction or whether the constraints are satisfied; and then determine the timing loss corresponding to the two-dimensional facial key point mapping result based on the difference representation data.
  • the above step 14 can be specifically as follows: based on the difference representation data between the two-dimensional facial key point mapping result corresponding to the target image and the two-dimensional facial key point detection result of the target image, the difference representation data between the three-dimensional facial reconstruction result corresponding to the target image and the three-dimensional facial parameter prediction result of the target image, the timing loss corresponding to the three-dimensional facial reconstruction result, and the timing loss corresponding to the two-dimensional facial key point mapping result, the three-dimensional facial reconstruction result corresponding to the target image is updated, so that the updated three-dimensional facial reconstruction result is at the same time as the two-dimensional key point re-recognition result.
  • the error presented on the projection is smaller than the error presented on the reprojection of the two-dimensional key points of the three-dimensional facial reconstruction result before the update, and the gap between the updated three-dimensional facial reconstruction result and the three-dimensional facial parameter prediction result of the target image is smaller.
  • the loss presented by the updated three-dimensional facial reconstruction result in terms of timing constraints is also smaller, so that the updated three-dimensional facial reconstruction result can better represent the facial state of the object in the target image, so that the above step 12 and subsequent steps can be returned to and continued based on the updated three-dimensional facial reconstruction result, and the iterative cycle is repeated until a preset stop condition is reached.
  • the preset stop condition may specifically include: the loss of the 3D facial reconstruction result of the current round is lower than a preset loss threshold.
  • the preset stop condition may specifically include: the rate of change of the loss of the 3D facial reconstruction result of the current round is lower than a preset rate of change threshold.
  • the preset stop condition may specifically include: the number of updates of the 3D facial reconstruction result reaches a preset number threshold.
  • the loss may be determined based on difference representation data between the 2D facial key point mapping result corresponding to the target image and the 2D facial key point detection result of the target image, difference representation data between the 3D facial reconstruction result corresponding to the target image and the 3D facial parameter prediction result of the target image, the temporal loss corresponding to the 3D facial reconstruction result, and the temporal loss corresponding to the 2D facial key point mapping result.
  • this application also provides an implementation method for the aforementioned step of "updating the 3D facial reconstruction result corresponding to the target image.”
  • the step can specifically include updating other parameters in the 3D facial reconstruction result, such as facial expression parameters and facial posture parameters, in addition to the facial identification parameters.
  • the update process for the 3D facial reconstruction result can be completed by fixing the facial identification parameters in the 3D facial reconstruction result and updating the facial expression parameters and facial posture parameters in the 3D facial reconstruction result. This effectively avoids interference with the facial identification parameters during the update process, thereby improving the reconstruction effect.
  • a three-dimensional face model can be first constructed based on the three-dimensional face reconstruction result; then, the three-dimensional face model can be mapped into a two-dimensional facial key point mapping result corresponding to the target image; then, based on the difference representation data between the two-dimensional facial key point mapping result corresponding to the target image and the two-dimensional facial key point detection result of the target image, the difference representation data between the three-dimensional face reconstruction result corresponding to the target image and the three-dimensional facial parameter prediction result of the target image, the timing loss corresponding to the three-dimensional face reconstruction result, and the timing loss corresponding to the two-dimensional facial key point mapping result, the loss of the three-dimensional face reconstruction result is determined, so that the loss can represent the performance of the three-dimensional face reconstruction result corresponding to the target image in the current round, such as accuracy + stability; finally Under
  • step 12 it is then possible to continue to perform step 12 and subsequent steps based on the updated 3D facial reconstruction result, and repeat the iterative cycle until a preset stop condition is reached.
  • the 3D facial reconstruction result finally obtained can more accurately represent the facial state of the object in the target image in the 3D space, thereby improving the facial reconstruction effect.
  • the initial value of the 3D facial reconstruction result is determined based on the 3D facial parameter prediction result of the target image, the initial value of the 3D facial reconstruction result can more accurately represent the facial state of the object in the target image, so that subsequent convergence can be achieved quickly and stably based on the initial value, which is conducive to improving the reconstruction effect.
  • this application also provides a method for obtaining facial identifier parameters in the 3D facial reconstruction result corresponding to the target image described above.
  • the facial identifier parameters in the 3D facial reconstruction result corresponding to the target image can be determined based on the average value of the facial identifier parameters in the 3D facial parameter prediction results of at least two frames in the reference video.
  • the at least two frames include the target image.
  • this application does not limit the implementation of the at least two frames; for example, the at least two frames can refer to all images in the reference video. For another example, after the reference video is divided into multiple video segments, the at least two frames can refer to all images in the video segment that includes the target image, so that the at least two frames can better describe the facial features of the subject in the target image.
  • the process of obtaining facial identification parameters in the 3D facial reconstruction result corresponding to the target image may include: first obtaining the 3D facial parameter prediction results of each frame image in the reference video; then calculating the average value of the facial identification parameters in the 3D facial parameter prediction results of all images in the reference video as the facial identification parameters in the 3D facial reconstruction result corresponding to the target image, so that the 3D facial reconstruction result corresponding to the target image can more accurately represent the facial features of the object in the target image, such as facial contour, distribution of facial features, etc., thereby improving the facial reconstruction effect.
  • i is a positive integer
  • I is a positive integer
  • I represents the total number of image frames in the reference video.
  • the target image is subjected to two-dimensional facial key point detection processing to obtain a two-dimensional facial key point detection result of the target image, so that the two-dimensional facial key point detection result can represent the facial state of the object in the target image in the two-dimensional space
  • the target image is subjected to three-dimensional facial parameter prediction processing to obtain a three-dimensional facial parameter prediction result of the target image, so that the three-dimensional facial parameter prediction result can more accurately represent the facial state of the object in the three-dimensional space
  • the three-dimensional facial parameter prediction result is fine-tuned to obtain a three-dimensional facial reconstruction result corresponding to the target image, so that the three-dimensional facial reconstruction result can more accurately represent the facial state of the object in the three-dimensional space, which is conducive to improving the facial reconstruction effect.
  • the fine-tuning processing based on the three-dimensional facial parameter prediction result can converge quickly and stably, so that the three-dimensional facial reconstruction result obtained by the fine-tuning processing can more accurately represent the facial state of the object in the three-dimensional space, which is conducive to improving the facial reconstruction effect, such as accuracy, stability and efficiency.
  • the present application does not limit the execution subject of the face reconstruction method provided in the embodiments of the present application.
  • the face reconstruction method provided in the embodiments of the present application can be applied to a terminal device or a server.
  • the face reconstruction method provided in the embodiments of the present application can also be implemented with the help of the data interaction process between the terminal device and the server.
  • the terminal device can be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc.
  • PDA personal digital assistant
  • the server can be a standalone server, a cluster server, or a cloud server.
  • the present application does not limit the application scenarios of the face reconstruction method provided in the embodiments of the present application.
  • the following is an illustration with examples.
  • the face reconstruction method when the face reconstruction method provided in this application is applied to a certain video processing task, such as an audio adjustment task in a video, the face reconstruction method may include the following steps 21 to 25.
  • Step 21 Acquire a target image, which refers to any frame image in the reference video.
  • the i-th frame image in the reference video can be considered as the target image, so that the 3D facial reconstruction result corresponding to the i-th frame image can be determined by using the relevant processing of the target image.
  • i is a positive integer
  • i ⁇ I is a positive integer
  • I represents the total number of image frames in the reference video.
  • Step 22 Perform two-dimensional facial key point detection on the target image to obtain a two-dimensional facial key point detection result of the target image.
  • Step 23 Perform three-dimensional facial parameter prediction processing on the target image to obtain a three-dimensional facial parameter prediction result of the target image.
  • Step 24 Based on the 2D facial key point detection result of the target image, fine-tune the 3D facial parameter prediction result of the target image to obtain a 3D facial reconstruction result corresponding to the target image.
  • Step 25 Generate a video corresponding to the audio sequence based on the 3D facial reconstruction results corresponding to each frame image in the reference video.
  • the object presented in the video corresponding to the audio sequence is consistent with the object presented in the reference video.
  • the video corresponding to the audio sequence is used to describe the changes in the facial state of the object under the audio sequence.
  • the audio sequence refers to the audio required for reference when processing the reference video, such as lip adjustment processing; and this application does not limit the implementation method of the audio sequence.
  • the audio sequence satisfies the following constraints: the total number of frames in the audio sequence is consistent with the total number of frames in the reference video.
  • the video corresponding to the audio sequence refers to the video generated for the audio sequence, so that the video satisfies the following constraints: the object presented in the video corresponding to the audio sequence is consistent with the object presented in the reference video, and the video corresponding to the audio sequence is used to describe the changes in the facial state of the object under the audio sequence.
  • the present application does not limit the implementation method of the above step 25.
  • it can be implemented with the help of any existing or future method that can generate a video based on some three-dimensional facial reconstruction results and an audio sequence.
  • the present application also provides a possible implementation method of this step 25.
  • this step 25 can specifically include: using a pre-constructed video generation model to generate a video corresponding to the audio sequence based on the three-dimensional facial reconstruction results corresponding to all images in the reference video, so that the object presented in the video corresponding to the audio sequence is consistent with the object presented in the reference video, and the video corresponding to the audio sequence is used to describe the changes in the facial state of the object under the audio sequence.
  • the video generation model refers to a pre-constructed model with video generation function, such as a machine learning model; and the present application does not limit the implementation method of the video generation model.
  • a two-dimensional facial key point detection process is performed on the i-th frame image in the reference video to obtain a two-dimensional facial key point detection result of the target image
  • a three-dimensional facial parameter prediction process is performed on the i-th frame image to obtain a three-dimensional facial parameter prediction result of the target image, where i is a positive integer, i ⁇ I, I is a positive integer, and I represents the total number of image frames in the reference video; then, based on the two-dimensional facial key point detection result of the i-th frame image, a three-dimensional facial parameter prediction process is performed on the i-th frame image.
  • the parameter prediction results are fine-tuned to obtain a 3D facial reconstruction result corresponding to the i-th frame image, where i is a positive integer, i ⁇ I, and I is a positive integer representing the total number of image frames in the reference video.
  • a video corresponding to the audio sequence is generated based on the 3D facial reconstruction results corresponding to all images in the reference video, such that the object presented in the video corresponding to the audio sequence is consistent with the object presented in the reference video, and the video corresponding to the audio sequence is used to describe the facial state changes of the object in the audio sequence. This can achieve adjustment of the facial state changes of the object in the reference video based on the audio sequence.
  • these 3D facial reconstruction results can accurately represent the facial state of the object in the reference video, such as facial features and facial expressions such as large facial amplitudes when speaking, the video generated based on these 3D facial reconstruction results can better represent the facial state changes of the object in the audio sequence, thereby improving the video generation effect.
  • FIG. 4 is a schematic structural diagram of the facial reconstruction device provided in the embodiments of this application. It should be noted that for technical details of the facial reconstruction device provided in the embodiments of this application, please refer to the relevant content of the facial reconstruction method above.
  • the face reconstruction device 400 provided in an embodiment of the present application includes:
  • An acquisition unit 401 is configured to acquire a target image
  • the processing unit 402 is configured to perform a two-dimensional facial key point detection process on the target image to obtain a two-dimensional facial key point detection result of the target image, and perform a three-dimensional facial parameter prediction process on the target image to obtain a three-dimensional facial parameter prediction result of the target image;
  • the fine-tuning unit 403 is configured to fine-tune the 3D facial parameter prediction result based on the 2D facial key point detection result to obtain a 3D facial reconstruction result corresponding to the target image.
  • the fine-tuning unit 403 is specifically configured to: initialize the 3D facial reconstruction result based on the 3D facial parameter prediction result; construct a 3D facial model corresponding to the target image based on the 3D facial reconstruction result; map the 3D facial model to a 2D image space to obtain a 2D facial key point mapping result corresponding to the target image; and update the 3D facial reconstruction result based on difference representation data between the 2D facial key point mapping result and the 2D facial key point detection result.
  • the fine-tuning unit 403 is specifically configured to update the three-dimensional facial reconstruction result based on difference representation data between the two-dimensional facial key point mapping result and the two-dimensional facial key point detection result, and difference representation data between the three-dimensional facial reconstruction result and the three-dimensional facial parameter prediction result.
  • the constraint strength of the update imposed by the difference representation data between the three-dimensional facial reconstruction result and the three-dimensional facial parameter prediction result is weaker than the constraint strength of the update imposed by the difference representation data between the two-dimensional facial key point mapping result and the two-dimensional facial key point detection result.
  • the target image refers to a frame image in a reference video
  • the reference video includes a frame image previous to the target image
  • the three-dimensional face reconstruction result is updated according to the temporal loss corresponding to the three-dimensional face reconstruction result and/or the temporal loss corresponding to the two-dimensional facial key point mapping result;
  • the temporal loss corresponding to the 3D facial reconstruction result is determined based on motion state representation data between the 3D facial parameter prediction result of the target image and the 3D facial parameter information of the previous frame image, and motion state representation data between the 3D facial reconstruction result corresponding to the target image and the 3D facial parameter information of the previous frame image;
  • the 3D facial parameter information of the previous frame image is determined based on the 3D facial parameter prediction result of the previous frame image and/or the 3D facial reconstruction result corresponding to the previous frame image;
  • the temporal loss corresponding to the two-dimensional facial key point mapping result is determined based on the motion state representation data between the two-dimensional facial key point detection result of the target image and the two-dimensional facial key point detection result of the previous frame image, and the motion state representation data between the two-dimensional facial key point mapping result corresponding to the target image and the two-dimensional facial key point detection result of the previous frame image.
  • the fine-tuning unit 403 is specifically configured to update other parameters in the 3D face reconstruction result except for the face identification parameters.
  • the target image refers to any frame image in a reference video; the facial identification parameters in the three-dimensional facial reconstruction result corresponding to the target image are determined based on the average value of the facial identification parameters in the three-dimensional facial parameter prediction results of at least two frames of images in the reference video, where the at least two frames of images include the target image.
  • the target image refers to any frame image in a reference video
  • the face reconstruction device 400 further includes:
  • a generation unit is configured to generate a video corresponding to an audio sequence based on a 3D facial reconstruction result corresponding to each frame image in the reference video, wherein the object presented in the video corresponding to the audio sequence is consistent with the object presented in the reference video, and the video corresponding to the audio sequence is used to describe the facial state changes of the object under the audio sequence.
  • the working principle of the facial reconstruction device 400 may include: after obtaining the target image, first, performing two-dimensional facial key point detection processing on the target image to obtain a two-dimensional facial key point detection result of the target image, so that the two-dimensional facial key point detection result can represent the facial state of the object in the target image in the two-dimensional space, and performing three-dimensional facial parameter prediction processing on the target image to obtain a three-dimensional facial parameter prediction result of the target image, so that the three-dimensional facial parameter prediction result can more accurately represent the facial state of the object in the three-dimensional space; then, based on the two-dimensional facial key point detection result, fine-tuning the three-dimensional facial parameter prediction result to obtain a three-dimensional facial reconstruction result corresponding to the target image, so that the three-dimensional facial reconstruction result can more accurately represent the facial state of the object in the three-dimensional space, which is conducive to improving the facial reconstruction effect.
  • the fine-tuning processing based on the three-dimensional facial parameter prediction result can converge quickly and stably, so that the three-dimensional facial reconstruction result obtained by the fine-tuning processing can more accurately represent the facial state of the object in the three-dimensional space, which is conducive to improving the facial reconstruction effect, such as accuracy, stability and efficiency.
  • an embodiment of the present application also provides an electronic device, which includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device performs any implementation of the face reconstruction method provided in the embodiment of the present application.
  • Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers.
  • mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers.
  • PDAs personal digital assistants
  • PADs tablet computers
  • PMPs portable multimedia players
  • in-vehicle terminals e.g., in-vehicle navigation terminals
  • fixed terminals such as digital TVs and desktop computers.
  • the electronic device shown in FIG5 is merely
  • electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503.
  • ROM read-only memory
  • RAM random access memory
  • Various programs and data required for the operation of electronic device 500 are also stored in RAM 503.
  • Processing device 501, ROM 502, and RAM 503 are connected to each other via a bus 504.
  • An input/output (I/O) interface 505 is also connected to bus 504.
  • the following devices may be connected to the I/O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509.
  • the communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data.
  • FIG5 shows the electronic device 500 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502.
  • the processing device 501 When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.
  • An embodiment of the present application further provides a computer-readable medium having instructions or a computer program stored therein.
  • the instructions or the computer program are executed on a device, the device executes any implementation of the face reconstruction method provided in the embodiment of the present application.
  • the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
  • a computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • the program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.
  • the client and server can communicate using any currently known or later developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network).
  • HTTP Hyper Text Transfer Protocol
  • Examples of communication networks include a local area network ("LAN”), a wide area network ("WAN”), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device can perform the method.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
  • each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram and/or flowchart, and the combination of the boxes in the block diagram and/or flowchart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit/module does not, in some cases, limit the unit itself.
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chip
  • CPLDs complex programmable logic devices
  • a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment.
  • a machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing.
  • a more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the foregoing.
  • At least one (item) means one or more, and “plurality” means two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, “A and/or B” can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character “/” generally indicates that the previous and next associated objects are in an “or” relationship. "At least one of the following items” or similar expressions refers to any combination of these items, including any combination of single items or plural items.
  • At least one of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c", where a, b, c can be single or multiple.
  • the steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two.
  • the software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Architecture (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开了一种脸部重建方法、装置、设备、介质、产品，该方法包括：在获取到目标图像之后，先对目标图像进行二维脸部关键点检测处理，得到目标图像的二维脸部关键点检测结果，以使该二维脸部关键点检测结果能够表示出目标图像中对象在二维空间内所处的脸部状态，并对目标图像进行三维脸部参数预测处理，得到目标图像的三维脸部参数预测结果，以使该三维脸部参数预测结果能够较准确地表示出该对象在三维空间内所处的脸部状态；然后，依据该二维脸部关键点检测结果，对该三维脸部参数预测结果进行微调处理，得到目标图像对应的三维脸部重建结果。

Description

一种脸部重建方法、装置、设备、介质、产品
本申请要求于2024年04月08日提交的，申请号为202410418018.7、发明名称为“一种脸部重建方法、装置、设备、介质、产品”的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据处理技术领域，尤其涉及一种脸部重建方法、装置、设备、介质、产品。
背景技术
对于一些应用场景，如视频中音频调整场景或者其他具有脸部重建需求的场景来说，这些应用场景可能存在以下需求：针对一个单图或者一个视频进行脸部重建，以便后续能够利用脸部重建结果完成某些任务，如三维脸部模型构建任务、二维视频转换成三维视频的任务或者视频中音频调整任务等。
发明内容
本申请提供了一种脸部重建方法、装置、设备、介质、产品，有利于提高脸部重建效果。
为了实现上述目的，本申请提供的技术方案如下：
本申请提供一种脸部重建方法，所述方法包括：
获取目标图像；
对所述目标图像进行二维脸部关键点检测处理，得到所述目标图像的二维脸部关键点检测结果，以及对所述目标图像进行三维脸部参数预测处理，得到所述目标图像的三维脸部参数预测结果；
依据所述二维脸部关键点检测结果，对所述三维脸部参数预测结果进行微调处理，得到所述目标图像对应的三维脸部重建结果。
在一种可能的实施方式下，所述微调处理，包括：
依据所述三维脸部参数预测结果，对所述三维脸部重建结果进行初始化处理；
依据所述三维脸部重建结果，构建所述目标图像对应的三维脸部模型；
将所述三维脸部模型映射至二维图像空间，得到所述目标图像对应的二维脸部关键点映射结果；
依据所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据，更新所述三维脸部重建结果。
在一种可能的实施方式下，所述依据所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据，更新所述三维脸部重建结果，包括：
依据所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据、以及所述三维脸部重建结果与所述三维脸部参数预测结果之间的差异表征数据，更新所述三维脸部重建结果。
在一种可能的实施方式下，所述三维脸部重建结果与所述三维脸部参数预测结果之间的差异表征数据对所述更新的约束强度，弱于，所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据对所述更新的约束强度。
在一种可能的实施方式下，所述目标图像是指参考视频中的一帧图像，而且所述参考视频包括所述目标图像的前一帧图像；
所述三维脸部重建结果是依据所述三维脸部重建结果对应的时序损失和/或所述二维脸部关键点映射结果对应的时序损失进行更新的；
所述三维脸部重建结果对应的时序损失是依据所述目标图像的三维脸部参数预测结果与所述前一帧图像的三维脸部参数信息之间的运动状态表征数据、以及所述目标图像对应的三维脸部重建结果与所述前一帧图像的三维脸部参数信息之间的运动状态表征数据所确定的；所述前一帧图像的三维脸部参数信息是依据所述前一帧图像的三维脸部参数预测结果和/或所述前一帧图像对应的三维脸部重建结果所确定的；
所述二维脸部关键点映射结果对应的时序损失是依据所述目标图像的二维脸部关键点检测结果与所述前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据、以及所述目标图像对应的二维脸部关键点映射结果与所述前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据所确定的。
在一种可能的实施方式下，所述更新所述三维脸部重建结果，包括：
更新所述三维脸部重建结果中除了脸部标识参数以外的其他参数。
在一种可能的实施方式下，所述目标图像是指参考视频中的任意一帧图像；
所述目标图像对应的三维脸部重建结果中脸部标识参数是依据所述参考视频中至少两帧图像的三维脸部参数预测结果中脸部标识参数的平均值所确定的，所述至少两帧图像包括所述目标图像。
在一种可能的实施方式下，所述目标图像是指参考视频中的任意一帧图像；
所述方法还包括：
依据所述参考视频中各帧图像对应的三维脸部重建结果，生成音频序列对应的视频，所述音频序列对应的视频中所呈现的对象与所述参考视频中所呈现的对象保持一致，所述音频序列对应的视频用于描述所述对象在所述音频序列下所处的脸部状态变化。
本申请提供了一种脸部重建装置，包括：
获取单元，用于获取目标图像；
处理单元，用于对所述目标图像进行二维脸部关键点检测处理，得到所述目标图像的二维脸部关键点检测结果，以及对所述目标图像进行三维脸部参数预测处理，得到所述目标图像的三维脸部参数预测结果；
微调单元，用于依据所述二维脸部关键点检测结果，对所述三维脸部参数预测结果进行微调处理，得到所述目标图像对应的三维脸部重建结果。
本申请提供了一种电子设备，所述设备包括：处理器和存储器；
所述存储器，用于存储指令或计算机程序；
所述处理器，用于执行所述存储器中的所述指令或计算机程序，以使得所述电子设备执行本申请提供的脸部重建方法。
本申请提供了一种计算机可读介质，所述计算机可读介质中存储有指令或计算机程序，当所述指令或计算机程序在设备上运行时，使得所述设备执行本申请提供的脸部重建方法。
本申请提供了一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行本申请提供的脸部重建方法的程序代码。
本申请提供的技术方案中，对于目标图像，如某个单图或者某个视频中第i帧图像来说，在获取到该目标图像之后，首先，对该目标图像进行二维脸部关键点检测处理，得到该目标图像的二维脸部关键点检测结果，以使该二维脸部关键点检测结果能够表示出该目标图像中对象在二维空间内所处的脸部状态，以及对该目标图像进行三维脸部参数预测处理，得到该目标图像的三维脸部参数预测结果，以使该三维脸部参数预测结果能够较准确地表示出该对象在三维空间内所处的脸部状态；然后，依据该二维脸部关键点检测结果，对该三维脸部参数预测结果进行微调处理，得到该目标图像对应的三维脸部重建结果。
附图说明
为了更清楚地说明本申请实施例或相关技术中的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为本申请实施例提供的一种脸部重建方法的流程图；
图2为本申请实施例提供的一种视频中音频调整任务的实现流程的示意图；
图3为本申请实施例提供的一种脸部重建流程的示意图；
图4为本申请实施例提供的一种脸部重建装置的结构示意图；
图5为本申请实施例提供的一种电子设备的结构示意图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
为了更好地理解本申请所提供的技术方案，下面先结合一些附图对本申请提供的脸部重建方法进行说明。如图1所示，本申请实施例提供的脸部重建方法，包括下文S1-S4。其中，该图1为本申请实施例提供的一种脸部重建方法的流程图。
S1：获取目标图像。
其中，目标图像是指需要进行脸部重建处理的图像，而且本申请不限定该目标图像的实施方式，为了便于理解，下面结合两种情况进行说明。
情况1，在一些应用场景，如单图处理场景下，上文目标图像可以是指单图处理任务所涉及的图像，如由用户指定的图像或者由其他任务所提供的图像等。需要说明的是，本申请不限定该单图处理任务，比如，该单图处理任务至少涉及针对单图的脸部重建处理。
情况2，在一些应用场景，如类似于视频中音频调整这类的视频处理场景下，上文目标图像可以是指参考视频中的任意一帧图像，如第i帧图像，i为正整数，i≤I，I为正整数，I表示该参考视频中的图像总帧数。其中，该参考视频是指在某个视频处理任务中所涉及的、需要进行脸部重建处理的视频，如图2或者图3所示的参考视频；而且本申请不限定该参考视频的实施方式，比如，该参考视频可以采用单人口播视频进行实施。另外，本申请不限定该视频处理任务的实施方式，比如，该视频处理任务至少涉及针对视频中部分或者全部图像的脸部重建处理。又如，该视频处理任务可以采用类似于视频翻译、视频口误校正、视频中部分音频替换这类任务进行实施。
可见，在一种可能的实施方式下，上文目标图像可以是指参考视频中的第i帧图像，i为正整数，i≤I，I为正整数，I表示该参考视频中的图像总帧数。
另外，本申请不限定上文目标图像的获取方式。
S2：对目标图像进行二维脸部关键点检测处理，得到该目标图像的二维脸部关键点检测结果。
其中，目标图像的二维脸部关键点检测结果用于描述该目标图像中对象在二维空间内所处的脸部状态，如表情状态等；而且本申请不限定该二维脸部关键点检测结果的实施方式，比如，其可以采用现有的或者未来出现的任意一种二维脸部关键点，如二维landmarks进行实施。需要说明的是，本申请不限定该对象的实施方式，比如，该对象可以采用动物或者虚拟形象进行实施。
另外，本申请不限定上文S2中的二维脸部关键点检测处理的实施方式，比如，其可以采用现有的或者未来出现的任意一种能够针对一个图像进行二维脸部关键点检测处理的方法，如借助预先构建的具有二维脸部关键点检测处理功能的机器学习模型进行实施。
S3：对目标图像进行三维脸部参数预测处理，得到该目标图像的三维脸部参数预测结果。
其中，目标图像的三维脸部参数预测结果用于描述该目标图像中对象在三维空间内所处的脸部状态，如表情状态等。
另外，本申请不限定上文目标图像的三维脸部参数预测结果的实施方式，比如，其可以包括脸部标识(Identity document，ID)参数、脸部表情参数以及脸部姿态参数。其中，该脸部标识参数用于描述该目标图像中对象所具有的脸部特点，如脸部轮廓、五官分布等特点，以使基于该脸部标识参数所构建的三维脸部模型能够表示出该对象在无表情以及无姿态情况下所处的脸部状态。该脸部表情参数用于描述该对象在该目标图像中所呈现的表情状态，以使基于该脸部表情参数所构建的三维脸部模型能够表示出该对象在无ID以及无姿态情况下所处的脸部状态；而且本申请不限定该脸部表情参数的实施方式，比如，该脸部表情参数可以采用blendshape系数进行实施。该脸部姿态参数用于描述该对象在该目标图像中所呈现的脸部姿态，如正脸、侧脸等姿态，以使基于该脸部姿态参数所构建的三维脸部模型能够表示出该对象在无ID以及无表情情况下所处的脸部状态；而且本申请不限定该脸部姿态参数的实施方式，比如，该脸部姿态参数可以包括旋转、平移、缩放等系数。可见，在一种可能的实施方式下，该目标图像的三维脸部参数预测结果可以采用三维脸部形变模型(3D Morphable Model，3DMM)进行实施。
此外，本申请不限定上文S3的实施方式，比如，在一些应用场景下，为了更好地提高脸部重建效果，该S3具体可以为：利用预先构建的三维脸部参数预测模型，对目标图像进行三维脸部参数预测处理，得到该目标图像的三维脸部参数预测结果。其中，该三维脸部参数预测模型用于针对该三维脸部参数预测模型的输入数据进行三维脸部参数预测处理；而且本申请不限定该三维脸部参数预测模型的实施方式，比如，其可以采用现有的或者未来出现的任意一种具有三维脸部参数预测功能的模型，如机器学习模型进行实施。
还有，本申请不限定上文S3的执行时间与上文S2的执行时间之间的关联关系，比如，前者早于后者。又如，前者晚于后者。还如，两者相同。
基于上文S3的相关内容可知，在一些应用场景，如视频处理场景下，对于参考视频中的第i帧图像来说，在获取到该第i帧图像之后，可以将该第i帧图像输入预先构建的三维脸部参数预测模型，以使该三维脸部参数预测模型能够针对该第i帧图像进行三维脸部参数预测处理，得到并输出该第i帧图像的三维脸部参数预测结果。其中，因该三维脸部参数预测模型具有较好的三维脸部参数预测性能，以使利用该三维脸部参数预测模型针对该第i帧图像预测所得的三维脸部参数预测结果，能够比较准确地表示出该第i帧图像中对象在三维空间内所呈现的脸部状态，以便后续能够以该三维脸部参数预测结果作为初始值更好地确定出针对该第i帧图像的三维脸部重建结果。i为正整数，i≤I，I为正整数，I表示该参考视频中的图像总帧数。
S4：依据目标图像的二维脸部关键点检测结果，对该目标图像的三维脸部参数预测结果进行微调处理，得到该目标图像对应的三维脸部重建结果。
其中，目标图像对应的三维脸部重建结果是指针对该目标图像的三维脸部参数预测结果的微调处理结果，以使该目标图像对应的三维脸部重建结果能够更准确地表示出该目标图像中对象在三维空间内所处的脸部状态。
另外，本申请不限定上文目标图像对应的三维脸部重建结果的实施方式，比如，该目标图像对应的三维脸部重建结果的实施方式类似于上文目标图像的三维脸部参数预测结果的实施方式。可见，在一种可能的实施方式下，该目标图像对应的三维脸部重建结果可以包括脸部标识参数、脸部表情参数以及脸部姿态参数。
此外，本申请不限定上文S4的实施方式，比如，在一些应用场景下，该S4具体可以为：将目标图像的二维脸部关键点检测结果以及该目标图像的三维脸部参数预测结果输入预先构建的参数微调模型，以使该参数微调模型能够依据目标图像的二维脸部关键点检测结果，对该目标图像的三维脸部参数预测结果进行微调处理，得到并输出该目标图像对应的三维脸部重建结果。其中，该参数微调模型是指预先构建的、具有三维脸部参数微调处理功能的模型，如某种机器学习模型等；而且本申请不限定该参数微调模型的实施方式。
此外，为了更好地提高重建效果，本申请还提供了上文S4的一种可能的实施方式，在该实施方式下，该S4具体可以包括下文步骤11-步骤14。
步骤11：依据目标图像的三维脸部参数预测结果，对该目标图像对应的三维脸部重建结果进行初始化处理。
需要说明的是，本申请不限定上文步骤11的实施方式，比如，其具体可以为：将目标图像的三维脸部参数预测结果，确定为该目标图像对应的三维脸部重建结果的初始值。
可见，在一种可能的实施方式下，当目标图像的三维脸部参数预测结果包括脸部标识参数、脸部表情参数以及脸部姿态参数时，上文步骤11具体可以为：利用该三维脸部参数预测结果中的脸部标识参数，对该目标图像对应的三维脸部重建结果中的脸部标识参数进行初始化处理，以使该三维脸部重建结果中的脸部标识参数的初始值与该三维脸部参数预测结果中的脸部标识参数保持一致；利用该三维脸部参数预测结果中的脸部表情参数，对该三维脸部重建结果中的脸部表情参数进行初始化处理，以使该三维脸部重建结果中的脸部表情参数的初始值与该三维脸部参数预测结果中的脸部表情参数保持一致；利用该三维脸部参数预测结果中的脸部姿态参数，对该三维脸部重建结果中的脸部姿态参数进行初始化处理，以使该三维脸部重建结果中的脸部姿态参数的初始值与该三维脸部参数预测结果中的脸部姿态参数保持一致。
步骤12：依据目标图像对应的三维脸部重建结果，构建该目标图像对应的三维脸部模型。
其中，目标图像对应的三维脸部模型是指依据该目标图像对应的三维脸部重建结果所构建的三维脸部模型，以使该模型能够呈现出该目标图像中对象在三维空间内所处的脸部状态。
另外，本申请不限定上文步骤12的实施方式，比如，当目标图像对应的三维脸部重建结果采用3DMM进行实施时，该步骤12可以采用现有的或者未来出现的任意一种能够基于3DMM构建三维脸部模型的方法进行实施。
步骤13：将目标图像对应的三维脸部模型映射至二维图像空间，得到该目标图像对应的二维脸部关键点映射结果。
其中，目标图像对应的二维脸部关键点映射结果是通过将该目标图像对应的三维脸部模型映射至二维图像空间所得到的，以使该二维脸部关键点映射结果能够表示出该三维脸部模型在二维空间内所处状态，从而使得该二维脸部关键点映射结果在一定程度上能够表示出该目标图像中对象在二维空间内所处的脸部状态。
另外，本申请不限定上文步骤13的实施方式，比如，其具体可以为：先将目标图像对应的三维脸部模型映射成二维图像；再对该二维图像进行二维脸部关键点检测处理，得到该目标图像对应的二维脸部关键点映射结果。需要说明的是，本申请不限定该二维图像的获取方式，比如，其可以采用现有的或者未来出现的任意一种能够将三维脸部模型映射回二维图像的方法进行实施。
步骤14：依据目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据，更新该目标图像对应的三维脸部重建结果，并返回继续执行上文步骤12及其后续步骤，直至达到预设停止条件。
其中，对于目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据来说，该差异表征数据用于表征该二维脸部关键点映射结果与该二维脸部关键点检测结果之间的差异性，以使该差异表征数据能够表示出当前轮下该目标图像对应的三维脸部重建结果在二维关键点的重投影下所呈现的误差，从而使得该差异表征数据能够在一定程度上表示出当前轮下目标图像对应的三维脸部重建结果的准确性，而且本申请不限定该差异表征数据的确定过程，比如，其可以采用现有的或者未来出现的能够衡量不同二维脸部关键点之间差异性的损失函数进行实施。
另外，本申请不限定上文步骤14的实施方式，比如，其具体可以为：依据目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据，更新该目标图像对应的三维脸部重建结果，以使更新后的三维脸部重建结果在二维关键点的重投影上所呈现的误差小于更新前的三维脸部重建结果在二维关键点的重投影上所呈现的误差，并基于该更新后的三维脸部重建结果继续执行上文步骤12及其后续步骤，以开启下一轮过程，如此迭代循环，直至达到预设停止条件。其中，该预设停止条件是指该迭代循环结束时所需达到的条件，如当前轮的三维脸部重建结果在二维关键点的重投影上所呈现的误差低于预设误差阈值等条件。可见，在一种可能的实施方式下，可以通过不断地减少该目标图像对应的三维脸部重建结果在二维关键点的重投影上所呈现的误差的方式，对该三维脸部重建结果进行迭代更新处理。
经研究发现，如果在更新过程中只考虑三维脸部重建结果在二维关键点的重投影上所呈现的误差，则可能会出现以下两种缺陷：①当目标图像的二维脸部关键点检测结果不准确时，会导致更新所依据的指导信息不准确，从而会导致该目标图像对应的三维脸部重建结果沿着错误的指导方向进行更新，进而影响了三维脸部重建效果。②当一味地减少在二维关键点的重投影上所呈现的误差时，可能会导致三维空间的脸部稳定性和连续性变差。
经研究还发现，对于上文目标图像的三维脸部参数预测结果来说，该三维脸部参数预测结果能够比较准确地表示出该目标图像中对象在三维空间内所处的脸部状态，故后续可以在距离该三维脸部参数预测结果不太远的变化范围内，确定该目标图像对应的三维脸部重建结果，如此有利于提高准确性以及效率。
基于上述两段所示的发现，为了更好地提高重建效果，本申请还提供了上文步骤14的一种可能的实施方式，在该实施方式下，该步骤14具体可以为：依据目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据、以及该目标图像对应的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间的差异表征数据，更新该目标图像对应的三维脸部重建结果，并返回继续执行上文步骤12及其后续步骤，直至达到预设停止条件。
另外，对于目标图像对应的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间的差异表征数据来说，该差异表征数据用于表征该三维脸部重建结果与该三维脸部参数预测结果之间的差异性，以使该差异表征数据能够表示出当前轮下该目标图像对应的三维脸部重建结果与该三维脸部参数预测结果之间的相对距离，从而使得该差异表征数据能够在一定程度上表示出当前轮下该目标图像对应的三维脸部重建结果相对于该三维脸部参数预测结果的变化剧烈程度，进而使得后续可以借助该差异表征数据，在更新时约束该三维脸部重建结果相对于该三维脸部参数预测结果的变化不剧烈，如此有利于提高重建准确性。
此外，为了更好地提高重建效果，上文两种差异表征数据可以满足以下约束：目标图像对应的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间的差异表征数据对更新的约束强度，弱于，该目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据对更新的约束强度，以便后续基于该约束的更新过程能够达到更好地优化效果。需要说明的是，本申请不限定该约束的实现方式，比如，其可以通过一个超参数来控制这两种差异表征数据对更新的约束强度。
基于上述三段内容可知，在一些应用场景下，对于当前轮下目标图像对应的三维脸部重建结果来说，该三维脸部重建结果的更新过程不仅参考上文“目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据”这一影响因素，还要参考上文“目标图像对应的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间的差异表征数据”这一影响因素，以使更新后的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间差距较小，并使得该更新后的三维脸部重建结果在二维关键点的重投影上所呈现的误差小于更新前的三维脸部重建结果在二维关键点的重投影上所呈现的误差，如此能够确保在不偏离该三维脸部参数预测结果太远的前提下沿着该二维脸部关键点检测结果的指导方向进行优化，从而有利于实现精确度和稳定性的平衡。
需要说明的是，本申请不限定上段中更新过程对应的预设停止条件的实施方式，比如，该预设停止条件具体可以包括：当前轮的三维脸部重建结果的损失低于预设损失阈值。又如，该预设停止条件具体可以包括：当前轮的三维脸部重建结果的损失的变化率低于预设变化率阈值。还如，该预设停止条件具体可以包括：三维脸部重建结果的更新次数达到预设次数阈值。其中，该损失用于表征该三维脸部重建结果的性能，如准确性+稳定性等；而且该损失是依据目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据、以及该目标图像对应的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间的差异表征数据所确定的。
实际上，在一些应用场景，如视频处理场景下，为了更好地提高重建效果，本申请还提供了上文目标图像对应的三维脸部重建结果的更新过程的一种可能的实施方式，在该实施方式下，当该目标图像是指参考视频中的一帧图像，而且该参考视频包括该目标图像的前一帧图像时，该目标图像对应的三维脸部重建结果可以是依据该三维脸部重建结果对应的时序损失和/或上文二维脸部关键点映射结果对应的时序损失进行更新的，以使该三维脸部重建结果的更新过程满足该参考视频中的时序约束。
对于上段所示的目标图像和前一帧图像来说，两者之间满足以下约束：该目标图像在参考视频中所处排列位置与该前一帧图像在参考视频中所处排列位置相邻，而且该目标图像在参考视频中所处排列位置比该前一帧图像在参考视频中所处排列位置靠后。
另外，为了避免因视频中出现的动作切换导致的干扰，可以将上文参考视频切分成多段，以使每个视频段均用于描述一个动作的变化情况，并使得不同视频段用于描述不同动作，以便后续能够在各视频段上进行时序约束。基于此，在一种可能的实施方式下，上文目标图像和前一帧图像之间可以满足以下约束：该目标图像与该前一帧图像来自于同一个视频段，该目标图像在该视频段中所处排列位置与该前一帧图像在该视频段中所处排列位置相邻，而且该目标图像在该视频段中所处排列位置比该前一帧图像在该视频段中所处排列位置靠后。
对于上文目标图像对应的三维脸部重建结果来说，该三维脸部重建结果对应的时序损失用于表示该三维脸部重建结果在时序约束方面所呈现的状态；而且该三维脸部重建结果对应的时序损失是依据该目标图像的三维脸部参数预测结果与前一帧图像的三维脸部参数信息之间的运动状态表征数据、以及该目标图像对应的三维脸部重建结果与前一帧图像的三维脸部参数信息之间的运动状态表征数据所确定的。其中，该前一帧图像的三维脸部参数信息用于描述该前一帧图像中对象在三维空间中所处的脸部状态；而且该前一帧图像的三维脸部参数信息是依据该前一帧图像的三维脸部参数预测结果和/或该前一帧图像对应的三维脸部重建结果所确定的，以使该前一帧图像的三维脸部参数信息可以包括该前一帧图像的三维脸部参数预测结果，和/或，该前一帧图像对应的三维脸部重建结果。其中，该前一帧图像的三维脸部参数预测结果是通过对该前一帧图像进行三维脸部参数预测处理所得到的；而且该前一帧图像的三维脸部参数预测结果的实施方式类似于上文目标图像的三维脸部参数预测结果的实施方式。该前一帧图像对应的三维脸部重建结果是依据该前一帧图像的二维脸部关键点检测结果，对该前一帧图像的三维脸部参数预测结果进行微调处理所得到的；而且该前一帧图像对应的三维脸部重建结果的实施方式类似于上文目标图像对应的三维脸部重建结果的实施方式。其中，该前一帧图像的二维脸部关键点检测结果是通过对该前一帧图像进行二维脸部关键点检测处理所得到的；而且该前一帧图像的二维脸部关键点检测结果的实施方式类似于上文目标图像的二维脸部关键点检测结果的实施方式。
另外，对于上文目标图像的三维脸部参数预测结果与前一帧图像的三维脸部参数信息之间的运动状态表征数据来说，该运动状态表征数据用于表示由该三维脸部参数预测结果所表征的对象相对于该前一帧图像中对象在三维空间内所处的运动状态，如速度、加速度等状态，以使该运动状态表征数据能够在一定程度上表示出该目标图像中对象相对于该前一帧图像中对象在三维空间内所处的运动状态，从而使得该运动状态表征数据在一定程度上能够表示出该目标图像对应的三维脸部重建结果所需满足的时序约束；而且本申请不限定该运动状态表征数据的实施方式，比如，其可以包括速度和/或加速度等。另外，本申请也不限定该运动状态表征数据的获取方式，比如，其可以采用现有的或者未来出现的任意一种能够衡量两帧数据之间的运动状态的方法进行实施。
此外，对于上文目标图像对应的三维脸部重建结果与前一帧图像的三维脸部参数信息之间的运动状态表征数据来说，该运动状态表征数据用于表示由该三维脸部重建结果所表征的对象相对于该前一帧图像中对象在三维空间内所处的运动状态，如速度、加速度等状态；而且本申请不限定该运动状态表征数据的实施方式，比如，其可以包括速度和/或加速度等。另外，本申请也不限定该运动状态表征数据的获取方式，比如，其可以采用现有的或者未来出现的任意一种能够衡量两帧数据之间的运动状态的方法进行实施。
还有，本申请不限定上文三维脸部重建结果对应的时序损失的确定方法，比如，其具体可以为：先计算上文“目标图像对应的三维脸部重建结果与前一帧图像的三维脸部参数信息之间的运动状态表征数据”与上文“目标图像的三维脸部参数预测结果与前一帧图像的三维脸部参数信息之间的运动状态表征数据”之间的差异表征数据，以使该差异表征数据能够表示出该三维脸部重建结果在时序约束方面所呈现的状态，如约束满足程度或者是否满足约束等状态；再依据该差异表征数据，确定该三维脸部重建结果对应的时序损失。
对于上文目标图像对应的二维脸部关键点映射结果来说，该二维脸部关键点映射结果对应的时序损失用于表示该二维脸部关键点映射结果在时序约束方面所呈现的状态；而且该二维脸部关键点映射结果对应的时序损失是依据该目标图像的二维脸部关键点检测结果与前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据、以及该目标图像对应的二维脸部关键点映射结果与前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据所确定的。其中，该前一帧图像的二维脸部关键点检测结果用于描述该前一帧图像中对象在二维空间中所处的脸部状态。
另外，对于上文目标图像的二维脸部关键点检测结果与前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据来说，该运动状态表征数据用于表示由该二维脸部关键点检测结果所表征的对象相对于该前一帧图像中对象在二维空间内所处的运动状态，如速度、加速度等状态，以使该运动状态表征数据能够在一定程度上表示出该目标图像中对象相对于该前一帧图像中对象在二维空间内所处的运动状态，从而使得该运动状态表征数据在一定程度上能够表示出该目标图像对应的二维脸部关键点映射结果所需满足的时序约束；而且本申请不限定该运动状态表征数据的实施方式，比如，其可以包括速度和/或加速度等。另外，本申请也不限定该运动状态表征数据的获取方式，比如，其可以采用现有的或者未来出现的任意一种能够衡量两帧数据之间的运动状态的方法进行实施。
此外，对于上文目标图像对应的二维脸部关键点映射结果与前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据来说，该运动状态表征数据用于表示由该二维脸部关键点映射结果所表征的对象相对于该前一帧图像中对象在二维空间内所处的运动状态，如速度、加速度等状态；而且本申请不限定该运动状态表征数据的实施方式，比如，其可以包括速度和/或加速度等。另外，本申请也不限定该运动状态表征数据的获取方式，比如，其可以采用现有的或者未来出现的任意一种能够衡量两帧数据之间的运动状态的方法进行实施。
还有，本申请不限定上文二维脸部关键点映射结果对应的时序损失的确定方法，比如，其具体可以为：先计算上文“目标图像对应的二维脸部关键点映射结果与前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据”与上文“目标图像的二维脸部关键点检测结果与前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据”之间的差异表征数据，以使该差异表征数据能够表示出该二维脸部关键点映射结果在时序约束方面所呈现的状态，如约束满足程度或者是否满足约束等状态；再依据该差异表征数据，确定该二维脸部关键点映射结果对应的时序损失。
基于上文时序约束的相关内容可知，在一种可能的实施方式下，上文步骤14具体可以为：依据目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据、该目标图像对应的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间的差异表征数据、该三维脸部重建结果对应的时序损失、以及该二维脸部关键点映射结果对应的时序损失，更新该目标图像对应的三维脸部重建结果，以使更新后的三维脸部重建结果在二维关键点的重投影上所呈现的误差小于更新前的三维脸部重建结果在二维关键点的重投影上所呈现的误差、并使得该更新后的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间差距较小、还使得该更新后的三维脸部重建结果在时序约束方面所呈现的损失更小，从而使得该更新后的三维脸部重建结果能够更好地表示出该目标图像中对象所处的脸部状态，以便后续能够基于该更新后的三维脸部重建结果返回继续执行上文步骤12及其后续步骤，如此迭代循环，直至达到预设停止条件。
需要说明的是，本申请不限定上段中预设停止条件的实施方式，比如，该预设停止条件具体可以包括：当前轮的三维脸部重建结果的损失低于预设损失阈值。又如，该预设停止条件具体可以包括：当前轮的三维脸部重建结果的损失的变化率低于预设变化率阈值。还如，该预设停止条件具体可以包括：三维脸部重建结果的更新次数达到预设次数阈值。其中，该损失可以是依据目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据、该目标图像对应的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间的差异表征数据、该三维脸部重建结果对应的时序损失、以及该二维脸部关键点映射结果对应的时序损失所确定的。
实际上，在一些应用场景，如脸部标识参数、脸部表情参数以及脸部姿态参数互相解耦场景下，为了更好地提高重建效果，本申请还提供了上文“更新该目标图像对应的三维脸部重建结果”这一步骤的实施方式，在该实施方式下，该步骤具体可以为：更新该三维脸部重建结果中除了脸部标识参数以外的其他参数，如脸部表情参数以及脸部姿态参数。可见，对于当前轮下目标图像对应的三维脸部重建结果来说，可以通过固定该三维脸部重建结果中的脸部标识参数，并对该三维脸部重建结果中脸部表情参数以及脸部姿态参数进行更新的方式，完成针对该三维脸部重建结果的更新处理，如此能够有效地避免更新过程对脸部标识参数所造成的干扰，从而有利于提高重建效果。
基于上文步骤11至步骤14的相关内容可知，在一些应用场景下，对于当前轮下目标图像对应的三维脸部重建结果来说，在获取到该三维脸部重建结果之后，可以先依据该三维脸部重建结果构建三维脸部模型；再将该三维脸部模型映射成该目标图像对应的二维脸部关键点映射结果；然后，依据该目标图像对应的二维脸部关键点映射结果与该目标图像的二维脸部关键点检测结果之间的差异表征数据、该目标图像对应的三维脸部重建结果与该目标图像的三维脸部参数预测结果之间的差异表征数据、该三维脸部重建结果对应的时序损失、以及该二维脸部关键点映射结果对应的时序损失，确定该三维脸部重建结果的损失，以使该损失能够表示出当前轮下目标图像对应的三维脸部重建结果的性能，如准确性+稳定性等；最后，在固定该三维脸部重建结果中的脸部标识参数的前提下，依据该三维脸部重建结果的损失，对该三维脸部重建结果中脸部表情参数以及脸部姿态参数进行更新，得到更新后的三维脸部重建结果，以使该更新后的三维脸部重建结果中的脸部标识参数与更新前的三维脸部重建结果中的脸部标识参数保持一致，并使得该更新后的三维脸部重建结果包括更新后的脸部表情参数以及更新后的脸部姿态参数，从而使得该更新后的三维脸部重建结果具有更好的性能，以便后续能够基于该更新后的三维脸部重建结果继续执行上文步骤12及其后续步骤，如此迭代循环，直至达到预设停止条件，如此使得最终获取到的三维脸部重建结果能够更准确地表示出该目标图像中对象在三维空间中所处的脸部状态，从而有利于提高脸部重建效果。其中，因该三维脸部重建结果的初始值是依据该目标图像的三维脸部参数预测结果所确定的，以使该三维脸部重建结果的初始值能够比较准确地表示出该目标图像中对象所处的脸部状态，从而使得后续能够基于该初始值快速地且稳定地达到收敛，如此有利于提高重建效果。
实际上，在一些应用场景，如脸部标识参数、脸部表情参数以及脸部姿态参数互相解耦场景下，为了更好地提高重建效果，本申请还提供了上文目标图像对应的三维脸部重建结果中脸部标识参数的一种获取方式，在该方式下，当该目标图像是指参考视频中的任意一帧图像时，该目标图像对应的三维脸部重建结果中脸部标识参数可以是依据该参考视频中至少两帧图像的三维脸部参数预测结果中脸部标识参数的平均值所确定的。其中，该至少两帧图像包括该目标图像；而且本申请不限定该至少两帧图像的实施方式，比如，该至少两帧图像可以是指该参考视频中所有图像。又如，在将该参考视频切分成多个视频段之后，该至少两帧图像可以是指包括该目标图像的视频段中所有图像，以使该至少两帧图像能够更好地描述出该目标图像中对象所具有的脸部特点。
可见，在一种可能的实施方式下，当上文目标图像是指参考视频中的任意一帧图像，如第i帧图像时，该目标图像对应的三维脸部重建结果中脸部标识参数的获取过程可以包括：先获取该参考视频中各帧图像的三维脸部参数预测结果；再计算该参考视频中所有图像的三维脸部参数预测结果中脸部标识参数的平均值，作为该目标图像对应的三维脸部重建结果中脸部标识参数，以使该目标图像对应的三维脸部重建结果能够更准确地表示出该目标图像中对象的脸部特点，如脸部轮廓、五官分布等特点，如此有利于提高脸部重建效果。其中，i为正整数，i≤I，I为正整数，I表示该参考视频中的图像总帧数。
基于上文S1至S4的相关内容可知，对于本申请实施例提供的脸部重建方法来说，在获取到目标图像之后，首先，对该目标图像进行二维脸部关键点检测处理，得到该目标图像的二维脸部关键点检测结果，以使该二维脸部关键点检测结果能够表示出该目标图像中对象在二维空间内所处的脸部状态，以及对该目标图像进行三维脸部参数预测处理，得到该目标图像的三维脸部参数预测结果，以使该三维脸部参数预测结果能够较准确地表示出该对象在三维空间内所处的脸部状态；然后，依据该二维脸部关键点检测结果，对该三维脸部参数预测结果进行微调处理，得到该目标图像对应的三维脸部重建结果，以使该三维脸部重建结果能够更准确地表示出该对象在三维空间内所处的脸部状态，如此有利于提高脸部重建效果。其中，因该三维脸部参数预测结果能够较准确地表示出该对象在三维空间内所处的脸部状态，以使基于该三维脸部参数预测结果的微调处理能够快速且稳定地达到收敛，从而使得通过微调处理所得到的三维脸部重建结果能够更准确地表示出该对象在三维空间内所处的脸部状态，如此有利于提高脸部重建效果，如准确性、稳定性以及效率等。
另外，本申请不限定本申请实施例提供的脸部重建方法的执行主体，例如，本申请实施例提供的脸部重建方法可以应用于终端设备或者服务器。又如，本申请实施例提供的脸部重建方法也可以借助终端设备与服务器之间的数据交互过程进行实现。其中，该终端设备可以为智能手机、计算机、个人数字助理(Personal Digital Assitant，PDA)、平板电脑等。服务器可以为独立服务器、集群服务器或云服务器。
此外，本申请不限定本申请实施例提供的脸部重建方法的应用场景，为了便于理解，下面结合示例进行说明。
作为示例，当本申请提供的脸部重建方法应用于某种视频处理任务，如视频中音频调整任务时，该脸部重建方法可以包括下文步骤21-步骤25。
步骤21：获取目标图像，该目标图像是指参考视频中的任意一帧图像。
本申请中，在一些应用场景，如某种视频处理场景下，在获取到参考视频之后，可以将该参考视频中第i帧图像视为目标图像，以便后续能够借助该目标图像的相关处理过程，确定该第i帧图像对应的三维脸部重建结果。其中，i为正整数，i≤I，I为正整数，I表示该参考视频中的图像总帧数。
另外，步骤21的相关内容请参见上文S1的相关内容。
步骤22：对目标图像进行二维脸部关键点检测处理，得到该目标图像的二维脸部关键点检测结果。
需要说明的是，步骤22的相关内容请参见上文S2的相关内容。
步骤23：对目标图像进行三维脸部参数预测处理，得到该目标图像的三维脸部参数预测结果。
需要说明的是，步骤23的相关内容请参见上文S3的相关内容。
步骤24：依据目标图像的二维脸部关键点检测结果，对该目标图像的三维脸部参数预测结果进行微调处理，得到该目标图像对应的三维脸部重建结果。
需要说明的是，步骤24的相关内容请参见上文S4的相关内容。
步骤25：依据参考视频中各帧图像对应的三维脸部重建结果，生成音频序列对应的视频，该音频序列对应的视频中所呈现的对象与该参考视频中所呈现的对象保持一致，该音频序列对应的视频用于描述该对象在音频序列下所处的脸部状态变化。
其中，音频序列是指在对参考视频进行处理，如口型调整处理时所需参考的音频；而且本申请不限定该音频序列的实施方式，比如，在一些应用场景下，该音频序列满足以下约束：该音频序列中的总帧数与该参考视频中的总帧数保持一致。
另外，对于上文音频序列来说，该音频序列对应的视频是指针对该音频序列所生成的视频，以使该视频满足以下约束：该音频序列对应的视频中所呈现的对象与该参考视频中所呈现的对象保持一致，而且该音频序列对应的视频用于描述该对象在音频序列下所处的脸部状态变化。
此外，本申请不限定上文步骤25的实施方式，比如，其可以借助现有的或者未来出现的任意一种能够基于一些三维脸部重建结果以及音频序列生成视频的方法进行实施。又如，在一些应用场景下，为了更好地提高视频生成效果，本申请还提供了该步骤25的一种可能的实施方式，在该实施方式下，该步骤25具体可以包括：由预先构建的视频生成模型依据参考视频中所有图像对应的三维脸部重建结果，生成音频序列对应的视频，以使该音频序列对应的视频中所呈现的对象与该参考视频中所呈现的对象保持一致，并使得该音频序列对应的视频用于描述该对象在音频序列下所处的脸部状态变化。其中，该视频生成模型是指预先构建的、具有视频生成功能的模型，如机器学习模型等；而且本申请不限定该视频生成模型的实施方式。
基于上文步骤21至步骤25的相关内容可知，在一些应用场景，如视频处理场景下，在获取到参考视频之后，首先，对该参考视频中第i帧图像进行二维脸部关键点检测处理，得到该目标图像的二维脸部关键点检测结果，以及对该第i帧图像进行三维脸部参数预测处理，得到该目标图像的三维脸部参数预测结果，i为正整数，i≤I，I为正整数，I表示该参考视频中的图像总帧数；然后，依据该第i帧图像的二维脸部关键点检测结果，对该第i帧图像的三维脸部参数预测结果进行微调处理，得到该第i帧图像对应的三维脸部重建结果，i为正整数，i≤I，I为正整数，I表示该参考视频中的图像总帧数；最后，依据该参考视频中所有图像对应的三维脸部重建结果，生成音频序列对应的视频，以使该音频序列对应的视频中所呈现的对象与该参考视频中所呈现的对象保持一致，并使得该音频序列对应的视频用于描述该对象在音频序列下所处的脸部状态变化，如此能够实现基于音频序列调整该参考视频中对象的脸部状态变化。其中，因这些三维脸部重建结果能够准确地表示出该参考视频中对象的脸部状态，如脸部特点、类似于表情幅度比较大这类在说话时所呈现的表情特点等，以使基于这些三维脸部重建结果所生成的视频能够更好地表示出该对象在音频序列下所处的脸部状态变化，如此有利于提高视频生成效果。
基于本申请实施例提供的脸部重建方法，本申请实施例还提供了一种脸部重建装置，下面结合图4进行解释和说明。其中，图4为本申请实施例提供的一种脸部重建装置的结构示意图。需要说明的是，本申请实施例提供的脸部重建装置的技术详情，请参照上文脸部重建方法的相关内容。
如图4所示，本申请实施例提供的脸部重建装置400，包括：
获取单元401，用于获取目标图像；
处理单元402，用于对所述目标图像进行二维脸部关键点检测处理，得到所述目标图像的二维脸部关键点检测结果，以及对所述目标图像进行三维脸部参数预测处理，得到所述目标图像的三维脸部参数预测结果；
微调单元403，用于依据所述二维脸部关键点检测结果，对所述三维脸部参数预测结果进行微调处理，得到所述目标图像对应的三维脸部重建结果。
在一种可能的实施方式下，所述微调单元403，具体用于：依据所述三维脸部参数预测结果，对所述三维脸部重建结果进行初始化处理；依据所述三维脸部重建结果，构建所述目标图像对应的三维脸部模型；将所述三维脸部模型映射至二维图像空间，得到所述目标图像对应的二维脸部关键点映射结果；依据所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据，更新所述三维脸部重建结果。
在一种可能的实施方式下，所述微调单元403，具体用于：依据所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据、以及所述三维脸部重建结果与所述三维脸部参数预测结果之间的差异表征数据，更新所述三维脸部重建结果。
在一种可能的实施方式下，所述三维脸部重建结果与所述三维脸部参数预测结果之间的差异表征数据对所述更新的约束强度，弱于，所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据对所述更新的约束强度。
在一种可能的实施方式下，所述目标图像是指参考视频中的一帧图像，而且所述参考视频包括所述目标图像的前一帧图像；
所述三维脸部重建结果是依据所述三维脸部重建结果对应的时序损失和/或所述二维脸部关键点映射结果对应的时序损失进行更新的；
所述三维脸部重建结果对应的时序损失是依据所述目标图像的三维脸部参数预测结果与所述前一帧图像的三维脸部参数信息之间的运动状态表征数据、以及所述目标图像对应的三维脸部重建结果与所述前一帧图像的三维脸部参数信息之间的运动状态表征数据所确定的；所述前一帧图像的三维脸部参数信息是依据所述前一帧图像的三维脸部参数预测结果和/或所述前一帧图像对应的三维脸部重建结果所确定的；
所述二维脸部关键点映射结果对应的时序损失是依据所述目标图像的二维脸部关键点检测结果与所述前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据、以及所述目标图像对应的二维脸部关键点映射结果与所述前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据所确定的。
在一种可能的实施方式下，所述微调单元403，具体用于：更新所述三维脸部重建结果中除了脸部标识参数以外的其他参数。
在一种可能的实施方式下，所述目标图像是指参考视频中的任意一帧图像；所述目标图像对应的三维脸部重建结果中脸部标识参数是依据所述参考视频中至少两帧图像的三维脸部参数预测结果中脸部标识参数的平均值所确定的，所述至少两帧图像包括所述目标图像。
在一种可能的实施方式下，所述目标图像是指参考视频中的任意一帧图像；
所述脸部重建装置400还包括：
生成单元，用于依据所述参考视频中各帧图像对应的三维脸部重建结果，生成音频序列对应的视频，所述音频序列对应的视频中所呈现的对象与所述参考视频中所呈现的对象保持一致，所述音频序列对应的视频用于描述所述对象在所述音频序列下所处的脸部状态变化。
基于上述脸部重建装置400的相关内容可知，本申请提供的脸部重建装置400的工作原理可以包括：在获取到目标图像之后，首先，对该目标图像进行二维脸部关键点检测处理，得到该目标图像的二维脸部关键点检测结果，以使该二维脸部关键点检测结果能够表示出该目标图像中对象在二维空间内所处的脸部状态，以及对该目标图像进行三维脸部参数预测处理，得到该目标图像的三维脸部参数预测结果，以使该三维脸部参数预测结果能够较准确地表示出该对象在三维空间内所处的脸部状态；然后，依据该二维脸部关键点检测结果，对该三维脸部参数预测结果进行微调处理，得到该目标图像对应的三维脸部重建结果，以使该三维脸部重建结果能够更准确地表示出该对象在三维空间内所处的脸部状态，如此有利于提高脸部重建效果。其中，因该三维脸部参数预测结果能够较准确地表示出该对象在三维空间内所处的脸部状态，以使基于该三维脸部参数预测结果的微调处理能够快速且稳定地达到收敛，从而使得通过微调处理所得到的三维脸部重建结果能够更准确地表示出该对象在三维空间内所处的脸部状态，如此有利于提高脸部重建效果，如准确性、稳定性以及效率等。
另外，本申请实施例还提供了一种电子设备，所述设备包括处理器以及存储器：所述存储器，用于存储指令或计算机程序；所述处理器，用于执行所述存储器中的所述指令或计算机程序，以使得所述电子设备执行本申请实施例提供的脸部重建方法的任一实施方式。
参见图5，其示出了适于用来实现本公开实施例的电子设备500的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图5示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图5所示，电子设备500可以包括处理装置(例如中央处理器、图形处理器等)501，其可以根据存储在只读存储器(ROM)502中的程序或者从存储装置508加载到随机访问存储器(RAM)503中的程序而执行各种适当的动作和处理。在RAM503中，还存储有电子设备500操作所需的各种程序和数据。处理装置501、ROM 502以及RAM 503通过总线504彼此相连。输入/输出(I/O)接口505也连接至总线504。
通常，以下装置可以连接至I/O接口505：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置506；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置507；包括例如磁带、硬盘等的存储装置508；以及通信装置509。通信装置509可以允许电子设备500与其他设备进行无线或有线通信以交换数据。虽然图5示出了具有各种装置的电子设备500，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置509从网络上被下载和安装，或者从存储装置508被安装，或者从ROM502被安装。在该计算机程序被处理装置501执行时，执行本公开实施例的方法中限定的上述功能。
本公开实施例提供的电子设备与上述实施例提供的方法属于同一发明构思，未在本实施例中详尽描述的技术细节可参见上述实施例，并且本实施例与上述实施例具有相同的有益效果。
本申请实施例还提供了一种计算机可读介质，所述计算机可读介质中存储有指令或计算机程序，当所述指令或计算机程序在设备上运行时，使得所述设备执行本申请实施例提供的脸部重建方法的任一实施方式。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(Hyper Text Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备可以执行上述方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元/模块的名称在某种情况下并不构成对该单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
需要说明的是，本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的系统或装置而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。
应当理解，在本申请中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。
还需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本申请。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本申请的精神或范围的情况下，在其它实施例中实现。因此，本申请将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。

Claims (12)

  1. 一种脸部重建方法，其中，所述方法包括：
    获取目标图像；
    对所述目标图像进行二维脸部关键点检测处理，得到所述目标图像的二维脸部关键点检测结果，以及对所述目标图像进行三维脸部参数预测处理，得到所述目标图像的三维脸部参数预测结果；
    依据所述二维脸部关键点检测结果，对所述三维脸部参数预测结果进行微调处理，得到所述目标图像对应的三维脸部重建结果。
  2. 根据权利要求1所述的方法，其中，所述微调处理，包括：
    依据所述三维脸部参数预测结果，对所述三维脸部重建结果进行初始化处理；
    依据所述三维脸部重建结果，构建所述目标图像对应的三维脸部模型；
    将所述三维脸部模型映射至二维图像空间，得到所述目标图像对应的二维脸部关键点映射结果；
    依据所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据，更新所述三维脸部重建结果。
  3. 根据权利要求2所述的方法，其中，所述依据所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据，更新所述三维脸部重建结果，包括：
    依据所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据、以及所述三维脸部重建结果与所述三维脸部参数预测结果之间的差异表征数据，更新所述三维脸部重建结果。
  4. 根据权利要求3所述的方法，其中，所述三维脸部重建结果与所述三维脸部参数预测结果之间的差异表征数据对所述更新的约束强度，弱于，所述二维脸部关键点映射结果与所述二维脸部关键点检测结果之间的差异表征数据对所述更新的约束强度。
  5. 根据权利要求2所述的方法，其中，所述目标图像是指参考视频中的一帧图像，而且所述参考视频包括所述目标图像的前一帧图像；
    所述三维脸部重建结果是依据所述三维脸部重建结果对应的时序损失和/或所述二维脸部关键点映射结果对应的时序损失进行更新的；
    所述三维脸部重建结果对应的时序损失是依据所述目标图像的三维脸部参数预测结果与所述前一帧图像的三维脸部参数信息之间的运动状态表征数据、以及所述目标图像对应的三维脸部重建结果与所述前一帧图像的三维脸部参数信息之间的运动状态表征数据所确定的；所述前一帧图像的三维脸部参数信息是依据所述前一帧图像的三维脸部参数预测结果和/或所述前一帧图像对应的三维脸部重建结果所确定的；
    所述二维脸部关键点映射结果对应的时序损失是依据所述目标图像的二维脸部关键点检测结果与所述前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据、以及所述目标图像对应的二维脸部关键点映射结果与所述前一帧图像的二维脸部关键点检测结果之间的运动状态表征数据所确定的。
  6. 根据权利要求2-5任一项所述的方法，其中，所述更新所述三维脸部重建结果，包括：
    更新所述三维脸部重建结果中除了脸部标识参数以外的其他参数。
  7. 根据权利要求1所述的方法，其中，所述目标图像是指参考视频中的任意一帧图像；
    所述目标图像对应的三维脸部重建结果中脸部标识参数是依据所述参考视频中至少两帧图像的三维脸部参数预测结果中脸部标识参数的平均值所确定的，所述至少两帧图像包括所述目标图像。
  8. 根据权利要求1所述的方法，其中，所述目标图像是指参考视频中的任意一帧图像；
    所述方法还包括：
    依据所述参考视频中各帧图像对应的三维脸部重建结果，生成音频序列对应的视频，所述音频序列对应的视频中所呈现的对象与所述参考视频中所呈现的对象保持一致，所述音频序列对应的视频用于描述所述对象在所述音频序列下所处的脸部状态变化。
  9. 一种脸部重建装置，其中，包括：
    获取单元，用于获取目标图像；
    处理单元，用于对所述目标图像进行二维脸部关键点检测处理，得到所述目标图像的二维脸部关键点检测结果，以及对所述目标图像进行三维脸部参数预测处理，得到所述目标图像的三维脸部参数预测结果；
    微调单元，用于依据所述二维脸部关键点检测结果，对所述三维脸部参数预测结果进行微调处理，得到所述目标图像对应的三维脸部重建结果。
  10. 一种电子设备，其中，所述设备包括：处理器和存储器；
    所述存储器，用于存储指令或计算机程序；
    所述处理器，用于执行所述存储器中的所述指令或计算机程序，以使得所述电子设备执行权利要求1-8任一项所述的方法。
  11. 一种计算机可读介质，其中，所述计算机可读介质中存储有指令或计算机程序，当所述指令或计算机程序在设备上运行时，使得所述设备执行权利要求1-8任一项所述的方法。
  12. 一种计算机程序产品，其中，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行权利要求1-8任一项所述的方法的程序代码。
PCT/CN2024/139769 2024-04-08 2024-12-16 一种脸部重建方法、装置、设备、介质、产品 Pending WO2025213835A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410418018.7 2024-04-08
CN202410418018.7A CN120782943A (zh) 2024-04-08 2024-04-08 一种脸部重建方法、装置、设备、介质、产品

Publications (1)

Publication Number Publication Date
WO2025213835A1 true WO2025213835A1 (zh) 2025-10-16

Family

ID=97281941

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/139769 Pending WO2025213835A1 (zh) 2024-04-08 2024-12-16 一种脸部重建方法、装置、设备、介质、产品

Country Status (2)

Country Link
CN (1) CN120782943A (zh)
WO (1) WO2025213835A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109657583A (zh) * 2018-12-10 2019-04-19 腾讯科技（深圳）有限公司 脸部关键点检测方法、装置、计算机设备和存储介质
CN110533777A (zh) * 2019-08-01 2019-12-03 北京达佳互联信息技术有限公司 三维人脸图像修正方法、装置、电子设备和存储介质
CN111223175A (zh) * 2018-11-27 2020-06-02 财团法人交大思源基金会 三维人脸重建方法
WO2021238126A1 (zh) * 2020-05-29 2021-12-02 北京达佳互联信息技术有限公司 三维人脸重建方法及装置
CN114067059A (zh) * 2021-11-29 2022-02-18 广州虎牙科技有限公司 三维人脸重建方法、电子设备及计算机可读存储介质
WO2022143354A1 (zh) * 2020-12-29 2022-07-07 百果园技术(新加坡)有限公司 虚拟对象的脸部生成方法、装置、设备和可读存储介质
CN117593493A (zh) * 2023-09-27 2024-02-23 书行科技(北京)有限公司 三维脸部拟合方法、装置、电子设备及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111223175A (zh) * 2018-11-27 2020-06-02 财团法人交大思源基金会 三维人脸重建方法
CN109657583A (zh) * 2018-12-10 2019-04-19 腾讯科技（深圳）有限公司 脸部关键点检测方法、装置、计算机设备和存储介质
CN110533777A (zh) * 2019-08-01 2019-12-03 北京达佳互联信息技术有限公司 三维人脸图像修正方法、装置、电子设备和存储介质
WO2021238126A1 (zh) * 2020-05-29 2021-12-02 北京达佳互联信息技术有限公司 三维人脸重建方法及装置
WO2022143354A1 (zh) * 2020-12-29 2022-07-07 百果园技术(新加坡)有限公司 虚拟对象的脸部生成方法、装置、设备和可读存储介质
CN114067059A (zh) * 2021-11-29 2022-02-18 广州虎牙科技有限公司 三维人脸重建方法、电子设备及计算机可读存储介质
CN117593493A (zh) * 2023-09-27 2024-02-23 书行科技(北京)有限公司 三维脸部拟合方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN120782943A (zh) 2025-10-14

Similar Documents

Publication Publication Date Title
WO2023138498A1 (zh) 生成风格化图像的方法、装置、电子设备及存储介质
CN110413812A (zh) 神经网络模型的训练方法、装置、电子设备及存储介质
CN112258622B (zh) 图像处理方法、装置、可读介质及电子设备
WO2024240222A1 (zh) 图像风格化处理方法、装置、设备、存储介质和程序产品
CN115439631A (zh) 一种三维扫描数据处理方法、装置、电子设备及存储介质
CN114964296A (zh) 车辆行驶路径规划方法、装置、设备和计算机可读介质
WO2024234998A1 (zh) 视频生成方法、装置、电子设备、计算机可读介质
WO2022033444A1 (zh) 动态流体效果处理方法、装置、电子设备和可读介质
CN111311712B (zh) 视频帧处理方法和装置
CN115086541A (zh) 一种拍摄位置确定方法、装置、设备及介质
WO2023284412A1 (zh) 图像处理方法、装置、电子设备及存储介质
WO2025167333A1 (zh) 一种图像生成方法、装置、设备、介质、产品
WO2025213835A1 (zh) 一种脸部重建方法、装置、设备、介质、产品
WO2025167320A1 (zh) 物体建模方法、装置、电子设备、存储介质和程序产品
WO2025200625A1 (zh) 一种视频生成方法、装置、设备、介质、产品
WO2025026428A1 (zh) 一种图像生成模型构建方法以及三维数字模型确定方法
WO2024174871A1 (zh) 一种图像处理方法、装置、设备及介质
WO2024152797A1 (zh) 视频补全方法、装置、介质和电子设备
WO2024169893A1 (zh) 模型构建方法、虚拟形象生成方法、装置、设备、介质
CN113587949A (zh) 导航路线推送方法和设备
CN113935139A (zh) 动态网络时钟同步仿真方法、系统、设备及介质
CN111968028A (zh) 图像生成方法、装置、设备和计算机可读介质
CN115879320B (zh) 网格模型生成方法、装置、电子设备及计算机可读存储介质
WO2025213838A1 (zh) 一种数据处理方法、装置、设备、介质、产品
WO2025213845A1 (zh) 一种视频生成方法、装置、设备、介质、产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24934898

Country of ref document: EP

Kind code of ref document: A1