WO2023130819A1 - 图像处理方法、装置、设备、存储介质及计算机程序 - Google Patents

图像处理方法、装置、设备、存储介质及计算机程序 Download PDF

Info

Publication number
WO2023130819A1
WO2023130819A1 PCT/CN2022/128637 CN2022128637W WO2023130819A1 WO 2023130819 A1 WO2023130819 A1 WO 2023130819A1 CN 2022128637 W CN2022128637 W CN 2022128637W WO 2023130819 A1 WO2023130819 A1 WO 2023130819A1
Authority
WO
WIPO (PCT)
Prior art keywords
description information
shape
information
target part
training
Prior art date
Application number
PCT/CN2022/128637
Other languages
English (en)
French (fr)
Inventor
周志强
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to US18/333,781 priority Critical patent/US20230368395A1/en
Publication of WO2023130819A1 publication Critical patent/WO2023130819A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/155—Segmentation; Edge detection involving morphological operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/60—Methods for processing data by generating or executing the game program
    • A63F2300/66—Methods for processing data by generating or executing the game program for rendering three dimensional images
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8082—Virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20036—Morphological image processing
    • G06T2207/20041—Distance transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2021—Shape modification

Definitions

  • This application relates to the field of computer technology, in particular to image processing technology.
  • 3D three-dimensional
  • virtual characters such as player characters and monster characters in the applications.
  • expressions for the 3D characters such as smiling expressions, sad expressions, etc.
  • the production of 3D character expressions is generally realized through facial controllers.
  • the controller parameters of the facial controller bound to the face of the 3D character can be edited and adjusted according to the image including the face of the character, so that Create corresponding facial expressions on the 3D character.
  • Embodiments of the present application provide an image processing method, device, equipment, storage medium, and computer program, which improve the efficiency of shape correction for target parts of an object.
  • an embodiment of the present application provides an image processing method executed by a computer device, including:
  • the image to be processed includes a target part of the object, the shape of the target part is a first shape, and the first shape does not match the expected shape;
  • an image processing device including:
  • An acquisition unit configured to acquire an image to be processed, the image to be processed includes a target part of the object, the form of the target part is a first form, and the first form does not match the expected form;
  • the acquiring unit is further configured to acquire description information corresponding to the first form, and acquire outline information of the target part in the first form;
  • a correction unit configured to modify the shape of the target part based on the description information and the contour information, and modify the shape of the target part from the first shape to a second shape, and the second shape is the same as The expected morphology matches.
  • an embodiment of the present application provides a computer device, including: a processor, adapted to implement one or more computer programs; a computer storage medium, the computer storage medium stores one or more computer programs, and the one One or more computer programs are suitable for being loaded by a processor and executing the image processing method provided by the embodiment of the present application.
  • the embodiment of the present application provides a computer storage medium, the computer storage medium stores a computer program, and when the computer program is executed by the processor of the image processing device, it is used to perform the image processing provided by the embodiment of the present application method.
  • the embodiment of the present application provides a computer program product or computer program, the computer program product includes a computer program, the computer program is stored in a computer storage medium; the processor of the image processing device reads the computer program from the computer storage medium program, the processor executes the computer program, so that the image processing device executes the image processing method provided in the embodiment of the present application.
  • the target part of the object in the image to be processed is in the first form. If the first form does not match the expected form, the description information corresponding to the first form and the target part in the first form can be obtained.
  • the contour information of the target part further, based on the description information corresponding to the first form and the contour information of the target part, the shape correction is performed on the target part, so as to modify the shape of the target part from the first form to the second form, and the second form is the same as If the expected shape matches, in this way, there is no need to manually correct the first shape, but according to the description information of the first shape and the contour information of the target part in the first shape, the shape correction of the target part is automatically performed, so that the target part The shape is as expected. The consumption of human resources caused by manual correction is eliminated, and the efficiency of shape correction is improved.
  • Fig. 1a is a schematic diagram of the binding of a 3D character image and a shape controller provided by an embodiment of the present application;
  • Figure 1b is a schematic diagram of correcting lip shape provided by the embodiment of the present application.
  • FIG. 2 is a schematic flow diagram of an image processing method provided in an embodiment of the present application.
  • Fig. 3 is a schematic diagram of the association between an image to be processed and an image grid provided by an embodiment of the present application;
  • Fig. 4a is a schematic diagram of the outline and mesh vertices of a target site provided by the embodiment of the present application;
  • Fig. 4b is a schematic diagram of contour information of a target site provided by an embodiment of the present application.
  • Fig. 4c is a schematic diagram of outline information of another target site provided by the embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of a parameter prediction model provided by an embodiment of the present application.
  • Fig. 6 is a schematic flowchart of another image processing method provided by the embodiment of the present application.
  • Fig. 7a is a schematic diagram of training a parameter prediction model provided by an embodiment of the present application.
  • Fig. 7b is a schematic diagram of training a position prediction model provided by the embodiment of the present application.
  • Fig. 8 is a schematic diagram of training another parameter prediction model provided by the embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an image processing device provided in an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • the embodiment of the present application provides an image processing method. After the image to be processed is acquired, it is determined that the target part of the object in the image to be processed is the first form, and it is further detected whether the first form of the target part in the image to be processed is consistent with When the first form of the target part does not match the expected form, the target part can be morphologically corrected according to the contour information of the target part in the first form and the description information corresponding to the first form, so that The morphology of the target site is corrected from the first morphology to a second morphology matching the expected morphology.
  • the image processing method can be performed by an image processing device, and the image processing device can be a terminal, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, a smart home appliance, a smart voice Interactive devices, etc.; or, the image processing device may also be a server, such as an independent physical server, a server cluster or distributed system composed of multiple physical servers, a cloud server providing cloud computing services, and the like.
  • a terminal such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle terminal, a smart home appliance, a smart voice Interactive devices, etc.
  • the image processing device may also be a server, such as an independent physical server, a server cluster or distributed system composed of multiple physical servers, a cloud server providing cloud computing services, and the like.
  • the image processing method can be executed cooperatively by the image processing device and the image management device, for example, the image management device can store the image to be processed, and send the image to be processed to the image processing device, and the image processing device obtains the image to be processed Perform the above image processing method after taking the image; as another example, the image processing device can obtain the image to be processed from the local storage, and then obtain the description information corresponding to the first form of the target part in the image to be processed and the target part in the first form. Contour information, then, the image processing device transmits the description information and contour information to the image management device, and the image management device corrects the shape of the target part based on the description information and contour information, and finally feeds back the correction result to the image processing device.
  • the image management device can store the image to be processed, and send the image to be processed to the image processing device, and the image processing device obtains the image to be processed Perform the above image processing method after taking the image; as another example, the image processing device can obtain the image to be processed from the local storage, and
  • image processing device and image management device are both computer devices.
  • the target part of the object in the image to be processed can refer to any one or more of the facial features included in the face of the object, such as eyes, lips, nose, ears, eyebrows, etc.; or, the target part of the object in the image to be processed
  • a part may also refer to other parts of the subject's body, such as a finger, an arm, and the like.
  • the embodiment of the present application takes the lip as an example for introduction.
  • the image processing method provided by the embodiment of the present application can be applied in the scene of 3D character expression production.
  • the image to be processed refers to the 3D character image
  • the 3D character in the 3D character image is obtained by the following steps (1) and (2). Create an expression, and the form of the target part in the produced expression is the first form. specifically:
  • Fig. 1a is a schematic diagram of binding the 3D character image with the shape controller provided by the embodiment of the present application
  • 101 represents the 3D character image
  • Each part of the object's face in the 3D character image can be described by the vertices of the corresponding grid, for example, the lips can be described by the vertices of the grid in 103.
  • 102 represents a part control corresponding to the 3D character image in the form controller, for example, 1021 represents a part control bound to the lips of the 3D character in the form controller.
  • the controller parameters in the shape controller may include the parameters of each part control, and the parameters of each part control may determine the size information, shape information, etc. of the part bound to it.
  • the animator edits the parameters of the shape controller according to the image including the character's face, so as to create the corresponding expression.
  • the shape of the character's lips in the image is shown in 11 in Figure 1b, and the shape of the lips is Expected shape: edit the parameters of the shape controller to create the expression of the 3D character image.
  • the shape of the lips in the created expression is shown as 12 in Figure 1b, and the shape of the lips can be regarded as the first shape.
  • the image processing method provided by the embodiment of the present application can be used to correct the lip shape.
  • the description information corresponding to the current shape when the lip is in the current shape is obtained, and the description information can be used in the image grid.
  • the shape of the lips can be regarded as the second shape.
  • the shape corrected by the image processing method provided by the embodiment of the present application has a high degree of matching with the expected shape, and the image processing method of the present application does not require artificial correction, which can save correction time and improve correction efficiency.
  • the above-mentioned 3D character can be a virtual character in a game application. Before the game application is officially released, in order to achieve a more realistic game effect, some expressions can be pre-designed for the virtual character.
  • the virtual character can be Present different expressions, for example, in the game battle scene, the virtual character can show a more excited expression, after the game battle is won, the virtual character can show a smiling expression, and so on. Different expressions can be designed for the virtual character through the above steps (1) and (2). If the shape of the target part in the designed expression does not match the expected shape, the image processing method provided by this application is used to modify the shape of the target part. shape to make the expression of the avatar more accurate.
  • the above-mentioned 3D character can also be a virtual object in a scene of human-computer interaction based on virtual objects.
  • the scene of human-computer interaction based on virtual objects can be as follows: the virtual object is product customer service, and users can chat and interact with the virtual object Come to discuss or learn about related products.
  • several expressions can be pre-designed for the virtual object, and the virtual object can display different labels in different chat scenarios. For example, in the chat scene where the user praises the product, the virtual object can show a smiling expression; in the chat scene where the user expresses dissatisfaction with the product, the virtual object can show a sad expression.
  • an expression can be designed for the virtual object through the above steps (1) and (2). If the shape of the target part in the designed expression does not match the expected shape, the image processing method provided by this application is used to correct the shape of the target part.
  • the scene of human-computer interaction based on virtual objects can also be: the virtual object acts as a guide for using the application. When the user uses the application for the first time, the virtual object can instruct the user how to use the application through some physical operations. application.
  • the scene of human-computer interaction based on virtual objects can also be: the virtual object can be the virtual image of the user in a certain application program, and the user can use the virtual image to represent himself, and the virtual image can be used in the application program to communicate with the user. Friends can interact with each other, or use the virtual image in the application to participate in some activities that simulate real life, such as growing vegetables, holding meetings, making friends, and so on.
  • any application scenario involving object shape correction can use the image processing method proposed in the embodiment of the application to save correction time and improve The purpose of correcting efficiency.
  • FIG. 2 it is a schematic flowchart of an image processing method provided in the embodiment of the present application.
  • the image processing method shown in Figure 2 can be executed by a computer device, specifically, it can be executed by a processor of the computer device, and the computer device can specifically be the above image processing device or image management device; the image processing method shown in Figure 2 can include the following step:
  • Step S201 acquiring an image to be processed, the image to be processed includes a target part of an object, the shape of the target part is a first shape, and the first shape does not match the expected shape.
  • the image to be processed can be stored locally in the computer device, and at this time, the computer device can directly obtain the image to be processed from the local storage; or, the image to be processed can be generated by other devices and stored in other devices, at this time, the computer The device may receive the image to be processed sent by other devices.
  • the image to be processed can include the face of the subject, and the object can be any object with expression, such as a human or an animal.
  • the image to be processed can refer to the 3D character image in the production of 3D character expression, and the image to be processed can be synthesized by 3D technology, for example Referring to 101 in FIG. 1a may represent an image to be processed.
  • the target part of the object in the image to be processed may refer to any one of the facial features of the object, such as eyes, eyebrows, nose, lips, and ears.
  • the shape of the target part can be the first shape, and the first shape can be generated by the face controller.
  • the first form may include a state of opening the mouth, a state of smiling, or a state of laughing out loud, etc.
  • the first form of the target part in the image to be processed can be compared with the expected form to determine the matching between the first form and the expected form; if the first form does not match the expected form, it is necessary to Execute subsequent steps S202 and S203 to correct the shape of the target site; if the first shape matches the expected shape, step S202 and step S203 may not be executed.
  • the target part is the lips
  • the first form is the state of smiling
  • the expected form is the state of opening the mouth and laughing
  • the first form and the expected form do not match at this time; or, the first form and the expected form are both in the smiling state, But varying degrees of smiling can also cause the first form to not match the expected form.
  • comparing the first form of the target part in the image to be processed with the expected form, and determining the matching between the first form and the expected form may include: artificially comparing the first form with the expected form Visual comparison, according to the comparison result, feeding back the matching between the first form and the expected form to the computer equipment.
  • comparing the first form of the target part in the image to be processed with the expected form, and determining the matching between the first form and the expected form may include: using image similarity analysis technology to The target part in the first form is compared with the target part in the expected form to obtain the similarity between the target parts in the two forms; if the similarity comparison result is less than or equal to the similarity threshold, the first The morphology does not match the expected morphology; if the similarity result is greater than a similarity threshold, it is determined that the first morphology matches the expected morphology.
  • the expected form can be any form specified in advance, and the expected form can be the form of the target part of the subject's face in any picture or video.
  • the referenced image The shape in which the target part of the character is is pre-specified as the expected shape.
  • Step S202 acquiring description information corresponding to the first form, and acquiring contour information of the target part in the first form.
  • the image to be processed can be associated with an image grid.
  • the image grid can include M grid vertices.
  • the grid vertex refers to the points that form the grid.
  • the image grid includes multiple grids, and each grid can be composed of multiple vertex composition.
  • the target part corresponds to N grid vertices in the M grid vertices, and the position change of the N grid vertices (here, the position change of the N grid vertices can refer to any one or more grids in the N grid vertices Vertex change) will cause the target part to present different forms, so the position information of the N grid vertices can be used as the description information of the first form.
  • FIG. 3 is a schematic diagram of the association between an image to be processed and an image grid provided by the embodiment of the present application.
  • the image grid may include multiple grid vertices.
  • 301 and 302 both represent grids.
  • Vertices each part of the subject's face in the image to be processed may correspond to several mesh vertices, for example, the mesh vertices in the area 303 are the mesh vertices corresponding to the lips.
  • the target part is a lip
  • the description information when the lip is in the first form is the position information of multiple mesh vertices in the area 303 .
  • the target part may include an inner contour and an outer contour, the inner contour may correspond to L mesh vertices, and the outer contour may correspond to P mesh vertices.
  • the target part is a lip, as shown in FIG. 4a
  • the embodiment of the application provides a corresponding relationship between the contour of the target part and the vertices of the grid.
  • 401 represents the target part
  • the black dotted line in 402 represents the inner contour of the target part
  • the black solid line represents the outer contour of the target part
  • the solid circle represents The L grid vertices corresponding to the inner contour
  • the hollow circles represent the P grid vertices corresponding to the outer contour.
  • the contour information of the target part may include the first distance between the inner contour and the outer contour, specifically, the first distance between the inner contour and the outer contour may include a corresponding relationship between L grid vertices and P grid vertices
  • the distance between the grid vertices for example, see Figure 4b, the solid circles and hollow circles that are connected to each other in Figure 4b are a set of corresponding grid vertices, and the corresponding solid circles and hollow circles
  • the distance between is called the first distance between the inner contour and the outer contour, for example, the length of line segment A, the length of line segment B and the length of line segment C are all the first distance.
  • the contour information of the target part can also include the second distance between every two mesh vertices in the P mesh vertices corresponding to the outer contour, for example, referring to Fig. 4c, every two solid circles in Fig. 4c (in Fig. 4c).
  • the distance between the solid circles is the second distance, for example, the length of the line segment D, the length of the line segment E and the length of the line segment F are all the second distance.
  • Step S203 modifying the shape of the target part based on the description information and the contour information, changing the shape of the target part from the first shape to a second shape, and the second shape matches the expected shape.
  • the shape of the target part is corrected based on the description information and the contour information, and the shape of the target part is corrected from the first shape to the second shape, including: predicting a target parameter based on the description information and the contour information, calling the shape
  • the controller adjusts the description information based on the target parameter, and the adjusted description information is used to make the shape of the target part appear as the second shape.
  • predicting target parameters based on description information and profile information can be realized by calling a parameter prediction model, and the parameter prediction model can be obtained through pre-training.
  • the shape of the target part is corrected based on the description information and contour information, and the shape of the target part is corrected from the first shape to the second shape, including: splicing the description information and contour information; calling parameters
  • the prediction model performs parameter prediction based on the splicing processing results to obtain the target parameters of the shape controller; the shape controller is called to adjust the description information based on the target parameters, and the adjusted description information is used to make the shape of the target part present a second shape.
  • a dimensionality reduction matrix may be obtained first, and the dimensionality reduction matrix is used to perform dimensionality reduction processing on the description information, and then the dimensionality reduction processing description information and contour information are used for splicing processing.
  • the principal component analysis (PCA) algorithm can be used to reduce the dimensionality of the description information.
  • the parameter prediction model can be any deep network structure, see Figure 5,
  • Figure 5 is a schematic structural diagram of a parameter prediction model provided by the embodiment of the present application
  • the parameter prediction model shown in Figure 5 can include three fully connected layers , each fully connected layer can correspond to an activation function, and the activation function can be ReLU.
  • the target part of the subject's face in the image to be processed is in the first form. If the first form does not match the expected form, the description information corresponding to the first form can be obtained, and the target part in the first form can be obtained.
  • the outline information of the part further, based on the description information corresponding to the first form and the outline information of the target part, the form correction is performed on the target part, so as to modify the form of the target part from the first form to the second form, and the second form is the same as In this way, there is no need to manually correct the first form, and the target part can be automatically morphologically corrected according to the description information of the first form and the contour information of the target part in the first form, so that the shape of the target part satisfies Expected shape.
  • the consumption of human resources caused by manual correction is eliminated, and the efficiency and accuracy of shape correction are improved.
  • FIG. 6 is a schematic flowchart of another image processing method provided in the embodiment of the present application.
  • the image processing method shown in FIG. 6 may be executed by a computer device, specifically, by a processor of the computer device, and the computer device may specifically be the above image processing device or image management device.
  • the image processing method shown in Figure 6 may include the following steps:
  • Step S601 acquiring an image to be processed, the image to be processed includes a target part of an object, the shape of the target part is a first shape, and the first shape does not match the expected shape.
  • Step S602 acquiring description information corresponding to the first form, and acquiring outline information of the target part in the first form.
  • step S601 reference may be made to the relevant description in step S201 in the embodiment in FIG. 2 , which will not be repeated here.
  • step S602 reference may be made to the relevant description in step S202 in the embodiment of FIG. 2 , which will not be repeated here.
  • Step S603 splicing the description information and the outline information, and calling the parameter prediction model to predict the parameters based on the splicing processing results, and obtaining the target parameters of the morphological controller.
  • the parameter prediction model can be trained based on the first training sample, specifically, the following steps s1-s4 can be included:
  • the first training sample may include the first sample image and the label information corresponding to each first sample image, the first sample image includes the target part of the subject's face, the first The label information corresponding to this image may include first description information corresponding to when the target part in the first sample image is in the first training form, and second description information corresponding to when the target part in the first sample image is in the second training form .
  • the first sample image corresponds to the image grid
  • the facial features of the subject’s face in the first sample image correspond to several grid vertices.
  • the target part can be the same as the target part in step S601, and can be any of the facial features.
  • the first description information corresponding to the target part in the first sample image when it is in the first training form may be the first position information of a plurality of grid vertices corresponding to the target part in the image grid associated with the first sample image
  • the second description information corresponding to the target part in the second training form in the first sample image may be the second position information of a plurality of mesh vertices corresponding to the target part.
  • the second training modality matches the expected training modality.
  • the manner of determining the expected training form may be the same as the aforementioned manner of determining the expected form, both of which may refer to the form of the target part of the person in the image.
  • the number of first sample images can be X, and X is an integer greater than or equal to 1.
  • the first description information is expressed as M coarse
  • the second description information is expressed as M fine
  • the number of grid vertices corresponding to the target part is N1
  • the dimension of each grid vertex is 3 dimensions
  • the dimension of M coarse is (X, N1*3), where N1 is much smaller than X.
  • the first description information includes the first description information of each first sample image
  • the first description information of each first sample image can be viewed becomes a collection of position information of N1 grid vertices.
  • the first description information may include X rows, and each row includes first position information of N1 grid vertices in a first sample image.
  • the first position information of a grid vertex may be passed through a three-dimensional coordinate (x, y, z), then each line of the first description information can be expressed as (x 1 , y 1 , z 1 , x 2 , y 2 , z 3 , ..., x N1 , y N1 , z N1 ), That is, each row of the first description information is composed of position information of N1 grid vertices in a first sample image.
  • each row in the second description information is the second position information of the N1 grid vertices in the first sample image.
  • s2 Acquiring the training contour information of the target part in the first sample image when the target part is in the first training form, and splicing the training contour information and the first description information.
  • a first sample image is used as an example to expand the description here, and the same processing steps are used for other first sample images, and the target part in each first sample image can be obtained Training profile information while in the first training modality.
  • the contour information of the target part in the first sample image when the target part is in the first training form in the first sample image can be acquired in the same way as the contour information obtained in step S202 when the target part in the image to be processed is in the first form.
  • the training contour information of the target part please refer to the description in step S202, which will not be repeated here.
  • the dimensionality reduction processing can be performed on the first description information, and the reduced The dimensionally processed first description information and the training contour information are concatenated.
  • a PCA algorithm may be used to perform dimensionality reduction processing on the first description information.
  • the principle of dimensionality reduction processing by the PCA algorithm is to generate a dimensionality reduction matrix based on the first description information and the second description information; multiply the first description information and the dimensionality reduction matrix to obtain the dimensionality reduction first description information.
  • M coarse the first description information
  • D K the dimensionality reduction processing of the first descriptive information is represented by the following formula (1):
  • MD coarse represents the first description information after dimensionality reduction processing, assuming that the dimensionality of the dimensionality reduction matrix is (N1*3, K), and the dimensionality of the first description information is (X, N1*3), after dimensionality reduction processing, The dimensions of the first description information are reduced to (X, K), where K is an integer much smaller than N1*3.
  • generating a dimensionality reduction matrix based on the first description information and the second description information includes:
  • D represents the eigenvector matrix with dimensions (N1*3, N1*3)
  • V is the eigenvalue matrix
  • V is a non-negative rectangular diagonal matrix with dimensions (X*2, N1* 3)
  • the value on the diagonal of the eigenvalue matrix V is the eigenvalue corresponding to each eigenvector
  • U is a matrix orthogonal to D
  • the dimension is the same as D.
  • K eigenvectors are selected in order, and the matrix composed of these K eigenvectors is determined as a dimensionality reduction matrix, and the eigenvalues corresponding to the K eigenvectors are and are greater than or equal to the feature threshold.
  • the selected K eigenvectors must ensure that information greater than or equal to the feature threshold in the description information matrix is retained.
  • K is the first value that allows the following formula (4) to be established:
  • the feature threshold can be taken as 99%.
  • the matrix composed of the first K eigenvectors is determined as a dimensionality reduction matrix, and the dimensionality of the dimensionality reduction matrix is (N1*3, K), where K is much smaller than N1*3.
  • s3 Call the parameter prediction model to be trained to perform parameter prediction based on the splicing processing results, and obtain the training parameters of the shape controller.
  • the number of first sample images can be X
  • the number of first description information and the number of training contour information are both X
  • the splicing processing result here can be any first sample image
  • the corresponding first description information and training contour information are spliced; that is, the first description information and training contour information corresponding to each first sample image in the X first sample images are spliced, A splicing processing result will be obtained, then, for the first training sample, the number of splicing processing results is also X.
  • the X splicing results can be input into the parameter prediction model one by one.
  • the dimension of the first description information is expressed as (X, N1*3,)
  • the dimension of the first description information after dimension reduction processing is expressed as (X, K)
  • the dimension of the training profile information is (X, D).
  • the process of the parameter prediction model performing parameter prediction processing based on a splicing processing result can be represented by Fig. 7a.
  • the parameter prediction model can also be called (Meah2Params).
  • the dimension of the splicing processing result input into the parameter prediction model each time is (1, K+C), and the dimension of the training parameters output by the parameter prediction model is (1, H). H represents the number of parameters used to control the shape of the target part in the shape controller.
  • s4 Predict the description information based on the training parameters, obtain the corresponding first prediction description information when the target part is in the second training form, and train the parameter prediction model according to the first prediction description information and the second description information.
  • the parameter prediction model is trained according to the first prediction description information and the second description information, which may specifically include: calculating the first loss function based on the first prediction description information and the second description information; using the backpropagation algorithm, based on The first loss function updates network parameters in the parameter prediction model.
  • calculating the first loss function based on the first prediction description information and the second description information can be expressed by the following formula (5):
  • L loss represents the first loss function
  • MD predict represents the first prediction description information
  • MD fine represents the second description information
  • MeanSquaredError( ⁇ ) represents the mean square error calculation function
  • w t represents the network parameters before updating in the parameter prediction model
  • w t+1 is the updated network parameters
  • ⁇ is the learning rate, which is generally around 10e-3.
  • the network parameters are optimized iteratively until the first loss function no longer changes.
  • the description information prediction based on the training parameters is performed above to obtain the corresponding first prediction description information when the target part is in the second training form, which can be executed by calling the position prediction model.
  • the training parameters are used as the input of the location prediction model, and the location prediction model is called to predict the description information to obtain the first prediction description information. It can be seen that the role of the position prediction model is to obtain a description information based on the parameter prediction of the shape controller. In other words, the position prediction model can imitate the process of the shape controller controlling the description information based on parameters.
  • the position prediction model can be a pre-trained model, and the position prediction model can be trained based on the second training sample, the second training sample includes the second sample image, and the relationship between the third description information and the second training parameters of the shape controller
  • the corresponding relationship, the second sample image includes the target part of the subject's face, and the third description information is the corresponding description information when the target part in the second sample image is in the third form.
  • the corresponding relationship between the third description information in the second training samples and the second training parameters in the morphological controller may include multiple groups.
  • the third form of the target part in the second sample image can be a form that matches an expected training form, or a form that does not match an expected training form.
  • the shape of the target part then the third shape can be a shape automatically generated by the shape controller based on the expression of the character. This shape may have errors from the expected training shape and is not very accurate.
  • the third form may be a form after the form controller automatically generates a form based on the expression of the person in the reference image, and manually adjusts the automatically generated form, and the third form at this time matches the expected training form.
  • the expected training form here may be the same as or different from the aforementioned expected training form.
  • the second sample image may correspond to the image grid
  • the target part corresponds to some grid vertices in the image grid
  • the third description information when the target part is in the third form refers to the position information of the grid vertices corresponding to the target part.
  • the correspondence between the third description information and the second training parameter in the shape controller can be determined based on the shape controller and the third description position information, specifically, a second training parameter can be randomly generated in the shape controller, based on the The second training parameter modifies the shape of the target part in the second sample image.
  • the third description information corresponding to the shape of the target part is the third description information corresponding to the current second training parameter.
  • a set of third description information and the second training parameters are used as a training data pair, and the position prediction model is invoked to predict the description information based on the second training parameters in a training data pair to obtain the second prediction description information.
  • the third description information used here is processed by dimension reduction matrix.
  • the dimension of each description information is 3
  • Each descriptive information corresponds to N1 grid vertices, and the dimension of the third descriptive information is (X, N1*3); the third descriptive information is subjected to dimensionality reduction processing by using the matrix obtained above, and the third descriptive information after dimensionality reduction
  • the dimensions of the information are (X, K); assuming that the second training parameters include H parameters, then the dimensions of the second training parameters are (X, H).
  • FIG. 7b it is a schematic diagram of training a position prediction model provided by the embodiment of the present application.
  • the position prediction model can also be called (Params2Mesh, P2M), and each time the second training parameter in a training data pair is input to the P2M model , the dimension of the second training parameter input into the P2M model at this time is (1, H), and the P2M model predicts the description information based on the second training parameter, and outputs the second prediction description information, and the dimension of the second prediction description information is (1,K).
  • the position prediction model is trained based on the second prediction description information and the third description information. Specifically, based on the second prediction description information and the third description information, determine the second loss function corresponding to the position prediction model; by calculating the partial derivative of the second loss function with respect to the network parameters in the position prediction model, update the position prediction model Network parameters, so as to realize the training of the position prediction model. For example, determining the second loss function corresponding to the position prediction model based on the second prediction description information and the third description information can be expressed by the following formula (7):
  • L loss' represents the second loss function
  • M predict' represents the second prediction description information
  • M ground truth represents the third description information
  • w't +1 represents the updated network parameters in the location prediction model
  • w't represents the network parameters before updating in the location prediction model
  • the position prediction model can also be any deep network structure, as long as it can meet the requirements of input and output, and can be iteratively optimized through training, and finally input a controller parameter to obtain correct description information.
  • the position prediction model provided by the embodiment of the present application may have the same network structure as the parameter prediction model, see Figure 5; the difference between the position prediction model and the parameter prediction model is: the input data is different, the input data of the position prediction model is the The parameters of the controller, the input data of the parameter prediction model are description information; and, the output data is different, the output data of the position prediction model is description information, and the output data of the parameter prediction model is the parameter of the shape controller; and, each full The number of hidden units in the connection layer is different.
  • Step S604 calling the shape controller to adjust the description information based on the target parameters, and the adjusted description information is used to make the shape of the target part present a second shape, and the second shape matches the preset shape.
  • step S604 for some feasible implementation manners included in step S604, reference may be made to the relevant description of step S203 in the embodiment in FIG. 2 , which will not be repeated here.
  • the embodiment of the present application proposes a schematic diagram of training a parameter prediction model, as shown in FIG. 8 .
  • the target part refers to the lip
  • the label information corresponding to the first sample image includes the first description information corresponding to the lip when the first sample image is in the first training form
  • the first description information does not match the expected training form
  • the label information also includes the second lip corresponding to the second training form in the first sample image.
  • the second descriptive information matches the expected training shape, so the second descriptive information can be understood as the mesh vertex position of the accurate mouth shape.
  • the training contour information can refer to the contour information of the mouth.
  • the contour information of the mouth includes the distance between the inner contour and the outer contour, and the distance between every two mesh vertices in the outer contour. These are collectively referred to as the inner contour and the outer contour of the mouth. Outline distance feature.
  • the data obtained after splicing is input into the parameter prediction model M2P, and M2P performs parameter prediction based on the input data , output training parameters;
  • the training parameters are used as the input data of the position prediction model P2M, P2M predicts the description information based on the input training parameters, and obtains the first prediction description information, and the first prediction description information can be understood as the predicted mouth shape grid vertex position; further, according to the grid vertex position of the predicted mouth shape and the grid vertex position of the accurate mouth shape after dimensionality reduction, calculate the loss function of the parameter prediction model M2P, based on the loss function M2P model for training.
  • the first sample training is used to obtain an M2P model
  • the M2P model can predict accurate description information according to rough description information and contour information of the target part.
  • Use the trained M2P model to correct the shape of the target part to obtain the shape of the target part that matches the expected shape.
  • the trained M2P model can save the human resources consumed by manual correction and avoid the damage caused by manual correction. Error, thus improving the efficiency and accuracy of shape correction.
  • FIG. 9 it is a schematic structural diagram of an image processing device provided in the embodiment of the present application.
  • the image processing device shown in Figure 9 can run the following units:
  • An acquisition unit 901 configured to acquire an image to be processed, the image to be processed includes a target part of the subject's face, the form of the target part is a first form, and the first form does not match the expected form;
  • the acquiring unit 901 is further configured to acquire description information corresponding to the first form, and acquire outline information of the target part in the first form;
  • the correction unit 902 is configured to modify the shape of the target part based on the description information and the contour information, and modify the shape of the target part from the first shape to a second shape, and the second The morphology matches the expected morphology.
  • the target part is bound to a shape controller, and based on the description information and the contour information, the correction unit 902 corrects the shape of the target part, and converts the shape of the target part to When the form is changed from the first form to the second form, perform the following steps:
  • the shape controller to adjust the description information based on the target parameter, and the adjusted description information is used to make the shape of the target part present the second shape.
  • the correction unit 902 performs the following steps when splicing the description information and the outline information:
  • Splicing processing is performed on the descriptive information after dimensionality reduction processing and the outline information.
  • the image to be processed is associated with an image grid
  • the image grid includes M grid vertices
  • the target site corresponds to N grid vertices in the M grid vertices
  • M and N are both integers greater than 1, and N is less than or equal to M
  • the description information corresponding to the first form includes position information of the N grid vertices.
  • the target site includes an inner contour and an outer contour, the inner contour corresponds to L mesh vertices, the outer contour corresponds to P mesh vertices, and the sum of L and P is less than N;
  • the contour information of the target part includes a first distance between the inner contour and the outer contour, and a second distance between every two mesh vertices among the P grid vertices corresponding to the outer contour;
  • the first distance includes the distance between the L grid vertices and the corresponding grid vertices among the P grid vertices.
  • the image processing device described in FIG. 9 further includes a processing unit 903:
  • the acquiring unit 901 is further configured to acquire a first training sample, the first training sample includes a first sample image and label information corresponding to the first sample image; the first sample image includes a target face of an object part, the tag information includes first description information corresponding to when the target part in the first sample image is in the first training form, and the target part in the first sample image is in the second training form The corresponding second description information; the first training form does not match the expected training form, and the second training form matches the expected training form;
  • the obtaining unit 901 is further configured to obtain training contour information when the target part in the first sample image is in the first training form, and perform splicing processing on the training contour information and the first description information ;
  • the processing unit 903 is configured to call the parameter prediction model to be trained to perform parameter prediction based on the splicing processing result, and obtain the training parameters of the shape controller;
  • Predict description information based on the training parameters, obtain first prediction description information corresponding to when the target part is in the second training form, and calculate the description information according to the first prediction description information and the second description information.
  • the parameter prediction model is trained.
  • the processing unit 903 when the processing unit 903 predicts the description information based on the training parameters and obtains the first predicted description information corresponding to when the target part is in the second training form, it performs the following steps:
  • the training parameters are used as the input of the position prediction model, and the position prediction model is called to predict the description information to obtain the first prediction description information; the position prediction model is used to predict the description information according to the parameters of the shape controller.
  • the obtaining unit 901 is further configured to obtain a second training sample, the second training sample includes the second sample image, and the relationship between the third description information and the second training parameter of the shape controller Corresponding relationship, the second sample image includes a target part of the subject's face, and the third description information refers to the corresponding description information when the target part in the second sample image is in a third form;
  • the processing unit 902 is further configured to call the position prediction model to be trained, perform description information prediction based on the second training parameters, and obtain second prediction description information; based on the second prediction description information and the third description information to train the position prediction model.
  • the processing unit 903 performs the following steps when training the parameter prediction model according to the difference between the first prediction description information and the second description information:
  • the acquiring unit 901 is configured to perform the following steps when splicing the training contour information and the first description information:
  • Splicing processing is performed on the dimensionally reduced first description information and the training contour information.
  • the acquiring unit 901 is configured to perform the following steps when acquiring the dimensionality reduction matrix:
  • the eigenvector matrix includes a plurality of eigenvectors;
  • the eigenvalue matrix includes eigenvalues corresponding to each eigenvector matrix;
  • the eigenvector matrix is adjusted, and the multiple eigenvectors in the adjusted eigenvector matrix are arranged in descending order according to the corresponding eigenvalues;
  • K eigenvectors are selected in order, and the matrix formed by the selected K eigenvectors is determined as a dimensionality reduction matrix, and the K eigenvectors correspond to The sum of the eigenvalues of is greater than or equal to the feature threshold.
  • each step involved in the image processing method shown in FIG. 2 and FIG. 6 may be executed by each unit in the image processing apparatus shown in FIG. 9 .
  • step S201 and step S202 shown in FIG. 2 may be executed by the acquisition unit 901 in the image processing device shown in FIG. 9
  • step S203 may be executed by the correction unit 902 in the image processing device shown in FIG. 9.
  • Step S601 and step S602 can be performed by the acquisition unit 901 in the image processing device shown in FIG. 9
  • step S603 and step S604 can be performed by the processing unit 903 in the image processing device shown in FIG. 9 .
  • the various units in the image processing device shown in Fig. 9 can be respectively or all combined into one or several other units to form, or some (some) units can be further disassembled. Divided into a plurality of functionally smaller units, this can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • the above-mentioned units are divided based on logical functions. In practical applications, the functions of one unit may also be realized by multiple units, or the functions of multiple units may be realized by one unit. In other embodiments of the present application, the image processing-based device may also include other units. In practical applications, these functions may also be implemented with the assistance of other units, and may be implemented cooperatively by multiple units.
  • a general-purpose computing device such as a computer that includes processing elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM) and storage elements
  • processing elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM) and storage elements
  • CPU central processing unit
  • RAM random access storage medium
  • ROM read-only storage medium
  • Running a computer program (including program code) capable of executing the steps involved in the corresponding methods as shown in Figure 2 and Figure 9, to construct the image processing device as shown in Figure 9, and to realize the image processing of the embodiment of the present application method.
  • the computer program may be recorded in, for example, a computer-readable storage medium, loaded into the image processing device through the computer-readable storage medium, and executed therein.
  • the target part of the subject's face in the image to be processed is in the first form. If the first form does not match the expected form, the description information corresponding to the first form and the target part in the first form can be obtained.
  • the contour information of the part further, based on the description information corresponding to the first form and the contour information of the target part, the form correction is performed on the target part, so as to modify the form of the target part from the first form to the second form, and the second form is If it matches the expected form, in this way, there is no need to manually correct the first form, but according to the description information of the first form and the outline information of the target part in the first form, the form correction of the target part is automatically performed, so that the target part The shape is in line with the expected shape. The consumption of human resources caused by manual correction is eliminated, and the efficiency of shape correction is improved.
  • FIG. 10 it is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • the computer device shown in FIG. 10 may include a processor 1001 , an input interface 1002 , an output interface 1003 and a computer storage medium 1004 .
  • the processor 1001, the input interface 1002, the output interface 1003, and the computer storage medium 1004 may be connected through a bus or other means.
  • the computer storage medium 1004 may be stored in the memory of the image processing device, the computer storage medium 1004 is used for storing computer programs, and the processor 1001 is used for executing the computer programs stored in the computer storage medium 1004 .
  • Processor 1001 (or CPU (Central Processing Unit, central processing unit)) is the calculation core and control core of computer equipment, which is suitable for implementing one or more computer programs, and is specifically suitable for loading and executing the program provided by the embodiment of the present application. image processing method.
  • CPU Central Processing Unit, central processing unit
  • the embodiment of the present application also provides a computer storage medium (Memory), the computer storage medium is a memory device of a computer device, and is used to store programs and data. It can be understood that the computer storage medium here may include the built-in storage medium of the computer device, and of course may also include the extended storage medium supported by the computer device.
  • a computer storage medium provides a storage space that stores an operating system of a computer device. Moreover, one or more computer programs suitable for being loaded and executed by the processor 1001 are also stored in the storage space.
  • the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory; computer storage media.
  • one or more computer programs stored in the computer storage medium can be loaded by the processor 1001 to execute the image processing method provided by the embodiment of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Graphics (AREA)
  • Architecture (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)

Abstract

本申请实施例公开了一种图像处理方法、装置、设备、存储介质及计算机程序，其中方法包括：获取待处理图像，该待处理图像中包括对象的目标部位，目标部位的形态为第一形态，第一形态与预期形态是不匹配的；获取第一形态对应的描述信息，以及获取处于第一形态时目标部位的轮廓信息；基于描述信息和轮廓信息对目标部位的形态进行修正，将目标部位的形态由第一形态修正为第二形态，第二形态与预期形态是相匹配的。采用本申请实施例，可以节省人力资源，提高形态修正效率。

Description

图像处理方法、装置、设备、存储介质及计算机程序
本申请要求于2022年01月10日提交中国专利局、申请号为2022100241125、申请名称为“图像处理方法、装置、设备、存储介质及计算机程序”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及图像处理技术。
背景技术
在一些虚拟现实应用以及游戏应用中，为了带给使用者更加逼真的效果，通常采用3D(three-dimensional)技术制作在应用中的玩家角色、怪物角色等虚拟角色。更进一步的，为了使得3D角色更加逼真，还会为3D角色制作表情，比如微笑的表情、难过的表情等。
3D角色表情制作一般通过面部控制器实现，举例来说，制作3D角色表情时，可以根据包括人物面部的图像，对与3D角色的面部绑定的面部控制器的控制器参数进行编辑调整，从而在3D角色上制作出对应的面部表情。
但是，由于3D角色面部部位的形态与图像中人物面部部位的形态通常存在差别，且不同部位的形态对表情的诠释都起着至关重要的作用，即使是微小的差异对表情的情感表达也可能有很大影响，因此，在很多情况下，通过上述方法制作出的3D角色表情与所预期的图像中的人物面部表情相差较大，此时，需要通过人工调整的方式来修正3D角色的面部形态。而人工纠正的方式不仅耗费人力资源，且修正效率较低。
发明内容
本申请实施例提供了一种图像处理方法，装置、设备、存储介质及计算机程序，提高了对于对象目标部位的形态修正效率。
一方面，本申请实施例提供了一种图像处理方法，由计算机设备执行，包括：
获取待处理图像，所述待处理图像中包括对象的目标部位，所述目标部位的形态为第一形态，所述第一形态与预期形态不匹配；
获取所述第一形态对应的描述信息，以及获取处于所述第一形态时所述目标部位的轮廓信息；
基于所述描述信息和所述轮廓信息对所述目标部位的形态进行修正，将所述目标部位的形态由所述第一形态修正为第二形态，所述第二形态与所述预期形态相匹配。
一方面，本申请实施例还提供了一种图像处理装置，包括：
获取单元，用于获取待处理图像，所述待处理图像中包括对象的目标部 位，所述目标部位的形态为第一形态，所述第一形态与预期形态不匹配；
所述获取单元，还用于获取所述第一形态对应的描述信息，以及获取处于所述第一形态时所述目标部位的轮廓信息；
修正单元，用于基于所述描述信息和所述轮廓信息对所述目标部位的形态进行修改，将所述目标部位的形态由所述第一形态修正为第二形态，所述第二形态与所述预期形态相匹配。
一方面，本申请实施例提供了一种计算机设备，包括：处理器，适用于实现一条或多条计算机程序；计算机存储介质，所述计算机存储介质存储有一条或多条计算机程序，所述一条或多条计算机程序适于由处理器加载并执行本申请实施例提供的图像处理方法。
一方面，本申请实施例提供了一种计算机存储介质，所述计算机存储介质存储有计算机程序，所述计算机程序被图像处理设备的处理器执行时，用于执行本申请实施例提供的图像处理方法。
一方面，本申请实施例提供了一种计算机程序产品或计算机程序，所述计算机程序产品包括计算机程序，计算机程序存储在计算机存储介质中；图像处理设备的处理器从计算机存储介质中读取计算机程序，该处理器执行计算机程序，使得图像处理设备执行本申请实施例提供的图像处理方法。
本申请实施例中，在待处理图像中对象的目标部位处于第一形态，如果该第一形态与预期形态不匹配，则可以获取第一形态对应的描述信息、以及处于第一形态时目标部位的轮廓信息，进一步的，基于第一形态对应的描述信息和目标部位的轮廓信息对目标部位进行形态修正，以将目标部位的形态由第一形态修正为第二形态，该第二形态是与预期形态相匹配的，如此，无需人为手动对第一形态进行修正，而是根据第一形态的描述信息和第一形态时目标部位的轮廓信息，自动对目标部位进行形态修正，使得目标部位的形态符合预期形态。省去了人工手动修正带来的人力资源消耗，提高了形态修正的效率。
附图说明
图1a是本申请实施例提供的一种3D角色图像与形态控制器绑定的示意图；
图1b是本申请实施例提供的一种对唇部形态进行修正的示意图；
图2是本申请实施例提供的一种图像处理方法的流程示意图；
图3是本申请实施例提供的一种待处理图像与图像网格关联的示意图；
图4a是本申请实施例提供的一种目标部位的轮廓和网格顶点示意图；
图4b是本申请实施例提供的一种目标部位的轮廓信息的示意图；
图4c是本申请实施例提供的另一种目标部位的轮廓信息示意图；
图5是本申请实施例提供的一种参数预测模型的结构示意图；
图6是本申请实施例提供的另一种图像处理方法的流程示意图；
图7a是本申请实施例提供的一种对参数预测模型的训练示意图；
图7b是本申请实施例提供一种对位置预测模型的训练示意图；
图8是本申请实施例提供的另一种参数预测模型的训练示意图；
图9是本申请实施例提供的一种图像处理装置的结构示意图；
图10是本申请实施例提供的一种计算机设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
本申请实施例提供了一种图像处理方法，获取到待处理图像后，确定待处理图像中对象的目标部位为第一形态，进一步的检测该待处理图像中对象目标部位的第一形态是否与预期的形态相匹配，当目标部位的第一形态与预期的形态不匹配时，可以根据处于第一形态时目标部位的轮廓信息以及第一形态对应的描述信息对目标部位进行形态修正，以使得目标部位的形态由第一形态修正为与预期的形态相匹配的第二形态。
在一个实施例中，该图像处理方法可由图像处理设备执行，图像处理设备可以是终端，比如智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表、车载终端、智能家电、智能语音交互设备等；或者，图像处理设备也可以是服务器，比如独立的物理服务器、多个物理服务器构成的服务器集群或者分布式系统、提供云计算服务的云服务器等。
另一个实施例中，该图像处理方法可以由图像处理设备和图像管理设备协同执行，例如，图像管理设备可以存储待处理图像，并向图像处理设备发送待处理图像，图像处理设备获取到待处理图像后执行上述图像处理方法；再如，图像处理设备可以从本地存储中获取待处理图像，然后，获取待处理图像中目标部位的第一形态对应的描述信息以及目标部位处于第一形态时的轮廓信息，接着，图像处理设备将描述信息和轮廓信息传输给图像管理设备，由图像管理设备基于描述信息和轮廓信息对目标部位的形态进行修正，最后将修正结果反馈给图像处理设备。
需要说明的是，上述图像处理设备和图像管理设备均为计算机设备。
可选的，待处理图像中对象的目标部位可以指对象面部包括的五官部位中任意一个或多个，比如眼睛、唇部、鼻子、耳朵以及眉毛等等；或者，待处理图像中对象的目标部位还可以指对象身体的其他部位，比如手指、胳膊等等。为了方便描述，在无特殊说明的情况下，本申请实施例以目标部位为唇部为例进行介绍。
本申请实施例提供的该图像处理方法可以应用在3D角色表情制作场景中，例如，待处理图像是指3D角色图像，通过下述步骤(1)和(2)为3D角色图像中的3D角色制作表情，在制作得到的表情中目标部位的形态为第一形态。具体地：
(1)将3D角色图像与形态控制器绑定，图1a为本申请实施例提供的一种将3D角色图像与形态控制器绑定的示意图，101表示3D角色图像，3D角色图像与图像网格相关联，3D角色图像中对象面部的每个部位都可以通过对应网格的顶点来描述，比如唇部可以通过103内的网格顶点来描述。102表示形态控制器中与3D角色图像对应的部位控件，比如1021表示形态控制器中与3D角色的唇部绑定的部位控件。形态控制器中的控制器参数可以包括每个部位控件的参数，每个部位控件的参数可以决定其绑定的部位的尺寸信息、形状信息等等。
(2)动画师根据包括人物面部的图像，对形态控制器参数进行编辑，从而制作出对应的表情，例如，图像中人物唇部的形态如图1b中11所示，该唇部的形态为预期的形态；对形态控制器参数进行编辑，制作出3D角色图像的表情，在制作出的表情中唇部的形态如图1b中12所示，该唇部形态可以视为第一形态。
通过对比可以发现，12所示的第一形态与图像中人物的唇部形态(预期形态)不同。因此，可以采用本申请实施例提供的图像处理方法对唇部进行形态修正，具体地，获取唇部处于当前形态时该当前形态对应的描述信息，该描述信息可以是在图像网格中用于表示唇部的多个网格的位置信息，以及获取处于当前形态时唇部的轮廓信息，进一步的，基于轮廓信息和描述信息进行形态修正，得到修正后的唇部形态如图1b中13所示，该唇部形态可以视为第二形态。
由图1b可见，采用本申请实施例提供的图像处理方法修正后的形态与预期的形态匹配度较高，且本申请的图像处理方法不需要人为的修正，可节省修正时间以及提高修正效率。
可选的，上述3D角色可以是游戏应用中的虚拟角色，在正式发布游戏应用之前，为了达到更加逼真的游戏效果，可以为虚拟角色预先设计一些表情，在不同游戏场景中，该虚拟角色可以呈现不同的表情，比如，在游戏对战场景中，虚拟角色可以呈现比较亢奋的表情，游戏对战胜利后，虚拟角色可以显现出微笑的表情，等等。可以通过上述步骤(1)和(2)为虚拟角色设计不同的表情，如果设计出的表情中目标部位的形态与预期的形态不匹配，则通过本申请提供的图像处理方法来修改目标部位的形态，以使得虚拟角色的表情更加精准。
可选的，上述3D角色还可以是一个基于虚拟对象进行人机交互场景中的虚拟对象，基于虚拟对象进行人机交互的场景可以为：虚拟对象为产品客服，用户可以通过与虚拟对象聊天交互来讨论或者了解相关产品。为了让用户体验到与真人聊天的逼真感，可以预先为该虚拟对象设计几种表情，不同聊天场景下虚拟对象可以表现不同标签。比如，聊天场景为用户对产品提出表扬，那么虚拟对象可以呈现微笑的表情；聊天场景为用户对产品表现出不 满意，那么虚拟对象可以呈现难过的表情。同样可以通过上述步骤(1)和(2)为虚拟对象设计表情，如果设计出的表情中目标部位的形态与预期的形态不匹配，采用本申请提供的图像处理方法来修正目标部位的形态。
其他实施例中，基于虚拟对象进行人机交互的场景还可以是：虚拟对象作为应用程序的使用引导者，当用户第一次使用该应用程序时，虚拟对象可以通过一些肢体操作指示用户如何使用应用程序。再或者，基于虚拟对象进行人机交互的场景还可以是：虚拟对象可以是用户在某个应用程序中的虚拟形象，用户可以利用该虚拟形象代表自己，在应用程序中可以通过该虚拟形象与好友进行会话交互，或者在应用程序中以该虚拟形象来参加模拟现实生活中的一些活动，比如种菜、开会、交友等等。
上述只是本申请实施例列举的图像处理方法的应用场景，在具体实现中，凡是涉及到对象形态修正的应用场景，均可以采用本申请实施例提出的图像处理方法，来达到节省修正时间和提高修正效率的目的。
基于上述的图像处理方案，本申请实施例提供了一种图像处理方法，参见图2，为本申请实施例提供的一种图像处理方法的流程示意图。图2所示的图像处理方法可由计算机设备执行，具体可由计算机设备的处理器执行，该计算机设备具体可以是上文的图像处理设备或图像管理设备；图2所示的图像处理方法可包括如下步骤：
步骤S201、获取待处理图像，待处理图像中包括对象的目标部位，目标部位的形态为第一形态，第一形态与预期形态不匹配。
其中，该待处理图像可以存储在计算机设备本地，此时计算机设备可以从本地存储中直接获取该待处理图像；或者，该待处理图像可以由其他设备生成并存储在其他设备中，此时计算机设备可以接收其他设备发送的该待处理图像。
待处理图像中可以包括对象面部，该对象可以是人或动物等任意具备表情的对象，该待处理图像可以指3D角色表情制作中的3D角色图像，该待处理图像可以通过3D技术合成，例如参见图1a中101可以表示一张待处理图像。
在待处理图像中对象的目标部位可以指对象面部的五官中任意一个，比如眼睛、眉毛、鼻子、唇部以及耳朵。目标部位的形态可以为第一形态，该第一形态可以通过面部控制器生成，详细描述可参见前文中通过面部控制器或形态控制器确定面部部位形态的相关介绍内，在此不再赘述。假设目标部位为唇部，第一形态可以包括张嘴状态、微笑状态或者大笑状态等等。
在一个实施例中，可以将待处理图像中目标部位的第一形态与预期形态进行比对，确定第一形态与预期形态之间的匹配情况；如果第一形态与预期形态不匹配，则需要执行后续的步骤S202和步骤S203，对目标部位进行形 态修正；如果第一形态与预期形态相匹配，则可以不执行步骤S202和步骤S203。例如，目标部位为唇部，第一形态是微笑状态，预期形态为张嘴大笑状态，那么此时第一形态与预期形态就是不匹配的；或者，第一形态和预期形态都是微笑状态，但是微笑的程度不同也可以导致第一形态与预期形态不匹配。
在一个实施例中，将待处理图像中目标部位的第一形态与预期形态进行比对，确定第一形态与预期形态之间的匹配情况，可以包括：人为的将第一形态与预期形态进行视觉上的比对，根据比对结果向计算机设备反馈第一形态与预期形态之间的匹配情况。
在另一个实施例中，将待处理图像中目标部位的第一形态与预期形态进行比对，确定第一形态与预期形态之间的匹配情况，可以包括：采用图像相似度分析技术，对处于第一形态的目标部位和处于预期形态的目标部位进行相似度比较处理，得到两种形态下的目标部位之间的相似度；如果相似度比较结果小于或等于相似度阈值，则可以确定第一形态与预期形态是不匹配的；如果相似度结果大于相似度阈值，则确定第一形态与预期形态是匹配的。
预期形态可以是预先指定的任意一种形态，该预期形态可以是任意一张图片或者任意一段视频中对象面部中目标部位所处的形态，例如，在制作3D角色表情中，所参照的图像中人物的目标部位所处的形态即被预先指定为预期形态。
步骤S202、获取第一形态对应的描述信息，以及获取处于第一形态时目标部位的轮廓信息。
待处理图像可以与图像网格相关联，图像网格可以包括M个网格顶点，网格顶点是指组成网格的点，图像网格包括多个网格，每个网格可以由多个顶点组成。目标部位与M个网格顶点中N个网格顶点对应，N个网格顶点的位置变化(此处N个网格顶点的位置变化可以指N个网格顶点中任意一个或多个网格顶点变化)会导致目标部位呈现不同的形态，因此可以将N个网格顶点的位置信息作为第一形态的描述信息。
举例来说，图3为本申请实施例提供的一种待处理图像与图像网格关联的示意图，图像网格中可以包括多个网格顶点，比如在图3中301和302均表示网格顶点，待处理图像中对象面部的各个部位均可以与若干个网格顶点相对应，比如303区域内的网格顶点是与唇部对应的网格顶点。假设目标部位为唇部，那么唇部处于第一形态时的描述信息就是303区域内多个网格顶点的位置信息。
在一个实施例中，目标部位可以包括内轮廓和外轮廓，内轮廓可以与L个网格顶点对应，外轮廓可以与P个网格顶点对应，例如，目标部位为唇部，图4a为本申请实施例提供的一种目标部位的轮廓与网格顶点对应关系，图4a中401表示目标部位，402中黑色虚线表示目标部位的内轮廓，黑色实线表 示目标部位的外轮廓；实心圆表示与内轮廓对应的L个网格顶点，空心圆表示与外轮廓对应的P个网格顶点。
目标部位的轮廓信息可以包括内轮廓和外轮廓之间的第一距离，具体地，内轮廓和外轮廓之间的第一距离可以包括L个网格顶点与P个网格顶点中具有对应关系的网格顶点之间的距离，例如，参见图4b，图4b中彼此间存在连线的实心圆和空心圆即为一组具有对应关系的网格顶点，具有对应关系的实心圆和空心圆之间的距离被称为内轮廓和外轮廓之间的第一距离，例如，线段A的长度、线段B的长度以及线段C的长度均为第一距离。
目标部位的轮廓信息还可以包括外轮廓对应的P个网格顶点中每两个网格顶点之间的第二距离，例如，参见图4c，图4c中每两个实心圆(图4c中的实心圆实质上为图4a和图4b中的空心圆)之间的距离为第二距离，例如，线段D的长度、线段E的长度以及线段F的长度均为第二距离。
步骤S203、基于描述信息和轮廓信息对目标部位的形态进行修正，将目标部位的形态由第一形态修正为第二形态，第二形态与预期形态相匹配。
在一个实施例中，基于描述信息和轮廓信息对目标部位的形态进行修正，将目标部位的形态由第一形态修正为第二形态，包括：基于描述信息和轮廓信息预测一个目标参数，调用形态控制器基于该目标参数调整该描述信息，调整后的描述信息用于使目标部位的形态呈现为第二形态。
其中，基于描述信息和轮廓信息预测目标参数可以调用参数预测模型实现，参数预测模型可以是预先训练得到的。具体实现中，步骤S203中基于描述信息和轮廓信息对目标部位的形态进行修正，将目标部位的形态由第一形态修正为第二形态，包括：对描述信息和轮廓信息进行拼接处理；调用参数预测模型基于拼接处理结果进行参数预测，得到形态控制器的目标参数；调用形态控制器基于目标参数调整描述信息，调整后的描述信息用于使目标部位的形态呈现第二形态。
具体对描述信息和轮廓信息进行拼接处理时，可以先获取降维矩阵，采用该降维矩阵对描述信息进行降维处理，进而再利用降维处理后的描述信息和轮廓进行进行拼接处理。此处可以采用主成分分析(Principal Component Analysis，PCA)算法对描述信息进行降维处理。
此外，参数预测模型可以是任意的深度网络结构，参见图5，图5为本申请实施例提供的一种参数预测模型的结构示意图，图5所示的参数预测模型可以包括三个全连接层，每个全连接层可以对应一个激活函数，该激活函数可以是ReLU。ReLU的工作原理可以通过公式y＝ReLU(w*x+b)表示，其中，x为输入，w为全连接层的权重，b为偏置，y为当前层的输出。
本申请实施例中，在待处理图像中对象面部的目标部位处于第一形态，如果该第一形态与预期形态不匹配，则可以获取第一形态对应的描述信息，以及处于第一形态时目标部位的轮廓信息，进一步的，基于第一形态对应的描述信息和目标部位的轮廓信息对目标部位进行形态修正，以将目标部位的形态由第一形态修正为第二形态，该第二形态与预期形态相匹配，如此，无需人为手动对第一形态进行修正，可以根据第一形态的描述信息和第一形态时目标部位的轮廓信息，自动对目标部位进行形态修正，使得目标部位的形态满足预期形态。省去了人工手动修正带来的人力资源消耗，提高了形态修正的效率和准确性。
基于上述的图像处理方法，本申请实施例还提供了另一种图像处理方法，参见图6，为本申请实施例提供的另一种图像处理方法的流程示意图。图6所示的图像处理方法可由计算机设备执行，具体可由计算机设备的处理器执行，该计算机设备具体可以是上文的图像处理设备或图像管理设备。图6所示的图像处理方法可包括如下步骤：
步骤S601、获取待处理图像，待处理图像中包括对象的目标部位，目标部位的形态为第一形态，第一形态与预期形态不匹配。
步骤S602、获取第一形态对应的描述信息，以及获取处于第一形态时目标部位的轮廓信息。
其中，步骤S601中包括的一些可行的实施方式可参见图2实施例中步骤S201中的相关描述，在此不再赘述。步骤S602中包括的一些可行的实施方式可参见图2实施例中步骤S202中的相关描述，在此不再赘述。
步骤S603、对描述信息和轮廓信息进行拼接处理，并调用参数预测模型基于拼接处理结果进行参数预测，得到形态控制器的目标参数。
在一个实施例中，参数预测模型可以基于第一训练样本训练得到，具体可包括如下步骤s1-s4:
s1：获取第一训练样本，该第一训练样本中可以包括第一样本图像以及每个第一样本图像对应的标签信息，第一样本图像中包括对象面部的目标部位，第一样本图像对应的标签信息可以包括第一样本图像中目标部位处于第一训练形态时对应的第一描述信息、以及第一样本图像中目标部位处于第二训练形态时对应的第二描述信息。
其中，第一样本图像与图像网格对应，第一样本图像中对象面部的五官均与若干个网格顶点对应，目标部位可以与步骤S601中的目标部位相同，均可以是五官中任意一个部位。第一样本图像中目标部位处于第一训练形态时对应的第一描述信息，可以是第一样本图像关联的图像网格中与目标部位对应的多个网格顶点的第一位置信息，第一样本图像中目标部位处于第二训练形态时对应的第二描述信息，可以是目标部位对应的多个网格顶点的第二位 置信息。第一训练形态与预期训练形态不匹配的，第二训练形态与预期训练形态相匹配。预期训练形态的确定方式与前述预期形态的确定方式可以相同，均可以是参照图像中人物的目标部位所处的形态。
第一样本图像的数量可以为X个，X为大于或等于1的整数，假设第一描述信息表示为M coarse，第二描述信息表示为M fine，目标部位对应的网格顶点的数量为N1个，每个网格顶点的维度为3维，那么M coarse的维度为(X，N1*3)，其中N1远小于X。换句话说，如果第一样本图像的数量为X个，那么第一描述信息包括每个第一样本图像的第一描述信息，每个第一样本图像的第一描述信息都可以看成是N1个网格顶点的位置信息的集合。
第一描述信息中可以包括X行，每行包括一个第一样本图像中N1个网格顶点的第一位置信息，比如，一个网格顶点的第一位置信息可通过一个三维坐标(x,y,z)表示，则第一描述信息的每行可以表示为(x 1，y 1，z 1，x 2，y 2，z 3，...，x N1，y N1，z N1)，也即第一描述信息的每行是由一个第一样本图像中N1个网格顶点的位置信息组成的。同理，第二描述信息中每行为一个第一样本图像中N1个网格顶点的第二位置信息。
s2：获取第一样本图像中目标部位处于第一训练形态时该目标部位的训练轮廓信息，并对训练轮廓信息和第一描述信息进行拼接处理。
需要说明的是，在无特殊说明时，这里以一个第一样本图像为例展开描述，对于其他的第一样本图像采用相同的处理步骤，可以得到每个第一样本图像中目标部位处于第一训练形态时的训练轮廓信息。
可选的，本申请实施例可以采用与步骤S202中获取待处理图像中目标部位处于第一形态时的轮廓信息相同的方式，来获取第一样本图像中目标部位处于第一训练形态时该目标部位的训练轮廓信息，具体实现方式可参见步骤S202中的描述，在此不再赘述。
在一个实施例中，为了降低特征的稀疏性，提高参数预测模型的学习效率，在对训练轮廓信息和第一描述信息进行拼接处理之前，可以先对第一描述信息进行降维处理，对降维处理后的第一描述信息与训练轮廓信息进行拼接处理。对第一描述信息进可以采用PCA算法进行降维处理。PCA算法进行降维处理的原理是，基于第一描述信息和第二描述信息生成一个降维矩阵；对第一描述信息和降维矩阵进行相乘处理得到降维处理后的第一描述信息。假设第一描述信息表示为M coarse，降维矩阵表示为D K，通过下述公式(1)表示对第一描述信息进行降维处理：
MD coarse＝M coarse*D K    (1)
其中，MD coarse表示降维处理后的第一描述信息，假设降维矩阵的维度为(N1*3，K)，第一描述信息的维度为(X，N1*3)，降维处理后，第一描述信息的维度降低为(X，K)，K为远小于N1*3的整数。
在一个实施例中，基于第一描述信息和第二描述信息生成降维矩阵，包 括：
s21、对所述第一描述信息和所述第二描述信息进行拼接，得到描述信息矩阵；假设第一描述信息表示为M coarse，第二描述信息可以表示为M fine，第一描述信息和第二描述信息的维度均为(X，N1*3)。将第一描述信息和第二描述信息拼接后得到描述信息矩阵，表示为M，M的维度为(X*2，N1*3)，将第一描述信息和第二描述信息拼接可以通过下述公式(2)表示：
Figure PCTCN2022128637-appb-000001
s22、对描述信息矩阵进行奇异值分解处理，得到特征向量矩阵和特征值矩阵；特征向量矩阵中包括多个特征向量；特征值矩阵包括每个特征向量对应的特征值，特征向量矩阵中的特征向量和特征值矩阵中的特征值是一一对应的。对描述信息矩阵进行奇异值分解处理的过程可以通过如下公式(3)表示：
M＝UVD T   (3)
在公式(3)中，D表示特征向量矩阵，维度为(N1*3，N1*3)，V为特征值矩阵，V是一个非负矩形对角矩阵，维度为(X*2，N1*3),特征值矩阵V的对角线上的值为每个特征向量对应的特征值，U是与D正交的矩阵，维度与D相同。
s23、对特征向量矩阵进行调整，调整后的特征向量矩阵中多个特征向量按照对应的特征值由大到小的顺序排列。假设多个特征向量的特征值从大到小的排列可以表示为x 1，x 2，x 3，...，x N1*3。
s24、根据调整后的特征向量矩阵中每个特征向量的特征值，按序选取K个特征向量，将这K个特征向量组成的矩阵确定为降维矩阵，K个特征向量对应的特征值之和大于或等于特征阈值。换句话说，选取出的K个特征向量要保证保留了描述信息矩阵中大于或等于特征阈值的信息。
具体实现中，假设令x＝x 1+x 2+x 3+...+x N1*3，K为第一个让以下公式(4)成立的值：
Figure PCTCN2022128637-appb-000002
一般情况下特征阈值可以取99％。通过上述公式(4)求解出K之后，将前K个特征向量组成的矩阵确定为降维矩阵，降维矩阵的维度为(N1*3，K)，其中，K远小于N1*3的。
s3：调用待训练的参数预测模型基于拼接处理结果进行参数预测，得到形态控制器的训练参数。
由前述可知，第一样本图像的数量可以为X个，那么第一描述信息的数量和训练轮廓信息的数量均为X个，此处的拼接处理结果可以是对任意一个 第一样本图像对应的第一描述信息和训练轮廓信息进行拼接处理得到的；也就是说，对X个第一样本图像中每个第一样本图像对应的第一描述信息和训练轮廓信息进行拼接处理，都会得到一个拼接处理结果，那么，对于第一训练样本来说，拼接处理结果的数量也为X个。在对参数预测模型进行训练时，可以将这X个拼接结果逐一输入到该参数预测模型。具体来讲，假设第一样本图像的数量为X个，在每个第一样本图像中目标部位均与N1个网格顶点对应，每个网格顶点的位置信息是3维的，那么第一描述信息的维度表示为(X，N1*3，)，降维处理后第一描述信息的维度表示为(X，K)，训练轮廓信息的维度为(X，D)。对降维处理后的第一描述信息和训练轮廓信息进行拼接后，拼接处理结果的维度为(X，K+D)。
在对参数预测模型进行训练时，每次将一个第一样本图像对应的第一描述信息和训练轮廓信息进行拼接处理后，将拼接处理结果输入到该参数预测模型中，高参数预测模型输出一个训练参数，该参数预测模型基于一个拼接处理结果进行参数预测处理的过程可以通过图7a表示。参数预测模型又可以称为(Meah2Params)，每次输入到参数预测模型中的拼接处理结果的维度为(1，K+C)，参数预测模型输出的训练参数的维度为(1，H)，H表示形态控制器中用于控制目标部位形态的参数个数。
s4：基于训练参数进行描述信息预测，得到目标部位处于第二训练形态时对应的第一预测描述信息，根据第一预测描述信息和第二描述信息，对参数预测模型进行训练。
其中，根据第一预测描述信息和第二描述信息，对参数预测模型进行训练，具体可以包括：基于第一预测描述信息和第二描述信息，计算第一损失函数；采用反向传播算法，基于第一损失函数更新参数预测模型中的网络参数。
例如，基于第一预测描述信息和第二描述信息计算第一损失函数，可以通过如下公式(5)表示：
L loss＝MeanSquaredError(MD predict，MD fine)    (5)
在公式(5)中，L loss表示第一损失函数，MD predict表示第一预测描述信息，MD fine表示第二描述信息，MeanSquaredError(·)表示均方差计算函数。
通过计算第一损失函数对网络参数的偏导数来更新网络参数，可以通过下述公式(6)表示：
Figure PCTCN2022128637-appb-000003
其中，w t表示参数预测模型中更新之前的网络参数，w t+1为更新后的网络参数，α为学习率，一般取值为10e-3左右。网络参数不断迭代优化，直到 第一损失函数不再变化为止。
在一个实施例中，上述基于训练参数进行描述信息预测，得到目标部位处于第二训练形态时对应的第一预测描述信息，可以调用位置预测模型执行。具体实现中，将训练参数作为位置预测模型的输入，调用位置预测模型进行描述信息预测，得到第一预测描述信息。可见，位置预测模型的作用是根据形态控制器的参数预测得到一个描述信息，换句话说，位置预测模型可以模仿形态控制器基于参数控制描述信息的过程。
该位置预测模型可以是一个预训练的模型，位置预测模型可以基于第二训练样本训练得到，第二训练样本包括第二样本图像、以及第三描述信息与形态控制器的第二训练参数之间的对应关系，第二样本图像包括对象面部的目标部位，第三描述信息是第二样本图像中目标部位处于第三形态时对应的描述信息。第二训练样本中第三描述信息与形态控制器中第二训练参数之间的对应关系可以包括多组。其中，第二样本图像中目标部位的第三形态可以是与一个预期训练形态相匹配的形态，也可以是与一个预期训练形态不匹配的形态，比如，预期训练形态是参照图像中人物表情中目标部位的形态，那么该第三形态可以是形态控制器基于该人物表情自动生成的一种形态，这种形态可能与预期训练形态存在误差，不是非常准确。或者，第三形态可以是形态控制器基于参照图像中人物表情自动生成了一种形态后，人工对自动生成的形态进行手工调整后的形态，此时的第三形态与预期训练形态匹配。此处的预期训练形态与前述的预期训练形态可以是相同的，也可以是不同的。
第二样本图像可以与图像网格对应，目标部位与图像网格中某些网格顶点对应，目标部位处于第三形态时的第三描述信息是指目标部位对应的网格顶点的位置信息。第三描述信息与形态控制器中第二训练参数之间的对应关系可以基于形态控制器和第三描述位置信息确定，具体地，可以在形态控制器中随机生成一个第二训练参数，基于该第二训练参数对第二样本图像中目标部位的形态进行修改，此时目标部位的形态对应的第三描述信息就是与当前的第二训练参数对应的第三描述信息。重复几次上述过程，便可以得到多组第三描述信息和第二训练参数之间的对应关系。
进一步的，将一组第三描述信息与第二训练参数作为一个训练数据对，调用位置预测模型，基于一个训练数据对中的第二训练参数进行描述信息预测，得到第二预测描述信息。此处用到的第三描述信息是经过降维矩阵进行降维处理后的，假设得到了X组第三描述信息和第二训练参数之间的对应关系，每个描述信息的维度为3，每个描述信息与N1个网格顶点对应，则第三描述信息的维度为(X，N1*3)；采用前述得到矩阵对该第三描述信息进行降维处理，降维后的第三描述信息的维度为(X，K)；假设第二训练参数中包括H个参数，那么第二训练参数的维度为(X，H)。参见图7b，为本申请实施例提供的一种位置预测模型的训练示意图，位置预测模型还可以称为 (Params2Mesh，P2M)，每次将一个训练数据对中的第二训练参数输入到P2M模型中，此时输入到P2M模型中的第二训练参数的维度为(1，H)，P2M模型基于第二训练参数预测描述信息，输出第二预测描述信息，该第二预测描述信息的维度为(1，K)。
然后，基于第二预测描述信息和第三描述信息，对位置预测模型进行训练。具体地，基于第二预测描述信息和第三描述信息，确定位置预测模型对应的第二损失函数；通过计算第二损失函数对位置预测模型中网络参数的偏导数，来更新位置预测模型中的网络参数，从而实现对位置预测模型的训练。例如，基于第二预测描述信息和第三描述信息确定位置预测模型对应的第二损失函数，可以通过如下公式(7)表示：
L loss’＝MeanSquareError(M predict‘M ground truth)    (7)
其中，L loss‘表示第二损失函数，M predict‘表示第二预测描述信息，M ground truth表示第三描述信息。
通过计算第二损失函数对位置预测模型中网络参数的偏导数来更新网络参数，可以通过如下公式(8)表示：
Figure PCTCN2022128637-appb-000004
在公式(8)中，w’ t+1表示位置预测模型中更新后的网络参数，w’ t表示位置预测模型中更新之前的网络参数。
可选的，位置预测模型也可以是任意的深度网络结构，只要能满足输入输出的要求，且能通过训练进行迭代优化，最后输入一个控制器的参数得到正确的描述信息即可。本申请实施例提供的位置预测模型可以与参数预测模型具有相同的网络结构，参见图5；位置预测模型和参数预测模型之间不同的是：输入数据的不同，位置预测模型的输入数据为形态控制器的参数，参数预测模型的输入数据为描述信息；以及，输出数据的不同，位置预测模型的输出数据为描述信息，参数预测模型的输出数据为形态控制器的参数；以及，每个全连接层中隐藏元的个数不同。
步骤S604、调用形态控制器基于目标参数调整描述信息，调整后的描述信息用于使目标部位的形态呈现第二形态，第二形态与预的形态相匹配。
在一个实施例中，步骤S604中包括的一些可行的实施方式可参见图2实施例中步骤S203的相关描述，此处不再赘述。
基于上述步骤s1-s4的描述，本申请实施例提出一种参数预测模型的训练示意图，参见图8。在图8中，假设目标部位是指唇部，第一样本图像对应的标签信息中包括第一样本图像中处于第一训练形态时唇部对应的第一描述信息，该第一描述信息与预期训练形态不匹配，因此该第一描述信息又可以 理解为粗糙嘴部形态的网格顶点位置，标签信息中还包括第一样本图像中处于第二训练形态时唇部对应的第二描述信息，第二描述信息与预期训练形态相匹配，因此该第二描述信息又可以理解为准确嘴部形态的网格顶点位置。
训练轮廓信息可以指嘴部的轮廓信息，嘴部的轮廓信息包括内轮廓和外轮廓之间的距离、以及外轮廓中每两个网格顶点之间的距离，将这些统称为嘴部内轮廓与外轮廓距离特征。
对粗糙嘴部形态的网格顶点位置进行PCA降维处理后，与嘴部内轮廓与外轮廓距离特征进行拼接；将拼接后得到的数据输入到参数预测模型M2P中，M2P基于输入数据进行参数预测，输出训练参数；该训练参数作为位置预测模型P2M的输入数据，P2M基于输入的训练参数进行描述信息预测，得到第一预测描述信息，该第一预测描述信息又可以理解为预测的嘴部形态的网格顶点位置；进一步的，根据预测的嘴部形态的网格顶点位置和降维处理后的准确嘴部形态的网格顶点位置，计算参数预测模型M2P的损失函数，基于该损失函数对M2P模型进行训练。
本申请实施例中，采用第一样本训练训练得到一个M2P模型，该M2P模型可以根据粗糙的描述信息和目标部位的轮廓信息来预测准确的描述信息。采用训练完成的M2P模型对目标部位的形态进行修正，得到与预期形态相匹配的目标部位形态，通过训练完成的M2P模型，可以省去人工修正所消耗的人力资源，以及避免人工修正带来的误差，从而提高了形态修正的效率和准确性。
基于上述的图像处理方法实施例，本申请实施例提供了一种图像处理装置，参见图9，为本申请实施例提供的一种图像处理装置的结构示意图。图9所示的图像处理装置可运行如下单元：
获取单元901，用于获取待处理图像，所述待处理图像中包括对象面部的目标部位，所述目标部位的形态为第一形态，所述第一形态与预期形态不匹配；
所述获取单元901，还用于获取所述第一形态对应的描述信息，以及获取处于所述第一形态时所述目标部位的轮廓信息；
修正单元902，用于基于所述描述信息和所述轮廓信息，对所述目标部位的形态进行修改，将所述目标部位的形态由所述第一形态修正为第二形态，所述第二形态与所述预期形态相匹配。
在一个实施例中，所述目标部位与形态控制器绑定，所述基于所述描述信息和所述轮廓信息，修正单元902在对所述目标部位的形态进行修正，将所述目标部位的形态由第一形态修正为第二形态时，执行如下步骤：
对所述描述信息和所述轮廓信息进行拼接处理；
调用参数预测模型基于拼接处理结果进行参数预测，得到所述形态控制 器的目标参数；
调用所述形态控制器基于所述目标参数调整所述描述信息，调整后的描述信息用于使所述目标部位的形态呈现所述第二形态。
在一个实施例中，修正单元902在对所述描述信息和所述轮廓信息进行拼接处理时，执行如下步骤：
获取降维矩阵，并采用所述降维矩阵对所述描述信息进行降维处理；
对降维处理后的描述信息和所述轮廓信息进行拼接处理。
在一个实施例中，所述待处理图像与图像网格相关联，所述图像网格包括M个网格顶点，所述目标部位与所述M个网格顶点中的N个网格顶点对应，M和N均为大于1的整数，且N小于或等于M；所述第一形态对应的描述信息包括所述N个网格顶点的位置信息。
在一个实施例中，所述目标部位包括内轮廓和外轮廓，所述内轮廓与L个网格顶点对应，所述外轮廓与P个网格顶点对应，L和P之和小于N；
所述目标部位的轮廓信息包括所述内轮廓和所述外轮廓之间的第一距离、以及所述外轮廓对应的P个网格顶点中每两个网格顶点之间的第二距离；所第一距离包括所述L个网格顶点与所述P个网格顶点中具有对应关系的网格顶点之间的距离。
在一个实施例中，图9所述的图像处理装置还包括处理单元903：
获取单元901，还用于获取第一训练样本，所述第一训练样本中包括第一样本图像以及第一样本图像对应的标签信息；所述第一样本图像中包括对象面部的目标部位，所述标签信息包括所述第一样本图像中所述目标部位处于第一训练形态时对应的第一描述信息、以及所述第一样本图像中所述目标部位处于第二训练形态时对应的第二描述信息；所述第一训练形态与预期训练形态不匹配，所述第二训练形态与所述预期训练形态相匹配；
获取单元901，还用于获取所述第一样本图像中所述目标部位处于所述第一训练形态时的训练轮廓信息，并对所述训练轮廓信息和所述第一描述信息进行拼接处理；
处理单元903，用于调用待训练的所述参数预测模型基于拼接处理结果进行参数预测，得到所述形态控制器的训练参数；
基于所述训练参数进行描述信息预测，得到所述目标部位处于所述第二训练形态时对应的第一预测描述信息，根据所述第一预测描述信息和所述第二描述信息，对所述参数预测模型进行训练。
在一个实施例中，处理单元903在基于所述训练参数进行描述信息预测，得到所述目标部位处于所述第二训练形态时对应的第一预测描述信息时，执行如下步骤：
将所述训练参数作为位置预测模型的输入，调用所述位置预测模型进行描述信息预测，得到第一预测描述信息；所述位置预测模型用于根据形态控 制器的参数预测描述信息。
在一个实施例中，获取单元901，还用于获取第二训练样本，所述第二训练样本包括第二样本图像、以及第三描述信息与所述形态控制器的第二训练参数之间的对应关系，所述第二样本图像包括对象面部的目标部位，所述第三描述信息是指所述第二样本图像中所述目标部位处于第三形态时对应的描述信息；
处理单元902，还用于调用待训练的所述位置预测模型，基于所述第二训练参数进行描述信息预测，得到第二预测描述信息；基于所述第二预测描述信息和所述第三描述信息，对所述位置预测模型进行训练。
在一个实施例中，处理单元903在根据所述第一预测描述信息和所述第二描述信息之间的差异，对所述参数预测模型进行训练时，执行如下步骤：
基于所述第一预测描述位置信息和所述第二描述信息，确定所述参数预测模型对应的第一损失函数；
采用反向传播算法，基于所述第一损失函数更新所述参数预测模型中的网络参数。在一个实施例中，获取单元901在对所述训练轮廓信息和所述第一描述信息进行拼接处理时，用于执行如下步骤：
获取降维矩阵，并采用所述降维矩阵对所述第一描述信息进行降维处理；
对降维处理后的第一描述信息和所述训练轮廓信息进行拼接处理。
在一个实施例中，获取单元901在获取降维矩阵时，用于执行如下步骤：
对所述第一描述信息和所述第二描述信息进行拼接，得到描述信息矩阵；
对所述描述信息矩阵进行奇异值分解处理，得到特征向量矩阵和特征值矩阵；所述特征向量矩阵中包括多个特征向量；所述特征值矩阵包括每个特征向量矩阵对应的特征值；
对所述特征向量矩阵进行调整，调整后的特征向量矩阵中多个特征向量按照对应的特征值由大到小的顺序排列；
根据所述调整后的特征向量矩阵中每个特征向量的特征值，按序选取K个特征向量，将被选取的K个特征向量组成的矩阵确定为降维矩阵，所述K个特征向量对应的特征值之和大于或等于特征阈值。
根据本申请的一个实施例，图2和图6所示的图像处理方法所涉及各个步骤可以是由图9所示的图像处理装置中的各个单元来执行的。例如，图2所示的步骤S201和步骤S202可由图9所示的图像处理装置中的获取单元901来执行，步骤S203可由图9所示的图像处理装置中的修正单元902来执行再如，步骤S601和步骤S602可由图9所示的图像处理装置中的获取单元901来执行；步骤S603和步骤S604可由图9所示的图像处理装置中的处理单元903来执行。
根据本申请的另一个实施例，图9所示的图像处理装置中的各个单元可以分别或全部合并为一个或若干个另外的单元来构成，或者其中的某个(些) 单元还可以再拆分为功能上更小的多个单元来构成，这可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。上述单元是基于逻辑功能划分的，在实际应用中，一个单元的功能也可以由多个单元来实现，或者多个单元的功能由一个单元实现。在本申请的其它实施例中，基于图像处理装置也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由多个单元协作实现。
根据本申请的另一个实施例，可以通过在包括中央处理单元(CPU)、随机存取存储介质(RAM)、只读存储介质(ROM)等处理元件和存储元件的例如计算机的通用计算设备上运行能够执行如图2和图9所示的相应方法所涉及的各步骤的计算机程序(包括程序代码)，来构造如图9中所示的图像处理装置，以及来实现本申请实施例图像处理方法。所述计算机程序可以记载于例如计算机可读存储介质上，并通过计算机可读存储介质装载于图像处理设备中，并在其中运行。
本申请实施例中，在待处理图像中对象面部的目标部位处于第一形态，如果该第一形态与预期形态不匹配，则可以获取第一形态对应的描述信息、以及处于第一形态时目标部位的轮廓信息，进一步的，基于第一形态对应的描述信息和目标部位的轮廓信息对目标部位进行形态修正，以将目标部位的形态由第一形态修正为第二形态，该第二形态是与预期形态相匹配的，如此，无需人为手动对第一形态进行修正，而是根据第一形态的描述信息和第一形态时目标部位的轮廓信息，自动对目标部位进行形态修正，使得目标部位的形态符合预期形态。省去了人工手动修正带来的人力资源消耗，提高了形态修正的效率。
基于上述的图像处理方法实施例以及图像处理装置实施例，本申请实施例提供了一种计算机设备。参见图10，为本申请实施例提供的一种计算机设备的结构示意图，图10所示的计算机设备可包括处理器1001、输入接口1002、输出接口1003以及计算机存储介质1004。其中，处理器1001、输入接口1002、输出接口1003以及计算机存储介质1004可通过总线或其他方式连接。
计算机存储介质1004可以存储在图像处理设备的存储器中，所述计算机存储介质1004用于存储计算机程序，所述处理器1001用于执行所述计算机存储介质1004存储的计算机程序。处理器1001(或称CPU(Central Processing Unit，中央处理器))是计算机设备的计算核心以及控制核心，其适于实现一条或多条计算机程序，具体适于加载并执行本申请实施例提供的图像处理方法。
本申请实施例还提供了一种计算机存储介质(Memory)，所述计算机存储介质是计算机设备的记忆设备，用于存放程序和数据。可以理解的是，此处的计算机存储介质既可以包括计算机设备的内置存储介质，当然也可以包 括计算机设备所支持的扩展存储介质。计算机存储介质提供存储空间，该存储空间存储了计算机设备的操作系统。并且，在该存储空间中还存放了适于被处理器1001加载并执行的一条或多条的计算机程序。需要说明的是，此处的计算机存储介质可以是高速RAM存储器，也可以是非不稳定的存储器(non-volatile memory)，例如至少一个磁盘存储器；可选的还可以是至少一个位于远离前述处理器的计算机存储介质。
在一个实施例中，所述计算机存储介质中存储的一条或多条计算机程序可由处理器1001加载并执行本申请实施例提供的图像处理方法。

Claims (15)

  1. 一种图像处理方法，由计算机设备执行，包括：
    获取待处理图像，所述待处理图像中包括对象的目标部位，所述目标部位的形态为第一形态，所述第一形态与预期形态不匹配；
    获取所述第一形态对应的描述信息，以及获取处于所述第一形态时所述目标部位的轮廓信息；
    基于所述描述信息和所述轮廓信息对所述目标部位的形态进行修正，将所述目标部位的形态由所述第一形态修正为第二形态，所述第二形态与所述预期形态相匹配。
  2. 如权利要求1所述的方法，所述目标部位与形态控制器绑定，所述基于所述描述信息和所述轮廓信息对所述目标部位的形态进行修正，将所述目标部位的形态由第一形态修正为第二形态，包括：
    对所述描述信息和所述轮廓信息进行拼接处理；
    调用参数预测模型基于拼接处理结果进行参数预测，得到所述形态控制器的目标参数；
    调用所述形态控制器基于所述目标参数调整所述描述信息，调整后的描述信息用于使所述目标部位的形态呈现所述第二形态。
  3. 如权利要求2所述的方法，所述对所述描述信息和所述轮廓信息进行拼接处理，包括：
    获取降维矩阵，并采用所述降维矩阵对所述描述信息进行降维处理；
    对降维处理后的描述信息和所述轮廓信息进行拼接处理。
  4. 如权利要求1所述的方法，所述待处理图像与图像网格相关联，所述图像网格包括M个网格顶点，所述目标部位与所述M个网格顶点中的N个网格顶点对应，M和N均为大于1的整数，且N小于或等于M；
    所述第一形态对应的描述信息包括所述N个网格顶点的位置信息。
  5. 如权利要求4所述的方法，所述目标部位包括内轮廓和外轮廓，所述内轮廓与L个网格顶点对应，所述外轮廓与P个网格顶点对应，L和P之和小于N；
    所述目标部位的轮廓信息包括所述内轮廓和所述外轮廓之间的第一距离、以及所述外轮廓对应的P个网格顶点中每两个网格顶点之间的第二距离；所述第一距离包括所述L个网格顶点与所述P个网格顶点中具有对应关系的网格顶点之间的距离。
  6. 如权利要求2所述的方法，所述方法还包括：
    获取第一训练样本，所述第一训练样本中包括第一样本图像以及所述第一样本图像对应的标签信息；所述第一样本图像中包括对象的目标部位，所述标签信息包括所述第一样本图像中所述目标部位处于第一训练形态时对应的第一描述信息、以及所述第一样本图像中所述目标部位处于第二训练形态 时对应的第二描述信息；所述第一训练形态与预期训练形态不匹配，所述第二训练形态与所述预期训练形态相匹配；
    获取所述第一样本图像中所述目标部位处于所述第一训练形态时所述目标部位的训练轮廓信息，并对所述训练轮廓信息和所述第一描述信息进行拼接处理；
    调用待训练的所述参数预测模型基于拼接处理结果进行参数预测，得到所述形态控制器的训练参数；
    基于所述训练参数进行描述信息预测，得到所述目标部位处于所述第二训练形态时对应的第一预测描述信息，根据所述第一预测描述信息和所述第二描述信息，对所述参数预测模型进行训练。
  7. 如权利要求6所述的方法，所述基于所述训练参数进行描述信息预测，得到所述目标部位处于所述第二训练形态时对应的第一预测描述信息，包括：
    将所述训练参数作为位置预测模型的输入，调用所述位置预测模型进行描述信息预测，得到所述第一预测描述信息；所述位置预测模型用于根据形态控制器的参数预测描述信息。
  8. 如权利要求7所述的方法，所述方法还包括：
    获取第二训练样本，所述第二训练样本包括第二样本图像、以及第三描述信息与所述形态控制器的第二训练参数之间的对应关系，所述第二样本图像包括对象的目标部位，所述第三描述信息是指所述第二样本图像中所述目标部位处于第三训练形态时对应的描述信息；
    调用待训练的所述位置预测模型，基于所述第二训练参数进行描述信息预测，得到第二预测描述信息；
    基于所述第二预测描述信息和所述第三描述信息，对所述位置预测模型进行训练。
  9. 如权利要求6所述的方法，所述根据所述第一预测描述信息和所述第二描述信息，对所述参数预测模型进行训练，包括：
    基于所述第一预测描述位置信息和所述第二描述信息，确定所述参数预测模型对应的第一损失函数；
    采用反向传播算法，基于所述第一损失函数更新所述参数预测模型中的网络参数。
  10. 如权利要求6所述的方法，所述对所述训练轮廓信息和所述第一描述信息进行拼接处理，包括：
    获取降维矩阵，并采用所述降维矩阵对所述第一描述信息进行降维处理；
    对降维处理后的第一描述信息和所述训练轮廓信息进行拼接处理。
  11. 如权利要求10所述的方法，所述获取降维矩阵，包括：
    对所述第一描述信息和所述第二描述信息进行拼接，得到描述信息矩阵；
    对所述描述信息矩阵进行奇异值分解处理，得到特征向量矩阵和特征值 矩阵；所述特征向量矩阵中包括多个特征向量；所述特征值矩阵包括每个所述特征向量对应的特征值；
    对所述特征向量矩阵进行调整，调整后的特征向量矩阵中多个特征向量按照对应的特征值由大到小的顺序排列；
    根据所述调整后的特征向量矩阵中每个特征向量的特征值，按序选取K个特征向量，将所述K个特征向量组成的矩阵确定为所述降维矩阵，所述K个特征向量对应的特征值之和大于或等于特征阈值。
  12. 一种图像处理装置，包括：
    获取单元，用于获取待处理图像，所述待处理图像中包括对象的目标部位，所述目标部位的形态为第一形态，所述第一形态与预期形态不匹配；
    所述获取单元，还用于获取所述第一形态对应的描述信息，以及获取处于所述第一形态时所述目标部位的轮廓信息；
    修正单元，用于基于所述描述信息和所述轮廓信息对所述目标部位的形态进行修改，将所述目标部位的形态由所述第一形态修正为第二形态，所述第二形态与所述预期形态相匹配。
  13. 一种计算机设备，包括：
    处理器，适于实现一条或多条计算机程序；以及，
    计算机存储介质，所述计算机存储介质存储有一条或多条计算机程序，所述一条或多条计算机程序适于由所述处理器加载并执行如权利要求1-11任一项所述的图像处理方法。
  14. 一种计算机存储介质，所述计算机存储介质中存储有计算机程序，所述计算机程序被处理器执行时，用于执行如权利要求1-11任一项所述的图像处理方法。
  15. 一种计算机程序产品或计算机程序，所述计算机程序产品中包括计算机程序，所述计算机程序存储在计算机存储介质中，所述计算机存储介质中的计算机程序被处理器执行时，用于加载并执行如权利要求1-11任一项所述的图像处理方法。
PCT/CN2022/128637 2022-01-10 2022-10-31 图像处理方法、装置、设备、存储介质及计算机程序 WO2023130819A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/333,781 US20230368395A1 (en) 2022-01-10 2023-06-13 Image processing method, apparatus and device, storage medium and computer program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210024112.5 2022-01-10
CN202210024112.5A CN114373033A (zh) 2022-01-10 2022-01-10 图像处理方法、装置、设备、存储介质及计算机程序

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/333,781 Continuation US20230368395A1 (en) 2022-01-10 2023-06-13 Image processing method, apparatus and device, storage medium and computer program

Publications (1)

Publication Number Publication Date
WO2023130819A1 true WO2023130819A1 (zh) 2023-07-13

Family

ID=81144989

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/128637 WO2023130819A1 (zh) 2022-01-10 2022-10-31 图像处理方法、装置、设备、存储介质及计算机程序

Country Status (3)

Country Link
US (1) US20230368395A1 (zh)
CN (1) CN114373033A (zh)
WO (1) WO2023130819A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114373033A (zh) * 2022-01-10 2022-04-19 腾讯科技（深圳）有限公司 图像处理方法、装置、设备、存储介质及计算机程序

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080170777A1 (en) * 2007-01-16 2008-07-17 Lucasfilm Entertainment Company Ltd. Combining multiple session content for animation libraries
CN103914806A (zh) * 2013-01-09 2014-07-09 三星电子株式会社 用于调整拍摄的用户的眼睛的显示装置和控制方法
US9747716B1 (en) * 2013-03-15 2017-08-29 Lucasfilm Entertainment Company Ltd. Facial animation models
CN112132764A (zh) * 2020-09-27 2020-12-25 广州虎牙科技有限公司 脸型处理方法、装置、用户终端和计算机可读存储介质
CN114373033A (zh) * 2022-01-10 2022-04-19 腾讯科技（深圳）有限公司 图像处理方法、装置、设备、存储介质及计算机程序

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080170777A1 (en) * 2007-01-16 2008-07-17 Lucasfilm Entertainment Company Ltd. Combining multiple session content for animation libraries
CN103914806A (zh) * 2013-01-09 2014-07-09 三星电子株式会社 用于调整拍摄的用户的眼睛的显示装置和控制方法
US9747716B1 (en) * 2013-03-15 2017-08-29 Lucasfilm Entertainment Company Ltd. Facial animation models
CN112132764A (zh) * 2020-09-27 2020-12-25 广州虎牙科技有限公司 脸型处理方法、装置、用户终端和计算机可读存储介质
CN114373033A (zh) * 2022-01-10 2022-04-19 腾讯科技（深圳）有限公司 图像处理方法、装置、设备、存储介质及计算机程序

Also Published As

Publication number Publication date
US20230368395A1 (en) 2023-11-16
CN114373033A (zh) 2022-04-19

Similar Documents

Publication Publication Date Title
EP3944200A1 (en) Facial image generation method and apparatus, device and storage medium
US10540817B2 (en) System and method for creating a full head 3D morphable model
US11741668B2 (en) Template based generation of 3D object meshes from 2D images
CN108961369A (zh) 生成3d动画的方法和装置
KR102491140B1 (ko) 가상 아바타 생성 방법 및 장치
US11514638B2 (en) 3D asset generation from 2D images
CN115049799B (zh) 3d模型和虚拟形象的生成方法和装置
US11941737B2 (en) Artificial intelligence-based animation character control and drive method and apparatus
US20180276870A1 (en) System and method for mass-animating characters in animated sequences
WO2023155775A1 (zh) 一种运动画面生成方法、装置、计算机设备和存储介质
CN111127309A (zh) 肖像风格迁移模型训练方法、肖像风格迁移方法以及装置
WO2023130819A1 (zh) 图像处理方法、装置、设备、存储介质及计算机程序
CN114202615A (zh) 人脸表情的重建方法、装置、设备和存储介质
CN113380269B (zh) 视频图像生成方法、装置、设备、介质和计算机程序产品
WO2021106855A1 (ja) データ生成方法、データ生成装置、モデル生成方法、モデル生成装置及びプログラム
CN114373034A (zh) 图像处理方法、装置、设备、存储介质及计算机程序
CN115690276A (zh) 虚拟形象的视频生成方法、装置、计算机设备和存储介质
WO2022197385A1 (en) Layered clothing that conforms to an underlying body and/or clothing layer
WO2022026603A1 (en) Object recognition neural network training using multiple data sources
CN113222178A (zh) 模型训练方法、用户界面的生成方法、装置及存储介质
CN113223128A (zh) 用于生成图像的方法和装置
US20240013500A1 (en) Method and apparatus for generating expression model, device, and medium
KR102652652B1 (ko) 아바타 생성 장치 및 방법
US20240111894A1 (en) Generative machine learning models for privacy preserving synthetic data generation using diffusion
CN117011430A (zh) 游戏资源处理方法、装置、设备、存储介质和程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22918271

Country of ref document: EP

Kind code of ref document: A1