WO2018205801A1 - 人脸动画实现的方法、计算机设备及存储介质 - Google Patents

人脸动画实现的方法、计算机设备及存储介质 Download PDF

Info

Publication number
WO2018205801A1
WO2018205801A1 PCT/CN2018/083326 CN2018083326W WO2018205801A1 WO 2018205801 A1 WO2018205801 A1 WO 2018205801A1 CN 2018083326 W CN2018083326 W CN 2018083326W WO 2018205801 A1 WO2018205801 A1 WO 2018205801A1
Authority
WO
WIPO (PCT)
Prior art keywords
deformation coefficient
feature
standard
face
computer device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/083326
Other languages
English (en)
French (fr)
Inventor
陈静聪
王新亮
李斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Publication of WO2018205801A1 publication Critical patent/WO2018205801A1/zh
Priority to US16/509,305 priority Critical patent/US11087519B2/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/40—Three-dimensional [3D] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/175—Static expression

Definitions

  • the present application relates to the field of computer processing, and in particular, to a method, a computer device, and a storage medium for implementing facial animation.
  • Facial Animation refers to a technique that captures facial expressions through an expression capture sensor and then maps and transfers them to virtual animated characters. It has been widely used in movies, games, and other fields, with virtual reality in recent years ( With the development of VR) and Reality Enhancement (AR) technology, this technology has broad application prospects.
  • the implementation of traditional face animation is either expensive (for example, based on depth camera), difficult to apply to the general consumer field, or identifies the exact difference, and often requires the user to make a series of presets before use, for example, different entries.
  • the gestures and expressions of the photos are cumbersome, which also improves the user's threshold.
  • a method, a computer device, and a storage medium for implementing facial animation are provided.
  • a method for implementing face animation comprising:
  • the computer device collects a face image
  • the computer device extracts a face feature point in the face image
  • the computer device compares the facial feature point with a standard feature point to obtain a first deformation coefficient corresponding to the geometric feature
  • the computer device processes the local area according to the facial feature point to obtain a second deformation coefficient corresponding to the appearance feature
  • the computer device drives the three-dimensional virtual object to execute a corresponding expression according to the first deformation coefficient and the second deformation coefficient.
  • a computer device comprising a memory and one or more processors having stored therein computer readable instructions, the computer readable instructions being executed by the one or more processors such that the one or more The processor implements the following steps when executing the computer readable instructions:
  • One or more non-transitory computer readable storage media storing computer executable instructions, which when executed by one or more processors, cause the one or more processors to perform the following steps :
  • the three-dimensional virtual object is driven to execute a corresponding expression according to the first deformation coefficient and the second deformation coefficient.
  • FIG. 1 is a schematic diagram showing the internal structure of a computer device in an embodiment
  • FIG. 2 is a flow chart of a method for implementing face animation in an embodiment
  • FIG. 3 is a schematic diagram of a face feature point obtained by marker positioning in an embodiment
  • FIG. 4 is a flow chart of a method for extracting a local region according to a face feature point to obtain a second deformation coefficient corresponding to the appearance feature according to an embodiment
  • Figure 5 is a schematic illustration of HOG features of a local region of the left eye in one embodiment
  • Figure 6 is a schematic view showing the position of the center of the eyeball in one embodiment
  • FIG. 7 is a schematic view of calculating an angle of rotation of an eyeball in one embodiment
  • Figure 8 is a schematic view showing a affine deformation of a triangle in one embodiment
  • Figure 9 is a schematic diagram of conversion to a standard face image in one embodiment
  • FIG. 10 is a flow chart of a method for determining a third deformation coefficient corresponding to a local region according to a texture feature in one embodiment
  • FIG. 11 is a schematic diagram of the principle of an SVM classifier in an embodiment
  • FIG. 12 is a flowchart of a method for comparing a face feature point with a standard feature point to obtain a first deformation coefficient corresponding to a geometric feature in an embodiment
  • FIG. 13 is a flowchart of a method for comparing a three-dimensional coordinate corresponding to a face feature point with a standard three-dimensional coordinate corresponding to a standard feature point to obtain a first deformation coefficient corresponding to the geometric feature;
  • FIG. 14 is a flow chart of a method for driving a three-dimensional virtual object to perform a corresponding expression according to a first deformation coefficient and a second deformation coefficient in one embodiment
  • Figure 15 is a schematic diagram showing the principle of smoothing of a least squares filter in one embodiment
  • Figure 16 is a schematic diagram of comparison before and after smoothing in one embodiment
  • 17 is a schematic diagram of performing a corresponding expression on a three-dimensional virtual object in one embodiment
  • 19 is a block diagram showing the structure of an apparatus for implementing face animation in an embodiment
  • FIG. 20 is a structural block diagram of a processing module in an embodiment
  • Figure 21 is a block diagram showing the structure of the comparison module in one embodiment.
  • the computer device may be a terminal, the terminal may be a personal computer or a mobile electronic device, and the mobile electronic device includes at least one of a mobile phone, a tablet computer, a personal digital assistant, or a wearable device.
  • the computer device includes a processor, a network interface, a display screen, an input device, and an image capture device connected by a system bus.
  • the memory comprises a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium of the computer device can store an operating system and computer readable instructions that, when executed, cause the processor to perform a method of facial animation implementation.
  • the processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device.
  • the internal memory can store computer readable instructions that, when executed by the processor, cause the processor to perform a method of face animation implementation.
  • the network interface of the computer device is used for network communication.
  • the display screen of the computer device may be a liquid crystal display or an electronic ink display screen
  • the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touchpad provided on the computer device casing, and It can be an external keyboard, trackpad or mouse.
  • the touch layer and display form a touch screen.
  • the image capture device can be a camera. It will be understood by those skilled in the art that the structure shown in FIG.
  • FIG. 1 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • a method for implementing facial animation is proposed.
  • the method is applied to a computer device, and specifically includes the following steps:
  • Step 202 Acquire a face image.
  • a face image refers to an image containing facial information of a person.
  • the collection of the face image can be collected by the front or rear camera in the terminal.
  • the method of capturing can be obtained by taking a photo or by taking a video, because the video itself is composed of a sequence of images.
  • Step 204 Extract a face feature point in the face image.
  • the face feature point is also called a face key point, and refers to a face position in the face area capable of embodying an expression state, including but not limited to a face position such as an eye, a mouth, a nose, and an eyebrow.
  • the face mark location technology can be used to extract the face feature points in the face image, which can be divided into two steps, one is face detection and the other is face mark. Firstly, the rough position of the face in the image is obtained by face detection, which is generally a rectangular frame that frames the face, and then based on the rectangular frame, the face mark is used to find a more precise position, and then returns to a series of people. The coordinates of the face feature point position. As shown in FIG.
  • FIG. 3 it is a schematic diagram of a face feature point obtained by marker positioning in one embodiment.
  • the method for locating the face mark can adopt an existing method, for example, an AAM (Active Appearance Models), an ERT (Ensemble of Regression Tree), and the like can be used. There is no limit to the method of face mark positioning.
  • step 206 the face feature points are compared with the standard feature points to obtain corresponding first deformation coefficients of the geometric features.
  • the geometric feature refers to a feature in which the feature points of the face become geometrical, for example, when the mouth is opened, the face feature points near the mouth are significantly displaced and deformed.
  • the change of the geometric features of the face needs to be compared with the standard feature points, that is, a reference value is needed.
  • the face feature point corresponding to the neutral face is used as the standard feature point, and the neutral face refers to the face state of the person without expression.
  • the displacement and deformation of each part of the face can be obtained, and then the deformation coefficient corresponding to each part can be determined, for example, the coefficient of eyebrow elevation, the mouth The coefficient of expansion, etc., the deformation coefficient corresponding to each part is the deformation coefficient of the geometric feature.
  • the deformation coefficient on the geometric feature is called the "first deformation coefficient".
  • Step 208 Extract a local area according to the facial feature point to perform processing to obtain a second deformation coefficient corresponding to the appearance feature.
  • the appearance feature refers to a change in the appearance of the face, and it is difficult to identify the feature only with geometric features. For example, blinking, sticking out tongues, etc. are difficult to identify with geometric features, so it is necessary to use a feature based recognition method to identify changes in the appearance features.
  • the facial features can be identified by extracting the facial partial images. The corresponding deformation coefficient is determined according to the change in the appearance feature identified, which is called "second deformation coefficient". Specifically, the local region is extracted according to the face feature point, and then the texture feature corresponding to the local region is calculated, and the trained classification model is used to predict the recognition result corresponding to the local region according to the texture feature.
  • the deformation coefficient can be set to 0, if For the split, set the deformation factor to 100.
  • Step 210 Drive the three-dimensional virtual object to execute the corresponding expression according to the first deformation coefficient and the second deformation coefficient.
  • the three-dimensional virtual object refers to a network virtual animation (Avatar), wherein Avatar is a 3D image.
  • Avatar is a 3D image.
  • the three-dimensional virtual object can be a cartoon image or a real-life image.
  • the expression of the captured face is displayed by the three-dimensional virtual object.
  • the calculated first deformation coefficient and the second deformation coefficient are used as deformation parameters of the three-dimensional virtual object model to control the three-dimensional virtual object to execute the corresponding expression.
  • the face of the three-dimensional virtual object is controlled to perform a corresponding expression by recognizing the obtained first deformation coefficient and the second deformation coefficient, for example, performing corresponding smiles, open mouths, blinks, and the like.
  • the animation rendering engine controls the face of the three-dimensional virtual object to execute the corresponding expression, thereby realizing the restoration of the facial expression to the face of the three-dimensional virtual object (Avatar).
  • the expression of the three-dimensional virtual object is controlled by values corresponding to a number of Blendshapes.
  • Blendshape is used to control facial detail expressions.
  • dozens of Blendshapes can be set on the face.
  • Each Blendshape only controls one facial detail.
  • the corners of the eyes, the mouth, and the corners of the mouth can be controlled by different Blendshapes.
  • Each Blendshape is taken.
  • the value range is 0-100.
  • Very complex facial expressions can be synthesized with dozens of Blendshape combinations.
  • the above process of recognizing the face image is to determine the value of the Blendshape corresponding to each part of the face, and by combining the geometric feature based on the appearance feature, a more accurate recognition effect can be obtained, thereby according to the first deformation coefficient and the second deformation.
  • Coefficients can drive 3D virtual objects to perform more detailed expressions.
  • the least squares filter method can be used to smooth the corresponding deformation coefficients.
  • the face image is captured by the camera, and then the face feature points in the face image are extracted, and the corresponding feature points are obtained by comparing the standard feature points corresponding to the face feature points with the neutral face.
  • a deformation coefficient because the geometric feature-based recognition method is unrecognizable or unrecognizable for some regions, it is also necessary to extract the local region according to the facial feature points for processing to obtain the second deformation coefficient corresponding to the appearance feature, and then, The three-dimensional virtual object is driven to perform a corresponding expression according to the first deformation coefficient and the second deformation coefficient.
  • the user does not need to make any preset values in the whole process, and the operation is simple, and a more accurate recognition effect can be obtained by combining the recognition on the finer appearance features, and the method can be realized by using an ordinary monocular camera, which greatly reduces the The cost of use.
  • the second strain coefficient includes a third deformation coefficient and a fourth deformation coefficient.
  • the step 208 of extracting the local region according to the facial feature point and performing the processing to obtain the second deformation coefficient corresponding to the appearance feature includes:
  • Step 208A extracting a local area from the face image according to the face feature point.
  • the recognition of the geometric features is identified based on the geometric deformation of the face feature points, it is difficult to identify for some of the more subtle expressions, such as whether it is blinking, whether the tongue is stretched or the like is difficult to recognize. Therefore, it is also necessary to identify the appearance features of some areas that are not easily identifiable.
  • the face feature point can identify the position of the face, including the eyes, the mouth, the nose, the eyebrows, etc., according to the face feature points, the corresponding partial region can be extracted from the face image, for example, the eye region is extracted. The left eye area and the right eye area can be extracted separately.
  • Step 208B calculating a texture feature corresponding to the local area.
  • a texture feature refers to a feature parameter extracted by a certain image processing technique that can represent the image attribute, and is used to describe information of the image. After the local area to be identified is extracted from the face image, the texture features corresponding to the local area are calculated, so as to facilitate subsequent classification according to the texture features.
  • a HOG Heistogram of Oriented Gradient
  • the child which computes and counts the gradient direction histogram of the local region of the image, in which the appearance and shape of the local target can be well described by the gradient density of the gradient or edge. As shown in FIG.
  • the left eye partial image (the left eye is closed) is visualized by the computed HOG feature. It can be seen from the figure that the horizontal gradient distribution of the middle region is more, and the image of the closed eye has a distinct horizontal edge in the middle, so the HOG feature can well reflect the local texture features of the image.
  • Step 208C Determine a third deformation coefficient corresponding to the local region according to the texture feature.
  • the trained classifier is used to predict the appearance features of the local regions according to the texture features, wherein the classifier learns the texture features in the mark samples. owned. For example, if the local area is an eye, the trained classifier can be used to predict whether to close the eye according to the obtained texture feature, and then determine the third deformation coefficient corresponding to the local area. If the result is closed, the corresponding deformation coefficient can be set to 0. If the result is blink, the corresponding deformation coefficient can be set to 100.
  • step 208D the eyeball tracking algorithm is used to determine the center position of the eyeball, and the rotation angle of the eyeball relative to the head-up state is calculated according to the center position of the eyeball, and the fourth deformation coefficient is determined according to the rotation angle.
  • the recognition on the appearance feature further includes recognizing the rotation of the eyeball, using the eyeball in the eye-level state as a reference standard, and determining the center position of the eyeball by using an eyeball tracking algorithm, and then according to the eyeball
  • the center position calculates the angle of rotation of the eyeball relative to the head-up state.
  • the left eyeball is calculated to rotate 25 degrees on the X-axis and 5 degrees on the Y-axis.
  • the fourth deformation coefficient is determined according to the rotation angle. For example, 25 and 5 can be directly used as the corresponding deformation coefficients.
  • the center of the eyeball is determined. As shown in Fig.
  • the transposed matrix representing di is only the center of the eyeball center position on the 2D image, and based on this, it is also necessary to estimate the rotation angle ⁇ of the eyeball.
  • the camera is a weak perspective model based on the pinhole, and the vertical direction (Y-axis) of the head relative to the focal plane of the camera is ⁇ .
  • the projection coordinates of the eye angle x direction are x1 and x2, and the center of the eyeball in the case of the eye plane is projected in the X direction of the focal plane as x', and the center of the eyeball is projected as x in the X direction of the focal plane, assuming that the radius of the eyeball is r, application geometry can be obtained ⁇ , the formula is as follows: Where a is a constant. ⁇ is the obtained rotation angle on the X-axis. Based on the same method, the rotation angle on the Y-axis can be obtained, and the corresponding fourth deformation coefficient is determined according to the calculated rotation angle.
  • the step 208A of extracting a partial region from the face image according to the face feature point includes: transforming the face image into a standard face image using the segmental affine deformation according to the face feature point, from the standard face A local area is extracted from the image.
  • the face images have different angles and postures, directly using the images may result in poor image processing robustness, so it is necessary to transform the face image into a standard face image through deformation, that is, a face
  • the image is deformed to a standard shape and then the local area of the image is extracted.
  • the face image can be converted into a standard face image according to the face feature point using a Piecewise Affine Warping method.
  • the extracted face feature points may be set to 68.
  • the standard triangle mesh corresponding to the standard face image may be preset according to the preset division rule, and after the face feature point of the face image is extracted, the face feature point corresponding according to the face feature point is obtained according to the same connection rule.
  • the triangular mesh has a one-to-one correspondence between the triangular mesh and the standard triangular mesh because the two are divided in the same way. Then an affine deformation is made for each triangle of the pixel, that is, each pixel (x, y) T of the source triangle mesh is deformed by affine to W(x, p), as shown in FIG.
  • a schematic diagram of a affine deformation of a triangle The acquired face image can be converted into a standard face image by performing affine deformation on each triangular pixel. As shown in FIG. 9, the triangular mesh corresponding to the face feature point in one embodiment is converted into A schematic diagram of a standard triangle mesh, and below is a schematic diagram of a corresponding face image being converted into a standard face image.
  • the step 208C of determining the third deformation coefficient corresponding to the local region according to the texture feature comprises:
  • Step 1002 The trained classifier is used to determine the recognition result corresponding to the local area according to the texture feature, and the classifier is obtained by learning the texture feature in the mark sample.
  • the pre-trained classifier is used to predict the recognition result corresponding to the local region according to the calculated texture feature.
  • the classifier is obtained by learning the texture features in the mark samples, and by discovering the rules, and predicting which class the new samples belong to according to the learned rules.
  • the prediction based on the appearance feature can be viewed as a two-category question, such as whether to close the eye, whether to stick the tongue, and the like.
  • algorithms such as decision tree, SVM (support vector machine), and neural network.
  • a training sample is obtained, and each sample is marked as belonging to which category, for example, a face image is a sample, and whether the eye is closed in the face image is a corresponding mark; in order to improve the robustness of the prediction,
  • the sample is also subjected to pre-processing, wherein the pre-processing includes mirroring the extracted partial image (Y-axis rotation), and the purpose of the image processing is to improve the effect and speed of the training. Because the face can be seen as bilaterally symmetric, in order to improve the speed of training, the other side can be predicted simultaneously by mirroring.
  • the HOG feature of each sample local region is extracted.
  • the calculated HOG feature can be normalized.
  • the normalization method can use the sample value minus the mean value and divide by the standard deviation.
  • the corresponding formula is as follows: (X- ⁇ )/ ⁇ , where X is the sample value, ⁇ is the mean value, and ⁇ is the standard deviation. This formalization method can solve the numerical stability problem and can prevent some features from being too large.
  • the HOG features in the tag samples are learned and the rules are learned until it is possible to accurately predict which class the new samples belong to according to the learned rules.
  • the SVM constructs a hyperplane during the learning process to maximize the interval between the two types of samples. The larger the interval, the more accurate the corresponding prediction effect.
  • the sample is two-dimensional, it is a straight line, three-dimensional is a plane, and the three-dimensional position is a hyperplane.
  • Step 1004 Determine a third deformation coefficient corresponding to the texture feature according to the recognition result.
  • the trained localizer is used to predict the extracted local region according to the texture feature, output a corresponding recognition result, and determine a third deformation coefficient corresponding to the texture feature according to the recognition result.
  • the output result is two values of 1 or -1 (corresponding to true and false), for example, whether to predict whether to close the eye, if the output result is 1, it means closed eyes, if the output is -1, it means blinking.
  • the corresponding coefficient of deformation of the closed eye can be set to 0, and the deformation coefficient of the blink is 100.
  • a probability model in order to obtain more accurate prediction results, can be trained to represent the probability that the output is true, not just the outputs 1 and -1, which facilitates subsequent adjustment of the threshold to improve the accuracy of the prediction. Sex. For example, for some users with small eyes, the HOG features of blinking and closed eyes may not be obvious, so it is easy to misjudge when predicting, so if you get a probability value of closed eyes, then adjust the closed
  • the threshold of the eye can improve the prediction accuracy. For example, the threshold of the original closed eye is set to 60%, that is, when the probability value of the closed eye is predicted to be greater than 60%, the eye is determined to be closed.
  • the threshold of the closed eye for example, adjust the threshold to 70%, so that only when the predicted closed eye probability value is greater than 70%, it is determined that the eye is closed, that is, judge The conditions for closing the eyes are more stringent.
  • f) 1 / [1 + exp(Af + B)], where A and B are the fitted parameters.
  • f is the output of the original SVM classifier
  • P is the probability that the predicted result is true.
  • the step 206 of comparing the face feature points with the standard feature points to obtain the first first deformation coefficient corresponding to the geometric features includes:
  • Step 206A calculating three-dimensional coordinates corresponding to the face feature points.
  • the obtained 2D face image is converted into a human.
  • the 3D image of the face is then compared, and the three-dimensional coordinates corresponding to the face feature points and the three-dimensional coordinates corresponding to the standard feature points are specifically compared. Therefore, it is necessary to calculate the three-dimensional coordinates corresponding to the feature points of the face.
  • the two-dimensional coordinates corresponding to the face feature points are obtained, and then the 3D coordinates corresponding to the two-dimensional coordinates of the face feature points are acquired by using the 3D reconstruction technique.
  • the three-dimensional coordinates corresponding to the face feature points can be calculated by the above formula.
  • the three-dimensional coordinates corresponding to the face feature points are subjected to certain scaling, rotation, displacement, and local deformation, and then the two-dimensional coordinates are projected to correspond to the squared sum of the errors between the previously calculated actual two-dimensional coordinates y i .
  • Step 206B Compare the three-dimensional coordinates corresponding to the obtained facial feature points with the standard three-dimensional coordinates corresponding to the standard feature points to obtain the first deformation coefficient corresponding to the geometric feature.
  • the calculated three-dimensional coordinates are compared with the three-dimensional feature points corresponding to the standard feature points, and the change values of the respective parts of the face relative to the standard three-dimensional coordinates are obtained. For example, the change in the displacement of the eyebrows relative to the standard three-dimensional coordinates, the change in the width of the mouth relative to the standard three-dimensional coordinates, and the like.
  • Corresponding deformation coefficients are determined according to the obtained variation values, thereby determining a first deformation coefficient corresponding to the geometric features.
  • the step 206B of comparing the obtained three-dimensional coordinates of the face feature points with the standard three-dimensional coordinates corresponding to the standard feature points to obtain the first first deformation coefficient corresponding to the geometric features includes:
  • Step 1302 Calculate standard three-dimensional coordinates corresponding to the standard feature points.
  • the standard feature point refers to the standard feature point corresponding to the neutral face, so the standard three-dimensional coordinates are actually the three-dimensional coordinates corresponding to the neutral face, and the neutral face refers to the face state of the person in the absence of expression.
  • the calculation of the standard three-dimensional coordinates can use the median-based online prediction algorithm. Of course, other methods can also be used. For example, the three-dimensional coordinates corresponding to the recorded neutral face can be directly used as the standard three-dimensional coordinates, and the standard three-dimensional coordinates are not calculated here.
  • the method is limited. The following is an example based on the median online prediction method. Specifically, the video data on a window of time (for example, 100 frames of video data) is acquired, and the facial feature points in each frame of the 100 frames of video data are obtained. Corresponding three-dimensional coordinates, the three-dimensional coordinates of the face feature points at the same position are sorted in order from small to large or large to small, and then the intermediate value is taken as the standard three-dimensional coordinates.
  • the method is based on a simple assumption that the user is expressionless for most of the time, so after sorting by size, the data at both ends is treated as a special state with expressions, and the median is expressionless neutral. status.
  • each frame of video data corresponds to a face image, and the face feature points in the face image are extracted, and the three-dimensional coordinates corresponding to the face feature points are calculated.
  • the number of face feature points corresponding to the face image is the same, and the three-dimensional coordinates corresponding to the face feature points at the same position are sorted, for example, the three-dimensional coordinates (X, Y, Z) of a feature point corresponding to the mouth. Sorting, the default extracted video data is in an expressionless state in most cases, so after sorting the three-dimensional coordinates, the intermediate value is taken as the standard three-dimensional coordinates. And so on, the standard three-dimensional coordinates corresponding to all feature points are obtained.
  • Step 1304 Compare the three-dimensional coordinates corresponding to the obtained facial feature points with the standard three-dimensional coordinates to determine a change value corresponding to each part of the face.
  • the standard three-dimensional coordinates can be used as a reference value to calculate the change of each face feature point in the user's face relative to the standard three-dimensional coordinates, thereby determining the face.
  • the change value corresponding to each part of the part for example, the change of the displacement of the eyebrows, the mouth, and the like.
  • Step 1306 normalizing the determined change value to obtain a first deformation coefficient.
  • the displacement is normalized to between 0 and 1.
  • the step 210 of driving the three-dimensional virtual object to execute the corresponding expression according to the first deformation coefficient and the second deformation coefficient comprises:
  • step 210A the first deformation coefficient and the second deformation coefficient are subjected to coefficient smoothing processing by a least squares filter method.
  • the three-dimensional coordinates calculated according to the two-dimensional coordinates may have local jitter between the video frames, so that the predicted deformation coefficient is not Smooth, seriously affecting the animation effect. Therefore, it is necessary to smooth the obtained deformation coefficient.
  • the first deformation coefficient and the second deformation coefficient are subjected to coefficient smoothing processing using a least squares filter method.
  • the specific processing method is as follows: As shown in FIG. 15, assuming that the filter processing window is 5 frames, it is necessary to retain 4 frames of historical data.
  • the point y' at which the time t falls on the straight line is the result of the filtered output, the filter is continuously applied for each frame, and the history data is updated to obtain a very smooth coefficient.
  • FIG. 16 it is an effect of smoothing the rotation coefficient of the head in an X direction, wherein the black thin line in the figure indicates the original coefficient, the jitter is large, and the thick black line indicates the result after the smoothing process, Smooth a lot.
  • step 210B the first deformation coefficient and the second deformation coefficient processed by the coefficient smoothing are used to drive the three-dimensional virtual object to execute the corresponding expression.
  • the three-dimensional virtual object refers to a virtual animation (Avatar), which may be a cartoon image or a real-life image.
  • the first deformation coefficient and the second deformation coefficient processed by the coefficient smoothing drive the three-dimensional virtual object to execute the corresponding expression.
  • the deformation coefficient can be represented by a Blendshape. The above process of recognizing a facial expression is to determine the value corresponding to each Blendshape in the face, and combine the actual expression of the adult according to the value of each Blendshape.
  • FIG. 17 is an expression of a blink (right eye closed) displayed by a three-dimensional virtual object in one embodiment.
  • FIG. 18 is a flow diagram of a process for implementing face animation in an embodiment.
  • the face feature points of the collected face images are extracted by the face mark location technology, and then the geometric feature and the appearance feature are respectively identified according to the face feature points.
  • the process of identifying the geometric features includes: adopting The 3D reconstruction technology calculates the corresponding three-dimensional coordinates according to the extracted two-dimensional coordinates of the face feature points, and calculates the standard three-dimensional coordinates corresponding to the neutral face, and compares the three-dimensional coordinates with the standard three-dimensional coordinates corresponding to the neutral face.
  • the change value corresponding to each part of the face is obtained, and the change value is converted into a normalized coefficient by coefficient conversion, thereby determining the corresponding first deformation coefficient.
  • the recognition process on the appearance features includes: obtaining a standard face image by using affine deformation according to the face feature point, and then extracting the local region in the face image, and then extracting the HOG feature of the local region by calculation, and adopting the training well
  • the SVM classifier obtains the corresponding prediction result and determines the third deformation coefficient.
  • the eyeball tracking algorithm is used to perform eyeball tracking to determine the center position of the eyeball, and the rotation angle of the eyeball relative to the head-up state is calculated according to the center position of the eyeball, and is determined according to the rotation angle.
  • the fourth deformation coefficient is used to perform eyeball tracking to determine the center position of the eyeball, and the rotation angle of the eyeball relative to the head-up state is calculated according to the center position of the eyeball, and is determined according to the rotation angle.
  • the fourth deformation coefficient is used to perform eyeball tracking to determine the center position of the eyeball, and the rotation angle of the eyeball relative to the head-up state is calculated according to the center position of the
  • the first deformation coefficient and the second deformation coefficient are smoothed by using a least squares filter method, and the three-dimensional virtual object is driven according to the first deformation coefficient and the second deformation coefficient of the smoothing process to execute the corresponding expression, that is, generating Deformation animation.
  • a computer device is further provided, and the internal structure of the computer device is as shown in FIG. 1 , the computer device includes a device implemented by a face animation, and the device implemented by the face animation includes each module.
  • Each module may be implemented in whole or in part by software, hardware, or a combination thereof.
  • an apparatus for implementing facial animation comprising:
  • the acquisition module 1902 is configured to collect a face image.
  • the extraction module 1904 is configured to extract a facial feature point in the face image.
  • the matching module 1906 is configured to compare the facial feature points with the standard feature points to obtain corresponding first deformation coefficients of the geometric features.
  • the processing module 1908 is configured to extract a local region according to the facial feature point for processing to obtain a second deformation coefficient corresponding to the appearance feature.
  • the driving module 1910 is configured to drive the three-dimensional virtual object to execute the corresponding expression according to the first deformation coefficient and the second deformation coefficient.
  • the second strain coefficient includes a third strain coefficient and a fourth strain coefficient.
  • the processing module 1908 includes:
  • the local area extraction module 1908A is configured to extract a local area from the face image according to the facial feature point.
  • the texture feature calculation module 1908B is configured to calculate a texture feature corresponding to the local region.
  • the third deformation coefficient determining module 1908C is configured to determine a third deformation coefficient corresponding to the local region according to the texture feature.
  • the fourth deformation coefficient determining module 1910D is configured to determine an eyeball center position by using an eyeball tracking algorithm, calculate a rotation angle of the eyeball relative to the heading state according to the eyeball center position, and determine a fourth deformation coefficient according to the rotation angle.
  • the local region extraction module 1908A is further configured to convert the facial image into a standard facial image by segmental affine deformation according to the facial feature point, and extract the local region from the standard facial image.
  • the third deformation coefficient determining module 1908C is further configured to determine, by using the trained classifier, a recognition result corresponding to the local region according to the texture feature, where the classifier is obtained by learning the texture feature in the mark sample, according to The recognition result determines a third deformation coefficient corresponding to the texture feature.
  • the comparison module 1906 includes:
  • the three-dimensional coordinate calculation module 1906A is configured to calculate three-dimensional coordinates corresponding to the facial feature points.
  • the three-dimensional coordinate matching module 1906B is configured to compare the three-dimensional coordinates corresponding to the obtained facial feature points with the standard three-dimensional coordinates corresponding to the standard feature points to obtain a first deformation coefficient corresponding to the geometric feature.
  • the three-dimensional coordinate comparison module 1906A is further configured to calculate standard three-dimensional coordinates corresponding to the standard feature points, and compare the obtained three-dimensional coordinates corresponding to the face feature points with the standard three-dimensional coordinates to determine corresponding parts of the face.
  • the change value is normalized to the determined change value to obtain a first deformation coefficient.
  • the driving module 1910 is further configured to perform coefficient smoothing processing on the first deformation coefficient and the second deformation coefficient by using a least squares filter method, and drive the first deformation coefficient and the second deformation coefficient after the coefficient smoothing process.
  • the 3D virtual object performs the corresponding expression.
  • a computer apparatus comprising a memory and one or more processors having stored therein computer readable instructions that, when executed by one or more processors, cause one or more
  • the processor executes the computer readable instructions, the following steps are performed: collecting a face image; extracting a face feature point in the face image; comparing the face feature point with the standard feature point to obtain a first deformation coefficient corresponding to the geometric feature And extracting a local region according to the face feature point to obtain a second deformation coefficient corresponding to the appearance feature; driving the three-dimensional virtual object to execute the corresponding expression according to the first deformation coefficient and the second deformation coefficient.
  • the second deformation coefficient includes a third deformation coefficient and a fourth deformation coefficient; and the processor performs a process of extracting the local region according to the facial feature point to obtain a second deformation coefficient corresponding to the appearance feature, including: according to the person The face feature points extract the local region from the face image; calculate the texture feature corresponding to the local region; determine the third deformation coefficient corresponding to the local region according to the texture feature; determine the eyeball center position by using the eyeball tracking algorithm, and calculate the eyeball relative to the eyeball center position
  • the rotation angle in the head-up state determines the fourth deformation coefficient according to the rotation angle.
  • the processor extracts the local region from the facial image according to the facial feature point, including: transforming the facial image into a standard facial image according to the facial feature point using the segmental affine deformation; The local area is extracted from the face image.
  • determining, by the processor, the third deformation coefficient corresponding to the local region according to the texture feature comprises: determining, by using the trained classifier, the recognition result corresponding to the local region according to the texture feature, and the classifier is in the labeled sample The texture feature is learned; and the third deformation coefficient corresponding to the texture feature is determined according to the recognition result.
  • the processor performs comparing the facial feature points with the standard feature points to obtain the first first deformation coefficient corresponding to the geometric features, including: calculating the three-dimensional coordinates corresponding to the facial feature points; and obtaining the human face The three-dimensional coordinates corresponding to the feature points are compared with the standard three-dimensional coordinates corresponding to the standard feature points to obtain corresponding first deformation coefficients of the geometric features.
  • the three-dimensional coordinates corresponding to the obtained feature points of the face are compared with the standard three-dimensional coordinates corresponding to the standard feature points, and the first deformation coefficient corresponding to the geometric feature is obtained, including: calculating the standard feature points.
  • the three-dimensional coordinates corresponding to the obtained face feature points are compared with the standard three-dimensional coordinates to determine corresponding change values of the respective parts of the face; and the determined change values are normalized to obtain the first deformation coefficient.
  • the processor performs driving the three-dimensional virtual object according to the first deformation coefficient and the second deformation coefficient to execute the corresponding expression, including: performing coefficients on the first deformation coefficient and the second deformation coefficient by using a least square filter method Smoothing processing; the first deformation coefficient and the second deformation coefficient processed by the coefficient smoothing drive the three-dimensional virtual object to execute the corresponding expression.
  • one or more non-transitory computer readable storage media having computer readable instructions stored by one or more processors are provided, such that one or more The processor performs the following steps: acquiring a face image by computer readable instructions; extracting a face feature point in the face image; comparing the face feature point with the standard feature point to obtain a first deformation coefficient corresponding to the geometric feature; The facial feature points extract the local regions for processing to obtain the second deformation coefficients corresponding to the appearance features; and the three-dimensional virtual objects are driven according to the first deformation coefficient and the second deformation coefficient to execute the corresponding expressions.
  • the second deformation coefficient includes a third deformation coefficient and a fourth deformation coefficient; and the processor performs a process of extracting the local region according to the facial feature point to obtain a second deformation coefficient corresponding to the appearance feature, including: according to the person The face feature points extract the local region from the face image; calculate the texture feature corresponding to the local region; determine the third deformation coefficient corresponding to the local region according to the texture feature; determine the eyeball center position by using the eyeball tracking algorithm, and calculate the eyeball relative to the eyeball center position
  • the rotation angle in the head-up state determines the fourth deformation coefficient according to the rotation angle.
  • the processor extracts the local region from the facial image according to the facial feature point, including: transforming the facial image into a standard facial image according to the facial feature point using the segmental affine deformation; The local area is extracted from the face image.
  • determining, by the processor, the third deformation coefficient corresponding to the local region according to the texture feature comprises: determining, by using the trained classifier, the recognition result corresponding to the local region according to the texture feature, and the classifier is in the labeled sample The texture feature is learned; and the third deformation coefficient corresponding to the texture feature is determined according to the recognition result.
  • the processor performs comparing the facial feature points with the standard feature points to obtain the first first deformation coefficient corresponding to the geometric features, including: calculating the three-dimensional coordinates corresponding to the facial feature points; and obtaining the human face The three-dimensional coordinates corresponding to the feature points are compared with the standard three-dimensional coordinates corresponding to the standard feature points to obtain corresponding first deformation coefficients of the geometric features.
  • the three-dimensional coordinates corresponding to the obtained feature points of the face are compared with the standard three-dimensional coordinates corresponding to the standard feature points, and the first deformation coefficient corresponding to the geometric feature is obtained, including: calculating the standard feature points.
  • the three-dimensional coordinates corresponding to the obtained face feature points are compared with the standard three-dimensional coordinates to determine corresponding change values of the respective parts of the face; and the determined change values are normalized to obtain the first deformation coefficient.
  • the processor performs driving the three-dimensional virtual object according to the first deformation coefficient and the second deformation coefficient to execute the corresponding expression, including: performing coefficients on the first deformation coefficient and the second deformation coefficient by using a least square filter method Smoothing processing; the first deformation coefficient and the second deformation coefficient processed by the coefficient smoothing drive the three-dimensional virtual object to execute the corresponding expression.
  • the various steps in the various embodiments of the present application are not necessarily performed in the order indicated by the steps. Except as explicitly stated herein, the execution of these steps is not strictly limited, and the steps may be performed in other orders. Moreover, at least some of the steps in the embodiments may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be executed at different times, and the execution of these sub-steps or stages The order is also not necessarily sequential, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of the other steps.
  • Non-volatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM) or external cache memory.
  • RAM is available in a variety of formats, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization chain.
  • Synchlink DRAM SLDRAM
  • Memory Bus Radbus
  • RDRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本申请提出了一种人脸动画实现的方法，包括：采集人脸图像；提取所述人脸图像中的人脸特征点；将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数；根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情。

Description

人脸动画实现的方法、计算机设备及存储介质
本申请要求于2017年5月12日提交中国专利局、申请号为2017103355095、发明名称为“人脸动画实现的方法、装置、计算机设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机处理领域，特别是涉及一种人脸动画实现的方法、计算机设备及存储介质。
背景技术
人脸动画(Facial Animation)是指通过表情捕捉传感器捕捉人脸表情，然后映射、转移到虚拟动画人物的一种技术，其已经广泛应用于电影、游戏等领域，随着最近几年虚拟现实(VR)和现实增强(AR)技术的发展，该技术更是有广泛的应用前景。传统的人脸动画的实现，要么设备昂贵(比如，需要基于深度摄像头)，很难应用于普通消费领域；要么识别精确差，且往往使用前需要用户做一系列的预设置，比如，录入不同姿态、表情的照片，操作繁琐，这也一定程度上提高了用户使用门槛。
发明内容
根据本申请的各种实施例，提供一种人脸动画实现的方法、计算机设备及存储介质。
一种人脸动画实现的方法，包括：
计算机设备采集人脸图像；
所述计算机设备提取所述人脸图像中的人脸特征点；
所述计算机设备将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；
所述计算机设备根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数；及
所述计算机设备根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情。
一种计算机设备，包括存储器和一个或多个处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述一个或多个处理器执行时，使得所述一个或多个处理器执行所述计算机可读指令时实现以下步骤：
采集人脸图像；
提取所述人脸图像中的人脸特征点；
将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；
根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数；及
根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情。
一个或多个存储有计算机可执行指令的非易失性计算机可读存储介质，所述计算机可执行指令被一个或多个处理器执行时，可使得所述一个或多个处理器执行以下步骤：
计算机可读指令采集人脸图像；
提取所述人脸图像中的人脸特征点；
将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；
根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数；及
根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应 的表情。
本申请的一个或多个实施例的细节在下面的附图和描述中提出。本申请的其它特征、目的和优点将从说明书、附图以及权利要求书变得明显。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为一个实施例中计算机设备的内部结构示意图；
图2为一个实施例中人脸动画实现的方法流程图；
图3为一个实施例中标记定位得到的人脸特征点的示意图；
图4为一个实施例中根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数的方法流程图；
图5为一个实施例中左眼局部区域的HOG特征的示意图；
图6为一个实施例中确定眼球中心位置的示意图；
图7为一个实施例中计算眼球转动角度的示意图；
图8为一个实施例中三角形进行仿射变形的示意图；
图9为一个实施例中转换成标准人脸图像的示意图；
图10为一个实施例中根据纹理特征确定局部区域对应的第三形变系数的方法流程图；
图11为一个实施例中SVM分类器的原理示意图；
图12为一个实施例中将人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数的方法流程图；
图13为一个实施例中人脸特征点对应的三维坐标与标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数的方法流程图；
图14为一个实施例中根据第一形变系数和第二形变系数驱动三维虚拟 对象执行相应的表情的方法流程图；
图15为一个实施例中最小二乘滤波器平滑原理的示意图；
图16为一个实施例中平滑处理前后的对比示意图；
图17为一个实施例中三维虚拟对象执行相应表情的示意图；
图18为一个实施例中实现人脸动画的流程架构图；
图19为一个实施例中人脸动画实现的装置结构框图；
图20为一个实施例中处理模块的结构框图；及
图21为一个实施例中比对模块的结构框图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
如图1所示，为一个实施例中计算机设备的内部结构示意图。该计算机设备可以是终端，终端可以是个人计算机或者移动电子设备，移动电子设备包括手机、平板电脑、个人数字助理或者穿戴式设备等中的至少一种。参照图1，该计算机设备包括通过系统总线连接的处理器、网络接口、显示屏、输入装置和图像采集装置。其中，存储器包括非易失性存储介质和内存储器。该计算机设备的非易失性存储介质可存储操作系统和计算机可读指令，该计算机可读指令被执行时，可使得处理器执行一种人脸动画实现的方法。该计算机设备的处理器用于提供计算和控制能力，支撑整个计算机设备的运行。该内存储器中可储存有计算机可读指令，该计算机可读指令被处理器执行时，可使得处理器执行一种人脸动画实现的方法。计算机设备的网络接口用于进行网络通信。计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。触摸层和显示屏构成触控屏。图像采集装置可以是摄像头。本领域技术 人员可以理解，图1中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
如图2所示，在一个实施例中，提出了一种人脸动画实现的方法，该方法应用于计算机设备中，具体包括以下步骤：
步骤202，采集人脸图像。
在一个实施例中，人脸图像是指包含有人脸部信息的图像。其中，人脸图像的采集可以通过终端中的前置或后置摄像头来进行采集。采集的方式可以通过拍摄照片的形式，也可以是通过拍摄视频的形式来获取，因为视频本身是由图像序列组成的。
步骤204，提取人脸图像中的人脸特征点。
在一个实施例中，人脸特征点也称人脸关键点，是指人脸区域中能够体现表情状态的脸部位置，包括但不限于眼睛、嘴巴、鼻子、眉毛等脸部位置。可以采用人脸标记定位技术来提取人脸图像中的人脸特征点，具体可以分为两个步骤，一个是人脸检测，一个是人脸标记。首先通过人脸检测获取图像中人脸所处的粗略位置，一般是框住人脸的一个矩形框，然后在该矩形框的基础上，通过人脸标记寻找更精密的位置，然后返回一系列人脸特征点位置的坐标。如图3所示，为一个实施例中，通过标记定位得到的人脸特征点的示意图。人脸标记定位的方法可以采用现有的方法，比如可以采用AAM(Active Appearance Models人脸外观模型)、ERT(Ensemble of Regression tree，回归树组合)等。这里并不对人脸标记定位的方法进行限制。
步骤206，将人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数。
在一个实施例中，几何特征是指人脸特征点发生几何形变得到的特征，比如，张嘴时，嘴巴附近的人脸特征点会发生明显的位移和形变。而人脸的几何特征的变化是需要与标准特征点进行比对才能得到的，即需要有一个参 考值。为了能够更好的体现人脸表情的变化，采用中性脸对应的人脸特征点作为标准特征点，中性脸是指人在无表情的情况下的人脸状态。通过将人脸特征点与中性脸对应的标准特征点进行比对就可以得到脸部各个部位的位移和形变，进而就可以确定各个部位对应的形变系数，比如，眉毛抬高的系数，嘴巴张开的系数等，上述各个部位对应的形变系数都是几何特征上的形变系数，为了与后文中外貌特征上的形变系数进行区分，将几何特征上的形变系数称为“第一形变系数”。
步骤208，根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数。
在一个实施例中，外貌特征是指人脸在外貌形态上发生的变化，仅仅用几何特征难以识别得到的特征。比如，眨眼、伸舌头等用几何特征难以识别得到，所以需要采用基于外貌特征的识别方法来识别该外貌特征上的变化。为了更精细的识别外貌特征，可以通过提取脸部局部图像来进行外貌特征的识别。根据识别得到的外貌特征上的变化确定相应的形变系数，称为“第二形变系数”。具体地，根据人脸特征点进行局部区域的提取，继而计算局部区域对应的纹理特征，采用训练好的分类模型根据纹理特征来预测局部区域对应的识别结果。比如，提取眼部区域，根据提取的眼部区域的纹理特征来预测眼睛是睁开的还是闭合的，进而确定相应的形变系数，如果眼睛是闭合的，可以将形变系数设为0，如果是睁开的，将形变系数设为100。通过将难以用几何特征识别的局部区域提取出来，采用外貌特征上的方法对该提取的局部区域进行处理，弥补了基于几何特征识别的不足，将基于几何特征的识别和基于外貌特征的识别进行结合，提高了表情识别的精细度和准确度。
步骤210，根据第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
在一个实施例中，三维虚拟对象是指网络虚拟动画(Avatar)，其中，Avatar是3D形象。三维虚拟对象可以是卡通形象，也可以是真实还原的真人形象。通过该三维虚拟对象来展示捕获到的人脸的表情。将计算得到的第一形变系 数和第二形变系数作为三维虚拟对象模型的形变参数控制三维虚拟对象执行相应的表情。具体地，通过识别得到的第一形变系数和第二形变系数来控制三维虚拟对象的脸部执行相应的表情，比如，执行相应的微笑、张嘴、眨眼等表情。也就是说，通过采用识别真实人脸表情得到的形变系数通过动画渲染引擎控制三维虚拟对象的脸部执行相应的表情，从而实现了将人脸表情真实还原到三维虚拟对象(Avatar)的脸部。
在一个实施例中，三维虚拟对象的表情是由若干个Blendshape(表情形变)对应的数值来控制的。Blendshape用于控制面部细节表情，一般脸部可以设置几十个Blendshape，每个Blendshape只控制一个面部细节，比如，眼角、嘴巴、嘴角等可以分别采用不同的Blendshape来进行控制，每个Blendshape的取值范围是0-100，举个例子，一个控制眼皮的Blendshape，如果取值为0时表示闭着眼，取值为100时表示完全张开。通过几十个Blendshape组合就可以合成非常复杂的面部表情。上述对人脸图像进行识别的过程就是确定人脸各个部位对应的Blendshape的值，通过将基于几何特征和基于外貌特征进行结合能够得到更加准确的识别效果，从而根据第一形变系数和第二形变系数就可以驱动三维虚拟对象执行更加精细的表情。
在一个实施例中，为了防止出现大幅度抖动，在根据第一形变系数和第二形变系数控制三维虚拟对象执行相应的表情之前，还需要对得到的第一形变系数和第二形变系数进行平滑处理，比如，可以采用最小二乘滤波器法对相应的形变系数进行平滑处理。
在一个实施例中，通过摄像头采集人脸图像，然后提取人脸图像中的人脸特征点，通过将人脸特征点与中性脸对应的标准特征点进行比对得到几何特征上对应的第一形变系数，由于该基于几何特征的识别方法对某些区域无法识别或不能识别的准确，所以还需要根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数，之后，根据第一形变系数和第二形变系数共同来驱动三维虚拟对象执行相应的表情。整个过程中不需要用户做任何预设值，操作简便，而且通过结合更精细的外貌特征上的识别能够得到 更加准确的识别效果，且该方法采用普通的单目摄像头就可以实现，大大降低了使用的成本。
如图4所示，在一个实施例中，第二形变系数包括第三形变系数和第四形变系数。根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数的步骤208包括：
步骤208A，根据人脸特征点从人脸图像中提取局部区域。
在一个实施例中，由于几何特征的识别是基于人脸特征点发生的几何形变来进行识别的，对于一些更细微的表情则难以识别，比如，是否眨眼，是否伸舌头等难以识别。所以还需要对一些不容易识别的区域进行外貌特征上的识别。由于人脸特征点可以标识脸部的位置，包括眼睛、嘴巴、鼻子、眉毛等位置，所以根据人脸特征点就可以从人脸图像中提取到相应的局部区域，比如，提取眼部区域，可以分别提取左眼区域和右眼区域。
步骤208B，计算局部区域对应的纹理特征。
在一个实施例中，纹理特征是指通过一定的图像处理技术提取的可以代表该图像属性的特征参数，用于描述图像的信息。从人脸图像中提取到待识别的局部区域后，计算该局部区域对应的纹理特征，便于后续根据纹理特征进行分类。在一个实施例中，可以采用HOG(Histogram of Oriented Gradient，方向梯度直方图)特征作为局部区域的纹理特征，其中，HOG特征是一种在计算机视觉和图像处理中用来进行物体检测的特征描述子，它通过计算和统计图像局部区域的梯度方向直方图来构成特征，在一副图像中，局部目标的表象和形状(appearance and shape)能够被梯度或边缘的方向密度分布很好地描述。如图5所示，为一个实施例中，左眼局部图像(左眼是闭着的)经过计算HOG特征得到的可视化结果。从图中可以看到中间区域的水平方向梯度分布较多，而闭眼的图像中间会有一条明显的水平边缘，所以HOG特征能很好的反应图像局部的纹理特征。
步骤208C，根据纹理特征确定局部区域对应的第三形变系数。
在一个实施例中，计算得到局部区域对应的纹理特征后，采用训练好的 分类器根据该纹理特征对局部区域进行外貌特征的预测，其中，分类器是通过对标记样本中的纹理特征进行学习得到的。比如，若局部区域为眼部，可以采用训练好的分类器根据得到的纹理特征预测是否闭眼，进而确定该局部区域对应的第三形变系数。若得到的结果为闭眼，那么相应的形变系数可以设置为0，若得到的结果为睁眼，那么相应的形变系数可以设置为100。
步骤208D，采用眼球跟踪算法确定眼球中心位置，根据眼球中心位置计算眼球相对于平视状态下的转动角度，根据转动角度确定第四形变系数。
在一个实施例中，为了更加形象地识别相应的表情，外貌特征上的识别还包括识别眼球的转动，将眼睛平视状态下的眼球作为参考标准，采用眼球跟踪算法确定眼球中心位置，然后根据眼球中心位置计算眼球相对于平视状态下的转动角度，比如，计算得到左眼球分别在X轴上旋转25度，在Y轴上旋转5度。根据该转动角度来确定第四形变系数，比如，可以直接将25和5作为对应的形变系数。在一个实施例中，首先，确定眼球中心位置。如图6所示，在不知道眼球中心的情况下，假设c为可能的眼球中心，gi为位置xi的梯度向量，如果c为眼球中心，那么c到xi的向量d和gi有相同的方向。由于向量di和gi之间的角度可以采用点积来表示，因为眼球的虹膜是近似圆形，那么计算得到di和gi点积的和最大时对应的c *点就是虹膜的中心，即眼球的中心。用数学公式表示如下：
Figure PCTCN2018083326-appb-000001
其中，
Figure PCTCN2018083326-appb-000002
表示di的转置矩阵；其次，通过上述目标函数求出来的极大值只是眼球中心位置在2D图像上的中心，基于此，还需要估计出眼球的转动角度α。在一个实施例中，如图7所示，假设眼球模型是一个近似的球型，摄像机是基于针孔的弱透视模型，头部相对于摄像机焦平面的的垂直方向(Y轴)转角为θ，眼角x方向的投影坐标为x1和x2，眼球平视情况下的中心在焦平面的X方向的 投影为x’眼球转动情况下的中心在焦平面的X方向投影为x，假设眼球的半径为r，应用几何学可以求得α，公式如下：
Figure PCTCN2018083326-appb-000003
其中，a为常数。α为求得的在X轴上的转动角度，基于同样的方法，可以求得在Y轴上的转动角度，根据计算得到的转动角度确定相应的第四形变系数。
在一个实施例中，根据人脸特征点从人脸图像中提取局部区域的步骤208A包括：根据人脸特征点采用分段仿射变形将人脸图像转变为标准人脸图像，从标准人脸图像中提取局部区域。
在一个实施例中，由于人脸图像存在不同的角度、姿态，直接使用这些图像会导致图像处理的鲁棒性变差，所以需要将人脸图像通过变形转变为标准人脸图像，即将人脸图像变形到一个标准的形状，然后再提取图像的局部区域。可以根据人脸特征点采用分段仿射变形(Piecewise Affine Warping)方法将人脸图像转变为标准人脸图像。
在一个实施例中，由于针对不同的人脸图像提取的人脸特征点的个数是相同的，比如，可以设置提取的人脸特征点都是68个。那么可以按照预设的划分规则预先设置标准人脸图像对应的标准三角形网格，提取到人脸图像的人脸特征点后，按照同样的连接规则根据人脸特征点划分得到人脸特征点对应的三角形网格，由于两者划分的方式是一致的，那么该三角形网格和标准三角形网格就有了一一对应的关系。然后针对每一个三角形的像素做一个仿射变形，即源三角形网格的每一个像素(x，y) T通过仿射变形到W(x，p)，如图8所示为一个实施例中三角形进行仿射变形的示意图。通过对每个三角形像素都进行仿射变形就可以将采集到的人脸图像转变为标准人脸图像，如图9所示，上面为一个实施例中人脸特征点对应的三角形网格转换成标准三角形网格的示意图，下面为相应的人脸图像转变为标准人脸图像的示意图。
如图10所示，在一个实施例中，根据纹理特征确定局部区域对应的第三形变系数的步骤208C包括：
步骤1002，采用训练好的分类器根据纹理特征确定局部区域对应的识别结果，分类器是通过对标记样本中的纹理特征进行学习得到的。
在一个实施例中，在计算得到局部区域对应的纹理特征后，采用预先训练好的分类器根据该计算得到的纹理特征来预测该局部区域对应的识别结果。其中，分类器是通过对标记样本中的纹理特征进行学习得到的，通过发现其中规律，进而根据学习到的规律来预测新的样本属于哪一类。在一个实施例中，基于外貌特征的预测可以看作一个二分类的问题，比如，是否闭眼，是否伸舌头等。建立分类器的方法有很多，比如，可以采用决策树、SVM(支持向量机)、神经网络等算法来实现。
下面以SVM分类器为例来说明如何建立分类器。首先，获取训练样本，每个样本被标记属于哪一类，比如，一张人脸图片为一个样本，该人脸图片中眼睛是否闭眼为对应的标记；为了提高预测的鲁棒性，在一个实施例中，还需要对样本进行预处理，其中，预处理包括对提取的局部图像做镜像处理(Y轴方向旋转)，镜像处理的目的是为了提高训练的效果和速度。因为人脸可以看作是左右对称的，为了提高训练的速度，可以通过镜像处理来同时预测另一边。比如，训练一个眨眼的分类器，希望能够同时来预测左眼和右眼，经过镜像处理后再进行学习，不仅可以增加样本的多样性，也可以一定程度上提高训练的效果。其次，提取每个样本局部区域的HOG特征，为了防止某些特征过大，可以对计算得到的HOG特征做正规化处理。正规化的方法可以采用样本值减去均值，再除以标准差，相应的公式如下：(X-μ)/σ，其中，X为样本值，μ表示均值，σ表示标准差。该正规化的方法能够解决数值稳定性问题，并且能够防止某些特征过大。最后，对标记样本中的HOG特征进行学习，学习其中规则，直到根据学习到的规则能够准确预测新的样本属于哪一类。如图11所示，SVM在学习的过程中构建了一个超平面，以使两类样本间的间隔最大，间隔越大，相应的预测效果越准确。其中，如果样本是二维，则为一条直线，三维为一个平面，大于三维位为超平面。
步骤1004，根据识别结果确定纹理特征对应的第三形变系数。
在一个实施例中，采用训练好的分类器根据纹理特征对提取的局部区域进行预测，输出相应的识别结果，根据识别结果来确定纹理特征对应的第三形变系数。其中，对于二分类的问题，输出结果为1或-1(对应真和假)两个数值，比如，对是否闭眼进行预测，如果输出结果为1，则说明为闭眼，如果输出结果为-1，则说明为睁眼。相应的闭眼的形变系数可以设为0，睁眼的形变系数为100。
在另一个实施例中，为了能够得到更加精确的预测结果，可以训练一个概率模型来表示输出为真的概率，而不仅仅是输出1和-1，这样便于后续通过调整阈值来提高预测的准确性。举个例子，对于有些眼睛很小的用户，可能睁眼和闭眼的HOG特征不是很明显，那么预测的时候很容易误判，所以如果得到的是一个闭眼的概率数值，那么通过调整闭眼的阈值，就能提高预测准确率。比如，原来闭眼的阈值设为60％，即当预测到闭眼的概率数值大于60％时，则判定眼睛为闭着的。而对于眼睛很小的用户，可以调整闭眼的阈值，比如，将阈值调整为70％，这样只有当预测闭眼概率数值大于70％时，才判定眼睛是闭着的，也就是说，判断闭眼的条件更加严格。在一个实施例中，可以采用以下公式来预测输出结果的概率，P(y＝1|f)＝1/[1+exp(Af+B)]，其中，A和B为拟合得到的参数，f为原始SVM分类器的输出结果，P为预测得到的结果为真的概率。
如图12所示，在一个实施例中，将人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数的步骤206包括：
步骤206A，计算人脸特征点对应的三维坐标。
在一个实施例中，由于真实人脸存在不同的角度、位移和缩放等刚性参数的干扰，为了能够更好的得到几何特征上对应的第一形变系数，通过将得到的2D人脸图像转换为人脸的3D图像后再进行比对，具体比对的是人脸特征点对应的三维坐标和标准特征点对应的三维坐标。所以需要计算人脸特征点对应的三维坐标。首先，获取人脸特征点对应的二维坐标，然后采用3D 重建技术来获取与人脸特征点的二维坐标对应的三维坐标。在一个实施例中，3D点分布模型能被参数p＝[s，R，q，t]描述，其中，s为缩放参数，R为模型旋转参数，t为2D位移参数，q为3D柔性参数。具体实现公式如下：
Figure PCTCN2018083326-appb-000004
其中，x i＝(x,y)表示第i个人脸特征点的二维坐标，
Figure PCTCN2018083326-appb-000005
表示第i个人脸特征点对应的三维坐标。Φ i表示从训练集中得到的第i个特征向量，用来描述柔性形状的线性变化，Φ i可以看作柔性形状变化的基，q是变化的参数。所以在已知参数p＝[s，R，q，t]的情况下，就可以采用上述公式计算得到人脸特征点对应的三维坐标。而参数p＝[s，R，q，t]可以采用最优化以下代价函数求得:
Figure PCTCN2018083326-appb-000006
其中，
Figure PCTCN2018083326-appb-000007
其中下标2代表向量范数，向量的2范数是向量中各个平方之和再开根号，即目标函数J(p)取最小值时，参数p＝[s，R，q，t]对应的值为确定的参数值。也就是将人脸特征点对应的三维坐标经过一定的缩放、旋转、位移和局部变形，然后投影得到二维坐标与之前计算得到的实际的二维坐标y i之间的误差平方和最小时对应的参数就是要确定的参数p＝[s，R，q，t]。这用通常的优化方法都可以求解，比如梯度下降方法、牛顿法。
在一个实施例中，使用梯度下降方法的计算过程如下：(1)初始化形变参数p＝[s，R，q，t]全为0；(2)应用以下公式求得模型的3D柔性形状：
Figure PCTCN2018083326-appb-000008
(3)应用以下公式求得刚性变换以后的2D投影形状：
Figure PCTCN2018083326-appb-000009
其中，
Figure PCTCN2018083326-appb-000010
为2D投影坐标，s为缩放参数，R为模型旋转参数，Φ可以看作柔性形状变化的基，q是变化的参数；(4)对J(p)中的参数向量p求偏导数：
Figure PCTCN2018083326-appb-000011
(5)更新参数p：p＝p-α·Δp，α称为学习率， 比如，可以设置α＝0.5；(6)计算2D投影坐标
Figure PCTCN2018083326-appb-000012
和人脸检测到的二维坐标y之间的距离(残差)ε是否收敛，
Figure PCTCN2018083326-appb-000013
即判断ε是否小于某个阈值(比如，0.001)，如果收敛则计算到的参数p为最优化值，否则继续步骤上述步骤(2)。
步骤206B，将得到的人脸特征点对应的三维坐标与标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数。
在一个实施例中，计算得到人脸特征点对应的三维坐标后，将计算得到的三维坐标与标准特征点对应的三维特征点进行比对得到脸部各个部位相对于标准三维坐标的变化值，比如，眉毛相对于标准三维坐标的位移变化，嘴巴相对于标准三维坐标的宽度变化等。根据得到的变化值确定相应的形变系数，进而确定在几何特征上对应的第一形变系数。
如图13所示，在一个实施例中，将得到的人脸特征点对应的三维坐标与标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数的步骤206B包括：
步骤1302，计算标准特征点对应的标准三维坐标。
在一个实施例中，为了将人脸特征点对应的三维坐标与标准特征点对应的三维坐标进行比对，首先需要计算标准特征点对应的标准三维坐标。标准特征点是指中性脸对应的标准特征点，所以标准三维坐标实际上就是中性脸对应的三维坐标，中性脸是指人在无表情的情况下的人脸状态。标准三维坐标的计算可以采用基于中值的在线预测算法，当然也可以采用别的方法，比如，可以直接将录入的中性脸对应的三维坐标作为标准三维坐标，这里并不对计算标准三维坐标的方法进行限制。下面以基于中值在线预测方法为例来进行说明，具体地，获取一段时间窗口上的视频数据(比如，100帧视频数据)，获取该100帧视频数据中每一帧视频中人脸特征点对应的三维坐标，将同一位置的人脸特征点的三维坐标按照从小到大或从大到小的顺序进行排序，然后取中间的值作为标准的三维坐标。该方法是基于一个简单的假设， 即用户在大多数时间内都是无表情状态，因此按照大小进行排序后，两端的数据看作是有表情的特殊状态，中值即为无表情的中性状态。举个例子，在获取到的连续的视频数据中，每一帧视频数据对应一张人脸图像，提取人脸图像中的人脸特征点，计算人脸特征点对应的三维坐标，每一张人脸图片对应的人脸特征点的数目是相同的，将处于同一位置的人脸特征点对应的三维坐标进行排序，比如，将嘴巴对应的一个特征点的三维坐标(X，Y，Z)进行排序，默认提取的视频数据在大多数情况下都是无表情的状态，所以将三维坐标进行排序后，取中间的值作为标准三维坐标。依次类推，获取所有特征点对应的标准三维坐标。
步骤1304，将得到的人脸特征点对应的三维坐标与标准三维坐标进行比对确定脸部各个部位对应的变化值。
在一个实施例中，计算得到标准特征点对应的标准三维坐标后，就可以将该标准三维坐标作为参考值来计算用户脸部中各个人脸特征点相对于标准三维坐标的变化，进而确定脸部各个部位对应的变化值，比如，眉毛、嘴巴等的位移变化。
步骤1306，对确定的变化值进行规范化处理得到第一形变系数。
在一个实施例中，为了对得到的数值采用统一的标准衡量，在得到脸部各个部位相对于中性脸的变化值后，还需要对该变化值进行规范化处理，比如，得到眉毛的相对位移后，将该位移规范化到0到1之间。规范化的方法可以简单的除以最大值：θ'＝θ/max(θ)，其中，θ代表规范前的值，max(θ)代表相应的最大值。举个例子，假设获取到眉毛抬高的位移为1mm，而根据经验值眉毛抬高的最大位移为5mm，那么规范化后的眉毛对应的值为0.2。由于驱动三维虚拟对象的形变系数Blendshape的取值范围为0-100，所以在对变化值进行规范化处理后，可以相应的乘以100作为相应的形变系数。
如图14所示，在一个实施例中，根据第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情的步骤210包括：
步骤210A，采用最小二乘滤波器法对第一形变系数和第二形变系数进行 系数平滑处理。
在一个实施例中，由于提取到的人脸特征点的二维坐标会存在误差，那么根据二维坐标计算得到的三维坐标在视频帧之间会存在局部抖动，使得预测出来的形变系数很不平滑，严重影响动画效果。所以还需要对得到的形变系数进行平滑处理。在一个实施例中采用最小二乘滤波器法对第一形变系数和第二形变系数进行系数平滑处理。具体处理的方法如下：如图15，假设滤波器处理窗口为5帧，则需要保留4帧的历史数据。假设t时刻的系数值为y1，采用最小二乘法利用t，t-1，t-2，t-3，t-4时刻的历史数据拟合一条直线y＝ax+b，求得参数a和b。t时刻落在该直线上的点y’即为滤波输出的结果，针对每一帧不断应用该滤波器，并且更新历史数据就能得到很平滑的系数。如图16所示，为一个X方向的头部转动系数平滑的效果，其中，图中黑色细线表示的是原始的系数，抖动很大，黑色粗线表示的是平滑处理后的结果，相对平滑很多。
步骤210B，采用系数平滑处理过的第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
在一个实施例中，三维虚拟对象是指网络虚拟动画(Avatar)，可以是卡通形象，也可以是真实还原的真人形象。采用系数平滑处理过的第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。在一个实施例中，形变系数可以采用Blendshape来表示，上述识别人脸表情的过程就是为了确定脸部中各个Blendshape所对应的数值，根据各个Blendshape的值组合成人的真实表情。然后根据该Blendshape的值驱动三维虚拟对象执行相应的表情，即将采集到的真实人脸的表情还原到三维虚拟对象的脸部，实现相应的人脸动画。如图17为一个实施例中三维虚拟对象展示的眨眼(右眼闭着)的表情。
如图18，为一个实施例中，实现人脸动画的流程架构图。首先，采用人脸标记定位技术提取采集到的人脸图像的人脸特征点，然后根据人脸特征点分别进行几何特征和外貌特征上的识别，其中，几何特征上的识别的过程包括：采用3D重建技术根据提取到的人脸特征点的二维坐标计算得到相应的 三维坐标，并计算得到中性脸对应的标准三维坐标，通过将三维坐标与中性脸对应的标准三维坐标进行比对得到脸部各个部位对应的变化值，并将该变化值通过系数转换转变为规范化的系数，进而确定相应的第一形变系数。外貌特征上的识别过程包括：根据人脸特征点采用仿射变形得到标准的人脸图像，然后提取人脸图像中的局部区域，之后一方面是通过计算提取局部区域的HOG特征，采用训练好的SVM分类器得到相应的预测结果，确定第三形变系数；另一方面采用眼球跟踪算法进行眼球跟踪确定眼球中心位置，根据眼球中心位置计算眼球相对于平视状态下的转动角度，根据转动角度确定第四形变系数。其中，第三形变系数和第四形变系数共同构成了与外貌特征对应的第二形变系数。之后，采用最小二乘滤波器法对计算得到的第一形变系数和第二形变系数进行平滑处理，根据平滑处理的第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情，即生成形变动画。
在一个实施例中，还提供了一种计算机设备，该计算机设备的内部结构可如图1所示，该计算机设备包括人脸动画实现的装置，该人脸动画实现的装置中包括各个模块，每个模块可全部或部分通过软件、硬件或其组合来实现。
如图19所示，在一个实施例中，提出了一种人脸动画实现的装置，该装置包括：
采集模块1902，用于采集人脸图像。
提取模块1904，用于提取人脸图像中的人脸特征点。
比对模块1906，用于将人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数。
处理模块1908，用于根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数。
驱动模块1910，用于根据第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
如图20所示，在一个实施例中，第二形变系数包括第三形变系数和第四 形变系数。处理模块1908包括：
局部区域提取模块1908A，用于根据人脸特征点从人脸图像中提取局部区域。
纹理特征计算模块1908B，用于计算局部区域对应的纹理特征。
第三形变系数确定模块1908C，用于根据纹理特征确定局部区域对应的第三形变系数。
第四形变系数确定模块1910D，用于采用眼球跟踪算法确定眼球中心位置，根据眼球中心位置计算眼球相对于平视状态下的转动角度，根据转动角度确定第四形变系数。
在一个实施例中，局部区域提取模块1908A还用于根据人脸特征点采用分段仿射变形将人脸图像转变为标准人脸图像，从标准人脸图像中提取局部区域。
在一个实施例中，第三形变系数确定模块1908C还用于采用训练好的分类器根据纹理特征确定局部区域对应的识别结果，分类器是通过对标记样本中的纹理特征进行学习得到的，根据识别结果确定纹理特征对应的第三形变系数。
如图21所示，在一个实施例中，比对模块1906包括：
三维坐标计算模块1906A，用于计算人脸特征点对应的三维坐标。
三维坐标比对模块1906B，用于将得到的人脸特征点对应的三维坐标与标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数。
在一个实施例中，三维坐标比对模块1906A还用于计算标准特征点对应的标准三维坐标，将得到的人脸特征点对应的三维坐标与标准三维坐标进行比对确定脸部各个部位对应的变化值，对确定的变化值进行规范化处理得到第一形变系数。
在一个实施例中，驱动模块1910还用于采用最小二乘滤波器法对第一形变系数和第二形变系数进行系数平滑处理，采用系数平滑处理过的第一形变 系数和第二形变系数驱动三维虚拟对象执行相应的表情。
在一个实施例中，提出一种计算机设备，包括存储器和一个或多个处理器，存储器中存储有计算机可读指令，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行计算机可读指令时实现以下步骤：采集人脸图像；提取人脸图像中的人脸特征点；将人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数；根据第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
在一个实施例中，第二形变系数包括第三形变系数和第四形变系数；处理器执行的根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数，包括：根据人脸特征点从人脸图像中提取局部区域；计算局部区域对应的纹理特征；根据纹理特征确定局部区域对应的第三形变系数；采用眼球跟踪算法确定眼球中心位置，根据眼球中心位置计算眼球相对于平视状态下的转动角度，根据转动角度确定第四形变系数。
在一个实施例中，处理器执行的根据人脸特征点从人脸图像中提取局部区域，包括：根据人脸特征点采用分段仿射变形将人脸图像转变为标准人脸图像；从标准人脸图像中提取局部区域。
在一个实施例中，处理器执行的根据纹理特征确定局部区域对应的第三形变系数，包括：采用训练好的分类器根据纹理特征确定局部区域对应的识别结果，分类器是通过对标记样本中的纹理特征进行学习得到的；根据识别结果确定纹理特征对应的第三形变系数。
在一个实施例中，处理器执行的将人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数，包括：计算人脸特征点对应的三维坐标；将得到的人脸特征点对应的三维坐标与标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数。
在一个实施例中，处理器执行的将得到的人脸特征点对应的三维坐标与标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系 数，包括：计算标准特征点对应的标准三维坐标；将得到的人脸特征点对应的三维坐标与标准三维坐标进行比对确定脸部各个部位对应的变化值；对确定的变化值进行规范化处理得到第一形变系数。
在一个实施例中，处理器执行的根据第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情，包括：采用最小二乘滤波器法对第一形变系数和第二形变系数进行系数平滑处理；采用系数平滑处理过的第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
在一个实施例中，提出了提供了一个或多个存储有计算机可读指令的非易失性计算机可读存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行以下步骤：计算机可读指令采集人脸图像；提取人脸图像中的人脸特征点；将人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数；根据第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
在一个实施例中，第二形变系数包括第三形变系数和第四形变系数；处理器执行的根据人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数，包括：根据人脸特征点从人脸图像中提取局部区域；计算局部区域对应的纹理特征；根据纹理特征确定局部区域对应的第三形变系数；采用眼球跟踪算法确定眼球中心位置，根据眼球中心位置计算眼球相对于平视状态下的转动角度，根据转动角度确定第四形变系数。
在一个实施例中，处理器执行的根据人脸特征点从人脸图像中提取局部区域，包括：根据人脸特征点采用分段仿射变形将人脸图像转变为标准人脸图像；从标准人脸图像中提取局部区域。
在一个实施例中，处理器执行的根据纹理特征确定局部区域对应的第三形变系数，包括：采用训练好的分类器根据纹理特征确定局部区域对应的识别结果，分类器是通过对标记样本中的纹理特征进行学习得到的；根据识别结果确定纹理特征对应的第三形变系数。
在一个实施例中，处理器执行的将人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数，包括：计算人脸特征点对应的三维坐标；将得到的人脸特征点对应的三维坐标与标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数。
在一个实施例中，处理器执行的将得到的人脸特征点对应的三维坐标与标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数，包括：计算标准特征点对应的标准三维坐标；将得到的人脸特征点对应的三维坐标与标准三维坐标进行比对确定脸部各个部位对应的变化值；对确定的变化值进行规范化处理得到第一形变系数。
在一个实施例中，处理器执行的根据第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情，包括：采用最小二乘滤波器法对第一形变系数和第二形变系数进行系数平滑处理；采用系数平滑处理过的第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
应该理解的是，虽然本申请各实施例中的各个步骤并不是必然按照步骤标号指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，各实施例中至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，所述的程序可存储于一非易失性计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和/或易失性存储器。非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易 失性存储器可包括随机存取存储器(RAM)或者外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDRSDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)等。计算机可读指令计算机可读指令以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (21)

  1. 一种人脸动画实现的方法，包括：
    计算机设备采集人脸图像；
    所述计算机设备提取所述人脸图像中的人脸特征点；
    所述计算机设备将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；
    所述计算机设备根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数；及
    所述计算机设备根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情。
  2. 根据权利要求1所述的方法，其特征在于，所述第二形变系数包括第三形变系数和第四形变系数；
    所述计算机设备根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数，包括：
    所述计算机设备根据所述人脸特征点从人脸图像中提取局部区域；
    所述计算机设备计算所述局部区域对应的纹理特征；
    所述计算机设备根据所述纹理特征确定所述局部区域对应的第三形变系数；
    所述计算机设备采用眼球跟踪算法确定眼球中心位置，根据所述眼球中心位置计算眼球相对于平视状态下的转动角度，根据转动角度确定第四形变系数。
  3. 根据权利要求2所述的方法，其特征在于，所述计算机设备根据所述人脸特征点从人脸图像中提取局部区域，包括：
    所述计算机设备根据所述人脸特征点采用分段仿射变形将所述人脸图像转变为标准人脸图像；
    所述计算机设备从所述标准人脸图像中提取局部区域。
  4. 根据权利要求2所述的方法，其特征在于，所述计算机设备根据所述 纹理特征确定所述局部区域对应的第三形变系数，包括：
    所述计算机设备采用训练好的分类器根据所述纹理特征确定所述局部区域对应的识别结果，所述分类器是通过对标记样本中的纹理特征进行学习得到的；
    所述计算机设备根据所述识别结果确定所述纹理特征对应的第三形变系数。
  5. 根据权利要求1所述的方法，其特征在于，所述计算机设备将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数，包括：
    所述计算机设备计算所述人脸特征点对应的三维坐标；
    所述计算机设备将得到的所述人脸特征点对应的三维坐标与所述标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数。
  6. 根据权利要求5所述的方法，其特征在于，所述计算机设备将得到的所述人脸特征点对应的三维坐标与所述标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数，包括：
    所述计算机设备计算所述标准特征点对应的标准三维坐标；
    所述计算机设备将得到的所述人脸特征点对应的三维坐标与所述标准三维坐标进行比对确定脸部各个部位对应的变化值；
    对确定的所述变化值进行规范化处理得到第一形变系数。
  7. 根据权利要求1所述的方法，其特征在于，所述计算机设备根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情，包括：
    所述计算机设备采用最小二乘滤波器法对所述第一形变系数和第二形变系数进行系数平滑处理；
    所述计算机设备采用系数平滑处理过的第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
  8. 一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行 所述计算机可读指令时实现以下步骤：
    采集人脸图像；
    提取所述人脸图像中的人脸特征点；
    将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；
    根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数；及
    根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情。
  9. 根据权利要求8所述的计算机设备，其特征在于，所述第二形变系数包括第三形变系数和第四形变系数；
    所述根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数，包括：
    根据所述人脸特征点从人脸图像中提取局部区域；
    计算所述局部区域对应的纹理特征；
    根据所述纹理特征确定所述局部区域对应的第三形变系数；
    采用眼球跟踪算法确定眼球中心位置，根据所述眼球中心位置计算眼球相对于平视状态下的转动角度，根据转动角度确定第四形变系数。
  10. 根据权利要求9所述的计算机设备，其特征在于，所述根据所述人脸特征点从人脸图像中提取局部区域，包括：
    根据所述人脸特征点采用分段仿射变形将所述人脸图像转变为标准人脸图像；
    从所述标准人脸图像中提取局部区域。
  11. 根据权利要求9所述的计算机设备，其特征在于，根据所述纹理特征确定所述局部区域对应的第三形变系数，包括：
    采用训练好的分类器根据所述纹理特征确定所述局部区域对应的识别结果，所述分类器是通过对标记样本中的纹理特征进行学习得到的；
    根据所述识别结果确定所述纹理特征对应的第三形变系数。
  12. 根据权利要求8所述的计算机设备，其特征在于，所述将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数，包括：
    计算所述人脸特征点对应的三维坐标；
    将得到的所述人脸特征点对应的三维坐标与所述标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数。
  13. 根据权利要求12所述的计算机设备，其特征在于，所述将得到的所述人脸特征点对应的三维坐标与所述标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数，包括：
    计算所述标准特征点对应的标准三维坐标；
    将得到的所述人脸特征点对应的三维坐标与所述标准三维坐标进行比对确定脸部各个部位对应的变化值；
    对确定的所述变化值进行规范化处理得到第一形变系数。
  14. 根据权利要求8所述的计算机设备，其特征在于，所述根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情，包括：
    采用最小二乘滤波器法对所述第一形变系数和第二形变系数进行系数平滑处理；
    采用系数平滑处理过的第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
  15. 一个或多个存储有计算机可读指令的计算机可读非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得所述一个或多个处理器执行以下步骤：
    采集人脸图像；
    提取所述人脸图像中的人脸特征点；
    将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数；
    根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二 形变系数；及
    根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情。
  16. 根据权利要求15所述的存储介质，其特征在于，所述第二形变系数包括第三形变系数和第四形变系数；
    所述根据所述人脸特征点提取局部区域进行处理得到外貌特征上对应的第二形变系数，包括：
    根据所述人脸特征点从人脸图像中提取局部区域；
    计算所述局部区域对应的纹理特征；
    根据所述纹理特征确定所述局部区域对应的第三形变系数；
    采用眼球跟踪算法确定眼球中心位置，根据所述眼球中心位置计算眼球相对于平视状态下的转动角度，根据转动角度确定第四形变系数。
  17. 根据权利要求16所述的存储介质，其特征在于，所述根据所述人脸特征点从人脸图像中提取局部区域，包括：
    根据所述人脸特征点采用分段仿射变形将所述人脸图像转变为标准人脸图像；
    从所述标准人脸图像中提取局部区域。
  18. 根据权利要求16所述的存储介质，其特征在于，根据所述纹理特征确定所述局部区域对应的第三形变系数，包括：
    采用训练好的分类器根据所述纹理特征确定所述局部区域对应的识别结果，所述分类器是通过对标记样本中的纹理特征进行学习得到的；
    根据所述识别结果确定所述纹理特征对应的第三形变系数。
  19. 根据权利要求15所述的存储介质，其特征在于，所述将所述人脸特征点与标准特征点进行比对得到几何特征上对应的第一形变系数，包括：
    计算所述人脸特征点对应的三维坐标；
    将得到的所述人脸特征点对应的三维坐标与所述标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数。
  20. 根据权利要求19所述的存储介质，其特征在于，所述将得到的所述人脸特征点对应的三维坐标与所述标准特征点对应的标准三维坐标进行比对得到几何特征上对应的第一形变系数，包括：
    计算所述标准特征点对应的标准三维坐标；
    将得到的所述人脸特征点对应的三维坐标与所述标准三维坐标进行比对确定脸部各个部位对应的变化值；
    对确定的所述变化值进行规范化处理得到第一形变系数。
  21. 根据权利要求15所述的存储介质，其特征在于，所述根据所述第一形变系数和所述第二形变系数驱动三维虚拟对象执行相应的表情，包括：
    采用最小二乘滤波器法对所述第一形变系数和第二形变系数进行系数平滑处理；
    采用系数平滑处理过的第一形变系数和第二形变系数驱动三维虚拟对象执行相应的表情。
PCT/CN2018/083326 2017-05-12 2018-04-17 人脸动画实现的方法、计算机设备及存储介质 Ceased WO2018205801A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/509,305 US11087519B2 (en) 2017-05-12 2019-07-11 Facial animation implementation method, computer device, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710335509.5 2017-05-12
CN201710335509.5A CN108876879B (zh) 2017-05-12 2017-05-12 人脸动画实现的方法、装置、计算机设备及存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/509,305 Continuation US11087519B2 (en) 2017-05-12 2019-07-11 Facial animation implementation method, computer device, and storage medium

Publications (1)

Publication Number Publication Date
WO2018205801A1 true WO2018205801A1 (zh) 2018-11-15

Family

ID=64105140

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/083326 Ceased WO2018205801A1 (zh) 2017-05-12 2018-04-17 人脸动画实现的方法、计算机设备及存储介质

Country Status (3)

Country Link
US (1) US11087519B2 (zh)
CN (1) CN108876879B (zh)
WO (1) WO2018205801A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111614925A (zh) * 2020-05-20 2020-09-01 广州视源电子科技股份有限公司 人物图像处理方法、装置、相应终端及存储介质

Families Citing this family (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3420887B1 (en) * 2017-06-30 2025-01-29 Essilor International Method for determining the position of the eye rotation center of the eye of a subject, and associated device
CN111460871B (zh) * 2019-01-18 2023-12-22 北京市商汤科技开发有限公司 图像处理方法及装置、存储介质
CN109840019B (zh) * 2019-02-22 2023-01-10 网易（杭州）网络有限公司 虚拟人物的控制方法、装置及存储介质
JP7161111B2 (ja) * 2019-03-26 2022-10-26 日本電信電話株式会社 重みマスク生成装置、重みマスク生成方法及びプログラム
CN110141857A (zh) 2019-04-26 2019-08-20 腾讯科技（深圳）有限公司 虚拟角色的面部显示方法、装置、设备及存储介质
WO2020222785A1 (en) * 2019-04-30 2020-11-05 Hewlett-Packard Development Company, L.P. Facial action unit detection
US10976829B1 (en) * 2019-06-03 2021-04-13 Facebook, Inc. Systems and methods for displaying augmented-reality objects
CN111651033B (zh) * 2019-06-26 2024-03-05 广州虎牙科技有限公司 一种人脸的驱动显示方法、装置、电子设备和存储介质
WO2020263277A1 (en) * 2019-06-28 2020-12-30 1Hewlett-Packard Development Company, L. P. Landmark temporal smoothing
CN110503700A (zh) * 2019-08-26 2019-11-26 北京达佳互联信息技术有限公司 生成虚拟表情的方法、装置、电子设备及存储介质
CN110517339B (zh) 2019-08-30 2021-05-25 腾讯科技（深圳）有限公司 一种基于人工智能的动画形象驱动方法和装置
CN110807364B (zh) * 2019-09-27 2022-09-30 中国科学院计算技术研究所 三维人脸与眼球运动的建模与捕获方法及系统
CN113129456B (zh) * 2019-12-30 2023-07-25 百度在线网络技术（北京）有限公司 车辆三维模型变形方法、装置和电子设备
CN111460945A (zh) * 2020-03-25 2020-07-28 亿匀智行(深圳)科技有限公司 一种基于人工智能获取rgb视频中3d表情的算法
CN114429214B (zh) 2020-10-29 2025-08-01 阿里巴巴集团控股有限公司 运算单元、相关装置和方法
CN112101306B (zh) * 2020-11-10 2021-02-09 成都市谛视科技有限公司 基于rgb图像的精细化人脸表情捕获方法及装置
CN112562027A (zh) * 2020-12-02 2021-03-26 北京百度网讯科技有限公司 人脸模型的生成方法、装置、电子设备及存储介质
CN112614213B (zh) * 2020-12-14 2024-01-23 杭州网易云音乐科技有限公司 人脸表情确定方法、表情参数确定模型、介质及设备
US11756334B2 (en) * 2021-02-25 2023-09-12 Qualcomm Incorporated Facial expression recognition
CN113066155A (zh) * 2021-03-23 2021-07-02 华强方特（深圳）动漫有限公司 一种3d表情处理方法及装置
CN113255457A (zh) * 2021-04-28 2021-08-13 上海交通大学 基于人脸表情识别的动画角色面部表情生成方法及系统
EP4377896A4 (en) * 2021-07-29 2025-05-21 Digital Domain Virtual Human (US), Inc. System and method for animation of secondary features
CN114332365B (zh) * 2021-12-24 2025-08-15 北京百度网讯科技有限公司 虚拟人物形象生成方法、装置、电子设备及存储介质
CN114359449A (zh) * 2022-01-13 2022-04-15 北京大橘大栗文化传媒有限公司 一种面部数字资产制作方法
US11659043B1 (en) 2022-01-27 2023-05-23 Meta Platforms Technologies, Llc Systems and methods for predictively downloading volumetric data
US12320888B2 (en) * 2022-05-12 2025-06-03 Microsoft Technology Licensing, Llc Controlling computer-generated facial expressions
CN114627218B (zh) * 2022-05-16 2022-08-12 成都市谛视无限科技有限公司 一种基于虚拟引擎的人脸细微表情捕捉方法及装置
CN115719505A (zh) * 2022-10-27 2023-02-28 西安邮电大学 基于真实人脸的切尔诺夫脸谱图的数据可视化方法及装置
CN115997239A (zh) * 2022-11-25 2023-04-21 广州酷狗计算机科技有限公司 人脸图像生成方法、装置、设备以及存储介质
CN116189254A (zh) * 2022-11-30 2023-05-30 上海哔哩哔哩科技有限公司 面部表情控制方法和装置
CN115953813B (zh) * 2022-12-19 2024-01-30 北京字跳网络技术有限公司 一种表情驱动方法、装置、设备及存储介质
CN115937372B (zh) * 2022-12-19 2023-10-03 北京字跳网络技术有限公司 一种面部表情的模拟方法、装置、设备及存储介质
JP7339420B1 (ja) 2022-12-20 2023-09-05 カバー株式会社 プログラム、方法、情報処理装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101354795A (zh) * 2008-08-28 2009-01-28 北京中星微电子有限公司 基于视频的三维人脸动画驱动方法和系统
CN102479388A (zh) * 2010-11-22 2012-05-30 北京盛开互动科技有限公司 基于人脸跟踪和分析的表情互动方法
US20150035825A1 (en) * 2013-02-02 2015-02-05 Zhejiang University Method for real-time face animation based on single video camera

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6879709B2 (en) * 2002-01-17 2005-04-12 International Business Machines Corporation System and method for automatically detecting neutral expressionless faces in digital images
US7720284B2 (en) * 2006-09-08 2010-05-18 Omron Corporation Method for outlining and aligning a face in face processing of an image
CN102376100A (zh) * 2010-08-20 2012-03-14 北京盛开互动科技有限公司 基于单张照片的人脸动画方法
US8311319B2 (en) * 2010-12-06 2012-11-13 Seiko Epson Corporation L1-optimized AAM alignment
CN102157010A (zh) * 2011-05-25 2011-08-17 上海大学 基于分层建模及多体驱动的三维人脸动画实现方法
CN102393951B (zh) * 2011-06-30 2014-10-29 Tcl集团股份有限公司 一种人脸模型的变形方法
WO2013020248A1 (en) * 2011-08-09 2013-02-14 Intel Corporation Image-based multi-view 3d face generation
US9936165B2 (en) * 2012-09-06 2018-04-03 Intel Corporation System and method for avatar creation and synchronization
EP2915101A4 (en) * 2012-11-02 2017-01-11 Itzhak Wilf Method and system for predicting personality traits, capabilities and suggested interactions from images of a person
CN105190700A (zh) * 2013-06-04 2015-12-23 英特尔公司 基于化身的视频编码
US9378576B2 (en) * 2013-06-07 2016-06-28 Faceshift Ag Online modeling for real-time facial animation
CN105139438B (zh) * 2014-09-19 2018-01-12 电子科技大学 视频人脸卡通动画生成方法
CN104599309A (zh) * 2015-01-09 2015-05-06 北京科艺有容科技有限责任公司 一种基于元表情的三维动画角色的表情生成方法
EP3232368A1 (en) * 2016-04-14 2017-10-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Determining facial parameters
CN106056650A (zh) * 2016-05-12 2016-10-26 西安电子科技大学 基于快速表情信息提取和泊松融合的人脸表情合成方法
US10586370B2 (en) * 2018-01-08 2020-03-10 Facebook Technologies, Llc Systems and methods for rendering avatars with deep appearance models
US10896535B2 (en) * 2018-08-13 2021-01-19 Pinscreen, Inc. Real-time avatars using dynamic textures

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101354795A (zh) * 2008-08-28 2009-01-28 北京中星微电子有限公司 基于视频的三维人脸动画驱动方法和系统
CN102479388A (zh) * 2010-11-22 2012-05-30 北京盛开互动科技有限公司 基于人脸跟踪和分析的表情互动方法
US20150035825A1 (en) * 2013-02-02 2015-02-05 Zhejiang University Method for real-time face animation based on single video camera

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LEI, TENG: "Study on movement of virtual human eyes and expression synthesis of human faces", ELECTRONIC TECHNOLOGY& INFORMATION SCIENCE, no. 9, 15 September 2014 (2014-09-15) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111614925A (zh) * 2020-05-20 2020-09-01 广州视源电子科技股份有限公司 人物图像处理方法、装置、相应终端及存储介质

Also Published As

Publication number Publication date
US20190333262A1 (en) 2019-10-31
CN108876879B (zh) 2022-06-14
US11087519B2 (en) 2021-08-10
CN108876879A (zh) 2018-11-23

Similar Documents

Publication Publication Date Title
WO2018205801A1 (zh) 人脸动画实现的方法、计算机设备及存储介质
US12307621B2 (en) Adjusting a digital representation of a head region
Park et al. Learning to find eye region landmarks for remote gaze estimation in unconstrained settings
CN104680121B (zh) 一种人脸图像的处理方法及装置
Tsalakanidou et al. Real-time 2D+ 3D facial action and expression recognition
CN103443804B (zh) 面部标志检测方法
WO2019128932A1 (zh) 人脸姿态分析方法、装置、设备、存储介质以及程序
WO2019128508A1 (zh) 图像处理方法、装置、存储介质及电子设备
CN108985210A (zh) 一种基于人眼几何特征的视线追踪方法及系统
CN103116902A (zh) 三维虚拟人头像生成方法、人头像运动跟踪方法和装置
CN101964064A (zh) 一种人脸比对方法
CN102654903A (zh) 一种人脸比对方法
CN103430218A (zh) 用3d脸部建模和地标对齐扩增造型的方法
WO2021196721A1 (zh) 一种舱内环境的调整方法及装置
Ioannou et al. Robust feature detection for facial expression recognition
Unzueta et al. Efficient generic face model fitting to images and videos
CN110751097A (zh) 一种半监督的三维点云手势关键点检测方法
Achenbach et al. Accurate Face Reconstruction through Anisotropic Fitting and Eye Correction.
CN114332136A (zh) 人脸属性数据标注方法、计算机设备及存储介质
Venkatesh et al. A novel approach to classification of facial expressions from 3D-mesh datasets using modified PCA
CN111523406A (zh) 一种基于生成对抗网络改进结构的偏转人脸转正方法
Fooprateepsiri et al. A general framework for face reconstruction using single still image based on 2D-to-3D transformation kernel
JP2014229012A (ja) 人物属性推定装置、人物属性推定方法及びプログラム
Heo 3d generic elastic models for 2d pose synthesis and face recognition
Gürel Development of a face recognition system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18797752

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18797752

Country of ref document: EP

Kind code of ref document: A1