WO2017092196A1

WO2017092196A1 - Method and apparatus for generating three-dimensional animation

Info

Publication number: WO2017092196A1
Application number: PCT/CN2016/076742
Authority: WO
Inventors: 黄源浩; 肖振中; 许宏淮
Original assignee: 深圳奥比中光科技有限公司
Priority date: 2015-12-01
Filing date: 2016-03-18
Publication date: 2017-06-08
Also published as: CN105513114B; CN105513114A

Abstract

A method for generating a three-dimensional animation, comprising: acquiring a depth image of a first main body, and acquiring a pre-established first three-dimensional model corresponding to the depth image of the first main body, wherein the main body is a human body or an animal with bones (S110); acquiring a first feature point matching the depth image of the first main body, mapping the first feature point to the first three-dimensional model to obtain a corresponding model first feature point, and acquiring influence weight information of the model first feature point on a skin point (S120); and acquiring a movement trajectory of the first feature point according to the depth image of the first main body, and generating a first three-dimensional animation corresponding to the first three-dimensional model according to the movement trajectory and the influence weight information (S130). The present invention means it is not necessary to wear a sensor to collect three-dimensional position information, and is simple and convenient. In addition, further provided is an apparatus for generating a three-dimensional animation.

Description

Method and device for generating 3D animation

[Technical Field]

The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for generating a three-dimensional animation.

【Background technique】

With the development of computer technology and the advancement of multimedia technology, two-dimensional animation can no longer meet people's visual needs. Three-dimensional animation is more and more popular because of its strong sense of space and realism.

Existing 3D animation generation methods In order to generate 3D animations that match human or animal movements, it is necessary to wear sensors on the human or animal body, capture the sensor, obtain the motion of the human or animal, capture the trajectory of the sensor, and then add Models to 3D animation use traditional 3D animation techniques to create expressions, bones, etc. to generate 3D animations. This method requires sensors and is highly complex.

[Summary of the Invention]

Based on this, it is necessary to provide a method for generating a three-dimensional animation for the above technical problems, and to improve the convenience of three-dimensional animation generation.

A method for generating a three-dimensional animation, the method comprising:

Obtaining a first body depth image, and acquiring a pre-established first three-dimensional model corresponding to the first body depth image, the body being a human body or an animal having bones;

Obtaining a first feature point that is matched by the first body depth image, and mapping the first feature point to a first three-dimensional model to obtain a corresponding first feature point of the model;

Obtaining weight information of the first feature point of the model on the skin point;

Obtaining a motion trajectory of the first feature point according to the first body depth image, the first body depth image is a depth image in an RGBD image, and the RGBD image further includes a corresponding color image, the first three-dimensional The animation is a colored three-dimensional animation, and the first three-dimensional animation corresponding to the first three-dimensional model is generated according to the motion trajectory and the influence weight information;

Obtaining an accessory model, and acquiring weight information of the accessory influence attribute of the first feature point of the model on the accessory model;

Changing a form of the accessory model according to position information of the first feature point of the model and the accessory influence weight information;

The changed accessory model is worn to a position corresponding to the first three-dimensional model.

In one embodiment, before the step of acquiring the first body depth image, the method further includes:

Obtaining a body depth image of different forms, and establishing different three-dimensional models for the body depth images of the different shapes;

And setting feature points corresponding to the different depths of the body depth image, and mapping the feature points to the three-dimensional model to obtain corresponding model feature points;

Establishing a three-dimensional animated skeleton of the three-dimensional model according to the model feature point;

The weight information of the influence of the model feature points on the skin points is determined according to the positional relationship between the skin point position and the three-dimensional animated bone.

In one embodiment, the step of determining the influence weight information of the model feature point on the skin point according to the positional relationship between the skin point position and the three-dimensional animated bone comprises:

Determining the influence range of the three-dimensional animated bone according to the skeleton characteristics of the subject;

The weight coefficient of the influence of the model feature points on the skin points at different positions within the influence range is determined according to the body skeleton feature, wherein the influence of the model feature points on the skin points is inversely proportional to the distance between the two.

In one embodiment, the step of generating the first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information comprises:

Mapping the feature points on the motion trajectory according to the depth information to the first three-dimensional model to obtain spatial three-dimensional coordinates of the model feature points;

Acquiring, according to the impact weight information, a first influence range of the model feature point;

Obtaining a first skin point in the first influence range, and calculating a spatial position relationship between the first skin point and the model feature point according to the original space three-dimensional coordinates of the first skin point and the spatial three-dimensional coordinates of the model feature point;

Obtaining a weight coefficient of the first skin point according to the spatial position relationship;

Calculating an updated spatial three-dimensional coordinate of the first skin point according to the weight coefficient, and moving the first skin point from the original space three-dimensional coordinate to the update space three-dimensional coordinate.

Determining whether the skin point of the three-dimensional animation to be generated is a skin point of a corresponding position on the depth image acquired by the camera, and if so, directly generating a skin point of the first three-dimensional animation corresponding to the position according to the depth image, otherwise according to the motion The trajectory and influence weight information generates other skin points of the first three-dimensional animation corresponding to the first three-dimensional model.

A method for generating a three-dimensional animation, the method comprising:

Obtaining a motion trajectory of the first feature point according to the first body depth image, and generating a first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information.

In one embodiment, the method further includes:

In one embodiment, the first body depth image is a depth image in an RGBD image, the RGBD image further comprising a corresponding color image, the first three-dimensional animation being a colored three-dimensional animation.

A device for generating a three-dimensional animation, the device comprising:

a depth image and model acquisition module, configured to acquire a first body depth image, and acquire a pre-established first three-dimensional model corresponding to the first body depth image, the body being a human body or an animal having bones;

a feature point and weight acquisition module, configured to acquire a first feature point matched by the first body depth image, map the first feature point to a first three-dimensional model to obtain a corresponding first feature point of the model, and acquire the model Weight information of the influence of the first feature point on the skin point;

a three-dimensional animation generating module, configured to acquire a motion trajectory of the first feature point according to the first body depth image, and generate a first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information.

In one embodiment, the apparatus further includes:

a pre-processing module, configured to acquire a body depth image of different shapes, establish different three-dimensional models for the body depth images of the different shapes, set feature points corresponding to the body depth images of the different shapes, and map the feature points to The three-dimensional model obtains corresponding model feature points, and the three-dimensional animated bone of the three-dimensional model is established according to the model feature points, and the weight information of the model feature points on the skin points is determined according to the positional relationship between the skin point position and the three-dimensional animated bone.

In one embodiment, the pre-processing module is further configured to determine an influence range of the three-dimensional animated bone according to a subject bone feature, and determine, according to the subject bone feature, a skin point effect of the model feature point on the different position within the influence range The weight coefficient, wherein the influence of the model feature points on the skin points when determining the weight coefficients is inversely proportional to the distance between the two.

In one embodiment, the three-dimensional animation generation module includes:

a feature point coordinate unit, configured to map the feature points on the motion track to the first three-dimensional model according to the depth information to obtain spatial three-dimensional coordinates of the model feature points;

a spatial relationship calculation unit, configured to acquire a first influence range of the model feature point according to the influence weight information, acquire a first skin point in the first influence range, and obtain a three-dimensional coordinate of the original space according to the first skin point The spatial three-dimensional coordinates of the model feature points are used to calculate the spatial positional relationship between the first skin point and the model feature points;

An update unit, configured to obtain a weight coefficient of the first skin point according to the spatial position relationship, calculate an updated spatial three-dimensional coordinate of the first skin point according to the weight coefficient, and use the first skin point from the original space three-dimensional coordinate Move to the update space three-dimensional coordinates.

In one embodiment, the apparatus further includes:

An accessory module for acquiring an accessory model, obtaining a first feature point of the model and matching the accessory model The influence weight information is changed, the shape of the accessory model is changed according to the position information of the first feature point of the model and the accessory influence weight information, and the changed accessory model is worn to a position corresponding to the first three-dimensional model.

In one embodiment, the three-dimensional animation generation module includes:

a determining unit, configured to determine whether a skin point of the three-dimensional animation to be generated is a skin point corresponding to a position on the depth image collected by the camera, and if yes, enter the first generating unit, and otherwise enter the second generating unit;

a first generating unit, configured to directly generate a skin point of the first three-dimensional animation corresponding to the position according to the depth image;

And a second generating unit, configured to generate, according to the motion trajectory and the influence weight information, other skin points of the first three-dimensional animation corresponding to the first three-dimensional model.

The method and device for generating the three-dimensional animation, the first body depth image is obtained by acquiring the first body depth image, the body is a human body or an animal having bones, and acquiring a first three-dimensional model corresponding to the first body depth image to obtain the first body depth image matching. a first feature point, mapping the first feature point to the first three-dimensional model to obtain a corresponding first feature point of the model, acquiring weight information of the first feature point of the model on the skin point, and acquiring the first feature according to the first body depth image The motion trajectory of the point generates a first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information. Since the depth image carries the depth information, it is three-dimensional spatial information, so that the motion of the first feature point acquired according to the depth image is obtained. The trajectory is a three-dimensional motion trajectory, and the first three-dimensional animation corresponding to the first three-dimensional model can be automatically generated according to the motion trajectory and the influence weight information of the first feature point of the model on the skin point, and the three-dimensional position information is not required to be worn by the sensor, which is simple and convenient.

[Description of the Drawings]

1 is a flow chart of a method for generating a three-dimensional animation in an embodiment;

2 is a flow chart of establishing a three-dimensional model and determining weight information in one embodiment;

3 is a flow chart of determining weight information in an embodiment;

4 is a flow chart of generating a first three-dimensional animation corresponding to a first three-dimensional model according to a motion trajectory and an influence weight information in one embodiment;

Figure 5 is a flow chart of wearing an accessory model in one embodiment;

6 is a structural block diagram of an apparatus for generating a three-dimensional animation in an embodiment;

7 is a structural block diagram of an apparatus for generating a three-dimensional animation in another embodiment;

FIG. 8 is a structural block diagram of a three-dimensional animation generating module in an embodiment; FIG.

9 is a structural block diagram of an apparatus for generating a three-dimensional animation in still another embodiment;

Figure 10 is a schematic view showing the generation of skin according to feature points in one embodiment;

Figure 11 is a schematic diagram of a three-dimensional animated diagram in one embodiment;

12 is a schematic diagram of model feature points on a three-dimensional human body model in an embodiment;

FIG. 13 is a schematic diagram of a three-dimensional skeleton of a human body half body established in an embodiment; FIG.

14 is a schematic diagram of a three-dimensional skeleton of a whole body of a human body established in an embodiment;

Figure 15 is a schematic diagram of a three-dimensional skeleton of a dog established in an embodiment;

16 is a schematic diagram of a three-dimensional human body animation generated in one embodiment;

17 is a three-dimensional animation diagram of a dog generated in one embodiment;

Figure 18 is a block diagram showing the structure of a three-dimensional animation generating module in one embodiment.

【detailed description】

In one embodiment, as shown in FIG. 1, a method for generating a three-dimensional animation is provided, including the following steps:

Step S110: Acquire a first body depth image, and acquire a pre-established first three-dimensional model corresponding to the first body depth image, where the body is a human body or an animal having bones.

Specifically, the body is a human body or an animal with bones, such as a dog, and the depth image can be acquired by a depth image camera, such as a binocular camera or a plurality of pairs of camera devices, which are obtained by averaging different depth images of the same image. After obtaining the first body depth image, the first body depth image is processed, such as removing the background to separate the body contour and the like. After the body contour is obtained, the analysis is performed, for example, the body contour is the complete master Body, then obtain the corresponding complete personal 3D model. If the body contour is only the head, the head is recognized, and a three-dimensional model of the head corresponding to the head is acquired. If the body contour includes the head and the arms or the two forelimbs of the animal, a three-dimensional three-dimensional model corresponding to the head and the arms or the two forelimbs is acquired. Since the first three-dimensional model is pre-established, it can be quickly matched according to the first body depth image, which improves the efficiency. In one embodiment, the pre-established first three-dimensional model may adjust the shape according to the first body depth image, such as adjusting the height, adjusting the length ratio of the limbs, and the like. The first three-dimensional model is made to match the first body depth image more closely. In an embodiment, after acquiring the first body depth image, establishing a first three-dimensional model corresponding to the first body depth image according to the first body depth image, since the first three-dimensional model is established in real time according to the first body depth image, The first three-dimensional model is dynamically generated according to the first body depth image, and is more matched. When the first three-dimensional model is established, the first body color image corresponding to the first body depth image may be acquired, and the first three-dimensional model is established according to the chromaticity information in the first body color image, such as adjusting the skin color of the skin color or the animal , clothes color, etc. It can be understood that if the three-dimensional animation is generated only from the depth image, the generated three-dimensional animation has no color, and if the three-dimensional animation is generated according to the depth image and the corresponding color image, the generated three-dimensional animation is colored.

Step S120: Acquire a first feature point that is matched by the first body depth image, map the first feature point to the first three-dimensional model to obtain a corresponding first feature point of the model, and obtain information about the influence weight of the first feature point of the model on the skin point.

Specifically, different depth images correspond to different first feature points, and the position and number of the first feature points correspond to the depth image. If the depth image corresponds to the body contour as a complete body, the first feature point is the body limb end, the joint and the head facial position. If the depth image corresponds to the body contour as the head, the first feature point is the facial facial position. When acquiring the first feature point of the first body depth image matching, the color image corresponding to the first body depth image may be acquired, and the color image corresponding image point position is obtained according to the image recognition of the color image, such as the facial features position recognition, due to the color The image corresponds to the depth image, and the feature point position on the depth image can be matched according to the position of the feature point on the color image. The number of the first feature points can be customized as needed. If the number of the first feature points is appropriate, the accuracy of the three-dimensional animation obtained by the subsequent feature points is high. The key parts such as the facial feature point density can be set high, the obtained 3D animation effect is more accurate, and the expression is more realistic. It should be noted that since the depth image carries the depth information, that is, the size of the pixel value in the depth map reflects the depth of the depth of field, the position of the obtained first feature point is It is the position in three-dimensional space. Since the first body depth image corresponds to the first three-dimensional model, according to the mapping relationship between the first body depth image and the first three-dimensional model, mapping is performed to obtain a first feature point of the model corresponding to the first feature point on the first three-dimensional model. s position.

The first feature point generally includes a plurality of feature points, and each of the feature points may be connected to form a corresponding bone according to the distribution of the body, and the first feature points of the model may be connected according to the distribution of the body to form a corresponding three-dimensional animated bone. The first three-dimensional model is independent of the three-dimensional animated bone. The first three-dimensional model is equivalent to the skin, including the individual skin points. Once the skin is bound to the three-dimensional animated bone, the skin can follow the corresponding motion of the movement of the three-dimensional animated bone. In order for the skin to follow the motion of the 3D animated bone, it is necessary to set the corresponding weight of the 3D animated bone to each skin point on the first 3D model. The corresponding influence weight of the three-dimensional animated bone on each skin point on the first three-dimensional model is converted into the weight of the first feature point of the model on the skin point, and the influence weight information includes: the range of the skin point affected by the first feature point of the model, the model The first feature point affects the weight coefficient of the skin point. The weight coefficient size and the skin point are related to the position of the first feature point of the model. The weight coefficient of the first feature point of the model to the skin points at different positions is generally determined in combination with the skeleton characteristics of the subject.

Step S130: Acquire a motion trajectory of the first feature point according to the first body depth image, and generate a first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information.

Specifically, the first body depth image acquired at different times is obtained, and the motion track of the first feature point is obtained according to the coordinate position change of the corresponding first feature point on the first body depth image at different time points. The motion trajectory of the first feature point is mapped to the motion trajectory of the first feature point of the model on the first three-dimensional model. Since the motion trajectory of the first feature point is obtained from the depth image, the depth information is a spatial three-dimensional motion trajectory, and the model The motion trajectory of the first feature point is also a spatial three-dimensional motion trajectory. According to the influence weight information of the first feature point of the model on the motion trajectory, the range of the skin point and the weight coefficient affecting the skin point are determined, and the update of the skin point by the first feature point of the model is calculated according to the weight coefficient and the motion trajectory. The spatial coordinates, which result in the updated skin, and the continuous skin changes form the first three-dimensional animation. If the first body depth image is the motion of the body head, the three-dimensional animated expression corresponding to the expression change is generated according to the feature points on the motion track and the influence weight information according to the motion track corresponding to the different expression changes collected. For example, when laughing, the corners of the mouth rise, and the feature points corresponding to the corners of the mouth form a movement trajectory toward the left and right sides. According to the weight coefficient of the characteristic points corresponding to the corners of the mouth on the skin at different positions, the change of the skin points within the influence range when the mouth angle is raised is obtained. Thereby a convex muscle change effect is obtained. As shown in FIG. 10, a schematic diagram of generating skin points corresponding to the position according to the motion trajectory of the feature point and the influence of the weight information on the skin point, the position of each skin point can be according to the formula:

Calculated, where d ₁ represents the starting skin point, d ₂ represents the skin point after exercise, a ₁ , b ₁ , c ₁ respectively represent the starting feature points, and a ₂ , b ₂ and c ₂ respectively represent the characteristics after exercise. Points, α, β, and λ are the weight values of the respective feature points, f() represents the trajectory of the calculated feature points, and g(_) represents the trajectory of the skin points. As shown in FIG. 11 , a three-dimensional animated map generated after the skin position is completely determined is shown in FIG. 16 as a three-dimensional human body animation diagram, and FIG. 17 is a three-dimensional animated diagram of the produced dog.

In this embodiment, by acquiring the first body depth image, acquiring a pre-established first three-dimensional model corresponding to the first body depth image, acquiring a first feature point matched by the first body depth image, and mapping the first feature point to The first three-dimensional model obtains the corresponding first feature point of the model, acquires the weight information of the first feature point of the model on the skin point, acquires the motion trajectory of the first feature point according to the first body depth image, and generates the motion trajectory according to the motion trajectory and the influence weight information. The first three-dimensional animation corresponding to the first three-dimensional model, because the depth image carries the depth information, is three-dimensional spatial information, so that the motion trajectory of the first feature point acquired according to the depth image is a three-dimensional motion trajectory, and according to the motion trajectory and the model The weight information of the first feature point on the skin point can automatically generate the first three-dimensional animation corresponding to the first three-dimensional model, and the sensor does not need to wear the sensor to collect the three-dimensional position information, which is simple and convenient.

In an embodiment, as shown in FIG. 2, before step S110, the method further includes:

Step S210: acquiring body depth images of different forms, and establishing different three-dimensional models for body depth images of different forms.

Specifically, the body depth image of different forms is processed, such as removing the background to separate the body contour. After the body contour is obtained, the analysis is performed. If the body contour is a complete body, a corresponding complete personal three-dimensional model is established. If the body contour is only half-length and does not include the arm, the head is recognized, and a three-dimensional model of the head corresponding to the head is established. If the body contour includes the head and the arm, a half-length three-dimensional model corresponding to the head and the arm is established. When the three-dimensional model is established, a body color image corresponding to the body depth image may be acquired, and a three-dimensional model may be established according to the chromaticity information in the body color image, such as adjusting skin color, clothing color, and the like. Create a three-dimensional model corresponding to the body depth image of different forms in advance, and generate a three-dimensional animation The matching established 3D model can be directly searched according to the real-time acquired body depth image, and the speed of 3D animation generation is accelerated.

Step S220, setting feature points corresponding to the body depth images of different forms, and mapping the feature points to the three-dimensional model to obtain corresponding model feature points.

Specifically, the position and the number of feature points corresponding to the body depth image of different forms may be customized. For example, if the depth image corresponds to the body contour as a complete body, the feature points are the end of the main limb, the joint, and the facial features of the head, such as depth. The image corresponds to the outline of the body as the head, and the feature point is the facial features. The position of the feature point can be manually calibrated or automatically recognized. In the automatic recognition, the color image corresponding to the depth image of the body can be obtained, and the color image can be image-recognized according to the facial image, such as the facial features recognition. The image corresponds to the depth image, and the feature point position on the depth image can be matched according to the position of the feature point on the color image. Since the body depth image corresponds to the three-dimensional model, according to the mapping relationship between the body depth image and the three-dimensional model, the mapping is performed to obtain the position of the model feature point corresponding to the feature point on the three-dimensional model. As shown in FIG. 12, it is a schematic diagram of model feature points on the three-dimensional model of the human body. It can be seen from the figure that the feature points of the head and the hand are more dense, and a more accurate facial expression and hand motion three-dimensional animation can be generated. As shown in FIG. 10, it is a schematic diagram of feature points on a three-dimensional model of a dog.

Step S230, the three-dimensional animated skeleton of the three-dimensional model is established according to the model feature points, and the weight information of the influence of the model feature points on the skin points is determined according to the positional relationship between the skin point position and the three-dimensional animated bone.

Specifically, the range of influence of the model feature points on the skin points can be customized according to requirements, for example, the distance between the feature points and the skin points is set when the distance from the feature points is less than the preset threshold. The weight coefficient can be customized according to the positional relationship between the skin point position and the three-dimensional animated bone, such as calculating the vertical distance between the skin point and the three-dimensional animated bone, and determining the weight coefficient of the feature point connecting the three-dimensional animated bone to the skin point according to the vertical distance. As shown in FIG. 13 , a schematic diagram of a three-dimensional skeleton of a human body half body is established, as shown in FIG. 14 , which is a schematic diagram of a three-dimensional skeleton of a human body. As shown in Figure 15, a three-dimensional skeleton diagram of the dog is established.

In an embodiment, as shown in FIG. 3, step S230 includes:

Step S231, determining an influence range of the three-dimensional animated skeleton according to the skeleton feature of the subject.

Specifically, according to the skeleton characteristics of the subject, the feature points of the model are connected according to the distribution of the subject. The 3D animated bone determines the range of skin points affected by the 3D animated bone according to the skeleton characteristics of the subject. The 3D animated bone is connected by the model feature points. The model feature points on the 3D animated bone affect the skin point range. Animated bones affect the range of skin points.

Step S232, determining, according to the skeleton characteristics of the subject, a weight coefficient of the influence of the model feature points on the skin points at different positions within the influence range, wherein the influence of the model feature points on the skin points when determining the weight coefficient is inversely proportional to the distance between the two main body.

Specifically, the weight coefficient of the model feature points on the skin points is determined by the distance relationship between the model feature points and the skin points, and the body bone characteristics are determined according to the body bone characteristics, and the stretching degrees of the muscles at different positions are different, for example, the three-dimensional animated bone is an arm. Since the bone movement on the arm has little effect on the stretching motion of the muscle, a smaller weight coefficient a1 of the main body bone feature is set, and then the distance weight coefficient a2 is obtained according to the distance between the skin point and the feature point on the three-dimensional animated bone. Multiply a1 and a2 to obtain the final weight coefficient. The final weight coefficient is expressed as a function of the distance between the skin point and the feature point. When the 3D animation is subsequently generated, the corresponding weight coefficient can be directly obtained according to the distance.

In an embodiment, as shown in FIG. 4, step S130 includes:

Step S131, mapping feature points on the motion trajectory according to the depth information to the first three-dimensional model to obtain spatial three-dimensional coordinates of the model feature points.

Specifically, since the motion trajectory is obtained from the depth image, the depth image includes depth information, and the feature point includes three-dimensional spatial information, so that the feature point is mapped to the first three-dimensional model to obtain a spatial three-dimensional coordinate.

Step S132: Acquire a first influence range of the model feature point according to the influence weight information, obtain a first skin point in the first influence range, and calculate the first space according to the original space three-dimensional coordinate of the first skin point and the spatial three-dimensional coordinate of the model feature point. The spatial positional relationship between skin points and model feature points.

Specifically, the influence weight information includes a range of the skin points affected by each feature point, and the first skin point in the first influence range is obtained, and the other skin points are not affected by the feature points because they are behind the influence range, for example, A model of 2 arms in which the movement of one arm does not affect the skin points on the other arm. The spatial positional relationship can be calculated according to needs. If the obtained weight coefficient is related to the distance between the skin point and the model feature point, the point-to-point distance between the skin point and the model feature point is directly calculated according to the spatial three-dimensional coordinate. If the obtained weight coefficient is formed with skin points and different model feature points The bone distance is related, and the point-to-line distance between the skin points and the bones formed by the model feature points can be calculated according to the spatial three-dimensional coordinates.

Step S133, obtaining a weight coefficient of the first skin point according to the spatial positional relationship.

Specifically, the weight coefficient corresponding to the distance may be obtained as the weight coefficient of the first skin point according to the calculated spatial distance. It can be understood that if there are multiple model feature points affecting the first skin point, the weight coefficients of the first skin point affected by the plurality of model feature points are obtained according to the spatial position relationship, and the weight coefficients are further weighted. Get the final weighting factor.

Step S134, calculating the updated spatial three-dimensional coordinates of the first skin point according to the weight coefficient, and moving the first skin point from the original space three-dimensional coordinates to the update space three-dimensional coordinates.

Specifically, the motion trajectory of the feature point obtains the motion trajectory of each model feature point by mapping to the first three-dimensional model, and calculates the updated space coordinate of the first skin point according to the weight coefficient and the trend of the motion trajectory of each model feature point, such as the model feature. The point is upward motion, the motion distance is b1, and the weight coefficient is b2, then the updated space coordinate of the first skin point is obtained as a function of the three-dimensional coordinates of b1 and b2 and the original space, and the three-dimensional coordinates of the update space are calculated according to the function. Specific function formulas can be customized as needed. Changes in the spatial coordinates of successive skin points form a three-dimensional animation. For the head model, when the captured body depth image corresponds to the head rotation, the three-dimensional animation also follows the head rotation. If the captured body depth image corresponds to a smile face, the three-dimensional animation also follows the smile face.

In an embodiment, as shown in FIG. 5, the method further includes:

Step S310, acquiring an accessory model, and acquiring weight information of the accessory influence weight of the first feature point of the model on the accessory model.

Specifically, according to the type matching of the accessory model, the target model feature points of the accessory model can be affected. For example, if the accessory model is a hat, the target model feature point is a head model feature point, and if the accessory model is glasses, the target model feature point is a nose. Part model feature points. In addition to the target model feature points, the other model feature points have a weighting factor of 0 for the accessories of the accessory model. The influence weight information includes the weight coefficient and influence range of the accessory's first feature point on the accessory model. Only the points on the accessories in the affected range will be affected by the first feature points. For example, the point on the edge of the hat will be affected by the first feature point of the model, and the point at the top of the hat will not be affected by the first feature point of the model. The influence weight coefficient determines the influence of each model feature point on the accessory model when the first feature point of the model is moved. For example, the accessory model is a hat and a hat. The point of influence of the point closer to the head is larger.

Step S320, changing the form of the accessory model according to the position information of the first feature point of the model and the accessory influence weight information, and wearing the changed accessory model to a position corresponding to the first three-dimensional model.

Specifically, the position information of the first feature point of the model includes a distance between two model feature points and a movement trajectory of the feature point, and the shape of the accessory model can be changed according to the position information of the first feature point of the model and the weight information of the accessory influence, such as Size, location, etc. The changed accessory model is matched with the first three-dimensional model so that the changed accessory model can be worn to a position corresponding to the first three-dimensional model. If the head of the user A is wider, the distance between the two model feature points of the user A's head is larger, so that the original accessory model, such as a hat, is enlarged so that it matches the user's head.

In one embodiment, the first three-dimensional model is a three-dimensional head model, and the first three-dimensional animation is an avatar three-dimensional animation.

Specifically, when the first three-dimensional model is a three-dimensional model of the head, the obtained weight information of the first feature point on the skin point is matched with the structure of the head, and the motion of the skin is obtained according to the motion track of the feature point of the head. Thereby getting different expressions and getting dynamic avatar 3D animation. It can be used for real-time video chat between users to generate corresponding three-dimensional animations with facial expression changes, which increases the interest of communication.

In an embodiment, step S130 includes: determining whether the skin point of the three-dimensional animation to be generated is a skin point of a corresponding position on the depth image acquired by the camera, and if yes, generating the first three-dimensional animation of the corresponding position according to the depth image. Skin points, otherwise other skin points of the first three-dimensional animation corresponding to the first three-dimensional model are generated according to the motion trajectory and the influence weight information.

Specifically, since the depth image carries the depth information and is the three-dimensional spatial information, the spatial position coordinates can be directly obtained according to the depth information for each point on the depth image, thereby determining the position of each skin point, such as the side facing the camera. The skin points have corresponding points on the depth image, and the three-dimensional positions of the skin points can be directly obtained, thereby generating corresponding three-dimensional animated skin points. For the part that is not captured by the camera, the part facing away from the camera does not have a corresponding point on the depth image, and these skin points need to be generated according to the motion trajectory and the influence weight information. For skin points with depth information, the corresponding 3D animation can be generated directly from the depth information, which can speed up the generation of 3D animation.

In one embodiment, the first body depth image is a depth image in the RGBD image, and the RGBD image further includes a corresponding color image, the first three-dimensional animation being a colored three-dimensional animation.

Specifically, the RGBD image is a synchronized depth image and a color image collected by the camera. Since the collected information includes a color image, and the color image corresponds to each point of the depth image one by one, the generated three-dimensional animation is colored.

In one embodiment, as shown in FIG. 6, an apparatus for generating a three-dimensional animation is provided, including:

The depth image and model acquisition module 410 is configured to acquire a first body depth image, and acquire a pre-established first three-dimensional model corresponding to the first body depth image, where the body is a human body or an animal having bones.

The feature point and weight acquisition module 420 is configured to acquire a first feature point of the first body depth image matching, map the first feature point to the first three-dimensional model to obtain a corresponding first feature point of the model, and acquire a first feature point pair of the model Skin point influence weight information.

The three-dimensional animation generating module 430 is configured to acquire a motion trajectory of the first feature point according to the first body depth image, and generate a first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information.

In an embodiment, as shown in FIG. 7, the apparatus further includes:

The pre-processing module 440 is configured to acquire body depth images of different forms, establish different three-dimensional models for body depth images of different forms, set feature points corresponding to body depth images of different forms, and map the feature points to the three-dimensional model to obtain corresponding The model feature points are based on the model feature points to establish a three-dimensional animated skeleton of the three-dimensional model, and the weight information of the model feature points on the skin points is determined according to the positional relationship between the skin point position and the three-dimensional animated bone.

In one embodiment, the pre-processing module 440 is further configured to determine a range of influence of the three-dimensional animated bone according to the subject bone feature, and determine, according to the subject bone feature, a weight coefficient of the model feature point affecting the skin point of the different position within the influence range, wherein When the weight coefficient is determined, the influence of the model feature points on the skin points is inversely proportional to the distance between the two.

In one embodiment, as shown in FIG. 8, the three-dimensional animation generating module 430 includes:

The feature point coordinate unit 431 is configured to map feature points on the motion track to the first three-dimensional model according to the depth information to obtain spatial three-dimensional coordinates of the model feature points.

The spatial relationship calculation unit 432 is configured to acquire the first impact range of the model feature point according to the influence weight information, and acquire the first skin point in the first influence range, according to the original space three-dimensional coordinate of the first skin point and the space of the model feature point. Calculate the spatial position of the first skin point and the model feature point in three-dimensional coordinates system.

The updating unit 433 is configured to obtain a weight coefficient of the first skin point according to the spatial position relationship, calculate a three-dimensional coordinate of the update space of the first skin point according to the weight coefficient, and move the first skin point from the original space three-dimensional coordinate to the update space three-dimensional coordinate.

In an embodiment, as shown in FIG. 9, the apparatus further includes:

The accessory module 450 is configured to acquire an accessory model, obtain the accessory influence weight information of the first feature point of the model on the accessory model, and change the shape of the accessory model according to the position information of the first feature point of the model and the influence weight information of the accessory, and then change The accessory model is worn to the position corresponding to the first three-dimensional model.

In one embodiment, as shown in FIG. 18, the three-dimensional animation generating module 430 includes:

The determining unit 434 is configured to determine whether the skin point of the three-dimensional animation to be generated is a skin point of a corresponding position on the depth image collected by the camera, and if yes, enter the first generating unit, and otherwise enter the second generating unit.

The first generating unit 435 is configured to directly generate a skin point of the first three-dimensional animation corresponding to the position according to the depth image.

The second generating unit 436 is configured to generate other skin points of the first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information.

A person skilled in the art can understand that all or part of the process of implementing the above embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium, such as the present invention. In an embodiment, the program can be stored in a storage medium of the computer system and executed by at least one processor in the computer system to implement a process comprising an embodiment of the methods as described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

The technical features of the above-described embodiments may be arbitrarily combined. For the sake of brevity of description, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction between the combinations of these technical features, All should be considered as the scope of this manual.

The above-described embodiments are merely illustrative of several embodiments of the present invention, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be determined by the appended claims.

Claims

A method for generating a three-dimensional animation, the method comprising:

Obtaining a first body depth image, and acquiring a pre-established first three-dimensional model corresponding to the first body depth image, the body being a human body or an animal having bones;

Obtaining a first feature point that is matched by the first body depth image, and mapping the first feature point to a first three-dimensional model to obtain a corresponding first feature point of the model;

Obtaining weight information of the first feature point of the model on the skin point;

Obtaining a motion trajectory of the first feature point according to the first body depth image, the first body depth image is a depth image in an RGBD image, and the RGBD image further includes a corresponding color image, the first three-dimensional The animation is a colored three-dimensional animation, and the first three-dimensional animation corresponding to the first three-dimensional model is generated according to the motion trajectory and the influence weight information;

Obtaining an accessory model, and acquiring weight information of the accessory influence attribute of the first feature point of the model on the accessory model;

Changing a form of the accessory model according to position information of the first feature point of the model and the accessory influence weight information;

The changed accessory model is worn to a position corresponding to the first three-dimensional model.
The method according to claim 1, wherein before the step of acquiring the first body depth image, the method further comprises:

Obtaining a body depth image of different forms, and establishing different three-dimensional models for the body depth images of the different shapes;

And setting feature points corresponding to the different depths of the body depth image, and mapping the feature points to the three-dimensional model to obtain corresponding model feature points;

Establishing a three-dimensional animated skeleton of the three-dimensional model according to the model feature point;

The weight information of the influence of the model feature points on the skin points is determined according to the positional relationship between the skin point position and the three-dimensional animated bone.
The method according to claim 2, wherein the step of determining the influence weight information of the model feature point on the skin point according to the positional relationship between the skin point position and the three-dimensional animated bone include:

Determining the influence range of the three-dimensional animated bone according to the skeleton characteristics of the subject;

The weight coefficient of the influence of the model feature points on the skin points at different positions within the influence range is determined according to the body skeleton feature, wherein the influence of the model feature points on the skin points is inversely proportional to the distance between the two.
The method according to claim 1, wherein the step of generating the first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information comprises:

Mapping the feature points on the motion trajectory according to the depth information to the first three-dimensional model to obtain spatial three-dimensional coordinates of the model feature points;

Acquiring, according to the impact weight information, a first influence range of the model feature point;

Obtaining a first skin point in the first influence range, and calculating a spatial position relationship between the first skin point and the model feature point according to the original space three-dimensional coordinates of the first skin point and the spatial three-dimensional coordinates of the model feature point;

Obtaining a weight coefficient of the first skin point according to the spatial position relationship;

Calculating an updated spatial three-dimensional coordinate of the first skin point according to the weight coefficient, and moving the first skin point from the original space three-dimensional coordinate to the update space three-dimensional coordinate.
The method according to claim 1, wherein the step of generating the first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information comprises:

Determining whether the skin point of the three-dimensional animation to be generated is a skin point of a corresponding position on the depth image acquired by the camera, and if so, directly generating a skin point of the first three-dimensional animation corresponding to the position according to the depth image, otherwise according to the motion The trajectory and influence weight information generates other skin points of the first three-dimensional animation corresponding to the first three-dimensional model.
A method for generating a three-dimensional animation, the method comprising:

Obtaining a first body depth image, and acquiring a pre-established first three-dimensional model corresponding to the first body depth image, the body being a human body or an animal having bones;

Obtaining a first feature point that is matched by the first body depth image, and mapping the first feature point to a first three-dimensional model to obtain a corresponding first feature point of the model;

Obtaining weight information of the first feature point of the model on the skin point;

Obtaining a motion trajectory of the first feature point according to the first body depth image, and generating a first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information.
The method according to claim 6, wherein before the step of acquiring the first body depth image, the method further comprises:

Obtaining a body depth image of different forms, and establishing different three-dimensional models for the body depth images of the different shapes;

And setting feature points corresponding to the different depths of the body depth image, and mapping the feature points to the three-dimensional model to obtain corresponding model feature points;

Establishing a three-dimensional animated skeleton of the three-dimensional model according to the model feature point;

The weight information of the influence of the model feature points on the skin points is determined according to the positional relationship between the skin point position and the three-dimensional animated bone.
The method according to claim 7, wherein the step of determining the influence weight information of the model feature point on the skin point according to the positional relationship between the skin point position and the three-dimensional animated bone comprises:

Determining the influence range of the three-dimensional animated bone according to the skeleton characteristics of the subject;

The weight coefficient of the influence of the model feature points on the skin points at different positions within the influence range is determined according to the body skeleton feature, wherein the influence of the model feature points on the skin points is inversely proportional to the distance between the two.
The method according to claim 6, wherein the step of generating the first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information comprises:

Mapping the feature points on the motion trajectory according to the depth information to the first three-dimensional model to obtain spatial three-dimensional coordinates of the model feature points;

Acquiring, according to the impact weight information, a first influence range of the model feature point;

Obtaining a first skin point in the first influence range, and calculating a spatial position relationship between the first skin point and the model feature point according to the original space three-dimensional coordinates of the first skin point and the spatial three-dimensional coordinates of the model feature point;

Obtaining a weight coefficient of the first skin point according to the spatial position relationship;

Calculating, according to the weight coefficient, an updated spatial three-dimensional coordinate of the first skin point, the first The skin point is moved from the original space three-dimensional coordinates to the three-dimensional coordinates of the update space.
The method of claim 6 wherein the method further comprises:

Obtaining an accessory model, and acquiring weight information of the accessory influence attribute of the first feature point of the model on the accessory model;

Changing a form of the accessory model according to position information of the first feature point of the model and the accessory influence weight information;

The changed accessory model is worn to a position corresponding to the first three-dimensional model.
The method according to claim 6, wherein the step of generating the first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information comprises:

Determining whether the skin point of the three-dimensional animation to be generated is a skin point of a corresponding position on the depth image acquired by the camera, and if so, directly generating a skin point of the first three-dimensional animation corresponding to the position according to the depth image, otherwise according to the motion The trajectory and influence weight information generates other skin points of the first three-dimensional animation corresponding to the first three-dimensional model.
The method according to claim 6, wherein the first body depth image is a depth image in an RGBD image, the RGBD image further comprising a corresponding color image, the first three-dimensional animation being a colored three-dimensional animation .
A device for generating a three-dimensional animation, characterized in that the device comprises:

a depth image and model acquisition module, configured to acquire a first body depth image, and acquire a pre-established first three-dimensional model corresponding to the first body depth image, the body being a human body or an animal having bones;

a feature point and weight acquisition module, configured to acquire a first feature point matched by the first body depth image, map the first feature point to a first three-dimensional model to obtain a corresponding first feature point of the model, and acquire the model Weight information of the influence of the first feature point on the skin point;

a three-dimensional animation generating module, configured to acquire a motion trajectory of the first feature point according to the first body depth image, and generate a first three-dimensional animation corresponding to the first three-dimensional model according to the motion trajectory and the influence weight information.
The device according to claim 13, wherein the device further comprises:

a pre-processing module, configured to acquire body depth images of different forms, and establish different three-dimensional models for the body depth images of different shapes, and set features corresponding to the body depth images of the different shapes Pointing, mapping the feature point to the three-dimensional model to obtain a corresponding model feature point, establishing a three-dimensional animated bone of the three-dimensional model according to the model feature point, and determining the model feature according to a positional relationship between the skin point position and the three-dimensional animated bone Point weight-to-skin point influence weight information.
The apparatus according to claim 14, wherein the pre-processing module is further configured to determine an influence range of the three-dimensional animated bone according to a subject skeleton feature, and determine that the model feature point is within the influence range according to the subject skeleton feature The weight coefficient of the skin point effect at different positions, wherein the influence of the model feature point on the skin point when determining the weight coefficient is inversely proportional to the distance between the two. main body
The device according to claim 13, wherein the three-dimensional animation generating module comprises:

a feature point coordinate unit, configured to map the feature points on the motion track to the first three-dimensional model according to the depth information to obtain spatial three-dimensional coordinates of the model feature points;

a spatial relationship calculation unit, configured to acquire a first influence range of the model feature point according to the influence weight information, acquire a first skin point in the first influence range, and obtain a three-dimensional coordinate of the original space according to the first skin point The spatial three-dimensional coordinates of the model feature points are used to calculate the spatial positional relationship between the first skin point and the model feature points;

An update unit, configured to obtain a weight coefficient of the first skin point according to the spatial position relationship, calculate an updated spatial three-dimensional coordinate of the first skin point according to the weight coefficient, and use the first skin point from the original space three-dimensional coordinate Move to the update space three-dimensional coordinates.
The device according to claim 13, wherein the device further comprises:

The accessory module is configured to obtain an accessory model, obtain the accessory influence weight information of the first feature point of the model on the accessory model, and change the shape of the accessory model according to the position information of the first feature point of the model and the accessory influence weight information. The changed accessory model is worn to a position corresponding to the first three-dimensional model.
The device according to claim 13, wherein the three-dimensional animation generating module comprises:

a determining unit, configured to determine whether a skin point of the three-dimensional animation to be generated is a skin point corresponding to a position on the depth image collected by the camera, and if yes, enter the first generating unit, and otherwise enter the second generating unit;

a first generating unit, configured to directly generate a skin point of the first three-dimensional animation corresponding to the position according to the depth image;

And a second generating unit, configured to generate, according to the motion trajectory and the influence weight information, other skin points of the first three-dimensional animation corresponding to the first three-dimensional model.
The apparatus according to claim 13, wherein said first body depth image is a depth image in an RGBD image, said RGBD image further comprising a corresponding color image, said first three-dimensional animation being a colored three-dimensional animation .