KR100933957B1 - 3D Human Body Pose Recognition Using Single Camera - Google Patents

Abstract

The present invention relates to a three-dimensional human body pose recognition method using a single camera, and more specifically, to generate and store a plurality of reference pose models based on the feature points of the silhouette image of the human body as a reference having a different pose, and a single camera The present invention relates to a three-dimensional human body pose recognition method using a single camera for estimating a three-dimensional pose and a shooting direction of a human body to be detected by generating a detection pose model of a human body to be detected using the reference pose models.

3D Pose, Single Camera, Silhouette, Feature Point, Histogram, Boundary Image

Description

Recognition method of three-dimensional human pose using a single camera {RECOGNITION METHOD OF 3 DIMENSIONAL HUMAN POSE USING A SINGLE CAMERA}

The present invention relates to a three-dimensional human body pose recognition method using a single camera, and more specifically, to generate and store a plurality of reference pose models based on the feature points of the silhouette image of the human body as a reference having a different pose, and a single camera The present invention relates to a three-dimensional human body pose recognition method using a single camera for estimating a three-dimensional pose and a shooting direction of a human body to be detected by generating a detection pose model of a human body to be detected using the reference pose models.

The pose of the human body represents the intention of human behavior, and thus it is interpreted and used in various computer science fields. Mainly, the pose of the human body is used in the security field, the interaction field with the robot or the pose recognition field.

Generally, multiple or single cameras are used to recognize the pose of the human body. The method of using multiple cameras compensates for the disadvantages of different poses of the human body depending on the direction of the human body or the position of the camera. The disadvantage is that the calculation is complicated.

In addition, when the single camera is used, the cost is simple and the calculation is simple, so it is easy to apply in the actual field, but it is difficult to directly obtain a three-dimensional pose of the human body.

The present invention was devised to solve the above problems, and an object of the present invention is to provide a three-dimensional human body pose recognition method using a single camera that can estimate and recognize the three-dimensional pose of the human body using a single camera.

It is also an object of the present invention to provide a three-dimensional human body pose model generation method that can generate a three-dimensional human body pose model that is an image feature of the human body having a three-dimensional pose using a plurality of cameras.

According to an embodiment of the present invention, a three-dimensional pose pose recognition method using a single camera according to an embodiment of the present invention may include: obtaining a plurality of reference two-dimensional pose images by photographing a reference object having an arbitrary three-dimensional pose in different directions; Step 1, extracting a reference silhouette image which is a silhouette image of each of the reference two-dimensional pose images, and generating a reference pose model that is an image characteristic of the reference object based on the reference silhouette image, based on the reference pose model The third step of storing in the pose model database, the fourth step of generating the reference pose models of the reference objects having different poses by repeating the first to third steps to store in the reference pose model database, a single camera Detects two-dimensional pose images of the detection subject in an arbitrary direction using A fifth step of obtaining, a sixth step of extracting a detection silhouette image which is a silhouette image of the detection two-dimensional pose image, and generating a detection pose model that is an image characteristic of the detection target based on the detection silhouette image and the reference pose model And a seventh step of estimating a three-dimensional pose of the detection target by deriving a reference pose model having the greatest similarity with the detection pose model among the reference pose models stored in the database.

In a preferred embodiment, the second step may include steps 2-1 of extracting the reference silhouette images by removing a background of each reference two-dimensional pose image, steps 2-2 of obtaining feature points of each reference silhouette image, Steps 2-3 of obtaining angle histogram values between the feature points of the respective reference silhouette images, steps 2-4 of obtaining average histograms by averaging the angle histogram values of the respective reference silhouette images, and principal component analysis of the average histogram. PCA: Principle Component Analysis (PCA).

In a preferred embodiment, the sixth step may include extracting the detection silhouette image by removing a background of the detection two-dimensional pose image, and extracting the detection silhouette image, and obtaining the feature points of the detection silhouette image. Steps 6-3 to obtain angle histogram values between the feature points of the silhouette image and Steps 6-4 to generate the detection pose model by projecting the angle histogram of the detection silhouette image to a principal component analysis space.

According to a preferred embodiment, after the seventh step, an angle histogram having the greatest similarity to the angle histogram value of the detection silhouette image among the angle histogram values of the reference silhouette images is derived, and the imaging direction of the detection target is estimated. 8 more steps.

In a preferred embodiment, the similarity of the seventh step is calculated as the Eucladian distance value.

In a preferred embodiment, the similarity of the eighth step is calculated as a diffusion distance value.

Method for generating a three-dimensional human body pose model according to an embodiment of the present invention for achieving the above object in the method for generating a human body pose model for providing an image feature of the human body having an arbitrary three-dimensional pose, arbitrary three-dimensional pose A first step of acquiring a plurality of two-dimensional pose images by photographing a human body having a different direction; a second step of obtaining silhouette images from which backgrounds of the two-dimensional pose images are removed; and a third step of obtaining feature points of each silhouette image And a fourth step of obtaining an angular histogram between the feature points of each of the silhouette images, and a fifth step of deriving an average histogram by averaging the angular histograms of the respective silhouette images.

In a preferred embodiment, after the fifth step, the method further includes a sixth step of projecting the average histogram into a principal component analysis space and transforming the average histogram into a characteristic point in the principal component analysis space, and converting the characteristic point in the principal component analysis space into the body pose model. Generated by image features.

As described above, according to the present invention, the three-dimensional pose of the human body can be estimated only by using the two-dimensional pose image of the human body photographed by a single camera by using the reference pose models. There is an effect that can provide a human body pose recognition method.

In addition, according to the present invention, by comparing the angular histogram values of the reference pose model and the angular histogram derived from the two-dimensional pose image of the human body has an effect that can provide a three-dimensional human body pose recognition method capable of estimating the human body shooting direction have.

In addition, according to the present invention, there is an effect that can provide a three-dimensional human body pose model generation method that can create a three-dimensional human body pose model that can be used in the field of security, interaction with robots, pose recognition field or real-time game production field have.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 to 5 are views for explaining a three-dimensional human body pose model generation method according to an embodiment of the present invention, Figure 6 is a flowchart of a three-dimensional human body pose model generation method according to an embodiment of the present invention.

Referring to the drawings, a method of generating a three-dimensional human body pose model according to an embodiment of the present invention, first, by photographing the human body 10 having an arbitrary three-dimensional pose from a plurality of cameras 20 from different directions, the camera A plurality of two-dimensional pose images P1, P2,..., And P8 corresponding to the number of pieces 20 are obtained (S1000).

In addition, the cameras 20 are composed of eight cameras and are spaced apart from each other at an angle of 45 degrees with respect to the human body 10.

However, the number of the cameras 20 may vary and may be spaced apart from each other at different angles. That is, the number and position of the cameras 20 are provided as the number and position for acquiring two-dimensional pose images P1, P2,..., And P8 capable of estimating a three-dimensional pose of the human body 10. Is enough.

Next, the plurality of two-dimensional pose images P1, P2,..., And P8 are obtained as silhouette images S1, S2,..., S8, respectively (S1100).

Meanwhile, the silhouette images S1, S2,..., And S8 remove the background portion 10b of the two-dimensional pose images P1, P2,..., And P8, and extract only the human body portion 10a. This is a shaded image.

That is, the silhouette images S1, S2,..., S8 are extracted as images of different types according to the photographing direction of the camera 20.

Next, feature points of each of the silhouette images S1, S2,..., S8 are obtained (S1200).

The feature points may be vertices of convex or concave polygons, such as convex hull points or concave points. However, the feature points may be points extracted according to a predetermined criterion among various points located on the outer lines of the silhouette images S1, S2,..., S8.

For example, a point where the curvature greatly changes in a point section of the outer line may be obtained as a feature point.

In addition, the convex hull points are points that allow the region C1 including the entire silhouette image to be derived when the convex hull points are connected to the outermost positions of the silhouette images S1, S2,..., S8. Say.

Next, an angular histogram V1 between the feature points of each of the silhouette images S1, S2,..., S8 is obtained (S1300).

In other words, one angular histogram V1 is derived for each of the silhouette images S1, S2, ..., S8.

FIG. 4 is a diagram showing that the angle histogram V1 is derived for one silhouette image S1. The same applies to the angle histogram derivation of all the silhouette images S1, S2,..., S8.

In addition, the angle histogram V1 is obtained through Equation 1 below.

Next, an average histogram Va1 is derived by averaging the angle histograms of the silhouette images S1, S2,..., S8 (S1400), and the average histogram Va1 is a human body having the arbitrary three-dimensional pose. Create a human body pose model of 10.

That is, the average histogram Va1 is generated as feature information of a human body pose model, which is an image feature of the human body 10.

In addition, the average histogram Va1 may be obtained from Equation 2 below.

Where P _i Denotes the average histogram Va1.

Meanwhile, in the exemplary embodiment of the present invention, since the average histogram Va1 has a large amount of data, the average histogram Va1 is projected to the feature point P ₁ in the principal component analysis space through PCA (Principle Component Analysis) to pose the feature point in the human body pose. Generated by the feature information of the model (S1500).

In addition, briefly explaining the generation process of the feature point (P ₁ ) First, the principal component analysis to obtain the eigen basis vector of the covariance matrix of the average histogram (Va1) by the following equation (3), and then, The feature point is selected by constructing only three vectors with high variance among the eigenbased vectors of the covariance matrix by 4 and constructing the basis vector of the principal component space, and then converting them to points in the three-dimensional basis vector space by Equation 5 below. (P ₁ ) is derived.

Where P _i Denotes the average histogram Va1.

That is, the method for generating a three-dimensional human body pose model according to an embodiment of the present invention may provide an image feature of a human body having an arbitrary three-dimensional pose as a human body pose model that is one feature point P ₁ in the principal component analysis space. The angle histogram V1 of each of the silhouette images S1, S2,..., S8 calculated in the derivation process of the human body pose model may be used as a data for deriving a photographing direction of the camera. .

7 to 8 are flowcharts of a three-dimensional human body pose recognition method using a single camera according to an embodiment of the present invention.

Hereinafter, the same reference numerals are used for the same components as those of FIGS. 1 to 6, and descriptions thereof will be omitted.

Referring to the drawings, a three-dimensional human body pose recognition method using a single camera according to an embodiment of the present invention, first, a plurality of reference two-dimensional poses by photographing a reference object, which is a human body 10 having an arbitrary three-dimensional pose, from different directions. Acquire images P1, P2, ... P8.

Also, the reference two-dimensional pose images P1, P2,..., P8 are images of a plurality of cameras installed toward the reference object and photographed in different directions with respect to an arbitrary pose of the reference object.

In addition, in one embodiment of the present invention, the reference two-dimensional pose images P1, P2, ... P8 are eight reference two-dimensional poses of eight cameras 20 arranged 45 degrees apart from the reference object. The images P1, P2, ... P8 were acquired (S2000).

Next, the reference silhouette images S1, S2,..., S3 which are silhouette images of each of the reference two-dimensional pose images P1, P2,..., P8 are extracted, and the reference silhouette images S1, A reference pose model P ₁ is generated based on S2,..., S3 (S2100).

Meanwhile, as shown in FIG. 8, the reference pose model P ₁ acquires reference silhouette images S1, S2,..., S3 of each reference two-dimensional pose image P1, P2,. After extracting the convex hull points of the respective reference silhouette images S1, S2, ..., S3 (S2120), an angle histogram value between the convex hull points is obtained (S2130), and then the angle is obtained. The average histogram of the histogram values is derived (S2140), and the average histogram is generated by projecting the average histogram into the principal component analysis space (S2150) (S2160).

In addition, since the reference pose model P ₁ is the same as the feature point P ₁ generated by the method for generating a three-dimensional human body pose model according to an embodiment of the present invention, a detailed description thereof will be omitted.

Next, the reference pose model P ₁ is stored in a reference pose model database (not shown) (S2200).

Next, the steps S2000, S2100, and S2200 are repeated to generate respective pose models for reference objects having different poses and store them in the pose model database (not shown) (S2300).

That is, the reference pose models, which are respective feature information, are generated and stored for the human body 10 having different poses.

Next, a detection two-dimensional pose image is acquired by photographing a detection target which is a human body to be detected in an arbitrary direction using a single camera (S2400).

Next, a detection silhouette image which is a silhouette image of the detection two-dimensional pose image is extracted, and a detection pose model is generated based on the detection silhouette image (S2500).

In addition, the method of generating the detection pose model is the same as the method of generating the reference pose model P ₁ , but since the detection pose model has only the angular histogram value derived from one detection two-dimensional pose image, an average histogram is obtained. There is a difference in that the detection pose model is generated by projecting the angular histogram of the detection silhouette image into the principal component analysis space without deriving it.

Next, a reference pose model having the highest similarity with the detection pose model among the reference pose models stored in the reference pose model database is derived to estimate the three-dimensional pose of the detection target as the three-dimensional pose of the reference pose model with the highest similarity. (S2600).

In addition, the similarity comparison between the detection pose model and the reference pose model is calculated as a Eucladian distance value.

Next, among the eight angle histograms of the silhouette image of the reference pose model having the largest similarity, the angle histogram having the greatest similarity to the angle histogram of the detection silhouette image of the detection pose model is derived to determine the camera of the camera. Estimate the shooting direction.

That is, the reference pose models stored in the reference pose model database may be used to estimate the three-dimensional pose of the human body photographed by a single camera, and may also estimate the detection object to the photographing direction.

Therefore, since a single camera is used, the manufacturing cost is small compared to the method using a plurality of conventional cameras, and the three-dimensional pose of the human body can be recognized by a simple comparison, and compared to the method using a conventional single camera, the three-dimensional pose of the human body There is an advantage that can be obtained directly.

In the above, the configuration and operation of the present invention has been shown in accordance with the above description and drawings, but this is merely described, for example, and various changes and modifications are possible without departing from the spirit and scope of the present invention. .

1 to 5 are views for explaining a three-dimensional human body pose model generation method according to an embodiment of the present invention,

6 is a flowchart of a method for generating a 3D human body pose model according to an embodiment of the present invention;

7 to 8 are flowcharts of a three-dimensional human body pose recognition method using a single camera according to an embodiment of the present invention.

In the drawings according to the present invention, the same reference numerals are used for components having substantially the same configuration and function.

<Description of the symbols for the main parts of the drawings>

10: human body 10a: human body part

10b: background part 20: camera

P1, P2, P3, P4, P5, P6, P7, P8: Two-dimensional pose image

S1, S2, S3, S4, S5, S6, S7, S8: Silhouette image

V1: Angle histogram Va1: Average histogram

P ₁ : Features

Claims (8)

delete delete A first step of photographing a reference object having an arbitrary three-dimensional pose in different directions to obtain a plurality of reference two-dimensional pose images; Extracting a reference silhouette image which is a silhouette image of each reference two-dimensional pose image, and generating a reference pose model that is an image characteristic of the reference object based on the reference silhouette image; Storing the reference pose model in a reference pose model database; A fourth step of repeating the first to third steps to generate reference pose models of reference objects having different poses and store them in the reference pose model database; A fifth step of acquiring a detection two-dimensional pose image of the detection object in an arbitrary direction using a single camera; Extracting a detection silhouette image which is a silhouette image of the detection two-dimensional pose image, and generating a detection pose model that is an image feature of the detection target based on the detection silhouette image; And A seventh model of estimating a reference pose model having the largest similarity with the detection pose model among the reference pose models stored in the reference pose model database, and estimating the derived reference pose model as the three-dimensional pose of the detection target; Comprising; The reference pose model of the second step is Extracting the reference silhouette images by removing a background of each reference two-dimensional pose image; A second step of obtaining feature points of each reference silhouette image; A third step of obtaining an angle histogram value between feature points of each reference silhouette image by Equation 1 below; <Equation 1>

(Histogram _si (i: 1 ~ 8) is the angle histogram value of each reference silhouette image, and feature _j (j: 1 ~ N-2) is an arbitrary feature point of each reference silhouette image.) A second to fourth step of obtaining an average histogram by Equation 2 below by averaging the angle histogram values of the respective reference silhouette images; And <Equation 2>

(here,

(i: 1 ~ N) is the average histogram value of each reference silhouette image, and Histogram _si is the angle histogram value of each reference silhouette image.) Generating the reference pose model by projecting the average histogram into a principal component analysis (PCA) space; The detection pose model of the sixth step is Extracting the detection silhouette image by removing a background of the detection two-dimensional pose image; A sixth step of obtaining feature points of the detection silhouette image; A sixth step of obtaining an angle histogram value between the feature points of the detection silhouette image by Equation 3 below; And <Equation 3>

(Where Histogram _s is an angle histogram value of the detection silhouette image, and feature _j (j: 1 to N-2) is an arbitrary feature point of the detection silhouette image.) And generating a detection pose model by projecting the angular histogram of the detection silhouette image into a principal component analysis space. The method of claim 3, wherein After the seventh step, the angle histogram value having the greatest similarity to the angle histogram value of the detection silhouette image among the angle histogram values of the reference silhouette images is derived, And an eighth step of estimating the photographing direction of the reference two-dimensional pose image, which is the basis for calculating the derived angle histogram value having the highest similarity, as the photographing direction of the detection target. Recognition method. The method of claim 4, wherein 3D human body pose recognition method using a single camera, characterized in that the similarity of the seventh step is calculated by the Euclidean distance value. The method of claim 4, wherein The similarity of the eighth step is calculated as a diffusion distance value 3D human body pose recognition method using a single camera. A method of generating a human body pose model for providing an image feature of a human body having an arbitrary three-dimensional pose, A first step of photographing a human body having an arbitrary three-dimensional pose in different directions to obtain a plurality of two-dimensional pose images; A second step of obtaining silhouette images from which backgrounds of the two-dimensional pose images are removed; Obtaining feature points of each of the silhouette images; A fourth step of obtaining an angle histogram between the feature points of each silhouette image by Equation 4 below; And <Equation 4>

(Histogram _si (i: 1 ~ 8) is the angle histogram value of each reference silhouette image, and feature _j (j: 1 ~ N-2) is an arbitrary feature point.) And a fifth step of deriving an average histogram by Equation 5 below by averaging the angle histograms of the respective silhouette images. <Equation 5>

(here,

(i: 1 ~ N) are average histogram values of silhouette images, and histogram _si is angle histogram values of each silhouette image.) And generating the average histogram as the human body pose model. The method of claim 7, And after the fifth step, converting the average histogram into a principal component analysis space and converting the mean histogram into a characteristic point in the principal component analysis space, wherein the characteristic point in the principal component analysis space is generated as the human body pose model. How to create a 3D human body pose model.

Images