KR20090069704A - Method and apparatus for tracking human body in 3d space - Google Patents

Abstract

A method and an apparatus for tracking a human body in a 3D space are provided to set human body in various poses accurately when tracing the motion of a human through a 3 dimensional human body model. A person detecting unit(110) detects a human by using view angle difference information obtained from a stereo image, and a model initializing unit(120) is initiated according to the formats of preset initial posture and silhouette. A motion tracing unit(130) traces the motion of the person based on the initiated 3 dimensional human body model.

Description

Method and apparatus for tracking human motion in three-dimensional space {Method and apparatus for tracking human body in 3D space}

The present invention relates to a method and apparatus for tracking human motion in three-dimensional space, and more particularly, to a method and apparatus for tracking human motion using a three-dimensional human body model from an input image.

Recently, many researches on tracking human movement based on images have been conducted. Analysis of human movement is required in a variety of applications, including image-based surveillance, motion capture, human behavior analysis, human interface such as Human Computer Interface (HCI) and Human Robot Interface (HRI). Because.

As a technique for tracking the movement of a person, there has been conventionally a method of tracking the movement of a person by attaching a marker capable of detecting the movement to a human body and tracking the marker itself. However, there is a problem in that it is necessary to track the movement of a person only in a special situation in which the marker and the marker can be attached directly to the human body.

Therefore, the technique of tracking the movement of a person using a three-dimensional human body model approximated to the shape of the person from the input image has been in the spotlight. To do this, it is first necessary to initialize the 3D human body model to the person present in the input image. It is not easy to initialize the 3D human body model for an arbitrary pose. Therefore, in the prior art, a three-dimensional human body model was initialized by taking a specific posture at a specific position manually.

The present invention is to provide a human motion tracking method and apparatus capable of accurately initializing a three-dimensional human body model for various postures when a human motion is to be tracked using a three-dimensional human body model from an input image. There is.

Furthermore, the technical problems to be achieved by the present invention are not limited to those mentioned above, and other technical problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above technical problem, the human motion tracking method in the three-dimensional space according to the present invention comprises the steps of: (a) detecting a person using visual difference information obtained from a stereo image; (b) determining any one of a plurality of predetermined postures according to a feature of the human silhouette as an initial posture, and initializing a three-dimensional human body model in the person according to the determined initial posture and the shape of the silhouette; And (c) tracking the movement of the person based on the initialized three-dimensional human body model.

Here, the step (a) may include detecting a human candidate using the time difference information; And verifying that the person is a person by detecting a face with respect to the detected person candidate.

The silhouette may be extracted using a time difference image or a difference image between an image corresponding to the time difference image and a background image obtained in advance.

The step (b) may include extracting a silhouette image of the person; Calculating a hu moment as a feature of the silhouette from the extracted silhouette image; And determining the initial posture according to the moment after the calculation.

Furthermore, the step (b) may include generating horizontal and vertical histograms from the extracted silhouette image; And initializing the 3D human body model according to a result of analyzing the determined initial posture and the horizontal and vertical histograms.

In addition, step (c), (c1) using a particle filter to roughly track the movement of the person; And (c2) tracking the correct movement using an iterative closest point (ICP) algorithm based on the approximately tracked movement of the person.

Also, in the step (c1), the reference position for generating the particle sample may be determined using the past motion information of the person.

In addition, when the human motion tracking fails in step (c2), in step (c1), a particle sample may be generated by enlarging the distribution area of the sample.

In addition, the human motion tracking method may further include applying a constraint such that the three-dimensional human body model exists within a range in which the human body can move in the ergonomic structure.

In accordance with another aspect of the present invention, there is provided a human motion tracking apparatus in a three-dimensional space, including: a person detection unit detecting a person using visual difference information obtained from a stereo image; A model initialization unit which determines any one of a plurality of predetermined postures according to a feature of the silhouette of the person as an initial posture, and initializes a three-dimensional human body model in the person according to the determined initial posture and the shape of the silhouette; And a motion tracking unit for tracking the movement of the person based on the initialized three-dimensional human body model.

Here, the person detector may include: a candidate detector for detecting a person candidate using the time difference information; And a face detector that detects a face by detecting a face with respect to the detected person candidate.

The silhouette may be extracted using a time difference image or a difference image between an image corresponding to the time difference image and a background image obtained in advance.

The model initialization unit may include a silhouette extractor configured to extract a silhouette image of the person; A calculator configured to calculate a hu moment as a feature of the silhouette from the extracted silhouette image; And a determiner configured to determine the initial posture according to the calculated moment.

Further, the model initialization unit, histogram generator for generating a horizontal and vertical histogram from the extracted silhouette image; And an initialization unit for initializing the three-dimensional human body model according to a result of analyzing the determined initial posture and the horizontal and vertical histograms.

The motion tracking unit may further include: a first tracking unit which roughly tracks the movement of the person using a particle filter; And a second tracking unit that tracks an accurate movement using an iterative closest point (ICP) algorithm based on the approximately tracked movement of the person.

The first tracker may determine a reference position for generating a particle sample using the past motion information of the person.

In addition, when the human motion tracking fails in the second tracking unit, the first tracking unit may enlarge the distribution area of the sample to generate particle samples.

In addition, the human motion tracking device may further include a constraint providing unit for applying a constraint so that the three-dimensional human body model exists in a range in which a human can move in an ergonomic structure.

In order to solve the above another technical problem, there is provided a computer-readable recording medium having recorded thereon a program for executing the above-described human motion tracking method.

According to the present invention, when a human motion is to be tracked using a 3D human body model from an input image, the 3D human body model can be initialized accurately for various postures.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, the substantially identical components are represented by the same reference numerals, and thus redundant description will be omitted. In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted.

1 is a block diagram of a human motion tracking apparatus in a three-dimensional space according to an embodiment of the present invention. Referring to FIG. 1, the human motion tracking apparatus 100 according to the present exemplary embodiment includes a person detector 110, a model initializer 120, a motion tracker 130, and a constraint conditioner 140. .

The person detector 110 receives a stereo image obtained from a stereo camera (not shown). The stereo image refers to two two-dimensional images captured at different photographing positions with respect to a subject, that is, two-dimensional images in a pair relationship. The person detector 110 obtains disparity information from the stereo image and detects a person using the disparity information. Specifically, the person detector 110 includes a candidate detector 112 and a face detector 114 as shown in FIG. 1.

The candidate detector 112 obtains time difference information, for example, a time difference map from a stereo image, and detects a human candidate using the time difference map. More specifically, the candidate detector 112 generates a two-dimensional space-time difference histogram by projecting the visual difference map in the height direction (Y-axis direction), which is the height direction of a person. Human candidates are detected by convolutional operation of a two-dimensional spatial-disparity histogram and an object-oriented scale-adaptive filter (OOSAF). When the two-dimensional space-time difference histogram and the OOSAF are multiplied, the human candidate appears as a blob with a soft outline on the two-dimensional space-time difference histogram.

The face detector 114 verifies that the person is a person by detecting a face with respect to the person candidate detected by the candidate detector 112. Since the person candidate obtained by the candidate detector 112 may or may not be a real person, face detection is used to know that it is a real person. The face detector 114 may use an AdaBoost algorithm, which is one of appearance-based pattern recognition methods, for face detection.

The model initializer 120 determines one of a plurality of predetermined postures as an initial posture according to a feature of the silhouette of the person detected by the person detector 110, and determines the initial posture and the shape of the silhouette of the person. Initialize the 3D human model. Specifically, the model initializer 120 includes a silhouette extractor 122, a calculator 124, a determiner 126, a histogram generator 128, and an initializer 129 as illustrated in FIG. 1. .

The silhouette extractor 122 extracts a silhouette image of a person. Here, the silhouette extractor 122 extracts a silhouette image by performing an AND operation on a time difference image obtained from a stereo image, an image corresponding to the time difference image, and a background image obtained in advance. The image corresponding to the visual difference image and the background image obtained in advance are obtained from one of two cameras constituting the stereo camera.

FIG. 2 is a scene provided for initialization of a three-dimensional human body model, with a human arm open. FIG. 3 shows a time difference image (left) obtained from the scene shown in FIG. 2 and a silhouette image (right) extracted by performing an AND operation on the time difference image and the difference image described above. Here, the silhouette extractor 122 may use the visual difference image or the difference image itself as a silhouette image. However, by performing the AND operation on the visual difference image and the difference image, an accurate silhouette image may be extracted even when the illumination state and the background are poorly distinguished.

The calculator 124 calculates a parameter that can reflect the feature of the silhouette from the silhouette image extracted by the silhouette extractor 122. Here, as a parameter capable of reflecting the characteristics of the silhouette, a hu moment may be used. The post moment is a 7-dimensional vector that is a useful parameter for shape analysis of an image regardless of its position, size, or orientation. The post moment is described in [M. Hu, "Visual Pattern Recognition by Moment Invariants," IRE Trans. Information Theory, vol. 8, no. 2, pp. 179-187, 1962, detailed description thereof will be omitted.

The determiner 126 determines one of a plurality of predetermined postures as an initial posture according to a parameter calculated by the calculator 124, for example, a post moment. The plurality of postures may consist of representative actions that a person may take. For example, it may be defined as four postures: lowering arms, open arms, lower left arm, and lower right arm. The larger the number of predetermined postures, the higher the accuracy of initialization in initializing the 3D human body model. If the scene shown in Fig. 2 is provided for the initialization of the three-dimensional human body model, the initial pose will be determined as the open arms of the four poses described above.

The determiner 126 may use a neural network to determine an initial posture according to a parameter reflecting the characteristic of the silhouette. Neural networks are electronic networks made by mimicking human brain neurons and their associated structures. They are highly capable of type identification and nonlinear mapping. The determination unit 126 may use an SVM classifier (Support Vector Machine) or the like as a classification means capable of mapping inputs and outputs in addition to neural networks.

The histogram generator 128 generates horizontal and vertical histograms from the silhouette image extracted by the silhouette extractor 122. 4A and 4B show horizontal and vertical histograms generated by the histogram generator 128, respectively. The horizontal histogram illustrated in FIG. 4A represents the sum of pixel values of vertical lines according to pixel coordinates in the horizontal direction with respect to the silhouette image illustrated on the right side of FIG. 3. Referring to the right image of FIG. 3 and FIG. 4A, since the bright pixels occupy a large portion of the vertical line at the center point in the horizontal direction of the silhouette image, the sum of the pixel values is high, whereas at both end points of the human silhouette It can be seen that the sum of the pixel values is low because light pixels occupy a small portion in the vertical line. The vertical histogram illustrated in FIG. 4B represents a sum of pixel values of horizontal lines according to vertical coordinates with respect to the silhouette image illustrated on the right side of FIG. 3. Referring to the right image of FIG. 3 and FIG. 4B, in the vertical coordinates where the horizontal line penetrates the arms, the sum of the pixel values is high in the vertical coordinates where the horizontal line penetrates the face. You can see that the sum of the values is low.

The initializing unit 129 detects the person detected by the person detecting unit 110 according to the result of analyzing the initial posture determined by the determining unit 126 and the horizontal and vertical histograms of the silhouette image generated by the histogram generating unit 128. Initialize the dimensional human body model. In the present exemplary embodiment, a cylinder model obtained by approximating a human body structure may be used as the 3D human body model. 5 is an example of such a cylinder model, in which the human body is modeled with nine cylinders and one sphere as shown. Here, the cylinder model forms a tree structure based on the torso (Torso) as shown in FIG. The movement of each sphere or cylinder in the cylinder model is represented by a rotation vector that determines the direction of the axis of rotation and a position vector that determines the position of the axis of rotation. In the case of using such a cylinder model, when the initial posture is determined, the directions of the cylinders are known, and the horizontal and vertical histograms are analyzed based on the directions of the cylinders to determine the position and size of each cylinder. . For example, the coordinates of a point corresponding to the four corners of the torso cylinder can be seen in the image.

In addition, the initialization unit 129 may further use human anatomical proportional information in initializing the 3D human body model. For example, when the width or height of the torso is found, the relative length of the leg or the length of the leg can be predicted, and the upper and lower arms can be estimated only by the initial posture and histogram analysis. If the lower arm is not well distinguished, the upper and lower arms can be distinguished using ergonomic proportional information.

7 illustrates a result of initializing a 3D human body model for the scene shown in FIG. 2 according to the present embodiment. Referring to FIG. 7, it can be seen that nine cylinders and one sphere overlap with a person present in the image.

Referring back to FIG. 1, the motion tracker 130 tracks the motion of a person based on the three-dimensional human body model initialized by the model initializer 120. Here, the tracking of human motion is for fitting a 3D human body model to a person present in an image obtained from a stereo camera. In detail, the motion tracker 130 includes a first tracker 132 and a second tracker 134 as illustrated in FIG. 1.

The first tracking unit 132 roughly tracks the movement of the person using a particle filter. Tracking a person's movement using a particle filter is largely divided into two steps. In the first step, the state transition step, a particle sample, that is, a sample of a cylinder model, is generated by using the motion information of the person to be tracked. In the second step, the observation measurement step, human motion is estimated using the measurements of each particle sample. Therefore, in order to track human movement using particle filter, we need to define two models. The first model is a state transition model that models the movement information of the person to be tracked. The second model is the observation model, which finds the relationship between the sample of the cylinder model and the observation data, that is, the image received from the stereo camera, in the current motion state.

In this embodiment, the state transition model uses a person's past motion information, for example, the velocity of the cylinder model in the previous frame, to determine a reference position for generating particle samples in the current frame. A reference position for generating particle samples using the velocity of the cylinder model in the previous frame can be obtained according to the following equation.

,

는 각각 t 프레임 및 t-1 프레임에서 예측된 속도를,

및

는 각각 t-1 프레임 및 t-2 프레임에서의 모션 파라미터를,

는 파티클 샘플을 생성하기 위한 기준 위치를 나타내는 모션 파라미터를 의미한다. 모션 파라미터란 객체(실린더)의 위치를 나타내기 위한 파라미터로서, 이동(translation)과 회전(rotation) 정보를 가지는 파라미터이다. 그리고

는 0과 1 사이의 임의의 값이다. 이처럼 사람의 과거 움직임 정보를 이용하여 각 프레임마다 파티클 샘플을 생성하기 위한 기준 위치를 변경하고, 일정 영역에서 파티클 샘플을 생성한다.here,

,

Is the predicted velocity in t frames and t-1 frames,

And

Are the motion parameters in t-1 and t-2 frames, respectively.

Denotes a motion parameter indicating a reference position for generating a particle sample. The motion parameter is a parameter for indicating the position of an object (cylinder) and is a parameter having translation and rotation information. And

Is any value between 0 and 1. As described above, the reference position for generating the particle sample is changed in each frame by using the past motion information of the person, and the particle sample is generated in a certain area.

The observation model uses a maximum likelihood function between the particle sample and the observation data. The maximum likelihood function is defined as the average of the sum of the inner product of the normal vector of each point of the cylinder model and the normal vector of each point of the person shown in the image, which is observation data. The maximum likelihood function can be defined according to the following equation.

및

는 각각 포인트 i에 대한 실린더 모델의 각 포인트의 법선 벡터 및 사람의 각 포인트의 법선 벡터를,

는 벡터의 내적 연산을 의미한다. 제1 추적부(132)는

를 기준으로 샘플링된 파티클 중 최대 공산 함수가 최소가 되는 모션 파라미터

를 찾아서 이를 사람 움직임으로 추정한다. Where N is the number of points

And

Is the normal vector of each point of the cylinder model for each point i and the normal vector of each point of the person,

Means the dot product of a vector. The first tracking unit 132

Motion parameter that maximizes the maximum likelihood function among the samples sampled from

Find and estimate this as human motion.

The second tracking unit 134 tracks the exact movement of the person using an iterative closest point (ICP) algorithm based on the movement of the person approximately tracked by the first tracking unit 132. The above-described particle filtering by the first tracking unit 132 and the ICP algorithm by the second tracking unit 134 are independently applied to each cylinder constituting the 3D human body model. In addition, in the case of using the cylinder model as shown in FIG. 5, each of the ten cylinder models is fitted to a person, and the ICP algorithm is applied in a certain order, for example, the body, the head, the upper arm, the upper arm, and the lower leg. Can be fitted.

In this embodiment, the process of tracking the movement of a person with respect to any cylinder consists of the following steps.

를 기준으로, 스테레오 영상에서 얻어진 사람 위의 점 IP_i와, 실린더 모델 위의 점 MP_i 사이의 거리 d(IP_i, MP_i)가 최소화되도록 하는 대응점의 집합 Y=C(IP_i, MP_i)를 구한다. Step 1. Motion parameters obtained by the first tracking unit 132 described above

A set of corresponding points Y = C (IP _i , MP _i ) to minimize the distance d (IP _i , MP _i ) between the point IP _i on the person obtained from the stereo image and the point MP _i on the cylinder model. )

Step 2. The amount of change of motion ΔP is obtained through a registration process between Y = C (IPi, MPi). Regarding the matching process between correspondence points, see P. J. Besl and N. MacKay, "A method for registration of 3-d shapes," IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, 1992, detailed description thereof will be omitted.

Step 3. The motion transformation amount ΔP obtained in Step 2 is applied to the point MP _i on the cylinder model as in the following equation.

는 상기 스텝 3에서 움직임 변환 적용된 모션 파라미터가 된다. The steps 1 to 3 described above are iterated until there is no change in the fitting error as a result of applying the motion change amount ΔP. Of course, if the step is repeated, the motion parameter as a reference in step 1

Becomes a motion parameter to which the motion transformation is applied in step 3.

가 구해지므로, 사람 움직임 추적이 실패하더라도 제2 추적부(134)가 다시 사람의 움직임을 정확히 추적할 수 있게 된다. When the human movement is fast, the tracking may fail while the above-described second tracking unit 134 tracks the human movement. If the tracking fails, the difference between the cylinder model and the person to be tracked is greatly increased. For example, if the fitting error becomes very large and does not fall below a certain value, the tracking fails. In this embodiment, in the case of generating the particle sample in the first tracking unit 132, the particle distribution area is enlarged to generate the particle sample. In this way, the motion parameters close to the person to be tracked in the first tracking unit 132 can be obtained.

Since is obtained, even if the human motion tracking fails, the second tracking unit 134 can accurately track the human motion again.

The constraint granter 140 applies constraints such that the 3D human body model fitted by the motion tracker 130 exists within a range in which a person can move in terms of an ergonomic structure. This is to prevent the three-dimensional human body model from moving incorrectly or ergonomically in a range that cannot be moved so that more accurate movement can be tracked. Constraints include articulation constraints, articulation range constraints, penetration constraints, torso height constraints, and the like.

Joint constraints mean that the human body's torso must be connected to the head, upper arm, and upper leg, and the lower arm and lower leg should be connected to the upper arm and upper leg, respectively. Joint range constraints are conditions in which each joint of a person has a limit of the range within which it can move. For example, the leg cannot change more than 180 degrees in the Z-axis direction. Therefore, set the limit angle at which joints can move on each axis to allow movement only within that range. Intrusion constraints are conditions in which each limb cannot invade each other in three-dimensional space. Therefore, the angle and position are adjusted to prevent intrusion between cylinders during human movement tracking. Torso height constraints are intended to remove ambiguity in the torso area of the human body. Since the three-dimensional human body model is represented by a hierarchical structure centered on the torso, as described above, the torso is the highest level and thus provides the most stable tracking. shall. Therefore, the three-dimensional body model at the time of initialization of the three-dimensional space of the height constraints to ensure a stable tracking.

FIG. 8 illustrates an experiment result in which a human motion is tracked based on a three-dimensional human body model initialized as shown in FIG. 7 according to the above-described embodiment. In FIG. 8, only four frames are shown among the various frames for convenience. In each frame, the image on the left is an image obtained through a stereo camera, and the image on the right represents a result of tracking a human motion. The result of this experiment is the image of the first frame (corresponding to FIG. 2) is used for human detection and initialization of the 3D human model, and the movement of the human is tracked based on this.

9 is a flowchart illustrating a human motion tracking method in three-dimensional space according to an embodiment of the present invention. The human motion tracking method according to the present embodiment consists of steps processed in the human motion tracking apparatus described with reference to FIG. 1. Therefore, although omitted below, the content described with reference to FIG. 1 is also applied to the human motion tracking method according to the present embodiment.

In operation 910, the human motion tracking apparatus receives a stereo image and detects a person using visual difference information obtained from the stereo image.

In operation 920, the apparatus for tracking a person's motion determines one of a plurality of predetermined postures as an initial posture according to the feature of the silhouette of the person detected in step 910, and according to the determined initial posture and the feature of the silhouette, step 910. The 3D human body model is initialized to the detected person.

In operation 930, the apparatus for tracking human motion tracks the movement of a person based on the 3D human body model initialized in operation 920.

In operation 940, the apparatus for tracking a human motion applies a constraint so that the 3D human model based on the result of tracking the movement of the human person is present in a range in which the human body can move in step 930.

10 is a flowchart illustrating step 910 of FIG. 9 in more detail according to an embodiment of the present invention. Operation 910 according to the present exemplary embodiment includes steps processed by the person detector 110 illustrated in FIG. 1. Therefore, even if omitted below, the content described with respect to the person detector 110 of FIG. 1 is also applied to step 910 according to the present embodiment.

In operation 911, the human motion tracking apparatus detects a human candidate using visual difference information obtained from a stereo image.

In step 912, the human motion tracking apparatus verifies that the person is a person by detecting a face with respect to the person candidate detected in step 911.

FIG. 11 is a flowchart illustrating step 920 of FIG. 9 in more detail according to an embodiment of the present invention. Operation 920 according to the present exemplary embodiment includes steps processed by the model initialization unit 120 illustrated in FIG. 1. Therefore, even if omitted below, the contents described with respect to the model initializer 120 of FIG. 2 are also applied to step 920 according to the present embodiment.

In operation 921, the human motion tracking apparatus extracts a silhouette image of the person detected in operation 910.

In operation 922, the human motion tracking apparatus calculates a hu moment as a feature of the silhouette from the silhouette image of the person extracted in operation 921.

In operation 923, the human motion tracking apparatus determines any one of a plurality of predetermined postures as the initial posture according to the moment after calculating in step 922.

In operation 924, the human motion tracking apparatus generates horizontal and vertical histograms from the silhouette image extracted in operation 921.

In step 925, the human motion tracking apparatus initializes the 3D human body model to the person detected in step 910 based on the analysis of the initial posture determined in step 923 and the horizontal and vertical histograms generated in step 924.

12 is a flowchart illustrating step 930 of FIG. 9 in more detail according to an embodiment of the present invention. Operation 930 according to the present exemplary embodiment includes operations processed by the motion tracker 130 illustrated in FIG. 1. Therefore, even if omitted below, the content described with respect to the motion tracker 130 of FIG. 2 is also applied to step 930 according to the present embodiment.

In operation 931, the human motion tracking apparatus tracks a human motion by using a particle filter. Here, in applying the particle filter, a reference position for generating the particle sample may be determined by using human past motion information.

In operation 932, the human motion tracking apparatus tracks an accurate movement using an ICP algorithm based on the movement of the person approximately tracked in operation 931. Although not shown in FIG. 12, the steps 931 and 932 are repeatedly performed. If the human motion tracking fails in step 932, the particle distribution area of the sample is enlarged by applying the particle filter in step 931. Samples can be generated.

According to the present invention described above, any one of a plurality of predetermined postures is determined as the initial posture according to the characteristics of the silhouette of the person detected from the stereoscopic image, and three-dimensional to the person according to the initial posture and the shape of the silhouette of the person. By initializing the human body model, it is possible to accurately initialize the three-dimensional human body model for various postures. In addition, by using a particle filter to track the movement of the person roughly, based on this based on the ICP algorithm it is possible to track in real time even in fast movements.

Meanwhile, the above-described embodiments of the present invention can be written as a program that can be executed in a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable recording medium. The computer-readable recording medium may be a magnetic storage medium (for example, a ROM, a floppy disk, a hard disk, etc.), an optical reading medium (for example, a CD-ROM, DVD, etc.) and a carrier wave (for example, the Internet). Storage medium).

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. Therefore, the disclosed embodiments should be considered in descriptive sense only and not for purposes of limitation. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

1 is a block diagram of a human motion tracking apparatus in a three-dimensional space according to an embodiment of the present invention.

2 is a scene provided for initialization of a three-dimensional human body model.

FIG. 3 illustrates a silhouette image extracted by performing an AND operation on a time difference image and a time difference image obtained from the scene shown in FIG. 2 and the difference image described above.

4A and 4B show horizontal and vertical histograms generated by the histogram generator 128, respectively.

5 is an example of a cylinder model approximating a human body structure.

The tree structure of a cylinder model is shown in FIG.

7 illustrates a result of initializing a 3D human body model for the scene shown in FIG. 2 according to an embodiment of the present invention.

FIG. 8 illustrates an experiment result in which a human motion is tracked based on a three-dimensional human body model initialized as shown in FIG. 7 according to an embodiment of the present invention.

9 is a flowchart illustrating a human motion tracking method in three-dimensional space according to an embodiment of the present invention.

10 is a flowchart illustrating step 910 of FIG. 9 in more detail according to an embodiment of the present invention.

FIG. 11 is a flowchart illustrating step 920 of FIG. 9 in more detail according to an embodiment of the present invention.

12 is a flowchart illustrating step 930 of FIG. 9 in more detail according to an embodiment of the present invention.

Claims (19)

In the human motion tracking method in three-dimensional space, (a) detecting a person using visual difference information obtained from a stereo image; (b) determining any one of a plurality of predetermined postures according to a feature of the human silhouette as an initial posture, and initializing a three-dimensional human body model in the person according to the determined initial posture and the shape of the silhouette; And (c) tracking the movement of the person based on the initialized three-dimensional human body model. According to claim 1, wherein the step (a), Detecting a person candidate using the time difference information; And Verifying that the person is a person by detecting a face with respect to the detected person candidate. The method of claim 1, The silhouette is a human motion tracking method, characterized in that the extraction using the difference image between the image corresponding to the time difference image, or the background image obtained in advance. According to claim 1, wherein step (b), Extracting a silhouette image of the person; Calculating a hu moment as a feature of the silhouette from the extracted silhouette image; And And determining the initial posture according to the moment after the calculation. The method of claim 4, wherein step (b) comprises: Generating horizontal and vertical histograms from the extracted silhouette image; And And initializing the three-dimensional human body model according to a result of analyzing the determined initial posture and the horizontal and vertical histograms. The method of claim 1, wherein step (c) comprises: (c1) roughly tracking the movement of the person using a particle filter; And (c2) tracking the exact movement using an iterative closest point (ICP) algorithm based on the approximately tracked movement of the person. The method of claim 6, In the step (c1), a reference position for generating a particle sample using the past motion information of the person is determined. The method of claim 6, If the human motion tracking fails in the step (c2), in the step (c1), a particle sample is generated by enlarging the distribution area of the sample. The method of claim 1, And applying constraints such that the three-dimensional human body model exists within a range in which the human body can move in the ergonomic structure. A computer-readable recording medium having recorded thereon a program for executing the human motion tracking method according to any one of claims 1 to 9. In the human motion tracking device in three-dimensional space, A person detector for detecting a person using visual difference information obtained from a stereo image; A model initialization unit which determines any one of a plurality of predetermined postures according to a feature of the silhouette of the person as an initial posture, and initializes a three-dimensional human body model in the person according to the determined initial posture and the shape of the silhouette; And And a motion tracker for tracking the motion of the person based on the initialized three-dimensional human body model. The method of claim 11, wherein the person detection unit, A candidate detector for detecting a human candidate using the time difference information; And And a face detector configured to verify that the person is a person by detecting a face with respect to the detected person candidate. The method of claim 11, And the silhouette is extracted using a difference image between a visual difference image or an image corresponding to the visual difference image and a background image obtained in advance. The method of claim 11, wherein the model initialization unit, A silhouette extractor extracting a silhouette image of the person; A calculator configured to calculate a hu moment as a feature of the silhouette from the extracted silhouette image; And And a determination unit for determining the initial posture according to the moment after the calculation. The method of claim 14, wherein the model initialization unit, A histogram generator for generating horizontal and vertical histograms from the extracted silhouette image; And And an initialization unit configured to initialize the three-dimensional human body model according to a result of analyzing the determined initial posture and the horizontal and vertical histograms. The method of claim 11, wherein the motion tracking unit, A first tracking unit which roughly tracks the movement of the person using a particle filter; And And a second tracking unit for tracking an accurate movement using an iterative closest point (ICP) algorithm based on the approximately tracked human movement. The method of claim 16, And the first tracker determines a reference position for generating a particle sample using the past motion information of the person. The method of claim 16, When the human motion tracking fails in the second tracking unit, the first tracking unit enlarges the distribution area of the sample to generate a particle sample. The method of claim 1, The human motion tracking device further comprises a constraint providing unit for applying a constraint so that the three-dimensional human body model exists within the range in which the human body can move.

Images