KR20190042197A - Regression-Based Landmark Detection Method on Dynamic Human Models and Apparatus Therefor - Google Patents

Abstract

Disclosed are a regression analysis-based landmark detection method for a dynamic human model and an apparatus thereof. According to an embodiment of the present invention, the regression analysis-based landmark detection method comprises the steps of: generating a pose invariant coordinate which defines a pose invariant descriptor independent of a pose and a position of a landmark independent of a dynamic pose through training of human models for predetermined poses including the landmarks; obtaining regression coordinates of a human-shaped landmark input through a regression analysis using the pose invariant descriptor and the pose invariant coordinates; and detecting the input human-shaped landmark using the obtained regression coordinates.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a regression-based landmark detection method for dynamic human models,

The present invention relates to a landmark detection technique, and more particularly, to a landmark detection method and apparatus capable of detecting a landmark based on a regression analysis on a dynamic human model.

Analyzing and understanding human anatomy is an important issue in computer graphics research, along with many applications such as registration, goal redirection, and shape retrieval. Anatomical landmarks of the human body are essential functions for obtaining human anthropometric information, but detection of anatomical landmarks remains a difficult problem because of the variety of human shapes and poses.

A method of detecting a landmark according to the related art is a method of registering a mesh, which searches for correspondence between a template body model having an annotated landmark and a specific human body shape. These registration-based methods are effective for template-like human shapes, but have limited functionality for generalizing to the full range of human shapes with different poses.

Conventionally, a landmark detection method according to another embodiment learns and predicts a relationship between various human shapes and landmark positions. Here, a good way to detect landmarks should not require complex pre-processing such as sorting on input data.

Because of its importance in shape analysis, researchers have developed a variety of ways to describe local (or partial) and global (or global) features of object shapes. Examples of local descriptors that represent the geometry of small neighbors or points include spin images, hit kernel markers, and wave kernel markers. The global descriptor represents the overall feature of the shape. Among them, global descriptors generated by combining local descriptors have generally been used for shape retrieval problems. Representative global descriptors include ShapeDNA, histograms of area projection transformations, and aggregation of local statistical functions. The most common way to create a global descriptor is to use the bag of feature (BoF) method, which is a global feature of the feature in terms of the frequency of occurrence of the local feature. This method is similar to the bag-of-words method Do.

The problem of predicting anthropometric landmarks has received increasing attention. Giachetti et al. Have reported cutting-edge technologies for landmark prediction. Among the reported techniques, a surface-to-surface registration method (CR03; CHUI H., RANGARAJAN A .: A new point matching algorithm for non- (AGM06, AZOUZ ZB, SHU C., MANTEL A: Automatic locating of anthropometric landmarks on 3d human models In 3D Data Processing, Visualization, and Transmission, Third International Symposium on (2006), IEEE, pp. 750-757, 2, 5) performed better than other comparative techniques.

The surface-to-surface registration method finds landmark points through non-rigid spatial mapping from template meshes to input meshes. ICP methods are used for registration, but generally this pure geometric mesh registration method does not guarantee human similarity of the registered mesh and also requires an iterative alignment step. The graphical model method shows the structure of a landmark using the Markov network. This method shows high accuracy when training with a database of uniform pose (A-pose) with relatively high computation time. Both methods use averaging data, for example, to find the correspondence between template meshes and input data.

Therefore, when the input data is different from the average data, the accuracy decreases. Moreover, these methods are not suitable for various poses due to the limitations of non-rigid ICP methods or the use of pose variation local features.

The method based on spectrum analysis is independent of the pose change, but its accuracy is not yet competitive.

Kernel Canonical Correlation Analysis (KCCA) is a method for finding the relationship between two multidimensional variables. In computer graphics, KCCA is used for face recognition and face re-targeting. For example, one conventional method used a KCCA to simultaneously control deformation of many local areas with a small number of control points.

The present invention provides a new method of determining the relationship between a person's body shape and anatomical landmark location through a statistical regression model.

Embodiments of the present invention provide a landmark detection method and apparatus that can detect a landmark based on a regression analysis on a dynamic human model.

A regression-based landmark detection method according to an exemplary embodiment of the present invention is a method of detecting a pose invariant descriptor independent of a pose and a dynamic pose independent of a pose through training of human models on predetermined pose including landmarks Creating a pose invariant coordinate where a position of an in-landmark is defined; Obtaining regression coordinates of a human land shaped landmark input through a regression analysis using the pose invariant descriptor and the pose invariant coordinates; And detecting the input human-shaped landmark using the obtained regression coordinates.

Wherein the creating step defines a local descriptor for a local area of each of the landmarks, defines a global shape descriptor characterizing a shape of the entire body using the defined local descriptor, The pause invariant descriptor representing a portion of the human shape associated with a particular landmark can be generated.

The generating may collect only the elements related to each of the landmarks among the geometric elements provided by the bag of features (BoF) method, and may generate the pose invariant descriptor using only the collected elements.

The generating step may generate the pose invariant coordinate using a feature vector indicating a position of the landmark independent of the dynamic pose.

The creating step may represent the spatial relationship between the independent landmarks using the feature vector and the geodesic distance, and may generate the pose invariant coordinates using the spatial relationship.

Wherein the acquiring includes learning a regression analysis model between the pose invariant descriptor, the pose invariant coordinate, and a global shape descriptor characterizing the shape of the entire body, and using the learned regression analysis model, It is possible to obtain regression coordinates for the landmark.

The acquiring step may learn the regression analysis model using a kernel matching correlation analysis (KCCA) technique.

Wherein the obtaining comprises performing a kernel matching correlation analysis (KCCA) between the pose invariant descriptor and the location of the independent landmark, and performing a kernel matching correlation analysis between the spatial relationship between the independent landmarks and the global shape descriptor , And regression coordinates of the inputted human body shape landmark can be obtained through regression analysis on the kernel correlation correlation analysis result.

In accordance with another embodiment of the present invention, a regression-based landmark detection method includes: training a human model for preset pose including landmarks, thereby generating a pose constant coordinate defined by a position of a landmark independent of a dynamic pose ; Acquiring regression coordinates of a human land shaped landmark input through regression analysis using the generated pose invariant coordinates; And detecting the input human-shaped landmark using the obtained regression coordinates.

The regression-based landmark detection apparatus according to an exemplary embodiment of the present invention includes a pose-invariant pseudo-invariant descriptor independent of a pose invariant descriptor and a dynamic pose-independent pose through learning of human models for predetermined pose including landmarks A preprocessing unit for generating a pose constant coordinate in which a position of an in-landmark is defined; An analyzer for obtaining regression coordinates of a human land shaped landmark inputted through a regression analysis using the pose invariant descriptor and the pose invariant coordinates; And a detector for detecting the input landmark of the human body using the obtained regression coordinates.

Wherein the preprocessor defines a local descriptor for a local region of each of the landmarks, defines a global shape descriptor characterizing the shape of the entire body using the defined local descriptor, and uses the defined global shape descriptor To generate the pose invariant descriptor representing a portion of the human shape associated with a particular landmark.

The preprocessor may collect only the elements related to each of the landmarks among the geometric elements provided by the bag of features (BoF) method, and generate the pose invariant descriptor using only the collected elements.

The preprocessor may generate the pose invariant coordinate using a feature vector indicating the position of the landmark independent of the dynamic pose.

The preprocessor may represent the spatial relationship between the independent landmarks using the feature vector and the geodesic distance, and may generate the pose invariant coordinate using the spatial relationship.

Wherein the analysis unit learns a regression analysis model between the pose invariant descriptor, the pose invariant coordinate, and the global shape descriptor characterizing the shape of the entire body, and uses the learned regression analysis model to calculate the input human shape landmark Can be obtained.

The analyzer may learn the regression analysis model using a kernel matching correlation analysis (KCCA) technique.

The analyzing unit performs a kernel matching correlation analysis (KCCA) between the pose invariant descriptor and the position of the independent landmark, and performs a kernel matching correlation analysis between the spatial relationship between the independent landmarks and the global shape descriptor , And regression coordinates of the inputted human-shaped landmark can be obtained through regression analysis on the kernel correlation correlation analysis result.

The regression-based landmark detection apparatus according to another embodiment of the present invention detects a position of a landmark independent of a dynamic pose through training of human models on predetermined pose including landmarks, Processing unit; An analyzer for obtaining regression coordinates of a human land shaped landmark inputted through regression analysis using the generated pose invariant coordinates; And a detector for detecting the input landmark of the human body using the obtained regression coordinates.

According to embodiments of the present invention, an anatomical landmark of a human model having a dynamic pose can be detected by training a statistical regression analysis model that learns the relationship between a human body shape and a landmark.

According to embodiments of the present invention, a regression-based method can achieve higher accuracy than a registration-based method when learning with a data set that has a wide range of human shapes and poses, Kernel Canonical Correlation Analysis (KCCA) method can successfully model the correlation between human shape and landmark.

According to the embodiments of the present invention, an accurate anatomical measurement can be performed within a tolerance range by learning a collected 3D human image and giving a person of a new body shape of various pose. It can be applied to the field. For example, anatomical analysis of human characters on computer animations or simulations is an essential element and can also be used as a medical analysis tool through 3D imagery.

The present invention can be applied to digital entertainment applications such as virtual clothing fitting and avatar generation, and software in image medical field, which automatically detects anatomical feature points of an arbitrary human body.

Joint position estimation is essential for natural movement of human characters on digital media, and the anatomical feature point estimation technique of the present invention can find the joints of characters accurately within the error range.

In addition, the process of registering characters among graphics is an important process that can grasp the geometrical structure between characters. The present invention can greatly reduce the time of the registration process by finding a correspondence point between characters.

In addition, the present invention can easily generate human character models of various body types by constructing a statistical model, and can also be used for character motion re-targeting.

In addition, since the present invention locates important feature points of a user simultaneously in an augmented reality (AR) / virtual reality (VR) space, it can help generate motions between users, and can be used for creating virtual avatars and fitting avatar garments Technology.

를 시각화한 일 예시도를 나타낸 것이다.
도 4는 160명의 인간 피실험자의 랜드마크 중 p의 처음 세 좌표를 보여주는 일 예시도를 나타낸 것이다.
도 5는 랜드마크 사이의 측지선 거리와 각 꼭지점

에 대한

의 처음 세 차원 값을 나타낸 것이다.
도 6은 본 발명에 따른 장치에 의해 검출된 랜드마크에 대한 일 예시도를 나타낸 것이다.
도 7은 본 발명의 일 실시예에 따른 랜드마크 검출 방법에 대한 동작 흐름도를 나타낸 것이다.Figure 1 shows an exemplary diagram of authorization models for dynamic pose in a data set comprising annotated landmarks.
2 shows a configuration of a landmark detection apparatus according to an embodiment of the present invention.
3 is the color for the landmark i for each vertex x _j

As shown in Fig.
Fig. 4 shows an example showing the first three coordinates of p among the landmarks of 160 human subjects.
FIG. 5 is a graph showing the relationship between the geodesic distances between landmarks and each vertex

For

The first three dimensions of
6 shows an example of a landmark detected by an apparatus according to the present invention.
7 is a flowchart illustrating an operation of the landmark detection method according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. In addition, the same reference numerals shown in the drawings denote the same members.

The anatomical measurement of a person on digital is very important. Important anatomical landmarks, for example, joint positions, etc., are used as an indispensable element in identifying and defining a person's body shape. However, these landmarks vary greatly depending on the person's body shape and the pose taken. Detection of anatomical landmarks is therefore a professionally demanding and still challenging task.

Embodiments of the present invention provide a landmark detection technique that detects a relationship between a human body shape and an anatomical landmark position through a statistical regression model.

Here, the present invention can automatically detect a landmark in an input human model given a human model having a new pose and body shape on a digital basis.

This invention includes two important concepts.

First, define the landmark coordinates that are not affected by the pose. To do this, we show the location of the landmark using feature vectors that are not affected by the pose, and show the spatial relationship between the landmarks using the feature vector and the geodesic distance.

Here, the present invention may define or generate pose invariant coordinates of landmarks representing local and global shape features using pose-invariant local shape descriptors and their spatial relationships. At this time, the present invention can provide the pose constant coordinate of the landmark by expressing the position of the landmark having the feature vector in the pseudo-constant local descriptor space.

In addition, since the mapping from vertices to the local shape descriptor space is non-implicit (e.g., the left and right Stylions have the same feature vector), this representation is expanded with spatial relationship information between the landmarks , This relationship expression can be described in a posture invariant manner.

Second, to increase accuracy, we create a two-step regression model for body parts and the whole body. The first-step regression model shows the linkage between the local features of the landmark and the shape of body parts. That is, the body level regression analyzer models the link between the local features of the landmark and the shape of the body part associated with the landmark. By using this regression model, body parts irrespective of the position of each landmark can be excluded. For example, one of the landmarks, the elbow joint, is affected by the shape of the arms but is not affected by other parts of the legs or head. In order to identify such landmark-specific body parts, a geometric shape dictionary obtained through a bag of feature (BoF) method, for example, a Supervised Bag of Features (S-BoF) method, can be used. The segmentation of the body part may block the effects of other unrelated body parts by training the low level regression analyzer with respect to the unique body part associated with the landmark.

The second-step regression model shows the connection between the whole body shape and the spatial relationship on the landmark. This can be used to prevent detection of a false landmark position due to symmetry of the human body. That is, a global-level regression analyzer overcomes the non-injectivity of the local features and almost automatically finds the landmark by associating the global shape feature with the spatial relationship between the landmarks.

The present invention automatically detects important anatomical landmarks for an input human model or a human body model, if only the left and right human bodies are selected through user input. In addition, using a test data set that includes a large change in human shape and poses without requiring any preconditioning process shows a prediction accuracy that is similar to or better than predictive accuracy.

The present invention is a framework of regression analysis for finding landmarks in a dynamic human model, and the framework of the present invention provides the following advantages.

First, regression analysis using KCCA not only can search landmarks for human shape that can be registered by template model, but also find landmarks for various human shapes.

Second, shape descriptors that combine global features and body part level shape features are invariant to isometric deformations, allowing for various pose changes. Pose invariance makes alignment such as registration, scale normalization, and origin adjustment unnecessary.

Finally, despite the short computation time, the accuracy of the landmark prediction is excellent and this accuracy is comparable to or better than other methods.

The present invention will now be described in detail.

data set

The present invention uses a data set consisting of 400 human models of 40 human subjects (20 males, 20 females) from a slim body to an obese body, and each model is composed of 10 dynamic postures. In the present invention, seven landmarks (Stylion, Radiale, Acromiale, Iliocristale, Trochanterion, Patella and Malleolus) can be annotated on the right side of the body as shown in Fig. 1 for landmark recognition. And pose data can be a great help in training landmark detectors that are generalized to a wide range of human subjects.

The present invention is based on the LBBC 14 method (LITMAN R., BRONSTEIN A., BRONSTEIN M., CASTELLANI U .: Supervised learning of bag-of-feature shape descriptors using sparse coding. In Computer Graphics Forum (2014), vol. 33, Wiley Online Library, pp. 127-136.

Here, the present invention may be more suited to changes in shape and pose because it uses a regression-based method of modeling the correlation between human shape and landmark position using pose-invariant local features.

The apparatus and method according to the present invention will be described in detail as follows.

The present invention detects a landmark of a human model having various shapes and poses, in which a link between a human shape and a landmark position is learned in a training data set.

FIG. 2 shows a configuration of a landmark detection apparatus according to an embodiment of the present invention. As shown in FIG. 2, the landmark detection apparatus includes a preprocessing unit 210, an analysis unit 220, and a detection unit 230.

The preprocessing unit 210 defines a divided body part descriptor expressing a part of a human shape related to a specific landmark through training or learning about a data set for a human model of various dynamic pose set in advance, And generates a pose constant coordinate of the landmark that can define the landmark position.

The analysis unit 220 finds a link between the coordinates of the body part descriptor and the landmark generated in the preprocessing process for the training human data set, and uses the trained predictor to determine the relationship between the landmark corresponding to the input human shape The coordinates are returned.

The detection unit 230 detects the position where the landmark is actually located in the human body shape inputted using the regression coordinates of the landmark.

Each configuration will be described in detail as follows.

1. Pre-

All landmarks of the training human model are labeled {1, ..., 7} in the order from the wrist (Stylion) to the ankle (Malleolus). The present invention does not require alignment of training models, and it is assumed that various pose models of the same subject are generated by isometric deformation. This assumption is necessary for geometric distance calculations when defining the relationship between landmarks. The position of the landmark depends only on the shape of the human body. Other elements such as poses should not affect landmark detection. To do this, we define pose invariant descriptors for human shapes and pose invariant coordinates for landmarks.

1.1 Pose Invariant Human Shape Descriptor

Since the geometric characteristics of the landmark are more strongly related to the local area than to other far areas, defining a descriptor suitable for the local area including the landmark is important for landmark detection. For example, the landmark of the wrist may be more dependent on the shape of the wrist than the elbow.

The present invention thus defines a separate body part descriptor that characterizes the geometric features of the body part. To this end, the preprocessor uses the global shape descriptor feature using the BoF method. Each atom in the geometric dictionary obtained from the BoF method represents a portion of a body part, so that it can define a local body part descriptor by collecting only the atoms associated with the body part to which the landmark belongs.

The first step is to define a BoF-based global shape descriptor, which will be described later, and then describe the procedure for defining a separate body region descriptor.

1.1.1 Local descriptor

를 갖는 매니폴드(manifold) M에 대한 열 방정식을 풀 수 있으며, 아래 <수학식 1>과 같이 나타낼 수 있다.A local descriptor represents the geometry of a point in a small neighborhood. The present invention uses Scale Invariant Heat kernel signature (SI-HKS), which is widely used to define the unique geometric features of an object shape, among many local descriptors. Invariance to isometric deformation makes the descriptor independent of human pose changes. SI-HKS is an improved version of Thermal Kernel Authentication (HKS), which uses the cotangent weights to generate m vertices

, And can be expressed as Equation (1) below. &Lt; EMI ID = 1.0 >

[Equation 1]

은 Laplace-Beltrami 연산자를 의미하고, u(x, t)는 시간 t에서 매니폴드 M에 대한 꼭지점 x에서의 열 분포를 의미할 수 있다.here,

Denotes the Laplace-Beltrami operator, and u (x, t) can denote the column distribution at the vertex x for the manifold M at time t.

(dirac-delta)가 주어지면, 상기 방정식에 대한 한 가지 솔루션이 열 커널로, 아래 <수학식 2>와 같이 나타낼 수 있다.As the initial condition, some heat distribution at time t = 0

(dirac-delta), one solution to the above equation can be expressed as a thermal kernel, as shown in Equation (2) below.

&Quot; (2) &quot;

와

는 라플라시안(Laplacian)의 고유항 분해(eigen-decomposition)에 의해 얻어진 고유 함수와 고유치를 의미할 수 있다.here,

Wow

Can mean eigenfunctions and eigenvalues obtained by the eigen-decomposition of Laplacian.

로 정의될 수 있다. 이 로컬 설명자를 푸리에 변환을 이용하여 주파수

로 대체함으로써, 열 커널 인증은 일정한 크기가 될 수 있으며, 꼭지점 x에 대한 로컬 설명자는 아래 <수학식 3>과 같이 나타낼 수 있다.The thermal kernels can provide local features for the geometry of the manifold. For example, the n-dimensional local descriptor for each vertex x is

. &Lt; / RTI > Using this Fourier transform,

, The thermal kernel authentication can be of a certain size, and the local descriptor for the vertex x can be expressed as Equation (3) below.

&Quot; (3) &quot;

행렬로 아래 <수학식 4>와 같이 정의될 수 있다.At this time, the SI-HKS for all m vertices of the human shape M

Can be defined as a matrix expressed by Equation (4) below.

&Quot; (4) &quot;

1.1.2 Global body descriptor (full body descriptor)

를 먼저 생성한 다음, 상기 수학식 4에 정의된 로컬 설명자

을 D에 대한 코드

로 표현한다. 마지막으로,

차원 글로벌 형상 설명자

는 코드

를 풀링(pooling)하여 얻어질 수 있다.In the present invention, BoF is used as a global shape descriptor. In general, BoF is a geometric dictionary matrix

And the local descriptor defined in equation (4)

Code for D

. Finally,

Dimension global geometry descriptor

The code

Can be obtained by pooling.

는 풀링 텀을 의미하고, 각 포인트를

와 같이 영역

에 비례하여 가중치를 부여할 수 있다.here,

Means a pooling term, and each point

Area

Weighting can be given in proportion to the weight.

For global shape descriptors, use the method of LBBC14. This method learns dictionary and sparse code using the objective function shown in Equation (5) below under the control by the dual level optimization, and improves the standard BoF constituting the dictionary managed by the uncontrolled method using clustering .

&Quot; (5) &quot;

는 양수 쌍과 음수 쌍의 비유사성 사이에서 적어도

만큼 분리하려고 하는 힌지 손실(hinge loss)를 의미하고, 코드

와

는 같은 대상의 일부 변형에 의해 영향을 받으며,

는 다른 대상에서 유래된 것일 수 있고,

는 정규화를 위한 가중 인자를 의미하며,

는 false positive 및 false negative 비율 사이의 절충을 위한 제어인자를 의미할 수 있다. here,

At least between the non-affinity of a positive pair and a negative pair

Hinge loss &quot; means a hinge loss to be separated by as much as

Wow

Are affected by some variation of the same subject,

May be derived from other objects,

Denotes a weighting factor for normalization,

May refer to control factors for the trade-off between false positive and false negative rates.

를 생성할 수 있으며, 각

에 대한 최적 코드는 아래 <수학식 6>과 같이 나타낼 수 있다.(5) is a dictionary that is optimally separated between a positive pair and a negative pair BoF

, And each

Can be expressed as Equation (6) below. &Quot; (6) &quot;

&Quot; (6) &quot;

1.1.3 On the landmark  The corresponding segmented body part descriptor

The present invention defines a global shape descriptor f that features the shape of the entire body independently of the pose and defines another shape descriptor that represents the shape of the body part associated with a particular landmark.

을 제공하는데, 여기서 열 벡터 d _i 는 신체 부위의

수를 나타내는 기하학적 요소이다. 일반적으로, 신체 부위는 서로 배타적이지 않기 때문에, 여러 요소들은 랜드마크와 관련되어 있다.BoF is a geometric dictionary

, Where the column vector d _i is the body part's

It is a geometric element that represents the number. In general, the body parts are not mutually exclusive, so the various elements are associated with landmarks.

The present invention collects only the elements related to the landmark and defines the shape descriptor using these elements.

Assuming that x _i is the vertex of the landmark i, the code is assumed as in Equation (7) below.

&Quot; (7) &quot;

는 d _i 에 해당하는 p _i 의 가중치를 의미할 수 있다.here,

Can be the weight of p _i corresponding to d _i .

Thus, an element with a non-zero value can be considered to be associated with p _i . Therefore, the method of the present invention for identifying a subset of elements associated with landmark i is as follows.

를 얻음으로써, 0이 아닌 가중치를 가지는 요소들을 찾는다. 트레이닝 세트의 모든 메쉬들 중에서 요소가 일정 비율 예를 들어, 62.5% 이상으로 0이 아닌 가중치를 갖는 경우, 요소는 랜드마크와 관련이 있는 것으로 결정될 수 있다.For every mesh in the training set, we calculate p _i for x _i

To find elements with non-zero weights. If, among all the meshes in the training set, the element has a weight that is not equal to a certain percentage, for example 62.5% or more, the element can be determined to be associated with the landmark.

Table 1 below lists the atoms associated with the landmark.

라고 불리는 분할된 신체 부위 설명자(SBD; segmented body part descriptor)를 아래 <수학식 8>과 같이 정의할 수 있다.From this analysis, for landmark i

A segmented body part descriptor (SBD) called &quot; segmented body part descriptor &quot; (SBD) can be defined as Equation (8) below.

&Quot; (8) &quot;

의 서브벡터

는 랜드마크 i와 관련된 원자들에 대응하는 요소들만을 포함할 수 있다.here,

The sub-

Lt; / RTI &gt; may contain only elements corresponding to the atoms associated with landmark i.

를 시각화함으로써, 각 랜드마크와 관련된 요소에 해당하는 꼭지점을 보여준다.

는 학습 및 랜드마크 검출에서 강력한 장점을 가지고 있고, 랜드마크와 관련된 유일한 신체 부위의 특징을 설명하기 때문에

를 기반으로 하는 랜드마크에 대한 예측은 관련 없는 다른 부분의 영향을 받지 않으며, 이는 글로벌 형상 설명자를 기반으로 한 예측의 경우 일 수 있다. 랜드마크 검출과 관련하여,

는 후보 꼭지점을 상당히 감소시키므로 검출을 빠르게 수행 할 수 있다.3 is the color for the landmark i for each vertex x _j

To show the vertices corresponding to the elements associated with each landmark.

Has a strong advantage in learning and landmark detection, and because it describes the unique body part features associated with landmarks

Are not affected by other unrelated parts, which may be the case for predictions based on global shape descriptors. Regarding landmark detection,

Can significantly reduce the candidate vertices and thus perform detection quickly.

1.2 Landmark  Pose invariant coordinates

Regardless of the change, to detect a landmark requires isometric deformations, poses, and appropriate coordinates that are independent of the magnitude and order of the vertices. World coordinates are affected by isometric deformations and pose changes, and vertex indices are affected by the size and ordering of the vertices.

In the present invention, defining the landmark coordinates uses both the spatial relationship between the landmark position and the landmark in the space of the local shape descriptor. However, the limitation of this approach is that the coordinates are symmetrical in the sagittal plane. For example, the left and right ankles have the same coordinates. To overcome this problem, the present invention can additionally use a geodesic distance between landmarks.

1.2.1 land mark  Local descriptor as a location

는

의 한 포인트이며 좌표로서의 몇 가지 중요한 속성을 만족한다. Local descriptor

The

And satisfies some important attributes as coordinates.

FIG. 4 shows the first three coordinates of p among the landmarks of 160 human subjects. As can be seen from FIG. 4, it can be seen that the points form clusters for each landmark. This means that the landmark can be distinguished when the local descriptor is used as the landmark coordinate.

에서

까지의 맵

는 주입적(injective)이지 않다. 즉, 신체상의 많은 포인트들이 로컬 설명자 공간에서는 동일한 위치로 매핑 될 수 있다. 이러한 점에 대처하기 위해, 좌표에 추가적인 요소 즉, 랜드마크 사이의 공간 관계를 필요로 할 수 있다.Also, p is independent of the pose change since it is invariant to the isometric strain. However,

in

Map up to

Is not injective. That is, many points on the body can be mapped to the same position in the local descriptor space. To cope with this, additional elements may be required in the coordinate, that is, a spatial relationship between the landmarks.

1.2.2 land mark  Interspatial relationship

Because the shape of the whole human is similar, the spatial relationship between the landmarks represents a consistent pattern. The present invention can use this feature to detect landmark locations, for which two types of spatial relationships are considered.

First, the relative position between the landmarks in the local descriptor space can be used, and the relative position can be expressed as Equation (9) below.

&Quot; (9) &quot;

Here, p _i may mean a local descriptor corresponding to landmark i.

는 랜드마크 2에서 무시될 수 있다.The positions of p _i associated with p _i-2 and p _i-1 can be selected assuming that the adjacent landmark exhibits a higher correlation than the far landmark. The relative position between p _i and p ₁ can be considered to correspond to the geodesic distance. The landmark is detected according to the order in which the landmarks increase from the landmark ₁ , so that p _i is compared with p _i-1 , p _i-1, and p ₁ . For example, in Equation 9,

Can be ignored in landmark 2.

Since p is the same for the left and right side landmarks, the relative position V _i is symmetrical at the center of the sagittal plane. To distinguish symmetrical landmarks, a measure of the additional distance, which is the geodesic distance between the landmarks, can be considered. Since geodesic distances are invariant to isometric deformations, it is for the purposes of the present invention that geodesic distances are invariant to isometric deformations, since human pose changes are considered to induce isometric skin deformations. Similar to the relative distance in the local descriptor space, the geodesic distances from landmark i to i-1, i-2 and 1 can be considered, and the geodesic distances from landmark i to i-1, i- Can be expressed as Equation (10) below.

&Quot; (10) &quot;

는 랜드마크 i에 대응하는 형상 M의 꼭지점을 의미하고, d(a, b)는 a와 b 사이의 측지선 거리를 의미할 수 있다.here,

And d (a, b) may represent a geodesic distance between a and b.

에 대한

의 처음 세 차원 값을 나타낸 것으로, 도 5에 도시된 몸체 모양에 그려진 색은 왼쪽에서 오른쪽으로 로컬 설명자 공간의 처음 세 차원에 있는 꼭지점의 값을 의미한다. 값은 비대칭 포즈의 경우에도 좌우 대칭을 나타내고, 빨간색 마커는 랜드마크를 나타내며 검은 선은 랜드마크 사이의 측지선 거리를 나타낸 것이다.FIG. 5 is a graph showing the relationship between the geodesic distances between landmarks and each vertex

For

The color drawn in the body shape shown in FIG. 5 means the value of the vertex in the first three dimensions of the local descriptor space from left to right. The values indicate symmetry even in the case of an asymmetric pose, the red markers represent the landmarks, and the black lines represent the geodesic distances between the landmarks.

A method for defining the spatial relationship between landmarks in the present invention is disclosed in ASM 06, WAS 10 (WUHRER S., AZOUZ ZB, SHU C .: Semi-automatic prediction of landmarks on human models in varying poses. In Computer and Robot Vision (CRV) WUHRER S., SHU C., XI P .: Landmark-free posture invariant human shape correspondence, The Visual Computer (2008), IEEE, pp. 136-142. 27, 9 (2011), 843-852. 2, 5, 8, 9).

공간에서의 유클리드 거리의 관점에서 공간적 관계를 정의하며, 이는 강제 변환(rigid transformations)이나 포즈 변경에 불변하지 않는다. WAS10와 WSX11은 유클리드 거리와 설명자 공간의 로컬 특징들 사이의 차이 벡터를 사용한다. 두 벡터 사이의 차이가 거리의 개념을 이미 포함하기 때문에 이러한 선택은 다소 중복된다. 이러한 접근 방식과 비교할 때, 본 발명에서는 두 개의 크게 독립적인 구성 요소 즉, 로컬 설명자 공간의 차이 벡터와 측지선 거리를 사용하므로 기존 방법에 비해 더 합리적이다.ASM06

It defines the spatial relationship in terms of Euclidean distance in space, which is not invariant to rigid transformations or pose changes. WAS10 and WSX11 use the difference vector between the Euclidean distance and the local features of the descriptor space. This choice is somewhat redundant because the difference between the two vectors already includes the concept of distance. Compared to this approach, the present invention is more reasonable than the existing method because it uses two largely independent components: difference vector of local descriptor space and geodesic distance.

2. Analysis department ( Landmark  Regression analysis to find out)

Global and separate body part descriptors and pose constants already defined are used to locate the landmark. To this end, the present invention is able to learn a regression analysis model between global and separate body part descriptors and pose constant coordinates that are already defined. Once the regression model is learned, global and body region shape descriptors for the input body shape search for appropriate landmark coordinates to locate the landmarks.

로 구성된 랜드마크의 좌표를 얻을 수 있다.Because the geometry descriptor and landmark coordinates are both high-dimensional and nonlinear, a kernel matching correlation analysis (KCCA) technique can be used as a regression analysis model. In the KCCA-based regression analysis, landmark coordinate estimation is performed using the connection parameters of the KCCA. That is, given a new human-like, perform regression analysis based on the learned parameters and connection Then, position elements p _i and the space between element

Can be obtained.

2.1 Two-step nonlinear correlation analysis

The KCCA maps the nonlinear variables to the feature space of the upper dimension and examines the correlation between the two variables in the space. The present invention performs regression analysis for two types of data pairs, namely, KCCA-based regression analysis, one for performing KCCA between the SBD and the landmark location in the local descriptor space, and the other for performing the spatial relationship between the landmarks And the global geometry descriptor. The former examines the relationship between local features, while the latter examines the relationship between global features. A detailed process of the KCCA in the present invention will be described as follows.

2.1.1 Analysis of Location

를 정의한다.Kernel functions before performing KCCA

.

는

를 더 높은 차원 공간으로 변환하는 투영 매핑을 의미할 수 있다.here,

The

To a higher dimensional space. &Lt; RTI ID = 0.0 &gt;

와 랜드마크 위치 간의 연결을 찾는 것이다. 각 랜드마크 i에 대해 트레이닝 데이터 세트

의

쌍이 주어지면, 각 트레이닝 데이터에 대응하는 각 열을 가지는 데이터로서 행렬

와

를 구성한다. 그런 다음, 커널은 아래 <수학식 11>과 같이 Q _i 와 R _i 에 대해 계산한다.For each landmark i, the present invention is based on the fact that, for each landmark i,

And the landmark location. For each landmark i, a training data set

of

When pairs are given, as data having each column corresponding to each training data,

Wow

. Then, the kernel calculates Q _i and R _i as shown in Equation (11) below.

&Quot; (11) &quot;

와

는

및

를 만족할 수 있다.Here, the projection direction for Q _i and R _i

Wow

The

And

Can be satisfied.

와

와 같은 한 쌍의 방향을 나타낼 수 있다. 여기서,

와

은 가중치를 의미할 수 있다. 그러므로, 아래 <수학식 12>와 같이 커널 함수를 사용하여 CCA의 비선형 버전을 작성할 수 있다.Therefore,

Wow

Such as < / RTI > here,

Wow

Can mean a weight. Therefore, a nonlinear version of the CCA can be created using a kernel function as shown in Equation (12) below.

&Quot; (12) &quot;

At this time, the solution to the correlation problem can be expressed by Equation (13) below.

&Quot; (13) &quot;

은

상관 값에 대응하는 t 쌍의 방향을 의미할 수 있다.here,

silver

And may mean the direction of t pairs corresponding to the correlation value.

와

가 투영 행렬

와

상에 커널

와

의 투영이라 하면,

와

는 아래 <수학식 14>와 같이 나타낼 수 있다.

Wow

A projection matrix

Wow

On the kernel

Wow

In other words,

Wow

Can be expressed by Equation (14) below.

&Quot; (14) &quot;

가 1과 유사하고,

와

가 선형적으로 상관된다고 가정하면, 선형 계수 행렬

는 아래 <수학식 15>와 같이 계산될 수 있다.here,

Is similar to 1,

Wow

Is linearly correlated, a linear coefficient matrix < RTI ID = 0.0 >

Can be calculated as Equation (15) below.

&Quot; (15) &quot;

는 랜드마크 i에 대한 것이므로, 각 랜드마크에 대해 KCCA를 수행하고

를 얻을 수 있다.here,

Is for the landmark i, so KCCA is performed for each landmark

Can be obtained.

2.1.2 Spatial Relationship Analysis

와 공간 관계

사이의 상관 관계를 분석한다. 분석은 랜드마크 좌표와 동일한 과정을 거치며, 유일한 차이점은 이러한 분석에서 KCCA가 한 번만 수행되는 반면, 이전 분석에서는 랜드마크 마다 여러 개의 KCCA가 필요하다는 것이다. 특히, 트레이닝 데이터

의

쌍이 주어지면, 행렬

와

을 데이터로 구성한다.Global (or predecessor) geometry descriptor

And spatial relationship

. The analysis goes through the same process as landmark coordinates, with the only difference being that in this analysis the KCCA is performed only once, whereas the previous analysis requires several KCCAs per landmark. In particular,

of

Given a pair,

Wow

.

일 수 있다.here,

Lt; / RTI &gt;

와 공간 관계 사이의 선형 계수 행렬

를 얻을 수 있다.Through the KCCA analysis, global geometry descriptors

And a linear coefficient matrix between spatial relations

Can be obtained.

2.2 land mark  Regression analysis of features

를 얻었으며, 글로벌 설명자와 랜드마크 간의 공간 관계 간의

를 얻었다.As a result of the correlation analysis on the training data, the linear coefficient between the segmented body part and the landmark position

Between the global descriptor and the landmark,

.

가 주어지면 s+1 수의 계수를 사용한 회귀 분석을 통해 랜드마크 위치(로컬 형상 설명자 공간에서의 랜드마크 위치)와 공간 관계를 예측한다. 분석 프로세스와 마찬가지로, 회귀 분석은 두 단계로 수행된다.New Human Model

The landmark position (landmark position in the local shape descriptor space) and the spatial relationship are predicted by regression analysis using the s + 1 number of coefficients. As with the analysis process, regression analysis is performed in two steps.

에 대한 랜드마크 위치

를 예측하는 절차를 나타낸 것으로, 각 랜드마크에 대해

를 계산한 다음, 커널

를

로 투영하고

를 곱하여

의 투영을 예측한다(4번째 줄). 이러한 회귀 분석은 로컬 형상 설명자 공간이 아닌 고차원 특징 공간에서 수행될 수 있다.

에서

로의 역 매핑은 함수 근사를 위해 널리 사용되는 모델인 RBFN(radial basis function network)을 트레이닝함으로써 이루어질 수 있다.Algorithm 1, shown below,

Landmark Locations for

, And for each landmark,

And then the kernel

To

And

By multiplying

(4th line). This regression analysis can be performed in a high dimensional feature space rather than a local shape descriptor space.

in

Can be achieved by training a radial basis function network (RBFN), a widely used model for function approximation.

와

가 각각

와

로 대체된 것과 같은 방식으로 수행된다.Regression analysis of spatial relationships

Wow

Respectively

Wow

As shown in FIG.

에서 랜드마크 위치를 획득하고, 후술되는 랜드마크 꼭지점을 감지하는데 사용되는 랜드마크

사이의 공간 관계도 획득할 수 있다.Through the regression procedure, the local geometry descriptor space

Which is used to acquire a landmark position in the landmark and to detect a landmark vertex,

Can also be obtained.

3. Detector ( land mark  detection)

를 검출한다.The present invention resolves the optimization problem shown in Equation (16) below,

.

&Quot; (16) &quot;

는 랜드마크 i에 해당하는 분할된 신체 부위의 꼭지점 집합을 의미하고,

및

는 임계 값을 의미하며,

는 가중 유클리드 거리를 의미할 수 있는데, 가중 유클리드 거리는 아래 <수학식 17>과 같이 계산될 수 있다.here,

Denotes a vertex set of a divided body part corresponding to the landmark i,

And

Quot; means a threshold value,

May denote the weighted Euclidean distance, and the weighted Euclidean distance may be calculated as Equation (17) below.

&Quot; (17) &quot;

와

는 상기 수학식 9에 나타낸

의 열 벡터를 의미할 수 있다.here,

Wow

Lt; RTI ID = 0.0 &gt; (9)

Quot; column vector &quot;

가 더 높게 설정된다.Because the closer landmark has a higher correlation than the landmark that is farther away,

Is set higher.

The process of detecting the landmark in the detection unit is composed of two steps.

를 모은 다음

에서 랜드마크 꼭지점을 찾는다.

를 구성하기 위한 의사 코드(pseudocode)는 아래 도시된 알고리즘 2를 통해 제공될 수 있다. 분할된 신체 부위 함수

는 꼭지점 x가

에 포함되는지 여부를 검사하는 데 사용된다(4번째 줄).First, the vertex set of the divided body parts corresponding to the landmark i

Then collect

Find the landmark vertices in.

The pseudocode for constructing the pseudocode may be provided through the algorithm 2 shown below. Divided body part function

The vertex x

(Line 4). &Lt; / RTI &gt;

는 랜드마크를 찾기 위한 후보 꼭지점의 수를 상당히 감소시킨다. 또한, 관련 없는 신체 부위의 꼭지점을 배제함으로써 검출 정확도를 높일 수 있다.

를 찾은 후, 상기 수학식 16은 랜드마크의 위치를 찾을 수 있다. 특히, 모든

에 대해

를 계산하고

의 증가 순서로 제약 조건을 만족하는 꼭지점을 찾는다.The segmented body region obtained in algorithm 2

Significantly reduces the number of candidate vertices for finding a landmark. In addition, detection accuracy can be improved by excluding the vertexes of unrelated body parts.

, The above Equation (16) can find the position of the landmark. In particular,

About

&Lt; / RTI &

In order of increasing order of vertices.

3.1 First land mark  detection

로만 랜드마크를 찾는 것은 좌우측을 구별 할 수 없다. 따라서, 첫 번째 랜드마크의 경우에만, 시스템에서 총 30개의 후보 랜드마크를 시각화하여 사용자가 오른쪽에서 하나를 선택할 수 있도록 한다. 후보 랜드마크의 위치는 일반적으로 매우 편중되므로, 사용자는 실질적으로 좌측 또는 우측을 선택하도록 요구되며, 사용자 입력에 의해 좌측 또는 우측이 선택되면 이후의 모든 후속 랜드마크들은 자동적으로 검출될 수 있다.The landmarks are found in order of increasing from 1 to s as the above formula (16) is solved. Since the spatial relationship can not be used when searching for the first landmark, the above equation (16) is solved without constraints. But,

Finding roman landmarks can not distinguish left and right. Thus, for the first landmark only, the system visualizes a total of 30 candidate landmarks so that the user can select one from the right. Since the position of the candidate landmark is generally very biased, the user is required to select the left or right side substantially, and if the left or right side is selected by the user input, all subsequent subsequent landmarks can be automatically detected.

FIG. 6 shows an example of a landmark detected by an apparatus according to the present invention, showing landmark detection results for a human model of various dynamic pose for 8 men and 8 women.

As can be seen from FIG. 6, the accuracy of the position of the actual landmark (red marker) and the position of the landmark (green marker) detected by the present invention in the human model of various dynamic pose is high.

Thus, the apparatus according to the present invention can detect anatomical landmarks of a human model having a dynamic pose by training a statistical regression analysis model that learns the relationship between a human body shape and a landmark.

Moreover, when the apparatus according to the present invention learns with a data set having a wide range of human shape and pose, the regression-based method can achieve higher accuracy than the registration-based method, and in particular, Canonical Correlation Analysis (KCCA) method can successfully model the correlation between human shape and landmark.

FIG. 7 is a flowchart illustrating an operation of the landmark detection method according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating an operation of the apparatus of FIG. 2 to FIG.

Referring to FIG. 7, a landmark detection method according to an exemplary embodiment of the present invention detects a pose invariant descriptor independent of a pose through training of human models for predetermined pose including landmarks, And generates a pose constant coordinate in which a position of an independent landmark in the pose is defined (S710).

Here, step S710 defines a local descriptor for the local area of each of the landmarks, defines a global shape descriptor characterizing the shape of the entire body using the defined local descriptor, The pause invariant descriptor representing a portion of the human shape associated with a particular landmark can be generated.

Further, in step S710, only the elements related to each of the landmarks among the geometric elements provided by the bag of features (BoF) method may be collected, and the pose invariant descriptor may be generated using only the collected elements.

In addition, step S710 may generate the pose invariant coordinate using a feature vector indicating the position of the landmark independent of the dynamic pose, and may use the feature vector and the geodesic distance to generate the pose invariant coordinate between the independent landmarks And the pose constant coordinate may be generated using the spatial relationship.

When the pose invariant descriptor and the pose invariant coordinates are generated in step S710, the regression coordinates of the human land shaped landmark inputted through the regression analysis using the pose invariant descriptor and the pose invariant coordinate are obtained (S720).

At this time, the step S720 learns a regression analysis model between the pose invariant descriptor, the pose invariant coordinate, and the global shape descriptor characterizing the shape of the whole body, and uses the learned regression analysis model to calculate the input human shape The regression coordinates for the landmarks of the landmark can be obtained.

Here, in step S720, the regression analysis model may be learned using a KCCA technique.

Specifically, step S720 may be performed by performing a Kernel Accuracy Correlation Analysis (KCCA) between the pose invariant descriptor and the location of the independent landmark, and performing a kernel matching correlation analysis between the spatial relationship between the independent landmarks and the global shape descriptor , And regression coordinates of the input human-shaped landmark can be obtained through regression analysis of the kernel correlation correlation analysis performed.

When the regression coordinates of the human shape landmark input in step S720 are obtained, the input landmarks of the human body shape are detected using the obtained regression coordinates (S730).

Although the description of FIG. 7 is omitted, the method of FIG. 7 may include all of the elements described in FIGS. 2-6, which is obvious to those skilled in the art.

The system or apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the systems, devices, and components described in the embodiments may be implemented in various forms such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array ), A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to embodiments may be implemented in the form of a program instruction that may be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI &gt; or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (18)

Generating a pose invariant coordinate that defines a pose invariant descriptor independent of the pose and a position of the landmark independent of the dynamic pose through training of human models for preset pose including the landmarks;
Obtaining regression coordinates of a human land shaped landmark input through a regression analysis using the pose invariant descriptor and the pose invariant coordinates; And
Detecting the input landmark of the human body using the obtained regression coordinates
The method comprising the steps of:
The method according to claim 1,
The generating step
Defining a local descriptor for each local region of the landmarks, defining a global shape descriptor characterizing the shape of the entire body using the defined local descriptor, and using the defined global shape descriptor to define a specific land Wherein the pose invariant descriptor representing a portion of the human shape associated with the mark is generated.
The method according to claim 1,
The generating step
Collecting only the elements related to each of the landmarks among the geometric elements provided by the bag of features (BoF) method, and generating the pose invariant descriptor using only the collected elements.
The method according to claim 1,
The generating step
Wherein the pose constant coordinate is generated using a feature vector indicating a position of the landmark independent of the dynamic pose.
5. The method of claim 4,
The generating step
Wherein the feature vector and the geodesic distance are used to represent the spatial relationship between the independent landmarks and the pose invariant coordinates are generated using the spatial relationship.
The method according to claim 1,
The obtaining step
A regression analysis model between the pose invariant descriptor, the pose invariant coordinate, and the global shape descriptor characterizing the shape of the whole body is learned, and the regression analysis model is used to calculate the return And obtaining the coordinates of the landmark based on the regression analysis.
The method according to claim 6,
The obtaining step
Wherein the regression analysis model is learned using a kernel matching correlation analysis (KCCA) technique.
The method according to claim 6,
The obtaining step
Performing kernel correlation correlation analysis (KCCA) between the pose invariant descriptor and the position of the independent landmark, performing a kernel matching correlation analysis between the spatial relationship between the independent landmarks and the global shape descriptor, And obtaining regression coordinates of the inputted human figure-shaped landmark through a regression analysis on the result of the kernel matching correlation analysis.
Generating pose invariant coordinates defining a position of a landmark independent of a dynamic pose through training of human models for preset pose including landmarks;
Acquiring regression coordinates of a human land shaped landmark input through regression analysis using the generated pose invariant coordinates; And
Detecting the input landmark of the human body using the obtained regression coordinates
The method comprising the steps of:
A preprocessing unit for generating a pose invariant descriptor independent of the pose and a pose invariant coordinate defined for the position of the landmark independent of the dynamic pose through learning of the human models for the preset pose including the landmarks, ;
An analyzer for obtaining regression coordinates of a human land shaped landmark inputted through a regression analysis using the pose invariant descriptor and the pose invariant coordinates; And
A detecting unit for detecting the inputted human-shaped landmark using the obtained regression coordinates;
Based on the regression analysis.
11. The method of claim 10,
The pre-
Defining a local descriptor for each local region of the landmarks, defining a global shape descriptor characterizing the shape of the entire body using the defined local descriptor, and using the defined global shape descriptor to define a specific land And generates the pose invariant descriptor representing a part of the human body shape associated with the mark.
11. The method of claim 10,
The pre-
Wherein the pod invariant descriptor is generated by collecting only the elements related to each of the landmarks among the geometric elements provided by the bag of features method and using only the collected elements to generate the pose invariant descriptor.
11. The method of claim 10,
The pre-
Wherein the pose constant coordinate is generated using a feature vector indicating a position of the landmark independent of the dynamic pose.
14. The method of claim 13,
The pre-
Wherein the pseudo coordinates are generated by using the feature vector and the geodesic distance to represent the spatial relationship between the independent landmarks and using the spatial relationship.
11. The method of claim 10,
The analyzer
A regression analysis model between the pose invariant descriptor, the pose invariant coordinate, and the global shape descriptor characterizing the shape of the whole body is learned, and the regression analysis model is used to calculate the return And obtaining coordinates of the landmark of the landmark.
16. The method of claim 15,
The analyzer
Wherein the regression analysis model is learned using a kernel matching correlation analysis (KCCA) technique.
16. The method of claim 15,
The analyzer
Performing kernel correlation correlation analysis (KCCA) between the pose invariant descriptor and the position of the independent landmark, performing a kernel matching correlation analysis between the spatial relationship between the independent landmarks and the global shape descriptor, And obtaining regression coordinates of the inputted human figure-shaped landmark through a regression analysis on the result of the kernel matching correlation analysis.
A preprocessing unit for generating pose invariant coordinates in which a position of a landmark independent of a dynamic pose is defined through training of human models with respect to preset pose including landmarks;
An analyzer for obtaining regression coordinates of a human land shaped landmark inputted through regression analysis using the generated pose invariant coordinates; And
A detecting unit for detecting the inputted human-shaped landmark using the obtained regression coordinates;
The method comprising the steps of:

Images