KR20110126314A - Method for extracting 3d line using stereo camera based fuzzy k-means clustering scheme - Google Patents

Abstract

PURPOSE: A method for extracting a 3D line using a stereo camera based fuzzy k-means clustering scheme is provided to exactly detect a 3D line in a low complexity. CONSTITUTION: A detection apparatus creates a 3D point group corresponding to one or more 2D straight lines(S110). The detection apparatus creates a plurality of point groups corresponding one or more 2D lines through a grouping process of the 3D point group(S120). The detection apparatus removes a noise point group(S130). The detection apparatus groups the point groups of the same direction into a point group(S140). The detection apparatus estimates a plurality of 3D straight lines(S150).

Description

Method for extracting 3D line using stereo camera based fuzzy k-means clustering scheme}

The disclosed technique relates to a three-dimensional straight line estimation technique, and more particularly, but not exclusively, a three-dimensional straight line corresponding to a two-dimensional straight line included in the image based on images obtained through a stereo camera. Relates to a technique for estimating.

Three-dimensional object recognition and / or three-dimensional object modeling, which is applied to many applications such as industrial robots or home robots, generally requires three-dimensional features such as mesh, three-dimensional straight lines, and colors. . In particular, three-dimensional straight lines are treated as more important three-dimensional features when the object to be recognized or modeled is in the form of a box, such as most artificial objects (eg, boxes, tables, buildings made by humans). In addition, the three-dimensional straight line may be used as a landmark for vision-based Simultaneous Localization And Mapping (SLAM) in the robot automatic navigation field.

Examples of the prior art for detecting such three-dimensional straight lines are described in Dong-Min Woo, "Geeration of 3D Line Segment Based on Disparity Data," Electronic Computer Technology, 2009, International Conference, pp. 481-485, Feb. Techniques disclosed in 2009. (hereinafter referred to as Document 1), Zhaojin Lu, Seungmin Baek, Sukhan Lee, "Robust 3D Line Extraction from Stereo Point Clouds," Robotics, Automation and Mechatronics, 2008 IEEE Conference, pp. 1-5, 21-24 Sept. Although there are techniques disclosed in 2008. (hereinafter referred to as Document 2), these techniques have room for improvement in terms of performance such as accuracy of detection and computational complexity.

The technical problem to be achieved by the disclosed technology is to provide a technique for accurately detecting a three-dimensional straight line using stereo data.

In order to achieve the above technical problem, an aspect of the disclosed technology (a) clustering a three-dimensional point cluster to generate a plurality of point groups; And (b) removing the noise point group in consideration of the goodness of fit of each of the plurality of point groups.

In one embodiment, the method further includes (c) performing grouping on point groups other than the noise point group, using eigenanalysis. In one embodiment, the method further comprises (d) performing a three-dimensional straight line estimation based on the result of performing the step (c). In an embodiment, the step (c) includes grouping point groups in the same direction among the point groups other than the noise point group into one point group, and the step (d) is based on the point group. Estimating a three-dimensional straight line.

In one embodiment, the method further comprises generating the three-dimensional point clusters corresponding to each of the at least one two-dimensional straight line, prior to the step (a). The generating of the 3D point cluster may include: generating a 3D point cluster corresponding to each of the at least one 2D straight line, based on the left image and the right image obtained from the stereo camera. It includes. The generating of the 3D point cluster may include: extracting the at least one two-dimensional straight line from at least one of the left image and the right image; And projecting the extracted two-dimensional straight lines in a three-dimensional image to generate the three-dimensional point cluster.

In an embodiment, the step (b) includes removing the noise point group based on the energy content of the eigenvector for each of the plurality of point groups.

In an embodiment, the step (a) may include generating the plurality of point groups using a k-means clustering technique or a Fuzzy k-means clustering technique.

In one embodiment, step (a) is based on a k-means clustering technique using an objective function proportional to the sum of metrics corresponding to each of the three-dimensional points included in the three-dimensional point cluster. Generating a plurality of point groups, wherein the metric is proportional to the product of the probability that the three-dimensional point belongs to the point group and the distance between the three-dimensional point and the center of the point group. In one embodiment, the distance is a Mahalanobis distance.

In order to achieve the above technical problem, another aspect of the disclosed technology is to (a) clustering a three-dimensional point cluster to generate a plurality of point groups; And (b) merging the overlapped point groups in consideration of the goodness of fit of each of the plurality of point groups.

In order to achieve the above technical problem, another aspect of the disclosed technique comprises the steps of: (a) obtaining noisy three-dimensional point data representing at least one three-dimensional straight line; And (b) extracting the at least one three-dimensional straight line by applying a clustering technique to the obtained three-dimensional point data.

In one embodiment, the step (b) includes the step of removing the noisy clusters by evaluating the suitability of each of the clusters generated as a result of applying the clustering technique.

In one embodiment, the step (b) includes the step of merging the overlapping clusters by evaluating the fitness of each of the clusters generated as a result of applying the clustering technique.

In one embodiment, the step (b) includes evaluating the suitability of each of the clusters generated as a result of applying the clustering technique, removing the noisy clusters, and merging the duplicated clusters.

In one embodiment, step (b) includes applying a k-means or fuzzy k-means clustering technique.

In one embodiment, the step of removing the noisy cluster may include: applying a principal component analysis or an eigen value analysis to each of the clusters to evaluate a goodness of fit; And removing the noisy cluster by measuring the superiority of the size of the main component to the size of the non-main component as a result of the main component analysis.

In an embodiment, the merging of the duplicated clusters may include: applying a principal component analysis or an eigen value analysis to each of the clusters to evaluate fitness; And merging clusters having principal components in the same direction among clusters other than the noisy cluster into one cluster.

In order to achieve the above technical problem, another aspect of the disclosed technology provides a computer readable recording medium containing a computer program for realizing a method according to various embodiments.

Embodiments of the present invention presented above may have an effect including the following advantages. However, all the embodiments of the present invention are not meant to include them all, and thus the scope of the present invention should not be understood as being limited thereto.

With low complexity, accurate three-dimensional straight lines can be detected.

1 is a flowchart illustrating a 3D line detection method according to an exemplary embodiment.
2 illustrates a clustering algorithm according to one embodiment.
3 illustrates a 3D line estimation algorithm according to an embodiment.
4 illustrates an example in which a three-dimensional straight line is refined according to an embodiment.
5 and 6 are flowcharts and functional block diagrams illustrating the processes from S110 to S140 of FIG. 1 in more detail, respectively.
7 is a functional block diagram illustrating a three-dimensional line detection method according to the disclosed technology.
8 and 9 illustrate the performance of the disclosed and existing technologies.

Since descriptions of embodiments of the present invention are merely illustrated for structural to functional descriptions of the present invention, the scope of the present invention should not be construed as limited by the embodiments described in the present invention. That is, the embodiments of the present invention may be variously modified and may have various forms, and thus, it should be understood to include equivalents that may realize the technical idea of the present invention.

On the other hand, the meaning of the terms described in the present invention will be understood as follows.

Terms such as “first” and “second” are used to distinguish one component from other components, and the scope of the present invention should not be limited by these terms. For example, the first component may be named a second component, and similarly, the second component may also be named a first component.

The term “and / or” should be understood to include all combinations that can be presented from one or more related items. For example, "first item, second item, and / or third item" means "at least one or more of the first item, second item, and third item", and means first, second, or third item. A combination of all items that can be presented from two or more of the first, second and third items as well as the third item.

Singular expressions described herein are to be understood to include plural expressions unless the context clearly indicates otherwise, and the terms "comprise" or "having" include elements, features, numbers, steps, operations, and elements described. It is to be understood that the present invention is intended to designate that there is a part or a combination thereof, and does not exclude in advance the possibility of the presence or addition of one or more other features or numbers, steps, actions, components, parts or combinations thereof. .

Each step described in the present invention may occur out of the stated order unless the context clearly dictates the specific order. That is, each step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

Unless otherwise defined, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and shall be interpreted as having ideal or overly formal meanings unless expressly defined in this application. Can't be.

The disclosed technique is to provide a variety of devices (e.g., robots) equipped with stereo cameras (e.g. bumblebee cameras) and stereo cameras necessary for calculating left and right images, three-dimensional information obtained by taking data (e.g., stereo cameras). The camera model parameter information such as the distance between views, etc.). Of course, those skilled in the art can fully understand that the various devices and processing apparatuses may further perform 3D object recognition, 3D object modeling, SLAM, etc. based on the detected three-dimensional straight lines. Hereinafter, for convenience, the disclosed technique will be described by referring to the subject performing the three-dimensional line detection method of the present invention as a detection device.

1 is a flow chart illustrating a three-dimensional line detection method 100 according to an embodiment.

Referring to FIG. 1, in S110, the detection apparatus generates a three-dimensional point cluster corresponding to each of at least one two-dimensional straight line. The step S110 may be performed in the following two steps (called S110_1 and S110_2), but is not necessarily limited thereto.

In S110_1, the detection apparatus extracts at least one two-dimensional straight line based on at least one of a left image (which is a two-dimensional image) and a right image (which is a two-dimensional image) obtained from the stereo camera. In general, a method of extracting at least one two-dimensional straight line from one of a left image and a right image may be used. Here, an example of a method of extracting a 2D straight line may include using a canny edge detection technique from one image, but is not limited thereto.

In S110_2, the detection apparatus projects each of the extracted at least one two-dimensional straight line to a three-dimensional image to generate a three-dimensional point cluster corresponding to each of the two-dimensional straight lines. Here, the left image and the right image may be used during 3D projection.

Each three-dimensional point q = [x, y, z] ^T included in the three-dimensional point cluster obtained from the left image and the right image is expressed by Equation 1 by the uncertantiy of depth information due to the limitation of the stereo vision. Given a covariance matrix.

Since such covariance matrices are disclosed in many documents (eg, document 2), detailed descriptions are omitted.

In operation S120, the detection apparatus groups the three-dimensional point clusters to generate a plurality of point groups corresponding to each of the at least one two-dimensional straight line.

In one embodiment, the detection device is based on the k-means clustering technique using an objective function proportional to the sum of the metric corresponding to each of the three-dimensional point included in the three-dimensional point cluster, the at least one 2 A plurality of point groups corresponding to each of the dimensional straight lines are generated. Here, the metric is generally proportional to the product of the probability that the three-dimensional point belongs to the point group and the distance between the three-dimensional point and the center of the point group. The clustering technique of this property is called a fuzzy k-means clustering technique for convenience. An example of a fuzzy k-means clustering technique is a clustering technique for minimizing an objective function J _fuz, which is given by Equation 2. An example of the specific algorithm is shown in FIG.

In Equation 2, i, j represents the index of the three-dimensional point and the index of the cluster, respectively. In addition, n is the number of 3D points belonging to the cluster object, k is a parameter that can be set in advance as the number of clusters to be obtained through the cluster. Further, when φ is a fuzzy index and φ is 0, it can be seen that Equation 2 is an objective function according to a general k-means clustering technique. In one embodiment, φ may be set greater than one.

의 곱으로 표현되는데 전자는 해당 3차원 포인트 x _i가 c _j를 중심으로 하는 클러스터에 속할 확률을 의미하며, 후자는 수학식 3으로 주어지는 마할라노비스 거리(Mahalanobis distance)를 나타낸다In Equation 2, the metric to be summed

The former represents the probability that the three-dimensional point x _i belongs to a cluster centered on c _j , and the latter represents the Mahalanobis distance given by Equation 3.

The Mahalanobis distance is disclosed in Document 2 and in Mahalanobis, PC (1936), "On the generalized distance in statistics," Proceedings of the National Institute of Sciences of India 2 (1): 49-55, etc. do.

를 초기화하는 방법의 예로는, 균일(uniform) 확률 분포로 확률 값을 할당하는 방법을 들 수 있다.In the initialization step of FIG. 2, n, k, c ₁ , c ₂ , ..., c _k may be initialized in the same manner as a general k-means clustering technique. Further, φ is a design parameter corresponding to the fuzzy index, which is a value that can be preset through a preliminary experiment (for example, a value greater than 1). Also,

As an example of a method of initializing, a method of allocating probability values with a uniform probability distribution is given.

Equation 4 may be used in the normalization step of FIG. 2.

및 c _j는 목적 함수 J_fuz를 각각

및 c _j로 편미분한 결과를 0으로 하는 값으로서 수학식 5 및 6으로 주어진다.Meanwhile, the objective function J _fuz of Equation 2 is minimized.

And c _j represent the objective function J _fuz , respectively.

And values obtained by partial division by c _j are given by Equations 5 and 6.

When a sufficiently converged data is secured (e.g., a predetermined number of iterations is a predetermined number of times, or when members of a cluster are not changed, as in a general k-means clustering algorithm), the detection apparatus may have a value smaller than a predetermined threshold. 2), the calculations of Equations 5 and 6 are repeated.

In operation S130, the detection apparatus removes the noise point group based on the energy content of the unique vector for each point group. Here, the energy content of the eigenvector may be represented by Equation 7 to be described later.

을 각 포인트 그룹에 대해서 수학식 7과 같이 구한다.In one example, the detection apparatus first performs principal component analysis to determine eigenvalues and dominant eigenvectors (ie, eigenvectors with the largest eigenvalues) for each of a plurality of (ie k) point groups. ) For reference, three eigenvectors and three eigenvalues are obtained for one point group. Then, the detection device determines the ratio of the eigenvalues corresponding to the dominant eigenvectors to the sum of the eigenvalues.

Is obtained for each point group as in Equation (7).

이 미리 설정된 임계치(예컨대, 90%)보다 포인트 그룹은 노이즈 포인트 그룹으로 간주하여 제거될 수 있다.Said ratio

Point groups above this preset threshold (eg, 90%) may be considered as noise point groups and removed.

In S140, the detection apparatus groups the point groups in the same direction among the point groups that are not the noise point group (ie, surviving in step S130) into one point group. In one example, point groups that are somewhat similar in direction of the main eigenvectors (eg, belonging to a range within a π / 18 radian) may be merged into one point group.

In another embodiment, S130 and S140 may be performed simultaneously. The detection device in this case is a point with three-dimensional points in which the direction of the principal eigenvectors is somewhat similar (e.g., within a range of π / 18 radian) and the ratio has a predetermined threshold (e.g. 90%) or more. Performs the operation of merging groups into one group. Here, the principal component analysis technique itself is described in M. Turk and A. Pentland, "Eigen faces for recognition," Journal of Cognitive Neuroscience, 3 (1): 71-86, 1991. and P. Belhumeur, J. Hespanha, and D. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection," In ECCV (1), pages 45-58, 1996.

In S150, the detection apparatus estimates a plurality of three-dimensional straight lines based on the plurality of point groups. In one embodiment, the detection apparatus performs eigenvalue analysis to estimate the at least one three-dimensional straight line consisting of a direction vector and reference coordinates of the direction vector, based on the plurality of point groups. In this case, the reference coordinate indicates location information for distinguishing a starting point, a center, or an end point of the same direction vector, since they may be different. In one example, the detection apparatus uses the algorithm of FIG. 3 to perform three-dimensional line estimation.

In S160, the detection apparatus may connect or distinguish two or more 3D straight lines based on the directions of the plurality of 3D straight lines and the proximity between the plurality of 3D straight lines. For example, as shown in FIG. 4, three-dimensional straight lines which are in the same direction and adjacent to each other are merged to be finally determined as one three-dimensional straight line, and three-dimensional straight lines which are not in the same direction or are separated from each other are separated from each other. The final dimension is determined by a straight line. In one embodiment, the detection device may connect two or more three-dimensional straight lines based on the direction vector and the reference coordinate obtained in S130. Referring to FIG. 4, four three-dimensional straight line candidates (that is, three-dimensional straight lines formed by three-dimensional point groups 1, 2, 3, and 4) are merged and finally determined as one three-dimensional straight line (red straight line), The remaining three-dimensional straight line candidates (that is, three-dimensional straight lines formed by the three-dimensional point group 5) are finally determined as three-dimensional straight lines (blue straight lines) separate from the merged three-dimensional straight lines. Meanwhile, the three-dimensional straight line indicated by the dotted line in FIG. 4 corresponds to the three-dimensional straight line detected by the technique according to document 2 (that is, the three-dimensional straight line detection technique based on RANSAC) in the same environment. Thus, it can be seen that the disclosed technique is superior in terms of detection accuracy than the technique according to document 2.

According to an embodiment, the detection apparatus may further perform a verification operation on the 3D straight line determined in S160. According to an embodiment, the verification method may include generating two-dimensional straight lines by projecting each of the three-dimensional straight lines determined in S160 onto a two-dimensional image (S170);

Determining whether there is a similarity between the two-dimensional straight line generated in S170 and the two-dimensional straight line generated in S110_1 (for example, whether it is within a preset allowable range in terms of direction, position, and length) (S180); And if it is determined that the result of the determination is not similar, the process proceeds to S120. Otherwise, the process may include determining the final three-dimensional straight line to terminate the three-dimensional straight line detection procedure (S190).

5 and 6 illustrate the processes from S110 to S180 of FIG. 1, respectively.

Illustrative flowchart and functional block diagram.

7 is a functional block diagram illustrating a three-dimensional line detection method according to the disclosed technology.

8 and 9 show the performance of the disclosed technique and the existing technique (RANSAC-based three-dimensional line detection method according to Document 2).

Referring to the experimental results according to FIGS. 8 and 9, it can be seen that the disclosed technique has lower computational complexity than the existing technique (RANSAC-based three-dimensional linear detection method according to Document 2) and has excellent performance in terms of extraction rate and accuracy. have.

The present invention can also be embodied as computer-readable codes on a computer-readable recording medium. The computer readable recording medium includes all kinds of recording devices in which packets which can be read by a computer system are stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, which are also implemented in the form of carrier waves (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

The inventors of the present invention have been described with reference to the embodiments shown in the drawings for clarity, but this is merely exemplary, and those skilled in the art may various modifications and other equivalent embodiments therefrom. Will understand. Therefore, the true technical protection scope of the present invention will be defined by the appended claims.

Embodiments of the present invention presented above may have an effect including the following advantages. However, all the embodiments of the present invention are not meant to include them all, and thus the scope of the present invention should not be understood as being limited thereto.

With low complexity, accurate three-dimensional straight lines can be detected.

Claims (20)

(a) clustering a three-dimensional point cluster to generate a plurality of point groups; And
(b) removing the noise point group in consideration of the goodness of fit of each of the plurality of point groups. The method of claim 1,
(c) using eigen analysis, further comprising grouping the point groups other than the noise point group The method of claim 2,
and (d) performing a three-dimensional straight line estimation based on the result of the performing of step (c). The method of claim 3,
Step (c) includes grouping point groups in the same direction among the point groups other than the noise point group into one point group,
The step (d) comprises the step of estimating a three-dimensional straight line based on the point group. The method of claim 1,
Prior to the step (a), further comprising the step of generating the three-dimensional point clusters corresponding to each of the at least one two-dimensional straight line The method of claim 5, wherein generating the three-dimensional point clusters,
And generating a three-dimensional point cluster corresponding to each of the at least one two-dimensional straight lines, based on the left image and the right image acquired from the stereo camera. The method of claim 6, wherein generating the three-dimensional point clusters,
Extracting the at least one two-dimensional straight line from at least one of the left image and the right image; And
And projecting the extracted two-dimensional straight lines onto a three-dimensional image to generate the three-dimensional point cluster. According to claim 1, wherein step (b),
Removing the noise point group based on the energy content of the eigenvector for each of the plurality of point groups. According to claim 1, wherein the step (a),
generating the plurality of point groups using a k-means clustering technique or a Fuzzy k-means clustering technique.
The method of claim 1,
In step (a), the plurality of point groups are generated based on a k-means clustering technique using an objective function that is proportional to the sum of metrics corresponding to each of the three-dimensional points included in the three-dimensional point cluster. Including the steps of:
Wherein said metric is proportional to the product of the probability that said three-dimensional point belongs to said point group and the distance between said three-dimensional point and the center of said point group. The method of claim 10,
Wherein said distance is a Mahalanobis distance. (a) clustering a three-dimensional point cluster to generate a plurality of point groups; And
and (b) merging overlapping point groups in consideration of goodness of fit of each of the plurality of point groups.
(a) obtaining noisy three-dimensional point data representing at least one three-dimensional straight line; And
(b) extracting the at least one three-dimensional straight line by applying a clustering technique to the obtained three-dimensional point data; The method of claim 13, wherein step (b) comprises:
Evaluating the fitness of each of the clusters generated as a result of applying the clustering technique to remove the noisy clustering method The method of claim 13, wherein step (b) comprises:
And merging the overlapping clusters by evaluating the fitness of each of the clusters generated as a result of applying the clustering technique. The method of claim 13, wherein step (b) comprises:
And evaluating goodness of each of the clusters generated as a result of applying the clustering technique to remove the noisy clusters and merging the duplicated clusters. The method of claim 13, wherein step (b) comprises:
A three-dimensional straight line extraction method comprising applying the k-means or fuzzy k-means clustering technique. The method of claim 14, wherein removing the noisy community,
Evaluating fitness by applying Principle Component Analysis or Eigen value analysis to each of the clusters; And removing the noisy cluster by measuring the superiority of the size of the main component to the size of the non-main component as a result of the principal component analysis. The method of claim 16, wherein merging the duplicated clusters,
Evaluating fitness by applying Principle Component Analysis or Eigen value analysis to each of the clusters; And
And merging clusters having principal components in the same direction among the non-noisy clusters into one cluster. A computer-readable recording medium containing a computer program for realizing the method according to any one of claims 1, 12, and 13.

Images