KR101846390B1 - Apparatus and method of camera network calibration with a small calibration pattern - Google Patents

Abstract

In the camera network correction using a small correction pattern, a small correction pattern is installed around a target object to be reconstructed in three dimensions, and a small correction pattern is generated from a plurality of cameras provided in the camera network It is possible to more easily perform camera correction of the camera network disposed around the object in order to perform 3D reconstruction of arbitrary objects and persons using the matrix values.

Description

[0001] CAMERA NETWORK CALIBRATION WITH A SMALL CALIBRATION PATTERN [0002]

The present invention relates to a camera correction method, and more particularly to a camera correction method in which a small correction pattern is installed in the vicinity of a target object for three-dimensional restoration in camera network correction, Dimensional correction of a camera network disposed around a target object in order to reconstruct an object and a person using a matrix value generated by a plurality of cameras provided in the camera network And a camera network correction method and apparatus.

The camera calibration process for obtaining internal variables and external variables of a camera is a task that must be preceded for all operations for obtaining three-dimensional information using a camera.

Although there are many types of camera correction methods, the method of correcting the camera using the correction pattern can guarantee the most stable and accurate result. However, when the size of the object to be reconstructed is large, the size of the correction pattern is also proportional to the size of the object. Such a large correction pattern has a problem of maintenance, maintenance, and management. In addition, there is an inverse relationship between the cost and precision for manufacturing a large correction pattern, which is also a problem in system construction.

In order to solve such a problem, there is a method of moving a light point by hand and taking a picture of this point at a plurality of cameras at the same time and correcting it by a self-calibration method. In order to overcome the problem of making a correction pattern of a three-dimensional structure, there is a method of using a correction pattern of a two-dimensional structure. However, this method has a drawback that the size of the pattern must be large in proportion to the size of the object to be imaged. The accuracy of measuring the relative position of the camera is reduced.

Korean Patent Laid-Open Publication No. 10-2007-0101580 discloses a technique for estimating a camera correction matrix on October 17, 2007.

Accordingly, the present invention proposes a method for eliminating the necessity of a large correction pattern for restoration when the object is large when the object is three-dimensionally reconstructed using a camera network. For this method, we propose a method of capturing a correction pattern at several arbitrary positions within the volume occupied by the object. Since the position of each correction pattern is not known at this time, it is necessary to simultaneously calculate the position of the pattern as well as the correction of the camera.

Therefore, in the camera network correction, a small correction pattern is installed in the vicinity of an object to be subjected to three-dimensional reconstruction, and a matrix value generated from a plurality of cameras provided in the camera network is used And to provide a camera network correction method and apparatus using a small correction pattern that can perform camera correction of a camera network disposed around a target object for three-dimensional reconstruction of an object and a person.

There is provided a camera network correction method using a small correction pattern, comprising the steps of: receiving a projection matrix generated from each camera for a plurality of correction patterns and generating a portion of the projection matrix as a sub-projection matrix; (SVD) on the sub-measurement matrix, and based on the result of the SVD, the sub-measurement matrix is transformed into a matrix of rank 3 Performing a SVD on the transformed sub-measurement matrix; extracting a rotation value of the small correction pattern and an internal factor and a rotation value of the camera according to a result of the SVD; And performing a correction.

In addition, the sub-projection matrix is generated as a predetermined 3X3 part in front of the projection matrix.

In addition, the sub-projection matrix is generated by the number of cameras for one correction pattern.

Further, in the performing of the correction, the translational movement value of the camera and the translational movement value of the small correction pattern are calculated using the rotation value of the extracted small correction pattern, the internal factor of the camera, the rotation value and the projection matrix .

According to another aspect of the present invention, there is provided a camera network correction apparatus comprising: a sub-projection matrix generation unit for receiving a projection matrix generated from each camera for a plurality of correction patterns and generating a sub-projection matrix of a part of the projection matrix; A sub-measurement matrix generation unit for generating a sub-measurement matrix by vertically connecting the sub-projection matrices generated from the matrix generation unit, and performing a singular value decomposition (SVD) on the sub- And an SVD is performed on the transformed sub-measurement matrix, and a rotation value of the small correction pattern and an internal factor and a rotation value of the camera are extracted And a correction unit for performing network correction of the camera.

The sub-projection matrix generation unit may generate a sub-projection matrix of a predetermined 3X3 portion in front of the projection matrix.

In addition, the sub-projection matrix is generated by the number of cameras for one correction pattern.

The correction unit may calculate the translational movement value of the camera and the translational movement value of the small correction pattern using the rotation value of the small correction pattern, the internal factor of the camera, the rotation value, and the projection matrix .

In the camera network correction using a small correction pattern, a small correction pattern is installed around a target object to be reconstructed in three dimensions, and a small correction pattern is generated from a plurality of cameras provided in the camera network There is an advantage that camera correction of a camera network disposed around a target object can be more easily performed in order to reconstruct an arbitrary object and a person using the matrix value.

In addition, when the correction of the camera network is performed through the present invention, it is not necessary to use a large-sized correction pattern even when the object to be photographed is large, so that it is possible to reduce the cost and labor for maintenance, It is possible to overcome the limitations of precision and cost which are problematic in manufacturing the semiconductor integrated circuit device, thereby contributing to the reduction of the construction cost of the system and the improvement of the work efficiency of the operator.

1 is a block diagram of a camera network correction apparatus according to an embodiment of the present invention;
2 is an exemplary view of a camera network system to which the present invention is applied,
3 is an exemplary view of a camera correction pattern according to an exemplary embodiment of the present invention.
4 is a conceptual view of a photographing situation according to an embodiment of the present invention,
5 is a diagram illustrating an example of a correction pattern photographing result in a camera network according to an embodiment of the present invention.
FIG. 6 is a conceptual diagram illustrating a configuration and principle of a camera network correction according to an exemplary embodiment of the present invention;
7 is a conceptual diagram for explaining a principle of correcting a correction pattern production error according to an embodiment of the present invention.

Hereinafter, the operation principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. The following terms are defined in consideration of the functions of the present invention, and these may be changed according to the intention of the user, the operator, or the like. Therefore, the definition should be based on the contents throughout this specification.

FIG. 1 is a block diagram of a camera network correction apparatus using a small correction pattern according to an embodiment of the present invention. As shown in FIG. 2, a camera network system for three- To perform correction.

A small correction pattern designed and fabricated in a three-dimensional structure is placed at arbitrary positions in the photographing space and photographed using a camera disposed in the vicinity. At this time, each camera can calculate one projection matrix corresponding to one pattern position. Therefore, when a pattern is placed at several positions, each camera produces a corresponding number of projection matrices. The sub-projection matrix generation unit 100 generates a sub-projection matrix at the 3x3 portion of the projection matrix.

In the sub-measurement matrix generation unit 102, a plurality of sub-projection matrices measured in the above manner are connected in the horizontal and vertical directions to generate one matrix, i.e., a sub-measurement matrix. If the sub-projection matrices corresponding to the factors of the sub-metric matrix are adjusted to have a matrix of 1, the sub-metric matrix must be a matrix of rank 3.

However, since the rank is not 3 due to image noise, the optimizer performs the following optimization process. SVD (Singular Value Decomposition) is performed on the sub-measure matrix and based on the result of this SVD, a matrix with rank 3 closest to the sub-measure matrix is created. Next, SVD is performed on the sub-measurement matrix of rank 3 obtained as a result of the above optimization process, and based on the SVD, the internal parameters of the camera, the rotation value, and the correction pattern The rotation value can be extracted. At this time, the translational movement value of the camera and the translational movement value of the correction pattern can be calculated using the result and the projection matrix.

Hereinafter, the camera network correction of the present invention will be described in more detail with reference to FIG. 2 to FIG.

The present invention first requires a correction pattern of a three-dimensional structure as shown in Fig. The shape of the correction pattern may be of any form, but the camera must be able to observe at least six reference points that are not in one plane when viewed from any direction. The shape of the correction pattern in FIG. 3 can satisfy two or more planes in any direction, so that the condition can be satisfied if there are four or more reference points on one plane.

The correction pattern is photographed at various arbitrary positions in the space to be photographed as shown in FIG. At this time, as the sum of the positions where the correction patterns are placed covers the whole photographing space, a good correction result can be obtained. An example of such a photographing result is shown in Fig.

Assuming that x is a reference point on a correction pattern where a position is extracted on the image and X is a three-dimensional position of a reference point on a correction pattern in an arbitrary world coordinate system, as shown in FIG. 6, have.

기호는 행렬의 인자들의 상대적 크기 까지만 같다는 의미로 사용한다.K is the internal factor of the camera, R and t are the rotation matrix and the position vector corresponding to the position of the camera, respectively, and are collectively called the outsider of the camera. From now on

The symbol is used to mean that it is equal to the relative size of the parameters of the matrix.

In the above equation, X can be expressed by the following equation (2) as shown in FIG.

Where Xref is the three-dimensional coordinates of the reference points relative to the reference coordinate system on the correction pattern, and S and V are the rotation matrix and position vector corresponding to the position of the correction pattern, respectively.

As shown in FIG. 6, a camera projection matrix satisfying the following expression (3) can be obtained for each camera at the position of each correction pattern.

라 둔다.The matrix P, which is the product of the matrix of M and N, is called the projection matrix. This projection matrix can be obtained by using the DLT (Direct Linear Transform) method of the computer vision field. Now, let us consider the projection matrix obtained for the i-th camera and the j-th pattern

I will.

라 두고 이를 부투영행렬라 한다. 이 행렬의 행렬식이 1이 되도록 다음의 [수학식 4]와 같은 상수

를 곱한다.The 3x3 portion of the projection matrix

Which is called the inferior projection matrix. And the matrix equation of this matrix is 1, the following equation (4)

Lt; / RTI >

Subsequently, this sub-projection matrix is added in the horizontal and vertical directions as shown in the following equation (5), and this matrix is called a sub-measurement matrix.

Here, the number of cameras is m and the number of positions of the pattern is n. If the correction pattern of the specific position j is not seen at the specific camera position i, it can be obtained by the following equation (6).

,

,

는 각각 구할 수 있다고 가정한다.here

,

,

Respectively.

The sub-measurement matrix should have a rank of 3 according to the following expression (7).

Where ai, i = 1 ... m corresponds to an arbitrary constant. However, in the measurement of the reference point in the image, the noise can not be exactly 3 due to the noise, and it is optimized as follows. Assuming that SVD (Singular Value Decomposition) of the sub-measurement matrix is performed and the result of this SVD is as shown in the following equation (8)

The matrix with the rank 3 closest to this matrix can be obtained by the following equation (9).

이며

와

는 각각

와

의 앞 3열만을 남김 행렬이다. 이때 이와 같이 부측정행렬을 두 매트릭스의 곱으로 표현 할 때 이러한 형태는 다음의 [수학식 10]과 같이 임의의 행렬 T의 존재에 의해 무수히 많이 존재할 수 있다.here

And

Wow

Respectively

Wow

The first three columns are left. In this case, when the sub-measurement matrix is expressed by the product of two matrices, this form can exist in a large number by the presence of an arbitrary matrix T as shown in the following Equation (10).

의 수학식에 의해 구할 수 있다. 여기서 Rw는 임의의 월드좌표계의 설정과 관련되므로 임의로 단위행렬도 두어도 무방하다. 이제

라 두고

라 두면 Ki와 Ri는

의 RQ-decomposition으로 구할 수 있고

는

에 가장 가까운 회전행렬을 구함으로써 계산할 수 있다.However, this matrix T

Can be obtained by the following equation. Here, since Rw is related to the setting of any world coordinate system, an arbitrary unit matrix may be set. now

La

Ki and Ri

Can be obtained by RQ-decomposition of

The

Can be calculated.

를 투영행렬

의 4번째 열이라고 하면 다음의 [수학식 11]에서와 같은 관계식이 구해진다.At this time,

A projection matrix

The following relation is obtained as in the following equation (11).

로 구할 수 있다. 이 관계식을 통해

및

도 구 할 수 있다. 이로써 원하는 모든 인자를 다 구하였다. 이제 이러한 대수적 관계에 의해 구한 결과를 기하학적 최적화를 통해 보다 더 정밀하게 인자를 구한다. 우선 최적화 대상이 되는 비용함수는 다음의 [수학식 12]와 같이 재 투영 오차로 정한다.At this time

. Through this relationship

And

You can do it. All the desired parameters were obtained. Now, the geometric optimization of the results obtained by this algebraic relationship yields more precise parameters. First, the cost function to be optimized is determined by the re-projection error as shown in the following equation (12).

는 i번째 카메라에서 j번째 위치의 보정패턴을 볼 때, 보정패턴상에 보이는 기준점의 인덱스에 해당한다.

는 이미지 상에서 잠음을 내포한 채 측정한 기준점의 좌표이고,

는 주어진 인자에 의해 계산된 보정패턴상의 특정 기준점의 재투영 좌표이다. 대수적 관계에 의해 구한 결과를 초기값으로 하고, 첫번째 보정패턴의 위치를 상수로 고정시키고, 이러한 비용함수에 대해 Levenberg-Marquardt 방식의 최적화 과정을 수행하면 보다 정밀한 결과를 얻게 된다.here

Corresponds to the index of the reference point on the correction pattern when the correction pattern of the j-th position is viewed in the i-th camera.

Is the coordinates of the reference point measured with the latent sound on the image,

Is the re-projection coordinate of a specific reference point on the correction pattern calculated by the given factor. The result obtained by the algebraic relation is set as the initial value, the position of the first correction pattern is fixed as a constant, and the optimization process of the Levenberg-Marquardt method is performed on the cost function to obtain more accurate results.

However, there may be a slight error in the fabrication of the correction pattern. This error is mostly caused by the position error of each plane forming the correction pattern. The method of correcting this error is as follows.

First, as shown in Fig. 7, a rotation matrix Rk indicating a position of each plane k with respect to reference coordinates attached to the correction pattern and a position vector tk are defined. You can then define a cost function that includes these factors. Rk, tk can be obtained by optimizing the Levenberg-Marquardt method with the initial values of these factors as the unit matrix and the zero vector, respectively. In this case, if the lens is further distorted by the lens, the lens distortion factor can be obtained by adding the lens distortion factor to the above cost function, setting the initial value of this factor to 0, and performing the same optimization process.

As described above, in the present invention, in the camera network correction using the small correction pattern, a small correction pattern is provided around the object to be reconstructed in three dimensions, and a plurality It is possible to more easily perform the camera correction of the camera network disposed around the object in order to perform three-dimensional reconstruction of an arbitrary object and a person using the matrix value generated from the camera of the camera.

In addition, when the correction of the camera network is performed through the present invention, it is not necessary to use a large-sized correction pattern even when the object to be photographed is large, so that it is possible to reduce the cost and labor for maintenance, It is possible to overcome the limitations of accuracy and cost which are a problem in manufacturing the system, thereby contributing to reduction of the system construction cost and improvement of the work efficiency of the operator.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should not be limited by the described embodiments but should be defined by the appended claims.

100: sub-projection matrix generation unit 102: sub-measurement matrix generation unit
104: Optimization unit 106:

Claims (8)

Generating a sub-projection matrix of a part of the projection matrix by receiving a projection matrix generated from each camera with respect to a plurality of correction patterns;
Generating a sub-measurement matrix by vertically connecting the sub-projection matrices;
Performing singular value decomposition (SVD) on the sub-measure matrix and transforming the sub-measure matrix into a matrix of rank 3 based on the result of the SVD;
Performing the SVD again on the transformed sub-measurement matrix,
Performing a network correction of the camera by extracting a rotation value of the correction pattern and an internal factor and a rotation value of the camera according to a result of the SVD;
Lt; / RTI >
The correction pattern
Wherein the camera network correction method has a three-dimensional structure and is capable of observing at least six reference points that are not in one plane even when viewed from an arbitrary direction.
The method according to claim 1,
Wherein the sub-
And generating a predetermined 3X3 portion in front of the projection matrix.
The method according to claim 1,
Wherein the sub-
A method for calibrating a camera network as many as a number of cameras for a calibration pattern.
The method according to claim 1,
In the performing of the correction,
Calculating a translation value of the camera and a translation value of the correction pattern using the rotation value of the extracted correction pattern, the internal factor of the camera, the rotation value, and the projection matrix.
A sub-projection matrix generator for receiving a projection matrix generated from each camera for a plurality of correction patterns and generating a sub-projection matrix of the projection matrix;
A sub-metric matrix generator for vertically connecting the sub-projection matrices generated from the sub-projection matrix generator to generate a sub-metric matrix;
An optimization unit that performs singular value decomposition (SVD) on the sub-measurement matrix and transforms the sub-metric matrix into a matrix having rank 3 based on the result of the SVD;
Performing a SVD again on the transformed sub-measurement matrix, extracting an internal factor and a rotation value of the camera and a rotation value of the correction pattern,
Lt; / RTI >
The correction pattern
Wherein the camera network correction device has a three-dimensional structure and can observe at least six reference points that are not in one plane even when viewed from an arbitrary direction.
6. The method of claim 5,
Wherein the sub-
And generating a predetermined 3X3 portion in front of the projection matrix as a sub-projection matrix.
6. The method of claim 5,
Wherein the sub-
A camera network correction device as many as the number of cameras for one correction pattern.
6. The method of claim 5,
Wherein,
Wherein the translational movement value of the camera and the translational movement value of the correction pattern are calculated using the rotation value of the correction pattern, the internal factor of the camera, the rotation value, and the projection matrix.

Images