KR101985287B1 - Correcting device for color camera and thermal camera - Google Patents

Abstract

The present invention relates to an apparatus for correcting a thermal image camera and a color camera, which extracts a reference point of images of the thermal image camera and the color camera based on a jig simultaneously providing four heat sources and four light sources to perform correction. According to the present invention, the apparatus using a jig providing a reference point for two images to correct a thermal image camera (200) and a color camera (300) comprises: a correction jig (100) having four corners and provided with a circular lamp (110) for simultaneously providing a heat source and a light source to each corner; and a camera correction unit (400) receiving a thermal image and a rear image from the thermal image and color cameras (200, 300) photographing the correction jig (100), analyzing an image of the lamp (110) of the reference jig (100), which becomes a reference point of the two images, to estimate a correction coefficient of the thermal image and color cameras (200, 300), and correcting the cameras based on the estimated correction coefficient. Accordingly, even without using a conventional complicate correction jig, the thermal image and color cameras can be simply and quickly corrected.

Description

CORRECTING DEVICE FOR COLOR CAMERA AND THERMAL CAMERA < RTI ID = 0.0 >

The present invention relates to a thermal camera and a correction method for a color camera. More particularly, the present invention relates to a thermal camera and a method of correcting a color camera using a jig that simultaneously applies a heat source and a light source at four points, And a correction device for a color camera.

As the image processing technology develops, it is deviated from the simple utilization of the image obtained through the conventional one camera. Recently, a method of producing stereoscopic stereo image by photographing the object using two cameras and matching the photographed images I use it a lot.

In particular, after shooting the same object using a general color camera and a thermal camera, the captured images are matched to form a single image. In this case, when matching images captured through a heterogeneous camera, camera correction is necessary. In the past, chess board chessboards, in which black and white squares were staggered for camera calibration, were mainly used. In the case of this chess board, black pattern corners can be extracted through a general color camera. However, There is a problem that it is difficult to accurately extract the corners because the image of the chess board to be photographed is unclear.

In accordance with this problem, the present applicant has proposed a registration jig 10-1426314 entitled " jig for image matching between a general color camera and a thermal image camera ", which is comprised of a white board having a square or circular hole at regular intervals, A jig including a black inner plate which is detachably coupled to the back surface of the white background board and represents a hole formed in the white background board in a black color enables accurate images to be taken from both the general color camera and the thermal image camera, So that the matching reference position can be accurately extracted.

However, in the above patent, the white board and the black inner board must be used in combination, and the image of the thermal camera must be extracted due to the temperature difference between the white board and the black inner board.

Korean Registered Patent No. 10-1426314 (Registered on July 29, 2014)

An object of the present invention is to extract and correct a reference point of an infrared camera and a color camera image through a jig that simultaneously applies a heat source and a light source at four points And to provide a correction device for a color camera.

According to an aspect of the present invention, there is provided a camera correcting apparatus for correcting a thermal camera and a color camera using a jig that provides reference points of two images, A correction jig provided with a circular illumination light for simultaneously providing a heat source and a light source at respective corners; A thermal imaging camera for photographing the correction jig and a thermal image and a real image from a color camera are received and the correction illumination coefficients of the thermal imaging camera and the color camera are estimated by analyzing the illumination light image of the correction jig as the reference point of the two images, And a camera corrector for correcting the camera through the estimated correction coefficient.

Here, it is preferable that the correction jig has any one of H, X, and a shape so as to have four corners, and circular illumination lights are provided at each corner.

A corner point position measurement module for analyzing the illumination image of the correction jig obtained through the thermal camera and the color camera to extract the four corner positions of the correction jig, A correction coefficient estimation module for estimating correction coefficients of the two cameras using the four-point corner position information of the correction jig, and a correction unit for performing correction of the thermal imaging camera and the color camera through the camera correction coefficient estimated through the correction coefficient estimation module A camera correction module is provided.

The corner point position measurement module detects an outline image of each of the four illumination image images photographed through the thermal camera and the color camera, estimates the circle center point and the radius parameter of the detected outline image, The center point coordinates of the matched circles of the image are detected to extract the four corner corner positions of the correction jig.

In particular, the corner point position measurement module converts an illumination image into a binarized image based on a threshold value, converts the converted image into an outline image through a canny algorithm to detect an outline image, In order to obtain the center point and the radius of a circle, the equation of the circle with three or more coefficients is estimated, the estimated circle equation is differentiated by each coefficient, and then transformed into a matrix form. And the center point of the four circles for the thermal image and the real image are obtained through the estimated center point and the radial variable of the circle and the center point of the center point of the matched circle of the two camera images is obtained through the sort algorithm It is desirable to find the order and extract the four-point corner coordinates.

In addition, the correction coefficient estimation module estimates the correction coefficients of the thermal imaging camera and the color camera by performing Perspective Transform (perspective transformation) on the four-point corner position information detected through the corner point position measurement module.

The correction coefficient estimation module 420 calculates the correction coefficient by using the following Perspective Transform equation

With respect to x ', y'

, ≪ / RTI >

, And the following projection matrix

It is preferable to obtain the variables a, b, c, d, e, f, g, and h to estimate the correction coefficients of the thermal imaging camera and the color camera.

According to the present invention, by extracting and correcting the reference points of the thermal imaging camera and the color camera image using a jig that simultaneously applies the heat source and the light source at the four points, And color camera can be corrected easily and quickly.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a general block diagram of a thermal imaging camera and a color camera correction apparatus according to the present invention;
2 is a front perspective view of a correction jig according to the present invention,
FIG. 3 is a perspective view of a fixing jig manufactured according to the present invention,
4 is a view showing an example of a thermal image obtained by photographing a correction jig manufactured by the present invention with a thermal imaging camera,
5 is a view showing an example of a real image of a correction jig manufactured according to the present invention,
6 is a flowchart illustrating a process of calibrating a camera by calculating correction coefficients of a thermal camera and a color camera according to the present invention,
7 is an example of determining the circular center point sequence of a thermal image and a real image according to the present invention,
8 shows an example of a temperature distribution of a thermal image on a general room image after correction of a thermal camera and a color camera according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing the overall configuration of a thermal imaging camera and a color camera correcting apparatus according to an embodiment of the present invention; FIG.

As shown in FIG. 1, the camera correcting apparatus according to the present invention includes a correcting jig 100 manufactured to simultaneously apply a heat source and a light source at four points, a thermal image (not shown) for photographing the image of the correcting jig 100, The image of the correction jig 100 photographed through the camera 200 and the color camera 300 and the thermal camera 200 and the color camera 300 is analyzed to estimate correction coefficients of the two cameras, And a camera correcting unit 400 for correcting the camera.

In the present invention, the reference points of the images of the thermal camera 200 and the color camera 300 are extracted using the correction jig 100 for simultaneously applying the four heat sources and the light sources, 3 is a front perspective view of a correction jig according to an embodiment of the present invention, and Fig. 3 shows an example of a realization of the manufactured correction jig.

The correction jig 100 applied to the present invention may have a shape having four corners regardless of the material or shape. As shown in FIGS. 2 and 3, in the embodiment of the present invention, Are formed in an H shape having four corners.

A round illumination light 110 is provided at each of the four corners of the correction jig 100 having the H shape. The circular illumination light 110 illuminates the color camera 300, And circular thermal distribution is expressed when the thermal imaging camera 200 is photographed.

The thermal imaging camera 200 and the color camera 300 respectively acquire a thermal image and a real image by capturing an image of the correction jig 100. In the embodiment of the present invention, The camera 300 includes a hybrid camera device arranged side by side and capturing an image of the correction jig 100.

FIG. 4 shows an example of a thermal image of a correction jig fabricated according to an embodiment of the present invention. The thermal image of the illumination lamp 110 of the correction jig 100 taken by the thermal imaging camera 200 It can be seen that it is expressed in a red circular form.

5 shows an example of a real image of a correction jig taken by a color camera. It is assumed that a real image of the illumination lamp 110 of the correction jig 100 photographed by the color camera 300 is represented by a bright white circle Able to know.

The camera correcting unit 400 analyzes the image of the illumination lamp 110 of the correction jig 100 photographed through the thermal camera 200 and the color camera 300 and outputs the image to the thermal camera 200 and the color camera 300. [ The camera correcting unit 400 is provided with a corner point position measuring module 410, a correction coefficient estimating module 420 and a camera correcting module 430.

The corner point position measurement module 410 analyzes the image of the illumination lamp 110 of the correction jig 100 obtained through the thermal camera 200 and the color camera 300 and outputs the image of the illumination lamp 110 of the correction jig 100 The corner point position measurement module 410 finds each of the outline images of the image of the illumination lamp 110 photographed through the two cameras. The corner point position measurement module 410 detects the center of the outline image, The radial parameters are respectively estimated, and then the coordinates of the center point of the matched circle of the images of the two cameras are found, and the corner positions of the four corners of the correction jig 100 are grasped.

The correction coefficient estimation module 420 is a program module for estimating correction coefficients of two cameras using four-point corner position information of the correction jig 100 detected through the corner point position measurement module 410, The estimation module 420 estimates the correction coefficients of the two cameras by performing Perspective Transform (perspective transformation) on the four corner positions of the correction jig 100.

The camera correction module 430 is a program module that performs correction of the thermal imaging camera 200 and the color camera 300 through the camera correction coefficient estimated through the correction coefficient estimation module 420.

Hereinafter, the process of correcting the two cameras through the thermal imaging camera and the color camera correction apparatus will be described.

FIG. 6 is a flowchart illustrating a process of correcting a camera by calculating correction coefficients of a thermal camera and a color camera according to an embodiment of the present invention.

The thermal camera 200 and the color camera 300 first take an image of the H type correction jig 100 and determine the position of the illumination light 110 located at the four corners of the correction jig 100 A thermal image, and a real image, and then transmits the thermal image and the real image to the camera correcting unit 400. The thermal image and the real image of the illumination lamp 110 obtained through the thermal imaging camera 200 and the color camera 300 are expressed as shown in Figs. 3 and 4 described above.

Step S110: When the thermal image and the real image of the illumination lamp 110 are acquired through the thermal imaging camera 200 and the color camera 300, the corner point position measurement module 410 of the camera correction unit 400 receives the thermal image ) And extracts the outline image of the image. In the embodiment of the present invention, the corner point position measurement module 410 converts each of the thermal images of the obtained illumination light 110 and the measured threshold values of the real image into binarized images, And converted into an outline image.

Step S120: When the outline image of the image of the illumination lamp 110 is extracted through the above process, the corner point position measurement module 410 extracts the circle center point and the radius parameter of the outline image, respectively. In the embodiment of the present invention, the corner point position measurement module 410 estimates a circle equation with a minimum of three coefficients to find a circle of the outline line image. The circle equation model for the least squares method is expressed by the following mathematical expression Equation 1 is as follows.

Here, X _i , Y _i is a set of circular points in the contour image and requires at least three points in order to estimate the three coefficients A, B, and C. For the least squares method, the following equation (2) is obtained by differentiating the equation (1) by the coefficients of A, B, and C, respectively.

In order to estimate the coefficient of Equation (2), it is converted into a matrix form as shown in Equation (3).

Here, N is the number of points in the circle. If the coefficients A, B, and C are estimated through the least squares method from the above equation, the center point and the radius variable of the circle as shown in Equation (4) can be estimated.

Step S130: Through the above process, the center points of the four circles matched in each image are obtained. The center point of the circle thus obtained is four in the thermal image and four in the real image, and the corner point position measurement module 410 sequentially orders the coordinates to obtain their coordinates.

FIG. 7 shows an example of determining the order of circular center points of a thermal image and a real image. In the present invention, a method of determining the corner order of four points uses a sort algorithm to prioritize the coordinates of the upper left corner among the four- Then, the coordinates of the center point of the matched circle of the images of the two cameras are found by selecting the coordinates of the lower right and upper right.

Through the above process, the corner coordinates of the four points are found and the positions of the four corner points are extracted.

Step S140: When the corner positions for the four points are extracted, the correction coefficient estimation module 420 extracts the correction coefficients of the thermal imaging camera 200 and the color camera 300 through the four-point corner position information.

In the embodiment of the present invention, the correction coefficient estimation module 420 extracts correction coefficients of each camera by performing Perspective Transform (perspective transformation). The formula for the basic perspective transformation is as follows.

여기에서, (x, y, 1)은 입력 영상의 2차원 좌표를 의미하고, (wx', wy', w)는 (x, y, 1)에 대한 동차 좌표계(homogenious coordinate)에서의 영상 좌표(w는 1의 값을 가진다)를 나타내며, a, b, c, d, e, f. g, h는 Perspective Transform(원근 변환) 행렬을 의미한다.

(Wx ', wy', w) represents the coordinates of the image in the homogenious coordinate system for (x, y, 1) (w has a value of 1), and a, b, c, d, e, f. g and h represent a perspective transformation matrix.

Since Equation (5) uses a homogenious coordinate system, the equation for x 'and y' can be changed as shown in Equation (6).

The equation (6) can be summarized into the following equation (7).

(X, y) and the corresponding (x ', y') pairs are used as the correction coefficients corresponding to the correction coefficients a, b, c, d, e, f, If only four are known, a projection matrix can be obtained. The matrix for obtaining this is represented by the following equation (8).

Here, the inverse matrix of the 8 × 8 matrix is obtained, and then the matrix at the rear is multiplied to find the variables a, b, c, d, e, f, g and h.

Through the above process, the projection matrix is obtained to calculate the correction coefficients of the thermal imaging camera 200 and the color camera 300, respectively.

Step S150: When the correction coefficients of the thermal imager 200 and the color camera 300 are calculated through the above process, the camera correcting module 430 corrects the correction coefficients of the thermal imager 200 and the color camera 300, (300). FIG. 8 shows an example of a temperature distribution of a thermal image on a normal room image after correction of the thermal camera 200 and the color camera 300. FIG.

As described above, according to the present invention, the correction jig 100 for simultaneously applying heat sources and light sources at four points via the thermal imaging camera 200 and the color camera 300 is photographed, and the position coordinates The perspectives of the camera 200 and the color camera 300 can be corrected simply and quickly by calculating the correction coefficients of each camera by performing Perspective Transform.

Meanwhile, in the above-described embodiment, it has been described that the H type correction jig 100 is applied. However, the correction jig 100 can be changed to another form if only four corners can be formed. For example, the correction jig 100 may be formed in various shapes such as an X-shape, a sagittal shape, a machine shape, and the like, having four corners, and performs the same function by installing the illumination lamp 110 at the corners of four points can do.

As described above, the present invention is not limited to the above-described embodiments, and various changes and modifications within the scope of the technical idea of the present invention and equivalents of the claims defined below may be made by a person having ordinary skill in the art to which the present invention belongs. And modifications may be made.

100: Correction jig 110: illumination light
200: Thermal camera 300: Color camera
400: camera correction unit 410: corner point position measurement module
420: correction coefficient estimation module 430: camera correction module

Claims (7)

A correction jig 100 provided with a circular illumination lamp 110 at a corner of four points; The thermal image camera 200 receives the thermal image and the real image from the thermal camera 200 and the color camera 300 for photographing the correction jig 100 and analyzes the image of the illumination lamp 110 of the correction jig 100, And a camera corrector (400) for estimating correction coefficients of the thermal imaging camera (200) and the color camera (300) and correcting the camera through the estimated correction coefficients, as,
The image of the illumination lamp 110 of the correction jig 100 obtained through the thermal camera 200 and the color camera 300 is analyzed by the camera correction unit 400 so that the four corners of the correction jig 100 And a correction coefficient estimating unit 420 for estimating correction coefficients of the two cameras using the four-point corner position information of the correction jig 100 extracted through the corner point position measuring module 410, Module 420 and a camera correction module 430 for performing correction of the thermal imaging camera 200 and the color camera 300 through the camera correction coefficient estimated through the correction coefficient estimation module 420,
The corner point position measurement module 410 detects the outline images of the images of the four illumination lights 110 photographed through the thermal camera 200 and the color camera 300 and detects the outline image of the detected outline image And then calculates the center point coordinates of the matched circles of the two camera images and extracts the four-corner positions of the correction jig (100).
The method according to claim 1,
Wherein the correction jig (100) has a shape of H, X, or a shape so as to have four corners, and a circular illumination lamp (110) is provided at each corner, Device.
delete delete The method according to claim 1,
The corner point position measurement module 410
The illumination light 110 is converted into a binarized image based on a threshold value, and then the converted image is converted into an outline image through a canny algorithm to detect an outline image,
In order to obtain the center point and the radius of the circle in the detected outline image, a circle equation with three or more coefficients is estimated, the estimated circle equation is differentiated by each coefficient, and then transformed into a matrix form. By estimating each coefficient, we can estimate the center point and the radius of the circle,
We obtain the center points of four circles for the thermal image and the real image through the estimated center point and the radial parameter, find the order of the center coordinates of the matched circles of the two camera images through the sort algorithm, And the correction device of the color camera.
The method according to claim 1,
The correction coefficient estimation module 420
A perspective transformation is performed on the four-point corner position information detected through the corner point position measurement module 410 to estimate the correction coefficients of the thermal imaging camera 200 and the color camera 300, respectively A correction device of an infrared camera and a color camera.
The method according to claim 6,
The correction coefficient estimation module 420
The following Perspective Transform equation

(Wx ', wy', w) represents the two-dimensional coordinate of the input image, and (x, y, A, b, c, d, e, f, g, h represent the Perspective Transform matrix (where w represents a value of 1)
With respect to x ', y'

, ≪ / RTI >

, And the following projection matrix

Wherein the correcting coefficients of the thermal imaging camera (200) and the color camera (300) are estimated by obtaining the variables a, b, c, d, e, f, .

Images