KR102385461B1 - Apparatus of thermal pattern, method and apparatus for thermal camera calibration - Google Patents

Abstract

A calibration system includes: a hot wire pattern device including a grid-pattern hot wire whose power supply and power off are controlled by a processor; and a computing device that pre-processes the thermal image obtained by photographing the grid-pattern hot wire to extract a horizontal straight line image and a vertical straight line image, and fits linear regions included in the horizontal straight line image and the vertical straight line image with curve functions, extracts the intersection points of the curve functions as the feature points of the grid-pattern hot wire, and calibrates a camera that captured the thermal image using the feature points.

Description

Heat ray pattern device, method and device for calibrating a thermal imaging camera

This disclosure relates to thermal imaging camera calibration.

Camera calibration is an important procedure that must be preceded in order to obtain an image with accurate geometric information, and a checkerboard is mainly used.

Meanwhile, a thermal camera measures infrared thermal energy radiated from an object, not visible light, and generates a thermal image indicating the temperature of the object. The thermal imaging camera must also be calibrated first, but due to the characteristics of the thermal imaging camera that needs to measure thermal energy, the checkerboard used in the visible light camera cannot be used as it is.

Referring to FIG. 1 , as a method of calibrating a thermal imaging camera, a method of manufacturing a PCB substrate with a copper lattice pattern, applying heat to the PCB substrate through a hot air blower, and measuring the copper lattice pattern having an increased temperature is known. An intersection point where lines intersect in the grid pattern is extracted as a feature point for calibration, and the calibration accuracy is determined according to the positional accuracy of the feature point. However, since it is a method of directly applying heat to the PCB substrate, the temperature of the copper grid pattern is not uniform as a whole, and it is difficult to keep the temperature constant during calibration. Also, since the ambient temperature of the copper lattice pattern increases, copper lines or intersections appear widely spread in the thermal image, making it difficult to accurately extract the position of the feature point, and consequently, the calibration accuracy is lowered.

An object of the present disclosure is to provide a hot wire pattern device for uniformly and constantly controlling the temperature of a grid pattern.

The present disclosure provides a method for measuring a hot ray pattern of a hot ray pattern device and extracting feature points from the measured thermal image, and a method and apparatus for calibrating a thermal imager using the same.

According to the present disclosure, a horizontal straight line image and a vertical straight line image are extracted by preprocessing a thermal image, horizontal and vertical straight lines included in each image are fitted with a curve function to extract horizontal hot rays and vertical hot rays, and a grid pattern It is to provide a method of extracting a feature point that is an intersection of hot wires.

As a hot wire pattern device according to an embodiment, a flat panel, horizontal hot wires and vertical hot wires are disposed on the flat panel to form a grid pattern hot wire for camera calibration, and control power supply to the grid pattern hot wire includes a processor that In the flat panel, grooves corresponding to the grid pattern are formed on the front surface, and holes having a size through which a hot wire passes are formed at intersections of the grid patterns. The grid pattern hot wire is inserted into the groove, and at an intersection point where each horizontal hot wire and each vertical hot wire intersect, one hot wire is attached to the front surface of the flat panel, and the other hot wire is a hole formed in the flat panel. It has a woven structure that passes through the rear surface of the flat panel and comes out to the front surface.

The processor may control power supply and power cut-off to the grid pattern hot wire through a control cycle consisting of a power supply time and a power cut-off time.

The flat panel may be made of a bakelite material.

As a method of operating a computing device according to another embodiment, obtaining a thermal image obtained by photographing a grid pattern hot wire, pre-processing the thermal image to extract a horizontal linear image and a vertical linear image, the horizontal linear image and extracting horizontal hot wires and vertical hot wires by fitting straight regions included in the vertical straight image with curve functions, and extracting the intersection points of the horizontal hot wires and the horizontal hot wires as feature points for camera calibration include

The extracting of the horizontal linear image and the vertical linear image may include generating an RGB thermal image obtained by converting thermal energy of the thermal image into an RGB value, and converting a G-channel image of the RGB thermal image to a G-channel threshold Generating a binarized grid pattern image based on converting an image into the horizontal straight line image of the original size; and binarizing the vertical straight line emphasized image into vertical linear regions and a background, and converting the binarized image into the vertical straight line image of the original size.

The generating of the horizontal straight line-emphasized image and the vertical straight line-emphasized image includes reducing the grid pattern image on a horizontal axis and expanding it on a vertical axis to generate the horizontal straight line emphasized image emphasizing a horizontal straight line component, and converting the grid pattern image to a horizontal axis. The vertical straight line emphasized image in which the vertical straight line component is emphasized may be generated by enlarging and reducing the vertical axis.

The generating of the horizontal straight line-emphasized image and the vertical straight line-emphasized image includes projecting and transforming horizontal straight lines and vertical straight lines included in the grid pattern image to be parallel to the X-axis and Y-axis of the image, and in the projection-converted grid pattern image, the A horizontal straight line emphasized image and a vertical straight line emphasized image may be generated.

The converting to the horizontal straight line image includes binarizing the horizontal straight line emphasized image into horizontal straight regions and a background, performing inverse projection transformation on the binarized image for the projection transformation, and converting the inverse projection transformed image to the horizontal straight line region and the background of the original size. It can be converted to a straight image. The step of converting the vertical straight line image includes binarizing the vertical straight line emphasized image into vertical straight lines regions and a background, performing inverse projection transformation on the binarized image for the projection transformation, and converting the inverse projection transformed image to the original size of the vertical image. It can be converted to a straight image.

The operating method may further include calibrating the thermal imaging camera using the feature points.

In the grid pattern heating wire, horizontal heating wires and vertical heating wires are disposed on a flat panel, and power supply and power off are controlled by a processor to maintain a constant temperature.

As a calibration system according to another embodiment, a hot wire pattern device including a grid pattern hot wire in which power supply and power off are controlled by a processor, and a thermal image obtained by photographing the grid pattern hot wire are pre-processed to obtain a horizontal straight line image and a vertical line image Extracting a straight line image, fitting straight line regions included in the horizontal straight line image and the vertical straight line image with curve functions, extracting intersection points of the curve functions as feature points of the grid pattern hot wire, and using the feature points and a computing device for calibrating a camera that has captured the thermal image.

The hot wire pattern device includes a flat panel in which the grid pattern hot wire, grooves corresponding to the grid pattern are formed on the front surface, and holes of a size through which the hot wire passes are formed at intersections of the grid pattern, and the grid pattern hot wire It may include a processor for controlling the power supply.

The grid pattern hot wire is inserted into the groove, and at an intersection point where each horizontal hot wire and each vertical hot wire intersect, one hot wire is attached to the front surface of the flat panel, and the other hot wire is a hole formed in the flat panel. It may have a woven structure that passes through the rear surface of the flat panel and comes out to the front surface.

The computing device extracts a grid pattern image based on a G-channel threshold value from an RGB thermal image obtained by converting the thermal energy of the thermal image into an RGB value, and enlarges or reduces horizontal and vertical axes of the grid pattern image. After generating a horizontal straight line-emphasized image in which a horizontal straight line component is emphasized and a vertical straight line emphasis image in which a vertical straight line component is emphasized, the horizontal straight line emphasis image and the vertical straight line emphasis image are binarized into a straight line component and a background, the binarized image is converted to the original size , to extract the horizontal straight line image and the vertical straight line image.

The computing device converts the grid pattern extracted from the thermal image to be parallel to the X-axis and Y-axis of the image, extracts a horizontal straight line component and a vertical straight line component, and projects the horizontal straight line component and the vertical straight line component The horizontal linear image and the vertical linear image may be extracted by performing inverse projection transformation on the transformation.

The temperature of the grid pattern may be uniformly and constantly maintained through the hot wire pattern device of the present disclosure, thereby increasing the accuracy of thermal imaging camera calibration.

According to the present disclosure, the linear component of the grid pattern hot wire can be extracted without distortion regardless of the direction in which the grid pattern hot wire is photographed, and the horizontal hot wire and the vertical hot wire can be fitted with a curve function, so that the intersection of the grid pattern hot wire is Feature points can be accurately extracted.

According to the present disclosure, it is possible to accurately extract positions of feature points from a thermal image obtained by photographing a grid pattern hot wire, thereby increasing the accuracy of thermal imaging camera calibration.

1 is an example of a copper grid pattern for calibrating a conventional thermal imaging camera.
2 is a perspective view of a hot wire pattern device according to an embodiment.
3 is a view exemplarily illustrating hot wires disposed in a grid pattern in a hot wire pattern apparatus according to an embodiment.
4 is a view showing a front surface and a rear surface of a hot wire pattern device manufactured according to an embodiment.
5 is a flowchart of a method of calibrating a thermal imager by extracting feature points of a thermal image according to an exemplary embodiment.
6 is a view for explaining extraction of feature points of a thermal image according to an exemplary embodiment.
7 is an example of curve function fitting according to an embodiment.
8 is a block diagram of a computing device according to an embodiment.

Hereinafter, with reference to the accompanying drawings, the embodiments of the present disclosure will be described in detail so that those of ordinary skill in the art to which the present disclosure pertains can easily implement them. However, the present disclosure may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present disclosure in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as “…unit”, “…group”, and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. there is.

The calibration system includes the hot ray pattern device 100 and the computing device 200 , and a structure of the hot ray pattern device 100 and an operating method of the computing device 200 will be described in detail below.

FIG. 2 is a perspective view of a hot wire pattern device according to an embodiment, FIG. 3 is a view exemplarily illustrating hot wires disposed in a grid pattern in a hot wire pattern device according to an embodiment, and FIG. 4 is a hot wire pattern device according to an embodiment It is a diagram showing the front and rear surfaces of the manufactured hot wire pattern device.

Referring to FIG. 2 , the hot wire pattern device 100 includes a flat panel 110 , a grid pattern hot wire 130 attached to the flat panel 110 , and a frame 150 supporting the flat panel 110 . . Wheels capable of moving and rotating may be attached to the frame 150 . The wheels may include a motion lock, which prevents micro-movement. The heating wire pattern apparatus 100 may further include a power supply unit 170 for supplying power to the heating wire 130 , and a processor 190 for controlling the temperature of the heating wire 130 by controlling the power supply unit 170 .

The flat panel 110 is made of a material that is strong in heat and has low thermal conductivity. For example, the flat panel 110 may be made of a bakelite material. Since the flat panel 110 has to maintain planarity, it may be manufactured to have a predetermined thickness (eg, 12T). The area of the flat panel 110 may be manufactured in various ways, and may be manufactured to have a predetermined area or more so that many feature points are extracted for camera calibration.

The flat panel 110 has a groove to which the grid pattern hot wire 130 is attached and a hole through which the hot wire passes.

In the grid pattern hot wire 130 , horizontal hot wires and vertical hot wires are disposed to form a uniform grid size for camera calibration. Horizontal hot wires may be connected to one hot wire to receive power, and vertical hot wires may also be connected to one hot wire to receive power. Alternatively, the horizontal heating wires and the vertical heating wires may be connected as one heating wire. Each of the horizontal heating wires and the vertical heating wires may be independently supplied with power. The diameter and heating temperature of the heating wire may be selected according to the calibration environment. For example, a hot wire having a diameter of 4 pi and capable of being heated to 65°C can be used.

The frame 150 is made of a material and a structure capable of supporting the flat panel 110 , and wheels may be attached thereto. For example, the frame 150 may be made of an aluminum profile frame.

The power supply unit 170 supplies power to heat the heating wire 130 . The power supply unit 170 may supply or cut off power to the hot wire under the control of the processor 190 .

The processor 190 controls the power supply unit 170 to maintain the grid pattern heating wire 130 at a constant temperature. The processor 190 minimizes the temperature change of the grid pattern heating wire 130 while acquiring a thermal image for calibration through automatic temperature control control.

According to an embodiment, the processor 190 may periodically control power supply and power cut-off through a control cycle including a power supply time t1 and a power cut-off time t2 . The power supply time t1 is a time period for heating the heating wire, and may be calculated according to a target temperature for calibration, an ambient temperature, and a type of the heating wire. The power cut-off time t2 is a time period for preventing the temperature of the heating wire from continuously increasing. For example, the power supply time t1 may be set to a value between about 45 to 60 seconds, and the power cut-off time t2 may be set to a value between about 10 to 20 seconds.

According to another embodiment, the processor 190 may receive the temperature of the grid pattern heating wire 130 and control the power supply unit 170 according to the current temperature.

If the horizontal heating wires or the vertical heating wires are grouped and power can be independently supplied for each group, the processor 190 controls the power supply of the group, and the number of horizontal heating wires and the number of vertical heating wires according to a calibration requirement , the grid size, etc. can be adjusted to vary the shape and size of the grid pattern hot wire.

The processor 190 may be connected to a wired/wireless communication module (not shown), and may control the power supply unit 170 through a control signal received from the wired/wireless communication module.

Referring to FIG. 3A , a grid pattern having a predetermined area or more may be formed so that many feature points are extracted for camera calibration. For example, horizontal hot wires and vertical hot wires may be arranged at regular intervals to form a grid pattern having a grid size of 50 mm in width and 50 mm in length.

The flat panel 110 is manufactured to have a predetermined area or more so that feature points are sufficiently extracted, for example, it may be manufactured to have a width of 850 mm and a length of 1150 mm. In this case, the number of horizontal hot wires used for calibration is greater than the number of vertical hot wires.

Referring to FIG. 3B , the heating wire may be attached to the flat panel 110 in the form of being inserted into grooves formed in a grid pattern. The groove may be formed to a depth (eg, 5 mm) into which the heating wire can be inserted. At this time, when the horizontal heating wire and the vertical heating wire intersect at the intersection of the grid pattern, the temperature of the intersection may rise and reach the limit temperature that the heating wire can withstand. Therefore, in order to prevent the horizontal heating wire and the vertical heating wire from coming into direct contact with each other at the intersection, two holes are drilled in the flat panel 110 around the intersection, and either the horizontal heating wire or the vertical heating wire passes through the rear surface of the flat panel 110 to the front. make it come out

Referring to FIG. 4A , the horizontal heating wire may be inserted in a straight line into the horizontal groove formed on the front surface of the flat panel 110 . The vertical heating wire is inserted into the vertical groove formed on the front surface of the flat panel 110, and at the intersection with the horizontal heating wire, it passes through the back hole through the hole drilled in the flat panel 110 and comes out to the front through another hole.

Referring to FIG. 4B , a plurality of holes are drilled in the flat panel 110 for forming a lattice pattern using a hot wire and for forming a bypass path at a lattice point.

Horizontal pattern hot wires may be connected to each other and vertical pattern hot wires may be connected through holes drilled in edges of the flat panel 110 .

Through the holes drilled around the intersection of the flat panel 110 , the vertical patterned hot wire may pass through the intersection by detouring to the rear so that the vertical patterned hot wire does not intersect with the horizontally patterned hot wire.

In the case of calibration using a black-and-white checkerboard, a corner extraction algorithm that extracts feature points based on the contrast ratio of the black-and-white checkerboard is used. However, unlike the black-and-white checkerboard, the grid pattern hot wire has a non-linear characteristic that a grid pattern is formed with hot wire of a certain thickness and increases with the temperature around the hot wire. It is difficult to know exactly.

Next, a new feature point extraction method of extracting a linear component by preprocessing the thermal image, fitting the linear component with a curve function, and extracting the feature point of the grid pattern hot wire will be described.

5 is a flowchart of a method of calibrating a thermal imaging camera by extracting characteristic points from a thermal image according to an embodiment, FIG. 6 is a diagram for explaining extraction of characteristic points from a thermal image according to an embodiment, and FIG. 7 is an embodiment An example of curve function fitting according to an example.

Referring to FIGS. 5 and 6 , the computing device 200 acquires a thermal image 300 obtained by photographing a grid pattern hot wire at a constant temperature ( S110 ). The thermal image 300 is an image obtained by photographing the grid pattern heating wire 130 in which a thermal imaging camera is attached to the heating wire pattern apparatus 100 and controlled to a predetermined temperature.

The computing device 200 extracts the region of interest 400 in which the grid pattern is located from the thermal image 300 ( S120 ). The computing device 200 may separate the grid pattern region and the background by binarizing the thermal image 300 into a background and a region of interest. To this end, the computing device 200 may generate an RGB thermal image obtained by converting thermal energy (temperature) of the thermal image into an RGB value. At this time, looking at the RGB values of the RGB thermal image, the green channel (G channel) value of the grid pattern region having a high temperature has a significantly lower value than the background. Accordingly, the computing device 200 may set the G-channel threshold and binarize the pixels of the thermal image to 0 or 1 based on the G-channel threshold.

The computing device 200 extracts a grid pattern image 510 from the G-channel image 500 of the RGB thermal image through the G-channel value of the region of interest ( S130 ). The computing device 200 may extract pixels that are less than or equal to a G-channel threshold (eg, 20) from the G-channel image 500 as hot rays of a linear component. The computing device 200 may reduce noise included in an image by using a median filter in order to reduce noise generated during image conversion and noise generated from a camera.

The computing device 200 generates a horizontal straight line emphasized image 600 and a vertical straight line emphasized image 610 from the grid pattern image 510 ( S140 ). The horizontal straight line emphasized image 600 and the vertical straight line emphasized image 610 may minimize linear distortion and increase the amount of information of the hot wire to be extracted. The horizontal straight line emphasis image 600 is an image in which the horizontal straight line component is emphasized by reducing the grid pattern image 510 on the horizontal axis and expanding it on the vertical axis. The vertical straight line emphasis image 610 is an image in which the vertical straight line component is emphasized by enlarging the grid pattern image 510 on the horizontal axis and reducing the vertical axis. Meanwhile, depending on the photographing direction and angle, straight lines in the grid pattern image 510 may not be parallel to the X and Y axes of the image. Therefore, before converting the horizontal straight line-emphasized image and the vertical straight line-emphasized image, the computing device 200 may convert the horizontal straight line and the vertical straight line included in the grid pattern image to be parallel to the X-axis and the Y-axis of the image. That is, the computing device 200 may perform a perspective transform on the grid pattern image into a front shot, just as the thermal imaging camera photographed the grid pattern hot wire from the front.

The computing device 200 binarizes the horizontal straight line component/vertical straight line component and the background in the horizontal straight line emphasized image 600/vertical straight line emphasized image 610 to generate the horizontal straight line image 601/vertical straight line image 611 . generated (S150). The computing device 200 detects a horizontal straight line area/vertical straight line area through image filtering, and binarizes it into a value distinguished from the background, and a horizontal straight line image 601 / vertical straight line image ( 611) can be created.

The computing device 200 scales the horizontal axis and the vertical axis of the horizontal linear image 601/vertical linear image 611 to the original size, and performs re-perspective transformation into the original image, so that the horizontal axis of the original size and direction is horizontal. A straight line image 700/vertical straight line image 710 is generated (S160). In the inverse projection transformation, by performing step S140 inversely, the horizontal and vertical axes of the image may be enlarged/reduced, and pixel values may be moved to the coordinate system of the grid pattern image.

The computing device 200 finds a horizontal hot wire and a vertical hot wire by fitting the horizontal straight image 700/vertical straight image 710 with a curve function (S170). The computing device 200 extracts a straight line area by grouping pixels (hot line pixels) corresponding to the hot line from the horizontal straight image/vertical straight image binarized into the background and the hot wire, and It is possible to find a curve function with a minimum distance and extract it as a horizontal hot wire/vertical hot wire.

The computing device 200 extracts the intersection points of the horizontal hot wire and the vertical hot wire as feature points ( S180 ). Since each horizontal hot wire and each vertical hot wire are expressed as a function in space, a contact point where the functions meet can be calculated, and the contact point becomes an intersection and a feature point of the hot wires.

The computing device 200 calibrates the thermal image camera by using the feature points of the thermal image ( S190 ). Since the calibration method for calculating the relationship between the camera internal parameter and the external parameter using the feature point is a known method, a detailed description thereof will be omitted.

As described above, the computing device 200 may pre-process the thermal image to extract a horizontal linear image and a vertical linear image, and fit the linear component with a curve function to accurately extract the feature points of the grid pattern heating wire. Accordingly, the computing device 200 may increase the calibration accuracy of the thermal imager by using the exact position of the feature point extracted from the thermal image.

Next, a method of extracting feature points of a thermal image will be described in detail.

The computing device 200 needs to detect the position of the grid pattern hot wire in the thermal image 300 . To this end, the computing device 200 may generate an RGB thermal image obtained by converting thermal energy (temperature) of the thermal image 300 into an RGB value. Thermal-RGB conversion may convert the Kelvin temperature into R, G, and B values as shown in Equation 1.

In the RGB thermal image converted through Equation 1, the temperature of 65 °C of the heating wire is converted to a value with a G channel value less than 20. Accordingly, the computing device 200 may extract the region of interest 310 in which the grid pattern is located by binarizing the region having the G-channel value of 20 or less as 1, otherwise, as 0 as shown in Equation (2).

The computing device 200 extracts the G-channel image 500 from the region of interest of the RGB thermal image, and extracts the grid pattern image 510 through the G-channel value. The computing device 200, for example, as in Equation 3, receives a G-channel image I _G , extracts an area equal to or less than a G-channel threshold (eg, 20) as a straight line area, and obtains a grid pattern image 510 . ) can be obtained.

The computing device 200 converts the horizontal straight line area and the vertical straight line area included in the grid pattern image 510 to be parallel to the X-axis and the Y-axis of the image before converting into the horizontal straight line-emphasized image and the vertical straight line-emphasized image ( perspective transform). To this end, the computing device 200 may extract intersection points of some horizontal and vertical straight lines of the grid pattern image 510 , and use these as initial feature points serving as a reference for projection transformation. The computing device 200 may project the position (x, y) included in the ROI of the original image to the position (x', y') on the XY plane by using a projection transformation equation such as Equation (4). In Equation 4, [H] is a projection transformation matrix, and the value of the left term is a homogenous coordinate system for expressing as a single matrix of projection transformation.

The computing device 200 finds two vertical line segments parallel to the vertical axis in the grid pattern image 510 by using a rotating caliper algorithm capable of obtaining the maximum diameter of the hot wire, and the two vertical line segments Find two horizontal line segments that are perpendicular to and parallel to each other. In addition, the computing device 200 may obtain four intersection points where two vertical line segments and two horizontal line segments intersect.

Thereafter, the computing device 200 may generate a horizontal straight line emphasized image 600 and a vertical straight line emphasized image 610 from the grid pattern image projected on the XY plane. For example, the computing device 200 may reduce the X-axis of the image to 200 pixels and enlarge the Y-axis to 1500 pixels to generate the horizontal straight line emphasis image 600 . The computing device 200 may generate the vertical straight line emphasis image 610 by, for example, enlarging the X-axis of the image to 1700 pixels and reducing the Y-axis to 200 pixels. In this case, since the grid pattern hot wire is formed of 20 horizontal hot wires and 14 vertical hot wires and is a long pattern in the Y-axis direction, magnification values of the horizontal straight line emphasized image 600 and the vertical straight line emphasized image 610 may be set differently.

The computing device 200 searches for a linear component in the linearly emphasized image 600 and the vertical linearly emphasized image 610 through Sobel filtering as in Equation 5, and a linear component through a dilation filter can be inflated, and this process can be repeated a certain number of times (eg, twice). In Equation 5, G _x and G _y are filter values for the X-axis and Y-axis, G _x is a horizontal axis, and G _y is a vertical axis, and A is the image corresponding to the Sobel filter. is the area

The computing device 200 maps the linear components inflated in the straight line-emphasized image 600 and the vertical straight-line-emphasized image 610 to the binary images of the straight-line-emphasized image 600 and the vertical straight-line-emphasized image 610 , and the same value (the pixel value of the binary image of 255), the pixel value of 255 is changed to 0 in the binary image of the linearly emphasized image 600 and the vertical linearly emphasized image 610 and removed from the image, )/vertical straight line image 611 may be generated.

As described above, since the distorted straight line information is projected onto the XY plane of the image and extracted, a re-perspective transform is required to transform the extracted horizontal and vertical straight lines to fit the original image. The computing device 200 inverses the horizontal straight line image 601/vertical straight line image 611 of the XY plane to the original image by using the inverse matrix of the projection transformation matrix used in Equation 4 and the initial feature points that are the basis of the projection transformation. You can transform the projection. In addition, the computing device 200 maps the converted linear component to the binary image of the original image in order to minimize the distortion of the image inversely projected into the original image, and only pixels having the same value obtain a value corresponding to the straight line. can be set to have.

The computing device 200 finds horizontal hot rays and vertical hot rays by fitting the horizontal straight image 700 / vertical straight image 710 of the original size with a curve, and a curve defined by a cubic function such as Equation 6 can be used. there is. In addition, the computing device 200 may perform curve fitting using a determinant as in Equation 7, so that it can be robustly fitted with a curve function even in a thermal image with severe distortion or a lot of rotation.

Referring to FIG. 7 , the computing device 200 may specify a horizontal hot wire and a vertical hot wire of an original thermal image as a function by fitting a straight line component corresponding to a hot wire to a line rather than an area. Accordingly, the computing device 200 may calculate the junctions of the functions, and through this, an accurate feature point position may be known.

In particular, as a result of calibrating the thermal imaging camera using the feature points extracted in this way, the Reprojection Error, a measure of calibration accuracy, was 0.17% compared to the image size (1.11/640), providing a fairly high accuracy.

8 is a block diagram of a computing device according to an embodiment.

Referring to FIG. 8 , the computing device 200 includes one or more processors 210 , a memory 230 for loading a computer program executed by the processor 210 , and a storage device 250 for storing computer programs and various data. , a communication interface 270 , and a bus 290 connecting them. In addition, the computing device 200 may further include various components.

The processor 210 is a device for controlling the operation of the computing device 200 , and may be a processor of various types that processes instructions included in a computer program, for example, a central processing unit (CPU), a micro processor (MPU) Unit), a micro controller unit (MCU), a graphic processing unit (GPU), or any type of processor well known in the art of the present disclosure.

The memory 230 stores various data, commands and/or information. The memory 230 may load a corresponding computer program from the storage device 250 so that the instructions described to execute the operations of the present disclosure are processed by the processor 210 . The memory 230 may be, for example, read only memory (ROM), random access memory (RAM), or the like.

The storage device 250 may non-temporarily store a computer program and various data. The storage device 250 is a non-volatile memory such as a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, or in the art to which the present disclosure pertains. It may be configured to include any well-known computer-readable recording medium.

The communication interface 270 may be a wired/wireless communication module supporting wired/wireless communication.

The bus 290 provides a communication function between the components of the analysis device 200 .

The computer program includes instructions that are executed by the processor 210 , and is stored in a non-transitory computer readable storage medium, and the instructions are read by the processor 210 . Make the action of initiation to be executed. The computer program may be downloaded over a network or sold as a product.

The embodiments of the present disclosure described above are not implemented only through apparatus and methods, and may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present disclosure or a recording medium in which the program is recorded.

Although the embodiments of the present disclosure have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improved forms of the present disclosure are also provided by those skilled in the art using the basic concept of the present disclosure as defined in the following claims. is within the scope of the right.

Claims (15)

A flat panel with grooves corresponding to the grid pattern on the front,
A grid pattern hot wire in which horizontal hot wires and vertical hot wires are inserted into the groove of the flat panel to form a grid pattern for camera calibration, and
A processor for controlling power supply to the grid pattern hot wire,
The flat panel is
Holes of a size through which a hot wire passes are formed at the intersections of the grid patterns,
In the grid pattern heating wire, at an intersection point where each horizontal heating wire and each vertical heating wire intersect, one heating wire is attached to the front surface of the flat panel, and the other heating wire is attached to the rear surface of the flat panel through a hole formed in the flat panel. It has a woven structure coming out to the front through the woven structure, the hot wire pattern device does not come into contact with the intersecting point through the woven structure. In claim 1,
The processor controls power supply and power cut-off to the grid pattern hot wire through a control cycle consisting of a power supply time and a power cut-off time, a hot wire pattern device. In claim 1,
The flat panel is made of a bakelite (bakelite) material, a hot wire pattern device. delete A method of operating a computing device, comprising:
obtaining a thermal image obtained by photographing a grid pattern hot wire;
extracting a horizontal straight line image and a vertical straight line image by pre-processing the thermal image;
extracting horizontal hot wires and vertical hot wires by fitting straight regions included in the horizontal straight image and the vertical straight image with curve functions, and
and extracting the intersection points of the horizontal hot wires and the horizontal hot wires as feature points for camera calibration,
The step of extracting the horizontal straight image and the vertical straight image is
generating an RGB thermal image obtained by converting the thermal energy of the thermal image into an RGB value;
generating a lattice pattern image obtained by binarizing the G-channel image of the RGB thermal image based on the G-channel threshold value;
generating a horizontal straight line emphasized image and a vertical straight line emphasized image from the grid pattern image;
binarizing the horizontal straight line emphasized image into horizontal straight line regions and a background, and converting the binarized image into the horizontal straight line image of the original size; and
binarizing the vertical straight line emphasized image into vertical straight line regions and a background, and converting the binarized image into the vertical straight line image of the original size.
Including, a method of operation. In claim 5,
The step of generating the horizontal straight line emphasized image and the vertical straight line emphasized image includes:
The grid pattern image is reduced in the horizontal axis and enlarged in the vertical axis to generate the horizontal straight line emphasized image in which the horizontal straight line component is emphasized,
An operation method of generating the vertical straight line emphasized image in which a vertical straight line component is emphasized by enlarging the grid pattern image on a horizontal axis and reducing it on a vertical axis. In claim 5,
The step of generating the horizontal straight line emphasized image and the vertical straight line emphasized image includes:
Projection-converting horizontal straight lines and vertical straight lines included in the grid pattern image to be parallel to the X-axis and Y-axis of the image, and generating the horizontal straight line-emphasized image and the vertical straight line-emphasized image from the projection-converted grid pattern image. . In claim 7,
The step of converting the horizontal straight image is
binarizing the horizontal straight line emphasized image into horizontal linear regions and a background, performing inverse projection transformation on the binarized image for the projection transformation, and converting the inverse projection transformed image into the horizontal linear image of the original size,
The step of converting the vertical straight image is
An operation method of binarizing the vertical straight line emphasized image into vertical linear regions and a background, performing inverse projection transformation on the binarized image for the projection transformation, and converting the inverse projection transformed image into the vertical linear image of an original size. In claim 5,
calibrating the thermal imaging camera using the feature points
Further comprising, the method of operation. In claim 5,
The grid pattern hot wire is
An operating method, wherein horizontal heating wires and vertical heating wires are disposed on a flat panel, and power supply and power off are controlled by a processor to maintain a constant temperature. A heating wire pattern device including a grid pattern heating wire in which power supply and power off are controlled by a processor, and
A thermal image obtained by photographing the grid pattern hot wire is pre-processed to extract a horizontal straight line image and a vertical straight line image, and linear regions included in the horizontal straight line image and the vertical straight line image are fitted with curve functions, and A computing device that extracts intersection points as feature points of the grid pattern hot wire and calibrates a camera that captures the thermal image using the feature points,
The hot wire pattern device
A flat panel in which grooves corresponding to the grid pattern are formed on the front surface, and holes of a size through which a hot wire passes are formed at the intersections of the grid patterns;
the lattice pattern hot wire in which horizontal and vertical hot wires are inserted into the groove of the flat panel to form a lattice pattern for camera calibration; and
A processor that controls the supply of power to the grid pattern heating wire
Calibration system including delete In claim 11,
The grid pattern hot wire is
Inserted into the groove and at an intersection point where each horizontal heating wire and each vertical heating wire intersect, one heating wire is attached to the front surface of the flat panel, and the other heating wire is attached to the flat panel through a hole formed in the flat panel. A calibration system with a weave structure that passes through the back and comes out to the front. A heating wire pattern device including a grid pattern heating wire in which power supply and power off are controlled by a processor, and
A thermal image obtained by photographing the grid pattern hot wire is pre-processed to extract a horizontal straight line image and a vertical straight line image, and linear regions included in the horizontal straight line image and the vertical straight line image are fitted with curve functions, and A computing device that extracts intersection points as feature points of the grid pattern hot wire and calibrates a camera that captures the thermal image using the feature points,
the computing device
extracting a grid pattern image based on a G-channel threshold value from an RGB thermal image in which thermal energy of the thermal image is converted into an RGB value;
By enlarging or reducing the horizontal axis and the vertical axis of the grid pattern image, a horizontal straight line emphasized image in which a horizontal straight line component is emphasized and a vertical straight line emphasized image in which a vertical straight line component is emphasized is generated,
A calibration system for extracting the horizontal straight line image and the vertical straight line image by converting the binarized image to an original size after binarizing the horizontal straight line emphasized image and the vertical straight line emphasized image with a linear component and a background. A heating wire pattern device including a grid pattern heating wire in which power supply and power off are controlled by a processor, and
A thermal image obtained by photographing the grid pattern hot wire is pre-processed to extract a horizontal straight line image and a vertical straight line image, and linear regions included in the horizontal straight line image and the vertical straight line image are fitted with curve functions, and A computing device that extracts intersection points as feature points of the grid pattern hot wire and calibrates a camera that captures the thermal image using the feature points,
the computing device
Projection transformation of the grid pattern extracted from the thermal image to be parallel to the X-axis and Y-axis of the image, extracting a horizontal linear component and a vertical linear component, and inversely converting the horizontal linear component and the vertical linear component to the projection transformation A calibration system for extracting the horizontal straight line image and the vertical straight line image by projection conversion.

Images