KR101918887B1 - Distance measuring device using mono infrared camera and method thereof - Google Patents

Abstract

Disclosed are a device and a method for measuring a distance using a mono infrared camera, which can perform a function of a stereo thermal image camera such as distance data acquisition and 3D reconstruction through a plurality of image information obtained with a parallax by using a mono thermal image camera. According to an embodiment of the present invention, the device for measuring a distance comprises: a moving body; a sensor for measuring a displacement of the moving body according to movement of the moving body; the mono thermal image camera for generating thermal images according to the movement of the moving body; and a controller for calculating an actual distance to the thermal image by using the displacement of the moving body and the thermal images.

Description

TECHNICAL FIELD [0001] The present invention relates to a distance measuring apparatus using a mono-infrared camera,

The following embodiments relate to a distance measuring apparatus and method using a mono infrared camera.

A person can recognize a three-dimensional object using the difference between the images seen by the two eyes. This is the method of recognizing distances by using a stereo camera, and this method is also used when producing 3D movies.

To measure the distance to a distant object with two images taken by two cameras separated by a certain distance like a human eye, it is necessary to see how much the object has changed, that is, how far it has moved on the screen.

On the other hand, a person who is blinded by one eye can recognize three dimensions in a similar manner. Since a person can move, it is possible to know whether the object is three-dimensional or two-dimensional by moving his head or walking. In other words, although two images can not be seen at the same time, distances to objects can be recognized with images taken continuously for a short time.

A related prior art is disclosed in Korean Patent Publication No. 2005-0000766

Embodiments can provide a technique capable of performing the function of a stereo thermal camera such as distance data acquisition and 3D reconstruction through a plurality of image information obtained with a parallax using a mono thermal camera.

A distance measuring apparatus according to an embodiment of the present invention includes a moving body, a sensor for measuring a displacement of the moving body in accordance with the movement of the moving body, a mono thermal imaging camera for generating thermal images in accordance with the movement of the moving body, And a controller for calculating the actual distance to the thermal image using the thermal images.

The controller includes an image processing module for processing the first lunar image generated at the first position of the moving object and the second lunar image generated at the second position of the moving object to extract the feature points of the lunar image, And a distance calculation module for calculating the actual distance through a stereo graphic method using the displacement of the moving object.

Wherein the image processing module extracts first pixels of a high temperature value from the first lunar image, extracts second pixels of a high temperature value from the second lunar image, The points matched in the pixels can be extracted as feature points of the thermal image.

Wherein the image processing module specifies a region of interest (ROI) at a predetermined interval in the first pixels and the second pixels, and matches at least one of a gradient and a pixel value in a specified ROI Can be found.

Wherein the image processing module matches the first pixels with the second pixels using the SURF technique when the first and second thermal images have the same direction without being rotated, Can be found.

The image processing module may extract a matching point in the first pixels and the second pixels as deep feature points using deep running. The controller may use the actual distance to calculate And a 3D generation module for performing a 3D reconstruction of the object.

Wherein the actual distance includes at least one of a distance from the first position to the heat image, a distance from the second position to the heat image, and a distance from the intermediate position between the first position and the second position to the heat image .

The distance calculation module may calculate the distance traveled by the moving object by using the first displacement measured at the first position and the second displacement measured at the second position.

According to an embodiment of the present invention, there is provided a distance measuring method comprising: measuring a displacement of a moving object in accordance with movement of a moving object; generating thermal images by a mono-thermal camera in accordance with movement of the moving object; And calculating the actual distance up to the thermal image using the thermal images.

Wherein the step of calculating includes extracting feature points of the thermal image by processing a first thermal image generated at a first position of the moving object and a second thermal image generated at a second position of the moving object, And calculating the actual distance through a stereo graphic method using the displacement of the moving object.

Wherein the extracting comprises extracting first pixels of a high temperature value in the first thermal image, extracting second pixels of a high temperature value in the second thermal image, And extracting a matching point in the second pixels as feature points of the thermal image.

Wherein the step of extracting at least one of a gradient and a pixel value in a specified ROI comprises the steps of: specifying a region of interest (ROI) at a predetermined interval in the first pixels and the second pixels; And searching for a matching point using the search result.

The step of extracting the feature points of the thermal image includes matching the first pixels and the second pixels using the SURF technique when the first and second thermal images have the same direction without being rotated And searching for a matching point.

The step of extracting a feature point of the thermal image may include extracting a matching point of the first pixels and the second pixels as deep feature points using deep running. And performing a 3D reconstruction of the object corresponding to the thermal image using the 3D reconstruction.

Wherein the actual distance includes at least one of a distance from the first position to the heat image, a distance from the second position to the heat image, and a distance from the intermediate position between the first position and the second position to the heat image .

Wherein calculating the actual distance through the stereo graphic method comprises calculating a distance traveled by the moving object using the first displacement measured at the first position and the second displacement measured at the second position can do.

1 is a schematic block diagram of a distance measuring device according to an embodiment.
Fig. 2 shows an example of a distance measuring apparatus for photographing a thermal image at different positions.
3 is a schematic block diagram of the controller shown in Fig.
FIG. 4 is a diagram for explaining an example of a feature point extraction operation of the image processing module shown in FIG.
5 is a view for explaining another example of the feature point extracting operation of the image processing module shown in FIG.
6 is a diagram for explaining another example of the feature point extraction operation of the image processing module shown in FIG.
7 is a view for explaining an example of a distance calculation operation of the distance calculation module shown in FIG.
8 is a view for explaining an application example of the distance measuring apparatus according to an embodiment.

It is to be understood that the specific structural or functional descriptions of embodiments of the present invention disclosed herein are presented for the purpose of describing embodiments only in accordance with the concepts of the present invention, May be embodied in various forms and are not limited to the embodiments described herein.

Embodiments in accordance with the concepts of the present invention are capable of various modifications and may take various forms, so that the embodiments are illustrated in the drawings and described in detail herein. However, it is not intended to limit the embodiments according to the concepts of the present invention to the specific disclosure forms, but includes changes, equivalents, or alternatives falling within the spirit and scope of the present invention.

The terms first, second, or the like may be used to describe various elements, but the elements should not be limited by the terms. The terms may be named for the purpose of distinguishing one element from another, for example without departing from the scope of the right according to the concept of the present invention, the first element being referred to as the second element, Similarly, the second component may also be referred to as the first component.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between. Expressions that describe the relationship between components, such as "between" and "between" or "neighboring to" and "directly adjacent to" should be interpreted as well.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. It will be understood that, in this specification, the terms "comprises ", or" having ", and the like are to be construed as including the presence of stated features, integers, But do not preclude the presence or addition of steps, operations, elements, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

Fig. 1 is a schematic block diagram of a distance measuring apparatus according to an embodiment, and Fig. 2 shows an example of a distance measuring apparatus for photographing a laminar image at different positions.

1 and 2, the distance measuring apparatus 10 includes a moving object 100, a sensor 200, a thermal imaging camera 300, and a controller 400. [ The distance measuring apparatus 10 may further include a display 500.

The moving object 100 may refer to a moving object. For example, the mobile object 100 may include a moving robot (i.e., a mobile robot)

The moving body 100 can move under the control of the controller 400. [ For example, the moving body 100 may move to the second position with a parallax at the first position under the control of the controller 400. [

The sensor 200 can measure the displacement of the moving object 100 and transmit the measured displacement to the controller 400. [ For example, the sensor 200 may measure the first displacement at the first position of the mobile object 100 and transmit the measured first displacement to the controller 400. [ The sensor 200 may measure the second displacement at the second position where the moving body 100 has moved and transmit the measured second displacement to the controller 400. [

The sensor 200 may be implemented as a displacement measurement sensor. For example, the displacement measurement sensor may be implemented as at least one of an odometer and an IMU sensor.

The thermal imaging camera 300 can generate a thermal image at a position where the moving body 100 has moved according to the control of the controller 400 and transmit the thermal image to the controller 400. [ For example, the thermal imaging camera 300 may generate a first thermal image at a first location of the moving object 100 and transmit the generated first thermal image to the controller 400. [ The thermal imaging camera 300 may generate a second thermal image at a second position where the moving body 100 has moved and transmit the generated second thermal image to the controller 400. [

The thermal imaging camera 200 may be implemented with an infrared camera. Since the thermal imaging camera 200 uses a wavelength longer than that of a general camera, it is possible to acquire an image, that is, a thermal image even in a low-light condition in which the light amount is insufficient or in a limited clock environment due to smoke.

The controller 400 can calculate the actual distance between the distance measuring device 10 and the object representing the lining, for example, the lining, using the displacement transmitted from the sensor 200 and the thermal image transmitted from the thermal imaging camera 300 have. The actual distance may include a distance from the first position to the heat, a distance from the second position to the heat, and / or a distance from the intermediate position (or intermediate point) between the first and second positions to the heat.

In the case of recognizing an object based on a general image, a large amount of calculation is required because a learning algorithm is used for pixels in the entire image to detect an object in a general camera. However, by using the thermal imager 300, The controller 400 can focus on an object emitting heat (e.g., infrared), reduce the ROI for thermal image detection, and effectively recognize the object by utilizing the characteristics of the object.

In addition, the controller 400 may perform a 3D reconstruction of the object corresponding to the thermal image using the calculated actual distance, e.g., distance and / or depth data. The controller 400 may send the 3D reconstruction of the object to the display 500 for display.

The display 500 may be implemented in the distance measuring device 10, but may be implemented in a separately implemented electronic device. An electronic device, a personal computer (PC), a data server, or a portable device.

Portable devices include laptop computers, mobile phones, smart phones, tablet PCs, mobile internet devices (MIDs), personal digital assistants (PDAs), enterprise digital assistants (EDAs) A digital still camera, a digital video camera, a portable multimedia player (PMP), a personal navigation device or a portable navigation device (PND), a handheld game console, an e-book e-book, or a smart device. The smartvision can be implemented as a smart watch, a smart band, or a smart ring.

The distance measuring apparatus 10 measures the displacement (or displacement vector) of the moving object 100 and can perform the functions of two thermal imaging cameras, that is, a stereo thermal imaging camera, with the mono thermal imaging camera 300. [ In addition, the distance measuring apparatus 10 can achieve price reduction, weight reduction, and volume reduction through the movable movable body 100. [

In addition, since the distance measuring apparatus 10 can move the mono thermal camera 300 through the moving body 100 to create a disparity, the parallax can be changed as much as desired. Therefore, when the distance of the object to be measured is close It is possible to improve accuracy more actively by using small parallax and large parallax when it is far away.

FIG. 3 is a schematic block diagram of the controller shown in FIG. 1, FIG. 4 is a diagram for explaining an example of a feature point extracting operation of the image processing module shown in FIG. 3, FIG. 6 is a view for explaining another example of the feature point extracting operation of the image processing module shown in FIG. 3, and FIG. 7 is a view for explaining another example of the feature point extracting operation of the image processing module shown in FIG. FIG. 4 is a diagram for explaining an example of the distance calculation operation of the distance calculation module. FIG.

3 through 7, the controller 400 may include an image processing module 410, a distance calculation module 430, and a 3D generation module 450.

The image processing module 410 may process the thermal images to extract thermal feature points from the thermal images and output one or more feature points to the distance calculation module 430. For example, a feature point may mean a point (e.g., a pixel) that is separated in the image by reflecting the color, contrast, position, and distribution characteristics of the pixel.

The image processing module 410 extracts first pixels of a high temperature value from the first lunar image, extracts second pixels of a high temperature value from the second lunar image, Can be determined as a feature point.

In one example, the image processing module 410 may calculate the first and second pixels in consideration of the relationship between the first peripheral pixels of the first pixels and the second pixels (e.g., the first lunar image and the surrounding pixels in the second lunal image) It is possible to determine a matching point in the pixels and the second pixels as the characteristic point. An example of feature point extraction of the image processing module 410 may be as shown in FIG.

In another example, the image processing module 410 may assign a region of interest (ROI) to the first and second pixels at a predetermined interval and may be matched using a gradient and / or a pixel value at a specified ROI Points can be found.

For example, the image processing module 410 may perform a vision image technique to find matching points. Vision imaging techniques can include techniques such as SURF, BRIST, SIFT, DAISY, and ASIFT.

As shown in FIG. 5, if the first lunar image and the second lunar image have the same orientation without being rotated, the image processing module 410 can quickly and effectively utilize the SURF technique, Pixels may be matched to find a matching point.

It is not necessarily limited to the above-described vision image technique, and a matching point can be found by using various image techniques.

As another example, the image processing module 410 may use the deep learning to determine, as feature points, the points that match in the first pixels and the second pixels. As shown in Figure 6, the deep learning network receives as input the first and second thermal images, in which the respective pixel temperature values appear, and the coordinates of the points matched in the first and second pixels to the feature points Can be output as coordinates. The entire pixel values of the thermal image need not be input as inputs to the deep learning network, and only the first and second pixels may be input to the input of the deep learning network, depending on the embodiment.

The distance calculation module 430 may calculate the actual distance between the distance measuring device 10 and the object representing the lining, for example, the lining, using the feature points and the displacement.

As shown in FIG. 7, the distance calculation module 430 may perform the distance calculation through a stereography method. At this time, the distance calculation module 430 may use Equation (1) for distance calculation.

Here, x represents the actual distance between the object representing the thermal image and the distance measuring apparatus 10, and L represents the distance (for example, the distance between the first position and the second position) , And f may denote the focal length of the thermal imaging camera 300.

Also, dl may be a value corresponding to the feature point shown in the first lacquer image (e.g., the left image in FIG. 5) and the focus of the thermal imager 300 (e.g., the focus at the first position) And the center point of the first lunar image. dr is associated with the feature point shown in the second thermal image (e.g., the right image in FIG. 5) and the feature point corresponding (or projected) to the focus of the thermal imaging camera 300 (e.g., And the distance from the center point of the second thermal image.

The distance calculation module 430 may calculate L using the first displacement measured at the first location and the second displacement measured at the second location. The distance calculation module 430 may calculate dl and dr using the minutiae and f. The distance calculation module 430 may calculate x using the calculated result.

x, that is, the actual distance may include a distance from the first position to the heat, a distance from the second position to the heat, and / or a distance from the middle position (or intermediate point) have.

The 3D generation module 450 may perform a 3D reconstruction of the object corresponding to the thermal image using the calculated actual distance. The 3D generation module 450 can send the 3D reconstruction of the object to the display 500 and display it.

A module in this specification may mean hardware capable of performing the functions and operations according to the respective names described in this specification and may mean computer program codes capable of performing specific functions and operations , Or an electronic recording medium, e.g., a processor or a microprocessor, equipped with computer program code capable of performing certain functions and operations.

In other words, a module may mean a functional and / or structural combination of software for driving hardware and / or hardware for carrying out the technical idea of the present invention.

8 is a view for explaining an application example of the distance measuring apparatus according to an embodiment.

Referring to FIG. 8, the distance measuring apparatus 100 may be capable of detecting heat and flame of a person. In a disaster-affected area, a distance measuring device 100 may be inserted into a space where firefighters can not directly enter, and the position of a petitioner and a fire spot may be possible.

In the event of a disaster, there is a high possibility that the light will be blocked due to factors such as electricity cut-off, building collapse, and disaster situation in the underground space. And realizing 3D reconstruction of objects based on the distance measurement technique according to the embodiment, it is possible to design a local and global path for recognizing a dangerous area and evacuating a demanding person. In addition, it can be expected that an effective disaster response system for rapid disaster response and rescue assistance will be established.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (18)

Moving body;
A sensor for measuring a displacement of the moving object according to the movement of the moving object;
A mono thermal camera for generating thermal images according to the movement of the moving body; And
A controller for calculating a displacement of the moving body and an actual distance to the heat by using the thermal images,
Lt; / RTI >
The controller comprising:
An image processing module for processing the first lunar image generated at the first position of the moving object and the second lunar image generated at the second position of the moving object to extract the characteristic points of the lunar phase; And
A distance calculation module for calculating the actual distance through a stereo graphic method using the feature points of the thermal image and the displacement of the moving object,
.
delete The method according to claim 1,
The image processing module comprising:
Extracting first pixels of a high temperature value from the first lunar image, extracting second pixels of a high temperature value from the second lunar image, Extracting a point as a feature point of the thermal image
Distance measuring device.
The method of claim 3,
The image processing module comprising:
A ROI (Region of Interest) at a predetermined interval is specified in the first pixels and the second pixels, and a matching point is found using at least one of a gradient and a pixel value in a specified ROI
Distance measuring device.
The method of claim 3,
The image processing module comprising:
If the first and second thermal images have the same direction without being rotated, the first pixels and the second pixels are matched using the SURF technique to find a matching point
Distance measuring device.
The method of claim 3,
The image processing module comprising:
A point matching in the first pixels and the second pixels is extracted as a feature point of the thermal image using deep running
Distance measuring device.
The method according to claim 1,
The controller comprising:
A 3D generation module for performing a 3D reconstruction of an object corresponding to the lattice using the actual distance;
Further comprising a distance measuring device.
The method according to claim 1,
Wherein the actual distance includes at least one of a distance from the first position to the heat image, a distance from the second position to the heat image, and a distance from the intermediate position between the first and second positions to the heat image
Distance measuring device.
The method according to claim 1,
The distance calculation module calculates,
Calculating a distance traveled by the moving object using the first displacement measured at the first position and the second displacement measured at the second position
Distance measuring device.
Measuring a displacement of the moving object in accordance with the movement of the moving object;
Generating thermal images by a mono thermal camera according to the movement of the moving body; And
Calculating a displacement of the moving body and an actual distance to the thermal image using the thermal images
Lt; / RTI >
Wherein the calculating step comprises:
Extracting feature points of the thermal image by processing a first thermal image generated at a first position of the moving object and a second thermal image generated at a second position of the moving object; And
Calculating the actual distance through the stereo image method using the feature point of the thermal image and the displacement of the moving object
/ RTI >
delete 11. The method of claim 10,
Wherein the extracting comprises:
Extracting first pixels having a high temperature value in the first lunar image;
Extracting second pixels of a high temperature value from the second laminar image; And
Extracting a matching point in the first pixels and the second pixels as minutiae of the thermal image;
/ RTI >
13. The method of claim 12,
The step of extracting the feature points of the thermal image comprises:
Designating a region of interest (ROI) at a predetermined interval in the first pixels and the second pixels; And
Searching for a matching point using at least one of a gradient and a pixel value in a specified ROI
/ RTI >
13. The method of claim 12,
The step of extracting the feature points of the thermal image comprises:
If the first lunar image and the second lunar image have the same direction without being rotated, matching the first pixels with the second pixels using the SURF technique to find a matching point
/ RTI >
13. The method of claim 12,
The step of extracting the feature points of the thermal image comprises:
Extracting a matching point in the first pixels and the second pixels as deep feature points using deep learning;
/ RTI >
11. The method of claim 10,
Performing 3D reconstruction of the object corresponding to the lattice using the actual distance;
Further comprising:
11. The method of claim 10,
Wherein the actual distance includes at least one of a distance from the first position to the heat image, a distance from the second position to the heat image, and a distance from the intermediate position between the first and second positions to the heat image
Distance measurement method.
11. The method of claim 10,
Wherein the step of calculating the actual distance through the stereo-
Calculating a distance traveled by the moving object using the first displacement measured at the first position and the second displacement measured at the second position,
/ RTI >

Images