KR102191747B1 - Distance measurement device and method - Google Patents

Abstract

The present invention relates to a distance measurement technology, and more particularly, to an apparatus and method for measuring a distance using a plurality of image information acquired using a variable aperture. According to an embodiment of the present invention, manufacturing is not complicated and distance measurement is possible in real time, so that when applied to a vehicle, a robot, a drone, etc., it can be used for an autonomous moving object.

Description

Distance measuring device and method {DISTANCE MEASUREMENT DEVICE AND METHOD}

The present invention relates to a distance measurement technology, and more particularly, to a distance measurement apparatus and method for measuring a distance using a plurality of image information having different focus.

Distance measurement technology is a key technology for autonomous driving such as mobile robots and autonomous vehicles. The mobile robot can establish an optimized route plan based on the result of measuring the distance to a reliable object and perform safe driving control based on this. Therefore, even in recent years, a distance measurement technique has been widely studied.

In particular, the core technology of autonomous vehicles is the advanced driving assistance system, or ADAS (Advanced Driver Assistant System). The advanced driving assistance system is largely composed of technologies in three areas: recognition, judgment, and control. The core of the advanced driving assistance system is the sensor. The recognition area of advanced driving assistance systems is a technology that uses sensors to recognize obstacles, road signs, and traffic signals. Advanced driving assistance system sensors can be largely divided into cameras, radars, and LiDARs. A laser range finder or LiDAR sensor is a typical sensor used to measure the distance of a mobile robot. Since the laser or lidar distance sensor provides intuitive information on the surrounding environment, it is easy to implement an observation model, and has advantages such as high accuracy and precision, a wide viewing range, and robustness to lighting. However, the laser or lidar distance sensor is still expensive and has limitations in its application, and accuracy cannot be guaranteed in an optically specific environment, for example, an environment in which a large number of glass or mirrors exist. In addition, although it has excellent functionality for distance measurement, it cannot extract an image, so a separate camera for shooting video must be applied separately to simultaneously capture video and measure distance.

On the other hand, the camera can measure the distance by extracting the object from the captured image while capturing the image. However, not only is it difficult to measure distances with a single camera, but the accuracy is also low.

The background technology of the present invention is disclosed in Korean Patent Application Publication No. 10-2018-0078596.

The present invention provides a distance measuring apparatus for measuring a distance using a plurality of image information acquired using a variable aperture.

In addition, the present invention provides a distance measuring device that accurately measures a distance using one camera module without using a plurality of camera modules.

In addition, the present invention provides a distance measuring apparatus with higher accuracy by using a plurality of image information without using one image information.

In addition, the present invention provides a distance measurement apparatus that stores image information as well as distance measurement.

According to an aspect of the present invention, a distance measuring device is provided

A distance measuring apparatus according to an embodiment of the present invention includes a camera module that obtains an image with a different focus by adjusting a diameter of an aperture opening, and a distance measuring module that calculates a distance using a plurality of images with different focus obtained from the camera module It may include.

According to another aspect of the present invention, a computer-readable recording medium in which a method for measuring a distance and a computer program for executing the same is recorded is provided.

A computer-readable recording medium in which a method for measuring distance and a computer program for executing the same according to an embodiment of the present invention is recorded is acquiring an image having a different focus using a variable aperture, for calculating a distance from an image having a different focus. It may include detecting an object, calculating a pixel value for a source of confusion of the object, and estimating a distance based on a difference between the pixel values.

According to an embodiment of the present invention, manufacturing is not complicated and distance measurement is possible in real time, so that when applied to a vehicle, a robot, a drone, etc., it can be used for an autonomous moving object.

According to an embodiment of the present invention, manufacturing is not complicated and distance measurement is possible in real time, so that when applied to a vehicle, a robot, a drone, etc., it can be used for an autonomous moving object.

According to an embodiment of the present invention, a distance may be accurately calculated by using a plurality of image information captured by one camera module.

1 is a block diagram of a distance measuring apparatus according to an embodiment of the present invention.
2 is a block diagram of a distance measuring apparatus according to an embodiment of the present invention.
3 is a block diagram of a distance measurement module according to an embodiment of the present invention.
4 is a diagram illustrating a method of extracting a depth map by a distance measuring apparatus according to an embodiment of the present invention using a defocal depth algorithm.
5 is a diagram illustrating a change in a focal plane according to a variable aperture in an image acquired by a distance measuring apparatus according to an embodiment of the present invention.
6 is a flowchart of a distance measuring method according to an embodiment of the present invention.
7 is a cross-sectional view of a camera module of a distance measuring apparatus according to an embodiment of the present invention.
8 to 10 are views showing a beam of a distance measuring apparatus according to an embodiment of the present invention.
11 and 12 are diagrams showing images obtained by mounting a distance measuring device according to an embodiment of the present invention to an autonomous vehicle.
13 is a graph showing a blur value extracted by a distance measuring apparatus according to an embodiment of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided. Specific embodiments are illustrated in the drawings and will be described in detail through detailed description. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, when it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, and in the description with reference to the accompanying drawings, the same or corresponding components are assigned the same reference numbers, and redundant descriptions thereof will be omitted. To

1 is a configuration diagram of a distance measuring apparatus according to an embodiment of the present invention.

Referring to FIG. 1, the distance measuring apparatus 10 may be applied to automobiles, robots, drones, etc. by providing real-time distance information based on image information, and may be applied to autonomous vehicles, smart homes, and smart factories.

2 is a block diagram of a distance measuring apparatus according to an embodiment of the present invention.

Referring to FIG. 2, a distance measurement apparatus 10 according to an embodiment of the present invention includes a camera module 100 and a distance measurement module 200.

The camera module 100 simultaneously acquires a plurality of images having different focal points by adjusting the diameter of the peripheral image and the aperture opening, like a general camera or a black box. The camera module 100 stores a captured image and simultaneously transmits a plurality of images having different focuses to the distance measurement module 200. For example, the camera module 100 may use the captured image as a monitoring image like a black box. In addition, the camera module 100 may use the captured image as data to determine traffic sign recognition, traffic light recognition, blind spot detection, lane departure, and the like in the autonomous vehicle.

The camera module 100 includes a variable aperture unit 110 and a camera unit 160.

The variable aperture unit 110 adjusts the diameter of the aperture aperture in order to obtain an image with a different focus. The variable aperture unit 110 is coupled to the front end of the camera unit 160 and may be used in a manner of assembling, for example, a general camera without any modification.

The camera unit 160 generates a plurality of image information having different focus. The camera unit 160 includes a camera lens, an image sensor, and an infrared filter (IR filter).

The camera lens delivers light to the image sensor.

The infrared filter is a filter that blocks infrared rays. Since the image sensor detects not only visible rays but also near infrared rays, the near infrared rays are reflected.

The image sensor generates image information by converting the light passing through the infrared filter into an electrical signal.

The distance measurement module 200 measures a distance to an object using a plurality of image information having different focus. The distance measurement module 200 includes an input unit 210 and a distance measurement unit 220. The distance measurement module 200 will be described in more detail below with reference to FIG. 3.

3 is a block diagram of a distance measurement module according to an embodiment of the present invention.

Referring to FIG. 3, the distance measurement module 200 compares images with different focus among image information generated by the camera module 100 to measure a distance to an object. The distance measurement module 200 measures a distance using a Defocus Depth (Depth From Defocus) algorithm. The out-of-focus depth algorithm uses a proportional relationship between a depth value and a blur (blur), and can calculate a depth value by measuring the amount of blur. The distance measurement module 200 may increase measurement accuracy compared to a defocus depth algorithm method using a single image. The distance measurement module 200 includes an input unit 210 and a distance measurement unit 220.

The input unit 210 may obtain information on a plurality of images having different focuses using the variable aperture unit 110 through the camera module 100.

The distance measuring unit 220 measures a distance to an object by using information on a plurality of images having different focal points. In more detail, the distance measurement unit 220 extracts distance information by using image information of a shallow depth of field obtained using a large aperture and image information having a deep depth of field obtained using a small aperture. The distance measurement unit 220 includes a detection unit 221 and a depth calculation unit 223.

The detection unit 221 detects an object to measure a distance from the image information. The detection unit 221 may apply various known algorithms as an algorithm for object detection. The detection unit 221 may use, for example, the YOLO2 version, which is one of object recognition algorithms.

The depth calculator 223 calculates a distance to an object included in a plurality of image information. An out-of-focus image acquired by the distance measuring apparatus 10 using a variable iris forms a circle of confusion. The depth calculator 223 calculates a distance by using a difference between pixel values of a confusion circle in a plurality of images having different focal points. The depth calculator 223 has different sources of confusion between the image with a low depth of field acquired when the diameter of the variable iris is the minimum and the image with a high depth of field acquired when the diameter of the variable iris is the maximum. The diameter of the circle of confusion can be expressed in terms of the size of a pixel.

In the defocal depth algorithm using a single image, the distance can be converted in reverse by accurately counting the number of blurred pixels from the following [Equation 1].

In [Equation 1], S1 is the distance of the object plane in focus, S2 is the distance of the virtual image of the unfocused object, f is the effective focal length (EFL), f1 is the distance from the lens to the image plane, and c is It is the diameter of the circle of confusion in the image plane.

The defocal depth algorithm using a single image requires a technique to analyze how far the blurred section is in order to accurately count the number of blurred pixels and requires high accuracy. However, if the out-of-focus depth algorithm is used with two images with different focus, a more accurate distance can be extracted by using the difference in pixel values of the confusion circle between the two images.

4 is a diagram illustrating a method of extracting a depth map by a distance measuring apparatus according to an embodiment of the present invention using a defocus depth algorithm

Referring to FIG. 4, the distance measuring apparatus 10 extracts depth map information from an image having a different focus acquired using a variable aperture using a defocal depth algorithm.

5 is a diagram illustrating a change in a focal plane according to a variable aperture in an image acquired by a distance measuring apparatus according to an embodiment of the present invention.

Referring to FIG. 5, an image obtained by the distance measuring apparatus 10 using a variable iris forms a circle of confusion depending on the degree of blur, that is, out of focus. The depth calculation unit 223 uses the source of confusion of the image acquired when the diameter of the variable iris is the minimum and the maximum. To explain this in more detail, S1 shown in FIG. 5 is the distance of the object plane in focus, S2 is the distance of the virtual image of the unfocused object, f is the effective focal length (EFL), and f1 is the distance from the lens to the image plane. The distance, c, is the diameter of the circle of confusion in the image plane. If the defocal depth algorithm method is used with two images with different focus, a more accurate distance can be extracted by using the difference in pixel values of the confusion circle between the two images.

Table 1 is a table showing the results of the relative error, log error, and root mean square error between the actual distance and the value obtained by each method by performing various depth measurement methods with an arbitrary NYU-v2 data set. The distance measurement method according to the embodiment of the present invention is indicated by the SNU method. The relative error of the distance measurement method (SNU method) of the present invention is 0.028, the common log error is 0.012, and the root mean square error is 0.154. As shown in Table 1, it is confirmed that the distance measurement method of the present invention has the least error among various methods.

Comparison of errors between distance measurement methods Way Relative error Common log error Root mean square error Input value DFD 0.609 - 2.758 Out of focus single image Saxena 0.349 - 1.214 Single image in focus DT 0.350 0.131 1.2 Single image in focus DCCRF 0.335 0.127 1.06 Single image in focus Eigen 0.215 - 0.907 Single image in focus DCNF 0.213 0.087 0.759 Single image in focus JCNF 0.201 0.077 0.708 Single image in focus Eigen 0.158 - 0.641 Single image in focus Anwar 0.094 0.039 0.347 Single image in focus SNU 0.028 0.012 0.154 Out of focus double image

Relative error, common log error, and root mean square error are calculated using [Equation 2].

6 is a flowchart of a distance measurement method according to an embodiment of the present invention.

Referring to FIG. 6, in step S610, the distance measuring apparatus 10 acquires a plurality of images with different focuses captured by the camera module 100 using a variable iris.

In step S620, the distance measuring apparatus 10 detects an object for calculating a distance in an image having a different focus.

In step S630, the distance measuring apparatus 10 calculates the diameter of the circle of confusion, which is the degree of blurring of the object, using the defocal depth algorithm.

In step S640, the distance measuring apparatus 10 calculates a distance by using a pixel difference of a source of confusion of an object detected.

7 is a cross-sectional view of a camera module of a distance measuring apparatus according to an embodiment of the present invention.

Referring to FIG. 7, the camera module 100 may couple a variable aperture portion 110 having a variable aperture diameter to the front end of the camera portion 160. The distance measuring apparatus 10 may provide an image having a different focus using the variable aperture unit 110 having a variable diameter.

8 to 10 are views for explaining a variable aperture part of a distance measuring apparatus according to an embodiment of the present invention.

Referring to FIG. 8, the variable aperture part 110 may variably change the diameter of the aperture aperture using thermal expansion. The variable aperture part 110 includes a blade 120, a beam 130 and a fixed pad 140. The fixing pad 140 is coupled to the beam 130 to fix the beam 130. For example, two fixing pads 140 may fix one beam 130, and ten fixing pads 140 may be used. One side of the beam 130 is coupled to the fixed pad 140 and the other side is coupled to the blade 120. When a voltage is applied to the fixed pad 140, current is transmitted to the beam 130 coupled with the fixed pad 140 to cause thermal expansion. The beam 130 in which thermal expansion has occurred generates a displacement in the blade 120 coupled to the other side. The blade 120 is arranged in a pentagon, and when displacement occurs, the size of the pentagon decreases, and the amount of light source transmitted to the lens decreases. For example, the variable aperture part 110 may be manufactured using a silicon wafer, and the total size may be 10.2mm X 10.2mm X 0.365mm.

Referring to FIG. 9, when current is transmitted to the beam 130, thermal expansion may occur, so that the diameter of the variable aperture may vary. The beam 130 includes a first beam 131, a second beam 132 and a third beam 133. Both sides of the first beam 131 are coupled to the fixing pad 140, respectively, and the second beam 132 is coupled to the center. One side of the second beam 132 is combined with the first beam 131, the other side is combined with the third beam 133, and the force transmitted to the second beam 132 is transferred to the third beam 133 do. The two second beams 132 coupled to the third beam 133 are coupled to the third beam 133 so as to shift from each other, so that the third beam 133 rotates in a counterclockwise direction. The other side of the third beam 133 is coupled to the blade 120. When the third beam 133 rotates counterclockwise, the blade 120 coupled to the other side is displaced, and the blade 120 moves in the inner circle direction to change the diameter of the variable aperture.

Referring to FIG. 10, the first beam 131 has a chevron shape, and both sides are fixed to the fixing pad 140 and the second beam 132 may be coupled to the center. Thermal expansion of the first beam 131 occurs due to joule's heating according to an applied voltage transmitted from the fixed pad 140. When a voltage is applied to the fixed pad 140, current is transmitted to the first beam 131, and at this time, thermal expansion occurs in the longitudinal direction of the first beam 131 due to Joule heating. Both sides of the first beam 131 are fixed to the fixing pad 140 at an angle in a chevron shape (seagull shape). Thermal expansion of the first beam 131 occurs in a linear direction due to the inclined angle. Since one side of the first beam 131 is fixed to the fixing pad 140, linear expansion occurs in the vertical direction of the first beam 131 and a second beam coupled to the center of the first beam 131 ( 132). The displacement of the second beam 132 can be calculated by [Equation 3].

Referring back to FIG. 9, the displacement of the blade 120 may be amplified by the principle of a lever by aligning the second beam 132 coupled to the third beam 133 to be offset. For example, two second beams 132 coupled to the third beam 133 may be aligned to be 75 μm shifted from each other.

The displacement of the blade 120 amplified by the principle of the lever can be calculated using [Equation 4].

The blade 120 is made of five and is arranged in a pentagonal shape to function as a variable aperture. The blade 120 is displaced due to thermal expansion according to the applied voltage, and the diameter of the diaphragm is changed. For example, the aperture diameter is 1.6mm in the absence of voltage, and can be changed to at least 0.46mm when voltage is applied.

According to an embodiment of the present invention, the change in the aperture diameter according to the application of voltage is shown in Table 2.

Change in diameter of aperture Voltage[V] Current[A] Blade displacement [㎛] Area ratio of opening (%) 0 0 0 100.00 One 1.37 15.7 99.29 2 2.52 67.7 87.79 3 3.11 153 75.85

11 and 12 are diagrams illustrating images obtained by mounting a distance measuring device according to an embodiment of the present invention to an autonomous vehicle.

Referring to FIG. 11, for example, the distance measuring device 10 may utilize 3D point cloud data from LiDAR and image data values obtained through the distance measuring device of the present invention as data for deep learning learning. have. The distance measuring apparatus 10 may continuously acquire distance information for an image.

Referring to FIG. 12, the distance measuring apparatus 10 uses the image acquired by the lidar as a reference value, and the input unit 210 determines the degree of blurring of the image with different focus acquired by the variable aperture of F1.8 and F4.0. Can be analyzed.

13 is a graph showing a blur value extracted by a distance measuring apparatus according to an embodiment of the present invention.

Referring to FIG. 13, as the distance increases, the distance measuring apparatus 10 may measure the distance by using a tendency that the degree of blurring of each image increases and the difference in blur between the images increases.

The above-described embodiments of the present invention can be implemented through various means. For example, embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof. In the case of implementation by firmware or software, the method according to the embodiments of the present invention may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. A computer program in which software codes and the like are recorded may be stored in a computer-readable recording medium or a memory unit and driven by a processor. As such, those skilled in the art to which the present invention pertains will be able to understand that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. Therefore, the embodiments described above are illustrative in all respects and should be understood as non-limiting. The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. .

Distance Measuring Device(10)
Camera module (100)
Variable aperture (110)
Blade(120)
Beam(130)
First beam (131)
2nd beam (132)
3rd beam (133)
Fixed pad (140)
Camera unit 160
Distance Measurement Module(200)
Input unit 210
Distance measuring unit 220
Detection unit 221
Depth calculation unit 223

Claims (8)

In the distance measuring device,
A camera module that adjusts the diameter of the aperture opening to obtain an image having a different focus; And
Including a distance measurement module for calculating a distance using a plurality of images having different focus obtained from the camera module,
The camera module
Including a variable aperture portion of which the diameter of the aperture aperture changes,
The variable aperture part
By applying voltage to the fixed pad, thermal expansion is generated in the combined beam,
The beam is
Distance measuring device for adjusting the diameter of the aperture opening by amplifying the displacement of the blade coupled to one side by displacing one or more.
The method of claim 1,
The distance measurement module
An input unit for obtaining an image having a different focus using the variable aperture unit; And
A distance measuring device comprising a distance measuring unit that calculates a distance from the image to an object.
The method of claim 2,
The distance measuring unit,
A distance measuring apparatus comprising a depth calculator configured to calculate a distance by using a difference between pixel values of a circle of confusion in a plurality of out-of-focus images obtained using the variable iris.
The method of claim 3,
The distance measuring unit
Distance measuring apparatus further comprising a detector for detecting an object for calculating a distance in the image.
In the distance measurement method executed in the distance measurement device,
Acquiring an image having a different focus using a variable aperture;
Calculating a pixel value for a source of confusion of an object included in the image with a different focus; And
Including the step of estimating a distance based on the difference between the pixel values,
The variable aperture is
By applying voltage to the fixed pad, thermal expansion is generated in the combined beam,
The beam is
A distance measuring method for adjusting the diameter of the aperture opening by amplifying the displacement of the blade coupled to one side by displacing at least one.
The method of claim 5,
The step of calculating a pixel value for a circle of confusion of an object included in an image having a different focus is
A distance measurement method comprising calculating the diameter of a circle of confusion, which is a degree of blurring of an object detected using a defocal depth algorithm.
The method of claim 5,
The distance measurement method further comprising detecting an object for calculating a distance in the image having a different focus.
A computer-readable recording medium in which a computer program that executes any one of the methods of claim 5 to 7 is stored.

Images