KR20220080359A - Apparatus for LIDAR - Google Patents

Abstract

The present application relates to a lidar device, and the lidar device according to an embodiment of the present invention includes a laser light source unit for irradiating a line laser (line laser) in a target space; an image sensor unit for generating a detection image by detecting the line laser reflected in the target space; and an image processing unit generating a three-dimensional coordinate map for the target space by using the laser pattern displayed in the detection image, wherein the laser pattern is short-circuited from the reference pattern and vertically spaced apart from the reference pattern. of the target pattern.

Description

LIDAR device {Apparatus for LIDAR}

The present application relates to a lidar device using a line laser, and more particularly, to a lidar device capable of generating a three-dimensional coordinate map in a target space by performing image processing on a detected image.

Laser (Light Amplification by the Stimulated Emission of Radiation, LASER) is light amplified by stimulated emission and is used as a core technology in many fields such as electronics, optical communication, medicine, and national defense.

In addition, a LiDAR (Light Detection and Ranging, LiDAR) device uses a laser to measure the distance to an object, and has recently attracted attention as a core technology for autonomous vehicles, mobile robots, and drones.

Since the conventional lidar device applies the TOF (Time of Flight) method that calculates the distance through the flight time of the laser, it has the advantage of being able to measure with high precision and high resolution, but it is expensive, the overall size is large, etc. disadvantages exist.

An object of the present application is to provide a lidar device capable of image processing a detection image detected by a line laser to generate precise three-dimensional coordinate information for a target space.

An object of the present application is to provide a lidar device capable of performing more precise distance measurement by improving the resolution of a detection image.

An object of the present application is to provide a lidar device capable of preventing erroneous distance measurement due to a classification error with respect to a plurality of line lasers included in a detection image.

A lidar device according to an embodiment of the present invention relates to an image-based three-dimensional lidar (LiDAR) device, comprising: a laser light source unit for irradiating a line laser (line laser) in a target space; an image sensor unit for generating a detection image by detecting the line laser reflected in the target space; and an image processing unit generating a three-dimensional coordinate map for the target space by using the laser pattern displayed in the detection image, wherein the laser pattern is short-circuited from the reference pattern and the reference pattern in a vertical direction. It may include a target pattern of a spaced apart shape.

Here, the image processing unit extracts z-axis coordinates for each point included in the target pattern by using a height position value at which the target pattern is vertically spaced apart from the center line of the detection image, and the target pattern is The y-axis coordinates for each point included in the target pattern are extracted using a vertical position value spaced from the reference pattern in the vertical direction, and each point included in the target pattern moves horizontally from the reference point of the detection image. Extracting the x-axis coordinates of the respective points by using the spaced horizontal position values, and using the x-axis coordinates, y-axis coordinates, and z-axis coordinates to generate three-dimensional coordinates for each point in the target pattern have.

Here, the reference point may be a center point of the detection image, and the center line may be a horizontal line passing through the center point of the detection image.

Here, when the y-axis coordinate is extracted, the image processing unit may correct the y-axis coordinate using the horizontal position value, and specifically, the distance information to the point corresponding to the vertical position value and the horizontal position value is The y-axis coordinate can be corrected by using the stored table table.

Here, the image processing unit may extract pattern pixels corresponding to the laser pattern from the detection image, and divide the pattern pixels to add a plurality of virtual pixels.

Here, the image processing unit extracts pattern pixels corresponding to the laser pattern from the detection image and inserts a plurality of virtual pixels into the pattern pixels so that the virtual pixels form a vertical or horizontal layer within the pattern pixels. can be made to form.

Here, the image processing unit may generate a binarized image by binarizing the pattern pixel and the virtual pixel based on a threshold value, and extract the laser pattern from the binarized image.

Here, the image sensor unit may further include a lens unit for receiving the line laser, and the image processing unit may generate 3D coordinate information obtained by correcting distortion by the lens unit with reference to a correction table.

Here, the lidar device according to an embodiment of the present invention, the output control unit for controlling the emission timing and emission holding time for irradiating the line laser into the target space; and an exposure control unit configured to control an exposure timing and exposure holding time for detecting the line laser by exposing the image sensor unit.

Here, the output control unit may use a synchronization signal to match the exposure timing and the emission timing.

Here, the output control unit may control to emit the line laser for a first emission duration when an odd-numbered synchronization signal is received, and to emit the line laser for a second emission duration when an even-numbered synchronization signal is received.

Here, the output control unit, the first emission holding time may be consistent with the exposure holding time, and the second emission holding time may be maintained shorter than the exposure holding time.

Here, the image sensor unit generates a first detection image corresponding to the odd-numbered synchronization signal and a second detection image corresponding to the even-numbered synchronization signal, respectively, and the first detection image is higher than the second detection image. It can be generated by image processing to have a brightness gain value.

Here, when the number of line lasers detected in the first detection image is the same as the number of line lasers output from the laser light source unit, the image processing unit may Line numbers can be assigned.

Here, the image processing unit may classify the reference pattern and the target pattern and assign the line number to each.

Here, when the number of line lasers detected in the first detection image is less than the number of line lasers output by the laser light source unit, the image processing unit may refer to the second detection image to determine a line number for each of the line lasers. can be given

Here, the image processing unit, when the number of line lasers detected in the first detection image is less than the number of line lasers output from the laser light source unit, selects candidates for line numbers of the line lasers included in the first detection image After setting x and (x-1) respectively, the line number of the line laser identically detected in the second detection image is (x-1), and the line number of the line laser not detected in the detection image is x. Each can be given.

Incidentally, the means for solving the above problems do not enumerate all the features of the present invention. Various features of the present invention and its advantages and effects may be understood in more detail with reference to the following specific embodiments.

Since the lidar device according to an embodiment of the present invention can measure the distance in the target space using image processing instead of the TOF, it is possible to reduce the cost and reduce the product size required for manufacturing the lidar device.

Since the lidar device according to an embodiment of the present invention can generate distance information by further considering a horizontal position value in addition to a vertical position value of a laser pattern, it is possible to generate more accurate distance information.

Since the lidar device according to an embodiment of the present invention can increase the resolution of a detected image by using image processing, it is possible to extract a precise laser pattern and to generate accurate distance information using the same.

Since the lidar device according to an embodiment of the present invention can accurately distinguish a plurality of line lasers included in a detection image, it is possible to prevent erroneous distance measurement due to a classification error of the line lasers.

1 is a block diagram showing a lidar device according to an embodiment of the present invention.
2 is a schematic diagram illustrating emission of a line laser in a target space using a lidar device according to an embodiment of the present invention.
3 and 4 are schematic diagrams illustrating the generation of three-dimensional coordinates for a single-line laser using a lidar device according to an embodiment of the present invention.
5 and 6 are schematic diagrams illustrating the generation of 3D coordinates for a multi-line line laser using a lidar device according to an embodiment of the present invention.
7 and 8 are schematic diagrams showing distortion of a detected image by a lens.
9 is a schematic diagram showing correction for distortion of a detected image of a lidar device according to an embodiment of the present invention.
10 and 11 are schematic diagrams illustrating distance information generation using a lidar device according to an embodiment of the present invention.
12 and 13 are exemplary views showing generation of distance information using a lidar device according to an embodiment of the present invention.
14 is a schematic diagram illustrating virtual pixel generation of a lidar device according to an embodiment of the present invention.
15 is a schematic diagram illustrating detection of a laser pattern using virtual pixels of a lidar device according to an embodiment of the present invention.
16 is a schematic diagram illustrating distance information generation using a laser pattern of a lidar device according to an embodiment of the present invention.
17 and 18 are schematic diagrams illustrating a detection image according to a distance from an object in a target space.
19 is a block diagram illustrating a line laser emission timing and emission holding time of a lidar device according to an embodiment of the present invention.
20 is a block diagram illustrating generation of a first detection image and a second detection image of the lidar device according to an embodiment of the present invention.
Fig. 21 is an exemplary diagram showing line number setting for each line laser when the number of detected line lasers is the same as the number of emitted line lasers.
Fig. 22 is an exemplary diagram showing line number setting for each line laser when the number of detected line lasers is different from the number of emitted line lasers.

Hereinafter, preferred embodiments will be described in detail so that those of ordinary skill in the art can easily practice the present invention with reference to the accompanying drawings. However, in describing the preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and functions.

In addition, throughout the specification, when a part is 'connected' with another part, it is not only 'directly connected' but also 'indirectly connected' with another element interposed therebetween. include In addition, 'including' a certain component means that other components may be further included, rather than excluding other components, unless otherwise stated. In addition, terms such as "~ unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

1 is a block diagram illustrating a lidar device according to an embodiment of the present invention, and FIG. 2 is a schematic diagram illustrating emission of a line laser in a target space using a lidar device according to an embodiment of the present invention.

1 and 2 , the lidar device 100 according to an embodiment of the present invention includes a laser light source unit 110 , an image sensor unit 120 , an image processing unit 130 , an output control unit 140 , and exposure. A control unit 150 may be included.

Hereinafter, the lidar device 100 according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2 .

The laser light source unit 110 may irradiate a line laser in the target space (A). The laser light source unit 110 may output a single line laser or a plurality of line lasers in the target space A, and according to an embodiment, it is also possible to irradiate by adjusting the projection angle of the output line laser.

Here, the wavelength of the line laser may be preset, and it is also possible to set the wavelength of the line laser to an infrared wavelength region according to an embodiment. That is, by irradiating the line laser of infrared rays, it is possible to prevent people located in the target space A from recognizing the line laser.

In addition, line lasers can be energized to meet Eye-safety 1 class standards. For example, the laser light source unit 110 may emit laser light in a pulsed form according to a preset energy design, thereby reducing the risk of accidents such as eye injuries caused by the line laser emitted in the target space (A). can be minimized

On the other hand, the target space (A) to which the line laser is output may be a security area such as a house or a store, or a space around a vehicle being driven or parked. However, the present invention is not limited thereto, and any space may correspond to a space in which the lidar device 100 is installed and irradiating a line laser.

The image sensor unit 120 may include a plurality of pixels, and may generate a detection image by detecting line lasers reflected in the target space A as pixel values of the pixels. For example, the image sensor unit 130 may be implemented as a sensor array such as a CCD (Charge-Coupled Device) sensor or a CMOS (Complementary Metal-Oxide Semiconductor) sensor, and electrically transmits light detected in the target space (A). can be converted into a signal. That is, a two-dimensional detection image may be generated by setting the luminance value of the line laser reflected in the target space A to the pixel value of each pixel.

The image processing unit 130 may generate a three-dimensional coordinate map for the target space A by using the laser pattern displayed in the detection image. Specifically, the image processing unit 130 may measure the distance to each point in the target space A using the geometric distance and angle between the laser light source unit 110 and the image sensor unit 120 . . Here, the image processing unit 130 may use a triangulation technique, and may set three-dimensional coordinates for each point based on this. Thereafter, the image processing unit 130 may generate a three-dimensional coordinate map for the target space A by collecting the three-dimensional coordinates for each laser pattern.

However, when the object S is present in the target space A, the laser pattern in the detection image i may be displayed as a reference pattern and a target pattern short-circuited from the reference pattern and vertically spaced apart. That is, as shown in FIG. 3 , when the object S is located in the target space A, the laser pattern may appear in a form including the target pattern Lb spaced apart from the reference pattern La. . In this case, it may be difficult to measure the distance to the object S using the triangulation method, and the image processing unit 130 uses the relationship between the target pattern Lb and the reference pattern La, 3D coordinates can be set.

Specifically, as shown in FIG. 3( a ), when the line laser is horizontally irradiated into the target space, the reference pattern La is located at the center of the detection image i. Here, the height position value of the laser pattern may be based on the position of the center line RL of the detection image i, and in the case of FIG. The value corresponds to 0. Accordingly, the z-axis coordinate of the reference pattern La may also be set to 0.

Thereafter, when the object S is located in the target space as shown in FIG. 3B , the target pattern Lb may appear in the detection image i while being spaced apart from the reference pattern La. However, since the laser light source unit 110 horizontally outputs the line laser in the actual target space, the height position values corresponding to the line laser are all the same. That is, the interval between the reference pattern La and the target pattern Lb appears in the detection image i according to the interval or angle between the laser light source unit 110 and the image sensor unit 120 . Accordingly, the height position value of the target pattern Lb shown in FIG. 3(b) may also be set to 0, and accordingly, the z-axis coordinate of the object S may also be set to 0. In addition, as shown in Figure 3 (b), when the object (S) is located in the target space and the target pattern (Lb) appears, the distance from the reference point (RP) to the horizontally spaced target pattern (Lb) It can be measured and set as a horizontal position value. For example, based on the central point of the target pattern (Lb), it is possible to measure the horizontal separation distance from the reference point (RP). In the case of Figure 3 (b), since the horizontal distance from the reference point (RP) to the center point of the target pattern (Lb) corresponds to 0, the horizontal position value of the target pattern (Lb) can be set to 0. In addition, since the horizontal position value is 0, it can be determined that the object S is located at the center in the detection image i.

In addition, according to the embodiment, horizontal position values spaced apart from the reference point RP in the horizontal direction may be measured for each point included in the target pattern Lb, and in this case, the horizontal position values are at each point. corresponding to the x-axis coordinates for Here, the case where the reference point RP is the center point RP of the detection image i has been described as an example, but according to embodiments, it is also possible to set various points other than the center point as the reference point RP.

On the other hand, the distance between the lidar device 100 and the object S is proportional to the vertical position value P at which the target pattern Lb in the detection image i is vertically spaced apart from the reference pattern La. Accordingly, the image processing unit 130 may calculate the distance D1 to the object S by using the vertical position value p. Here, a function or table for obtaining the distance value to the object S corresponding to each vertical position value p may be preset, and the image processing unit 130 uses this to determine the vertical position value ( You can set the y-axis coordinates from P).

Accordingly, the image processing unit 130 may generate a three-dimensional coordinate value for each point included in the corresponding laser pattern by using the vertical position value, the horizontal position value, and the height position value.

In addition, in order to generate a three-dimensional coordinate map for the target space (A), the laser light source unit 110 may irradiate a line laser to various positions in the target space (A). As in a), it is possible to irradiate the line laser by changing the light emission angle. That is, as shown in Fig. 4(a), in the case of irradiating the line laser in the target space at the set light projection angle θ, the reference pattern La is lower than the center line RL of the detection image i. can be located in In this case, the image processing unit 130 may generate each three-dimensional coordinate value by further applying the projection angle θ of the line lasers to the vertical position value, the horizontal position value, and the height position value.

Specifically, when the object S is located in the target space A as shown in Fig. 4(b), the distance D2 between the lidar device 100 and the object S is, in the detection image i The target pattern Lb is proportional to the vertical position value p that is vertically spaced from the reference pattern La. Accordingly, the image processing unit 130 may calculate the distance D2 to the object S by using the vertical position value p. Here, a function or table table for obtaining the distance D2 to the object S corresponding to each vertical position value p may be preset, and the image processing unit 130 uses this to determine the vertical position The distance D2 from the value p to the object S can be calculated. Thereafter, the y-axis coordinate of the object S may be obtained as D2*cos(θ) by reflecting the light transmission angle θ.

In addition, the height position value may be set based on the center line RL of the detection image i, and the interval between the reference pattern La and the center line RL may be calculated as the height position value. Here, since the height value (z-axis coordinate) at which the reference pattern La is located is proportional to the interval between the reference pattern La and the center line RL, the height value up to the reference pattern La corresponding to the height position value A function or table table for obtaining ? may be preset. The image processing unit 130 may use this to set the z-axis coordinate from the height position value.

However, as shown in Fig. 4(b), since the object S is located at the upper end in the z-axis direction than the reference pattern La, in order to obtain the z-axis coordinates of the object S, the reference pattern La is It is necessary to perform additional calculations on the z-axis coordinates. For example, the z-axis coordinates of the object S may be obtained using D2*cos(θ)*tan(θ), and the z-axis coordinates of the object S may be obtained using various other methods.

Additionally, depending on the embodiment, there may be a case where the light transmission angle θ is not given. In this case, the image processing unit 130 derives the z-axis coordinates of the object s by using the height position value C at which the target pattern Lb is vertically spaced from the center line RL of the detection image i. can do. That is, since the height value (z-axis coordinate) at which the object S is located is proportional to the height position value C, a function for obtaining the z-axis coordinate of the object S corresponding to each height position value C Alternatively, a table table may be preset. Accordingly, the image processing unit 130 may obtain the z-axis coordinate of the object S from the height position value C using this. In this case, it is possible to accurately obtain height information compared to a method in which the height of the center line RL is designated as an arbitrary constant (eg, 0).

On the other hand, the horizontal position value can be measured in the same way as in the case of horizontally irradiating the line laser. That is, the target pattern Lb from the reference point RP may measure a horizontal position value spaced apart in the horizontal direction to measure a measurement distance, and in this case, the reference point RP may be the center point of the detection image i. Here, since the horizontal position value corresponds to the x-axis coordinate, it is possible to set the x-axis coordinate for each point by using the horizontal position values of each point included in the target pattern Lb.

As described above, the image processing unit 130 may generate 3D coordinate values for each point included in the laser pattern while changing the light transmission angle. That is, the image processing unit 130 may calculate each of the three-dimensional coordinates for the target space A by using the laser pattern extracted from the detected image, and may generate a three-dimensional coordinate map based on this.

In summary, the image processing unit 130 uses a height position value at which the target pattern Lb is vertically spaced from the center line RL of the detection image i for each point included in the target pattern Lb. You can extract the z-axis coordinates. In addition, the y-axis coordinates for each point included in the target pattern Lb may be extracted using the vertical position value P at which the target pattern Lb is vertically spaced from the reference pattern La. Thereafter, the x-axis coordinates of each point may be extracted using a horizontal position value at which each point included in the target pattern Lb is horizontally spaced apart from the reference point RP of the detection image i. When the x-axis coordinates, the y-axis coordinates, and the z-axis coordinates are extracted, the image processing unit 130 may generate three-dimensional coordinates for each point of the target pattern Lb.

On the other hand, FIG. 4 shows a case in which a single line laser is irradiated in the target space A while changing the projection angle, but depending on the embodiment, as shown in FIG. 5 , it is also possible to simultaneously irradiate a multi-line laser. do. That is, as shown in FIGS. 5 (a) and 5 (b), the lidar device 100 can irradiate four line lasers at the same time, and each line laser can be reflected from a flat wall. have. Here, when comparing the four endpoints where the four line lasers appeared, it can be confirmed that only the z coordinates (z1, z2, z3, z4) indicating the height are different, and the remaining x and y coordinates are the same because it is irradiated to a uniform wall. can That is, when irradiating a multi-line laser, it is necessary to calculate the z-coordinate for the height of each line laser.

When irradiating a multi-line laser as shown in FIG. 5 , the x-axis coordinate may be obtained from a horizontal position value of the target pattern Lb of each line laser. That is, it can be obtained in the same way as in the case of a single-line laser. The y-axis coordinate can be obtained from the vertical position value between the reference pattern and the target pattern of each line laser. That is, each reference pattern and a target pattern corresponding to each line laser may be distinguished, and a y-axis coordinate corresponding to each of the line lasers may be obtained using a vertical position value between the reference pattern and the target pattern. In the case of the Z-axis coordinate, as in the case of single-line lasers, it can be obtained from the light projection angle of each line laser and the distance between the reference pattern and the center line, or it can be calculated from the height position value between the target pattern and the center line.

Specifically, referring to FIG. 6( a ), a reference pattern and a target pattern for each line laser are set, and vertical position values ( P1 , P2 , P3 ) between each reference pattern and the target pattern can be obtained. . Thereafter, the y-axis coordinates of the respective target patterns may be calculated from the respective vertical position values P1, P2, and P3. In addition, referring to Figure 6 (b), after calculating the height position values (C1, C2, C3) between each target pattern and the center line (RL), each of the height position values (C1, C2, C3) You can set the corresponding z-axis coordinates.

That is, when irradiating a multi-line laser, by repeatedly applying the same method as a single-line laser to each line laser, three-dimensional coordinates for each point can be extracted.

Additionally, according to an embodiment of the present invention, the image sensor unit 120 may receive the line laser reflected in the target space A through the lens. That is, the image sensor unit 120 may collect and receive light through a lens similar to a camera in order to increase reception efficiency.

In this case, as shown in FIG. 7 , the shape of the laser pattern included in the detection image i may be distorted by the lens. That is, it can be seen that the distance between the center line RL and the line laser differs by 20 pixels at the end, but by 30 pixels at the center. Furthermore, as shown in Fig. 8(a), when a difference in the distance between the end and the center of the line laser occurs, as shown in Fig. 8(b), although the wall on which the line laser is actually reflected is flat, , may be mistakenly recognized as a curved shape.

Accordingly, the image processing unit 130 according to an embodiment of the present invention may correct distortion due to a lens included in the detection image i by using the correction table. That is, after pre-stored in the correction table the degree of distortion caused by the lens or the like, it is possible to correct the distortion based on this. For example, a difference between a non-distorted straight line laser and a line laser bent by distortion may be stored in the correction table, and the image processing unit 130 uses the correction table to generate the detected image (i). ) can be corrected so that the curved line laser in the inside is spread out. In this case, the distorted detection image (i) of FIG. 9(a) may be corrected in a straight line form as shown in FIG. 9(b). Then, by using the corrected detection image (i), it is possible to generate a three-dimensional coordinate map to appear as a flat wall shape as shown in Fig. 9(c). Therefore, after correcting the detected image (i) distorted by the lens, it is possible to generate an accurate three-dimensional coordinate map for the target space (A).

Also, according to an embodiment of the present invention, when extracting the y-axis coordinates, the image processing unit 130 may correct the y-axis coordinates by using a horizontal position value in addition to a vertical position value. That is, by further considering the position of the object on the x-axis when measuring the distance of the object, more accurate distance information can be measured.

Specifically, since the image processing unit 130 generates distance information to an object through image processing for the detected image, as the distance between the object and the lidar device 100 increases, the corresponding one pixel in the detected image increases. A phenomenon in which the actual distance increases may occur. Accordingly, when the distance between the lidar device 100 and the object is long, it may be difficult to accurately distinguish the distance to the object, and even a small error occurring in a laser pattern, etc. may greatly change the distance information about the object. That is, when the distance to the object is calculated using only the vertical position value of the target pattern displayed in the detection image, the calculated distance information to the object may be relatively inaccurate.

For example, the distance D1 to the first object S1 of FIG. 10 may be 3m, and the distance D2 to the second object S2 may be 4m. However, in the detection image i, the vertical position values p1 and p2 of the first object S1 and the second object S2 may be equally measured as shown in FIG. 11 . That is, since the vertical position values p1 and p2 are the same, despite the difference in distance between the actual first object S1 and the second object S2, the lidar device 100 is measured as being located at the same distance. problems such as

In order to solve this problem, the image processing unit 130 according to an embodiment of the present invention may more accurately generate distance information to the object by reflecting the horizontal position value together with the vertical position value of the target pattern Lb.

11 is a detection image of each of the first object S1 and the second object S2 of FIG. 10, and laser patterns La and Lb corresponding to the line laser may be included in the detection image i. , the laser pattern may be shorted by the objects S1 and S2. Here, since the first object S1 is located in front of the lidar device 100 , the target pattern Lb may be located in the center of the detection image i as shown in FIG. 11A . On the other hand, since the second object S2 is biased to the right from the lidar device 100, as shown in FIG. 11(b), the target pattern Lb is also located to the right from the center of the detection image i. can be checked.

In this case, the image processing unit 130 determines that the target pattern Lb is vertically spaced apart from the reference pattern La in vertical position values p1 and p2, and the target pattern Lb is horizontally spaced apart from the reference point in the horizontal direction. Position values (x1, x2) may be calculated, respectively, and distance information to the objects (S1, S2) may be generated using vertical position values (p1, p2) and horizontal position values (x1, x2), respectively. Here, the reference point may be set to an arbitrary point such as the left end, right end, or center point RP of the detection image (i) in the detection image (i). ), it is also possible to set a point corresponding to the position of the

Here, there may exist a table in which distance information corresponding to the vertical position values p1 and p2 and the horizontal position values x1 and x2 of the target pattern Lb is stored, and the image processing unit 130 uses them to Distance information corresponding to the objects S1 and S2 may be extracted. According to an embodiment, it is also possible to generate a table table by using machine learning or deep learning. In the case of FIG. 11 , although the vertical position values p1 and p2 of the first object S1 and the second object S2 are identically measured, the horizontal position values x1 and x2 are different, so each of the first objects ( Distance information for S1) and the second object S2 may be set differently. Accordingly, it is possible to more accurately generate distance information of an object.

Meanwhile, FIGS. 12 and 13 are diagrams illustrating distance information calculation by reflecting both the vertical position value and the horizontal position value of the target pattern Lb according to an embodiment of the present invention. 12 is a distance calculation result for an object S located at a distance of 100 cm, 150 cm, 200 cm, and 250 cm from the lidar device 100, respectively. Here, as the distance from the lidar device 100 increases, the size of the vertical position value of the target pattern Lb decreases, and accordingly, it can be confirmed that it is difficult to gradually extract accurate distance information to the object S only with the vertical position value. have.

13 is a diagram illustrating the calculation of distance information when the object S moves along a circle with a radius of 2 m of the lidar device 100 . Referring to FIG. 13 , it can be confirmed that, despite the change in the horizontal position value of the object S, the distance information to each object S is equally calculated to be 2m. That is, since the horizontal position value and the vertical position value for the target pattern Lb can be considered at the same time, it is possible to more accurately calculate the distance information.

Meanwhile, the image processing unit 130 according to an embodiment of the present invention may perform image processing on the detected image i to increase the resolution of the laser pattern included in the detected image i.

The detection image (i) may include laser patterns (La, Lb) corresponding to the line laser, and when a preset table is applied to each pixel representing the laser pattern (La, Lb), the distance for each pixel information can be calculated. Here, the table may be preset using a triangulation method.

However, due to the hardware performance of the laser light source unit 110 and the image sensor unit 120 and noise included in the detected image i, the laser pattern may vary for every frame, and accordingly, Distance information may change. The variation of the distance information has a small variation when the distance to the object S is short, but may appear larger as the distance to the object S increases. For example, when the distance to the object S is 200 cm, the actual distance difference due to 1 pixel difference of the laser pattern may appear about 5 cm, but when the distance to the object S is 700 cm, the laser pattern The difference in the actual distance by 1 pixel can be as large as 50 cm.

This occurs because the boundary of the laser pattern is unclear, and can be resolved when the boundary of the laser pattern is clearly displayed in the detection image (i). A method of simply applying an image sensor supporting a high resolution to the image sensor unit 120 or increasing the separation distance between the laser light source unit 110 and the image sensor unit 120 may be considered, but this has a physical limitation. , here we propose a method of improving the resolution through image processing.

Specifically, as shown in FIG. 14( a ), a pattern pixel Pa representing a laser pattern in general can be specified, and a laser pattern can be extracted using this. However, the image processing unit 130 according to an embodiment of the present invention may add a plurality of virtual pixels Pb by dividing the pattern pixels representing the laser pattern as shown in FIG. 14(b) . That is, the laser pattern can be displayed more precisely by using the virtual pixel Pb.

The virtual pixels Pb may be added to form a layer in a vertical direction or a horizontal direction within the pattern pixel Pa, and the pixel value of the virtual pixels Pb is the pixel value of other pattern pixels Pa located in the periphery. can be set using For example, the pixel value may be set as an average value of pixels located above, below, or left and right of the virtual pixel Pb, and in addition, various filling methods may be used.

Thereafter, as shown in FIG. 15 , the image processing unit 130 may binarize the pattern pixel Pa or the virtual pixel Pb based on the threshold value, and may extract a laser pattern from the binarized image. In general, as shown in FIGS. 14A and 15A , a corresponding laser pattern can be extracted by binarizing it based on a threshold value of 0.75 without adding a virtual pixel.

However, the image processing unit 130 according to an embodiment of the present invention may add virtual pixels Pb as shown in FIGS. 14(b) and 15(b), and a threshold value for the virtual pixel Binarization can be performed based on 0.75. In this case, the first and second rows of the virtual pixels are both set to “1” and extracted as a laser pattern, but the third row may be excluded from the laser pattern because both are set to “0”. That is, compared with FIG. 15( a ), it can be seen that in FIG. 15 ( b ), the laser pattern is set more precisely after increasing the resolution of the laser pattern by adding virtual pixels.

Specifically, referring to Fig. 16, Fig. 16 (a) is a result of generating distance information to an object (S) by extracting a laser pattern without adding a virtual pixel, and Fig. 16 (b) is a result of generating a distance information by adding a virtual pixel This is the result of generating distance information to the object S after extracting the laser pattern. According to Fig. 16 (a), distance information to the object (S) can be calculated based on the 100th row of the detection image (i). It is possible to calculate distance information to the object S based on the row. Through this, it can be confirmed that distance information to the object S can be more precisely calculated.

On the other hand, referring to FIGS. 17A and 17B , when the line laser is irradiated into the target space A using the lidar device 100 , the line laser is irradiated according to the position of the object S. The forming height may vary. However, as shown in FIG. 17 , when a plurality of line lasers are simultaneously irradiated, it is necessary to distinguish each line laser, and depending on the embodiment, it is possible to distinguish by assigning a line number to each line laser. do. Thereafter, distance information may be calculated according to a vertical position value of the target pattern of the line lasers corresponding to each line number compared to the reference line or the reference pattern.

However, in the case of using a plurality of line lasers, since the position of the line laser changes according to the distance in the detection image, the distinction from other line lasers may not be clear. That is, as shown in FIG. 18, depending on the position of the object S, the line laser may rise above the reference line, and in some cases, one of the line lasers is not focused in the detection image (i). can occur In this case, it is not possible to know which line laser corresponds to the three lines corresponding to the target pattern, and an error in the distance calculation result may occur due to incorrect line number assignment. In order to solve this problem, the lidar device 100 according to an embodiment of the present invention may utilize the output control unit 140 and the exposure control unit 150 .

Here, the output control unit 140 can control the emission timing and the emission holding time of the line laser output from the laser light source unit 110, and the emission timing of the output control unit 140 is the image sensor unit 120 by the synchronization signal. can match the exposure timing of In addition, the exposure control unit 150 may control the exposure timing and the exposure holding time of the image sensor unit 120 . The image sensor unit 120 may start exposure according to the synchronization signal, and may generate a detection image (i) by detecting light reflected in the target space (A) during the exposure holding time.

Referring to FIG. 19 , the output control unit 140 according to an embodiment of the present invention may receive a synchronization signal for generating each detected image i from the image sensor unit 120, and receive each synchronization signal. It can be divided into an odd-numbered synchronization signal and an even-numbered synchronization signal.

Thereafter, when an odd-numbered synchronization signal is received, the line laser is emitted for the first emission duration T1, and when an even-numbered synchronization signal is received, the line laser can be controlled to be emitted for a second emission duration T2. Here, the first emission holding time (T1) may be set longer than the second emission holding time (T2). According to the embodiment, it is also possible to make the first emission holding time (T1) coincide with the exposure holding time, and to keep the second emission holding time (T2) shorter than the exposure holding time.

If the output control unit 140 sets the emission holding time to be long, the line laser can be emitted with sufficient energy to be reflected from the wall located at the end of the target space (A) and return. It can be emitted with a small amount of energy that can be reflected and returned only from the object S located in a short distance. Therefore, by setting the first emission holding time T1 and the second emission holding time T2 differently, it is possible to separate the detection range into a long distance and a short distance, respectively, that is, the first emission holding time T1. During the detection of objects located at a long distance, during the second emission holding time T2, it is possible to detect objects located at a short distance.

In addition, as shown in FIG. 20 , the image sensor unit 120 generates a first detection image corresponding to an odd-numbered synchronization signal and a second detection image corresponding to an even-numbered synchronization signal, respectively, and then Different ISP (Image Signal Processor) processing may be applied to the first detection image and the second detection image. That is, the first detection image may be generated by image processing to have a higher brightness gain value than that of the second detection image. In this case, in the second detection image, since the line laser reflected from a distance is displayed darker, the line laser reflected from a short distance may be highlighted.

Thereafter, the image processing unit 140 may detect the line lasers in the first detection image and check the number of line lasers. Here, when the detected number of line lasers matches the number of line lasers output from the laser light source unit 110 , each line number may be assigned in an ascending order from the line laser positioned at the bottom in the first detection image. That is, as shown in FIG. 21 , when the number of line lasers output from the laser light source unit 110 is 4 and the number of line lasers detected in the first detection image i1 is also 4, the lowest line laser Line numbers can be set to 1, 2, 3, 4 in sequence from

Here, the image processing unit 140 may separate the line laser into a reference pattern and a target pattern, respectively, and then may individually assign a line number to each of the reference pattern and the target pattern.

On the other hand, if the number of detected line lasers is smaller than that of the line lasers output from the laser light source unit 110 , the image processing unit 140 may assign a line number to each of the line lasers with reference to the second detection image. That is, as shown in FIG. 22( a ), when the number of line lasers output from the laser light source unit 110 is 4 and the number of detected line lasers is 3, line lasers are referenced to the second detection image i2 . Line numbers of lasers can be assigned.

Specifically, candidates for line numbers of the line lasers included in the first detection image (i) may be set to x and (x-1), respectively, and among them, the line lasers identically detected in the second detection image (i2) (x-1) can be set as the line number. On the other hand, the line number of the line laser that is not detected in the second detection image i2 may be assigned as x.

Referring to FIG. 22 , each of the line lasers included in the first detection image i1 may have 1, 2, 3 or 2, 3, 4 line number candidates. Here, since the target pattern portion is detected in the same manner in the second detection image i2, (x-1) may be set as a line number for each target pattern portion. Accordingly, the target pattern portion can set the line number to 1, 2, and 3 instead of 2, 3, and 4, respectively. In addition, since the remaining reference pattern portion is not detected in the second detection image i2, (x) may be set as the line number. Accordingly, the reference pattern portion can set the line number to 2, 3, and 4, respectively.

The present invention is not limited by the above embodiments and the accompanying drawings. For those of ordinary skill in the art to which the present invention pertains, it will be apparent that the components according to the present invention can be substituted, modified and changed without departing from the technical spirit of the present invention.

100: lidar device 110: laser light source unit
120: image sensor unit 130: image processing unit
140: output control unit 150: exposure control unit

Claims (18)

In the image-based three-dimensional LiDAR (LiDAR) device,
A laser light source for irradiating a line laser (line laser) in the target space;
an image sensor unit for generating a detection image by detecting the line laser reflected in the target space; and
An image processing unit for generating a three-dimensional coordinate map for the target space by using the laser pattern displayed in the detection image,
The laser pattern is
A lidar device comprising a reference pattern and a target pattern having a shape that is short-circuited from the reference pattern and spaced vertically from the reference pattern.
The method of claim 1, wherein the image processing unit
extracting z-axis coordinates for each point included in the target pattern by using the height position value of the target pattern vertically spaced apart from the center line of the detection image,
Extracting the y-axis coordinates for each point included in the target pattern by using the vertical position value of the target pattern spaced from the reference pattern in the vertical direction,
Extracting the x-axis coordinates of each point included in the target pattern by using a horizontal position value that is horizontally spaced apart from the reference point of the detection image,
Using the x-axis coordinates, y-axis coordinates and z-axis coordinates, lidar device, characterized in that for generating three-dimensional coordinates for each point in the target pattern.
3. The method of claim 2,
The reference point is any one of the center point, the left end and the right end of the detection image,
The center line is a lidar device, characterized in that it is a horizontal line passing through the center point of the detection image.
The method of claim 2, wherein the image processing unit
When the y-axis coordinate is extracted, the lidar device, characterized in that the y-axis coordinate is corrected by further using the horizontal position value.
5. The method of claim 4, wherein the image processing unit
LiDAR apparatus, characterized in that the y-axis coordinate is corrected by using a table in which distance information to the point corresponding to the vertical position value and the horizontal position value is stored.
The method of claim 1, wherein the image processing unit
Extracting pattern pixels corresponding to the laser pattern from the detection image, and dividing the pattern pixels to add a plurality of virtual pixels.
The method of claim 1, wherein the image processing unit
extracting pattern pixels corresponding to the laser pattern from the detection image and inserting a plurality of virtual pixels into the pattern pixels so that the virtual pixels form a layer in a vertical direction or a horizontal direction within the pattern pixels a lidar device.
The method of claim 6 or 7, wherein the image processing unit
A binarization of the pattern pixel and the virtual pixel based on a threshold value to generate a binarized image, and extracting the laser pattern from the binarized image.
According to claim 1,
The image sensor unit further comprises a lens unit for receiving the line laser,
The image processing unit
With reference to the correction table, the lidar device, characterized in that for generating three-dimensional coordinate information corrected by the lens unit.
According to claim 1,
an output control unit for controlling an emission timing and an emission holding time of irradiating the line laser into the target space; and
LiDAR device, characterized in that it further comprises an exposure control unit for controlling the exposure timing and exposure holding time for detecting the line laser by exposing the image sensor unit.
11. The method of claim 10, wherein the output control unit
Using a synchronization signal, the lidar device, characterized in that the exposure timing and the emission timing coincide.
12. The method of claim 11, wherein the output control unit
When an odd-numbered synchronization signal is received, the line laser is emitted for a first emission duration, and when an even-numbered synchronization signal is received, the line laser is controlled to be emitted for a second emission duration.
13. The method of claim 12, wherein the output control unit
The first emission holding time coincides with the exposure holding time, and the second emission holding time is maintained shorter than the exposure holding time.
14. The method of claim 13, wherein the image sensor unit
generating a first detection image corresponding to the odd-numbered synchronization signal and a second detection image corresponding to the even-numbered synchronization signal, respectively;
The first detection image is generated by image processing so as to have a higher brightness gain value than that of the second detection image.
15. The method of claim 14, wherein the image processing unit
When the number of line lasers detected in the first detection image is the same as the number of line lasers output from the laser light source unit, each line number is assigned in ascending order from the line laser located at the bottom of the first detection image. Features a lidar device.
16. The method of claim 15, wherein the image processing unit
The lidar device, characterized in that by distinguishing the reference pattern and the target pattern, and assigning the line number to each.
The method of claim 16, wherein the image processing unit
When the number of line lasers detected in the first detection image is less than the number of line lasers output from the laser light source unit, a line number is assigned to each of the line lasers with reference to the second detection image lidar device.
18. The method of claim 17, wherein the image processing unit
When the number of line lasers detected in the first detection image is less than the number of line lasers output from the laser light source unit, candidates for line numbers of the line lasers included in the first detection image are respectively x, (x- After setting to 1), the line number of the line laser identically detected in the second detection image is (x-1), and the line number of the line laser that is not detected in the second detection image is given as x, respectively. Features a lidar device.

Images