KR102059244B1 - Apparatus for Light Detection and Ranging - Google Patents

Abstract

The present invention relates to a lidar device for measuring the distance, direction, speed, etc. to the object by analyzing the laser beam reflected by the object by irradiating the laser beam to the object,
Lidar device according to an embodiment of the present invention,
A laser module for emitting a laser beam toward an object; A camera sensor module configured to receive a laser beam emitted from the laser module and reflected by the object and installed in the moving object; And a controller configured to calculate a distance from the object by using the laser beam received from the camera sensor module.

Description

Lidar device {Apparatus for Light Detection and Ranging}

The present invention relates to a lidar apparatus for measuring a distance, a direction, a speed, and the like to an object by analyzing a laser beam reflected by the object by irradiating the laser beam to the object.

A laser (Light Amplification by the Stimulated Emission of Radiation, LASER) generates a stimulated emission of light and amplifies the laser light.

LiDAR (Light Detection and Ranging, LiDAR) is a technology that measures the distance using a laser, and has been developed in the form of constructing and visualizing terrain data for constructing three-dimensional Geographic Information System (GIS) information. It is applied to the field of defense.

Recently, it is attracting attention as a core technology while being applied to autonomous vehicles, mobile robots, and drones. When applied to a vehicle, the rider may perform warning or automatic vehicle control by measuring the distance between vehicles in real time so that the driving vehicle may avoid collision with the front vehicle or minimize impact.

1. Republic of Korea Patent Registration No. 10-1296780 2. Korean Patent Registration No. 10-1491289

An object of the present invention is to provide a lidar device that can reduce the installation cost by using a camera pre-installed on the moving object.

Lidar device according to an embodiment of the present invention,

A laser module for emitting a laser beam toward an object; A camera sensor module configured to receive a laser beam emitted from the laser module and reflected by the object and installed in the moving object; And a controller configured to calculate a distance from the object by using the laser beam received from the camera sensor module.

In the lidar apparatus according to an embodiment of the present invention, the movable body may be a vehicle equipped with a photographing apparatus including a camera or an unmanned aerial vehicle equipped with a camera or a mobile robot provided with a camera.

In the lidar apparatus according to an embodiment of the present invention, the movable body may be a vehicle equipped with a black box or an around view including a camera.

In the lidar apparatus according to an embodiment of the present invention, the laser module, may be installed in at least one or more of the position of the direction indicator light of the front lamp, rear lamp, both side mirrors of the vehicle.

In the lidar apparatus according to an embodiment of the present invention, the laser module is integrally formed on one side of the front lamp or the rear lamp so that the optical axis of the laser module is parallel to the irradiation direction of the front lamp or the rear lamp. Can be.

In the lidar apparatus according to an embodiment of the present invention, the control unit, the image of the image captured by the camera sensor module and the laser beam is reflected from the object and the laser beam image received from the laser beam image received pixel The apparatus may include a pixel position detector detecting position information, and a distance calculator configured to calculate a distance from the object using position information of the detected pixels.

In the lidar apparatus according to the embodiment of the present invention, the pixel position detector may be in synchronization with the laser module to collect the laser beam image according to the on (on) / off (off) of the laser module.

In the lidar apparatus according to the embodiment of the present invention, the pixel position detection unit, when the intensity of light intensity of a specific area in the laser beam image is greater than the intensity of light intensity of another area, the pixel area detection unit uses the corresponding area as the position information of the pixel. Can be detected.

In the lidar apparatus according to the embodiment of the present invention, the pixel position detector may obtain the laser beam image by removing a correction image correcting the image image from the image including the laser beam image.

In the lidar apparatus according to an embodiment of the present invention, the corrected image is obtained by dividing an exposure time t2 of an image including the laser beam image by an exposure time t1 of the image image with respect to the image image. Can be obtained by multiplying by value t2 / t1.

In the lidar apparatus according to an embodiment of the present invention, the camera sensor module, the electronic optical filter for transmitting the laser beam image and block the image image, the electronic optical filter is synchronized with the laser module It may be turned on / off according to the on / off of the laser module.

In a lidar apparatus according to an embodiment of the present invention, the pixel position detector includes an electronic filter synchronized with the laser module, wherein the electronic filter is synchronized with the laser module to turn on / off the laser module. It can be turned on / off according to the off.

In the lidar apparatus according to the embodiment of the present invention, the camera sensor module may include an image photographing image sensor for obtaining an image image and a distance measuring image sensor for obtaining a laser beam image.

In the lidar apparatus according to an embodiment of the present invention, in the camera sensor module, any one region among the image sensors constituting one camera sensor module is divided into the image capturing image sensor, and the other region is used for the distance measurement. It can be manufactured by dividing into an image sensor.

In the lidar apparatus according to an embodiment of the present invention, the camera sensor module may be manufactured by combining an image photographing image sensor for obtaining an image image and a distance measuring image sensor for obtaining a laser beam image.

In the lidar apparatus according to an embodiment of the present invention, the distance measuring image sensor may further include an optical filter for enhancing the resolution by blocking the non-laser image from the laser beam image.

Other specific details of embodiments according to various aspects of the present invention are included in the following detailed description.

According to the lidar apparatus according to the embodiment of the present invention, by using the camera pre-installed on the moving body as a camera sensor module, it is possible to reduce the installation cost of the lidar apparatus, such as a vehicle, a drone, a robot.

1 is a block diagram illustrating a lidar apparatus according to an embodiment of the present invention.
2A and 2B are views illustrating a state in which a lidar device is installed in a vehicle according to an embodiment of the present invention.
3 is a plan view of FIG. 2A.
4 is a block diagram illustrating a controller of a lidar apparatus according to an embodiment of the present invention.
5 is a photograph for describing an operation process of a pixel position detector.
6 is a view showing an application example of the camera sensor module of the lidar apparatus according to an embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present invention, the terms 'comprise' or 'have' are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof. Hereinafter, a lidar apparatus according to an embodiment of the present invention will be described with reference to the drawings.

The lidar device of the present invention is a lidar device that can reduce the installation cost by using a camera pre-installed on the moving body as a camera sensor module. The moving object may be, for example, a vehicle in which a photographing apparatus including a camera is installed, such as a black box or an around view (also called an “omni view”). In the following description, a case where the lidar apparatus of the present invention is applied to a vehicle will be described by way of example. Of course, the present invention is not necessarily limited to a vehicle, and may be applied to various moving objects such as an unmanned aerial vehicle (drone) equipped with a camera or a mobile robot provided with a camera.

1 is a block diagram showing a lidar apparatus according to an embodiment of the present invention, Figures 2a and 2b is a view illustrating a state in which a lidar apparatus is installed in a vehicle according to an embodiment of the present invention, 3 is a plan view of FIG. 2A, and FIG. 4 is a block diagram illustrating a control unit of a lidar apparatus according to an embodiment of the present invention.

As shown in FIG. 1, a Lidar apparatus according to an embodiment of the present invention includes a laser module 100, a camera sensor module 200, and a controller 300.

The laser module 100 emits a laser beam, and may be a line laser module that emits a laser beam forward or a laser scanning module that converts and emits a laser beam into various shapes.

The line laser module or the laser scanning module has a different structure, but basically a laser diode 110 for generating a laser beam, a collimating lens 120 for condensing the laser beam generated through the laser diode, and It includes an electronic circuit (not shown). The collimating lens 120 is an optical lens that focuses the laser beam generated through the laser diode, and is located behind the laser diode. In this case, the laser beam may have various forms such as a pulse wave and a continuous wave.

As shown in Figures 2a, 2b and 3, the laser module 100 is installed in at least one or more of the position of the front lamp, the rear lamp, the direction indicator lights of both side mirrors, the front and rear of the vehicle, The object present in at least one of the two directions may be detected.

At this time, the optical axis of the laser module 100 may be integrally formed on one side of the lamp to be parallel to the irradiation direction of the lamp. Therefore, when the front / rear lamps of the vehicle are capable of angle adjustment, the laser module formed integrally with the front / rear lamps of the vehicle may also be adjusted.

The camera sensor module 200 receives the laser beam that is emitted from the laser module 100 and reflected by the object A.

Most vehicles are equipped with a black box or around view according to the user's choice. The black box is a device that records information before and after an accident and provides information necessary to determine the situation, vehicle speed, and parking situation recording function. Around view is a device that provides the screen as if the car is taken from the top of the navigation screen by using four camera sensor modules, one at the front and rear of the vehicle and one at the bottom of the left and right side mirrors.

Such a black box or around view essentially includes a camera. The lidar apparatus of the present invention uses a camera included in a black box or around view installed in a vehicle as the camera sensor module 200.

The camera sensor module 200 may be positioned at the front black box position, the rear black box position, and the direction indicator light of both side mirrors. In addition, the camera sensor module 200 may be located in the front grille, the rear trunk position of the vehicle.

The controller 300 calculates a distance from the object using the laser beam received from the camera sensor module 200. That is, the controller 300 calculates the distance to the object by using the laser beam received by the camera (that is, the camera sensor module 200) installed in the black box or around view of the vehicle. Since the camera sensor module 200 installed in the view is a device for acquiring an image for image in principle, among the images received by the camera sensor module 200, the image sensor and the laser beam photographed by the camera sensor module 200 are used. The laser beam image received by the camera sensor module 200 after being irradiated to the object and reflected is separated.

To this end, the controller 300, as shown in Figure 4, the image of the image captured by the camera sensor module 200 and the laser beam is reflected from the object of the laser beam image received from the laser beam image received pixel The pixel position detector 310 detects the position information, and the distance calculator 320 calculates a distance from the object using the detected position information of the pixel. In addition, the storage unit 330 further includes.

The pixel position detector 310 is synchronized with the laser module to collect the laser beam image according to the on / off of the laser module. That is, when the laser module is in an on state, the image acquired by the camera sensor module is collected as a laser beam image, and when the laser module is in the off state, the image obtained by the camera sensor module is imaged. Collect as an image.

The pixel position detector 310 may compare the collected laser beam image and the image image and detect the corresponding region as the position information of the pixel when the intensity of the specific region is greater than the intensity of the other region.

5A is a graph showing the exposure time of the camera sensor module 200. An exposure time t1 for an image image per frame and an exposure time t2 for a laser beam image are shown, where t2 is exposed for a very small time compared to t1. For example, if t1 is 5ms, t2 may be 1ms.

5B is a video image and a laser beam image collected during one frame. The left image of FIG. 5B is an image image not including a laser beam image, and the right image is an image including a laser beam image. The two images of FIG. 5B are images having different exposure times. In particular, since the exposure time of the video image (left image) is very large, when the positional information of the laser beam is detected by these images, the noise caused by the video image is reduced. Many are included, making accurate measurements difficult.

Therefore, in order to reduce the influence of disturbance, the image image of FIG. 5B is corrected for the laser beam exposure time t2. The left image of FIG. 5C is an image obtained by correcting the video image (left image) of FIG. 5B, and is an image image in which the amount of noise light is reduced as much as the laser beam image of FIG. 5B. This image correction is obtained by multiplying the original video image (the left image of FIG. 5B) by the "exposure time t2 of the laser beam image / exposure time t1 of the video image".

Next, the corrected image image (the left image of FIG. 5C) is removed from the image including the laser beam image (the right image of FIGS. 5B and 5C) to obtain a laser beam image such as that of FIG. 5D, and the pixels therefrom. Position information is detected.

The distance calculator 320 calculates a distance by performing triangulation measurement calculation using the position information of the pixel detected by the pixel position detector 310 to provide information for recognizing a vehicle's magnetic position or preventing collision with an obstacle that is an object. to provide.

The distance calculator 320 is a laser beam irradiated according to the on / off control of the laser module 100 based on the installation position information between the laser module 100 and the camera sensor module 200 stored in the storage unit 330 When reflected to the object and received by the camera sensor module 200, the object is displayed on the screen of the camera sensor module 100 by using the angle information and the distance information between the laser module 100 and the camera sensor module 200. The distance of is calculated by using triangulation measurement.

The storage unit 330 stores a program related to the operation of the controller 300 and basic setting information. The basic setting information includes installation position information between the laser module 100 and the camera sensor module 200, that is, angle information and separation distance information between the laser module 100 and the camera sensor module 200.

Meanwhile, the camera installed in the black box or the around view of the vehicle may be a camera sensor module to which an electronic optical filter is applied. The electronic optical filter is a filter that controls the transmittance of a specific wavelength of light as an electric signal. It can be used to transmit a laser beam image and block an image image. It is synchronized with the laser module to turn on / off the laser module. electronic optical filter is also turned on / off. When the electronic optical filter is applied, the laser beam image of the moment when the laser module is turned on is obtained from the images captured by the camera sensor module, but the non-laser image (for example, an RGB image) is obtained from the obtained laser beam image. This prevents disturbance caused by the video image in advance to increase the accuracy of the distance measurement.

The electronic optical filter may not be applied to the camera sensor module, but the electronic filter may be applied to the controller 300. That is, the pixel position detector 310 includes an electronic filter synchronized with the laser module, and the pixel position detector 310 is synchronized with the laser module and collected according to on / off of the laser module. When the position information of the laser beam is detected using the image, the electronic filter is also synchronized with the laser module to generate an image (eg, an RGB image) that is not a laser beam in the laser beam image detected by the pixel position detector 310. By blocking, the disturbance caused by the video image can be prevented in advance, thereby increasing the accuracy of the distance measurement.

Meanwhile, the camera sensor module installed in the black box or the around view of the vehicle may be specially manufactured for the purpose of implementing the embodiments of the present invention. 6 is a view showing an application example of the camera sensor module of the lidar apparatus according to an embodiment of the present invention.

As illustrated in FIG. 6, the camera sensor module 200 may include an image capturing image sensor 210 obtaining an image image and a distance measuring image sensor 220 obtaining a laser beam image. Specifically, among the image sensors constituting the camera sensor module 200, image sensors of one region are divided into image capturing image sensors 210 for acquiring an image image, and image sensors of other regions acquire a laser beam image. The distance sensor may be manufactured by dividing into an image sensor 220. Alternatively, the camera sensor module 200 may be manufactured separately from an image capturing image sensor 210 for acquiring an image image and a distance measuring image sensor 220 for acquiring a laser beam image, and then two sensors 210 and 220 may be used. Can be prepared by merging. The distance measuring image sensor may include an optical filter 230 that increases resolution by blocking an image (eg, an RGB image) that is not a laser beam from the laser beam image. For example, the optical filter 230 may be an IR filter.

Next, an operation process of the lidar apparatus according to the embodiment of the present invention as described above will be described.

The laser beam is emitted from the laser module 100 installed in the front lamp, the rear lamp, and both side mirrors of the vehicle. When the emitted laser beam is reflected to the vehicle by the object A, the reflected laser beam is received by the camera sensor module 200 included in the black box or around view installed in the vehicle. At this time, since the camera sensor module 200 is a part of a black box or an around view, the camera sensor module 200 captures not only a laser beam but also an external image.

The controller 300 detects the position information of the pixel in which the laser beam is received from the image image photographed by the camera sensor module 200 and the laser beam image reflected from the object, and uses the detected position information of the pixel. Calculate the distance to the object. In this case, the pixel position detection unit 310 of the controller 300 is synchronized with the laser module to collect the image image and the laser beam image according to the on (on) / off (off) of the laser module. The distance calculator 320 of the controller 300 calculates the distance by performing triangulation measurement calculation using the position information of the pixel detected by the pixel position detector 310 to recognize the position of the vehicle or collide with an obstacle that is an object. Provide information for prevention.

The lidar device according to the embodiment of the present invention as described above, by using the camera pre-installed on the moving body as a camera sensor module, it is possible to reduce the installation cost of the lidar device, such as a vehicle, drone, robot.

As mentioned above, although an embodiment of the present invention has been described, those of ordinary skill in the art may add, change, delete or add components within the scope not departing from the spirit of the present invention described in the claims. The present invention may be modified and changed in various ways, etc., which will also be included within the scope of the present invention.

100: laser module 200: camera sensor module
300: control unit 310: pixel position detection unit
320: distance calculation unit 330: storage unit

Claims (16)

A laser module for emitting a laser beam toward an object;
A camera sensor module configured to receive a laser beam emitted from the laser module and reflected by the object and installed in the moving object; And,
And a controller configured to calculate a distance from the object by using the laser beam received from the camera sensor module.
The camera sensor module,
When the laser beam does not emit light, it receives light other than the laser beam reflected by the object to obtain an image image,
When the laser beam is emitted, receiving the laser beam reflected by the object, or receiving the other light with the laser beam reflected by the object
Lidar device.
The method according to claim 1, wherein the moving body,
Lidar device, characterized in that the vehicle is a vehicle equipped with a recording device including a camera sensor module or an unmanned aerial vehicle or a mobile robot equipped with a camera.
The method according to claim 1, wherein the moving body,
Lidar device, characterized in that the vehicle is a black box or around the view including the camera sensor module is installed.
The method of claim 3, wherein the laser module,
And at least one of a front lamp, a rear lamp, and a direction indicator light of both side mirrors of the vehicle.
The method according to claim 4,
And the laser module is integrally formed on one side of the front lamp or the rear lamp so that the optical axis of the laser module is parallel to the irradiation direction of the front lamp or the rear lamp.
The method according to claim 1, wherein the control unit,
A pixel position detector for detecting position information of a pixel on which the laser beam is received from the image image photographed by the camera sensor module and the laser beam image reflected from the object and received;
Distance calculation unit for calculating the distance to the object using the detected position information of the pixel
Lidar device comprising a.
The method of claim 6, wherein the pixel position detector,
And a laser beam image which is synchronized with the laser module to collect the laser beam image according to the on / off of the laser module.
The method of claim 6, wherein the pixel position detector,
And if the intensity of a specific region in the laser beam image is greater than the intensity of another region, the region is extracted as position information of a pixel.
The method of claim 6, wherein the pixel position detector,
And a laser beam image is obtained by removing the corrected image correcting the video image from the image including the laser beam image.
The method of claim 9, wherein the correction image,
A lidar apparatus for obtaining the image image by multiplying the exposure time t2 of the image including the laser beam image by a value t2 / t1 divided by the exposure time t1 of the image image .
The method according to claim 6, wherein the camera sensor module,
And an electronic optical filter for transmitting the laser beam image and blocking the image image.
And the electronic optical filter is synchronized with the laser module and turned on / off according to the on / off of the laser module.
The method of claim 6, wherein the pixel position detector,
An electronic filter synchronized with the laser module,
And the electronic filter is synchronized with the laser module and turned on / off according to the on / off of the laser module.
The method according to claim 1, wherein the camera sensor module,
And a distance measuring image sensor for acquiring a laser beam image.
The method according to claim 13, wherein the camera sensor module,
And an area of the image sensors constituting one camera sensor module is divided into the image capturing image sensor, and the other area is manufactured by being divided into the distance measuring image sensor.
The method according to claim 13, wherein the camera sensor module,
And after the image capturing image sensor and the distance measuring image sensor are manufactured separately, the two sensors are manufactured by merging.
The image sensor according to any one of claims 13 to 15, wherein the distance measuring image sensor
And an optical filter for removing a component other than the laser beam from the laser beam image to increase resolution.

Images