RU218665U1 - MULTISPECTRAL LIDAR SYSTEM - Google Patents

Abstract

The utility model relates to the field of lidar technology and can be used in machine vision systems for measuring the distance to distant targets, mapping and creating three-dimensional images.

A multispectral lidar contains two or more lasers generating laser radiation pulses in different spectral ranges, for which the spectra of signals reflected from different materials do not match in intensity.

The output optical system of the lidar ensures that all laser radiation beams coincide in space over the entire operating range. The aperture of the input optical system is directed to the region of space in which the rays of the reflected radiation propagate, and is configured in such a way that the radiation of all lasers reflected from the target falls into it. There is an array of spectrum-splitting elements that separate in space along the spectrum the laser radiation reflected from the target entering the input optical system. In the focal plane of the input optical system, an array of photodetectors is located in such a way that only one section of the reflected radiation spectrum falls on each independent element of the photodetector. The propagation delay of light pulses reflected from the target is determined by the time interval meter. The controller determines the parameters of the target based on the measurement of the delay time of the propagation of the light pulse from the laser to the target and the amplitude of the reflected radiation, and these characteristics are determined independently for each element of the massive photodetector.

The technical result is to improve the accuracy of object identification in machine vision systems based on lidar. 3 ill.

Description

The proposed utility model relates to the field of lidar technology and can be used as part of machine vision systems for measuring the distance to distant targets, mapping and creating three-dimensional images.

Recently, the high relevance of the tasks of robotization and computer control of systems has led to the rapid development of machine vision devices. Machine vision allows algorithms to determine the parameters of surrounding objects, assess the situation and decide on a behavior strategy. In many cases, for example, when an unmanned vehicle or aircraft is moving, the correctness of estimating the distance to a remote target and its identification directly determines the safety of such a system for humans and the environment. Now the main devices for identifying objects are various types of cameras, including infrared cameras and multispectral cameras. To estimate the distance to objects, ultrasonic sensors, radars or lidars are used. The disadvantage of the camera is that they do not allow to accurately and reliably estimate the distance to a remote target, and ultrasonic sensors, radars and lidars do not provide information about the possible material, shape and texture of a remote target. The proposed design of the device allows you to simultaneously obtain a whole array of useful information about the distance to a distant target and its reflection coefficient in several spectral ranges, which will significantly improve the process of identifying distant targets for machine vision systems.

There is a known patent [US 9823128 B2, IPC: G01J3/2823, published on 11/27/2017] for a device for multispectral shooting based on an array of narrow-band light filters, which includes a photosensitive element consisting of a plurality of pixels, an array of narrow-band light filters located in front of the photosensitive element , and each element of the light filter array corresponds to a single pixel of the photosensitive element, while the many elements of the light filter array are grouped by N elements, so that each individual element of the filter array i absorbs K _i narrow spectral sections and passes NK _i narrow spectral sections while N is greater than 3 and K _{i is} between 1 and N-1, i is between 1 and N, and K _i is different for at least two elements of the filter array. The device allows you to form an image in which data on the reflection coefficient from a distant target in more than 3 spectral ranges are used as a contrast, which makes it possible to more accurately identify a distant target by the specifics of the absorption and reflection spectra. The disadvantage of the device is the lack of active illumination, and the inability to determine the distance to a distant target.

Known patent [JP 2021518560 A, IPC: G01N21/21, published 08/02/2021] multispectral lidar transmitter, which includes an optical device, a scanning device, a spectrum splitter, a wavelength selection device and a detection device. The optical device consists of a wide spectrum laser from which one specific wavelength is selected using a spectrum splitter. The advantage of the device is the ability to create active illumination in a large number of spectral ranges. The disadvantage of the device is the need to use a laser with an extremely wide spectrum, more than a few tens of nanometers, which significantly increases energy consumption, since only a small part of the emitted energy is used at a time. Also, the disadvantage is the inability to simultaneously emit in more than one spectral range.

As a prototype, a lidar is used [US 10094925 B1, IPC: G01S17/08, published 10/09/2021], which includes: a laser configured to emit a light pulse including a first wavelength; a scanner configured to direct the emitted light pulse in accordance with the scan; a receiver comprising: a first detector configured to detect in the first field of view an emitted light pulse in the first spectral range scattered by a distant target and a second detector configured to detect in the second field of view an emitted light pulse in the second spectral range scattered by a distant target, the scanner provides correspondence between the first field of view and the second field of view; the prototype also contains an optical element configured to separate the received radiation into the first beam containing the first wavelength and practically not containing the second wavelength and the second beam containing the second wavelength and not containing the first wavelength, the optical element also directs the first beam to the first detector , and the second beam to the second detector, the lidar system also includes a controller that is configured to determine the distance using the light pulse recorded by the first detector, the controller also allows you to measure the parameters of a remote target using the light pulse recorded by the second detector, make adjustments to the measurements of the parameters of the remote target using distance data.

The prototype allows you to build a three-dimensional map of space while simultaneously measuring the reflectance of the object in the second wavelength. The disadvantage of the prototype is the presence of a source of laser radiation with only one wavelength. Creating an image with additional information obtained using the second detector is possible only if there is reflected radiation of natural nature in the second spectral range or if a distant target emits radiation. Also, the disadvantage is that in the prototype only two detectors are used to obtain information, one for determining the range and the second for determining one additional property of a distant target, while information about the reflection coefficient from a distant target in different spectral ranges would help improve the ability to object recognition for machine vision systems. Another disadvantage is that the field of view of the first detector and the field of view of the second detector do not coincide, which leads to the fact that scanning systems must be used to build a three-dimensional image, and when scanning fast moving distant targets, errors may occur related to the correlation of information about range and information about additional properties.

The task set in the basis of the proposed technical solution is to create a device that will allow you to simultaneously obtain information about the distance to a distant target and information about the reflection coefficient in several spectral ranges.

The technical result of solving the problem is to improve the accuracy of object identification in machine vision systems based on lidar. It is known that many materials have a unique reflectance spectrum, and data on the relative reflectivity of the surface of a distant target in several spectral ranges allows you to more accurately and reliably identify the material of a distant target and, accordingly, choose a more appropriate strategy for interacting with it. For example, multispectral lidar can significantly increase the safety of landing an unmanned aerial vehicle on an unprepared site with unknown coverage. Sand and clay have a similar reflectance in the visible spectrum, while in the wavelength range of 1400-1500 nm it differs twice [Li S. et al. Automatic near real-time flood detection using Suomi-NPP/VIIRS data //Remote sensing of environment. - 2018. - T. 204. - S. 672-689.], while landing on a clay surface is safer than on a sandy one.

The technical result is achieved due to the fact that the device uses two or more lasers that generate pulses of light radiation in spectral ranges that do not match in intensity level 1/e ² , an output optical system that ensures that all laser radiation beams coincide in space over the entire operating range range and provides illumination of the entire field of view of the multispectral lidar with all wavelengths used, the device also uses an input optical system, the aperture of which is directed to the region of space in which the rays of light radiation propagate, and the aperture of the optical device is configured in such a way that reflected radiation enters it from all lasers for any remote target within the field of view of the optical system, while the input optical system projects the image of the remote target onto the spectrum-splitting elements, which are performed using prisms, diffraction gratings or other optical devices capable of separating in space the spectrum that falls into the optical system reflects laser radiation from a distant target, the device also contains an array of photodetectors, which are located in the focal plane of the optical system in such a way that each independent element of the photodetector receives radiation corresponding to only one section of the spectrum reflected from a remote target, the device also contains a system for determining the parameters of a remote target based on measuring the delay time of the propagation of a light pulse from the laser to the remote target and the amplitude of the reflected light radiation, and the system for determining the parameters of the remote target allows you to determine these characteristics for each element of the massive photodetector independently.

The essential difference of the proposed device is the use of active illumination in two or more mismatched spectral ranges and photodetector devices, any of the independent pixels of which are sensitive to only one of the spectral ranges used in the illumination, while each pulse of light radiation , reflected from a distant target, hits all pixels corresponding to its image on the FPA and with the help of which a two-dimensional image is formed. This makes it possible to obtain data on the reflection spectrum of a distant target, its geometric dimensions and distance to it, even in the absence of additional illumination or luminescence of the target itself, in one FPU frame and one emitted laser pulse and without the use of scanning systems.

The technological implementation of the proposed multispectral lidar is based on the well-known basic methods for its manufacture, which are well developed and widely used by now. The proposal satisfies the criterion "Industrial applicability".

Brief description of the drawings

The proposed multispectral lidar is illustrated in figures 1-3.

Figure 1 - schematically depicts a functional diagram.

On FIG. 2 - shows the relative emission spectrum and the relative reflection spectrum from two distant targets made of snow and white plastic at the same distance, as well as the spectrum of lasers used in the lidar.

On FIG. 3 shows the result of the operation of a two-dimensional lidar with emitters in three spectral ranges.

Implementation of the PM

The device shown in Fig. 1 includes the following components:

L ₁ , L ₂ , L _n , are lasers emitting in the spectral range λ ₁ , λ ₂ .. λ _N ;

OS1 output optical system that allows you to group the beams λ ₁ , λ ₂ ..λ _N in one direction;

CA is a distant target, which is hit by the radiation of all lasers and which is located in the aperture of the input optical system;

OS2 is the input optical system, into the input aperture of which the radiation reflected from a distant target falls, and which projects its image onto the spectrum-splitting elements;

FE ₁ , FE ₂ , FE _K - spectrum-splitting elements that separate the radiation incident on them according to the radiation wavelength.

FPU array of photodetectors, consisting of independent pixels Ф ₁ , Ф ₂ ..FPU _KN ;

TW is a time meter that allows you to determine the radiation propagation delay for each independent FPU pixel by measuring the time between the moment of laser pulse emission t _start1 , t _start2 … t _startN and the arrival of the FPU signal reflected from a distant target t _stop1..N .

K - controller that receives information from the time meter about the radiation propagation delay t _1..N and information from the FPU about the target reflection coefficient R _1..KN, converts and processes this information to create a picture in which the surrounding space is divided into points, for each from which the range and reflection coefficient in N spectral ranges are known.

At times t_start1..t_startNemitters generate light pulses with wavelength λ₁..λ_Nand transmit a signal about the beginning of the countdown for the time meter. Light impulses λ₁..λ_Nwith the help of the output optical system, they are directed to one region of space, where, in the presence of a distant target, they are reflected into the aperture of the input optical system. In the input optical system, the image of a distant target is projected onto the spectrum-splitting elements; the reflected light is divided into a spectrum, and radiation of one wavelength falls on each independent FPU element. Each of the independent elements of the FPU transmits a signal t to the time meter_stop1..Nand to the controller a signal about the intensity of the reflected light pulse R_1..Nwhich is proportional to the reflection coefficient of the distant target in the corresponding spectral range. The time meter determines the delay from the difference between the times t_stopand t_startfor each spectral range and transmits this data to the controller. The controller converts and processes the received data and builds an image based on them, consisting of the distance to the remote target and the reflection coefficient in each used spectral range.

The graph depicted in Fig. 2 includes the relative total specular reflection spectrum (thick solid line) of the λ ₁ laser with a dominant wavelength of 905 nm, the λ ₂ laser with a dominant wavelength of 1345 nm, and the λ ₃ laser with a dominant wavelength of 1550 nm. The graph also shows the reflectance spectrum of white plastic (dashed line) and snow (dashed line) [Moshtaghi M. et al. Spectral reflectance of marine macroplastics in the VNIR and SWIR measured in a controlled environment //Scientific Reports. - 2021. - T. 11. - No. 1. - P. 1-12., Spectral reflectance of marine macroplastics in the VNIR and SWIR measured in a controlled environment Domine F. et al. Correlation between the specific surface area and the short wave infrared (SWIR) reflectance of snow //Cold Regions Science and Technology. - 2006. - T. 46. - No. 1. - P. 60-68.] and the intensity level of the reflected signal 1/e ² (thin solid line). It can be seen from the graph that for lidars with a wavelength of 1345 nm, the intensity of the reflected signal for these two materials from the same distance will be close, and the white color in the visible range also makes it difficult to identify with the camera. Due to the use of other spectral ranges together with the emitter in the proposed design of the multispectral lidar, these materials can be unambiguously distinguished in the image due to a significant difference in the reflectance at 905 nm and 1550 nm. The unique spectral composition of reflected light is characteristic of many materials, and by increasing the number of lasers in different spectral ranges of a multispectral lidar, their identification is simplified. Thus, the use of multispectral lidar technology makes it possible to determine not only the distance to a distant target, but also the material of which it consists.

The figure 3 shows a possible variant of the image output of a multispectral lidar with three lasers. Color encoding is done using an 8-bit RGB system where R is the reflectance at 905 nm, G is the reflectance at 1345 nm, and B is the reflectance at 1550 nm. The field of view of the multispectral lidar is 90 degrees. Based on the results of the constructed image, it is possible to determine the distance to distant targets, as well as the fact that objects consisting of three different materials fell into the field of view of the lidar.

Claims (1)

Multispectral lidar containing two or more lasers with the ability to generate laser pulses in mismatched spectral ranges; output optical system with the ability to ensure the coincidence in space of all beams of laser radiation over the entire operating range; an input optical system, the aperture of which is directed to the region of space in which, during the operation of the multispectral lidar, reflected radiation beams should propagate, and the aperture of the optical device is configured in such a way that radiation from all lasers reflected from a distant target enters it; an array of spectrum-splitting elements capable of spatially separating the laser radiation reflected from a distant target into the input optical system according to the spectrum; an array of photodetectors that are located in the focal plane of the input optical system with the possibility of hitting each independent element of the photodetector with radiation corresponding to only one section of the spectrum reflected from a distant target; a time interval meter that determines the propagation delay of light pulses reflected from a distant target; a controller for determining the parameters of a remote target based on measuring the delay time of the propagation of a light pulse from a laser to a remote target and the amplitude of the reflected radiation, and the system for determining the parameters of a remote target allows you to determine the specified characteristics for each element of a massive photodetector device independently, characterized in that lasers are selected with the ability to generation of laser radiation pulses in different spectral ranges, for which the signals reflected from different materials have spectra that do not match in terms of intensity.

Images