JP2022125966A - Ranging correction device, ranging correction method, ranging correction program, and ranging device - Google Patents

Abstract

To provide a ranging correction device and the like capable of improving ranging accuracy.SOLUTION: A ranging correction device corrects a ranging result of a ranging device which measures a distance to a reflection point by detecting, with a pixel, reflected light from a reflection point with respect to irradiation with light. The ranging correction device includes: a pixel information acquisition part 110 for acquiring distance information detected with a corresponding pixel for a plurality of reflection points; a normal calculation part 130 for calculating the normal direction for each reflection point; and a distance correction part 150 for correcting the distance to each reflection point based on the normal direction.SELECTED DRAWING: Figure 1

Description

The disclosure in this specification relates to a technique for measuring the distance to a reflection point by detecting reflected light from the reflection point with respect to irradiation of light.

Patent Literature 1 discloses a device that corrects the measurement result of a distance measuring device. This device measures the number of photons received by the light receiving section, and corrects the walk error caused by the intensity of the received light based on the number of photons.

International Publication No. 2017/42993

By the way, in a distance measuring device that detects reflected light, a distance detection error may occur depending on the inclination of the reflecting surface. The technique disclosed in Japanese Patent Application Laid-Open No. 2002-200010 cannot correct an error according to the inclination of the reflecting surface.

An object of the disclosure is to provide a distance measurement correction device, a distance measurement correction method, a distance measurement correction program, and a distance measurement device capable of improving distance measurement accuracy.

The multiple aspects disclosed in this specification employ different technical means to achieve their respective objectives. In addition, the symbols in parentheses described in the claims and this section are examples showing the corresponding relationship with the specific means described in the embodiment described later as one aspect, and limit the technical scope. is not.

One of the disclosed distance measurement correction devices is a distance measurement device that has a processor (102) and measures the distance to the reflection point by detecting the reflected light from the reflection point of the target with respect to the irradiation of light with pixels. A distance measurement correction device for correcting the distance measurement result of (1),
an acquisition unit (110) for acquiring related information, which is information related to distances detected at corresponding pixels for a plurality of reflection points;
feature quantity calculation units (130, 135) for calculating a tilt feature quantity related to the magnitude of tilt with respect to a reference plane (R) for a partial surface of a target constituting a reflection point;
a correction unit (150) that corrects the distance to each reflection point based on the inclination feature quantity;
Prepare.

One of the disclosed distance measurement correction methods is to detect the reflected light from the reflection point of the target with respect to the irradiation of the light with pixels, thereby measuring the distance to the reflection point, and the distance measurement result of the distance measurement device (1) is corrected. A ranging correction method executed by a processor (102) to correct, comprising:
an acquisition process (S100, S110; S210, S220) for acquiring relevant information, which is information related to distances detected at corresponding pixels, for a plurality of reflection points;
Feature quantity calculation processes (S130, S131, S132; S133, S134; S135, S136; S135, S136; S230) and
a correction process (S150, S160, S170; S250, S260, S270) for correcting the distance to each reflection point based on the inclination feature quantity;
including.

One of the disclosed distance measurement correction programs uses pixels to detect light reflected from a reflection point of a target against irradiation of light, thereby measuring the distance to the reflection point, thereby obtaining the distance measurement result of a distance measurement device (1). A ranging correction program comprising instructions to be executed by a processor (102) to correct, comprising:
the instruction is
an acquisition process (S100, S110; S210, S220) for acquiring related information, which is information related to distances detected at corresponding pixels, for a plurality of reflection points;
Feature quantity calculation processes (S130, S131, S132; S133, S134; S135, S136; S135, S136; S230) and
a correction process (S150, S160, S170; S250, S260, S270) for correcting the distance to each reflection point based on the inclination feature amount;
including.

One of the disclosed distance measuring devices is a distance measuring device which has a processor (102) and measures the distance to the reflection point by detecting the reflected light from the reflection point of the target with respect to the irradiation of the light with pixels. There is
an acquisition unit (110) for acquiring related information, which is information related to distances detected at corresponding pixels for a plurality of reflection points;
feature quantity calculation units (130, 135) for calculating a tilt feature quantity related to the magnitude of tilt with respect to a reference plane (R) for a partial surface of a target constituting a reflection point;
a correction unit (150) that corrects the distance to each reflection point based on the inclination feature quantity;
Prepare.

According to these disclosures, the distance to each reflection point is corrected based on the tilt feature amount of each reflection point. Therefore, the error corresponding to the inclination of the partial surface forming the reflection point can be corrected. As described above, a distance measurement correction device, a distance measurement correction method, and a distance measurement correction program that can improve distance measurement accuracy can be provided.

3 is a block diagram showing an example of functions of a distance measurement correction device; FIG. FIG. 4 is a diagram conceptually showing changes in detected waveforms according to inclinations; FIG. 4 is a diagram conceptually showing a method of calculating a normal vector; 5 is a flow chart showing an example of a distance measurement correction method executed by a distance measurement correction device; FIG. 10 is a diagram conceptually showing a method of calculating a normal vector in the second embodiment; 9 is a flow chart showing an example of a distance measurement correction method executed by a distance measurement correction device according to the second embodiment; FIG. 11 is a block diagram showing an example of functions of a ranging correction device according to a third embodiment; FIG. 10 is a diagram for explaining a feature amount of a waveform; FIG. 11 is a flow chart showing an example of a distance measurement correction method executed by a distance measurement correction device according to a third embodiment; FIG. 12 is a block diagram showing an example of functions of a distance measurement correction device according to a fourth embodiment; FIG. 14 is a flow chart showing an example of a distance measurement correction method executed by a distance measurement correction device according to a fourth embodiment; FIG. 11 is a block diagram showing an example of functions of a distance measurement correction device according to a fifth embodiment; FIG. 12 is a diagram conceptually showing setting of a scan speed in the fifth embodiment; 14 is a flow chart showing an example of a distance measurement correction method executed by the distance measurement correction device according to the fifth embodiment; FIG. 11 is a diagram conceptually showing setting of a scan speed in the sixth embodiment;

(First embodiment)
As shown in FIG. 1 , an image processing device 100 as a ranging correction device according to an embodiment of the present disclosure is mounted on a LiDAR (Light Detection and Ranging/Laser Imaging Detection and Ranging) device 1 . The LiDAR device 1 is a distance measuring device that measures the distance to a reflection point by detecting reflected light from the reflection point with respect to irradiation of light. The LiDAR device 1 is, for example, a sensor mounted on a vehicle equipped with at least one of an advanced driving support function and an automatic driving function. The LiDAR device 1 is communicably connected to the in-vehicle ECU 10 . The in-vehicle ECU 10 is an electronic control device that uses the measurement results of the LiDAR device 1 for processing.

The LiDAR device 1 includes a light emitting unit 2 and an imaging unit 3 in addition to the image processing device 100 .

The light emitting unit 2 is a semiconductor element, such as a laser diode, that emits directional laser light. The light emitting unit 2 emits intermittent pulsed beams of laser light toward the outside of the vehicle. The imaging unit 3 is composed of a light receiving element, such as SPAD (Single Photon Avalanche Diode), which is highly sensitive to light. The imaging unit 3 is exposed to light incident from the sensing area determined by the angle of view of the imaging unit 3 in the external world of the imaging unit 3 . A plurality of light-receiving elements constituting the imaging unit 3 are arranged in a two-dimensional array, for example. A set of a plurality of adjacent light-receiving elements constitutes a pixel in reflected light detection. That is, information indicating the relationship between the distance to the reflection point and the reflection intensity, which will be described later, is detected for each pixel configured by a set of a plurality of light receiving elements.

The actuator 4 controls the angle of reflection of a reflector that reflects the laser light emitted from the light emitting unit 2 to the emission surface of the LiDAR device 1 . The laser beam is scanned by the actuator 4 controlling the angle of reflection of the reflecting mirror. The scanning direction may be horizontal or vertical. The actuator 4 may scan the laser light by controlling the attitude angle of the housing of the LiDAR device 1 itself.

The image processing apparatus 100 is a computer including at least one memory 101 and at least one processor 102 . The memory 101 stores or stores computer-readable programs and data non-temporarily, for example, at least one type of non-transitory tangible storage medium such as a semiconductor memory, a magnetic medium, and an optical medium. storage medium). The memory 101 stores various programs executed by the processor 102, such as a distance correction program to be described later.

The processor 102 includes, as a core, at least one type of CPU (Central Processing Unit), GPU (Graphics Processing Unit), RISC (Reduced Instruction Set Computer), etc., for example. Processor 102 executes a plurality of instructions contained in a ranging correction program stored in memory 101 . Accordingly, the image processing apparatus 100 constructs a plurality of functional units for correcting the distance to the target T measured from the distance measurement result, that is, the detection information of the imaging unit 3 . As described above, in the image processing apparatus 100, the distance measurement correction program stored in the memory 101 causes the processor 102 to execute a plurality of instructions, thereby constructing a plurality of functional units. Specifically, as shown in FIG. 1, the image processing apparatus 100 includes functional units such as a pixel information acquisition unit 110, a point cloud generation unit 120, a normal line calculation unit 130, a reliability calculation unit 140, and a distance correction unit 150. is constructed.

The pixel information acquisition unit 110 controls exposure and scanning of a plurality of pixels in the imaging unit 3, and processes signals from the imaging unit 3 into data. In the reflected light mode in which the pixel information acquisition unit 110 exposes the imaging unit 3 by light irradiation from the light emitting unit 2, an object point within the sensing area becomes a reflection point of laser light. As a result, laser light reflected at the reflection point (hereinafter referred to as reflected light) enters the imaging section 3 through the incident surface. At this time, the pixel information acquisition unit 110 senses reflected light by scanning a plurality of pixels of the imaging unit 3 .

The pixel information acquisition unit 110 integrates the reflection intensity scanned in each pixel for each received light frequency, thereby obtaining the relationship between the distance to the reflection point and the reflection intensity as shown in FIG. is acquired for each pixel as information related to . Specifically, the pixel information acquisition unit 110 can acquire the relationship information as histogram information obtained by integrating the reflection intensity for each predetermined distance bin, or waveform information based on the reflection intensity for each distance bin in the histogram. In this embodiment, it is assumed that the relationship information is waveform information as shown in FIG. 2 and the like. Thereby, the pixel information acquisition unit 110 acquires pixel information including information related to the distance to the reflection point. A partial surface SA of the target T on which the reflected light is incident on the same pixel constitutes a reflection point detected by the pixel. The pixel information acquisition unit 110 can acquire two-dimensional data including such pixel information for each pixel as a range image.

On the other hand, in the external light mode in which the pixel information acquisition unit 110 exposes the imaging unit 3 while intermittent light irradiation from the light emitting unit 2 is stopped, an object point within the sensing area becomes a reflection point of external light. As a result, the external light reflected by the reflection point enters the imaging section 3 through the incident surface. At this time, the pixel information acquisition unit 110 senses reflected external light by scanning a plurality of pixels of the imaging unit 3 . Here, in particular, the pixel information acquisition unit 110 can acquire an external light image by converting the luminance value acquired for each pixel according to the intensity of the sensed external light into two-dimensional data as each pixel value. is. The external light image can also be called a background light image or an ambient light image.

The pixel information acquisition unit 110 determines whether or not the waveform information (detected waveform information) of the detected reflected wave is valid for the newly acquired pixel information. For example, the pixel information acquisition unit 110 may determine whether the detected waveform information is valid based on the S/N ratio of the waveform, the amplitude of the waveform, and the like. If it is determined that the detected waveform information is not valid, the pixel information acquiring section 110 rejects the acquired pixel information. The pixel information acquisition unit 110 acquires pixel information for all pixels in each control cycle. The pixel information acquisition unit 110 sequentially provides the acquired pixel information to the point group generation unit 120 .

Further, the pixel information acquisition unit 110 removes noise from the generated distance image. For example, the pixel information acquisition unit 110 determines a region on which the noise removal filter is applied for the current distance image based on the past distance image. More specifically, the pixel information acquiring unit 110 determines, for a range image frame, a non-existing area where no point cloud exists at the same position in a past (for example, one frame previous) range image, and an existing area where the point group exists. divided into The point group generation unit 120 skips application of the noise removal filter to non-existing regions. In addition, the pixel information acquisition unit 110 applies a noise removal filter with different parameters depending on the type of object to the existing region. For example, the pixel information acquisition unit 110 changes the parameters of the noise removal filter for objects with relatively many substantially flat portions and objects with relatively few flat portions. Objects with a relatively large number of flat portions are, for example, roads, buildings, and the like. Objects with relatively few flat portions include people, animals, and the like. It should be noted that the pixel information acquisition unit 110 may set the existing area to be larger than the actual area where the object actually exists, taking into consideration the movement of the object.

The point group generation unit 120 converts the distance value to the reflection point included in the acquired pixel information into three-dimensional coordinate information. The point group generation unit 120 converts the distance value into a three-dimensional coordinate value in the LiDAR coordinate system centered on the LiDAR device 1 based on the focal length of the optical system, the number of pixels of the image sensor, the size of the image sensor, and the like. do it. The point cloud generation unit 120 converts all the distance values into a three-dimensional coordinate system and generates point cloud data including coordinate information of the reflection point corresponding to each pixel.

The normal calculation unit 130 calculates the normal direction of the reflection point as the tilt feature amount. The tilt feature amount is a parameter related to the degree of tilt with respect to the reference plane R in the partial surface SA of the target T forming the reflection point. Here, the reference plane R is a virtual plane directly facing the line-of-sight direction DL for each pixel in the LiDAR device 1, which will be described later. The normal calculation unit 130 calculates the normal direction of each reflection point based on the three-dimensional position of the point cloud data. Specifically, the normal calculation unit 130 calculates a normal vector Vn including information on the direction of the normal. For example, the normal calculation unit 130 sets the outer product of two vectors (reference vectors) based on multiple reflection points corresponding to multiple pixels as the normal vector Vn.

Specifically, as shown in FIG. 3, the normal calculation unit 130 sets the reflection point (reflection point of interest) RPi for calculating the normal vector Vn as the starting point of the reference vector. Then, the normal calculation unit 130 selects two reflection points (reference reflection points) RPr located near the reflection point of interest RPi. The reference reflection point RPr may be, for example, reflection points detected at two pixels adjacent to the pixel corresponding to the target reflection point RPi. The normal calculation unit 130 sets a reference vector Vr having the target reflection point RPi as a starting point and each reference reflection point RPr as an end point. The normal calculating section 130 calculates the outer product vector of the reference vector Vr as the normal vector Vn of the target reflection point RPi. The normal calculation unit 130 calculates normal vectors for substantially all reflection points in one frame. The normal calculation unit 130 sequentially provides the calculated normal vector information to the distance correction unit 150 . The normal calculation unit 130 is an example of a “feature amount calculation unit”.

The reliability calculator 140 calculates the reliability of the normal vector Vn of each reflection point. In the following description, this reliability is referred to as normal reliability. Normal confidence is an estimate related to the magnitude of error in the calculated normal vector. The higher the normal reliability, the smaller the error of the normal vector Vn. The reliability calculation unit 140 estimates the normal reliability based on at least one of the signal light intensity and the external light intensity included in the detected waveform information detected at the corresponding pixel, for example. The higher the signal light intensity, the higher the reliability. Also, the higher the external light intensity, the lower the reliability. Normal reliability is an example of "calculated reliability."

A distance correction unit 150 corrects the distance value to each reflection point based on the normal vector Vn. For example, the distance correction unit 150 calculates the corrected distance value based on the normal vector Vn, the line-of-sight information of the LiDAR device 1, the distance value before correction, and the normal reliability.

Here, the line-of-sight information of the LiDAR device 1 is information regarding the line-of-sight direction DL of the pixels of the LiDAR device 1 in each reflection point detection. The line-of-sight direction DL is, for example, a direction facing the direction of receiving the reflected light. The line-of-sight direction DL, as indicated by the dotted arrow in FIG. 2, is the direction from the pixel center or the pixel position when the pixel is regarded as a point toward the center of the pixel detection range PR. It can also be said that the line-of-sight direction DL is the direction in which the center of the angle of view of the corresponding pixel is oriented from the pixel center or the pixel position when the pixel is regarded as a point.

Here, the partial surface SA of the target T on which the reflected light is incident on the same pixel constitutes the reflection point detected by the pixel. In FIG. 2, the partial surface SA is a portion of the surface of the target T that is included in the detection range PR. The greater the inclination of the partial surface SA with respect to the reference surface R, the greater the optical path length difference of the reflected light within the partial surface. Also, the peak of the waveform of the reflection intensity becomes smaller as the slope becomes larger. Therefore, the greater the slope, the broader the waveform can be. Also, the greater the slope, the smaller the signal intensity of the peak of the waveform, and the more delayed the light receiving time. Therefore, by calculating the correction amount for correcting the distance value in the direction (the direction of the dotted arrow in the graph of FIG. 2) to eliminate the delay in the light reception time, that is, the lengthening of the distance value, the distance after correction closer to the true value value can be calculated. It should be noted that in the present embodiment, the partial surface SA is treated as a substantially flat surface.

The distance correction unit 150 increases the correction amount as the relative inclination of the normal vector Vn with respect to the line-of-sight direction increases. Further, the distance correction unit 150 increases the correction amount as the distance value before correction increases. In addition, the distance correction unit 150 increases the correction amount as the normal reliability is lower. The distance correction unit 150 comprehensively determines the correction amount based on the above parameters. The distance correction unit 150 corrects the distance value with the determined correction amount. The distance correction unit 150 corrects the distance values for all reflection points for which normal vectors have been calculated, and generates a distance image based on the corrected distance values. The distance correction unit 150 provides the generated distance image to other in-vehicle ECUs 10 . Note that the distance correction unit 150 may generate point cloud data obtained by converting the distance image into a three-dimensional point cloud, and provide the data to the in-vehicle ECU 10 .

Next, the flow of the distance measurement correction method executed by the image processing apparatus 100 in cooperation with the functional blocks will be described below with reference to FIG. In the flow described later, "S" means multiple steps of the flow executed by multiple instructions included in the program.

First, in S100, the pixel information acquisition unit 110 acquires unacquired pixel information from the image sensor. Next, in S110, it is determined whether or not the waveform data of the pixel information is valid. If it is determined not to be valid, the flow returns to S100 to acquire other unacquired pixel information. If the waveform data is determined to be valid, the flow shifts to S120.

In S120, the point group generation unit 120 converts the pixel information determined to be valid into three-dimensional coordinate data. Next, in S130, the coordinates of the reference reflection point RPr in the vicinity of the reflection point of interest RPi are acquired. Then, in S131, two reference vectors are calculated based on the reflection point of interest RPi and the reference reflection point RPr. In subsequent S132, a normal vector is calculated as the outer product of the reference vectors.

Next, in S140, the normal calculation unit 130 calculates normal reliability. Next, in S150, the distance correction unit 150 corrects the distance to the reflection point based on the inclination of the normal vector with respect to the line-of-sight direction, the distance to the reflection point, and the degree of reliability. In subsequent S160, the distance correction unit 150 determines whether or not all pixels have been corrected in the current control cycle. When it is determined that all pixels have been corrected, the distance correction unit 150 outputs distance image data in S170.

Note that S100 and S110 described above are an example of the "pixel information acquisition process", and S120 is an example of the "point group generation process". Also, S130, S131, and S132 are an example of a "feature amount calculation process," S140 is an example of a "reliability calculation process," and S150, S160, and S170 are examples of a "correction process."

According to the first embodiment described above, the distance to each reflection point is corrected based on the normal vector of each reflection point. Therefore, the error according to the inclination of the reflecting surface can be corrected. As described above, the accuracy of distance measurement can be improved.

Further, according to the first embodiment, the normal vector is calculated by the outer product based on the positional information of the target reflection point RPi and the plurality of reference reflection points RPr. According to this, the inclination of the normal line is calculated by vector calculation. Therefore, the slope of the normal can be calculated relatively quickly.

Furthermore, according to the first embodiment, the larger the inclination of the normal vector Vn of the target reflection point RPi with respect to the observation direction in the corresponding pixel, the larger the correction amount. Therefore, when the inclination of the reflection point is large and the deviation from the true value is likely to be large, the distance can be corrected to a greater extent. Therefore, the distance can be corrected more accurately.

In addition, according to the first embodiment, the correction amount increases as the normal vector calculation reliability increases. According to this, the higher the reliability of the normal vector, the larger the correction amount corresponding to the inclination of the normal vector. Therefore, the distance can be corrected more accurately.

Further, according to the first embodiment, the correction amount increases as the distance before correction to the target reflection point RPi increases. The greater the distance to the target reflection point RPi, the greater the difference in the optical path length within one pixel to the reflection point. By increasing the correction amount, the distance can be corrected more accurately.

(Second embodiment)
In the second embodiment, a modified example of the image processing apparatus 100 in the first embodiment will be described. In FIGS. 5 and 6, constituent elements denoted by the same reference numerals as in the drawings of the first embodiment are similar constituent elements and have similar effects.

In the second embodiment, as shown in FIG. 5, the normal calculation unit 130 selects a plurality of reflection points whose point-to-point distances from the target reflection point RPi are within the allowable range as the reference reflection points RPr. The allowable range is a numerical range that is less than or equal to a threshold value related to the distance from the target reflection point RPi. That is, even if the reflection point is an adjacent pixel, if the distance based on the three-dimensional coordinates is outside the allowable range, the normal calculation unit 130 excludes the reflection point from the reference reflection point RPr.

In the second embodiment, the normal calculation unit 130 performs principal component analysis based on a plurality of reference reflection points RPr and the target reflection point RPi. The normal vector calculation unit 130 calculates the normal vector Vn based on the result of the principal component analysis.

Next, detailed processing of normal vector calculation among the distance measurement correction methods executed by the image processing apparatus 100 in the second embodiment will be described below with reference to the flowchart of FIG.

This flow shifts to S133 after the process of S120. In S133, the normal calculation unit 130 acquires the coordinates of a plurality of reference reflection points RPr whose distances from the target reflection point RPi are within the allowable range. Next, in S134, the normal vector Vn is calculated by the normal vector Vn based on the principal component analysis of the point group consisting of the reflection point of interest RPi and the reference reflection point RPr. When the process of S134 is completed, the flow shifts to S140.

According to the second embodiment described above, the normal line of the target reflection point RPi is calculated based on the target reflection point RPi and a plurality of reference reflection points RPr whose inter-point distances between the target reflection point RPi are within the allowable range. A direction is calculated. Therefore, the normal direction can be easily calculated based on multiple reflection points on the same reflecting object. Therefore, the calculation accuracy of the normal direction can be improved.

(Third embodiment)
In the third embodiment, a modified example of the image processing apparatus 100 in the first embodiment will be described. In FIGS. 7 to 9, constituent elements denoted by the same reference numerals as in the drawings of the first embodiment are similar constituent elements and have similar effects.

In the third embodiment, the image processing apparatus 100 has in advance a correspondence table CT that stores the correspondence between detected waveform information and normals for each pixel (see FIG. 7). Specifically, the correspondence table CT stores in the memory 101 the magnitude of the inclination of the normal corresponding to the detected waveform information for each reflection characteristic of the reflecting object and the distance to the reflecting object. In addition, the correspondence table CT stores the magnitude of the inclination of the normal corresponding to the detected waveform information for each Lambertian reflection characteristic and distance to the reflecting object. The detected waveform information stored in the correspondence table CT may be detected waveform information obtained by extracting characteristic points, or may be complete detected waveform information.

For example, the correspondence table CT stores at least one of the waveform peak value, pulse width, and bottom width as detected waveform information. The peak value is the maximum signal intensity of the waveform (signal intensity p at t3 in FIG. 8). The pulse width is the time width obtained as the absolute value of the difference between the half-value points (see t2 and t4 in FIG. 8) where the signal intensity is half the peak value at the rise and fall of the pulse. The tail width is the time width obtained as the absolute value of the difference between the pulse start time (see t1 in FIG. 8) and the pulse end time (see t5 in FIG. 8). The pulse start time is the time at which the difference in intensity between the background signal obtained by excluding the pulse signal from the acquired signal and the pulse signal is equal to or greater than a predetermined threshold value at the rising edge of the pulse. The background signal may include a signal derived from the ambient light, or may be obtained by removing the pulse signal from the acquired signal from which the signal derived from the ambient light has been removed. The pulse end time is the time at which the difference in intensity between the background signal and the pulse signal becomes equal to or less than a predetermined threshold or less than the threshold at the trailing edge of the pulse.

For example, for the same reflection characteristics and distance, the larger the peak value, the greater the inclination of the normal direction with respect to the reference direction. Also, the greater the pulse width, the greater the inclination in the normal direction with respect to the reference direction. Furthermore, the greater the skirt width, the greater the inclination in the normal direction with respect to the reference direction. The correspondence table CT stores such a relationship as a correspondence relationship between detected waveform information and normals. A parameter that correlates with the magnitude of the gradient in the normal direction, such as the peak value, pulse width, and bottom width, can also be called a waveform feature amount.

The normal line calculation unit 130 calculates the inclination of the normal line by comparing the reflection characteristics, the distance, and the detected waveform information at each reflection point with the correspondence table CT. That is, the normal calculation unit 130 calculates the magnitude of the inclination of the normal of one reflection point based on the information of the corresponding single pixel. If the reflection characteristic of the reflection point is unknown, the normal calculation section 130 assumes that the reflection point has the Lambertian reflection characteristic, and collates the distance and the detected waveform information with the correspondence table CT.

Next, detailed processing of normal vector calculation among the distance measurement correction methods executed by the image processing apparatus 100 in the third embodiment will be described below with reference to the flowchart of FIG. Note that the description in the first embodiment is used except for detailed processing. First, in S135, the reflection characteristic and the distance of the target reflection point RPi are obtained. Next, in S136, the slope of the normal line is calculated by collating the reflection characteristics, distance, and detected waveform information of the target reflection point RPi with the correspondence table CT.

According to the third embodiment described above, the reflection of interest is detected based on the detected waveform information obtained by detecting the reflected light from the reflection point of interest RPi and the relationship information between the predetermined distance and the waveform of the reflected light. A normal direction of the point RPi is calculated. According to this, since the normal direction of the target reflection point RPi is calculated based on the correspondence, it is possible to reduce the amount of calculation when specifying the normal direction. Also, the distance to a reflective object, such as a distant object or a small object, for which it is difficult to detect reflection data across a plurality of pixels, can be corrected more accurately.

(Fourth embodiment)
In the fourth embodiment, a modified example of the image processing apparatus 100 in the first embodiment will be described. In FIGS. 10 and 11, the components denoted by the same reference numerals as in the drawings of the first embodiment are the same components and have the same effects.

In the fourth embodiment, as shown in FIG. 10, the image processing apparatus 100 includes a reflecting object determination section 115 as a functional section. The reflecting object determination unit 115 determines whether or not reflected light information from a specific target T is within a single pixel. Whether or not the reflected light information from the target T is within a single pixel is a condition for switching the method of calculating the inclination feature amount of the target reflection point RPi. Therefore, the reflecting object determination unit 115 can also be said to be a condition determination unit that determines whether or not the condition is satisfied.

The normal calculation unit 130 changes the normal calculation method for each pixel based on the determination result. Specifically, the normal calculation unit 130 switches between calculation of a normal vector based on multi-pixel information and calculation of a normal vector based on single-pixel information according to the type of the reflecting object.

Specifically, when the reflected light information from the specific reflecting object is contained within a single pixel, the normal vector calculation unit 130 calculates the normal vector Vn based on the single pixel information. On the other hand, when the reflected light information from the specific reflecting object is detected over a plurality of pixels, the normal vector calculation unit 130 calculates the normal vector Vn based on the information of the plurality of pixels.

Next, the flow of the distance measurement method executed by the image processing apparatus 100 in cooperation with the functional blocks will be described below with reference to FIG.

If an affirmative determination is made in S110, the flow shifts to S115. In S115, the reflecting object determination unit 115 determines whether or not the reflecting object is contained within a single pixel. If it is determined that it does not fit, the normal vector calculation unit 130 calculates a normal vector based on the multiple-pixel information in S130 to S132. On the other hand, if it is determined that it fits, the normal vector calculation unit 130 calculates a normal vector based on the single pixel information in S135-S136.

According to the fourth embodiment described above, a normal vector is calculated based on single pixel information for a reflecting object that is so far or small that the reflected light information can be accommodated within a single pixel. Then, for a reflecting object whose reflected light information is close to or large enough to cover a plurality of pixels, a normal vector is calculated based on the information of a plurality of pixels.

(Fifth embodiment)
A modification of the image processing apparatus 100 according to the third embodiment will be described in the fifth embodiment. In FIGS. 12 to 14, constituent elements assigned the same reference numerals as in the drawings of the third embodiment are similar constituent elements and have similar effects.

As shown in FIG. 12, the image processing apparatus 100 according to the fifth embodiment includes functional units such as a scan setting unit 105, a pixel information acquisition unit 110, a change degree calculation unit 135, a reliability calculation unit 140, and a distance correction unit 150. be built.

The scan setting unit 105 sets the scanning speed of the laser beam by the actuator 4 . Specifically, the scan setting unit 105 sets a first scan cycle in which scanning is performed at a predetermined scanning speed and a second scan cycle in which scanning is performed at a faster or slower scanning speed than the first scan cycle. , to set different scan speeds. For example, the scan setting unit 105 sets the first scan period and the second scan period to be alternately repeated, thereby causing the scan to be performed over a plurality of periods. Alternatively, the scan setting unit 105 may repeat a pattern in which a plurality of scans in the first scan period are continuously performed, and then a plurality of scans in the second scan period.

The degree-of-change calculator 135 calculates the degree of shape change for each detected waveform for each scan speed, which is obtained by detecting reflected light from the same reflection point scanned at different scan speeds. Specifically, the degree-of-change calculator 135 calculates the degree of change in the detected waveform between the first scan period and the second scan period. The degree-of-change calculating unit 135 calculates, for example, the difference between the waveform feature amounts as the degree of change. The waveform feature amount is, for example, at least one of a peak value, pulse width, and bottom width. The degree of change may be an evaluation value of the magnitude of change obtained by integrating a plurality of feature quantities. The degree of change is an example of the “tilt feature amount”.

The distance corrector 150 corrects the distance according to the degree of change. Specifically, as shown in FIG. 13, when the inclination of the partial surface SA with respect to the reference plane R is large, the change in shape of the waveform is greater than when the inclination is relatively small. Specifically, the larger the slope, the smaller the peak value and the larger the pulse width and tail width. Therefore, the greater the degree of change, the greater the divergence from the true value of the distance. Therefore, distance correction section 150 increases the correction value as the degree of change increases.

Next, the flow of the distance measurement method executed by the image processing apparatus 100 in cooperation with the functional blocks will be described below with reference to FIG.

First, in S200, the scan setting unit 105 sets different scan speeds between the first scan cycle and the second scan cycle. In S210, the pixel information acquisition unit 110 acquires pixel information of the target reflection point RPi based on the above settings. At this time, the pixel information acquisition unit 110 acquires both pixel information in the first scan cycle and pixel information in the second scan cycle. The following S220 is the same processing as S110.

If an affirmative determination is made in S220, the flow shifts to S230. In S230, the degree-of-change calculator 135 calculates the degree of change in the waveform for each cycle. In subsequent S240, the reliability calculator 140 calculates the reliability of the detected waveform. The reliability of the detected waveform may be calculated based on the signal intensity, ambient light intensity, and the like. Then, in S250, the distance correction unit 150 performs distance correction based on the degree of change and reliability of the waveform. The following S260 and S270 are the same processes as S160 and S170. In the above, S210 and S220 are an example of the "acquisition process", S230 is an example of the "feature amount calculation process", and S250, S260, and S270 are examples of the "correction process".

(Sixth embodiment)
In the sixth embodiment, a modified example of the image processing apparatus 100 in the fifth embodiment will be described. In the sixth embodiment, the scan setting unit 105 changes the scan speed midway within the detection range PR corresponding to one pixel. Specifically, within the detection range PR corresponding to one pixel, as shown in FIG. A different range of scanning speed for B and B is set.

The pixel information acquisition unit 110 acquires pixel information for each small pixel obtained by dividing one pixel into a high speed range A and a low speed range B, respectively.

The degree-of-change calculating unit 135 detects the reflected light from the reflecting point whose scanning speed is changed in the middle, for each small pixel obtained by dividing the pixel corresponding to the reflecting point so as to correspond to the scanning speed. The degree of change in the shape of is calculated. Specifically, the change degree calculator 135 calculates the degree of change between the detected waveform in the high speed range A and the detected waveform in the low speed range B as the slope feature amount. The degree-of-change calculator 135 may use the amount of change in at least one or more waveform feature values as the degree of change, or may use the amount of change based on all points of the detected waveform as the degree of change. Such a degree of change is an example of the "tilt feature amount".

The distance correction unit 150 corrects the distance detected at the corresponding pixel according to the degree of change. When the inclination of the reflecting surface with respect to the reference plane is large, the change in shape of the waveform detected in each range A and B is greater than when the inclination is relatively small. Specifically, the larger the slope, the smaller the peak value and the larger the pulse width and tail width. Therefore, the greater the degree of change, the greater the divergence from the true value of the distance. Therefore, distance correction section 150 increases the correction value as the degree of change increases.

In the distance measurement method according to the sixth embodiment, in S200, the scan setting unit 105 selects a high-speed range A in which the scanning speed is high and a low-speed range B in which the scanning speed is lower than that of the high-speed range A. set. In S210, the pixel information acquisition unit 110 acquires pixel information of the target reflection point RPi based on the above settings. At this time, the pixel information acquiring unit 110 acquires pixel information for each of small pixels obtained by dividing one pixel into a high speed range A and a low speed range B, respectively. Further, in S230, the degree-of-change calculation unit 135 calculates the degree of change in the waveform for each small pixel.

(Other embodiments)
The disclosure herein is not limited to the illustrated embodiments. The disclosure encompasses the illustrated embodiments and variations thereon by those skilled in the art. For example, the disclosure is not limited to the combinations of parts and/or elements shown in the embodiments. The disclosure can be implemented in various combinations. The disclosure can have additional parts that can be added to the embodiments. The disclosure encompasses omitting parts and/or elements of the embodiments. The disclosure encompasses permutations or combinations of parts and/or elements between one embodiment and another. The disclosed technical scope is not limited to the description of the embodiments. The disclosed technical scope is indicated by the description of the claims, and should be understood to include all changes within the meaning and range of equivalents to the description of the claims. .

In the above-described embodiments, the dedicated computer that configures the image processing device 100 is the electronic control device that configures the LiDAR device 1 . Alternatively, the dedicated computer that constitutes the image processing apparatus 100 may be an operation control ECU mounted on the vehicle, or may be an actuator ECU. Alternatively, the dedicated computer configuring the image processing apparatus 100 may be a locator ECU or a navigation ECU. Alternatively, the dedicated computer configuring the image processing apparatus 100 may be an HCU.

In the above-described second embodiment, the normal vector calculation unit 130 calculates the normal vector Vn based on the principal component analysis. You may

In the above-described third embodiment, the normal line calculation unit 130 is based on the correspondence table CT that stores the magnitude of the inclination of the normal line corresponding to the detected waveform information for each reflection characteristic of the reflecting object and the distance to the reflecting object. and calculate the normal direction. Alternatively, the normal calculation unit 130 may calculate the normal direction based on a function representing the correspondence relationship.

In the modified examples of the fifth embodiment and the sixth embodiment described above, the change degree calculator 135 may further calculate the normal direction based on the change degree of the waveform. In this case, the distance correction unit 150 may perform distance correction based on the normal direction.

Image processing apparatus 100 may be a dedicated computer that includes at least one of a digital circuit and an analog circuit as a processor. Here, digital circuits in particular include, for example, ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), SOC (System on a Chip), PGA (Programmable Gate Array), and CPLD (Complex Programmable Logic Device). at least one of Such digital circuits may also include memory storing programs.

Image processing apparatus 100 may be provided by a single computer or a set of computer resources linked by a data communication device. For example, some of the functions provided by the image processing apparatus 100 in the above-described embodiments may be realized by another ECU.

1 LiDAR device (ranging device), 4 actuator, 100 image processing device (ranging correction device), 102 processor, 110 pixel information acquisition unit (acquisition unit), 130 normal line calculation unit (feature amount calculation unit), 135 change Degree calculation unit (feature amount calculation unit) 150 Distance correction unit (correction unit) DL Line of sight direction R Reference plane.

Claims (26)

A distance measurement device (1) having a processor (102) detects light reflected from a reflection point of a target against irradiation of light by pixels, thereby correcting a distance measurement result of a distance measurement device (1) for measuring a distance to the reflection point. A ranging correction device,
an acquisition unit (110) for acquiring related information, which is information related to the distances detected at corresponding pixels, for the plurality of reflection points;
a feature quantity calculation unit (130, 135) for calculating a tilt feature quantity related to a magnitude of tilt with respect to a reference plane (R) for a partial surface of the target constituting the reflection point;
a correction unit (150) for correcting the distance to each reflection point based on the inclination feature amount;
A ranging correction device. 2. The distance measurement correcting device according to claim 1, wherein the feature amount calculator calculates a normal direction of the reflection point as the tilt feature amount. For each of the reflection points, the feature amount calculation unit calculates the normal direction based on three-dimensional position information of a target reflection point for calculating the normal direction and a plurality of reference reflection points other than the target reflection point. 3. The distance measurement correcting device according to claim 2, which calculates . The feature amount calculation unit extracts a plurality of the reference reflection points whose point-to-point distances from the target reflection point are within an allowable range, and extracts the target reflection point based on the target reflection point and the plurality of reference reflection points. 4. The distance measurement correcting device according to claim 3, wherein the normal direction of the reflection point is calculated. The feature amount calculation unit calculates, for each of the reflection points, detection waveform information obtained by detecting the reflected light from the reflection point of interest for calculating the normal direction, and the predetermined distance and the reflected light. 5. The distance measurement correcting device according to claim 2, wherein the normal direction of the target reflection point is calculated based on the relationship information with the waveform. 6. The correction unit according to any one of claims 3 to 5, wherein the correction unit determines the correction amount of the distance based on the magnitude of the inclination of the normal direction of the target reflection point with respect to the line-of-sight direction (DL) of the corresponding pixel. 1. A range finding correction device according to claim 1. The range finder comprises an actuator (4) that scans the radiated light,
The feature amount calculation unit is obtained by detecting the reflected light from the same reflection point scanned at different scan speeds, based on the degree of change in shape of each detected waveform for each scan speed, 7. The distance measurement correction device according to claim 2, wherein the normal direction of the reflection point is calculated. The range finder comprises an actuator (4) that scans the radiated light,
The feature amount calculation unit detects the reflected light from the reflection point whose scanning speed is changed in the middle for each small pixel obtained by dividing the pixel corresponding to the reflection point so as to correspond to the scanning speed. 7. The distance measurement correcting device according to claim 2, wherein the normal direction of the reflection point is calculated based on the degree of change in shape of each detected waveform. The distance measurement correction device according to any one of claims 2 to 8, wherein the correction unit increases the correction amount of the distance as the calculation reliability of the normal direction increases. The distance measurement correcting device according to any one of claims 2 to 9, wherein the correction unit increases the correction amount of the distance as the distance to the reflection point before correction increases. The range finder comprises an actuator (4) that scans the radiated light,
The feature amount calculation unit calculates the degree of change in shape for each detection waveform for each of the scan speeds, which is acquired by detecting the reflected light from the same reflection point scanned at different scan speeds, by calculating the slope. 2. The distance measurement correcting device according to claim 1, wherein the distance measurement correction device is calculated as a feature amount. The range finder comprises an actuator (4) that scans the radiated light,
The feature amount calculation unit detects the reflected light from the reflection point whose scanning speed is changed in the middle for each small pixel obtained by dividing the pixel corresponding to the reflection point so as to correspond to the scanning speed. 2. A distance measurement correcting apparatus according to claim 1, wherein the degree of change in shape of each detected waveform is calculated as said inclination feature quantity. A processor (102) for correcting the distance measurement result of a distance measuring device (1) that measures the distance to a reflection point by detecting light reflected from a reflection point of a target with respect to irradiation of light by pixels. A ranging correction method implemented comprising:
an acquisition process (S100, S110; S210, S220) for acquiring related information, which is information related to the distance detected at corresponding pixels, for the plurality of reflection points;
Feature quantity calculation processes (S130, S131, S132; S133, S134; S135, S136; S230) and
a correction process (S150, S160, S170; S250, S260, S270) for correcting the distance to each of the reflection points based on the inclination feature amount;
Ranging correction method including. 14. The distance measurement correcting method according to claim 13, wherein in said feature amount calculation process, a normal direction of said reflection point is calculated as said inclination feature amount. In the feature amount calculation process, for each of the reflection points, the normal direction 15. The distance measurement correction method according to claim 14, wherein . In the feature amount calculation process, a plurality of the reference reflection points having point-to-point distances from the target reflection point within an allowable range are extracted, and the target reflection point is extracted based on the target reflection point and the plurality of reference reflection points. 16. The distance measurement correction method according to claim 15, wherein the normal direction of the reflection point is calculated. In the feature amount calculation process, for each of the reflection points, detection waveform information obtained by detecting the reflected light from the target reflection point for calculating the normal direction, the predetermined distance and the reflected light 17. The distance measurement correcting method according to any one of claims 14 to 16, wherein the normal direction of the target reflection point is calculated based on information related to the waveform. 18. Any one of claims 15 to 17, wherein in the correction process, the amount of correction of the distance is determined based on the magnitude of the inclination of the normal direction at the target reflection point with respect to the line-of-sight direction (DL) of the corresponding pixel. 1. A distance measurement correction method according to 1. The range finder comprises an actuator (4) that scans the radiated light,
In the feature amount calculation process, based on the degree of change in shape for each detection waveform for each of the scan speeds, obtained by detecting the reflected light from the same reflection point scanned at different scan speeds, 19. The distance measurement correcting method according to any one of claims 14 to 18, wherein the normal direction of the reflection point is calculated. The range finder comprises an actuator (4) that scans the radiated light,
In the feature amount calculation process, the reflected light from the reflection point whose scanning speed is changed midway is detected for each small pixel obtained by dividing the pixel corresponding to the reflection point so as to correspond to the scanning speed. 19. The distance measurement correcting method according to any one of claims 14 to 18, wherein the normal direction of the reflection point is calculated based on the degree of shape change for each detected waveform. 21. The distance measurement correction method according to any one of claims 14 to 20, wherein in the correction process, the correction amount of the distance is increased as the calculation reliability of the normal direction increases. 22. The distance measurement correction method according to any one of claims 14 to 21, wherein in the correction process, the larger the distance to the reflection point before correction, the larger the correction amount of the distance. The range finder comprises an actuator (4) that scans the radiated light,
In the feature amount calculation process, for each detection waveform for each of the scan speeds, obtained by detecting the reflected light from the same reflection point scanned at different scan speeds, the degree of change in shape is calculated as the slope. 14. The distance measurement correction method according to claim 13, wherein the distance measurement is calculated as a feature amount. The range finder comprises an actuator (4) that scans the radiated light,
In the feature amount calculation process, the reflected light from the reflection point whose scanning speed is changed midway is detected for each small pixel obtained by dividing the pixel corresponding to the reflection point so as to correspond to the scanning speed. 14. The distance measurement correcting method according to claim 13, wherein the degree of change in shape of each detected waveform is calculated as the inclination feature amount. A processor (102) for correcting a distance measurement result of a distance measuring device (1) that measures a distance to a reflection point by detecting reflected light from a reflection point of a target against irradiation of light by pixels. A ranging correction program including instructions to be executed,
Said instruction
an acquisition process (S100, S110; S210, S220) for acquiring related information, which is information related to the distance detected at the corresponding pixels, for the plurality of reflection points;
Feature quantity calculation processes (S130, S131, S132; S133, S134; S135, S136; S230) and
a correction process (S150, S160, S170; S250, S260, S270) for correcting the distance to each of the reflection points based on the inclination feature amount;
Ranging correction program including. A distance measuring device having a processor (102) and measuring a distance to a reflecting point of a target object by detecting light reflected from the reflecting point with respect to irradiation of light by pixels,
an acquisition unit (110) for acquiring related information, which is information related to the distances detected at corresponding pixels, for the plurality of reflection points;
a feature quantity calculation unit (130, 135) for calculating a tilt feature quantity related to a magnitude of tilt with respect to a reference plane (R) for a partial surface of the target constituting the reflection point;
a correction unit (150) for correcting the distance to each reflection point based on the inclination feature amount;
A rangefinder with a

Images