JP2023104435A - Obstacle detection system - Google Patents

Abstract

To provide an obstacle detection system which materializes reduction of the burden of arithmetic processing, and furthermore enables misdetection to be suppressed.SOLUTION: The obstacle detection system performs angle calculation for each pixel on the basis of distance image data acquired by a distance image sensor 1 to thereby detect the presence of an obstacle. The distance information of each pixel obtained when a flat ground surface is photographed, is obtained as reference data from the installed position of the distance image sensor, reference data is excluded from distance measurement data acquired by the distance image sensor, and orthogonal coordinate conversion is performed on the distance measurement data of the distance image sensor.SELECTED DRAWING: Figure 1

Description

The present invention relates to an obstacle detection system that detects the presence or absence of an obstacle by calculating an angle for each pixel based on distance image data (polar coordinate data) acquired by a distance image sensor.

Patent Literature 1 describes a ground detection method and a ground detection device for determining the ground from point cloud data.
Further, Patent Literature 2 describes a construction machine equipped with a distance measuring sensor for detecting obstacles and suppressing erroneous detection due to reflection of the machine itself.

JP 2020-42822 A JP 2018-159194 A

By the way, in the case of determination using point cloud data as in Patent Document 1, it is necessary to perform angle calculation such as orthogonal coordinate transformation for each pixel, so the amount of calculation increases. In addition, when projecting point cloud data onto a mesh, it is also necessary to allocate memory for areas where the point cloud is not projected (empty space), so the amount of memory used increases. Therefore, there is a problem that the load of arithmetic processing is large.

On the other hand, Patent Document 2 makes it possible to detect an obstacle in the vicinity of the undercarriage while preventing the undercarriage of a construction machine from being erroneously determined to be an obstacle. However, at a site where construction machinery is used, there is a possibility that a sand dune or a slope of the ground may be erroneously detected as an obstacle.

SUMMARY OF THE INVENTION The present invention has been made in view of the circumstances as described above, and an object of the present invention is to provide an obstacle detection system capable of suppressing erroneous detection while reducing the burden of arithmetic processing.

An obstacle detection system according to an aspect of the present invention is an obstacle detection system that detects the presence or absence of an obstacle by performing angle calculation for each pixel based on distance image data acquired by a distance image sensor, Distance information of each pixel when a flat ground is photographed from the installation position of the distance image sensor is obtained as reference data, the reference data is excluded from the distance measurement data obtained by the distance image sensor, and the distance image sensor is used. is characterized in that orthogonal coordinate transformation is performed on the distance measurement data.

According to the present invention, by removing the reference data from the distance measurement data acquired by the distance image sensor, it is possible to reduce the amount of calculation and the amount of memory used. Also, by excluding the reference data from the ranging data, it is possible to discriminate between the ground and the detection target, thereby suppressing erroneous detection. Therefore, it is possible to provide an obstacle detection system capable of suppressing erroneous detection while reducing the burden of arithmetic processing.

1 is a block diagram showing a schematic configuration of an obstacle detection system according to an embodiment of the invention; FIG. 4 is a flowchart showing a reference data creation procedure executed by a reference data creation unit; FIG. 4 is a schematic diagram for explaining creation of reference data executed by a reference data creation unit; 5 is a flowchart showing a procedure for selecting pixels to be excluded, which is executed by an excluded pixel selection unit; It is for explaining the first comparison method, and is a diagram showing a detection target and a non-detection target. It is for explaining the first comparison method, and is a diagram showing a difference between a detection target and a non-detection target. It is for explaining the second comparison method, and is a diagram showing a difference between a detection target and a non-detection target. FIG. 4 is a schematic diagram for explaining a case where a 2D image is created from actual ranging data; FIG. 4 is a diagram showing a simplified 2D image obtained by taking a difference between virtual ground data and actual distance measurement data; FIG. 10 is a diagram showing a simplified 2D image of an obstacle other than the ground remaining as a difference value; FIG. 10 is a diagram showing a simplified 2D image for explaining a portion with a large gap with surrounding pixels; FIG. 10 is a diagram for determining a ground pixel reference, and for explaining a difference in depth value between pixels. FIG. 4 is a diagram showing a simplified 2D image obtained by taking a difference between virtual ground data and actual distance measurement data; FIG. 10 is a diagram showing a simplified 2D image obtained by combining pixels with small gaps in the vicinity of ground pixels to form ground pixels; FIG. 4 is a simplified 2D image showing sensing target data that has not been combined with ground data; FIG. 10 is a diagram for explaining the exclusion of ground pixels, and is a diagram showing a simplified 2D image in which pixels determined to be ground pixels are excluded from data to be detected. FIG. 4 is a diagram showing distance measurement data detected by a distance image sensor; FIG. 18 is a diagram showing ranging data of the ground extracted from FIG. 17; FIG. 18 is a diagram showing ranging data of a detection target extracted from FIG. 17;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 shows a schematic configuration of an obstacle detection system according to an embodiment of the invention. This system includes a distance image sensor 1 such as a ToF (Time of Flight) sensor that acquires distance measurement data, and the distance of each pixel when the flat ground (virtual ground) is photographed from the installation position of this distance image sensor 1. It is provided with a reference data creation unit 2 that obtains information as reference data, an exclusion pixel selection unit 3 that selects exclusion pixels, and a detection processing unit 4 that detects obstacles such as irregularities on the ground and people.

The exclusion pixel selection unit 3 creates comparison data for comparing the distance measurement data acquired by the distance image sensor 1 and the distance information of each pixel obtained by the reference data creation unit 2, and measures the distance based on the created comparison data. The ground data is excluded from the data and output to the detection processing unit 4 . In this manner, each time the distance image sensor 1 acquires distance information, the distance measurement data is compared with the reference data.

FIG. 2 shows a reference data creation procedure executed by the reference data creation unit 2 . The creation of this reference data is performed for the first time, and processing for each frame is not performed.
First, information on the mounting position and angle of the distance image sensor 1 is obtained (step S1). Subsequently, the distance from each pixel to the virtual ground is calculated from the mounting position and angle of the distance image sensor 1, and reference data is created (step S2). That is, when the distance image sensor is activated, the output result of the distance image sensor (distance value at each pixel) when an image of an ideal (flat) virtual ground with no inclination or undulations from the installation position and angle of the distance image sensor is captured. is calculated, and the distance information of each pixel is used as reference data.

FIG. 3 is a schematic diagram for explaining how reference data is created by the reference data creation unit 2. As shown in FIG. Project the point cloud data onto the mesh and the unprojected part becomes the space. When creating a 2D image of a virtual ground without inclination or undulation, projection onto a plane will result in a shape with pincushion distortion, as shown in FIG. 3(a). As a result, the 2D image becomes as shown in FIG. 3(b).

FIG. 4 shows a procedure for selecting pixels to be excluded, which is executed by the pixel-to-exclude selecting section 3 . This process is performed each time the distance measurement data is acquired.
First, the distance image sensor 1 acquires distance measurement data (step S11), performs invalid pixel determination, and excludes abnormal values in the distance measurement data (step S12).

Next, comparison data is created by comparing the reference data created in the procedure of FIG. 2 with the distance measurement data created in step S12 (step S13). Subsequently, neighboring pixels with small change amounts of comparison data are combined (step S14). Next, ground pixel determination is performed (step S15). In the ground pixel determination, a ground pixel having a small difference between the reference data and the distance measurement data (comparison data ≈ 0) is regarded as a ground pixel, and a pixel connected to the ground pixel is also regarded as a ground pixel.

That is, pixels whose amount of change with respect to adjacent pixels is within a predetermined threshold value are regarded as ground pixels and excluded from the distance measurement data. In addition, pixels with a smaller amount of change from neighboring pixels than the predetermined threshold value are recognized as inclined surfaces of the ground, and larger pixels are detected as an obstacle. After that, exclusion pixel determination is performed to exclude ground pixels and abnormal pixels from obstacle detection targets (step S16).

5 and 6 are respectively for explaining the first comparison method. As shown in FIG. is an inclined surface such as a sand dune, the depth difference from the reference surface is as shown in FIGS. 6(a) and 6(b). FIG. 6A shows the depth difference between a crouched person and the ground, which is the reference plane, and the transition of the ranging data is not smooth as indicated by the arrows. On the other hand, as shown in FIG. 6(b), the depth difference between the inclined surface and the ground has a smooth transition of the ranging data as indicated by the arrow.

Thus, the first comparison method is to take the difference between the distance measurement data and the reference data, ie, "reference data - distance measurement data". This first comparison method is suitable when a high processing speed is required. However, when the resolution of the distance image sensor 1 is low or when the laser irradiation angle is close to horizontal, the effect is reduced (the error in the result for each pixel increases).

FIG. 7 is for explaining the second comparison method, and shows the difference between the detection target and the non-detection target. The second comparison method is to take the difference between the distance measurement data and the reference data, and divide the difference by the reference data "([reference data]-[range measurement data])/[reference data]". FIG. 7(a) shows the difference in height between a crouched person and the ground, which is the reference plane, and the transition of the distance measurement data is not smooth as indicated by the arrow.

On the other hand, as shown in FIG. 7(b), the height difference between the inclined surface and the ground makes the transition of the ranging data smoother as indicated by the arrows.
In this way, when the height difference from the reference plane is taken, the processing speed becomes slower than that of the first comparison method 1 by the division amount, but the same scale can be used for any pixel.

FIG. 8 is a schematic diagram for explaining a case where a 2D image is created from actual ranging data. As shown in FIG. 8(a), when the object to be detected is a crouched person, a 2D image as shown in FIG. 8(b) is obtained by acquiring distance measurement data. This 2D image is an image in which reflected light from a crouched person is superimposed on the front against the background of the 2D image shown in FIG. 3(b).

Next, when the difference between the actual distance measurement data shown in FIG. 8B and the virtual ground data shown in FIG. 3B is taken, a 2D image as shown in FIG. 9 is obtained. If the ground is flat (no slope or undulation), the difference value is "0". On the other hand, obstacles other than the ground remain as difference values as shown in FIG. In FIG. 11, portions with large gaps to peripheral pixels are indicated by x marks.

In a portion with a large gap from surrounding pixels, pixels whose amount of change from adjacent pixels is within a predetermined threshold are regarded as ground pixels and excluded from the distance measurement data. That is, pixels with a smaller amount of change from adjacent pixels than a predetermined threshold value are recognized as inclined surfaces of the ground, and larger pixels are detected as an obstacle. In other words, obstacles that can be overcome are excluded, and obstacles that cannot be overcome are detected as obstacles.

More specifically, when a specific ground pixel is used as a reference, it is possible to change neighboring pixels to ground pixels only when both the pixels above and below the ground pixel and the pixels on the left and right of the ground pixel are changeable. Neighboring pixels can be changed when one of the difference values of upper and lower (or left and right) pixels is the same and the other is the same, increases (including rapid increases), or decreases. Also, it is a case where one of the difference values between the upper and lower (or left and right) pixels increases (including a rapid increase) and the other decreases.
Here, the fact that the pixel difference value is the same value means that the amount of change from the adjacent pixel is within a predetermined threshold value.

On the other hand, if the reference ground pixel is an inflection point (the difference value between the upper and lower pixels is increasing and increasing (or decreasing and decreasing), or the difference value between the left and right pixels is increasing and increasing (or decreasing and decreasing), , the neighboring pixels cannot be changed, and if the upper and lower (or left and right) pixels include a drastic decrease, the neighboring pixels cannot be changed.

FIG. 12 is for determining the ground pixel reference, and shows the difference in depth value at each pixel. FIG. 13 is a 2D image obtained by taking the difference between the virtual ground data and the actual distance data. If the difference value is within the range of the predetermined threshold, it is taken as the reference pixel of the ground. In FIG. 13, pixel 11 is determined to be the ground reference pixel.

In FIG. 14, pixels 12-1 to 12-4 with small gaps near the ground pixels are combined with the reference pixel 11 of the ground to form the ground pixels. Pixels are combined only with data that can be regarded as ground pixels, and as shown in FIG. 15, the data not combined with the ground data are used as detection target data (pixels 13 hatched downward to the left).

FIG. 16 is for explaining the exclusion of ground pixels, in which the pixels 13 determined to be "ground pixels" are excluded from the data to be detected.
Note that in FIG. 16, when a jump point or the like occurs, it is not removed in this process, but is left to the subsequent detection process.

FIG. 17 shows distance measurement data detected by the distance image sensor 1 described above. FIG. 18 shows ranging data of the ground (inclined surface) extracted from FIG. 17, and FIG. 19 shows the detection target (person) extracted from FIG.
As shown in FIG. 19, by excluding the ground (sloping surface) ranging data from the ranging data detected by the range image sensor 1, the amount of data for objects to be detected for obstacles can be greatly reduced.

As described above, according to the present invention, processing can be performed without performing complicated calculations, resulting in high throughput. Also, the memory usage can be reduced to the number of pixels used in the sensor. Moreover, even when creating a point cloud or the like in subsequent detection processing, data can be thinned out in advance, so the amount of calculation can be minimized. Furthermore, it is possible to prevent erroneous detection of a sand dune, slope of the ground, or the like as an obstacle.

Therefore, according to the obstacle detection system of the present invention, by excluding the data of the ground portion, the amount of calculation and the amount of memory used for each frame can be suppressed, and the burden of arithmetic processing can be reduced. False positives can be suppressed by differentiating

The configurations, control procedures, and the like described in the above embodiments are merely schematic representations to the extent that the present invention can be understood and implemented. Therefore, the present invention is not limited to the described embodiments, but can be modified in various forms without departing from the scope of the technical idea indicated in the claims.

For example, the obstacle detection system may be mounted on the construction machine, and the distance image data of the construction machine itself reflected in the distance image may be further excluded from the distance image data acquired by the distance image sensor.

In addition, since the amount of light in the data of the edge portion tends to be small, detection accuracy can be improved by using the attenuation of the light amount value of the edge portion and differentiating the detection target and the ground data by edge extraction. A distance image sensor (MEMS: Micro Electro Mechanical Systems) can acquire distance data and light intensity data on the same axis. measured by the amount of light.

Furthermore, it is also possible to differentiate the object to be detected and the ground data by AI learning of combined data of the distance image data and the light amount data. Since significant features are detected in both the distance image and the light intensity image, AI learning can improve the accuracy compared to using only the data of the distance image.

1... distance image sensor, 2... reference data creation unit, 3... exclusion pixel selection unit, 4... detection processing unit, 11... pixel (ground reference pixel), 12-1 to 12-4... pixel (pixel with small gap ), 13... pixel (data to be detected)

Claims (5)

An obstacle detection system that detects the presence or absence of an obstacle by calculating an angle for each pixel based on distance image data acquired by a distance image sensor,
Distance information of each pixel when a flat ground is photographed from the installation position of the distance image sensor is obtained as reference data, the reference data is excluded from the distance measurement data obtained by the distance image sensor, and the distance image sensor is used. An obstacle detection system characterized by performing orthogonal coordinate transformation on distance measurement data. Each time the distance image sensor acquires the distance measurement data, the reference data and the distance measurement data are compared pixel by pixel, the difference between the distance measurement data and the reference data is obtained, and the amount of change from the adjacent pixels is calculated. 2. The obstacle detection system according to claim 1, wherein pixels within a predetermined threshold are regarded as ground pixels and are excluded from range finding data. 3. The obstacle detection system according to claim 2, wherein a pixel whose amount of change from said adjacent pixel is smaller than said predetermined threshold value is recognized as an inclined surface of the ground, and a pixel whose amount of change is larger than said predetermined threshold value is detected as an obstacle. 3. The obstacle detection system according to claim 2, wherein after obtaining a difference between said reference data and said ranging data, said difference is divided by said reference data. The obstacle detection system according to any one of claims 1 to 4 is mounted on a construction machine,
An obstacle detection system, wherein distance image data of the construction machine itself reflected in the distance image is further excluded from the distance image data acquired by the distance image sensor.