JP7227227B2 - Object identification system, automobile, vehicle lighting - Google Patents

Description

The present invention relates to object identification systems.

Candidates for automobile sensors include LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), cameras, millimeter wave radars, ultrasonic sonars, and the like. Compared to other sensors, LiDAR is capable of (i) object recognition based on point cloud data, and (ii) high-precision detection even in bad weather due to active sensing. and (iii) the ability to measure over a wide range.

FIG. 1 is a diagram showing a ranging system 1 using LiDAR2. The LiDAR2 has N lines of resolution in the vertical direction, and each layer L ₁ to L _N has a resolution of Δθ degrees in the horizontal direction. Light emitted from LiDAR2 is reflected from object OBJ. The point cloud data generated by LiDAR2 represents the distance _ri between LiDAR2 and each point _pi .

Since the point cloud data represents the shape of the object, the type of object (vehicle, pedestrian, etc.) can be determined by processing the point cloud data.

In object identification based on point cloud data generated by LiDAR, as preprocessing, clustering is performed to divide the point cloud data for each object. FIG. 2 is a diagram for explaining a conventional clustering algorithm. FIG. 2 shows a plan view of the measurement system viewed from above. In this algorithm, when a certain point (referred to as a point of interest) in the point cloud data is focused, the points included in a sphere with a radius of r centered on the point of interest are classified into the same cluster. This algorithm is also used in Autoware known as open source software for autonomous driving systems (Non-Patent Document 1).

In this example, five points p ₁ to p ₅ on the same layer are shown in a simplified manner, points p ₁ to p ₃ are located on object OBJ1, and points p ₄ and p ₅ are located on object OBJ2. are doing. Point p ₂ is included in the sphere with radius r centered at _point p ₁ , and point p ₃ is included in _{the sphere with radius r centered at point p 2 .} are determined to be the same cluster. Also, since the point p ₅ is included in the sphere with the radius r centered at the point p ₄ , the points p ₄ and p ₅ are determined as being in the same cluster. Since the distance between points p ₃ and p ₄ is greater than r, points p ₁ to p ₃ and points p ₄ and p ₅ are treated as different objects.

JP 2017-56935 A JP-A-2009-98023

Autonomous driving software: Autoware, [online], Nagoya University, Internet <URL: https://www.pdsl.jp/fot/autoware/>

As a result of examining conventional clustering algorithms, the inventors have come to recognize the following problems.

<First issue>
What is obtained by the LiDAR is the distance r _i between the LiDAR and each point p _i , while this algorithm needs to calculate the distance between the points p _i and p _j , which is computationally expensive.

In addition, although points on the same layer are compared in FIG. 2, it is also necessary to compare points on different heights. When the point cloud data is given by M×N matrices, the combination of two points is _M×N C ₂ =(M×N)(M×N−1)/2. increases, the computational cost increases exponentially. Even if a search algorithm such as kd-tree is used, a high-speed processor is required for processing.

<Second issue>
FIG. 3A is a side view of the ranging system 1. FIG. When implementing an identification system in a vehicle or in traffic infrastructure, the distance from the LiDAR to the object varies from several meters to hundreds of meters. If the object OBJ' is far away, most of the lower layers will be illuminated on the ground instead of the object OBJ'. Therefore, it may be useless to search for all distance measuring points.

<Third issue>
FIG. 3(b) is a top view of the distance measuring system 1. FIG. In the case shown in FIG. 3B, the distance between the points _p4 and _p5 is greater than the radius r, even though they are on the same object OBJ2. Therefore, points _p4 and _p5 are erroneously determined to be separate clusters 6_2 and 6_3.

The present invention has been made in such a situation, and one exemplary object of certain aspects thereof is to provide a system, apparatus, and method capable of clustering with a small amount of computation.

An aspect of the present invention includes a three-dimensional sensor that generates point cloud data indicating distances to each of a plurality of points, and an arithmetic processing unit that clusters objects based on the point cloud that forms one layer of the point cloud data. And prepare.

According to the present invention, clustering can be performed with a small amount of calculation.

1 is a diagram showing a ranging system using LiDAR; FIG. It is a figure explaining the algorithm of the conventional clustering. 3(a) is a side view of the ranging system, and FIG. 3(b) is a top view of the ranging system. 1 is a block diagram of an object identification system according to an embodiment; FIG. It is a figure explaining clusterization. FIG. 4 is a diagram showing a specific example of clustering based on horizontal layers; FIG. 4 is a diagram showing an example of the relationship between threshold th and distance r; 9 is a flowchart showing an example of processing of a clusterizing unit; FIGS. 9A and 9B are diagrams for explaining clustering based on the flowchart of FIG. 9 is a flow chart showing another example of processing of the clusterizing unit; 1 is a block diagram of an automobile with an object identification system; FIG. 1 is a block diagram showing a vehicle lamp with an object identification system; FIG.

(Overview of Embodiment)
One embodiment disclosed herein relates to an object identification system. The object identification system includes a three-dimensional sensor that generates point cloud data indicating distances to each of a plurality of points, an arithmetic processing unit that clusters objects based on the point cloud that forms one layer of the point cloud data, Prepare.

Many of the objects to be identified in relation to traffic are located above and above the ground. In other words, it basically does not occur that two different objects exist above and below the same position. Therefore, the amount of calculation can be greatly reduced by performing clustering focusing only on layers with appropriate heights.

One layer may lie substantially horizontally from the three-dimensional sensor. In other words, one layer is the layer for which the elevation/depression angle from the three-dimensional sensor is substantially zero. This allows the height of the layer from the ground to be substantially constant regardless of the distance to the object. Also, if a 3D sensor is installed at the height of a headlight or bumper in an automotive application, one layer will cross the human leg (or waist) and the height of the vehicle headlight or bumper. Objects to be identified can be captured reliably.

Focusing on two adjacent points in a group of points forming one layer, the arithmetic processing unit determines that the two points are in the same cluster when the difference in distance to each point is smaller than a threshold value. good too. Since this eliminates the need to calculate the distance between two points, the amount of calculation can be further reduced.

Three-dimensional sensors such as LiDAR emit laser beams radially in the horizontal direction. When an object is close to the LiDAR, two adjacent laser beams are projected onto two relatively close points on the object, and when the object moves away from the LiDAR, the same two adjacent laser beams are projected onto the object. The distance between the two irradiated points increases. Therefore, the threshold value may be variable according to the distance, and the threshold value may be increased as the distance increases. The threshold value may correspond to the average distance to points included in the same cluster.

When focusing on the first point and the second point that are in a non-adjacent relationship with an interval equal to or less than a predetermined value, the arithmetic processing unit has a difference between the distance to the first point and the distance to the second point that is smaller than a threshold. Sometimes, the first point and the second point may be determined to be the same cluster. By comparing two non-adjacent points as well, the influence of noise can be suppressed. Also, by limiting the search range to a predetermined value as a parameter, an increase in the amount of calculation can be suppressed.

(Embodiment)
BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described below based on preferred embodiments with reference to the drawings. The same or equivalent constituent elements, members, and processes shown in each drawing are denoted by the same reference numerals, and duplication of description will be omitted as appropriate. Moreover, the embodiments are illustrative rather than limiting the invention, and not all features and combinations thereof described in the embodiments are necessarily essential to the invention.

FIG. 4 is a block diagram of object identification system 10 according to an embodiment. This object identification system 10 is mounted on a vehicle such as an automobile or a motorcycle, and determines the types (also called categories) of objects OBJ existing around the vehicle.

The object identification system 10 mainly comprises a 3D sensor 20 and a processor 40 . The three-dimensional sensor 20 generates point cloud data S1 indicating distances to a plurality of points on the surface of the object OBJ in front. Points where the reflection from object OBJ cannot be measured are measured as a predetermined value (negative, zero or infinity).

Based on the point cloud data S1, the arithmetic processing unit 40 determines (categorizes) the types of the objects OBJ1, OBJ2, . As a pre-process for categorization, the point cloud data S1 is divided into sub-data S2_1, S2_2, . . . for each of the objects OBJ1, OBJ2, .

Although the three-dimensional sensor 20 is not particularly limited, it is preferable to use LiDAR when it is desired to accurately identify an object with small unevenness, such as a pedestrian. The configuration of the LiDAR is not particularly limited, and it does not matter whether it is a scanning type or a non-scanning type.

The arithmetic processing unit 40 can be implemented by a combination of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a processor (hardware) such as a microcomputer, and a software program executed by the processor (hardware). The arithmetic processing unit 40 may be a combination of multiple processors.

The arithmetic processing unit 40 includes a memory 42 , a clustering section 44 , a clipping section 46 and a categorizing section 48 . The point cloud data S1 are stored in the memory 42 . The clusterizing unit 44 determines the horizontal position (range) where each object OBJ exists.

The clipping unit 46 splits the point cloud data S1 into sub data S2 for each object based on the clustering result of the clustering unit 44 .

The categorizing unit 48 determines the types of objects included in each of the sub-data S2_1, S2_2, .

FIG. 5 is a diagram for explaining clusterization. A rectangle S1 indicates point cloud data generated by the three-dimensional sensor 20 . Rectangles S2_1 and S2_2 indicate sub data.

Next, clustering will be explained.
The point cloud data S1 acquired by the three-dimensional sensor 20 can be regarded as a set of point clouds with different heights from the ground, in other words, a set of point clouds with different elevation/depression angles viewed from the three-dimensional sensor 20. A set of points of is referred to as a "layer". The arithmetic processing unit 40 (clustering unit 44) clusters the objects based on the point cloud that forms one notable layer Lx of the point cloud data S1 rather than the entire point cloud data S1.

One layer Lx of interest for clustering is preferably positioned substantially horizontally from the three-dimensional sensor 20 . Hereinafter, the layer Lx of interest is also referred to as a horizontal layer. In other words, the horizontal layer can be grasped as a layer whose elevation/depression angle is close to zero. As a result, the height of the measurement point can be made constant without depending on the distance to the object OBJ.

Many of the objects OBJ (pedestrians and automobiles) to be identified in relation to traffic are located above and in contact with the ground. In other words, it basically does not occur that two different objects exist above and below the same position. Therefore, the amount of calculation can be greatly reduced by performing clustering focusing only on layers with appropriate heights.

FIG. 6 is a diagram showing a specific example of clustering based on horizontal layers. FIG. 6 shows only the point cloud p ₁ , p ₂ , p ₃ , p ₄ forming the layer Lx. Objects OBJ1 and OBJ2 have the same shape as objects OBJ1 and OBJ2 in FIG. 2(b).

When focusing on adjacent two points p _i and p _i+1 among points on one layer Lx, the arithmetic processing unit 40 (clustering unit 44) calculates the distances r _{i and} r to each of the points p _i and p _i+1 When the difference d _{i,i+1 =} |r _i −r _i+1 | of i+1 is smaller than the threshold value th, the two points p _i and p _i+1 are clustered into the same cluster. In the example of FIG. 6, points p ₁ and p ₂ are determined to be the same cluster 6_1, and points p ₃ and p ₄ are determined to be the same cluster 6_2.

Regarding the third problem, one of the advantages of the clustering according to the embodiment becomes clear from a comparison between FIG. 6 and FIG. 3(b). As shown in FIG. 3(b), the distance between points _p3 and _p4 is greater than the predetermined radius r, and the conventional method cannot determine that they are the same cluster. On the other hand, in FIG. 6, in order to focus on the distance difference d 3, ₄ between the points p ₃ , p ₄ and the three-dimensional sensor 20, the threshold value th equal to the radius r in FIG. 3 is applied. , the two points p ₃ and p ₄ can be determined to be in the same cluster. Although the same result can be obtained in the conventional method by increasing the radius r, a large radius r causes erroneous determination of points on different objects as being in the same cluster. On the other hand, in the present embodiment, since it is not necessary to increase the threshold value th, it can be understood that erroneous determination can be suppressed.

The threshold th may be varied (scaled) according to the distance r. FIG. 7A is a diagram for explaining the difference _d1,2 between the distance to an object and the distances between two points _p1 and _p2 on the object. When an object OBJ _is located near LIDAR2 (on the left side in the figure), the distances to two points _{p1 and p2 on} the object OBJ irradiated by adjacent laser beams LB1 and _LB2 are _r1 and r ₂ and their difference d _1,2 is r ₂ −r ₁ .

Consider the case where the same object OBJ is positioned relatively far from LIDAR2 (on the right side in the figure). At this time, the irradiation position of the laser beam LB ₁ on the object OBJ is the same (p ₁ ′=p ₁ ), but the irradiation position p ₂ ′ of the laser beam LB ₂ is shifted from the original position p ₂ (p ₂ ′ ≠ p ₂ ).

That is, when the object OBJ is close to the LiDAR, two adjacent laser beams are applied to two points _p1 and _p2 on the object that are relatively close to each other, and when the object moves away from the LiDAR, the same adjacent laser beams are applied. The distance between the two points p ₁ and p ₂ irradiated with the two laser beams increases. FIG. 7B is a diagram showing the relationship between the distance to an object and the difference in distance to two points. The distance difference d _1,2 ′ between the two points p ₁ ′ and p ₂ ′ when the object OBJ is far is greater than the distance difference d _1,2 between the two points p ₁ and p ₂ when the object OBJ is near. growing.

In other words, if the threshold value th determined with reference to when the object OBJ is close is applied to a distant object, there is a possibility that two points p ₁ ′ and p ₂ ′ on the same object are erroneously determined to be different objects. be. Therefore, as shown in FIG. 7C, the longer the distance r, the larger the threshold th. This allows accurate clustering independent of the distance to the object.

The threshold th may be set according to the average value of the distances to a plurality of points that have been determined to belong to the same cluster.

In another embodiment, the distances r _{i and r i+1} to the two points p i and p _i+1 to be compared are normalized by dividing by one distance r _i (or a value close to it), and the normalized distance The difference (1−r _i /r ₁₊₁ ) may be compared to a threshold. The threshold value in this case can be constant regardless of the distance to the object.

When focusing on a first point p _i and a second point p _j that are in a non-adjacent relationship with an interval equal to or less than a predetermined value K, the arithmetic processing unit 40 calculates the distance r _i to the first point p _i and the second point When the difference d _i,j of the distance r _j to p _j is smaller than the threshold value th, the first point p _i and the second point p _j are regarded as the same cluster. In other words, the second point _pj is allowed to be separated from the first point _pi by a maximum of K (K<<M). M is the horizontal resolution of the three-dimensional sensor 20; For example, K may be about 2-5.

FIG. 8 is a flow chart showing an example of the processing of the clusterizing section 44. As shown in FIG. Point groups forming the horizontal layer Lx are numbered in order from the end (S100). Points where reflection (distance) cannot be detected may not be numbered.

Subsequently, the variable i is initialized (S102), and the variable j is initialized (S104). The difference d _i , _j between the distances r _i , r _j between the two points p _{i , p j} is compared with the threshold value th (S106). When the difference d _i,j is smaller than the threshold value th, the points p _i , p _j are determined to be in the same cluster and stored in the cluster queue (S108). When the difference d _i,j is greater than the threshold value th, the points p _i , p _j are determined to be in the same cluster and stored in the cluster queue (S108). Note that if the two points p _i and p _j have already been stored in the cluster queue as a result of past determination, the comparison process can be omitted.

When j≤i+K (N in S110), j is incremented by 1, the second point is moved to the next point, and the process returns to step S106. Conversely, when j>i+K (Y in S110), i is incremented to shift the first point _pi , and the process returns to step S104.

FIGS. 9A and 9B are diagrams for explaining clustering based on the flowchart of FIG. FIG. 9A is a diagram showing horizontal layers to be clustered, where the horizontal axis represents numbers and the vertical axis represents distances. FIG. 9B is a diagram showing comparison results and cluster queues.

In this example, K=3. A broken line in FIG. 9B indicates that the comparison is omitted when the comparison target has already been clustered. In the case of FIG. 9A, points p ₁ to p ₆ are determined to be in the same cluster, and points p ₇ and p ₈ are determined to be in the same cluster.

In the processing of FIG. 8, if no numbers are assigned to points where reflection (distance) cannot be detected, a situation in which multiple objects (people or cars) are horizontally separated at substantially the same distance. below can incorrectly cluster them as the same object. This problem can be solved as follows.

FIG. 10 is a flow chart showing another example of the processing of the clusterizing unit. Two indices are assigned to a plurality of point groups (S101). More specifically, the points are numbered in order from the end including the point where no reflection is obtained, and the value is set as the first index (index1). Also, points where reflection (distance) cannot be detected are removed, numbered from the end, and the value is used as a second index (index2). The relationship between the two index1 and index2 is maintained.

The processes S102 to S112 are the same as in FIG. 8, and the values of the second index are used for i and j.

If j>i+K in process S110, index2 in the cluster queue is converted to index1 (S116). After the conversion, if the indexes in the cluster queue are continuous (Y of S118), the multiple indexes in the cluster queue are output as the same object (S120). When the indexes in the cluster queue are discontinuous after conversion (N of S118), the multiple indexes in the cluster queue are divided into multiple cluster queues and output (S122).

The processing of FIG. 10 enables correct clustering of multiple objects (persons and automobiles) as different objects in a situation in which a plurality of objects (people and automobiles) exist at substantially the same distance from each other in the horizontal direction.

FIG. 11 is a block diagram of a vehicle with object identification system 10 . Automobile 100 includes headlights 102L and 102R. At least the three-dimensional sensor 20 of the object identification system 10 is built into at least one of the headlights 102L and 102R. The headlamp 102 is positioned at the extreme tip of the vehicle body and is the most advantageous location for installing the three-dimensional sensor 20 in terms of detecting surrounding objects. The arithmetic processing unit 40 may be built in the headlamp 102 or may be provided on the vehicle side. For example, in the arithmetic processing unit 40, the intermediate data may be generated inside the headlamp 102, and the final data may be generated by the vehicle.

FIG. 12 is a block diagram showing a vehicle lamp 200 including the object identification system 10. As shown in FIG. A vehicle lamp 200 includes a light source 202 , a lighting circuit 204 and an optical system 206 . Furthermore, the vehicle lamp 200 is provided with a three-dimensional sensor 20 and an arithmetic processing unit 40 . Information about the object OBJ detected by the processing unit 40 is transmitted to the vehicle ECU 104 . The vehicle ECU may perform automatic driving based on this information.

Further, the information regarding the object OBJ detected by the processing unit 40 may be used for light distribution control of the vehicle lamp 200 . Specifically, the lamp ECU 208 generates an appropriate light distribution pattern based on the information about the type and position of the object OBJ generated by the processing unit 40 . The lighting circuit 204 and the optical system 206 operate so as to obtain the light distribution pattern generated by the lamp ECU 208 .

In the embodiment, the object identification system 10 for in-vehicle use has been described, but the application of the present invention is not limited to that. It is possible.

(Modification)
In the embodiment, clustering based on the horizontal layer Lx whose elevation/depression angle from the three-dimensional sensor 20 is zero has been described, but the one layer to be focused on is not limited to that, and is one layer above the horizontal layer, or one layer above the horizontal layer. Clustering may be performed based on the underlying layer.

Although the present invention has been described using specific terms based on the embodiment, the embodiment only shows one aspect of the principle and application of the present invention, and the embodiment does not include the claims. Many variations and rearrangements are permissible without departing from the spirit of the invention as defined in its scope.

The present invention relates to object identification systems.

REFERENCE SIGNS LIST 10 object identification system 20 three-dimensional sensor 40 arithmetic processing unit 42 memory 44 clusterizing unit 46 clipping unit 48 categorizing unit S1 point cloud data 100 automobile 102 headlight 104 vehicle ECU
200 vehicle lamp 202 light source 204 lighting circuit 206 optical system 208 lamp ECU

Claims (9)

a three-dimensional sensor that generates point cloud data indicating distances to each of a plurality of points;
an arithmetic processing unit for clustering objects based on the point cloud forming one layer of the point cloud data;
with
The arithmetic processing unit determines two points to be in the same cluster when a difference in distance to each point is smaller than a threshold when focusing on two adjacent points in the point group forming the one layer. An object identification system characterized by : When focusing on a first point and a second point that are in a non-adjacent relationship with an interval equal to or less than a predetermined value, the arithmetic processing unit determines that the difference between the distance to the first point and the distance to the second point is a threshold value. 2. The object identification system according to claim 1 , wherein said first point and said second point are determined to be in the same cluster when they are smaller than a value. a three-dimensional sensor that generates point cloud data indicating distances to each of a plurality of points;
an arithmetic processing unit for clustering objects based on the point cloud forming one layer of the point cloud data;
with
When focusing on a first point and a second point that are in a non-adjacent relationship with an interval equal to or less than a predetermined value, the arithmetic processing unit determines that the difference between the distance to the first point and the distance to the second point is a threshold value. An object identification system , wherein said first point and said second point are determined to be in the same cluster when they are smaller than a value . 4. An object identification system according to any preceding claim, wherein said one layer is positioned substantially horizontally from said three-dimensional sensor. 5. The object identification system according to claim 1, wherein said threshold is variable according to said distance. 6. The object identification system according to claim 5, wherein said threshold value corresponds to an average value of distances to points included in the same cluster. A motor vehicle comprising an object identification system according to any one of claims 1 to 6. 8. A vehicle according to claim 7, wherein said three-dimensional sensor is built in a headlight. A vehicle lamp comprising the object identification system according to any one of claims 1 to 6.

Images