JP5583523B2 - Object recognition device - Google Patents

Description

  The present invention relates to an object recognition apparatus that recognizes the type of a target object such as a vehicle from a reflection detection pattern of an in-vehicle radar.

  Conventionally, various preventive safety technologies have been researched and developed in the field of vehicles, and various preventive safety systems such as a front collision damage reduction system and a rear side warning system have been put into practical use. However, all of these conventional systems are intended for detection in only one direction of a vehicle (automobile), have a narrow application range, and are in a complicated traffic environment where many obstacles other than vehicles (bicycles, pedestrians, etc.) are mixed. It is difficult to apply.

  In order to deal with collisions from various directions under various traffic environments, it is considered necessary to recognize not only the rear shape of the target vehicle but also the shapes from all directions. In addition, recognizing the attributes (types) of various target objects (obstacles) including vehicles is useful in complex traffic environments, and can greatly enhance the application range of preventive safety systems and improve the effectiveness. It may be helpful but not easy.

  As a method of recognizing a target object from the reflection pattern of the vehicle-mounted radar, the detection pattern of the reflection point group (cluster) detected at a certain point of time by the laser radar as the vehicle-mounted radar has three types of set L, I, and O. It has been proposed to identify and recognize a target object to be recognized depending on which pattern it belongs to (for example, see Non-Patent Document 1).

  However, in the recognition described in Non-Patent Document 1, the target object is estimated and recognized from the detection pattern of the cluster of one frame detected at a certain time point by the laser radar. Moreover, the number of object patterns is small (specifically, one pattern of L, I, and O for each of three objects of rectangular shape, fence, wall, and other shapes).

  On the other hand, the target object moves in various directions (front-rear direction, lateral direction, etc.) with respect to the vehicle-mounted radar, and the direction, orientation, etc. with respect to the vehicle-mounted radar change over time. The detection pattern of the radar reflection cluster changes from time to time, and in the recognition described in Non-Patent Document 1, it is possible to accurately identify and apply the detection patterns of the clusters of various types of target objects to the corresponding object patterns. The recognition rate (correct answer rate) of the target object type is extremely low. If the type of the target object can be accurately identified, optimal driving support such as changing the alarm timing according to the recognized target object is realized.

  Therefore, the present applicant has already invented an object recognition apparatus that recognizes a target object with a high recognition rate by estimating the type (attribute) of the target object from a reflection detection pattern of an on-vehicle radar typified by a laser radar. (Japanese Patent Application No. 2009-215241). In this existing application, since the reflection of the laser radar can be obtained even by a bicycle or a pedestrian, for example, when the type of the target object is estimated (attribute estimation) by the laser radar, the position of the target object is also the same object. Paying attention to the difference in direction and orientation, prepare multiple patterns with different shapes as reflection point cloud patterns (clusters) for target vehicles such as four-wheeled vehicles, two-wheeled vehicles, and pedestrians, and extract their features by pattern matching Then, the detection pattern of each frame of the laser radar is identified, and the type of the target object is estimated by accumulating the recognition results (scores) of each frame in time series for each pattern by a known SVM discriminator used for pattern recognition. And recognize.

  In this case, by accumulating the detection patterns of multiple time-series frames, it absorbs changes in the time and direction of the target object, eliminates those effects, and accurately estimates the type of the target object. The target object can be recognized.

Stephan Wender, 3 others, “Classification of Laserscanner Measurement at Interstitial Measurement at Interstitial Measurement in Interstitial Stimulation Measurement.” 2005. Proceedings. IEEE, IEEE, June 2005, p. 94-99

  In the case of the object recognition device described in the above-mentioned application, the recognition accuracy of the target object is improved, but in order to estimate the type, it is necessary to classify and accumulate the detection patterns of a plurality of time series frames. Therefore, it is time consuming and it is desired to estimate the type of the target object more quickly. Also, it is difficult to know the overall size of the target object and the direction of movement.

  It is an object of the present invention to quickly estimate the type of a target object from the reflection detection pattern of an on-vehicle radar and to grasp the size of an obstacle and the direction of movement.

In order to achieve the above object, the object recognition apparatus of the present invention forms an image by plotting distance information of a plurality of reflection points of a target object obtained from an on-vehicle radar on a horizontal plane, and forming a frame of the formed image. An object recognition device for identifying a type of the target object based on a detection pattern of a reflection point group obtained by connecting the reflection points for each of the detection points, wherein the detection changes as the target object moves with respect to the in-vehicle radar An estimation means for estimating whether the pattern is a horizontal length or a vertical length of the target object, and at least one of the estimated horizontal length and vertical length An identification means for identifying the type of the target object is provided (claim 1).

  In the case of the invention of claim 1, most objects present in the traffic environment are rectangular, and the length in the horizontal direction and the length in the vertical direction differ depending on the type based on the provisions of the Road Transport Vehicle Law. The estimation means estimates the horizontal length and the vertical length of the target object from the reflection detection pattern of the on-vehicle radar, and at least one of the estimated horizontal length and vertical length is estimated. On the other hand, the type can be recognized by the identification means.

  In this case, there is no need to accumulate the detection patterns obtained by extracting the features of each time-series frame as in the above-mentioned application and perform many pattern matching of the accumulated results, and the type of the target object is estimated from the detection pattern of one frame. The type of the target object can be estimated quickly. In addition, since the length in the horizontal direction and the length in the vertical direction of the target object can be estimated, the overall size and the direction of movement of the target object can also be known.

It is a block diagram of one embodiment of the present invention. It is explanatory drawing of the class of the vehicle of FIG. (A), (b) is explanatory drawing of the example of a detection pattern of FIG. (A), (b) is explanatory drawing of the axis | shaft of the detection pattern of Fig.3 (a), (b). (A), (b) is explanatory drawing of the end point of the detection pattern of Fig.3 (a), (b). It is explanatory drawing of a rectangular shape. It is explanatory drawing of the motion vector of the rectangular shape of FIG. (A), (b) is explanatory drawing of the classification | category of a rectangular shape. (A)-(d) is explanatory drawing of the process which forms a rectangle from the detection pattern of each one frame of a time series. It is explanatory drawing of the vote result of the rectangle of several frames. (A)-(c) is explanatory drawing of the experimental example of a vehicle. (A)-(c) is explanatory drawing of the experimental example of a bicycle. (A), (b) is explanatory drawing of the experiment example in an intersection. (A), (b) is explanatory drawing of the recognition result of each 1 frame of the experimental example of a vehicle, and the voting result of several frames. It is explanatory drawing of the vote result of the experimental example of a bicycle. It is explanatory drawing of the vote result of the experiment example in an intersection.

  Next, in order to describe the present invention in more detail, an embodiment will be described in detail with reference to FIGS.

  FIG. 1 shows a configuration of an object recognition device of this embodiment provided in a vehicle (own vehicle) 1. A laser radar 2 as an in-vehicle radar can detect a wide range of objects and can continuously estimate the shape of an object. Necessary, for example, the specifications shown in Table 1 below are attached to the front vehicle bumper position, and the distance data of the moving object is acquired in a horizontal section according to the laser projection height. In other words, the laser radar 2 is a wide-angle ranging radar that searches a range of approximately 110 ° in front of the vehicle 1 and can also be reflected by a bicycle or a pedestrian by using near infrared rays. The front of the vehicle 1 attached to the bumper position is searched for every frame, for example, at intervals of 80 mS seconds, and reflected waves from other vehicles (automobiles), two-wheeled vehicles, pedestrians (people), etc. are received. The distance data of these target objects moving forward is acquired, and this distance data is output to the reception processing unit 3 as a reception output of the reflected wave. In the present embodiment, the target object is a two-wheeled vehicle, a light vehicle, a small vehicle, or a normal vehicle.

  The reception processing unit 3 receives the distance data and sends it to the clustering processing unit 4.

The clustering processing unit 4 plots the distance information of each reflection point obtained from the laser radar 2 on the horizontal plane based on the distance data of each reflection point of the reception output and forms an image of the reflection point. That is, since the reflection points of the target object are discrete, the clustering processing unit 4 connects the reflection points in the vicinity using, for example, a well-known morphological operation, and groups the reflection points in the vicinity (reflection point group). And a detection pattern (cluster) of each reflection point group for each frame of the target object is output to the tracking processing unit 5.

  The tracking processing unit 5 temporarily stores a detection pattern for each frame. For example, each time a reflection of a new frame is obtained, the detection pattern of the reflection is compared with the detection pattern of the immediately preceding frame, and the pattern of the same cluster. For example, a label for pattern identification is attached to the detected pattern.

  Information of each detection pattern of each frame, such as a label attached by the tracking processing unit 5, is sent to the rectangle estimation unit 6. The rectangular estimation unit 6 includes the estimation means of the present invention, and for each detection pattern, the detection pattern that changes due to the movement of the target object with respect to the laser radar 2 is either the horizontal length or the vertical length of the target object. And the rectangular shape of the target object is estimated.

That is, most objects (cars, bicycles, motorcycles, trucks, etc.) present in the traffic environment have a shape close to a rectangle. Therefore, the rectangle estimation unit 6 estimates the rectangle from the reflection point information (information of each detection pattern of the reflection point) on the assumption that all the target objects are rectangles. In the estimation of the rectangle, the direction of the object is also estimated.

  The estimation result of the rectangle estimation unit 6 is sent to the object attribute estimation unit 7 forming the identification means of the present invention, and the object attribute estimation unit 7 is based on the rectangle estimation result of each frame of the rectangle estimation unit 6 for each frame. Then, the type of the target object is estimated, and the type (attribute) is determined by the majority process by voting according to the types of motorcycles, light cars, small cars, and ordinary cars. The type estimated by the object attribute estimation unit 7 is sent to the driving support unit 8, and the driving support unit 8 performs driving support according to the estimated type.

FIG. 2 shows an outline processing procedure of the rectangle estimation. First, when one frame of reflection point information (laser radar) is obtained in the reception processing unit 3 (step S1), clustering is performed for each frame (every time). The processing unit 4 performs a clustering process on the reflection point information to obtain a detection pattern (step S2). The tracking processing unit 5 tracks the detection pattern (step S3), and the rectangle estimation unit 6 is based on the tracking result of the tracking processing unit 5 and the width (horizontal) of the reflection point information detection pattern orthogonal to the rectangle of the target object. Direction length) or depth (vertical length (vehicle length)) which corresponds to the pattern of the axis, and detect the position of the end point of at least one of the axes of the rectangle at that time (Step S4). Next, from the motion vector of the end point detected in one frame, the direction of the target object is estimated, the width of the rectangle and the length of the depth are estimated, and the vehicle size of the rectangle at that time is estimated, Furthermore, the estimated vehicle size is corrected based on the vehicle size of each vehicle defined by the road transport vehicle law stored in advance to update the rectangular size (step S5). Further, based on the estimated rectangular size, class (type) voting is performed in time series (step S6), and the class is determined by the vote value to identify the type of the target object (step S7). In the case of this embodiment, there are four types of classes for voting: motorcycles, mini cars, small cars, and ordinary cars.

  The above processing procedure will be described in more detail.

<Rectangle estimation>
"End point detection"
In the detection of the end points, first, the axis that becomes the width or the depth is detected. This axis is detected by using all the reflection points of the detection pattern obtained by clustering (FIGS. 3A and 3B show an example Dpa of the detection pattern in the case of a vehicle and another example Dpb) as the cluster center. Rotate by 1 degree with the origin as the origin, and in each case, perform density projection in the horizontal direction, find the axis that is at least one of the two sides of the rectangular L-shape from the rotation state where the density value peaks, and find only one axis If not, the other axis which is the remaining one of the two sides of the L-shape is set in a direction perpendicular to the angle formed by the axis (FIGS. 4A and 4B show detection patterns Dpa and Dpb). An example of detection of each axis is shown, (a) is a detection example of two horizontal and vertical axes, and (b) is a detection example of one vertical axis). When the density value is less than or equal to the threshold value, there are no axes, and the number of axes is 0, 1 or 2.

  In the case where two axes are found, the end points are three points, that is, a point where the axes intersect each other and a point at the end of the axis (the circles in FIG. 5A are the end points of the detection pattern Dpa). When only one axis is found, the end points are two reflection points existing at the end of the axis, but a rectangle cannot be generated with only two end points, so the third end point is temporarily generated. The third end point is a reflection point on the opposite side from the laser radar 1 on the other axis obtained by rotating one axis by 90 degrees (each circle mark in FIG. 5B indicates the detection pattern Dpb). An end point of which a circle away from the detection pattern Dpb on the broken line is the third end point).

"Determination of width and depth"
When estimating a rectangle, information on the width and depth of the target object is required, but it is not clear which is the width or depth only by the end points. Therefore, the width and depth are determined based on, for example, two frames of motion vectors in time series of the target object.

  For example, as shown in FIG. 6, when detection patterns Dpc and Dpd of a cluster of a plurality of reflection points indicated by circles are detected, the rectangular shapes of the detection patterns Dpc and Dpd are numbers with circles in the figure. 1 and 2 can be estimated and cannot be specified, but if the target object moves in the direction of the solid arrow from the motion vector as shown in FIG. 7, the direction of the rectangle is specified. The shape can be estimated.

  When the orientation of the rectangle is specified, the width and depth are considered from the direction of the wheel mounted on the vehicle (two-wheeled vehicle, light vehicle, small vehicle, ordinary vehicle), the axis perpendicular to the movement direction is the width, the axis of the movement direction Can be estimated as a length.

  By the way, the movement direction of the target object may be estimated from the movement of the center or center of gravity of the cluster. In this case, however, the stable movement direction is due to the influence of noise and the size of the cluster changing with time. Cannot be observed, and detection becomes unstable.

  Therefore, in the present embodiment, as information that can be stably detected, when a light vehicle, a small vehicle, a normal vehicle (four-wheeled vehicle), or a two-wheeled vehicle is observed with the laser radar 2, one of the corners of the rectangle is necessarily observed. The amount of movement of the corner of the target object is used as a motion vector. In the case of two axes, the corner of the object is the intersection of the axes, and in the case of one axis, it is a point close to the own vehicle 1 on the axis.

  When the motion vector is obtained in this way, a rectangular pattern of the target object is classified by looking at whether the motion vector and the axis of the rectangle are close to vertical or parallel. Compare the angular motion vector and the biaxial angle for the rectangle.

FIGS. 8 (a) and 8 (b) show the movement of the rectangle and the motion vector of the solid arrow in the two time-series detection patterns (patterns Dpe, Dpf and Dpg, Dph), respectively, and the detection pattern Dpe at time Δt, If the rectangular corner of Dpg is point P, the end points on the axis are points A and B, and the detection pattern Dpf of (Δt + N) time, and the corner of Dph is P ′, the relationship between the width and depth angles is 1 can be roughly classified into two patterns A and B shown in FIGS. 8A and 8B.

  The pattern A is a case where the front corner of the target object is visible, and the pattern B is a case where the rear corner of the target object is visible. In each case, ∠APP ′ and ∠BPP ′ are compared. In the case of the pattern A, if ∠APP ′ <∠BPP ′, AP is the depth and BP is the width. In the case of the pattern B, if | 180−∠APP ′ | <∠BPA, the axis AP is the depth and the axis BP is the width. Whether pattern A or pattern B is determined is pattern A when ∠APB + ∠BPA is 90 ° or less, and pattern B when ∠APB + ∠BPA is 90 ° or more. In this manner, the width of the rectangle of the detection pattern and the length of the depth are detected by determining the moving direction.

"Size estimation by vehicle structure pattern"
Next, the rectangle width and depth length of each frame has occlusion problems and does not necessarily match the original width and depth length of the target object. The size of the target object is estimated.

  As shown in Table 2 below, vehicle sizes of target objects such as motorcycles, light cars, small cars, and ordinary cars are determined by the Road Transport Vehicle Law.

  For this reason, the width and depth length (size) of the rectangle are updated depending on whether the rectangle corresponds to a specified size of a motorcycle, a light vehicle, a small vehicle, or a normal vehicle. That is, if at least one of the width and depth exceeds the condition of each vehicle in Table 2, the width and depth are updated to the size of the upper vehicle (classification) in the table. This update is performed by changing the position of the end point according to the estimated vehicle size.

  The estimation result of the target vehicle thus estimated is sent to the object attribute estimation unit 7, and the target object is classified into one of the attributes of a two-wheeled vehicle, a light vehicle, a small vehicle, and a normal vehicle.

  FIG. 9 shows the result of estimating the vehicle size from the observed width and depth of the rectangle. (A) in the figure shows a detection pattern of reflection points at the time of each frame when another vehicle to be detected passes from right to left in front of the own vehicle 1, and (b) in FIG. (C) shows a rectangle formed by connecting the end points of each frame observed without considering occlusion, etc., and (d) shows a vehicle defined by the Road Transport Vehicle Law. The size is updated in consideration of the size, and the result of class classification of the attribute of the target object from the rectangle of each updated frame is shown. Numbers 1 to 8 in FIGS. 9A to 9D indicate time-series frame numbers. In FIG. 9D, T indicates a motorcycle, L indicates a light vehicle, and C indicates a small vehicle classification result. Although not shown in FIG. 9 (d), the classification result for ordinary vehicles is O. Further, m in FIGS. 9A to 9D indicates the position of the host vehicle 1.

  Although the attribute of the target object can be recognized from the detection pattern of each frame of the rectangle estimation unit 6, in order to increase the recognition accuracy, the detection pattern changes according to the movement of the target object. Since the size of the depth changes, the estimation result of the rectangle estimation unit 6 is sent to the object attribute estimation unit 7 to vote for the types of motorcycles, light vehicles, small vehicles, and ordinary vehicles, and the majority processing of the time series of multiple frames To determine the type of the target object.

"Vehicle type determination by voting"
As described above, since the size of the rectangle changes from time to time as the object moves, the vehicle size of the object estimated from the estimated rectangle of each frame also changes from time to time. Therefore, the object attribute estimation unit 7 determines the estimation result of the rectangle estimation unit 6 based on the vehicle classification result (type) estimated from each rectangle of each frame for a certain period of time for a motorcycle, a light vehicle, a small vehicle, and a normal vehicle. Vote by category, and determine the type of the largest number of vehicles in the voting results.

  FIG. 10 shows the result of voting for each frame type for a plurality of frames as shown in FIG. 9 (d). In this way, the first frame is classified as a two-wheeled vehicle. It can be seen that in the seventh and eighth frames, the small car classification is maintained. That is, voting reduces the misrecognition of motorcycles, increases the recognition of mini cars or small cars, and continues to be recognized by small cars from the second half, improving the recognition accuracy.

  Then, the vehicle type (attribute) estimated by the object attribute estimation unit 7 based on the voting result at each time point is sent to the driving support unit 8, and the driving support unit 8 shifts the alarm timing according to the estimated vehicle type to optimize the vehicle type. Provide driving assistance. In this case, since the estimation accuracy of the vehicle type of the target object is improved, the driving support accuracy is improved.

<Experimental example>
In the experiment, in order to confirm that (1) the size of the rectangle was correctly estimated and (2) the direction of the object was correctly estimated, the target object was a four-wheeled vehicle (small car) or two-wheeled vehicle (bicycle). The vehicle type was estimated when traveling in front of the host vehicle 1 in a figure 8 and when various vehicles traveled in front of the host vehicle 1 stopped at the intersection.

  FIGS. 11A and 11B are a front view and a side view of a four-wheeled vehicle (small vehicle) of the target object TG, and FIG. 11C shows a four-wheeled vehicle (small vehicle) of the target object TG of the subject vehicle 1. An example of observation of 10 frames of a rectangular shape captured by the reflection point when traveling in front of the figure 8 is shown. 12A and 12B are a front view and a side view of the two-wheeled vehicle (bicycle) of the target object TG, and FIG. 12C is a front view of the two-wheeled vehicle (bicycle) of the target object TG 8 in front of the own vehicle 1. An example of observation of 10 frames of a rectangular shape captured by a reflection point when traveling in a letter is shown. FIG. 13 (a) shows a scene in which the vehicles of the target objects TG with the numbers 1 to 3 marked with a circle run in front of the own vehicle 1 that stops at the intersection to make a right turn. The traveling direction of the vehicle is indicated, and the broken arrow line indicates the right turn direction of the own vehicle m that has stopped. FIG. 5B shows an observation example of each vehicle in that case. For pedestrians, since no axis was detected, objects other than four-wheeled vehicles and two-wheeled vehicles were recognized as pedestrians.

(Experimental result)
11 and 12, assuming that the number of time-series frames is 420 frames, the type is estimated from the rectangular size for each frame, the type is estimated from the rectangular size of the voting results of several frames, and further, When the correct answer rate and the correct answer rate of the direction were confirmed, the following results were obtained.

  FIG. 14A shows the estimation result of the type from the rectangular size for each frame based on the observation of FIG. 11 of the four-wheeled vehicle (small car) that is the target object TG, and FIG. 14B is the target object TG. The estimation result of the classification from the voting result based on the observation of FIG. 12 of the four-wheeled vehicle (small vehicle) is shown. Moreover, (a) and (b) of the following Table 3 show the correct answer rate of those types and the correct answer rate of direction. Note that the arrow lines in FIGS. 14A and 14B indicate the traveling direction of the target object TG.

The target object TG in FIG. 13 is a small car (C), whereas in FIG. 14 (a), there is a frame that is a light car (L), and the class accuracy rate was 76%. In FIG. 14B, all the frames were correctly identified as the small car (C). In addition, the direction was erroneously estimated in two frames.

  FIG. 15 shows the type estimation result from the voting result based on the observation of the motorcycle (bicycle (T)) of the target object TG in FIG. Further, (a) and (b) of Table 4 below show the estimation result of one frame, the estimation result from the voting result, the correct answer rate of the type, and the correct answer rate of the direction. In addition, the arrow line of FIG. 15 shows the traveling direction of the target object TG.

  For the bicycle (T) in FIG. 15, the recognition rate by voting was 98%, and the wrong scene was only 4 frames until the voting value was obtained.

  FIG. 16 shows 170-frame voting results in a scene in which various vehicles travel in front of the host vehicle 1 that is stopped to make a right turn at the intersection of FIG. 13 (a). In FIG. The vertical axis indicates the classification type (type class). Numbers 1 to 3 with ○ indicate the small car with the number 1 in FIG. 13B, the ordinary car with the number 2, and the ordinary car with the number 3, and the vote value gradually becomes correct among them. Turned out to be close. In addition, all vehicle orientations were detected correctly. The correct answer rate was 83%, and the average frame time to reach the correct answer was 16.4 frames as shown in Table 5 below.

  As described above, in the case of the present embodiment, since the rectangular estimation unit 6 and the object attribute estimation unit 7 are provided, the type (attribute) of the target object can be estimated using only the laser radar 2. Specifically, based on the assumption that almost all objects present in the traffic environment are rectangular, the size of the rectangle of the detection pattern is referred to by referring to the pre-stored vehicle size specified in the Road Transport Vehicle Law. The type of two-wheeled vehicle, light vehicle, small vehicle, or ordinary vehicle can be accurately estimated by voting the type (class) in time series, and the direction of the target object can be estimated almost correctly.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. It is not limited, and may be a millimeter wave radar, an ultrasonic radar, or the like. Further, the search range of the on-vehicle radar may be any, and the on-vehicle radar may search for the rear and left and right sides of the vehicle 1 or may search the entire circumference of the vehicle 1. Good.

  Next, it goes without saying that the configurations and processing procedures of the rectangular estimation unit 6, the object attribute estimation unit 7, and the driving support unit 8 may be different from those in the above embodiment. The number or the like may be set to an appropriate number based on experiments or the like. In addition, pedestrians may be included in the target object, and the type and number of types are not limited to those in the above embodiment. Furthermore, simply, for each detection pattern of one frame, it is estimated whether the length in the horizontal direction or the vertical length of the target object, and the road corresponding to the length is calculated from the length. The classification may be performed by determining the horizontal length (width) and vertical length (depth) of the corresponding type from the length of Table 1 of the Transport Vehicle Law.

  The present invention can be applied to various vehicle object recognition apparatuses using on-vehicle radars.

1 Vehicle (own vehicle)
2 Laser radar 6 Rectangle estimation unit 7 Object attribute estimation unit

Claims (1)

The distance information of the plurality of reflection points of the target object obtained from the on- vehicle radar is plotted on a horizontal plane to form an image, and the reflection point group obtained by connecting the reflection points for each frame of the formed image is detected. An object recognition device for identifying a type of the target object based on a pattern,
Estimating means for estimating whether the detection pattern that changes due to the movement of the target object with respect to the on-vehicle radar is a horizontal length or a vertical length of the target object;
An object recognition apparatus comprising: an identification unit that identifies a type of the target object from at least one of the estimated length in the horizontal direction and the length in the vertical direction.

Images