JP5806647B2 - Object recognition device - Google Patents

Description

  The present invention relates to an object recognition device for recognizing an object existing around a host vehicle.

  As an object recognition device that recognizes objects such as pedestrians and vehicles that exist in the vicinity of the host vehicle, for example, in the forward direction of the host vehicle, the presence of the object is detected using both the radar device and the camera, and the detection results of both are used. There is also a type that determines the type of object.

  In Patent Document 1, the transmission output of the radar device is switched between large and small, and the detection result based on the reflected wave received at the small transmission output is removed from the detection result based on the reflected wave received at the large transmission output. Thus, a technique is described in which an object other than a vehicle is extracted, a pattern matching process is performed on the extracted object based on an image captured by a camera, and a determination is made as to whether or not the object is a pedestrian.

JP 2005-157765 A

  However, in the technique disclosed in Patent Document 1, since it is impossible to determine whether an object is a vehicle or a pedestrian with a radar device alone, from a detection result based on a reflected wave received at a large transmission output, When the detection result based on the reflected wave received at the small transmission output is removed, the pedestrian may be removed as noise.

  Further, as a technique different from the technique disclosed in Patent Document 1, pattern matching processing is performed on an object detected by a radar device based on an image captured by a camera, and the object is a vehicle or a pedestrian. Although it is possible to determine whether or not there is an object, generally in this case, if the object is not recognized by the camera multiple times, it is not determined that the object is a vehicle or a pedestrian. It will take. As a result, various control operation timings performed based on the determination result may be delayed.

  Therefore, the present invention provides an object recognition apparatus that can determine at an early stage whether or not a detected object is a specific object.

The object recognition apparatus according to the present invention employs the following means in order to solve the above problems.
The invention according to claim 1 is a transmitting / receiving unit (for example, described later) that transmits an electromagnetic wave toward a predetermined range around the host vehicle and receives a reflected wave that is reflected by an object existing around the host vehicle. Radar device 2) in the embodiment, imaging means for imaging a predetermined range around the host vehicle (for example, camera unit 3 in the embodiment described later), and an object detected by the transmitter / receiver means the moving target and the stationary object Moving target discriminating means for discriminating whether the target is a target (for example, a moving target discriminating unit 12 in an embodiment described later) and object extraction for extracting a pedestrian or a vehicle from an image captured by the imaging unit The transmission / reception is performed based on the means (for example, the object extraction unit 13 in the embodiment described later), the discrimination result of the moving target discrimination means and the extraction result of the object extraction means. Specific object determining means object detected by the step to determine whether or not the pedestrian or the vehicle (e.g., the specific object determination unit 14 in the embodiment described later) provided with a said object extraction means, The pedestrian or the vehicle is extracted both in the case of the moving target and in the case of the stationary target, and the moving object is detected when the object detected by the transmitting / receiving means is a moving target. When it is determined by the target determining means, the time until the specific object determining means determines that the pedestrian or the vehicle is determined, and the object detected by the transmitting / receiving means is the stationary target If it is determined by the moving target discriminating means, the object recognition apparatus characterized by shorter than the time to confirm that the more the pedestrian or the vehicle to the specific object identifying means (e.g. If an object recognition device 1) in the embodiment described later.

The invention according to claim 2 is the fault according to claim 1, wherein the specific object discriminating means is a count value set for each object that increases every time the object is detected by the transmitting / receiving means. The object reliability is added to the recognition reliability, which is a count value set for each pedestrian or vehicle that increases each time the object is extracted as the pedestrian or the vehicle by the object extraction means. The obstacle when the object is determined to be the pedestrian or the vehicle when the value is equal to or greater than a predetermined value, and the object is determined to be a moving target by the moving target determining means. The increase amount of the reliability is larger than the increase amount of the obstacle reliability when the moving target determination unit determines that the object is not a moving target.

  According to the first and second aspects of the invention, when it is determined that the object detected by the transmitting / receiving unit is a moving target, the object detected by the transmitting / receiving unit is not a moving target. It can be determined earlier that the object is a specific object, rather than the object determined to be. As a result, various control operation timings that the own vehicle can take with respect to the specific object can be advanced.

It is a block diagram in one embodiment of an object recognition device concerning this invention. It is a flowchart which shows the specific object discrimination | determination process in the object recognition apparatus of the said embodiment. It is a figure which shows an example of the object detection by a radar apparatus. It is a figure which shows an example of the object detection by a camera unit. It is a figure which shows an example of the state which fixed the specific thing at the early stage from the object detection by a camera unit. It is a figure which shows the increase state of the reliability until it determines that an object is a specific object, (A) is a case where an object is a moving object, (B) is a case where an object is a stationary object.

Embodiments of an object recognition apparatus according to the present invention will be described below with reference to the drawings of FIGS.
As shown in FIG. 1, the object recognition apparatus 1 of this embodiment uses, for example, a driving force of an internal combustion engine 21 as a driving source to transmit a transmission (T / T) such as an automatic transmission (AT) or a continuously variable automatic transmission (CVT). M) It is mounted on a vehicle that travels by being transmitted to the drive wheels of the vehicle via 22. In addition to the object recognition device 1, this vehicle includes a brake actuator 23, a steering actuator 24, and a notification device 25.

The object recognition device 1 includes a radar device (transmission / reception means) 2, a camera unit (imaging means) 3, a host vehicle state sensor 4, and an electronic control device 10.
The radar device 2 transmits an electromagnetic wave such as a laser beam or a millimeter wave toward the front in the traveling direction of the host vehicle, and the transmitted electromagnetic wave is an object outside the host vehicle (for example, a structure, a pedestrian, another vehicle). Etc.) is received, and the transmitted electromagnetic wave and the received electromagnetic wave (reflected wave) are mixed to generate a beat signal and output to the electronic control unit 10.

  The camera unit 3 includes a camera 3a composed of a CCD camera or a CMOS camera, and an image processing unit 3b. The image processing unit 3b performs predetermined image processing, such as filtering and binarization processing, on the image of the external world ahead of the traveling direction of the host vehicle obtained by capturing with the camera 3a. Image data is generated and output to the electronic control unit 10.

  The host vehicle state sensor 4 includes, as vehicle information about the host vehicle, for example, a vehicle speed sensor that detects the speed (vehicle speed) of the host vehicle, a yaw rate sensor that detects a yaw rate (rotational angular velocity about the vertical axis of the center of gravity of the vehicle), steering, A steering angle sensor that detects an angle (direction and magnitude of a steering angle input by the driver) and an actual steering angle (steering angle) according to the steering angle, a steering torque sensor that detects steering torque, and an artificial satellite, for example Positioning signals such as GPS (Global Positioning System) signals for measuring the position of the vehicle using GPS, position signals transmitted from information transmitting devices outside the host vehicle, and other appropriate gyro sensors and acceleration sensors Based on the detection results, the position sensor that detects the current position and traveling direction of the host vehicle, the sensor that detects the amount of depression of the accelerator pedal, and the depression state of the brake pedal are detected. Configured to include a sensor or the like. The own vehicle state sensor 4 outputs a vehicle information signal corresponding to the detected information to the electronic control device 10.

  The electronic control unit 10 includes an object detection unit 11, a moving target determination unit (moving target determination unit) 12, an object extraction unit (object extraction unit) 13, a specific object determination unit (specific object determination unit) 14, Vehicle control unit 15.

  The object detection unit 11 calculates the position, velocity, reflection level, etc. of the object that has reflected the electromagnetic wave based on the beat signal input from the radar device 2, and outputs the calculated information to the moving target determination unit 12. To do. Note that the speed of the object can be calculated from the relative speed of the host vehicle detected by the radar apparatus 2 with a time difference and calculated based on the position information of the object, and the speed of the host vehicle.

Based on the velocity of the object input from the object detection unit 11, the moving target discriminating unit 12 is an object that is moving (moving target) or an object that is not moving, that is, a stationary object (moving object). It is determined whether the object is a non-target object, and the determination result is output to the specific object determining unit 14.
Further, the object detection unit 11 calculates a predicted position of the object after a predetermined time based on the calculated speed (or relative speed) of the object, and outputs the predicted position information to the object extraction unit 13.

  Image data is input from the camera unit 3 to the object extraction unit 13 and predicted position information of the object is input from the object detection unit 11. The object extraction unit 13 sets, for example, a region having a predetermined size (hereinafter referred to as an integrated range) around the predicted position on the image data input from the camera unit 3 based on the input predicted position information.

  Further, the object extraction unit 13 extracts an object on the image data by edge extraction based on the luminance value of the pixels included in the set integration range, and stores the human body stored in advance for the extracted object. A pattern matching process is performed with the model image of the vehicle, and it is determined whether or not the extracted object matches the model image of the human body or the vehicle. Then, the object extraction unit 13 outputs the determination result to the specific object determination unit 14.

  The specific object discriminating unit 14 includes a discrimination result input from the moving target discriminating unit 12, that is, a discrimination result whether it is a moving target or a stationary object, and a discrimination result input from the object extracting unit 13, that is, an object extracted on the image data. Whether or not the detected object is a pedestrian or a vehicle is determined based on the determination result of whether or not the object matches the model image of the human body or the vehicle. In this embodiment, the “specific object” is a pedestrian or a vehicle.

The vehicle control unit 15 performs contact according to the determination result in the specific object determination unit 14 when, for example, an object determined as a pedestrian or a vehicle by the specific object determination unit 14 may come into contact with the host vehicle. The traveling of the host vehicle is controlled so as to avoid it. For example, the vehicle control unit 15 controls the driving force of the internal combustion engine 21, the control signal that controls the speed change operation of the transmission 22, the control signal that controls the deceleration operation by the brake actuator 23, and the own vehicle by the steering actuator 24. At least one of the control signals for controlling the steering operation of the steering mechanism (not shown) is output, and deceleration control or steering control of the host vehicle is executed as the contact avoiding operation.
Further, the vehicle control unit 15 controls at least one of the alarm output timing and the output content by the notification device 25 according to the degree of possibility of contact with a pedestrian or the vehicle.

Next, the specific object determination process performed in the specific object determination part 14 is demonstrated.
In this object recognition device 1, when the presence of an object is detected by the radar device 2, an integrated range is set on the image data obtained by the camera unit 3 based on the predicted position of the object, and extracted in the integrated range. When pattern matching processing is performed on an object and it is matched, it is counted as a pedestrian candidate or vehicle candidate, and this series of processing is executed a plurality of times every predetermined time, and the count value is integrated. When a predetermined determination threshold is reached, it is determined that the person is a pedestrian or a vehicle.
When a plurality of objects are detected by the radar apparatus 2, an integrated range is set on the image data for each detected object, the object is extracted, pattern matching processing is performed, and a pedestrian is detected for each object. The count values of candidates or vehicle candidates are integrated, and it is determined whether the object is a pedestrian or a vehicle based on the integrated value for each object.

  By the way, in the object detection by the radar device 2, an object that is determined to be a stationary object is highly likely to be an upper or lower object or noise that exists above or below without affecting the traveling of the host vehicle. An object that is determined to be a moving object is unlikely to be an upper / lower object or noise. Here, the upper and lower objects refer to, for example, a signboard installed at a high place or a short stationary object present on a road.

  Therefore, the object determined as the moving target by the moving target determination unit 12 is compared with the object on the image data of the camera unit 3 when compared with the object determined as a stationary object by the moving target determination unit 12. When the pattern matching process is performed, the probability of matching is high, and it can be said that the possibility of being a pedestrian or vehicle is high. In other words, an object that is determined as a moving target by the moving target determination unit 12 has a detection recognition reliability that is determined as a pedestrian or a vehicle, compared to an object that is determined as a stationary object by the moving target determination unit 12. It can be said that it is expensive.

  Therefore, in the object recognition device 1, an object that has been determined to be a moving target by the moving target determination unit 12 (in other words, an object that has been determined to be a moving object in object detection by the radar device 2). Is a pedestrian or vehicle rather than an object that is determined to be a stationary object by the moving target determination unit 12 (in other words, an object that is determined to be a stationary object in the object detection by the radar device 2). This makes it easier to confirm an affirmative determination.

Next, the specific object determination process in this embodiment will be described with reference to the flowchart of FIG.
The specific object determination processing routine shown in the flowchart of FIG. 2 is repeatedly executed by the electronic control device 10 at regular intervals.
First, in step S101, the radar device 2 detects an object existing ahead in the traveling direction of the host vehicle, detects the reflection level of the reflected wave, and calculates the position and speed of the object.
Next, the process proceeds to step S102, where it is determined whether or not the detected object is a moving target based on the speed of the object calculated in step S101.

  FIG. 3 is a diagram illustrating an example of object detection by the radar device 2, where the symbol V in the figure indicates the host vehicle, and the symbols Xa, Xb, Xc, Xd, and Xe in the diagram are objects detected by the radar device 2. Is shown. In FIG. 3, objects Xa, Xb, and Xc indicate objects that are determined as stationary objects, and objects Xd and Xe indicate objects that are determined as moving targets. However, the radar device 2 cannot determine what the detected object is.

If the determination result in step S102 is “NO”, the object detected by the radar apparatus 2 is a stationary object, so the process proceeds to step S103, and the previous value R _n−1 of the obstacle reliability is “1”. Is added as the current value R _n of the obstacle reliability (R _n = R _n−1 +1). The initial value of the obstacle reliability R _n is 0.
On the other hand, if the determination result in step S102 is “YES”, the object detected by the radar apparatus 2 is a moving target, so the process proceeds to step S104, and the previous value R _n−1 of the obstacle reliability is set. The value obtained by adding “2” is updated as the current value R _n of the obstacle reliability (R _n = R _n−1 +2). That is, when it is determined that the object detected by the radar device 2 is a moving target, the amount of increase in the obstacle reliability is increased compared to the case where it is determined that the object is a stationary object.

  The process proceeds from step S103, S104 to step S105, and based on the image data of the camera unit 3, an object is extracted within the integrated range set on the image data, and the lateral movement speed of the object (vehicle width of the host vehicle) is extracted. (Moving speed in a direction parallel to the direction) and lateral movement acceleration are detected. Note that the lateral movement speed of the object can be calculated based on the position information of the object on the plurality of image data obtained by the camera unit 3 with a time difference. Further, the lateral movement acceleration of the object can be calculated by differentiating the lateral movement speed with respect to time.

Next, proceeding to step S106, pattern matching processing is performed on the object detected at step S105, and it is determined whether or not the object looks like a person or a vehicle as a result of the pattern matching processing.
FIG. 4 is a diagram showing an example of object detection by the camera unit 3, and shows a case where an object is extracted in the integrated range for each object on the image data. That is, the case where the objects Xa to Xe detected by the radar device 2 are also detected on the image data of the camera unit 3 is shown. In FIG. 4, symbol V indicates the host vehicle.
In FIG. 4, objects Xc and Xd are objects determined as pedestrians by pattern matching processing, object Xe is an object determined as a vehicle by pattern matching processing, and objects Xa and Xb are pedestrians by pattern matching processing. Or the object determined not to be a vehicle is shown. That is, the object Xc is a stationary pedestrian-like object, the object Xd is a moving pedestrian-like object, and the object Xe is a moving vehicle-like object.

If the determination result in step S106 is “NO”, the object is not a person or vehicle-like object, so the process proceeds to step S107, and a value obtained by adding “0” to the previous value C _n−1 of the recognition reliability is obtained. And updated as the current value C _n of the recognition reliability (C _n = C _n−1 +0). The initial value of the recognition reliability C _n is set to 0.
On the other hand, if the determination result in step S106 is “YES”, the object is likely to be a person or a vehicle, so the process proceeds to step S108, and “1” is added to the previous value C _n−1 of the recognition reliability. The value is updated as the current value C _n of the recognition reliability (C _n = C _n−1 +1).
That is, the recognition reliability is increased only when it is determined that the object detected by the camera unit 3 is a person or a vehicle.

The process proceeds from step S107, S108 to step S109, the current value Cn of the recognition reliability is 1 or more, and the total reliability Rn + Cn, which is the sum of the current value Rn of the obstacle reliability and the current value Cn of the recognition reliability. Whether or not exceeds the object detection / recognition decision threshold N.
If the determination result in step S109 is “NO”, the process proceeds to step S110 to set the object detection / recognition confirmation flag to “0”.
On the other hand, if the determination result in step S109 is “YES”, the process proceeds to step S111, the object detection / recognition confirmation flag is set to “1”, and the execution of this routine is once ended. As a result, the object detected by the radar device 2 and the camera unit 3 is determined to be a pedestrian or a vehicle.

  FIG. 5 shows a state in which the object Xc is not yet determined as a pedestrian, the object Xd is determined as a pedestrian, and the object Xe is determined as a vehicle. That is, the object circled in FIG. 5 is an object that is determined to be a specific pedestrian or vehicle.

  This specific object determination process is executed in parallel for each object detected by the radar apparatus 2 with the series of processes in steps S101 to S111 as one cycle, and a pedestrian or vehicle is determined for each object. Is called.

  In the specific object determination process, when the cycle is executed a plurality of times, unless it is determined even once that the object extracted on the image data of the camera unit 3 in the process of step S106 is a pedestrian or vehicle-like object, No matter how many times the object is detected by the radar device 2, the object is not determined to be a pedestrian or a vehicle.

Further, in this specific object discrimination process, in one cycle, when the object detected by the radar device 2 is a stationary object, the obstacle reliability is increased by one time, and when the object is a moving target, the obstacle is detected. The reliability increases twice as many times. In addition, in one cycle, when it is determined that the object extracted on the image data of the camera unit 3 is a pedestrian or vehicle-like object, the recognition reliability increases by one time, and the pedestrian or vehicle-like object is increased. If it is determined that it is not, the recognition reliability does not increase.
Therefore, when it is determined that the object detected by the radar apparatus 2 is a moving target, an affirmative determination is made that the object is a pedestrian or a vehicle compared to a case where the object is determined to be a stationary object. Confirmation is easily performed.

For example, when an affirmative determination (moving target) is made at S102 and an affirmative determination (an object that seems to be a pedestrian or a vehicle) is made at step S106, the increase amount of the obstacle reliability “2” and the increase amount of the recognition reliability Since “1” is added, the increase amount of the total reliability in one cycle is “3”. For convenience of the following description, the cycle determined in this way is referred to as an A type cycle.
On the other hand, when a negative determination (stationary object) is made in S102 and an affirmative determination (a pedestrian or vehicle-like object) is made in step S106, the obstacle reliability increase amount “1” and the recognition reliability increase. Since the amount “1” is added, the increase amount of the total reliability in one cycle is “2”. For convenience of the following description, the cycle determined in this way is referred to as a B type cycle.
Therefore, the A type cycle makes it easier to reach the object detection / recognition determination threshold N in the total reliability than the B type cycle.

  FIG. 6 is a diagram showing the state of increase in reliability until the object is determined to be a specific object, with the time axis on the horizontal axis, and FIG. 6 (A) shows the case where the A type cycle continues. FIG. 6B shows a case where the B type cycle continues. Also from this figure, it can be seen that the total reliability reaches the object detection / recognition decision threshold N in the A type cycle in a shorter time than in the B type cycle.

As described above, according to the object recognition apparatus 1 of this embodiment, an object that is determined to be a moving object in the object detection by the radar apparatus 2 is a stationary object in the object detection by the radar apparatus 2. The positive determination that the object is a specific object (pedestrian or vehicle) can be established earlier than the object determined as “A”.
As a result, for example, the operation timing of the traveling control of the host vehicle for avoiding contact with the pedestrian or the vehicle can be advanced, and the traveling control can be executed early.

[Other Embodiments]
The present invention is not limited to the embodiment described above.
For example, in the above-described embodiment, the increase amount of the obstacle reliability when the object detected by the radar device 2 is determined to be a moving target is set to “2”, but this increase amount is set to “1. It may be “5” or “3”. In short, it may be a value larger than the increase amount of the obstacle reliability when it is determined that it is not a moving target, and can be set to an appropriate value.
In the above-described embodiment, the specific object is a pedestrian or a vehicle. However, an animal such as a dog or a cat can be added to the specific object.
The detection direction of the object is not limited to the front in the traveling direction of the host vehicle, but may be the rear in the front-rear direction of the host vehicle or the side of the host vehicle.

  Control using an object determined as a specific object by the object recognition device is not limited to travel control for avoiding contact between the own vehicle and the object, and the specific object is the preceding vehicle and the own vehicle is the preceding vehicle. Various controls that the vehicle can take with respect to a specific object are possible, such as follow-up running control for running following the vehicle.

  Each configuration in each embodiment, a combination thereof, and the like are examples, and the addition, omission, replacement, and other modifications of the configuration can be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Object recognition apparatus 2 Radar apparatus (transmission / reception means)
3 Camera unit (imaging means)
12 Moving target discrimination unit (moving target discrimination means)
13 Object extraction unit (object extraction means)
14 Specified object discriminating part

Claims (2)

Transmitting and receiving means for transmitting an electromagnetic wave toward a predetermined range around the host vehicle and receiving a reflected wave generated by the reflection of the electromagnetic wave by an object existing around the host vehicle;
Imaging means for imaging a predetermined range around the host vehicle;
A moving target discriminating means for discriminating whether the object detected by the transmitting / receiving means is a moving target or a stationary target;
An object extraction means for extracting a pedestrian or a vehicle from an image captured by the imaging means;
Specific object determining means for determining whether the object detected by the transmitting / receiving means is the pedestrian or the vehicle based on the determination result of the moving target determining means and the extraction result of the object extracting means; ,
With
The object extracting means extracts the pedestrian or the vehicle both in the case of the moving target and in the case of the stationary target,
When the moving target discriminating unit determines that the object detected by the transmitting / receiving unit is a moving target, the time until the specific object discriminating unit determines that the object is the pedestrian or the vehicle is determined. When the moving object discriminating means discriminates that the object detected by the transmission / reception means is the stationary target, the specific object discriminating means determines that the object is the pedestrian or the vehicle. An object recognition device characterized in that it is shorter than time. The specific object discriminating means includes an obstacle reliability that is a count value set for each object that is increased every time the object is detected by the transmitting / receiving means, and the object is extracted by the object extracting means. The object is the pedestrian or the vehicle when a value obtained by adding the recognition reliability that is a count value set for each pedestrian or the vehicle that is increased every time it is extracted as the vehicle is equal to or greater than a predetermined value. It is determined that
The amount of increase in the obstacle reliability when the moving target determining means determines that the object is a moving target is determined by the moving target determining means that the object is not a moving target. The object recognition apparatus according to claim 1, wherein the object recognition apparatus is larger than an increase in the obstacle reliability in a case.

Images