JP3999088B2 - Obstacle detection device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an obstacle detection apparatus that detects and warns an object that is likely to affect the running of a host vehicle from an image taken by an imaging means.
[0002]
[Prior art]
A conventional obstacle detection device extracts an object such as a pedestrian that may collide with the own vehicle from an image around the own vehicle captured by an imaging means such as an infrared camera, and automatically extracts the information. Provide to the vehicle driver. In this device, a portion having a high temperature in an image around the host vehicle captured by a pair of left and right stereo cameras is used as a target, and the distance to the target is calculated by obtaining the parallax of the target. From the moving direction of the object and the position of the object, an object that is likely to affect the traveling of the host vehicle is detected and an alarm is output (for example, see Patent Document 1).
In addition, there are some that prevent erroneous detection of an object by excluding a region showing a high temperature that cannot be an object such as a pedestrian in an image around the own vehicle captured by an imaging means such as an infrared camera. (For example, refer to Patent Document 2).
[0003]
[Patent Document 1]
JP 2001-6096 A [Patent Document 2]
Japanese Patent Laid-Open No. 2001-108758
[Problems to be solved by the invention]
However, in the conventional technology, by removing a region showing a high temperature that cannot be an object such as a pedestrian, an erroneous detection such as detecting an artificial structure showing an extreme temperature as an object can be prevented. Although there is a problem, it is difficult to recognize as a non-alarming object and to exclude it from the object such as an electric pole beside a road, a streetlight pole, or a vending machine showing a temperature equivalent to that of a pedestrian. For this reason, there is a problem that warnings for these artificial structures that are not pedestrians are troublesome for the driver.
[0005]
The present invention has been made in view of the above-described problems, and provides an obstacle detection device capable of causing a driver to accurately recognize an object such as a pedestrian that is likely to affect the traveling of the host vehicle. With the goal.
[0006]
[Means for Solving the Problems]
In order to solve the above problems, an obstacle detection apparatus according to the invention of claim 1 is based on an image captured by an infrared camera (for example, the infrared camera 2R, 2L of the embodiment), and the vehicle (for example, the vehicle of the embodiment). In an obstacle detection device that detects an object that affects the vehicle 10) and issues an alarm to the driver of the host vehicle, the object is detected and an alarm is issued based on the operation of the driver Object storage control means (for example, step S27 to step S28 in the embodiment) for storing the feature and position information of the object in the storage means , and the feature and position information of the detected object are stored in the storage means. If it is identical to the characteristics and location information of the physical object, the alarm output control means to stop the alarm output for the object (e.g. embodiment step S24~ step S25) Characterized by comprising.
[0007]
The obstacle detection device having the above configuration stores the feature such as the shape of the object detected by the object storage control unit and the position information in the storage unit by the driver's operation, so that the driver can It is possible to memorize information on an arbitrary object to be consciously excluded from the alarm target. Then, when an object whose information is stored in the storage means is detected again, the alarm output control means determines that the object is an object that the driver consciously wants to exclude from the alarm object, and the object The alarm output for can be stopped.
The obstacle detection apparatus according to the invention of claim 2 is characterized in that the object storage control means stores the shape of the object as a feature of the object stored in the storage means.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of an obstacle detection apparatus according to an embodiment of the present invention.
In FIG. 1, reference numeral 1 denotes an in-vehicle image processing unit including a CPU (central processing unit) that controls the obstacle detection device of the present embodiment, and includes two infrared cameras 2 </ b> R capable of detecting far infrared rays. 2L is connected, and a signal indicating the traveling state of the vehicle such as vehicle speed information and turning angle information of the vehicle on which the obstacle detection device of the present embodiment is mounted is input. As a result, the image processing unit 1 detects a pedestrian or animal in front of the vehicle from the infrared image around the vehicle and a signal indicating the running state of the vehicle, and warns when the possibility of a collision is high. To emit.
[0009]
Further, the image processing unit 1 displays a speaker 4 for issuing a warning by voice and images taken by the infrared cameras 2R and 2L, and allows the vehicle driver to recognize an object having a high risk of collision. For example, a meter-integrated display integrated with a meter that expresses the running state of the host vehicle, a NAVID display installed on the console of the host vehicle, and information displayed at a position on the front window that does not obstruct the driver's front view An image display device 5 including a HUD (Head Up Display) or the like is connected.
Further, a navigation device 6 is connected to the image processing unit 1, and the image processing unit 1 detects the distance of the object detected by the image processing unit 1 based on the position information acquired from the navigation device 6. -New position information can be stored in consideration of azimuth information.
[0010]
The image processing unit 1 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a CPU (central processing unit) that performs various arithmetic processes, RAM (Random Access Memory) used for storing data, ROM (Read Only Memory) for storing programs and tables executed by the CPU, maps, etc., driving signals for the speaker 4, display signals for the image display device 5, etc. Output signals, and signals indicating the vehicle running state, such as images taken by the infrared cameras 2R and 2L and vehicle speed information and turning angle information, are converted into digital signals and input to the CPU. It is configured to be.
[0011]
Furthermore, the image processing unit 1 and the navigation device 6 include a nonvolatile memory such as a flash memory, and a switch (not shown) for storing information on an object connected to the image processing unit 1. In response to the signal, information related to the shape of the object (object) detected in front of the vehicle is stored in the image processing unit 1, and information related to the position is stored in a non-volatile memory such as a flash memory provided in the navigation device 6. be able to.
[0012]
As shown in FIG. 2, the infrared cameras 2R and 2L are disposed in the front part of the host vehicle 10 at positions that are substantially symmetrical with respect to the center of the host vehicle 10 in the vehicle width direction. The optical axes of 2R and 2L are parallel to each other and are fixed so that the heights from both road surfaces are equal. The infrared cameras 2R and 2L have a characteristic that the output signal level increases (the luminance increases) as the temperature of the object increases.
Further, the HUD 7a is provided so that the display screen is displayed at a position that does not obstruct the driver's front view of the front window of the host vehicle 10.
Furthermore, the host vehicle 10 is also equipped with a yaw rate sensor 3 that measures the yaw rate for obtaining the turning angle information.
[0013]
Next, the operation of the present embodiment will be described with reference to the drawings.
FIG. 3 is a flowchart showing an object detection / alarm operation of a pedestrian or the like in the image processing unit 1 of the obstacle detection apparatus of the present embodiment.
First, the image processing unit 1 acquires an infrared image that is an output signal of the infrared cameras 2R and 2L (step S1), performs A / D conversion (step S2), and stores a grayscale image in an image memory (step S1). S3). Here, the right image is obtained by the infrared camera 2R, and the left image is obtained by the infrared camera 2L. In addition, since the horizontal position of the same object on the display screen is shifted in the right image and the left image, the distance to the object can be calculated from this shift (parallax).
[0014]
If a grayscale image is obtained in step S3, then the right image obtained by the infrared camera 2R is used as a reference image, and binarization processing of the image signal, that is, an area brighter than the luminance threshold value ITH is “1” ( White) and the dark area is set to “0” (black) (step S4).
FIG. 4A shows a grayscale image obtained by the infrared camera 2R, and binarization processing is performed on the grayscale image to obtain an image as shown in FIG. 4B. In FIG. 4B, for example, an object surrounded by a frame from P1 to P4 is an object displayed as white on the display screen (hereinafter referred to as “high luminance region”).
When the binarized image data is acquired from the infrared image, the binarized image data is converted into run-length data (step S5). The line represented by the run-length data is an area that is whitened by binarization at the pixel level, and has a width of one pixel in the y direction, and each has a width in the x direction. It has the length of the pixels constituting the run-length data.
[0015]
Next, the target object is labeled from the image data converted into run-length data (step S6), thereby performing a process of extracting the target object (step S7). That is, of the lines converted into run-length data, a line having a portion overlapping in the y direction is regarded as one object, so that, for example, the high luminance regions P1 to P4 shown in FIG. To be grasped as a value object).
When the extraction of the object is completed, next, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed rectangle of the extracted object are calculated (step S8).
[0016]
Here, the area S is (x [i], y [i], run [i], A) (i = 0, 1, 2,... N−1) ), The length of the run length data (run [i] −1) is calculated for the same object (N pieces of run length data). Also, the coordinates (xc, yc) of the center of gravity G of the object A are the length (run [i] -1) of each run length data and the coordinates x [i] or y [i] of each run length data. Are multiplied by the area S and calculated by multiplying them by the area S. Further, the aspect ratio ASPECT is calculated as a ratio Dy / Dx between the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object.
Since the run-length data is indicated by the number of pixels (number of coordinates) (= run [i]), the actual length needs to be “−1” (subtract 1) (= run [i]. ] -1). Further, the position of the center of gravity G may be substituted by the position of the center of gravity of the circumscribed rectangle.
[0017]
Once the center of gravity, area, and circumscribing aspect ratio of the object can be calculated, next, the object is tracked between times, that is, the same object is recognized for each sampling period (step S9). For tracking between times, the time obtained by discretizing the time t as an analog quantity with the sampling period is set as k. For example, when the objects A and B are extracted at the time k, the objects C and D extracted at the time (k + 1) and the target The identity determination with the objects A and B is performed. When it is determined that the objects A and B and the objects C and D are the same, the objects C and D are changed to labels of the objects A and B, respectively, so that tracking is performed for a time.
Further, the position coordinates (center of gravity) of each object recognized in this way are stored in the memory as time-series position data and used for the subsequent calculation processing.
[0018]
Note that the processes in steps S4 to S9 described above are performed on a binarized reference image (in this embodiment, a right image).
Next, the vehicle speed information detected by the vehicle speed sensor (not shown) and the turning angle information obtained by time integration of the yaw rate detected by the yaw rate sensor 3 are read, and the turning angle θr of the host vehicle 10 is calculated (step). S10).
[0019]
On the other hand, in parallel with the processing of step S9 and step S10, in steps S11 to S13, processing for calculating the distance z between the object and the host vehicle 10 is performed. Since this calculation requires a longer time than steps S9 and S10, it is executed in a cycle longer than that of steps S9 and S10 (for example, a cycle that is about three times the execution cycle of steps S1 to S10).
First, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (here, the entire region surrounded by the circumscribed rectangle is searched for from the right image). And the circumscribed rectangle of the search image is referred to as a target frame) (step S11).
[0020]
Next, a search area R2 for searching for an image (hereinafter referred to as “corresponding image”) corresponding to the search image R1 from the left image is set (step S12), and a correlation operation with the search image R1 is executed in the search area R2. Thus, a region having a high correlation with the search image R1 is obtained and set as a corresponding image. Then, the parallax / distance calculation calculation between the search image R1 and the corresponding image is performed, and the distance z between the host vehicle 10 and the object is calculated (step S13).
[0021]
When the calculation of the turning angle θr in step S10 and the calculation of the distance to the object in step S13 are completed, the turning angle correction for correcting the positional deviation on the image due to the turning of the host vehicle 10 is next performed. Perform (step S14). In the turning angle correction, when the host vehicle 10 turns, for example, to the left by the turning angle θr during the period from time k to (k + 1), the range of the image is shifted in the x direction by Δx on the image obtained by the camera. This is a process for correcting this.
[0022]
When the turning angle correction is completed, the coordinates (x, y) and the distance z in the image are converted into real space coordinates (X, Y, Z) (step S15).
Here, the real space coordinates (X, Y, Z) are, as shown in FIG. 2, with the origin O as the midpoint position (position fixed to the host vehicle 10) of the attachment position of the infrared cameras 2R, 2L. The coordinates in the image are determined as shown in the figure, with the center of the image as the origin and the horizontal direction as x and the vertical direction as y.
In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).
Next, when the turning angle correction for the real space coordinates is completed, N pieces of real space position data after the turning angle correction (for example, about N = 10) obtained during the monitoring period of ΔT for the same object, that is, From the time series data, an approximate straight line LMV corresponding to the relative movement vector between the object and the host vehicle 10 is obtained.
[0023]
Next, the latest position coordinates P (0) = (X (0), Y (0), Z (0)) and (N-1) position coordinates P (N−1) before the sample (before time ΔT). = (X (N-1), Y (N-1), Z (N-1)) is corrected to a position on the approximate straight line LMV, and the corrected position coordinates Pv (0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)) are obtained.
Thereby, a relative movement vector is obtained as a vector from the position coordinates Pv (N−1) to Pv (0) (step S16).
In this way, by calculating an approximate straight line that approximates the relative movement locus of the object with respect to the host vehicle 10 from a plurality (N) of data within the monitoring period ΔT, the influence of the position detection error is reduced. Thus, the possibility of collision with the object can be predicted more accurately.
[0024]
When the relative movement vector is obtained in step S16, next, an alarm determination process is performed to determine whether or not there is a possibility of collision with the object detected by the host vehicle 10 and output an alarm (step) S17). Details of the alarm determination process will be described later.
In addition, after the alarm determination process is completed, the process returns to step S1 and the above process is repeated.
[0025]
The above is the object detection / alarm operation in the image processing unit 1 of the obstacle detection apparatus of the present embodiment. Next, referring to the flowchart shown in FIG. 5, step S17 of the flowchart shown in FIG. The alarm determination process for determining the possibility of collision with the object and outputting an alarm will be described in more detail.
FIG. 5 is a flowchart showing the alarm determination processing operation of the present embodiment.
First, the image processing unit 1 calculates the relative speed of the object with respect to the host vehicle 10. Then, by dividing the distance between the host vehicle 10 and the target object by the relative speed, the time until the contact between the host vehicle 10 and the target object is calculated, and whether or not this is more than a margin time for avoiding the collision. Determine. It should be noted that the range in the height direction is also defined for determining whether or not to collide.
[0026]
If it is determined in step S21 that there is no possibility of collision between the host vehicle 10 and the object (NO in step S21), the alarm determination process is terminated without doing anything.
In Step S21, when it is determined that there is a possibility of collision between the host vehicle 10 and the object (YES in Step S21), the image processing unit 1 determines whether or not the object exists in the approach determination area. Is determined (step S22). Here, the approach determination area is an area in front of the host vehicle 10 where it is determined that the possibility of a collision with the host vehicle 10 is extremely high if the object continues to exist.
[0027]
If it is determined in step S22 that the object does not exist in the approach determination area (NO in step S22), the object exists in the approach determination area outside the approach determination area in the lateral direction (x coordinate direction). Then, an approach collision determination is performed to determine whether or not there is a possibility of an approach collision in which the target vehicle advances and enters the approach determination area and the host vehicle 10 and the target object collide (step S23).
If it is determined in step S23 that there is no possibility of an entry collision between the host vehicle 10 and the object (NO in step S23), the alarm determination process is terminated without doing anything.
[0028]
On the other hand, if it is determined in step S22 that the object is present in the approach determination area (YES in step S22), it is determined whether or not the position information of the object is a registered point stored in the navigation device 6 (step S24). ).
In step S24, when the position information of the object is a registered point stored in the navigation device 6 (YES in step S24), whether the shape of the object is a registered shape stored in the image processing unit 1 or not. Is determined (step S25).
In step S25, when the shape of the object is a registered shape stored in the image processing unit 1 (YES in step S25), the alarm determination process is terminated without doing anything.
[0029]
If it is determined in step S23 that there is a possibility of an approach collision between the host vehicle 10 and the object (YES in step S23), or position information of the object is stored in the navigation device 6 in step S24. If it is not a registered point (NO in step S24), or if the shape of the object is not a registered shape stored in the image processing unit 1 in step S25 (NO in step S25), the host vehicle 10 and the target An object that determines that there is a high possibility of contact with an object, issues an audio alarm through the speaker 4, and outputs an image obtained by, for example, the infrared camera 2R to the image display device 5 to approach The object is displayed as an emphasized image for the driver of the host vehicle 10 (step S26).
[0030]
At this time, the driver of the vehicle 10 must always be alerted at the same point, such as a power pole or streetlight pole or vending machine, where the object has a temperature equivalent to that of a pedestrian, with regard to the warning and highlighting that is issued. When it can be recognized that the structure is a man-made structure that is regarded as a target, features such as the shape of the target can be stored in the image processing unit 1 and position information can be stored in the navigation device 6.
That is, the image processing unit 1 operates a switch (not shown) for storing a feature such as a shape, which is information of an object connected to the image processing unit 1, and position information by a driver of the host vehicle 10. Then, it is determined whether or not the switch is turned on (pressed) (step S27).
[0031]
If the switch is not turned on in step S27 (NO in step S27), the alarm determination process is terminated without doing anything.
On the other hand, if the switch is turned on in step S27 (YES in step S27), the image processing unit 1 extracts the shape of the target object (artificial structure for which an alarm has been issued) and stores it as a registered shape. At the same time, the location information acquired from the navigation device 6 is added to the distance / orientation information of the object detected by the image processing unit 1 and stored in the navigation device 6 as a registered location again to execute the location registration (step). S28). Then, the alarm determination process ends.
[0032]
In the above-described embodiment, the image processing unit 1 includes the object storage control unit and the alarm output control unit. More specifically, steps S27 to S28 in FIG. 5 correspond to the object storage control means. Further, steps S24 to S25 in FIG. 5 correspond to the alarm output control means.
[0033]
As described above, the obstacle detection device according to the present embodiment has an artificial structure in which an object detected from images captured by the infrared cameras 2R and 2L is always an alarm target at the same point, for example. If the object is an object, the driver's operation on the image processing unit 1 stores the feature such as the shape of the object detected by the object storage control unit and the position information in the storage unit, so that the driver can It is possible to store information on an arbitrary object that is consciously excluded from the alarm target.
[0034]
When the object whose information is stored in the storage unit is detected again, the image processing unit 1 determines that the object is an object that the driver wants to exclude from the alarm target by the alarm output control unit. The alarm output for the object can be stopped.
Therefore, even for artificial structures that cannot be excluded from roadside utility poles, streetlight poles, or vending machines that show the same temperature as pedestrians, their features such as shape and location information Is stored and designated as a non-alarm object, so that an effect of suppressing an alarm that bothers the driver can be obtained.
[0035]
Further, in the present embodiment, the position information of the object is obtained by adding the position information that can be acquired from the navigation device 6 to the position information that is detected by the image processing unit 1 and adding new position information (registration point). However, since the alarm is performed when the object is in the vicinity of the vehicle, the position information of the object can be obtained from the navigation device 6 when the switch is operated. It is also possible to set a predetermined range with reference to.
[0036]
【The invention's effect】
As described above, according to the obstacle detection device of the first aspect, it is possible to store the characteristics and position information of any object that the driver wants to consciously exclude from the alarm object. And when the memorize | stored target object is detected again, the alarm output with respect to this target object can be stopped.
Therefore, even for an artificial structure that shows a temperature equivalent to that of a pedestrian and cannot be excluded from the object, the shape and other features and position information should be explicitly stored and designated as a non-alarm object. Thus, it is possible to suppress the output of an alarm bothering the driver.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an obstacle detection apparatus according to an embodiment of the present invention.
FIG. 2 is a diagram showing attachment positions of an infrared camera, a sensor, a display, and the like in a vehicle.
FIG. 3 is a flowchart showing an object detection / alarm operation in the obstacle detection apparatus according to the embodiment;
FIG. 4 is a diagram illustrating a grayscale image obtained by an infrared camera and a binarized image thereof.
FIG. 5 is a flowchart showing an alarm determination processing operation in the obstacle detection device according to the embodiment;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image processing unit 2R, 2L Infrared camera 4 Speaker 5 Image display apparatus 6 Navigation apparatus 10 Own vehicle S24-S25 Alarm output control means S27-S28 Object storage control means

Claims (2)

In an obstacle detection device that detects an object that affects the host vehicle from an image captured by an infrared camera and issues an alarm to the driver to the driver of the host vehicle.
Object storage control means for storing in the storage means the characteristics and position information of the detected and alarmed object based on the driver's operation;
When the detected feature and position information of the object is the same as the feature and position information of the object stored in the storage means, an alarm output control means for stopping the alarm output for the object is provided. An obstacle detection device characterized by that. The obstacle detection apparatus according to claim 1, wherein the object storage control unit stores a shape of the object as a feature of the object stored in the storage unit.

Images