JP2004106682A - Obstacle detection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an obstacle detection device allowing the driver to accurately recognize the obstacle to be alerted. <P>SOLUTION: An image processing unit, if it determines that there is a possibility of collision against an obstacle (YES in S21), and that the obstacle exists within a proximity determination area (YES in S22), determines if the location information of the obstacle is a registered location stored in the navigation system or not (S24). If the location information is a registered location, it determines if the shape of the obstacle is a registered shape stored in the image processing unit or not (S25). If the shape of the obstacle is not a registered shape, it determines there is a high possibility of a contact against the obstacle, and generates an alert to the driver and displays a highlighted image using an image display device (S26). If the alert is generated to an artificial structure, the shape and location information of the artificial structure are stored in the image processing unit and the navigation system respectively (S28), according to the driver's operation. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an obstacle detection device that detects an object likely to affect the traveling of a host vehicle from an image captured by an imaging unit and issues an alarm.
[0002]
[Prior art]
A conventional obstacle detection device extracts an object such as a pedestrian that may collide with the host vehicle from an image around the host vehicle captured by an imaging unit such as an infrared camera. Provide to the driver of the vehicle. In this device, a portion having a high temperature in an image around the host vehicle taken by a pair of left and right stereo cameras is set as a target object, and a distance to the target object is calculated by obtaining a parallax of the target object. From the moving direction of the object and the position of the object, an object that is likely to affect the traveling of the host vehicle is detected and an alarm is output (for example, see Patent Document 1).
In addition, in an image around the host vehicle captured by an imaging unit such as an infrared camera, there is a type in which a region indicating a high temperature that cannot be a target such as a pedestrian is excluded to prevent erroneous detection of the target. (For example, see Patent Document 2).
[0003]
[Patent Document 1]
JP 2001-6096 A [Patent Document 2]
JP 2001-108758 A
[Problems to be solved by the invention]
However, in the conventional technology, it is possible to prevent an erroneous detection such as detecting an artificial structure exhibiting an extreme temperature as an object by excluding a region indicating a high temperature that cannot be an object such as a pedestrian. Although it is possible, there is a problem that it is difficult to recognize an artificial structure such as a utility pole, a streetlight pole, a vending machine or the like as a non-warning object and remove it from the object, which shows a temperature similar to that of a pedestrian. For this reason, there is a problem that a warning for these artificial structures that are not pedestrians is troublesome for the driver.
[0005]
The present invention has been made in view of the above-described problems, and provides an obstacle detection device capable of causing a driver to accurately recognize an object such as a pedestrian likely to affect the traveling of the vehicle. With the goal.
[0006]
[Means for Solving the Problems]
In order to solve the above problem, the obstacle detection device according to the first aspect of the present invention uses an image captured by an infrared camera (for example, the infrared camera 2R or 2L of the embodiment) to read the vehicle (for example, the vehicle of the embodiment). In an obstacle detection device that detects an object that affects the vehicle 10) and issues a warning to the driver of the host vehicle, the characteristics of the detected object and the characteristics of the detected object are determined based on the operation of the driver. An object storage control unit (for example, steps S27 to S28 in the embodiment) for storing the position information in the storage unit, and an alarm for the object when the object stored in the storage unit is detected again. An alarm output control means for stopping the output (for example, steps S24 to S25 in the embodiment) is provided.
[0007]
The obstacle detection device having the above configuration allows the driver to store the characteristics such as the shape of the detected object and the position information in the storage unit by the operation of the driver, so that the driver can In addition, it is possible to store information of an arbitrary object that is intentionally excluded from the alarm target. When an object whose information is stored in the storage unit is detected again, the alarm output control unit determines that the object is an object that the driver intentionally wants to exclude from the alarm target, and Can be stopped.
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a configuration of an obstacle detection device according to an embodiment of the present invention.
In FIG. 1, reference numeral 1 denotes a vehicle-mounted image processing unit provided with a CPU (Central Processing Unit) for controlling the obstacle detection device according to the present embodiment, and includes two infrared cameras 2R capable of detecting far infrared rays. 2L is connected, and a signal indicating the running state of the vehicle, such as vehicle speed information and turning angle information, of the vehicle equipped with the obstacle detection device of the present embodiment is input. Thereby, the image processing unit 1 detects a pedestrian or an animal in front of the vehicle from the infrared image around the vehicle and the signal indicating the running state of the vehicle, and issues an alarm when it is determined that the possibility of collision is high. Emits.
[0009]
Further, the image processing unit 1 displays a speaker 4 for issuing a warning by voice and images taken by the infrared cameras 2R and 2L, and allows the driver of the vehicle to recognize an object having a high risk of collision. For example, a meter-integrated display integrated with a meter that indicates the running state of the own vehicle by a number, a NAVI Display installed on the console of the own vehicle, and further, information is displayed at a position on the front window that does not obstruct the driver's front view. An image display device 5 including a HUD (Head Up Display) is connected.
Further, a navigation device 6 is connected to the image processing unit 1, and the image processing unit 1 adds the distance information of the target detected by the image processing unit 1 to the position information acquired from the navigation device 6. -It is possible to store new position information in consideration of the direction information.
[0010]
The image processing unit 1 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a CPU (Central Processing Unit) that performs various types of arithmetic processing, (Random Access Memory) used to store data of the ROM, ROM (Read Only Memory) storing programs, tables, maps, and the like executed by the CPU, drive signals of the speaker 4, display signals of the image display device 5, and the like. The output circuit outputs an image captured by the infrared cameras 2R and 2L, and signals indicating the running state of the vehicle, such as vehicle speed information and turning angle information, are converted into digital signals and input to the CPU. It is configured to be.
[0011]
Further, the image processing unit 1 and the navigation device 6 include a non-volatile memory such as a flash memory, and a switch (not shown) for storing information on an object connected to the image processing unit 1. In response to the signal, information on the shape of the object (object) detected in front of the vehicle is stored in the image processing unit 1, and information on the position is stored in a nonvolatile memory such as a flash memory provided in the navigation device 6. be able to.
[0012]
Further, as shown in FIG. 2, the infrared cameras 2R and 2L are arranged at the front of the host vehicle 10 at positions substantially symmetric with respect to the center of the host vehicle 10 in the vehicle width direction. The optical axes of 2R and 2L are fixed so that the optical axes are parallel to each other and the heights from both roads are equal. Note that the infrared cameras 2R and 2L have a characteristic that the higher the temperature of the target, the higher the output signal level (the higher the luminance).
The HUD 7a is provided such that the display screen is displayed at a position on the front window of the host vehicle 10 where the front view of the driver is not obstructed.
Further, the host vehicle 10 is also equipped with a yaw rate sensor 3 for measuring a yaw rate for obtaining turning angle information.
[0013]
Next, the operation of the present embodiment will be described with reference to the drawings.
FIG. 3 is a flowchart illustrating the operation of detecting and warning an object such as a pedestrian in the image processing unit 1 of the obstacle detection device according to the present embodiment.
First, the image processing unit 1 acquires an infrared image which is an output signal of the infrared cameras 2R and 2L (step S1), performs A / D conversion (step S2), and stores a gray scale image in an image memory (step S1). S3). Here, a right image is obtained by the infrared camera 2R, and a left image is obtained by the infrared camera 2L. Further, in the right image and the left image, the horizontal position of the same object on the display screen is shifted, so that the distance (parallax) to the object can be calculated.
[0014]
After the grayscale image is obtained in step S3, the right image obtained by the infrared camera 2R is used as a reference image, and the image signal is binarized, that is, an area brighter than the luminance threshold ITH is set to "1" ( White) and the process of setting the dark area to “0” (black) (step S4).
FIG. 4A shows a grayscale image obtained by the infrared camera 2R. By performing a binarization process on the grayscale image, an image as shown in FIG. 4B is obtained. In FIG. 4B, for example, an object surrounded by a frame from P1 to P4 is an object displayed as white on the display screen (hereinafter, referred to as a “high luminance area”).
When the binarized image data is obtained from the infrared image, a process of converting the binarized image data into run-length data is performed (step S5). The line represented by the run-length data indicates, at a pixel level, an area that has been whitened by binarization, and each has a width of one pixel in the y direction and also has a width of one pixel in the x direction. It has the length of a pixel constituting the run-length data.
[0015]
Next, a process of extracting the target object is performed by labeling the target object from the image data converted to the run-length data (step S6) (step S7). That is, among the lines converted into run-length data, a line having a portion overlapping in the y-direction is regarded as one object, so that, for example, the high-luminance areas P1 to P4 shown in FIG. (Value object).
When the extraction of the object is completed, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed rectangle of the extracted object are calculated (step S8).
[0016]
Here, the area S is obtained by calculating the run length data of the target object of the label A by (x [i], y [i], run [i], A) (i = 0, 1, 2,..., N−1). ), The length (run [i] -1) of the run-length data is calculated by integrating the same object (N run-length data). The coordinates (xc, yc) of the center of gravity G of the object A are represented by the length (run [i] -1) of each run-length data and the coordinates x [i] or y [i] of each run-length data. Are multiplied by each other, and the sum of the multiplications for the same object is calculated by dividing by the area S. Further, the aspect ratio ASPECT is calculated as a ratio Dy / Dx of the length Dy in the vertical direction and the length Dx in the horizontal direction of the circumscribed rectangle of the object.
Since the run length data is indicated by the number of pixels (the number of coordinates) (= run [i]), the actual length needs to be “−1” (subtract 1) (= run [i]). ] -1). In addition, the position of the center of gravity G may be replaced by the position of the center of gravity of a circumscribed rectangle.
[0017]
When the center of gravity, area, and aspect ratio of the circumscribed rectangle of the object can be calculated, next, tracking of the object between times, that is, recognition of the same object in each sampling cycle is performed (step S9). In time tracking, the time at which the time t as the analog quantity is discretized at the sampling cycle is k, and for example, if the objects A and B are extracted at time k, the objects C and D extracted at time (k + 1) The identity with the objects A and B is determined. When it is determined that the objects A and B are the same as the objects C and D, the inter-time tracking is performed by changing the objects C and D to the labels of the objects A and B, respectively.
Further, the position coordinates (of the center of gravity) of each object recognized in this way are stored in the memory as time-series position data, and are used in subsequent arithmetic processing.
[0018]
Note that the processes of steps S4 to S9 described above are performed on the binarized reference image (the right image in the present embodiment).
Next, the turning angle information obtained by time-integrating the vehicle speed information detected by the vehicle speed sensor (not shown) and the yaw rate detected by the yaw rate sensor 3 is read, and the turning angle θr of the host vehicle 10 is calculated (step). S10).
[0019]
On the other hand, in parallel with the processing of steps S9 and S10, in steps S11 to S13, processing of calculating the distance z between the target object and the host vehicle 10 is performed. Since this calculation requires a longer time than steps S9 and S10, it is executed in a cycle longer than steps S9 and S10 (for example, about three times the execution cycle of steps S1 to S10).
First, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (here, the entire area surrounded by the circumscribed rectangle is searched from the right image) Then, a circumscribed rectangle of the search image is referred to as a target frame (step S11).
[0020]
Next, a search area R2 for searching for an image corresponding to the search image R1 (hereinafter referred to as "corresponding image") from the left image is set (step S12), and a correlation operation with the search image R1 is performed within the search area R2. Then, an area having a high correlation with the search image R1 is obtained and set as a corresponding image. Then, a parallax / distance calculation between the search image R1 and the corresponding image is performed to calculate a distance z between the host vehicle 10 and the target (step S13).
[0021]
When the calculation of the turning angle θr in step S10 and the calculation of the distance to the target in step S13 are completed, the turning angle correction for correcting the positional shift on the image due to the turning of the host vehicle 10 is then performed. Perform (Step S14). In the turning angle correction, if the host vehicle 10 turns, for example, to the left by a turning angle θr during the period from time k to (k + 1), the range of the image is shifted by Δx in the x direction on the image obtained by the camera. This is a process for correcting this.
[0022]
When the turning angle correction is completed, the coordinates (x, y) and the distance z in the image are converted into real space coordinates (X, Y, Z) (step S15).
Here, as shown in FIG. 2, the real space coordinates (X, Y, Z) are obtained by setting the middle point of the mounting positions of the infrared cameras 2R, 2L (the position fixed to the host vehicle 10) as the origin O. The coordinates in the image are defined as x in the horizontal direction and y in the vertical direction with the center of the image as the origin.
In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).
Next, when the turning angle correction for the real space coordinates is completed, N (for example, about N = 10) real space position data after the turning angle correction, obtained for the same object during the monitoring period of ΔT, that is, From the time-series data, an approximate straight line LMV corresponding to a relative movement vector between the target object and the host vehicle 10 is obtained.
[0023]
Next, the latest position coordinates P (0) = (X (0), Y (0), Z (0)) and the position coordinates P (N−1) before (N−1) samples (time ΔT before). = (X (N-1), Y (N-1), Z (N-1)) is corrected to a position on the approximate straight line LMV, and the corrected position coordinates Pv (0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)).
As a result, a relative movement vector is obtained as a vector from the position coordinates Pv (N-1) to Pv (0) (step S16).
In this way, the influence of the position detection error is reduced by calculating the approximate straight line that approximates the relative movement trajectory of the target object with respect to the vehicle 10 from the plurality (N) of data within the monitoring period ΔT. As a result, it is possible to more accurately predict the possibility of collision with the object.
[0024]
Further, when the relative movement vector is obtained in step S16, next, an alarm determination process of determining whether there is a possibility that the own vehicle 10 collides with the detected object and outputting an alarm is performed (step S16). S17). The details of the alarm determination process will be described later.
After the end of the alarm determination processing, the process returns to step S1, and the above processing is repeated.
[0025]
The above is the object detection / warning operation in the image processing unit 1 of the obstacle detection device according to the present embodiment. Next, referring to the flowchart shown in FIG. 5, step S17 of the flowchart shown in FIG. The alarm determination process of determining a possibility of collision with an object and outputting an alarm will be described in more detail.
FIG. 5 is a flowchart showing the alarm determination processing operation of the present embodiment.
First, the image processing unit 1 calculates a relative speed of the target object with respect to the host vehicle 10. Then, by dividing the distance between the host vehicle 10 and the target by the relative speed, the time until contact between the host vehicle 10 and the target is calculated, and it is determined whether or not this is equal to or longer than a margin time for avoiding a collision. Is determined. In determining whether or not a collision occurs, a range in the height direction is also defined.
[0026]
In step S21, when it is determined that there is no possibility of collision between the host vehicle 10 and the target (NO in step S21), the warning determination process is terminated without doing anything.
When it is determined in step S21 that there is a possibility of collision between the host vehicle 10 and the target (YES in step S21), the image processing unit 1 determines whether the target exists in the approach determination area. Is determined (step S22). Here, the approach determination area is an area in front of the host vehicle 10 where it is determined that the possibility of collision with the host vehicle 10 is extremely high if the target object continues to exist as it is.
[0027]
If it is determined in step S22 that the object does not exist in the approach determination area (NO in step S22), the object exists in the approach determination area laterally (x-coordinate direction) outside the approach determination area. Then, an approach collision determination is made to determine whether there is a possibility of an approach collision in which the subject advances into the approach determination area and the vehicle 10 collides with the object (step S23).
Then, in step S23, when it is determined that there is no possibility of the ingress collision between the vehicle 10 and the target object (NO in step S23), the warning determination process is terminated without doing anything.
[0028]
On the other hand, when it is determined in step S22 that the target is present in the approach determination area (YES in step S22), it is determined whether or not the position information of the target is a registered point stored in the navigation device 6 (step S24). ).
Also, in step S24, when the position information of the object is the registered point stored in the navigation device 6 (YES in step S24), whether or not the shape of the object is the registered shape stored in the image processing unit 1 Is determined (step S25).
If the shape of the target object is the registered shape stored in the image processing unit 1 in step S25 (YES in step S25), the warning determination process ends without performing any operation.
[0029]
When it is determined in step S23 that there is a possibility that the vehicle 10 and the target collide with each other (YES in step S23), or in step S24, the position information of the target is stored in the navigation device 6. If it is not a registered point (NO in step S24), or if the shape of the object is not the registered shape stored in the image processing unit 1 in step S25 (NO in step S25), the host vehicle 10 and the object It is determined that there is a high possibility of contact with an object, an alarm is issued by voice via the speaker 4, and an image obtained by, for example, the infrared camera 2 </ b> R is output to the image display device 5, and the approaching object is output. The object is displayed as an emphasized image for the driver of the vehicle 10 (step S26).
[0030]
At this time, if the driver of the own vehicle 10 issues a warning or highlighting, the warning is always issued at the same point, such as a power pole, a streetlight pole, a vending machine, or the like, on the roadside where the target object has the same temperature as the pedestrian. When it is recognized that the object is an artificial structure that can be regarded as an object, characteristics such as the shape of the object can be stored in the image processing unit 1 and position information can be stored in the navigation device 6.
That is, the image processing unit 1 operates the switch (not shown) for the driver of the vehicle 10 to store the shape information and the like, which are information of the object connected to the image processing unit 1, and the position information. Then, it is determined whether or not the switch is turned on (pressed) (step S27).
[0031]
If the switch is not turned on in step S27 (NO in step S27), nothing is performed and the alarm determination process ends.
On the other hand, if the switch is turned on in step S27 (YES in step S27), the image processing unit 1 extracts the shape of the target object (artificial structure for which an alarm has been issued) and stores it as a registered shape. At the same time, the location registration is executed by adding the location information acquired from the navigation device 6 to the distance / direction information of the object detected by the image processing unit 1 and storing the information again in the navigation device 6 as a registered location (step). S28). Then, the alarm determination processing ends.
[0032]
In the above-described embodiment, the image processing unit 1 includes the object storage control unit and the alarm output control unit. More specifically, steps S27 to S28 in FIG. 5 correspond to an object storage control unit. Steps S24 to S25 in FIG. 5 correspond to alarm output control means.
[0033]
As described above, the obstacle detection device according to the present embodiment has an artificial structure in which an object detected from images captured by the infrared cameras 2R and 2L is always set as a warning target at the same point, for example. If the object is a driver, the driver operates the image processing unit 1 to store the characteristics such as the shape of the detected target object and the position information in the storage unit by the target object storage control unit. It is possible to store information on any object that is intentionally excluded from the alarm target.
[0034]
When the object whose information is stored in the storage unit is detected again, the image processing unit 1 determines the object as an object that the driver wants to exclude from the alarm target by the alarm output control unit, and The alarm output for the object can be stopped.
Therefore, even for artificial structures that cannot be excluded from objects such as utility poles, streetlight poles, or vending machines that show the same temperature as pedestrians, such as shapes and other characteristics and location information. Is stored and designated as a non-warning target, an effect is obtained that the output of a warning that bothers the driver can be suppressed.
[0035]
Further, in the present embodiment, the position information of the object is obtained by adding new position information (registered location) by adding the distance / direction information of the object detected by the image processing unit 1 to the position information obtainable from the navigation device 6. ) Is accurately obtained by storing the information. However, since the warning is performed when the object is in the vicinity of the vehicle, the position information of the object is obtained from the navigation device 6 when the switch is operated. May be set to a predetermined range based on.
[0036]
【The invention's effect】
As described above, according to the obstacle detection device of the first aspect, it is possible to store the feature and position information of an arbitrary object that the driver wants to intentionally exclude from the alarm target. Then, when the stored object is detected again, the alarm output for the object can be stopped.
Therefore, even for an artificial structure that shows a temperature equivalent to that of a pedestrian and cannot be excluded from the object, explicitly store the characteristics such as its shape and position information and designate it as a non-warning object. Thus, the effect of suppressing the output of an alarm that bothers the driver can be obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of an obstacle detection device according to an embodiment of the present invention.
FIG. 2 is a diagram showing attachment positions of an infrared camera, a sensor, a display, and the like in a vehicle.
FIG. 3 is a flowchart showing an object detection / warning operation in the obstacle detection device of the embodiment.
FIG. 4 is a diagram showing a grayscale image obtained by an infrared camera and its binary image.
FIG. 5 is a flowchart showing an alarm determination processing operation in the obstacle detection device of the embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image processing unit 2R, 2L Infrared camera 4 Speaker 5 Image display device 6 Navigation device 10 Own vehicle S24-S25 Alarm output control means S27-S28 Object storage control means

Claims (1)

From an image captured by an infrared camera, an obstacle detecting device that detects an object affecting the own vehicle and issues an alarm to the driver of the own vehicle to the object.
Based on the operation of the driver, an object storage control unit that stores the detected feature and position information of the object in the storage unit,
An obstacle detection device comprising: an alarm output control unit that stops an alarm output for an object stored in the storage unit when the object is detected again.

Images