JP2001018738A - Apparatus for monitoring ambient condition of vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus for monitoring ambient conditions of a vehicle that can effectively arouse a driver's attention is a collision with an object detected is highly likely, especially during night driving. SOLUTION: When an object is detected ahead of a vehicle and a collision with the object is determined to be highly likely (S41-S44), an alarm in the form of voice is issued and an image of the object obtained by an infrared camera is displayed on the screen of a head-up display (S51, S52) if the position of the object detected is out of the range of irradiation of the headlight of the vehicle.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle for monitoring the periphery of a vehicle to detect an object that may collide with the vehicle and issuing a warning to a driver when the possibility of collision is high. Peripheral monitoring device.

[0002]

2. Description of the Related Art An obstacle existing in front of a vehicle is detected,
When the danger of the obstacle is high, a danger information warning device has been conventionally known in which a head-up display displays the obstacle in a red frame so as to reliably inform the driver of the danger. (Japanese Patent Laid-Open No. 6-215)
No. 300).

[0003]

However, in the above-mentioned conventional apparatus, the display position of the red frame is displayed so that the detected real image of the obstacle and the displayed red frame are superimposed on the front window. Since calculations are performed to control the display of the head-up display, the display control becomes complicated, and at night, the actual image of a distant obstacle cannot be seen, so that the effect as an alarm is not effective. There is a problem that is enough.

The present invention has been made in view of this point. In particular, when the possibility of collision with an object detected during traveling at night is high, it is possible to effectively draw the driver's attention. It is an object of the present invention to provide a vehicle surroundings monitoring device that can be used.

[0005]

According to a first aspect of the present invention, there is provided an object detecting means for detecting a position of an object existing around a vehicle, and a detecting means for detecting the position of the detected object. Determining means for determining the possibility of collision with the vehicle, and when the determination means determines that the possibility of collision is high,
In a vehicle periphery monitoring device provided with alarm means for issuing an alarm to a driver, an image pickup means capable of detecting infrared rays, and an image display means arranged at a position where the driver can see and displaying an image. The warning means includes: when the determination means determines that the possibility of collision is high, the position of the object detected by the position detection means is out of an irradiation range of headlights of the vehicle. At this time, an alarm by voice is issued and an image of the object obtained by the imaging means is displayed on the image display means.

According to this configuration, for an object that is determined to have a high possibility of collision and is out of the irradiation range of the headlight, an image of the object is displayed on the image display means together with an alarm sound. The driver's attention can be more effectively called as compared with the sound-only warning.

[0007] Even if the detected object is out of the irradiation range of the headlight, the alarm means is provided by the image display means when it is predicted that the detected object enters the irradiation range within a predetermined time. It is desirable not to display.
Here, the “predetermined time” is the sum of the line-of-sight movement time required to move the driver's line of sight toward the front outside the vehicle to the image display means and to move the driver back outside the vehicle again, and the recognition time required to recognize the target object. Time. According to this configuration, the display can be performed only when the effect of displaying the target object image on the image display means can be expected.

[0008]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of a vehicle periphery monitoring device according to one embodiment of the present invention. This device includes two infrared cameras 1R and 1L as imaging means capable of detecting far infrared rays, and A yaw rate sensor 5 for detecting a yaw rate and a traveling speed (vehicle speed) of the vehicle
A vehicle speed sensor 6 for detecting a VCAR, a brake sensor 7 for detecting an operation amount of a brake, and an object such as an animal in front of the vehicle are detected based on image data obtained by the cameras 1R and 1L, and a collision is detected. The image processing unit 2 that issues an alarm when the possibility of collision is high, the speaker 3 that issues an alarm by voice, the image obtained by the camera 1R or 1L, and the driving of an object with a high possibility of collision And a head-up display (hereinafter, referred to as “HUD”) 4 for allowing a person to recognize.

As shown in FIG. 2, the cameras 1R and 1L are arranged in front of the vehicle 10 at positions substantially symmetrical with respect to the central axis of the vehicle 10 in the lateral direction.
The optical axes of L are fixed so that they are parallel to each other and their heights from the road surface are equal. Infrared camera 1R,
1L has the characteristic that the higher the temperature of the object, the higher the output signal level (the higher the luminance).

The image processing unit 2 includes an A / D conversion circuit for converting an input analog signal into a digital signal, an image memory for storing digitized image signals, a CPU (Central Processing Unit) for performing various types of arithmetic processing, and a CPU for performing arithmetic operations. RAM used to store intermediate data (Ra
ROM (Read Only Memory) for storing programs, tables, maps, etc., executed by the CPU.
mory), an output circuit for outputting a drive signal for the speaker 3, a display signal for the HUD 4, and the like.
1L and the output signals of the sensors 5 to 7 are converted into digital signals and input to the CPU. As shown in FIG. 2, the HUD 4 is provided such that a screen 4a is displayed at a position in front of the driver in a front window of the vehicle 10.

FIG. 3 is a flowchart showing the procedure of processing in the image processing unit 2. First, the camera 1R, 1
The output signal of L is A / D converted and stored in the image memory (steps S11, S12, S13). The image stored in the image memory is a gray scale image including luminance information. FIGS. 5A and 5B respectively show the camera 1R,
1L (camera 1R)
To obtain a right image and a camera 1L to obtain a left image). The hatched area is an area of intermediate gradation (gray), and the area surrounded by a thick solid line is , An area of the object having a high luminance level (at a high temperature) and displayed on the screen as white (hereinafter, referred to as a “high luminance area”). In the right image and the left image, the horizontal position of the same object on the screen is shifted, so that the distance to the object can be calculated from the shift (parallax).

In step S14 of FIG. 3, the right image is used as a reference image, and the image signal is binarized, that is, an area brighter than an experimentally determined luminance threshold ITH is set to "1".
(White) and the dark area is set to “0” (black). FIG. 6 shows an image obtained by binarizing the image shown in FIG.
This figure shows that the hatched area is black, and the high-luminance area surrounded by the thick solid line is white.

In the following step S15, a process for converting the binarized image data into run-length data is performed. FIG. 7A is a diagram for explaining this. In FIG. 7A, an area whitened due to binarization is represented by a line L1 at a pixel level.
ＬL8. Each of the lines L1 to L8 has a width of one pixel in the y direction and is actually arranged without a gap in the y direction, but is shown apart for the sake of explanation. Lines L1 to L8 are respectively 2 pixels, 2 pixels, 3 pixels, 8 pixels, 7 pixels, 8 pixels, 8 pixels in the x direction.
Pixels have a length of 8 pixels. The run-length data indicates the lines L1 to L8 by using the coordinates of the start point of each line (the left end point of each line) and the length (the number of pixels) from the start point to the end point (the right end point of each line). It is shown.
For example, the line L3 is (x3, y5), (x4, y5)
And (x5, y5), the run-length data is (x3, y5, 3).

In steps S16 and S17, FIG.
A process of extracting the target object is performed by labeling the target object as shown in FIG. That is, among the lines L1 to L8 converted into run-length data, the lines L1 to L3 having a portion overlapping in the y direction are regarded as one object 1,
The lines L4 to L8 are regarded as one object 2, and object labels 1 and 2 are added to the run-length data. By this processing, for example, the high luminance areas shown in FIG.

In step S18, as shown in FIG. 7C, the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed rectangle indicated by the broken line are calculated. The area S is
The length of the run-length data is calculated by integrating the same object, and the coordinates of the center of gravity G are calculated as x-coordinates of a line that bisects the area S in the x-direction and y of a line that bisects the area S in the y-direction.
Calculated as coordinates, and the aspect ratio APECT is calculated as the ratio Dy / Dx between Dy and Dx shown in FIG. The position of the center of gravity G may be replaced by the position of the center of gravity of a circumscribed rectangle.

In step S19, the object is tracked over time, that is, the same object is recognized in each sampling cycle. The time at which the time t as the analog quantity is discretized at the sampling period is k, and when the objects 1 and 2 are extracted at the time k as shown in FIG.
The identity of the objects 3 and 4 extracted in 1) and the objects 1 and 2 is determined. Specifically, the following identity determination conditions 1)
When 3) are satisfied, it is determined that the objects 1 and 2 are the same as the objects 3 and 4, and the objects 3 and 4 are respectively 1 and 2.
The time tracking is performed by changing the label to "."

1) Object i at time k (= 1, 2)
The coordinates of the position of the center of gravity on the image of (xi (k),
yi (k)) and the object j at time (k + 1)
The coordinates of the position of the center of gravity of the image of (= 3, 4) are represented by (xj (k
+1), yj (k + 1)), | xj (k + 1) −xi (k) | <Δx | yj (k + 1) −yi (k) | <Δy. Here, Δx and Δy are allowable values of the moving amount on the image in the x direction and the y direction, respectively.

2) Object i at time k (= 1, 2)
Is the area on the image Si (k), and the area of the object j (= 3, 4) on the image at time (k + 1) is Sj
When (k + 1), Sj (k + 1) / Si (k) <1 ± ΔS. Here, ΔS is an allowable value of the area change.

3) Object i at time k (= 1, 2)
Let ASPECTi (k) denote the aspect ratio of the circumscribed rectangle of ASPECTj (k + 1), and let ASPECTj (k + 1) denote the aspect ratio of the circumscribed rectangle of the object j (= 3, 4) at time (k + 1). ) <1
± ΔASPECT. However, ΔASPECT is an allowable value of the change in the aspect ratio.

8 (a) and 8 (b), each object has a large size on the image, but objects 1 and 3 satisfy the above-described identity determination condition, and Things 2 and 4
Satisfy the above-described identity determination condition, the objects 3 and 4 are recognized as the objects 1 and 2, respectively. The position coordinates (of the center of gravity) of each object recognized in this way are stored in a memory as time-series position data, and are used in subsequent arithmetic processing. The processes in steps S14 to S19 described above are executed for the binarized reference image (the right image in the present embodiment).

In step S20 of FIG. 3, the vehicle speed sensor 6
Speed VCAR and yaw rate sensor 5 detected by
By reading the detected yaw rate YR and integrating the yaw rate YR with time, the turning angle θr of the host vehicle 10 is obtained.
(See FIG. 14).

On the other hand, in steps S31 to S33, processing for calculating the distance z between the object and the host vehicle 10 is performed in parallel with the processing in steps S19 and S20. Since this operation requires a longer time than steps S19 and S20, a period longer than steps S19 and S20 (for example, steps S11 to S20)
(The cycle is about three times the execution cycle of S20).

In step S31, a reference image (right image)
By selecting one of the objects tracked by the binarized image of (1), the search image R1 (here, the entire region surrounded by the circumscribed rectangle is searched from the right image as shown in FIG. Image). Subsequent step S32
Then, a search area for searching for an image corresponding to the search image (hereinafter referred to as “corresponding image”) is set from the left image, and a correlation operation is executed to extract the corresponding image. Specifically, as shown in FIG. 9B, a search region R2 is set in the left image according to each vertex coordinate of the search image R1, and the height of the correlation with the search image R1 in the search region R2. Brightness difference sum C indicating
(A, b) is calculated by the following equation (1), and the sum C
An area where (a, b) is minimum is extracted as a corresponding image. Note that this correlation operation is performed using a grayscale image instead of a binary image. When there is past position data for the same object, a region R2a (shown by a broken line in FIG. 9B) smaller than the search region R2 is set as a search region based on the position data.

(Equation 1)

Here, IR (m, n) is the luminance value at the position of the coordinates (m, n) in the search image R1 shown in FIG. 10, and IL (a + m-M, b + n-N) is the search value. The luminance value at the position of the coordinates (m, n) in the local region R3 having the same shape as the search image R1 with the coordinates (a, b) in the region as a base point. The position of the corresponding image is specified by changing the coordinates (a, b) of the base point and obtaining the position where the luminance difference total value C (a, b) is minimized.

As shown in FIG. 11, the search image R1 and the corresponding image R4 corresponding to the target object are extracted by the processing in step S32, so that in step S33, the center of gravity of the search image R1 and the center of the image are extracted. Distance d from line LCTR
R (the number of pixels) and the distance dL (the number of pixels) between the center of gravity of the corresponding image R4 and the image center line LCTR are obtained, and the following equation (2) is obtained.
To calculate the distance z between the host vehicle 10 and the target object.

(Equation 2)

Here, B is the base line length, that is, the horizontal direction (x) between the center position of the image sensor 11R of the camera 1R and the center position of the image sensor 11L of the camera 1L as shown in FIG.
Direction) (the distance between the optical axes of both cameras), F is the lens 1
The focal lengths of 2R and 12L and p are the image pickup devices 11R and 11R.
This is the pixel interval within L, and Δd (= dR + dL) is the amount of parallax.

In step S21, the coordinates (x,
y) and the distance z calculated by the equation (2) are expressed by the following equation (3).
To convert to real space coordinates (X, Y, Z). Here, the real space coordinates (X, Y, Z) are, as shown in FIG. 13A, the origin O, which is the position of the midpoint of the mounting position of the cameras 1R and 1L (the position fixed to the host vehicle 10). The coordinates in the image are defined as x in the horizontal direction and y in the vertical direction with the center of the image as the origin, as shown in FIG.

(Equation 3)

Here, (xc, yc) represents the coordinates (x, y) on the right image with the real space origin O based on the relative position relationship between the mounting position of the camera 1R and the real space origin O. This is converted into coordinates in a virtual image in which the center of the image is matched. F is a ratio between the focal length F and the pixel interval p.

In step S22, a turning angle correction for correcting a positional shift on the image due to the turning of the vehicle 10 is performed. As shown in FIG. 14, (k +
During the period up to 1), the own vehicle is turned to the left, for example, by a turning angle θr
Just turning around, on the image obtained by the camera,
Since it is shifted in the x direction by Δx as shown in FIG. 15, this is a process for correcting this. Specifically, by applying the real space coordinates (X, Y, Z) to the following equation (4), the corrected coordinates (Xr,
Yr, Zr) is calculated. The calculated real space position data (Xr, Yr, Zr) is stored in the memory in association with each object. In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).

(Equation 4)

In step S23, as shown in FIG. 16, for the same object, N real space position data (for example, about N = 10) obtained during the period of ΔT after turning angle correction, ie, time series From the data, the object and the vehicle 10
An approximate straight line LMV corresponding to the relative movement vector is obtained. Specifically, assuming that a direction vector L = (lx, ly, lz) (| L | = 1) indicating the direction of the approximate straight line LMV, a straight line represented by the following equation (5) is obtained.

(Equation 5)

Here, u is a parameter having an arbitrary value, and Xav, Yav, and Zav are the average value of the X coordinate, the average value of the Y coordinate, and the average value of the Z coordinate of the real space position data sequence, respectively. is there. The equation (5) becomes the following equation (5a) if the parameter u is eliminated. (X-Xav) / lx = (Y-Yav) / ly = (Z-Zav) / lz (5a)

FIG. 16 is a diagram for explaining the approximate straight line LMV, in which P (0), P (1), P (2),
, P (N-2) and P (N-1) indicate the time series data after the turning angle correction, and the approximate straight line LMV indicates the average position coordinates Pav (= (Xav, Yav, Za) of the time series data.
v)) is obtained as a straight line that minimizes the average value of the square of the distance from each data point. Here, the numerical value in parentheses attached to P indicating the coordinates of each data point indicates that the larger the value is, the more past the data is. For example, P
(0) indicates the latest position coordinates, P (1) indicates the position coordinates one sample cycle ago, and P (2) indicates the position coordinates two sample cycles ago. D (j), X (j), Y in the following description
The same applies to (j), Z (j), and the like.

More specifically, a vector D (j) = (DX (j), DY (j), DZ) from the average position coordinates Pav to the coordinates P (0) to P (N-1) of each data point.
(J)) = (X (j) -Xav, Y (j) -Yav, Z
The inner product s of (j) -Zav) and the direction vector L is calculated by the following equation (6), and the direction vector L = (lx, ly, lz) that maximizes the variance of the inner product s is obtained. s = lx.DX (j) + ly.DY (j) + lz.DZ (j) (6)

The variance-covariance matrix V of the coordinates of each data point is
Since the eigenvalue σ of this matrix corresponds to the variance of the inner product s, the directional vector L corresponding to the largest eigenvalue among the three eigenvalues calculated from this matrix is the direction vector L to be obtained. In order to calculate the eigenvalues and eigenvectors from the matrix of Expression (7), a method known as the Jacobi method (for example, shown in “Numerical Calculation Handbook” (Ohm)) is used.

(Equation 6)

Next, the latest position coordinates P (0) = (X
(0), Y (0), Z (0)) and the position coordinates P (N−1) = (X (N−) before (N−1) samples (before time ΔT).
1), Y (N-1), Z (N-1)) are approximated to the straight line LMV
Correct to the upper position. Specifically, Z in the above equation (5a)
By applying the coordinates Z (0) and Z (N-1), that is, by the following equation (8), the corrected position coordinates Pv
(0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1),
Zv (N-1)).

(Equation 7)

The position coordinates Pv (N−
A relative movement vector is obtained as a vector from 1) toward Pv (0). In this manner, the influence of the position detection error is reduced by calculating the approximate straight line that approximates the relative movement trajectory of the target object with respect to the own vehicle 10 from the plurality (N) of data within the monitoring period ΔT. As a result, it is possible to more accurately predict the possibility of collision with the object.

Returning to FIG. 3, in step S24, the possibility of collision with the detected object is determined, and an alarm determination process (FIG. 4 (a)) for issuing an alarm when the possibility is high is executed. In step S41, the relative velocity Vs in the Z direction is calculated by the following equation (9), and the following equations (10) and (11) are calculated.
Is satisfied, it is determined that there is a possibility of a collision, and the process proceeds to step S42. If Expression (10) and / or Expression (11) is not satisfied, the process ends. Vs = (Zv (N−1) −Zv (0)) / ΔT (9) Zv (0) / Vs ≦ T (10) | Yv (0) | ≦ H (11)

Here, Zv (0) is added to indicate the latest distance detection value (v is data after correction by the approximate straight line LMV, but the Z coordinate is the same value as before correction. And Zv (N−1) is the distance detection value before the time ΔT. T is a margin time, which is intended to determine the possibility of a collision by the time T before the predicted collision time, and is set to, for example, about 2 to 5 seconds.
H is a predetermined height that defines a range in the Y direction, that is, the height direction, and is set to, for example, about twice the height of the host vehicle 10. FIG. 17 shows the relationship of the equation (10), and the detected relative velocity Vs and the distance Zv (0)
The coordinates corresponding to and are in the hatched area,
When | Yv (0) | ≦ H, the determination in step S42 and subsequent steps is performed.

FIG. 18 shows an area that can be monitored by the cameras 1R and 1L by an outer triangular area AR0 indicated by a thick solid line, and furthermore, an area AR1 in the area AR0 closer to the vehicle 10 than Z1 = Vs × T. , AR2, and AR3 are defined as alarm determination areas. Here, the area AR1 is an area corresponding to a range obtained by adding a margin β (for example, about 50 to 100 cm) to both sides of the vehicle width α of the host vehicle 10, in other words, the host vehicle 1
This is an area having a width of (α / 2 + β) on both sides of the center axis in the horizontal direction of 0, and if the object continues to exist as it is, the possibility of collision is extremely high, and is therefore called an approach determination area. The areas AR2 and AR3 are areas where the absolute value of the X coordinate is larger than the approach determination area (outside the approach determination area in the horizontal direction).
The target in this area is referred to as an intrusion determination area because an intrusion collision determination, which will be described later, is performed. These regions have a predetermined height H in the Y direction as shown in the above equation (11).

The answer to step S41 is affirmative (YES).
Is when the object is present in any of the approach determination area AR1 and the intrusion determination areas AR2 and AR3. In a succeeding step S42, the object moves to the approach determination area A.
It is determined whether or not it is within R1, and this answer is affirmative (YES).
If, the process immediately proceeds to step S44, while if negative (NO), an intrusion collision determination is made in step S43. Specifically, the latest x coordinate xc on the image
(0) (c is added to indicate that the coordinates have been corrected so that the center position of the image matches the real space origin O as described above) and the x coordinate xc (N −
It is determined whether or not the difference from 1) satisfies the following equation (12).
If the conditions are satisfied, it is determined that the possibility of collision is high.

(Equation 8)

As shown in FIG. 19, the animal 20 traveling from a direction substantially 90 ° to the traveling direction of the vehicle 10
Xv (N-1) / Zv (N-1) = Xv
When (0) / Zr (0), in other words, the ratio Vp / Vs = Xr (N−1) / of the animal speed Vp and the relative speed Vs.
When Zr (N-1), the azimuth θd at which the vehicle 10 sees the animal 20 is constant, and the possibility of collision is high. Equation (12) determines this possibility in consideration of the vehicle width α of the host vehicle 10. Hereinafter, with reference to FIG.
The derivation method of 2) will be described.

A straight line passing through the latest position coordinates of the object 20 and the position coordinates before the time ΔT, that is, the approximate straight lines LMV and XY
Assuming that an X coordinate of an intersection with a plane (a plane including the X axis and the Y axis, that is, a plane including the line (X axis) corresponding to the tip of the vehicle 10 and perpendicular to the traveling direction of the vehicle 10) is XCL, Width α
Is given by the following equation (13). −α / 2 ≦ XCL ≦ α / 2 (13) On the other hand, a straight line obtained by projecting the approximate straight line LMV on the XZ plane is given by the following equation (14).

(Equation 9) When X = 0 and X = XCL are substituted into this equation to obtain XCL, the following equation (15) is obtained.

(Equation 10)

Since the real space coordinates X and the coordinates xc on the image have the relationship shown in the above equation (3), Xv (0) = xc (0) × Zv (0) / f (16) Xv (N−1) = xc (N−1) × Zv (N−1) / f (17) When these are applied to the equation (15), the intersection X coordinate XCL is given by the following equation (18). Can be This is given by equation (1)
By substituting into 3) and rearranging, the condition of equation (12) is obtained.

[Equation 11]

Returning to FIG. 4A, when it is determined in step S43 that the possibility of collision is high, the process proceeds to step S44, and when it is determined that the possibility of collision is low, the process ends. In step S44, an alarm output determination, that is, a determination as to whether or not to output an alarm is performed as follows. First, it is determined from the output of the brake sensor 7 whether or not the driver of the vehicle 10 is performing a brake operation. If the driver has not performed the brake operation, the process immediately proceeds to step S45 to execute an alarm process. When a brake operation is being performed, the acceleration Gs (the deceleration direction is assumed to be positive) generated thereby is calculated,
When the acceleration Gs is equal to or less than the predetermined threshold GTH, the process proceeds to step S45, while when Gx> GTH,
It is determined that the collision is avoided by the brake operation, and the process ends. Thus, when an appropriate brake operation is being performed, an alarm is not issued, and the driver can be prevented from being unnecessarily troublesome.

The predetermined threshold value GTH is determined as in the following equation (19). This is a value corresponding to a condition that the host vehicle 10 stops at a travel distance equal to or less than the distance Zv (0) when the brake acceleration Gs is maintained as it is.

(Equation 12) In step S45, an alarm determination process shown in FIG. In step S51 in the figure, the position PM of the target object after a lapse of a predetermined time TD from the present time is predicted based on the above-described movement vector of the target object, and the position PM of the headlight indicated by hatching in FIG. It is determined whether or not it is within the irradiation range AHL. Here, the predetermined time T
D is the total time of the line-of-sight moving time required to move the driver's line of sight toward the front outside the vehicle to the screen 4a of the HUD 4 and moving the driver's line of sight to the front outside the vehicle again, and the recognition time required to recognize the target object. Also, in the figure, P0 indicates the current time, that is, the latest position of the object 20, and the PM is a predicted position after a predetermined time TD has elapsed. The irradiation range A of the headlight
Since the HL is adjusted to be within a predetermined range, it can be easily determined from the position coordinates of the target whether or not the HL is within the irradiation range.

FIG. 9A shows a case where the light beam does not fall within the irradiation range AHL both at the present time and after the lapse of a predetermined time TD. In this case, the answer to step S51 is negative (NO). Therefore, an audio warning is issued via the speaker 3 and the HUD as shown in FIG.
4 displays an image obtained by the camera 1R on the screen 4
a, and the detected object is highlighted (for example, highlighted in a frame).

When the reaction time of the driver is compared between the case where the alarm is given only by the voice and the case where the voice and the image display are used together, the one using the voice and the image display gives the alarm only by the voice. It has been experimentally confirmed that the reaction time is shortened in some cases, and by using the audio warning and the image display together in this way, the driver can be surely recognized the object with a high possibility of collision. , It is possible to take appropriate countermeasures. The irradiation range AHL of the headlight shown in FIG. 21 is obtained when the low beam is selected.
While the length LBEAM is about 30 m, the images of the infrared cameras 1 R and 1 L can be captured up to about 100 m ahead. The object having a high possibility of being displayed can be displayed by the HUD 4.

As shown in FIG. 21 (b), the current time is outside the irradiation range AHL, but after a predetermined time TD has elapsed,
When entering the irradiation range AHL, the answer to step S51 is affirmative (YES), and the routine proceeds to step S53, where the HU
Only a voice alarm is issued without performing display by D4. In this case, after the lapse of the predetermined time TD, the driver can visually recognize the actual target object image, so that the display by the HUD 4 gives the driver trouble instead.

As described above, if the driver takes appropriate avoidance measures such as operating the brakes and the likelihood of a collision is reduced, the alarm output processing shown in FIG. Alternatively, the warning by any of the image displays is not output. As described above, in the present embodiment, when an object having a high possibility of collision is detected, if the object is out of the irradiation range of the headlight, an infrared image display by the HUD 4 is performed together with a sound alarm. As a result, it is possible to make the driver surely recognize an object having a high possibility of collision and to take an appropriate countermeasure.

In this embodiment, the HUD 4 constitutes an image display means, and the image processing unit 2 constitutes a part of a position detection means, a judgment means and a warning means. More specifically,
Steps S11 to S22 and S31 to S33 in FIG. 3 correspond to the position detecting means, steps S23 and steps S41 to S44 in FIG. 4A correspond to the determining means, and step S45 (FIG. 4B) in FIG. Corresponds to the alarm means.

The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, in this embodiment, a head-up display is used as the image display means, but a liquid crystal display device of a navigation system may be used. However, the head-up display is suitable for raising the driver's attention because the amount of movement of the driver's line of sight is small. The method of detecting an object having a possibility of collision is not limited to the method described above, and for example, JP-A-6-215300 and JP-A-9-2264.
For example, a technique disclosed in Japanese Patent Publication No. 90-90 may be used.

[0052]

As described above in detail, according to the present invention, an object which is determined to have a high possibility of collision and is out of the irradiation range of the headlight is displayed on the image display means together with an alarm sound. Since the image is displayed, the driver's attention can be more effectively raised as compared with the case of the sound-only warning.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a peripheral monitoring device according to an embodiment of the present invention.

FIG. 2 is a view for explaining a mounting position of the camera shown in FIG. 1;

FIG. 3 is a flowchart illustrating a procedure of processing in the image processing unit of FIG. 1;

FIG. 4 is a flowchart showing details of an alarm determination process in FIG. 3;

FIG. 5 is a diagram showing a gray scale image obtained by an infrared camera with hatching applied to a halftone portion;

FIG. 6 is a diagram showing a black area with hatching to explain an image obtained by binarizing a grayscale image.

FIG. 7 is a diagram for explaining a conversion process to run-length data and labeling.

FIG. 8 is a diagram for explaining time tracking of an object.

FIG. 9 is a diagram for explaining a search image in a right image and a search area set in a left image.

FIG. 10 is a diagram for explaining a correlation calculation process for a search area.

FIG. 11 is a diagram illustrating a method of calculating parallax.

FIG. 12 is a diagram for explaining a method of calculating a distance from parallax.

FIG. 13 is a diagram illustrating a coordinate system according to the present embodiment.

FIG. 14 is a diagram for explaining turning angle correction;

FIG. 15 is a diagram showing a displacement of an object position on an image caused by turning of the vehicle.

FIG. 16 is a diagram for explaining a calculation method of a relative movement vector.

FIG. 17 is a diagram for describing conditions for making an alarm determination.

FIG. 18 is a diagram for describing an area division in front of a vehicle.

FIG. 19 is a diagram illustrating a case where a collision is likely to occur.

FIG. 20 is a diagram for explaining a method of determining an intrusion alarm according to the width of a vehicle.

FIG. 21 is a diagram for explaining a positional relationship between an irradiation range of a headlight and an object.

FIG. 22 is a diagram for explaining a display on a head-up display.

[Explanation of symbols]

1R, 1L infrared camera (imaging means) 2 image processing unit (position detecting means, determining means, alarm means) 3 speaker (alarm means) 4 head-up display (image displaying means) 5 yaw rate sensor 6 vehicle speed sensor 7 brake sensor

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat ゛ (Reference) G08B 21/00 G08B 23/00 510D 23/00 510 520A 520 H04N 7/18 J H04N 7/18 G06F 15 / 62 380 (72) Inventor Shinji Nagaoka 1-4-1, Chuo, Wako-shi, Saitama Prefecture Inside Honda R & D Co., Ltd. (72) Inventor Masato Watanabe 1-4-1, Chuo, Wako-shi, Saitama Honda Co., Ltd. F-term in laboratory (reference)

Claims (1)

[Claims] 1. A position detecting means for detecting a position of an object existing around a vehicle, a judging means for judging a possibility of collision between the object and the vehicle, and a possibility of collision by the judging means. Is determined to be high, a vehicle surroundings monitoring device provided with an alarm unit for issuing an alarm to a driver, wherein an imaging unit capable of detecting infrared rays, and a position which can be viewed by the driver, Image display means for displaying the position of the object detected by the position detection means when the possibility of collision is determined to be high by the determination means, When the vehicle is out of the irradiation range of the headlight, an alarm is issued by sound and an image of the object obtained by the imaging unit is displayed on the image display unit. Location.