JP3949628B2 - Vehicle periphery monitoring device - Google Patents

Description

  In the present invention, a collision with a large animal such as a deer or a bear affects the vehicle traveling. Therefore, in order to avoid the collision, there is a possibility of colliding with the vehicle from an image obtained by an imaging means mounted on the vehicle. The present invention relates to a periphery monitoring device that monitors the periphery of the vehicle in order to detect an object.

Two CCD cameras are mounted on the vehicle, and the distance between the object and the vehicle is detected based on the image shift obtained from the two cameras, that is, parallax, and a pedestrian 30 to 60 meters ahead of the vehicle is detected. Such a crossing object detection device has been known (Patent Document 1).
JP-A-9-226490

  However, since the conventional apparatus determines the possibility of collision using the optical flow direction of the object detected based on the image obtained by the camera as it is, the relative distance between the host vehicle and the object is determined. Depending on the vehicle speed, there is a problem that the determination accuracy is lowered. That is, for example, when the vehicle speed is higher than the speed of the moving object, the optical flow on the image corresponding to the moving object moving toward the center of the road is actually regarded as a vector toward the outside of the road. In some cases, erroneous determination occurs.

  The present invention has been made paying attention to this point, and more accurately detects the movement of an object existing around the vehicle to determine the possibility of a collision, and more appropriately warns the driver. An object of the present invention is to provide a vehicle periphery monitoring device capable of performing the above.

In order to achieve the above object, the invention described in claim 1 is directed to a vehicle periphery monitoring device that detects an object existing around a vehicle from an image obtained by an image capturing unit mounted on the vehicle. Relative position detection means for detecting the relative position of the object with respect to the vehicle from the obtained image as position data, and actuality of the object based on position data about the object detected by the relative position detection means. A real space position calculating means for calculating a position in space; a first distance (Zv (N−1)) to the object at a first sampling time calculated by the real space position calculating means; A second distance (Zv (0)) to the object at a second sampling time after a lapse of a certain time (ΔT) from the first sampling time, and calculating the second distance A determination threshold value is obtained by multiplying a value obtained by subtracting the reciprocal of the first distance (1 / Zv (N−1)) from the reciprocal of the separation (1 / Zv (0)) by a predetermined value (α · f / 2). When the horizontal movement distance (| xc (0) -xc (N-1) |) of the object on the image is equal to or less than the determination threshold value, the object collides with the vehicle. And determining means for determining that there is a possibility of the occurrence.

  The imaging means is preferably two infrared cameras capable of detecting infrared rays. As a result, it is possible to easily extract animals, traveling vehicles, and the like when traveling at night, which is difficult for the driver to check.

  The imaging means is two television cameras that detect infrared or visible light, and based on the position of the object image included in the output image of one camera, the output image of the other camera corresponding to the object image A search area for searching for an object image included in the image, and performing a correlation operation in the search area to identify the corresponding object image, and from the vehicle from the parallax between the object image and the corresponding object image Calculate the distance to the object. The correlation calculation is preferably performed using a grayscale signal including intermediate gradation information. By using a grayscale signal instead of a binarized signal, a more accurate correlation calculation is possible, and it is possible to avoid erroneously specifying a corresponding object image.

  Using the signal obtained by binarizing the gray scale signal, the object moving in the output image of the imaging means is tracked. In this case, it is desirable to recognize the tracking target object based on the run-length encoded data. As a result, the amount of data about the object to be tracked can be reduced and the memory capacity can be saved.

Further, it is desirable to determine the identity of the tracking object based on the barycentric position coordinate, the area, and the aspect ratio of the circumscribed rectangle. As a result, the identity of the object can be accurately determined.
When it is determined by the determination means that the possibility of a collision with an object is high, the warning means is provided to issue a warning to the driver, the warning means is a case where the driver is performing a brake operation, When the acceleration by the brake operation is larger than a predetermined threshold value, it is desirable not to issue an alarm. When the driver has already noticed the object and is performing an appropriate brake operation, the driver can be prevented from giving a warning so as not to be annoyed by the driver.

According to the first aspect of the present invention, the first distance to the object at the first sampling time and the second distance to the object at the second sampling time after a predetermined time has elapsed from the first sampling time; Is calculated, and a determination threshold value is calculated by multiplying a value obtained by subtracting the reciprocal number of the first distance from the reciprocal number of the second distance, and the moving distance in the horizontal direction of the object on the image is determined as the determination threshold value. When it is below, it is determined that the object may collide with the host vehicle. The above determination condition is a condition for determining the possibility that an object traveling from a direction of approximately 90 degrees with respect to the traveling direction of the host vehicle collides, and so as to cross the traveling direction of the vehicle. it is possible to determine the possibility of collision with the moving object to be.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram showing a configuration of a vehicle periphery monitoring device according to an embodiment of the present invention. This device detects two infrared cameras 1R and 1L capable of detecting far infrared rays, and a yaw rate of the vehicle. Based on the yaw rate sensor 5, the vehicle speed sensor 6 for detecting the travel speed (vehicle speed) VCAR of the vehicle, the brake sensor 7 for detecting the operation amount of the brake, and the image data obtained by these cameras 1R and 1L. An object such as an animal in front of the vehicle is detected, and an image obtained by an image processing unit 2 that issues an alarm when there is a high possibility of a collision, a speaker 3 that issues an alarm by sound, and a camera 1R or 1L is displayed. In addition, a head-up display (hereinafter referred to as “HUD”) 4 is provided to allow the driver to recognize an object having a high possibility of collision.

  As shown in FIG. 2, the cameras 1R and 1L are arranged in front of the vehicle 10 at almost target positions with respect to the central axis in the lateral direction of the vehicle 10. The optical axes of the two cameras 1R and 1L are They are parallel to each other and fixed so that their height from the road surface is equal. The infrared cameras 1R and 1L have a characteristic that the output signal level increases (the luminance increases) as the temperature of the object increases.

The image processing unit 2 includes an A / D conversion circuit that converts an input analog signal into a digital signal, an image memory that stores a digitized image signal, a CPU (Central Processing Unit) that performs various arithmetic processes, and data that is being calculated by the CPU Output circuit for outputting RAM (Random Access Memory) used for storing CPU, ROM (Read Only Memory) for storing programs and tables executed by CPU, map, etc., driving signal for speaker 3, display signal for HUD4, etc. The output signals of the cameras 1R and 1L and the sensors 5 to 7 are converted into digital signals and input to the CPU.
As shown in FIG. 2, the HUD 4 is provided so that the screen 4 a is displayed at the front position of the driver on the front window of the vehicle 10.

  FIG. 3 is a flowchart showing a processing procedure in the image processing unit 2. First, the output signals of the cameras 1R and 1L are A / D converted and stored in the image memory (steps S11, S12, and S13). The image stored in the image memory is a grayscale image including luminance information. FIGS. 5A and 5B are diagrams for explaining gray scale images obtained by the cameras 1R and 1L (a right image is obtained by the camera 1R and a left image is obtained by the camera 1L), respectively. The hatched area is an intermediate gradation (gray) area, and the area surrounded by a thick solid line has a high luminance level (at a high temperature) and is an area of an object displayed as white on the screen (hereinafter, “ It is referred to as a “high luminance region”. In the right image and the left image, the horizontal positions of the same target object on the screen are shifted and displayed, and the distance to the target object can be calculated from this shift (parallax).

  In step S14 of FIG. 3, the right image is used as a reference image, and binarization of the image signal, that is, an area brighter than the experimentally determined luminance threshold value ITH is set to “1” (white), and a dark area is set to “0”. ”(Black). FIG. 6 shows an image obtained by binarizing the image of FIG. This figure shows that the hatched area is black and the high luminance area surrounded by the thick solid line is white.

  In the subsequent step S15, the binarized image data is converted into run-length data. FIG. 7A is a diagram for explaining this, and in this figure, whitened areas by binarization are shown as lines L1 to L8 at the pixel level. Each of the lines L1 to L8 has a width of one pixel in the y direction and is actually arranged with no gap in the y direction, but is shown separated for the sake of explanation. The lines L1 to L8 have lengths of 2 pixels, 2 pixels, 3 pixels, 8 pixels, 7 pixels, 8 pixels, 8 pixels, and 8 pixels, respectively, in the x direction. The run length data includes the coordinates of the start point of each line (the leftmost point of each line) and the length (number of pixels) from the start point to the end point (the rightmost point of each line). It is shown. For example, since the line L3 includes three pixels (x3, y5), (x4, y5), and (x5, y5), the run length data is (x3, y5, 3).

  In steps S16 and S17, processing for extracting an object is performed by labeling the object as shown in FIG. 7B. That is, of the lines L1 to L8 converted to run length data, the lines L1 to L3 having portions overlapping in the y direction are regarded as one object 1, the lines L4 to L8 are regarded as one object 2, and the run length data The object labels 1 and 2 are added to. By this processing, for example, the high luminance regions shown in FIG. 6 are grasped as the objects 1 to 4, respectively.

  In step S18, as shown in FIG. 7C, the center-of-gravity G of the extracted object, the area S, and the aspect ratio ASPECT of the circumscribed rectangle indicated by the broken line are calculated. The area S is calculated by integrating the lengths of the run length data for the same object, and the coordinates of the center of gravity G are equally divided into the x coordinate of the line that bisects the area S in the x direction and the y direction. The aspect ratio APECT is calculated as a ratio Dy / Dx between Dy and Dx shown in FIG. 7C. Note that the position of the center of gravity G may be substituted by the position of the center of gravity of the circumscribed rectangle.

  In step S19, tracking of the object is performed between times, that is, the same object is recognized for each sampling period. When the time t as an analog quantity is discretized by the sampling period is k, and the objects 1 and 2 are extracted at the time k as shown in FIG. 8A, the object extracted at the time (k + 1) The identity determination between 3 and 4 and the objects 1 and 2 is performed. Specifically, when the following identity determination conditions 1) to 3) are satisfied, it is determined that the objects 1 and 2 and the objects 3 and 4 are the same, and the objects 3 and 4 are set to 1, respectively. By changing the label to 2, tracking between times is performed.

1) The center-of-gravity position coordinates on the image of the object i (= 1, 2) at the time k are (xi (k), yi (k)), respectively, and the object j (= 3, 3) at the time (k + 1). 4) When the center-of-gravity position coordinates on the image are (xj (k + 1), yj (k + 1)),
| Xj (k + 1) −xi (k) | <Δx
| Yj (k + 1) −yi (k) | <Δy
Be. However, Δx and Δy are allowable values of the movement amount on the image in the x direction and the y direction, respectively.

2) The area of the object i (= 1, 2) on the image at time k is Si (k), and the area of the object j (= 3, 4) on the image at time (k + 1) is Sj ( k + 1)
Sj (k + 1) / Si (k) <1 ± ΔS
Be. However, ΔS is an allowable value of area change.

3) The aspect ratio of the circumscribed rectangle of the object i (= 1, 2) at time k is ASPECTTi (k), and the aspect ratio of the circumscribed rectangle of the object j (= 3,4) at time (k + 1) is ASPECTj ( k + 1)
ASPECTj (k + 1) / ASPECTTi (k) <1 ± ΔASPECT
Be. However, ΔASPECT is an allowable value of the aspect ratio change.

8 (a) and 8 (b), each object has a large size on the image, but the objects 1 and 3 satisfy the above-described identity determination condition, and the object 2 and 4 and 4 meet the identity determination condition, the objects 3 and 4 are recognized as the objects 1 and 2, respectively. The position coordinates (center of gravity) of each object recognized in this manner are stored in the memory as time-series position data and used for later calculation processing.
Note that the processing in steps S14 to S19 described above is executed for a binarized reference image (right image in the present embodiment).

  In step S20 in FIG. 3, the vehicle speed VCAR detected by the vehicle speed sensor 6 and the yaw rate YR detected by the yaw rate sensor 5 are read, and the yaw rate YR is integrated over time, whereby the turning angle θr of the host vehicle 10 (see FIG. 14). Is calculated.

  On the other hand, in steps S31 to S33, in parallel with the processes in steps S19 and S20, a process of calculating the object, the host vehicle 10, and the distance z is performed. Since this calculation requires a longer time than steps S19 and S20, it is executed in a longer cycle than steps S19 and S20 (for example, a cycle about three times the execution cycle of steps S11 to S20).

In step S31, by selecting one of the objects tracked by the binarized image of the reference image (right image), the search image R1 (in this case) is selected from the right image as shown in FIG. , The entire region surrounded by the circumscribed rectangle is set as a search image). In the subsequent step S32, a search area for searching for an image corresponding to the search image (hereinafter referred to as “corresponding image”) from the left image is set, and a correlation operation is executed to extract the corresponding image. Specifically, as shown in FIG. 9B, the search area R2 is set in the left image according to the vertex coordinates of the search image R1, and the correlation between the search image R1 and the search image R1 is high. Is calculated by the following equation (1), and a region where the total value C (a, b) is minimum is extracted as a corresponding image. This correlation calculation is performed using a gray scale image instead of a binary image. When there is past position data for the same object, an area R2a (shown by a broken line in FIG. 9B) narrower than the search area R2 is set as a search area based on the position data.

  Here, IR (m, n) is a luminance value at the position of the coordinate (m, n) in the search image R1 shown in FIG. 10, and IL (a + m−M, b + n−N) is in the search region. The luminance value at the position of the coordinates (m, n) in the local region R3 having the same shape as the search image R1, with the coordinates (a, b) as a base point. The position of the corresponding image is specified by determining the position where the luminance difference sum C (a, b) is minimized by changing the coordinates (a, b) of the base point.

Since the search image R1 and the corresponding image R4 corresponding to the object are extracted by the process of step S32 as shown in FIG. 11, in step S33, the position of the center of gravity of the search image R1, the image center line LCTR, Distance dR (number of pixels) and the distance dL (number of pixels) between the center of gravity of the corresponding image R4 and the image center line LCTR are obtained and applied to the following equation (2) to determine the distance between the host vehicle 10 and the object. z is calculated.

  Here, B is the baseline length, that is, the distance in the horizontal direction (x direction) between the center position of the image sensor 11R of the camera 1R and the center position of the image sensor 11L of the camera 1L as shown in FIG. Axis spacing), F is the focal length of the lenses 12R and 12L, p is the pixel spacing in the image sensors 11R and 11L, and Δd (= dR + dL) is the amount of parallax.

In step S21, the coordinates (x, y) in the image and the distance z calculated by the equation (2) are applied to the following equation (3) to convert to real space coordinates (X, Y, Z). Here, as shown in FIG. 13A, the real space coordinates (X, Y, Z) are obtained by setting the position of the middle point (the position fixed to the host vehicle 10) of the camera 1R, 1L as the origin O. The coordinates in the image are defined as x in the horizontal direction and y in the vertical direction with the center of the image as the origin, as shown in FIG.

  Here, (xc, yc) is the coordinate (x, y) on the right image based on the relative position relationship between the mounting position of the camera 1R and the real space origin O and the center of the image. Are converted into coordinates in a virtual image. F is a ratio between the focal length F and the pixel interval p.

In step S22, the turning angle correction for correcting the positional deviation on the image due to the turning of the host vehicle 10 is performed. As shown in FIG. 14, when the host vehicle turns, for example, to the left by the turning angle θr during the period from time k to (k + 1), on the image obtained by the camera, as shown in FIG. This is a process for correcting this. Specifically, correction coordinates (Xr, Yr, Zr) are calculated by applying real space coordinates (X, Y, Z) to the following equation (4). The calculated real space position data (Xr, Yr, Zr) is stored in the memory in association with each object. In the following description, the coordinates after the turning angle correction are displayed as (X, Y, Z).

In step S23, for the same object as shown in FIG. 16, N pieces of real space position data (for example, about N = 10) after turning angle correction obtained during the period of ΔT, that is, time-series data, An approximate straight line LMV corresponding to the relative movement vector between the object and the vehicle 10 is obtained. Specifically, when a direction vector L = (lx, ly, lz) (| L | = 1) indicating the direction of the approximate straight line LMV, a straight line represented by the following formula (5) is obtained.

Here, u is a parameter that takes an arbitrary value, and Xav, Yav, and Zav are the average value of the X coordinate, the average value of the Y coordinate, and the average value of the Z coordinate of the real space position data string, respectively. Equation (5) becomes the following equation (5a) if the parameter u is deleted.
(X-Xav) / lx = (Y-Yav) / ly = (Z-Zav) / lz
... (5a)

  FIG. 16 is a diagram for explaining the approximate straight line LMV, and P (0), P (1), P (2),..., P (N-2), P (N-1) in FIG. The time series data after the turning angle correction is shown, and the approximate straight line LMV passes through the average position coordinates Pav (= (Xav, Yav, Zav)) of this time series data, and the average value of the square of the distance from each data point Is obtained as a straight line that minimizes. Here, the numerical value in () attached to P indicating the coordinates of each data point indicates that the data increases as the value increases. For example, P (0) indicates the latest position coordinates, P (1) indicates the position coordinates one sample period before, and P (2) indicates the position coordinates two sample periods before. The same applies to D (j), X (j), Y (j), Z (j) and the like in the following description.

More specifically, a vector D (j) = (DX (j), DY (j), DZ (j) from the average position coordinate Pav to the coordinates P (0) to P (N−1) of each data point. ) = (X (j) -Xav, Y (j) -Yav, Z (j) -Zav)) and the inner product s of the direction vector L are calculated by the following equation (6), and the variance of the inner product s is maximum. A direction vector L = (lx, ly, lz) is obtained.
s = lx · DX (j) + ly · DY (j) + lz · DZ (j) (6)

The variance-covariance matrix V of the coordinates of each data point is expressed by the following equation (7). Since the eigenvalue σ of this matrix corresponds to the variance of the inner product s, the maximum of the three eigenvalues calculated from this matrix is The eigenvector corresponding to the eigenvalue is the direction vector L to be obtained. In order to calculate eigenvalues and eigenvectors from the matrix of equation (7), a method known as the Jacobian method (for example, shown in “Numerical Calculation Handbook” (Ohm)) is used.

Next, the latest position coordinate P (0) = (X (0), Y (0), Z (0)) and (N−1) position coordinate P (N−1) before the sample (before time ΔT) = (X (N-1), Y (N-1), Z (N-1)) is corrected to a position on the approximate straight line LMV. Specifically, by applying the Z coordinates Z (0) and Z (N−1) to the equation (5a), that is, according to the following equation (8), the corrected position coordinates Pv (0) = (Xv (0), Yv (0), Zv (0)) and Pv (N-1) = (Xv (N-1), Yv (N-1), Zv (N-1)) are obtained.

  A relative movement vector is obtained as a vector from the position coordinates Pv (N−1) calculated by Expression (8) toward Pv (0). In this way, by calculating an approximate straight line that approximates the relative movement locus of the object with respect to the host vehicle 10 from a plurality (N) of data within the monitoring period ΔT, the influence of the position detection error is reduced. Thus, the possibility of collision with the object can be predicted more accurately.

Returning to FIG. 3, in step S <b> 24, the possibility of collision with the detected object is determined, and an alarm determination process (FIG. 4) for issuing an alarm when the possibility is high is executed.
In step S41, the relative speed Vs in the Z direction is calculated by the following equation (9). When the following equations (10) and (11) are satisfied, it is determined that there is a possibility of a collision, and the process proceeds to step S42. If (10) and / or equation (11) is not established, this process is terminated.
Vs = (Zv (N−1) −Zv (0)) / ΔT (9)
Zv (0) / Vs ≦ T (10)
| Yv (0) | ≦ H (11)

Here, Zv (0) is the latest distance detection value (v is attached to indicate that it is data after correction by the approximate straight line LMV, but the Z coordinate is the same value as before correction). Yes, Zv (N−1) is a distance detection value before time ΔT. T is an allowance time and is intended to determine the possibility of a collision by a time T before the predicted collision time, and is set to about 2 to 5 seconds, for example. H is a predetermined height that defines a range in the Y direction, that is, the height direction, and is set to, for example, about twice the vehicle height of the host vehicle 10.
The relationship of equation (10) is illustrated in FIG. 17, where the coordinates corresponding to the detected relative velocity Vs and the distance Zv (0) are in the hatched area, and | Yv (0) | When ≦ H, the determination after step S42 is executed.

  FIG. 18 shows an area that can be monitored by the cameras 1R and 1L as an outer triangular area AR0 indicated by a thick solid line, and further, areas AR1, AR2, closer to the host vehicle 10 than Z1 = Vs × T in the area AR0. AR3 is used as an alarm determination area. Here, the area AR1 is an area corresponding to a range obtained by adding a margin β (for example, about 50 to 100 cm) to both sides of the vehicle width α of the host vehicle 10, in other words, both sides of the lateral central axis of the host vehicle 10. The region having a width of (α / 2 + β) is called an approach determination region because the possibility of a collision is extremely high if the object continues to exist as it is. The areas AR2 and AR3 are areas where the absolute value of the X coordinate is larger than that of the approach determination area (outside in the lateral direction of the approach determination area), and for an object in this area, an intrusion collision determination described later is performed. This is called an intrusion determination area. These regions have a predetermined height H in the Y direction as shown in the equation (11).

The answer to step S41 is affirmative (YES) when the object exists in either the approach determination area AR1 or the intrusion determination areas AR2 and AR3.
In subsequent step S42, it is determined whether or not the object is in the approach determination area AR1, and when the answer is affirmative (YES), the process immediately proceeds to step S44, whereas when the answer is negative (NO), In step S43, an intrusion collision determination is performed. Specifically, the latest x-coordinate xc (0) on the image (c is added to indicate that the correction has been made so that the center position of the image matches the real space origin O as described above. And the difference from the x coordinate xc (N−1) before the time ΔT satisfies the following formula (12), and if so, it is determined that the possibility of a collision is high.

  As shown in FIG. 19, when there is an animal 20 traveling from a direction of approximately 90 ° with respect to the traveling direction of the host vehicle 10, Xv (N−1) / Zv (N−1) = Xv (0) / Zv (0), in other words, when the ratio Vp / Vs = Xv (N−1) / Zv (N−1) between the animal speed Vp and the relative speed Vs, The viewing azimuth angle θd is constant and the possibility of collision is high. Formula (12) determines this possibility in consideration of the vehicle width α of the host vehicle 10. Hereinafter, the method of deriving the equation (12) will be described with reference to FIG.

  A straight line passing through the latest position coordinate of the object 20 and the position coordinate before time ΔT, that is, an approximate straight line LMV and an XY plane (a plane including the X axis and the Y axis, that is, a line corresponding to the tip of the vehicle 10 (X axis ) And the plane perpendicular to the traveling direction of the vehicle 10), the condition for the occurrence of a collision in consideration of the vehicle width α is given by the following equation (13).

この式にＺ＝０，Ｘ＝ＸＣＬを代入してＸＣＬを求めると下記式（１５）のようになる。

-Α / 2 ≦ XCL ≦ α / 2 (13)
On the other hand, a straight line obtained by projecting the approximate straight line LMV onto the XZ plane is given by the following equation (14).

Substituting Z = 0 and X = XCL into this equation to obtain XCL yields the following equation (15).

Further, since the real space coordinate X and the coordinate xc on the image have the relationship shown in the equation (3),
Xv (0) = xc (0) × Zv (0) / f (16)
Xv (N-1) = xc (N-1) * Zv (N-1) / f (17)
When these are applied to the equation (15), the intersection X coordinate XCL is given by the following equation (18). By substituting this into equation (13) and rearranging, the condition of equation (12) is obtained.

Returning to FIG. 4, when it is determined in step S43 that the possibility of a collision is high, the process proceeds to step S44, and when it is determined that the collision is low, this process is terminated.
In step S44, alarm output determination, that is, whether or not alarm output is to be performed is performed as follows. First, it is determined from the output of the brake sensor 7 whether or not the driver of the host vehicle 10 is performing a brake operation. If the brake operation is not being performed, the process immediately proceeds to step S45 to output an alarm. If the brake operation is being performed, the acceleration Gs generated (deceleration direction is positive) is calculated. If the acceleration Gs is less than or equal to the predetermined threshold GTH, the process proceeds to step S45 while Gx> GTH When it is, it determines with a collision being avoided by brake operation, and complete | finishes this process. As a result, when an appropriate brake operation is performed, an alarm is not issued, and the driver is not bothered excessively.

ステップＳ４５では、スピーカ３を介して音声による警報を発するとともに、図２１に示すようにＨＵＤ４により例えばカメラ１Ｒにより得られる画像を画面４ａに表示し、接近してくる対象物を強調表示する（例えば枠で囲んで強調する）する。図２１（ａ）はＨＵＤの画面４ａの表示がない状態を示し、同図（ｂ）が画面４ａが表示された状態を示す。このようにすることにより、衝突の可能性の高い対象物を運転者が確実に認識することができる。 The predetermined threshold GTH is determined as in the following formula (19). This is a value corresponding to a condition in which the host vehicle 10 stops at a travel distance equal to or less than the distance Zv (0) when the brake acceleration Gs is maintained as it is.

In step S45, an audio alarm is issued via the speaker 3, and an image obtained by the camera 1R, for example, is displayed on the screen 4a by the HUD 4 as shown in FIG. 21, and an approaching object is highlighted (for example, (Enhance with a frame). FIG. 21A shows a state where the HUD screen 4a is not displayed, and FIG. 21B shows a state where the screen 4a is displayed. In this way, the driver can reliably recognize an object having a high possibility of collision.

As described above, in the present embodiment, the position in the real space of the target object is calculated based on a plurality of time-series position data for the same target object, and the movement vector is calculated from the real space position. Since the possibility of collision between the target object and the host vehicle 10 is determined based on the movement vector calculated in the above manner, the erroneous determination as in the conventional apparatus does not occur, and the determination accuracy of the collision possibility Can be improved.
Also, an approximate straight line LMV that approximates the relative movement trajectory of the object is obtained, the position coordinates are corrected so that the detection position is located on this approximate line, and the movement vector is obtained from the corrected position coordinates. It is possible to reduce the influence of the error in the detection position and perform a more accurate collision possibility determination.
Further, since the collision determination is performed in consideration of the vehicle width α, it is possible to accurately determine the possibility of the collision and prevent the use of an unnecessary warning.

  In the present embodiment, the image processing unit 2 constitutes a part of relative position detection means, real space position calculation means, determination means, and alarm means. More specifically, steps S14 to S19 in FIG. 3 correspond to relative position detection means, steps S20 to S23 correspond to real space position calculation means, and steps S41 to S44 in FIG. 4 correspond to determination means. Step S45, the speaker 3 and the HUD 4 in FIG.

The present invention is not limited to the embodiment described above, and various modifications can be made. For example, in this embodiment, an infrared camera is used as the imaging means. However, for example, as shown in Japanese Patent Laid-Open No. 9-226490, a television camera that can detect only normal visible light may be used. However, by using an infrared camera, it is possible to simplify the extraction process of animals or running vehicles, and it can be realized even if the computing capability of the computing device is relatively low.
Moreover, although the example which monitors the front of a vehicle was shown in embodiment mentioned above, you may make it monitor any directions, such as the back of a vehicle.

It is a block diagram which shows the structure of the periphery monitoring apparatus concerning one Embodiment of this invention. It is a figure for demonstrating the attachment position of the camera shown in FIG. 3 is a flowchart showing a processing procedure in the image processing unit of FIG. 1. It is a flowchart which shows the warning determination process of FIG. 3 in detail. In order to explain a gray scale image obtained by an infrared camera, it is a diagram showing hatching in an intermediate gradation part. It is a figure which shows the black area | region which hatched, in order to demonstrate the image which binarized the gray scale image. It is a figure for demonstrating the conversion process and labeling to run length data. It is a figure for demonstrating tracking between the time of a target object. It is a figure for demonstrating the search image in a right image, and the search area | region set to a left image. It is a figure for demonstrating the correlation calculation process which made the search area | region object. It is a figure for demonstrating the calculation method of a parallax. It is a figure for demonstrating the method of calculating distance from parallax. It is a figure which shows the coordinate system in this embodiment. It is a figure for demonstrating a turning angle correction | amendment. It is a figure which shows the shift | offset | difference of the target object position on the image which generate | occur | produces by turning of a vehicle. It is a figure for demonstrating the calculation method of a relative movement vector. It is a figure for demonstrating the conditions which perform warning determination. It is a figure for demonstrating the area division ahead of a vehicle. It is a figure for demonstrating the case where a collision is easy to generate | occur | produce. It is a figure for demonstrating the method of the intrusion warning determination according to the width | variety of a vehicle. It is a figure for demonstrating the display on a head-up display.

Explanation of symbols

1R, 1L infrared camera (imaging means)
2 Image processing unit (relative position detection means, real space position calculation means, determination means, alarm means)
3 Speaker (alarm means)
4 Head-up display (alarm means)
5 Yaw rate sensor 6 Vehicle speed sensor 7 Brake sensor

Claims (1)

In the vehicle periphery monitoring device for detecting an object existing around the vehicle from an image obtained by an imaging means mounted on the vehicle,
Relative position detection means for detecting, as position data, a relative position of the object to the vehicle from an image obtained by the imaging means;
Real space position calculating means for calculating the position of the object in real space based on position data about the object detected by the relative position detecting means;
The first distance to the object at the first sampling time calculated by the real space position calculating means and the second distance to the object at the second sampling time after a lapse of a certain time from the first sampling time. And calculating a determination threshold by multiplying a value obtained by subtracting the reciprocal of the first distance from the reciprocal of the second distance, and calculating a horizontal value of the object on the image. A vehicle periphery monitoring device comprising: a determination unit that determines that the object may collide with the vehicle when a moving distance in a direction is equal to or less than the determination threshold.

Images