JP3452268B2 - Rear side monitoring method for vehicles - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video camera mounted on a vehicle such as an automobile, and uses a video image of the rear and side images of the video camera when the vehicle is running. The invention also relates to a vehicle rear side monitoring method for detecting a vehicle approaching from the side and giving a warning to a driver. 2. Description of the Related Art When a vehicle changes lanes while traveling on a road having two or more lanes on one side, if a vehicle running at a higher speed than the own vehicle is overlooked on an adjacent lane to be changed, a large problem occurs. The risk of accidents is great. In addition, when there is a following vehicle traveling in the same lane as the own vehicle, if a sudden brake is applied when approaching suddenly, there is a risk of a collision, and it is necessary to recognize a nearby vehicle. As a conventional example for recognizing a vehicle traveling in an adjacent lane and a following vehicle, Japanese Patent Application Laid-Open No. 1-189289 (vehicle information display device) has been proposed. Japanese Patent Application Laid-Open No. 1-189289 discloses a method of recognizing a vehicle traveling in an adjacent lane and a following vehicle by photographing the rear and side of the vehicle with a camera and displaying the photographed image on a monitor. ing. [0004] As described above, conventionally, the rear and side of a vehicle are photographed by a camera, the photographed images are displayed on a monitor, and a vehicle traveling in an adjacent lane and a following vehicle are conventionally photographed. Is made to recognize. Therefore, there is a possibility that the vehicle displayed on the monitor may be overlooked, and it is immediately determined whether or not the speed of the vehicle traveling in the adjacent lane and the speed of the following vehicle are higher than the speed of the own vehicle only by looking at the monitor. It was difficult. Accordingly, in view of the above-mentioned conventional problems, the present invention uses a rear side image from a video camera installed in a vehicle to automatically determine the degree of danger caused by a vehicle traveling behind or a vehicle traveling in an adjacent lane. It is an object of the present invention to provide a method for monitoring a vehicle rear side which can be determined in a specific manner. Further, the present invention provides a rear side monitoring method for a vehicle in which a rear side image picked up by a video camera can be quickly recognized as an optical flow at the same point every predetermined time as an optical flow. It is intended to be. [0007] In order to achieve the above object, a vehicle rear side monitoring method according to the present invention captures a rear side view from a running own vehicle and precedes a predetermined time phase. In the vehicle rear side monitoring method of detecting the movement of the same point in the two frames of images as an optical flow, and monitoring the relative relationship of the vehicle running in the rear lane or the adjacent lane to the own vehicle, the rear vehicle or determine a vector direction of the optical flow emanating from an infinitely far point in the image appearing for the points on the vehicle traveling the adjacent lane (FOE), determines the risk depending on the magnitude of the vector of the optical flow vehicle This is a side monitoring method for
In the previous image of the two images before and after
To one point that indicates the direction opposite to the traveling direction of the own vehicle.
From the corresponding infinity point to the point of interest
A window that is elongated in the direction of
While moving in the radial direction from the telephoto point, one point of interest is extracted.
Arrow, connecting the points between the previous and subsequent images
It is characterized as a optical flow . According to the above method, attention is paid to the fact that the optical flow increases as the distance to the vehicle behind or in the adjacent lane decreases, and as the relative speed increases. It is necessary to provide a range finder to measure the distance between the vehicle behind and the vehicle traveling in the adjacent lane, because the danger is determined based on the magnitude of the optical flow at a point on the vehicle traveling in the adjacent lane. Is gone. Focusing on the fact that the optical flow is formed in a radial direction from an infinity point corresponding to one point indicating the direction opposite to the traveling direction of the host vehicle, and in the previous image of the two frames, From the point at infinity corresponding to one point indicating the direction opposite to the traveling direction of the running vehicle, an elongated window is set in a radial direction with respect to one point of interest,
While moving the window in the radial direction from the infinity point on the subsequent screen, one point of interest is extracted, and an arrow connecting the points of the previous image and the subsequent image is defined as one optical flow. Therefore, the amount of calculation can be reduced and the processing speed can be increased. Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows the configuration of an apparatus for carrying out the method according to the present invention. In FIG. 1, reference numeral 1 denotes a video camera that captures a background image, 2 denotes an input of an image obtained by the video camera 1, and An arithmetic processing unit for performing the processing, an arithmetic processing unit for performing a risk determination process based on the image processing result of the arithmetic processing unit 2, and an alarm device. FIGS. 2A and 2B are diagrams for explaining changes in the rear-view image obtained by the video camera 1. FIG. 2B shows the situation in which the video camera 1 includes the vehicle shown in FIG.
Indicates an image captured at time t, and (c) indicates an image captured at time t + Δt. Now, assuming that the vehicle is traveling straight on a flat road, for example, focusing on a road sign and a building shown in FIG.
At + Δt, images as shown in (b) and (c) are obtained. By searching for corresponding points in these two images and connecting them, a velocity vector as shown in (d) is obtained. This is the optical flow. When the following vehicle approaches, the direction of the velocity vector of the optical flow shown in FIG. 2D is reversed. Here, these optical flows appear radially from one point called an FOE (Focus of Expansion) in the image. The FOE is called an infinity point or a vanishing point, and corresponds to one point indicating a direction directly opposite to a traveling direction of the own vehicle on an image when the vehicle is traveling straight. As described above, the optical flow required when the host vehicle is traveling is in a radial direction from the FOE. Here, the optical flow emitted from the vehicle traveling in the following or adjacent lane includes information including the position and the relative speed of the following or adjacent vehicle with respect to the own vehicle, the optical flow is longer, and the direction is larger than that of the FOE. If diverging, the risk is considered to be high. Next, the details will be described with reference to FIG. In the optical arrangement shown in the figure, 11 is the lens of the video camera, 12 is the image plane of the video camera, f is the distance from the lens 11 to the image plane 12, P
(X, Y, Z) is an arbitrary point on the following vehicle, p (x,
If y) is a point corresponding to the point P on the image plane 12, then x = f.X / Z (2) from the similarity ratio of the triangle. By transforming this equation and differentiating it over time, X '= (Δx / Δt · Z + x · Z ′) / f (3) Further, since the x-direction component u of the optical flow is u = Δx / Δt (4), using this, Z = (f × X′−xZ ′) / u (5) Become. Here, Z '= the relative speed between the following vehicle or the vehicle traveling in the adjacent lane and the own vehicle = -α (6). Therefore, the above equation (5) gives Z = (f × X ′ + xα). ) / U (7) Therefore, the x-direction component u of the optical flow is as follows: u = (f × X ′ + xα) / Z (8) The same applies to Y. Therefore, from the above equation (8), as Z is smaller, that is, as the distance to the following vehicle or a vehicle traveling in an adjacent lane is smaller, or as α is larger, that is, as the relative speed is larger,
The x component of the optical flow increases. This is the same for the Y direction. Accordingly, the optical flow becomes longer as the distance to the following vehicle or the like becomes smaller and further as the relative speed becomes larger, and the direction of the optical flow becomes FO.
It is considered that the risk for the following vehicle or the adjacent vehicle is relatively greater when the light diverges from E and the length is longer than when the length is shorter. In the present invention, the optical flow is FOE.
The optical flow is obtained at a high speed by utilizing the fact that the optical flow is obtained in a radial direction from. The method will be described below with reference to FIG. FIG. 4 is a diagram showing one embodiment of a method for obtaining an optical flow at high speed. First, a window is set in a radial direction from the FOE for a point of interest in an image at time t (FIG. 4A). Next, in the image at the time t + Δt, the sum of the absolute values of the luminance differences from the window at the time t is obtained while moving the window one point at a time in the radial direction from the FOE. Then, the moving amount of the window when the sum is minimized is obtained as a speed vector of one point of interest (FIG. 4B). The luminance difference is, for each pixel constituting the window, for example, between pixels at corresponding positions indicated by ● in (a) and (b). By repeating the above processing at all points of the image at time t, the optical flow of the entire image can be obtained. Alternatively, a pixel in the window may be scanned to extract a point of interest, and the extracted points may be connected to obtain an optical flow. Next, a method for obtaining the degree of risk will be described. If the direction of the optical flow is the direction toward the FOE, it indicates that the speed of the following vehicle is lower than the speed of the own vehicle and is away from the own vehicle, and conversely, the direction of the optical flow is diverging with respect to the FOE. In some cases, it indicates that the vehicle is approaching. The direction of the optical flow generated by the scenery or the mark on the road surface photographed in the set area is all toward the FOE, and can be easily distinguished from the following vehicle approaching. Therefore, for example, the length of the optical flow diverging from the FOE is weighted, and if the weighted value exceeds a certain threshold value, it is determined to be dangerous. In addition, several levels of thresholds can be set, and the level of risk can be determined. Finally, it is possible to call the driver's attention by sounding an alarm in accordance with the magnitude of the determined degree of danger. It is also possible to display which area is dangerous and how much is on the display. Further, it is also possible to change the type of alarm sound and the like according to the danger level. Further, by issuing a warning signal to other actuators and the like, those actuators can also be operated. Also, the warning in the adjacent lane area can be made to issue an alarm when the driver turns on the turn signal to change lanes. FIG. 5 summarizes the procedure of the image processing according to the method of the present invention described above. First, an image at time t is captured in step S1, and then an image at time t + Δt is captured in step S2.
After that, the FOE is set in step S3. Thereafter, the process proceeds to step S4, in which only the optical flow in the direction diverging from the FOE in the set area is obtained, and in the next step S5, the risk is calculated based on the weighted value. As described above, according to the present invention, it is possible to automatically determine the presence of a vehicle behind or a vehicle running in an adjacent lane or the degree of danger thereof, thereby enabling safe driving of the vehicle. Become. In particular, danger is determined based on the magnitude of the optical flow diverging from the FOE of a point on a vehicle traveling in a rear vehicle or an adjacent lane, and the background is imaged from the own vehicle traveling. It is not necessary to install one or several video cameras and to provide a distance meter for measuring the distance between the vehicle and the rear vehicle, which can be realized at low cost. Further, it is possible to obtain an effect such that an appropriate warning can be issued by knowing the dangerous area and its degree.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram illustrating an example of an apparatus for performing a method according to the present invention. FIG. 2 is a diagram showing a foreground, an image, and an optical flow obtained by the video camera shown in FIG. 1; FIG. 3 is a diagram for explaining how to detect an obstacle or the like by the method of the present invention. FIG. 4 is a diagram for explaining a method of obtaining an optical flow according to the method of the present invention. FIG. 5 is a diagram showing a series of processes of the method of the present invention. [Description of Signs] 1 Video cameras 2 and 3 Arithmetic processing unit 4 Alarm device

──────────────────────────────────────────────────続 き Continuation of the front page (56) References JP-A-2-241855 (JP, A) JP-A-2-218284 (JP, A) JP-A-63-9813 (JP, A) JP-A 5- 120428 (JP, A) JP-A-7-37099 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 7/18 B60R 21/00 621 G08B 21/00 H04N 5/225

Claims (1)

(57) [Claims 1] An image of a rear view is taken from a running own vehicle, and movement of the same point in two frames before and after a predetermined time is detected as an optical flow, In a vehicle rear side monitoring method for monitoring a relative relationship of a vehicle traveling in a rear lane or an adjacent lane with respect to the own vehicle, a point on a vehicle on the vehicle traveling in the rear lane or the adjacent lane is displayed in an image. A vector of an optical flow in a direction diverging from a point at infinity (FOE) is obtained, and the degree of risk is determined based on the magnitude of the vector of the optical flow.
This is a rear side monitoring method for a vehicle, wherein a predetermined period of time 2
In the previous image of the frame images,
Infinity corresponding to one point indicating the direction opposite to the direction of travel of the vehicle
Elongate in a radial direction from a distant point to one point of interest
Set a window and move the window from the point at infinity in a later image
While moving in the radial direction, extract one point of interest and
An arrow connecting one point of the image with the point of the next image
A rear side monitoring method for a vehicle, which is defined as a flow .