JP2000276697A - Periphery monitoring device for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a cost without requiring a means for detecting a white line separately by detecting the horizontal position of its own vehicle in its own lane based on the position of the white line in an image detected by a white line detecting means. SOLUTION: A white line detecting means 31-1 processes an image obtained by an image pickup means 1 to detect a pair of white lines positioned on both sides of its own lane on a road surface on which its own vehicle is traveling. A position detecting means 3a-5 detects the horizontal position of its own vehicle in its own lane based on the position of the white line in the image detected by the means 3a-1. Since the means 3a-5 detects the horizontal position of its own vehicle in its own lane by the position of the white line detected by the means 3a-1 like this. Thus, by appropriating the white line detected by the means 3a-1 to detect an optical flow, the means 8a-5 can detect the horizontal position of its own vehicle in its own vehicle by the position of the white line.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus for monitoring the periphery of a vehicle, and more particularly to an image pickup means mounted on a vehicle for picking up an image of the periphery of the vehicle to obtain an image. A point at which an intersection of a white line detecting means for detecting a pair of white lines located on both sides of the own lane on the road surface on which the vehicle is to travel and an extension of the white line detected by the white line detecting means is defined as a FOE (point at infinity). FOE setting means for setting, an optical flow detecting means for detecting, as an optical flow, a movement amount of the same point in two images obtained before and after a predetermined time by the imaging means using the FOE set by the FOE setting means, Danger judging means for judging danger by detecting another vehicle approaching the own vehicle based on the magnitude of the optical flow of the other vehicle traveling around the vehicle To an apparatus.

[0002]

2. Description of the Related Art Conventionally, as this kind of vehicle periphery monitoring device, one described in Japanese Patent Application Laid-Open No. 7-50769 is known. Hereinafter, a conventional vehicle periphery monitoring device will be described with reference to FIG. FIG. 14 is a diagram for explaining a change in an image on the rear side obtained by the camera. FIG. 14B illustrates an image captured by the camera 1 at time t in a situation including the own vehicle illustrated in FIG. c) is the time t +
Images taken at Δt are shown.

[0003] Now, assuming that the vehicle is traveling straight on a flat road, for example, if attention is paid to a road sign and a building shown in FIG.
At + Δt, images as shown in (b) and (c) are obtained. In these two images, a search is made for corresponding identical points and they are connected to obtain a velocity vector as shown in (d). This is the optical flow.

[0004] Here, these optical flows correspond to FOE (Focus of Expansion) in an image.
It appears radially from one point called. The FOE is called a point at infinity or a vanishing point, and corresponds to one point indicating the direction opposite to the traveling direction of the own lane on the image when the vehicle is traveling straight. As described above, the optical flow required when the host vehicle is traveling is in a radial direction from the FOE. Here, the optical flow emitted from the vehicle traveling in the succeeding or adjacent lane includes information including the position and the relative speed of the vehicle traveling in the adjacent lane with respect to the own lane, and the optical flow is long and its direction is long. It is considered that the degree of danger is higher when diverging than FOE.

In the prior art, the white lane of the own lane on which the host vehicle is traveling on a straight road is detected to identify the own lane running lane area and the adjacent lane area adjacent thereto. By determining the vehicle traveling lane area or the adjacent lane area as the monitoring area, the processing speed for those that do not need to be monitored is reduced and the processing speed is increased. Then, a point at which the detected extension of the white line intersects is set as the FOE point,
The approaching vehicle to the own vehicle is detected by calculating the optical flow radially from the FOE point for the own lane region and the adjacent lane region, and the degree of risk is determined based on the magnitude of the optical flow. Therefore, there is an advantage that the degree of danger can be automatically determined for another vehicle traveling in the adjacent lane, and no special distance meter is required.

[0006]

It is desired that the vehicle be driven with a stable mind and body at all times. However, in recent years, the number of people driving the vehicle has increased due to the development of the road network and an increase in leisure time. Because of this, it is becoming increasingly common to be aware of poor physical condition and lack of sleep and to neglect this and grasp the steering wheel. If such a driving ignoring the physical condition or continuous driving for a long time is performed, the possibility of causing an accident is increased due to accumulation of driver fatigue and a decrease in concentration.

In particular, when driving on a highway, the operation of the steering wheel becomes monotonous, and the possibility that the driver starts to fall asleep increases. Drivers falling asleep can cause lane departures and meandering and cause serious accidents.

[0008] Then, infrared rays are irradiated on the driver's face,
The image of the eye is taken by the camera, and C changes according to the opening and closing of the eye.
There has been proposed a drowsy driving alarm device that detects a change in blinking by performing image processing on a CD signal, detects drowsy driving, and alerts the driver of the drowsiness.

The above-mentioned dozing alarm device directly detects a driver's dozing by monitoring the driver's own behavior by image processing, so that high-speed calculation is required for image processing, which is costly. was there. Therefore, simply adding such a drowsiness alarm device to the above-described vehicle periphery monitoring device increases costs.

In general, a drowsy driving of a driver has a high possibility of causing lane departure and meandering driving. For this reason, it is conceivable to detect the lane departure and the meandering operation and indirectly detect the dozing operation, but the conventional vehicle periphery monitoring device monitors other vehicles but does not monitor the own vehicle. not going.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a vehicle periphery monitoring device capable of detecting a lateral position of a host vehicle in a host vehicle lane by focusing on the above problems.

[0012]

Means for Solving the Problems The invention according to claim 1 for solving the above-mentioned problems is mounted on a vehicle as shown in the block diagram of FIG. An image capturing means 1 for obtaining, a white line detecting means 3a-1 for processing an image obtained by the image capturing means and detecting a pair of white lines located on both sides of the own lane on a road surface on which the own vehicle is to travel; The FOE setting means 3a-2 for setting, as an FOE (point at infinity), a point at which the extension lines obtained by extending the white lines detected by the means intersect with each other, and the imaging means using the FOE set by the FOE setting means. Optical flow detection means 3a for detecting the movement amount of the same point in two images obtained before and after the time as an optical flow
And a danger determining means 3a-4 for detecting another vehicle approaching the own vehicle based on the magnitude of the optical flow of the other vehicle traveling around the own vehicle and determining danger. In the monitoring apparatus, there is provided a position detecting means 3a-5 for detecting a lateral position of the own vehicle in the own lane based on a position in the image of the white line detected by the white line detecting means. It exists in the vehicle periphery monitoring device.

According to the first aspect of the present invention, the position detecting means 3a-5 detects the lateral position of the own vehicle in the own lane based on the position of the white line detected by the white line detecting means 3a-1. The white line detected by the white line detecting means 3a-1 can be used to detect the position, and the position detecting means 3a-5 can detect the lateral position of the own vehicle in the own vehicle based on the position of the white line. No means for detecting a white line is required.

According to a second aspect of the present invention, the position detecting means is configured to detect the position of the white line or an extension of the white line, which changes according to the position of the white line, based on an angle formed by a reference line set on the image. 2. The vehicle periphery monitoring device according to claim 1, wherein a lateral position of the own vehicle in the own lane is detected.

According to the second aspect of the present invention, the angle formed by the white line and the reference line in the direction radially diverging from the FOE can be set even if the FOE position moves up and down with the vertical vibration of the vehicle.
Paying attention to the fact that there is no fluctuation, the position detecting means 3a
-5 detects the lateral position of the own vehicle in the own lane based on the angle between the white line detected by the white line detecting means 3a-1 or the extension thereof and the reference line set on the image, The lateral position can be detected without being affected by vibration.

According to a third aspect of the present invention, the position detecting means determines the position of the self-line based on the position of the intersection of the white line or an extension of the white line, which changes according to the position of the white line, and a reference line set on the image. 2. A vehicle periphery monitoring device according to claim 1, wherein a lateral position of the own vehicle in the lane is detected.

According to the third aspect of the present invention, the position detecting means 3a-5 automatically detects the position of the white line or its extension line which changes according to the position of the white line and the reference line set on the image. Since the lateral position of the host vehicle in the lane is detected, the white line detected by the white line detecting means 3a-1 is diverted to detect the optical flow, and the position detecting means 3a is used.
-5 can detect the position of the own vehicle in the lateral direction in the own vehicle by the intersection point of the white line and the reference line, and does not require a separate means for detecting the white line.

According to a fourth aspect of the present invention, there is provided the vehicle periphery monitoring apparatus according to the second or third aspect, wherein a peripheral edge on the image is used as the reference line.

According to the fourth aspect of the present invention, since the peripheral edge on the image is used as the reference line, the angle between the reference line and the portion of the captured white line closest to the vehicle, or The lateral position can be detected based on the intersection position.

According to a fifth aspect of the present invention, in the vehicle periphery monitoring device according to any one of the second to fourth aspects, the angle or the position of the intersection is determined by the white line or one of its extension lines. Exists.

According to the fifth aspect of the present invention, since the angle or the intersection point is determined by the white line or one of the extension lines thereof, the angle or the intersection point between one of the white line or the extension line and the reference line is determined. It is only necessary to determine the position, and it is not necessary to determine the angle or the intersection point between each of the white line or its extension and the reference line.

The invention according to claim 6 is based on lane departure detecting means 3a-6 for detecting the departure of the own vehicle from the lane based on the lateral position of the own vehicle in the own lane detected by the position detecting means. 6. The vehicle lane departure warning means 5-1 for issuing an alarm when the vehicle departure detecting means detects the vehicle's own lane departure traveling. Vehicle peripheral monitoring device.

According to the present invention, the lane departure detecting means 3a-6 detects the traveling of the own vehicle in the own lane based on the lateral position of the own vehicle in the own lane detected by the position detecting means 3a-5. And the lane departure warning means 5-1 is
When the lane departure detecting means 3a-6 detects the vehicle's own lane departure traveling, an alarm is issued to notify the driver of the lane departure. Therefore, the lane departure detecting means 3a-6 detects drowsiness by detecting the own lane departure traveling. , Vehicle departure warning means 5-
1 allows the driver to be notified. Furthermore, it can be used for detecting a lane departure due to dozing using the position detection means 3a-5.

According to a seventh aspect of the present invention, there is provided a meandering operation detecting means for detecting a meandering operation of the own vehicle based on a change with time of a lateral position of the own vehicle in the own lane detected by the position detecting means. 7. The vehicle according to any one of claims 1 to 6, further comprising: a meandering operation alarm unit 5-2 that issues an alarm when the meandering operation detecting unit detects the meandering operation of the vehicle. For peripheral monitoring equipment.

According to the seventh aspect of the present invention, the meandering operation detecting means 3a-7 performs the meandering operation based on the time-dependent change of the lateral position of the own vehicle in the own lane detected by the position detecting means 3a-5. Detection means 3a
When the meandering operation detecting means 3a-7 detects the meandering operation of the own vehicle, the warning is issued to notify the driver of the occurrence of the meandering operation. Therefore, the meandering operation detecting means 3a-7 detects the snaking operation to detect the dozing. The driver can be informed of this by the meandering operation warning means 5-2. Further, the position detecting means 3a-5 can be used to detect a meandering operation due to dozing.

[0026]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment An embodiment of the present invention will be described below with reference to the drawings.
FIG. 2 is a block diagram showing the configuration of the vehicle periphery monitoring device according to the present invention. In FIG.
An image in the angle of view range defined by the lens 1a is formed on the image plane 1b.

The storage unit 2 includes first and second frame memories 2
a, 2b, a differential image memory 2c, and a divergent optical flow memory 2d. First frame memory 2a
And the second frame memory 2b converts the captured image data D1 imaged on the image plane 1b of the camera 1 into pixels of m rows and n columns, for example, 512 * 512 pixels and a luminance of 0 to 255 gradations. The data is temporarily stored as data D2 and D3 and output to a microcomputer 3 (hereinafter, microcomputer 3). These first or second frame memories 2a and 2b are, for example, first frame memories 2a and 2b.
a is time t, and the second frame memory 2b is time t + Δ
Pixel data D2 and D3 obtained by converting an image captured every predetermined time Δt into pixels of m rows and n columns, such as t, are sequentially stored.

The differential image memory 2c stores differential image data D4 generated by differentiating the captured image data D1 formed on the image plane 1b of the camera 1. The divergent optical flow memory 2d stores optical flow data D5 in the divergent direction and outputs the data to the microcomputer 3.

The microcomputer 3 comprises a CPU 3a which operates according to a control program, and a ROM 3b which holds a control program of the CPU 3a and preset values.
And a RAM 3c for temporarily storing data necessary when the CPU 3a executes the calculation.

The above-mentioned CPU 3a uses a white line detecting means for detecting a pair of white lines located on both sides of the own lane on which the own vehicle is to travel, a FOE setting means for setting the FOE, and a predetermined time before or after the camera using the FOE. Optical flow detecting means for detecting an amount of movement of the same point in the two obtained images as an optical flow, a danger judgment for detecting another vehicle approaching the own vehicle based on the size of the optical flow of the other vehicle and determining danger Means, position detecting means for detecting the lateral position of the own vehicle in the own lane, lane departure detecting means for detecting lane departure traveling based on the lateral position of the own vehicle in the own lane, and It functions as a meandering operation detecting means for detecting the meandering operation based on the change of the lateral position with time.

The turn signal detecting section 4 is attached to the turn signal mechanism of the vehicle, and outputs an H-level turn signal S1 to the microcomputer 3 when the turn signal switch operated by the driver to turn the vehicle to the left or right is turned on. Output.

The alarm generator 5 as a lane departure alarm and a meandering operation alarm has a speaker 5a and a display 5b. The alarm generation unit 5 displays an image captured by the camera 1, or detects a meandering operation and a lane departure, or determines that there is a danger of contact due to a sudden approach of another vehicle to the own vehicle. Then, a message or the like is displayed to inform the driver of the danger with a video.

The loudspeaker 5a provides voice guidance or an alarm based on a voice signal S2 issued from the microcomputer 3 when a meandering operation and a lane departure are detected, or when it is determined that there is a risk of contact with another vehicle. Generates sound such as sound. Then, the speaker 5a notifies the driver of the danger by voice using the voice.

The operation of the above-described vehicle periphery monitoring device will be described below. First, the camera 1 outputs the captured image shown in FIG. 3 to the microcomputer 3 as captured image data D1. This captured image includes the road 10, the white lines 11 to 15 drawn on the road 10, and the walls 1 standing on both sides of the road 10.
The image 6 disappears at the horizontal center position on the screen.

As described above, the captured image in FIG. 3 has the camera 1 mounted rearward on the rear portion of the vehicle, so that the right side of the captured image corresponds to the left side with respect to the traveling direction. The left side in the captured image corresponds to the right side with respect to the traveling direction.

The CPU 3a converts the picked-up image data D1 at time t output from the camera 1 into the
Pixel data D2 of 2 * 512 pixels and luminance of 0 to 255 gradations
And outputs it to the first frame memory 2a. First
The frame memory 2a temporarily stores the pixel data D2 so that the microcomputer 3 can always read the pixel data D2. Further, after the time Δt, the microcomputer 3 outputs the pixel data D3 of the captured image captured at the time t + Δt to the second frame memory 2b. That is, the first and second frame memories 2
Pixel data D2 and D3 of an image captured every predetermined time Δt are sequentially stored in a and 2b.

The CPU 3a detects the departure of the lane or the meandering operation.
, The luminance values _Im , _n of the pixels in the m rows and n columns are scanned in the horizontal direction in FIG. 3, and the luminance values _Im , _{n + 1 of the} adjacent pixels are scanned.
When the difference _{I m, n + 1 -I m} , n is equal to or higher than the predetermined luminance value and, to a luminance value I _m, and _n = 1, when less than a predetermined luminance value, the luminance value I _{m, n} =
As 0, a differential image (edge image) which is an image of only the edge portion on the captured image as shown in FIG. 4 is generated, and the generated differential image is output to the differential image memory 2c as differential image data D4.

Next, a reference line _VSL as shown in FIG. 5 is set for this differential image. The reference line _VSL is set so as to traverse the differential image in the vertical direction at the center position in the horizontal direction on the differential image. That is, the reference line V
_SL is set at the center in the horizontal direction of the lane separated by the white lines 12 and 13 where the vehicle is traveling.

When the reference line V _SL is set, a pair of white lines 12 located on both sides of the own lane on the road surface on which the own vehicle should travel
, And 13 are searched. This white line 1
The search for the edge points that form 2 and 13 is performed upward from the horizontal line H _(LO) located at the lower end of the screen shown in FIG. That is, the reference line V _SL on the A and the lowermost point P _(SO), to search for a edge point toward the horizontal direction at both ends. By this search, point constituting the edge of the white line 13 that is present on the right side of the reference line V _SL left P _(LO) and the reference line V _SL points constituting the edge of the white line 12 in the presence of P
_(RO) is obtained.

Subsequently, a point P located second from the lowermost end
From _(S1), the white line 12 to search for edge points is performed toward the horizontal direction both end portions, that are contained in the left side of the reference line V _SL
P points constituting the edge of the white line 13 that is present on the right side of the point P constituting the edges _(L1) and the reference line V _{SL (R1)} is obtained.

The above processing is performed upward in the differential image.
Edge points are sequentially extracted. At this time, follow
Point P that constitutes another vehicle_{(L (m + 2))},
P_{(R (m + 2))}, P _{(L (m + 4))}And P_{(R (m + 4))}I also extracted
So, on the same straight line from this extracted edge point
Only those that are in As a result, the vehicle travels.
A pair of white lines 12 on either side of the lane
Only the edge points that make up
You. Then, the extracted edge points are calculated by the least squares method.
Generate an approximate line, and use the approximate lines as white lines 12 and 13.
To detect. And detected as white lines 12 and 13
Extend the approximation line and set the intersection as FOE
You. Further, the pixel data D3 of the captured image after the time Δt
The same applies to both lanes on the road where the vehicle should travel.
The pair of white lines 12 and 13 located on the side are detected and detected.
Set the FOE at the point where the extension lines of the white lines 12 and 13 intersect
I do.

Further, the CPU 3a obtains an optical flow for detecting the degree of danger to another vehicle traveling in a nearby lane or to another following vehicle. Hereinafter, a procedure for obtaining an optical flow will be described with reference to FIG. First, the pixel data D2 is fetched from the first frame memory 2a, and in the captured image at the time t, for one point of interest, the direction of the radiation from the FOE set as described above centering on this one point of interest ( That is, an elongated window is set in the direction connecting the FOE and one point of interest (FIG. 6).
(A)). Next, the pixel data D3 is fetched from the second frame memory 2b, and in the captured image at the time t + Δt, while moving the window one point at a time in the radial direction from the FOE, for each pixel constituting the window at the time t, The absolute value of the luminance difference between each pixel constituting the window at the time t + Δt corresponding to the pixel is obtained. That is, the pixel P at time t (FIG. 6)
The pixel Q at time t + Δt corresponding to (a)) (FIG. 6)
The absolute value of the luminance difference from (b)) is obtained. The amount of movement of the window when the sum of the obtained luminance differences is minimized is obtained as an optical flow of one point of interest.

By repeating the above-described processing at all points of the captured image, the optical flow of the entire captured image can be obtained. Alternatively, a pixel in the window may be scanned to extract a point of interest, and the extracted points may be connected to obtain an optical flow.

The direction of the obtained optical flow is FOE.
If it is the direction toward, it indicates that another vehicle traveling in the nearby lane or another vehicle traveling in the succeeding lane is slower than the speed of the own vehicle and separates from the own vehicle, and conversely, the direction of the optical flow is relative to the FOE. The direction of divergence indicates that the vehicle is approaching.

The directions of optical flows generated by landscapes in the captured image, marks on the road surface, and the like are all FO.
It becomes the direction toward E and can be easily distinguished from the approaching nearby or other following vehicles. The length of the optical flow emitted from this neighboring or following other vehicle is proportional to the relative speed between the own vehicle and the neighboring or following other vehicle. Therefore, FOE
When the length of the optical flow in the direction of divergence from the vehicle exceeds a predetermined length, it is determined that another vehicle is rapidly approaching the own vehicle. There is an alarm ", and the display 5
The message “other vehicle approaching” is displayed in b.

Further, the CPU 3a sets a perpendicular L from the FOE as a reference line as shown in FIG. 7 to detect the lateral position of the own vehicle in the own lane. Are detected, and the angle difference θ1−θ2 between the angle θ1 and the angle θ2 is determined. Next, the angle difference θ1−θ2 is similarly obtained for the pixel data D3 of the captured image after Δt. That is, the angle difference θ1−θ2 is obtained for each Δt.

As a result of calculating the angle difference θ1−θ2, if the vehicle is traveling in the center of the lane as shown in FIG.
1−θ2 = 0 (FIG. 7A). In addition, if the own vehicle is traveling leftward as viewed from the camera 1 in the own vehicle, the angle difference becomes θ1−θ2 <0 (FIG. 7 (b)). The value of -θ2 becomes smaller. Furthermore, if the own vehicle is traveling rightward in the own lane as viewed from the camera 1, the angle difference θ1−θ2> 0 (FIG. 7 (c)). The value increases. As described above, the angle difference θ
By obtaining 1−θ2, the lateral position of the host vehicle in the host lane can be detected at every time ΔT.

Also, the white line and the FOE detected or set to detect the optical flow are used to
Angles θ1 and θ2 between each of a perpendicular line from OE and a white line
By detecting the lateral position of the own vehicle in the own lane, in addition to the detection of other vehicles approaching the own vehicle by the conventional vehicle periphery monitoring device, the lateral position of the own vehicle in the own lane at low cost Can also be detected.

Next, to detect the lane departure from the lateral position of the own vehicle in the own lane, for example, as a result of obtaining an angle difference θ1−θ2, a turn signal signal S1 is output from the turn signal detecting section 4. If not, it is determined that the driver does not intend to change lanes, and the own vehicle is in the camera 1 of the own lane.
The angle difference θ1−θ2 obtained from the image captured when deviating to the left is a first predetermined value, and the angle difference θ1−θ2 obtained from the image captured when deviating to the right is the second value.
When the angle difference θ1−θ2 is equal to or larger than a first predetermined value or equal to or smaller than a second predetermined value, the lane departure traveling is detected. Then, in order to notify the driver of the lane departure due to falling asleep, a warning is given by, for example, a voice from the speaker 5a indicating that the vehicle is running in a meandering manner, and the display unit 5b is displayed to indicate "lane departure." For this reason, an accident due to falling asleep can be prevented. Furthermore, by using the detected lateral position of the vehicle to detect a lane departure due to dozing, it is possible to detect a lane departure due to dozing at low cost.

Further, in order to detect the meandering operation of the host vehicle based on the time-dependent change in the lateral position of the host vehicle in the host vehicle lane, for example, as shown in FIG. The points N1, N2,..., Which are the peaks of the trajectory obtained from the direction position, are extracted, and when the displacement ΔM of the lateral position between adjacent extracted points exceeds a third predetermined value, the predetermined number of times is determined within a predetermined time. When there is, meandering operation is detected. Then, in order to notify the user, the speaker 5a gives an alarm by sound from the speaker 5a to indicate that "meandering."
By causing b to display “meandering operation”, it is possible to warn of a meandering operation due to dozing or the like. For this reason, an accident due to falling asleep can be prevented. Furthermore, by using the detected lateral position of the host vehicle to detect a lane departure due to dozing, it is possible to detect a meandering operation due to dozing at low cost.

The details of the operation outlined above are described in the microcomputer 3
The processing procedure of the CPU 3a will be described below with reference to the flowchart shown in FIG. First, captured image data D1 of a captured image at time t as shown in FIG. 3 is captured from the camera 1 and stored in the first frame memory 2a as pixel data D2 corresponding to the captured image data D1 at time t (step SP1). ). Next, each captured image data D1 at time t + Δt is captured and stored in the second frame memory 2b as pixel data D3 (step SP).
2). Differentiated images are generated by differentiating the respective pixel data D2 and D3 (step SP3), and white lines forming both sides of the own lane on which the vehicle travels are detected (step SP4). Next, the detected white lines on both sides are extended, and their intersection is set as the FOE (step SP
5).

Next, an optical flow is detected in order to detect a danger to another vehicle traveling in a nearby lane or another following vehicle (step SP6), and the obtained optical flow is diverged from the FOE in a long direction. When the vehicle length exceeds a predetermined length, it is determined that another vehicle is approaching the own vehicle (Y in step SP7), and a warning that there is a vehicle approaching is issued by a voice from the speaker 5a, for example, to notify that fact. Then, the display unit 5b is caused to display the other vehicle's sudden approach (step SP8), and the process returns to step SP2. If there is no other vehicle approaching when the optical flow converges on the FOE or the length does not exceed the predetermined length even in the diverging direction (N in step SP7), the process proceeds to the next step. .

Next, as shown in FIG. 7, the angles .theta.1 and .theta.2 between the vertical lines L and the white lines 12 and 13 dropped from the FOE set on the pixel data D2 of the captured image at time t are detected, and the angles .theta.1 Angle θ1−θ2 between the angle θ2 and the angle θ2 is determined. Next, the angle difference θ1−θ2 is similarly obtained for the pixel data D3 of the captured image at time t + Δt (step SP9).

Next, if the turn signal S1 is not output from the turn signal detecting section 4 (N in step SP10), it is determined in step SP9 that it is determined that the driver has no intention to change lanes and the lane departure is detected. Angle difference θ1−θ2 at time t or angle difference θ at time t + Δt
It is determined whether or not the own vehicle has deviated from the own lane based on 1-θ2 (step SP11). As a result, when the vehicle deviates from the lane (Y in step SP11), the vehicle departs from the lane to notify the driver that the vehicle has departed from the lane.
The display unit 5b displays the lane departure (step SP1).
2) The process proceeds to the next step SP13. When it can be determined that the vehicle has not deviated from the lane (N in step SP11), the process directly proceeds to step SP13.

Next, a meandering operation is detected based on a temporal change in the lateral position of the vehicle in the vehicle lane detected from the angle difference θ1−θ2 (step SP13). As a result, if the driver is performing the meandering operation (Y in step SP13), for example, a warning is given by voice from the speaker 5a to notify that the driver is performing the meandering operation, and the meandering operation is displayed on the display unit 5b. Display is performed (step SP14).
The process returns to the next step SP2. When the meandering operation is not performed (N in step SP13), the process directly returns to step SP2.

When the blinker signal S1 is output from the blinker detector 4 (Y in step SP10),
It is determined that the driver intends to change lanes, and the process returns to step SP2 without detecting lane departure and meandering operation.

Second Embodiment In the first embodiment, the perpendicular L from the FOE is used as a reference line, and the angles θ1 and θ2 between the perpendicular L and the white lines 12 and 13 are determined by the magnitude relation. Although the lateral position in the own vehicle was detected, for example, as shown in FIG. 10, the angle θ11 between the reference line L1 and the white lines 12 and 13 which are horizontal with respect to a preset image, and The lateral position in the host vehicle may be detected based on the magnitude relationship of θ21.

The details of the lateral position detection in the second embodiment will be described below. First, as shown in FIG.
And the white lines 12 and 13 detect angles θ11 and θ21, and determine the angle difference θ11−θ between the angles θ11 and θ21.
Find 21.

As a result of calculating the angle difference θ1−θ2, if the vehicle is traveling in the center of the lane as shown in FIG. 10, the angle difference θ11−θ21 = 0 (FIG. 10 (a)). Further, if the own vehicle is traveling leftward as viewed from the camera 1 in the own lane, the angle difference θ11−θ21> 0 (see FIG. 1).
0 (b)), the more the vehicle is to the left of the lane, the more the angle difference θ
The value of 11-θ21 increases.

Further, if the vehicle is traveling rightward as viewed from the camera 1, the angle difference θ11−θ21 <0 (FIG. 10 (c)). The value increases. By obtaining the angle difference θ11−θ21 as described above, the lateral position of the own vehicle in the own lane is detected at every time ΔT.

The above-described reference line L1 may be set at the bottom constituting the periphery on the screen as shown in FIG. By the way, when a rear road curves, for example, a portion of a captured white line that is far from the vehicle fluctuates according to the curve. Thus, by setting the reference line at the bottom constituting the periphery on the screen in this manner, the lateral position is detected based on the angle between the reference line and the portion of the white line being imaged that is closest to the vehicle. Can be
The position in the lateral direction can be accurately detected.

Third Embodiment In the first and second embodiments described above, the angle between each of the white lines 12 and 13 and the perpendicular L or the reference line L1 is detected. As shown in FIG. 12, only the angle θ11 between the white line 12 and the reference line L1 is detected,
The lateral position of the host vehicle may be detected based on the magnitude of the angle θ11.

That is, the angle θ11 in FIG. 12 is obtained by using the angle θ11 when the vehicle travels in the center of the lane as the reference angle θr.
When ef is set (FIG. 12A), if the own vehicle is traveling leftward as viewed from the camera 1 in the own lane, the angle difference θ11−θref> 0 (FIG. 12A).
(B)), the angle difference θ11−θr as the left side of the own lane is shifted
The value of ef increases. Furthermore, if the own vehicle is traveling to the right as viewed from the camera 1 in the own lane, the angle difference θ11−θref <0 (FIG. 12 (c)). The value of θref becomes small and changes according to the lateral position in the own vehicle.

As in the third embodiment, if the lateral position of the host vehicle is detected, it is only necessary to find the angle between one of the white lines 12 and 13 and the reference line L1, and the white line 1
Since there is no need to determine the angle between each of the reference lines 2 and 13 and the reference line, the processing for detecting the lateral position can be reduced.

Fourth Embodiment Further, in each of the first to third embodiments, the white line 1
The lateral position in the own vehicle is detected based on the angle between the reference lines 2 and 13 and the reference line.
As shown in FIG. 7, a white line or an extension line for setting the FOE is used, and the periphery on the screen is used as a reference line, and the intersection points A and B between the periphery and the extension lines of the white lines 12 and 13 are used. The lateral position in the host vehicle may be detected.

The reference line may be set at a position other than the periphery of the screen. However, the reference line may be set at the periphery based on the position of the intersection between the reference line and the portion closest to the vehicle among the white lines being imaged. The lateral position can be detected, and the lateral position can be accurately detected. In addition, both white lines 12 and 13
It is also conceivable to detect the lateral position from the position of the intersection with either one of the white lines without detecting the position of the intersection of the periphery with the reference line.

However, the vehicle may vibrate up and down while traveling on an uneven road or the like. Then, along with the vertical vibration of the vehicle, the FOE in the image also vertically vibrates,
As shown in FIG. 13A, the intersection points A and B between the peripheral edge and the white lines 12 and 13 in the direction diverging from the FOE are also shifted, so that it is difficult to accurately detect the lateral position.

However, as shown in FIG. 13A, the angle between the reference line and the white line does not change due to the vertical vibration of the FOE. Therefore, the lateral position can be determined more accurately by obtaining the lateral position based on the angle between the reference line and the white line than in the fourth embodiment described above.

[0069]

As described above, according to the first aspect of the present invention, the white line detected by the white line detecting means is diverted to detect the optical flow, and the position detecting means determines the position of the vehicle based on the position of the white line. It is possible to detect the lateral position of the own vehicle within the vehicle, and does not need a separate means for detecting a white line. Therefore, it is possible to obtain a vehicle periphery monitoring device that reduces costs.

According to the second aspect of the present invention, since the lateral position can be detected without being affected by the vertical vibration of the vehicle, the vehicle position can be more accurately detected. A peripheral monitoring device can be obtained.

According to the third aspect of the present invention, the white line detected by the white line detecting means is diverted to detect an optical flow, and the position detecting means determines the position of the self-vehicle in the own vehicle based on the intersection between the white line and the reference line. Since the position of the vehicle in the lateral direction can be detected, and a separate means for detecting the white line is not required, it is possible to obtain a vehicle periphery monitoring device with a reduced cost.

According to the fourth aspect of the present invention, the lateral position can be detected based on the angle formed between the reference line and the portion of the white line being imaged that is closest to the vehicle, or the intersection position. Thus, it is possible to obtain a vehicle periphery monitoring device capable of detecting the lateral position more accurately.

According to the fifth aspect of the present invention, it is only necessary to determine the angle or the position of the intersection between one of the white line and its extended line and the reference line, and to form each of the white line or its extended line with the reference line. Since there is no need to determine the angle or the intersection position, it is possible to obtain a vehicle periphery monitoring device that can reduce the processing for detecting the lateral position.

According to the sixth aspect of the present invention, the lane departure detecting means detects the drowsiness by detecting the departure from the own lane, and the driver can be informed of the drowsiness by the vehicle departure warning means. Can be prevented beforehand. Furthermore, since the position detecting means can be used to detect a lane departure due to drowsiness, a vehicle periphery monitoring device capable of detecting a lane departure due to drowsiness can be obtained at low cost.

According to the seventh aspect of the present invention, the meandering operation detecting means detects the snaking operation to detect the drowsiness, and the meandering operation warning means can notify the driver of the drowsiness. Can be prevented beforehand. Furthermore, since the position detecting means can be used to detect the meandering operation due to dozing, it is possible to obtain a vehicle periphery monitoring device capable of detecting the meandering operation due to dozing at low cost.

[Brief description of the drawings]

FIG. 1 is a diagram showing a basic configuration of a vehicle periphery monitoring device according to the present invention.

FIG. 2 is a block diagram showing an embodiment of a vehicle periphery monitoring device according to the present invention.

FIG. 3 is a diagram illustrating an image captured by a camera.

FIG. 4 is a diagram illustrating a differentiated image (edge image) generated by a differentiated process of a captured image.

FIG. 5 is a diagram illustrating an operation for detecting a white line.

FIG. 6 is a diagram illustrating an operation for detecting an optical flow.

FIG. 7 is a diagram illustrating an operation for detecting a lateral traveling position of the own vehicle in the own lane.

FIG. 8 is a diagram illustrating an operation for detecting a meandering operation.

9 is a flowchart showing a processing procedure by a CPU of a microcomputer constituting the vehicle periphery monitoring device of FIG. 2;

FIG. 10 is a diagram illustrating an operation for detecting a lateral traveling position of the own vehicle in the own lane.

FIG. 11 is a diagram illustrating an operation for detecting a lateral traveling position of the own vehicle in the own lane.

FIG. 12 is a diagram illustrating an operation for detecting a lateral traveling position of the own vehicle in the own lane.

FIG. 13 is a diagram illustrating an operation for detecting a lateral traveling position of the own vehicle in the own lane.

FIG. 14 is a diagram illustrating a change in an image obtained by a camera.

[Explanation of symbols]

 Reference Signs List 1 imaging means 3a-1 white line detecting means 3a-2 FOE setting means 3a-3 optical flow detecting means 3a-4 danger determining means 3a-5 position detecting means 3a-6 lane departure detecting means 3a-7 meandering operation detecting means 5- 1 Lane departure warning means 5-2 Meandering driving warning means

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 1/00 G08G 1/00 J 7/20 H04N 7/18 J G08G 1/00 G05D 1/02 K H04N 7/18 G06F 15/62 380 // G05D 1/02 15/70 410

Claims (7)

[Claims] 1. An image pickup device mounted on a vehicle to obtain an image by taking an image of the periphery of the vehicle, and processing an image obtained by the image pickup device to position the vehicle on both sides of a vehicle lane on a road surface on which the vehicle should travel. FO for setting a point at which a white line detecting means for detecting a pair of white lines to intersect with each other and an extension line extending from the white line detected by the white line detecting means as an FOE (infinite point).
E setting means, and FO set by the FOE setting means.
E, an optical flow detecting means for detecting, as an optical flow, a movement amount of the same point in two images obtained before and after a predetermined time by the imaging means, and an optical flow of another vehicle traveling around the own vehicle. And a danger determining means for determining danger by detecting another vehicle approaching the own vehicle based on the size, and a danger determining means for determining the danger, wherein the white line detected by the white line detecting means is located at a position in the image. And a position detecting means for detecting a lateral position of the vehicle in the vehicle lane based on the vehicle lane. 2. The vehicle in the vehicle lane based on an angle between the white line or an extension thereof that changes according to the position of the white line and a reference line set on the image. 2. The vehicle periphery monitoring device according to claim 1, wherein a lateral position of the vehicle is detected. 3. The vehicle according to claim 1, wherein the position detecting unit is configured to determine a position of the vehicle in the vehicle lane based on an intersection of the white line or an extension thereof that changes according to the position of the white line and a reference line set on the image. 2. The vehicle periphery monitoring device according to claim 1, wherein a lateral position is detected. 4. The vehicle periphery monitoring device according to claim 2, wherein a peripheral edge on the image is used as the reference line. 5. The vehicle periphery monitoring device according to claim 2, wherein the angle or the position of the intersection is determined by the white line or one of extensions thereof. 6. A lane departure detecting means for detecting a lane departure traveling of the own vehicle based on a lateral position of the own vehicle in the own lane detected by the position detecting means; The vehicle surroundings monitoring device according to any one of claims 1 to 5, further comprising: an own lane departure warning unit that issues an alarm notifying that the vehicle has deviated from its own lane. 7. A meandering operation detecting means for detecting a meandering operation of the own vehicle based on a time-dependent change in a lateral position of the own vehicle in the own lane detected by the position detecting means; When detecting meandering operation of the vehicle,
The vehicle surroundings monitoring device according to any one of claims 1 to 6, further comprising a meandering operation warning unit that issues a warning notifying the fact.