JP2004038877A - Perimeter monitoring device and image processing apparatus for vehicles - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a perimeter monitoring device for accurately determining the danger of a vehicle contacting an object even in running on a general road. <P>SOLUTION: A predicted running course calculation means 52a calculates a predicted running course of a vehicle. A first monitoring area setting means 52b sets a monitoring area based on the predicted running course. A danger determining means 100 monitors an object around the vehicle and determines the danger of the object in the monitored area contacting the vehicle. A white line detecting means 52c processes an image picked up by an imaging means 10 and detects a pair of white lines on either side of the lane in which the vehicle is going. A second monitoring area setting means 52d sets a monitoring area based on the detected white lines. A switching means 52e switches the monitoring area used by the danger judging means 100 for determination of dangerousness between the monitoring area set by the first monitoring area setting means 52b and the monitoring area set by the second monitoring area setting means 52d. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a vehicle periphery monitoring device and an image processing device, and more particularly to a vehicle periphery monitoring device that detects a degree of danger of contact between an object and a vehicle, and, for example, an image processing device used in a vehicle periphery monitoring device About.
[0002]
[Prior art]
Conventionally, as the above-described vehicle periphery monitoring device, a camera mounted on the vehicle, a laser light source, an ultrasonic oscillator, and the like are used to detect the position of an object around the vehicle and the degree of approach to the host vehicle, and to detect the position of the host vehicle. 2. Description of the Related Art When an object is present in a close position or when an object is coming very close to a host vehicle, it is determined that there is a danger of contact and a warning is issued.
[0003]
However, as in the above-described vehicle periphery monitoring device, when there is an object at a position very close to the own vehicle, when it is determined that there is a risk of contact because the object has suddenly approached the own vehicle. However, the following problems occur. In other words, when there is no intention to change course, there is another vehicle running close to the own vehicle on the adjacent lane, or another vehicle suddenly approaches from the adjacent lane. Therefore, there is a problem that the danger of contact with an object cannot be accurately determined.
[0004]
Therefore, in order to prevent such a situation, some of the vehicular peripheral monitoring devices using a camera detect a white line that is a lane division line using a road image captured by the camera, and based on the detected white line. There are those that identify the own lane area and the adjacent lane area.
[0005]
When the turn signal is not operated and there is no intention to change the course, the vehicle periphery monitoring device determines the risk of contact with an object existing in the own lane area. On the other hand, when the turn signal is operated and there is an intention to change the course, the risk of contact is determined for an object existing in the adjacent lane area in the operation direction.
[0006]
[Problems to be solved by the invention]
The vehicle periphery monitoring device that performs the white line detection described above was able to accurately determine the risk of contact with an object on a highway where the traveling environment is relatively simple. However, on general roads such as urban roads, there are many objects such as pedestrians and telephone poles, there is not always a white line, and the driver's awareness of maintaining the driving lane is low, The first problem is that it is not possible to judge the danger of contact. There is also a problem that the vehicle periphery monitoring device that performs white line detection cannot be applied to a device that performs periphery monitoring using a laser light source or an ultrasonic oscillator.
[0007]
In addition, a vehicle periphery monitoring device using a camera is configured of two cameras arranged at a predetermined distance from each other, and provides a parallax between an object imaged by one camera and an object imaged by the other camera. There is a stereo camera type that detects the relative position of an object based on the information and determines the risk of contact. Further, based on the movement amounts of the objects in two images obtained before and after a predetermined time from one camera, the approaching degree of the objects around the vehicle to the own vehicle is detected, and the risk of contact is determined. There is also an optical flow type.
[0008]
However, a vehicular monitoring device using a camera needs to perform image processing on an image captured by the camera to determine the danger of contact, and in general, the amount of calculation is enormous, and real-time (real-time) There is also a second problem that it is difficult to determine the danger of contact and that a high-speed arithmetic processing unit is required to realize it, so that the apparatus itself is expensive.
[0009]
In view of the above, the present invention focuses on the above-described problems, there is no white line, and there are many objects such as pedestrians and telephone poles, or the driver's awareness of maintaining the driving lane is low. A first object is to provide a vehicle periphery monitoring device that can accurately determine the danger of contact with an object even when traveling on a road. It is a second object of the present invention to provide an image processing apparatus which makes it possible to judge the risk of contact in real time at low cost by simplifying the image processing of the image around the vehicle imaged by the imaging means. Make it an issue.
[0010]
[Means for Solving the Problems]
The invention according to claim 1 for solving the first problem described above monitors an object around a vehicle as shown in a basic configuration diagram of FIG. 1 and detects a risk of contact between the object and the vehicle. Danger determination means 100 for determining the gender, predicted path calculation means 52a for calculating a predicted path of the vehicle, and first monitoring area setting means 52b for setting a monitoring area based on the predicted path. The means exists in the vehicle periphery monitoring device, wherein the danger is determined for an object present in the monitoring area.
[0011]
According to the first aspect of the present invention, the predicted course calculating means calculates the predicted course of the vehicle. The first monitoring area setting means sets a monitoring area based on the predicted course. The danger determining means monitors an object around the vehicle and determines a risk of contact between the vehicle and an object present in the monitoring area. Therefore, by setting the monitoring area based on the predicted course, there are many objects such as pedestrians and utility poles, or when traveling on a general road where the driver's awareness of maintaining the traveling lane is low. Also, an optimum monitoring area can be set. In addition, the present invention can be applied to a laser light source or a device that performs peripheral monitoring using an ultrasonic oscillator.
[0012]
The invention according to claim 2 is the vehicle surroundings monitoring device according to claim 1, wherein the danger determining unit monitors an object in front of the vehicle, and monitors a relationship between the object in front of the vehicle and the vehicle. The present invention resides in a vehicle periphery monitoring device characterized by determining a danger of contact.
[0013]
According to the invention described in claim 2, the danger determining means monitors the object in front of the vehicle and determines the danger of contact between the vehicle and the object in front of the vehicle. Therefore, an optimal monitoring area can be set in front of the vehicle.
[0014]
The invention according to claim 3 is the vehicle surroundings monitoring device according to claim 1 or 2, wherein the first monitoring area setting means sets an area including the predicted course as the monitoring area. A feature of the present invention resides in a vehicle periphery monitoring device.
[0015]
According to the invention described in claim 3, focusing on the fact that the risk of contact is small even if an object exists outside the predicted course, the first monitoring area setting means sets the area including the predicted course as Set as a monitoring area. Therefore, even when traveling on a general road, an optimal monitoring area can be set even more.
[0016]
The invention according to a fourth aspect is the vehicle periphery monitoring device according to the third aspect, wherein the first monitoring area setting means sets an area in which the predicted course is enlarged in the vehicle width direction as the monitoring area. The present invention resides in a vehicle periphery monitoring device.
[0017]
According to the fourth aspect of the invention, the first monitoring area setting means sets an area in which the predicted course is enlarged in the vehicle width direction as a monitoring area. Therefore, even if the driver operates the steering wheel to change the predicted course to some extent, or the case where another vehicle traveling around the predicted course approaches suddenly, an even more optimal A monitoring area can be set.
[0018]
According to a fifth aspect of the present invention, in the vehicle surroundings monitoring apparatus according to any one of the first to fourth aspects, the first monitoring area setting means is configured to control the vehicle from the vehicle along the predicted course. A vehicle periphery monitoring device is characterized in that a range that is farther than a distance determined according to a speed is excluded from the monitoring area.
[0019]
According to the fifth aspect of the present invention, an object that is farther than the distance traveled by the vehicle (hereinafter referred to as a stop distance) from when the driver senses the danger to when the driver steps on the brake and actually stops. Paying attention to the danger that there is no danger of contact, the first monitoring area setting means monitors a range that is farther from the vehicle along a predicted course than a distance (for example, a stopping distance) determined according to the vehicle speed. Exclude from region. Therefore, even when traffic congestion or the like occurs and the vehicle travels at a low speed or runs at a high speed along the traffic flow, the optimal monitoring area can be set even more.
[0020]
According to a sixth aspect of the present invention, as shown in the basic configuration diagram of FIG. 1, the perimeter monitoring device for a vehicle according to any one of the first to fifth aspects, wherein the danger determining means captures an image of the periphery of the vehicle. A white line detecting unit 52c that performs image processing on an image captured by the image capturing unit, and detects a pair of white lines located on both sides of the own lane in which the own vehicle travels, based on the detected white line A monitoring area set by the first monitoring area setting means, a second monitoring area setting means 52d for setting a monitoring area, and a monitoring area used by the danger determining means for determining the danger. A switching device 52e for switching between the monitoring region set by the region setting device and the monitoring region is further provided.
[0021]
According to the invention described in claim 6, in the danger determining means, the imaging means captures an image of the periphery of the vehicle. The white line detecting means processes the image captured by the image capturing means to detect a pair of white lines located on both sides of the own lane in which the own vehicle travels. The second monitoring area setting means sets a monitoring area based on the detected white line. The switching means switches a monitoring area used by the danger determining means for determining danger between the monitoring area set by the first monitoring area setting means and the monitoring area set by the second monitoring area setting means. Therefore, on a highway where a white line exists and the traveling environment is relatively simple, the monitoring area is switched to the monitoring area set by the second monitoring area setting means, and there are no white lines and many objects such as pedestrians and telephone poles. On a general road, by switching to the monitoring area set by the second monitoring area setting means, it is possible to set an optimum monitoring area adapted to the driving environment.
[0022]
The invention according to claim 7 for solving the above second problem is a vehicle periphery monitoring device according to any one of claims 1 to 5, as shown in the basic configuration diagram of FIG. The danger determining means includes: an imaging means 10 for imaging the periphery of the vehicle; and a density change detection means for scanning an image taken by the imaging means to detect a density change amount between pixels existing in the vicinity of the scanning direction. Means 52f, and a feature point detecting means 52g for detecting a maximum point of the detected density change amount in the scanning direction as a feature point, performing image processing on an image composed of the feature points, and Is determined in the vehicle periphery monitoring device.
[0023]
An image processing apparatus according to a ninth aspect of the present invention for solving the above-mentioned second problem, performs image processing on an image captured by an imaging unit mounted on a vehicle and detects a feature point in the image. A density change amount detection unit configured to scan an image picked up by the imaging unit and detect a density change amount between pixels existing in the vicinity of the scanning direction; and a local maximum in the scanning direction of the detected density change amount. And a feature point detecting unit that detects a point as a feature point.
[0024]
According to the seventh and ninth aspects of the present invention, in the danger determining means, the imaging means images the periphery of the vehicle. The density change amount detection unit scans the image picked up by the image pickup unit, and detects a density change amount between pixels existing near the scanning direction. The characteristic point detecting means detects a maximum point in the scanning direction of the detected density change amount as a characteristic point. The danger determining means performs image processing on the image composed of the feature points to determine the danger. Therefore, by detecting the maximum point of the density change amount as a feature point, a region having a certain width along the edge is detected as a feature point even when the edge of the object is not clearly imaged. And the width can be reduced.
[0025]
The invention according to claim 8, which has been made to solve the above-mentioned second problem, is a vehicle periphery monitoring device according to any one of claims 1 to 5, as shown in the basic configuration diagram of FIG. The danger determining means includes two image capturing means 10 for capturing an image of the periphery of the vehicle disposed at a predetermined distance from each other, and the same inspection for detecting the same point in two images captured by the two image capturing means. Output means 52h, wherein the same point detecting means detects a feature point in two images picked up by the two image pickup means, and detects a feature point in one of the two images. Set a window centered on the point, in another image, set a window centered on a feature point present on an epipolar line that is a straight line where the same point with respect to the feature point appears, in the other image Of the images in the set window, Resides in a vehicle-surroundings monitor apparatus and detecting the same point feature points dissimilarity between images of the window constitutes a substantial center of the lowest window.
[0026]
According to a tenth aspect of the present invention, there is provided an image processing method comprising the steps of: performing image processing on two images taken by two image pickup units mounted on a vehicle and arranged at a predetermined distance from each other; An image processing apparatus that detects the same point in two images captured by the two imaging units, and detects feature points in the two images captured by the two imaging units; A window centered on a feature point in one of the two images is set, and a window centered on a feature point on a straight line on which the same point as the feature point appears in the other image. Among the images in the window set in the other image, the feature point constituting the approximate center of the window having the lowest degree of difference from the image of the window set in the one image is defined as the same point. There is an image processing apparatus characterized in that it is detected.
[0027]
According to the eighth and tenth aspects of the present invention, when the danger is determined based on the difference between the same points in two images captured by two imaging units arranged at a predetermined distance from each other, The same point detection means sets a window having a feature point in one of the two images substantially at the center. In the other image, a window is set centering on a feature point existing on an epipolar line which is a straight line on which the same point as the feature point for which the window is set appears. Then, of the images in the window set in the other image, a feature point that forms the approximate center of the window having the lowest difference from the image of the window set in one image is detected as the same point. According to the above configuration, when detecting the same point as a feature point in one image from another image, a window set for the feature point in one image is set to an epipolar point in the other image. By moving the window along the line and for each feature point present on the epipolar line and determining the degree of difference, the window can be scanned over the entire other image, or the epipolar in another image can be scanned. The same point can be obtained without moving the window one pixel at a time on the line and scanning.
[0028]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 2 is a block diagram showing an embodiment of a vehicle periphery monitoring device incorporating the image processing device of the present invention. In FIG. 1, a vehicle periphery monitoring device is mounted in front of a vehicle, and an imaging unit 10 as an imaging unit that captures an image of the front of the vehicle, a storage unit 20 that stores an image, and outputs a steering angle signal according to a steering angle of the steering wheel. A steering angle sensor 30, a speed sensor 40 for detecting a speed signal corresponding to the vehicle speed, a microcomputer (μCOM) 50 for executing various arithmetic processing, and an alarm generation unit 60 for generating an alarm.
[0029]
The imaging unit 10 includes a right CCD 11R and a left CCD 11L mounted on a vehicle, a right image plane 12R on which an image captured by the right CCD 11R is projected, and a left image plane 12L on which an image captured by the left CCD 11L is projected. And The imaging unit 10 is attached so that the front of the vehicle is an imaging area. The pair of CCDs 11R and 11L constituting the imaging unit 10 are mounted at a predetermined distance in the horizontal direction.
[0030]
The storage unit 20 includes a right frame memory 21R for temporarily storing an image projected on the right image plane 12R, a left frame memory 21L for temporarily storing an image projected on the left image plane 12L, and imaging by the CCDs 11R and 11L. And a frame memory 22 that is used in a process of performing image processing on the obtained image to determine a risk of contact with an object.
[0031]
The μCOM 50 has a ROM 51 in which an operation program for determining the danger of contact between the vehicle and an object is recorded, a CPU 52 that operates according to the operation program, and a RAM 53 that temporarily stores information necessary when the CPU 52 operates. are doing. The alarm generation unit 60 includes, for example, a display (not shown) and a speaker (not shown). When the μCOM 50 determines that there is a risk of contact with an object, the alarm generation unit 60 generates an alarm sound to notify the danger. Generates an alarm or displays a message to alert the driver.
[0032]
The operation of the vehicle periphery monitoring device having the above configuration will be described below with reference to the flowchart of FIG. The CPU 52 starts its operation, for example, by turning on an ignition switch (not shown). In an initial step (not shown), the CPU 52 initializes various areas formed in the RAM 53 in the μCOM 50, and then proceeds to the first step S1.
[0033]
In step S1, the CPU 52 stores the images projected on the image planes 12R and 12L in the right-side frame memories 21R and 21L, respectively, and acquires the images captured by the CCDs 11R and 11L (step S1). Next, the CPU 52 detects a point on the edge (contour) of the object reflected in each of the acquired captured images as a feature point, creates a feature point image including only the feature points, and 22 (step S2).
[0034]
A detailed operation of the above-described feature point image creation will be described. First, the CPU 52 functions as a density change amount detecting unit, scans each image stored in the right frame memories 21R and 21L in the x-axis direction (= image horizontal direction), and scans the pixels existing near the x-axis direction. Obtain the density change amount. A typical example of a method for obtaining the density change amount is a Sobel operator. Next, the CPU 52 functions as a feature point detecting means, and determines, as shown in FIG. ₁ , P ₂ , P ₃ , P ₄ , And a feature point image is created.
[0035]
By the way, conventionally, it has been general to detect, as a feature point, a point where the amount of density change is equal to or larger than a threshold value. However, in this conventional feature point detection method, as shown in FIG. 4, when the edge of the object is not clearly imaged and the density change at the edge is gentle, the edge has a certain width W equal to or larger than the threshold value A. All the points in the region are detected as feature points.
[0036]
Therefore, if the maximum point of the amount of change in density is detected as a feature point as in the present invention described above, all points in a region having a certain width W along the edge are not detected as feature points. , One point along the edge (= feature point P ₁ , P ₂ , P ₃ , P ₄ ..) Are detected as feature points. In other words, feature points are not excessively extracted in portions where edges are not clearly imaged, and image processing performed on a feature point image described later can be simplified.
[0037]
Next, as described above, the CPU 52 creates a parallax image using the two captured images captured by the right CCD 11R and the left CCD 11L and the feature point images respectively created for the two captured images, and generates a frame. It is stored in the memory 22 (step S3).
[0038]
Specifically, the CPU 52 first functions as the same point detecting means, and converts the same point as a feature point present in one feature point image (for example, a left feature point image) into the other feature point image (for example, (Right-side feature point image). The same point detection is performed for all the feature points existing in the left feature point image. Then, the position difference between all the same points, that is, the parallax is obtained. Next, a parallax image is created by converting a feature point at which parallax occurs into a luminance corresponding to the parallax value.
[0039]
At this time, if the conversion is performed so that the luminance increases as the parallax value increases, the parallax image becomes brighter as the feature point of the object existing near the vehicle (because the closer the object is, the larger the parallax becomes). The feature points of a moving object become darker (because a farther object, which is an image, has a smaller parallax).
[0040]
Next, the detailed operation of the same point detection performed in the process of creating the parallax image described above will be described with reference to FIG. First, the CPU 52, for example, places a feature point P of interest on an image captured by the left CCD 11L. ₅ N × N window W centered on ₁ Set. Next, from the right-side captured image, the feature point P ₅ Epipolar line L which is a straight line where the same point appears ₁ Feature point P existing on ₆ , P ₇ , P ₈ Is detected. When two CCDs 11R and 11L are horizontally arranged as in the present invention, the epipolar line L ₁ Is one line in the horizontal direction.
[0041]
Next, the CPU 52 sets the epipolar line L ₁ Feature point P existing on ₆ , P ₇ , P ₈ N × N window W centered on ₂₁ , W ₂₂ , W ₂₃ Of the images in the window W ₁ The feature point set in the window having the lowest degree of difference from the image in ₅ Are detected as the same point.
[0042]
As described above, by detecting the same point as the feature point in the left captured image from the right captured image, the window W ₁ Is scanned over the entire right-side captured image, or the epipolar line L in the right-side captured image is scanned. ₁ Window W one pixel above ₁ The same point can be obtained without moving and scanning. For this reason, image processing related to the same point detection can be simplified.
[0043]
Since the parallax image created as described above includes the parallax information of an object having no danger of contact, such as a character drawn on a road surface, the CPU 52 next calculates the parallax of an object having the same height as the road surface. Information is removed from the parallax image (step S4). The distance (parallax) to the road surface captured in the captured image is represented by the following equation (1).
d = A.y + B.x + C (1)
[0044]
In the above equation (1), d is the parallax, x is the position of the parallax image in the x-axis direction, y is the position of the parallax image in the y-axis direction (image vertical direction), and A, B, and C are road surface parameters. . The road surface parameters A, B, and C are determined from the installation parameters of the CCDs 11R and 11L with respect to the road surface. By removing the point having the luminance value corresponding to the parallax d obtained by the equation (1) from the parallax image, the parallax information of the object having the same height as the road surface can be removed from the parallax image.
[0045]
Next, the CPU 52 creates an interpolated parallax image using the created parallax image (Step S5). As described above, in the created parallax image, parallax is not calculated for all pixels in the image. Therefore, for pixels other than the feature points, that is, regions having a flat density, it is necessary to infer and interpolate from parallax values calculated from nearby feature points.
[0046]
Therefore, when edges having the same parallax in the x-axis direction continue in the parallax image, they are regarded as edges generated from the same vehicle. Then, by filling the horizontal direction between the same edges with a parallax value (luminance) having both edges, parallax of a portion where no feature point exists is interpolated to create an interpolated parallax image. Accordingly, a parallax image having luminance according to the parallax can be obtained not only for the contour of the captured object but also for the entire object (within the contour).
[0047]
Next, the CPU 52 creates a projection parallax image using the created interpolation parallax image (Step S6). Specifically, the x-axis is left as it is from the interpolated parallax image, the y-axis is the frequency distribution of the parallax d in the y-direction of the interpolated parallax image, and a projection parallax image having a horizontal axis x and a vertical axis d is created.
[0048]
For example, it is assumed that another vehicle F and another vehicle G are traveling ahead. The interpolated parallax image at this time is as shown in FIG. In the figure, since the other vehicle F is traveling closer to the own vehicle than the other vehicle G, the disparity generated from the other vehicle F is naturally larger than the disparity generated from the other vehicle G.
[0049]
Therefore, when the interpolated parallax image as shown in FIG. 6A is converted into a projected parallax image as described above, the projected parallax image is moved forward by another vehicle F as shown in FIG. 6B. Straight line L ₃ Is converted to a straight line L on the back side ₂ (However, it is assumed that the parallax d indicates a larger value toward the near side.) That is, since the parallax generated from the same object is substantially the same, the object is converted into a straight line according to the generated parallax, that is, the distance from the vehicle. Then, the straight line converted from the object existing near the own vehicle is located closer to the near side (because the closer the object is, the larger the parallax is). This indicates the position when the object is viewed from directly above.
[0050]
Next, the CPU 52 functions as a predicted course calculating means, and calculates a predicted course M on the XZ plane of the road surface as shown in FIG. 7 (step S7). The predicted trajectory M is obtained from an arc having a turning radius R corresponding to the steering angle of the steering wheel in consideration of the vehicle width. Note that the radius of gyration R is calculated by the following equation (2).
R = L _WB / Tanθ _steer … (2)
L _WB Is the wheelbase of the vehicle, θ _steer Represents the turning angle of the tire calculated from the steering angle of the steering wheel. The steering wheel angle is obtained based on the steering angle signal from the steering angle sensor 30.
[0051]
Next, the CPU 52 functions as first monitoring area setting means and sets a monitoring area (step S8). First, on a general road, focusing on the fact that the range in which the driver must pay attention varies depending on the traveling direction of the own vehicle, the expected course M obtained in step S7 is enlarged by ΔD in the vehicle width direction. The set area is set as a monitoring area.
[0052]
Further, the range in which the driver must pay attention varies depending on the traveling speed of the host vehicle. In other words, when driving at a considerably reduced speed, such as during a traffic jam or before an intersection, it is only necessary to pay attention to the short distance. On the other hand, when you're speeding along the traffic flow, you need to watch your eyes very far. Focusing on this, the distance L determined along the predicted course M according to the vehicle speed (determined based on the speed signal from the speed sensor 40) _Q Exclude more distant areas from the monitoring area.
[0053]
Note that the distance L determined according to the vehicle speed _Q Specifically, the distance traveled by the vehicle (= stop distance) between the time when the driver perceives danger, the time when the driver depresses the brakes, and the time when the vehicle actually stops can be considered. By the above operation, the monitoring area E can be set on the XZ plane of the road surface.
[0054]
Next, the points in the monitoring area E on the XZ plane set above are converted into xy planes on the image planes 12R and 12L using a predetermined conversion formula. By this conversion, as shown in FIG. 8, a monitoring area image obtained by capturing the monitoring area E drawn on the road surface by the CCDs 11R and 11L can be obtained.
[0055]
Next, the CPU 52 converts the monitoring area image into the projection parallax image as described above, and stores it in the frame memory 22 (Step S9). Thereafter, the CPU 52 compares the projection parallax image of the monitoring area with the projection parallax image of the captured image created in step S6, and in FIG. ₂ , L ₃ When the object represented by is present in the monitoring area, it is determined that there is a risk of contact with the object (Y in step S10), and a warning is given to the driver using the warning generation unit 60. (Step S11), returning to step S1.
[0056]
On the other hand, if no object exists in the monitoring area, it is determined that there is no danger of contact with the object (N in step S10), and the process immediately returns to step S1. From the above, it can be understood that the imaging unit 10 and the CPU 52 constitute the danger determining unit 100 in the claims.
[0057]
As described above, by setting the monitoring area E including the predicted course M, a general road in which a large number of objects such as pedestrians and telephone poles are present or the driver's awareness of maintaining the traveling lane is low. Even when traveling, the optimum monitoring area E can be set, so that even when traveling on a general road, the danger of contact with an object can be accurately determined.
[0058]
In addition, by setting the area obtained by enlarging the predicted course M by ΔD in the vehicle width direction as a monitoring area, the driver operates the steering wheel to change the predicted course M slightly. An optimal monitoring area E can be set further in consideration of a case where another vehicle traveling around the area suddenly approaches.
[0059]
Further, by excluding from the monitoring area E a range that is farther than a distance (for example, a stopping distance) determined according to the vehicle speed from the vehicle along the predicted course M, congestion or the like occurs, and the vehicle travels at a low speed. Even in the case where the vehicle travels or repeats traveling at a high speed along the traffic flow, the optimum monitoring area E can be set even more.
[0060]
In the embodiment described above, the monitoring area is set based on the predicted trajectory. However, for example, on a highway where the driving environment is relatively simple, a monitoring area is set based on a white line as in the past, and on a general road, a monitoring area is set based on the predicted course as described above. You may. In this case, the CPU 53 functions as a white line detection unit, a second monitoring area setting unit, and a switching unit. Switching between the monitoring area based on the white line and the monitoring area based on the predicted course may be performed as described below.
[0061]
When the vehicle speed is equal to or higher than a certain value and it can be determined that the vehicle is traveling on a highway, the monitoring area based on the white line is used.When the vehicle speed is less than a certain value and it can be determined that the vehicle is traveling on a general road, Then, a danger determination is made using the monitoring area based on the predicted course. It is also conceivable to switch between a general road and an expressway based on information from the car navigation device. Furthermore, when a white line cannot be detected, it may be determined that the vehicle is traveling on a general road, and when a white line can be detected, it may be determined that the vehicle is traveling on an expressway, and switching may be performed.
[0062]
Further, in the above-described embodiment, the predicted course is calculated based on the steering angle sensor 30, but the predicted course may be calculated using, for example, a gyro.
[0063]
Further, in the above-described embodiment, two CCDs 11R and 11L are used as means for monitoring the surroundings of the host vehicle. However, for example, the setting of the monitoring area can also be applied to a method of monitoring the periphery using one CCD or a method of monitoring the periphery using a laser light source and an ultrasonic oscillator. Further, the feature point detection described in step S2 can be applied to an apparatus that performs peripheral monitoring using one CCD.
[0064]
【The invention's effect】
As described above, according to the first aspect of the present invention, by setting the monitoring area based on the predicted course, there are a large number of objects such as pedestrians and telephone poles, or the driver is required to maintain the traveling lane. Even when driving on a general road where awareness is low, it is possible to set the optimal monitoring area, so even when driving on a general road, accurately judge the danger of contact with objects can do. In addition, it is possible to obtain a vehicle periphery monitoring device that can be applied to a device that monitors the periphery using a laser light source or an ultrasonic oscillator.
[0065]
According to the second aspect of the present invention, it is possible to set an optimum monitoring area in front of the vehicle, and therefore, even when traveling on a general road, there is a danger of contact with an object present ahead accurately. Can be obtained.
[0066]
According to the third aspect of the present invention, even when driving on a general road, an optimal monitoring area can be set even more. Therefore, even when driving on a general road, It is possible to obtain a vehicle periphery monitoring device capable of accurately determining the danger of contact with an object.
[0067]
According to the invention described in claim 4, when the driver operates the steering wheel to change the predicted course slightly, or when the other vehicle traveling around the predicted course approaches suddenly. Since the optimum monitoring area can be set even more in consideration of the surrounding area for vehicles, the danger of contact with an object can be more accurately determined even when traveling on a general road. A monitoring device can be obtained.
[0068]
According to the fifth aspect of the present invention, even when traffic congestion or the like occurs and the vehicle travels at a low speed or runs at a high speed along the traffic flow, it is more optimally monitored. Since the area can be set, it is possible to obtain a vehicle periphery monitoring device that can more accurately determine the danger of contact with an object even when traveling on a general road.
[0069]
According to the invention of claim 6, on a highway where a white line exists and the traveling environment is relatively simple, the monitoring area is switched to the monitoring area set by the second monitoring area setting means. On a general road where a large number of objects are present, switching to the monitoring area set by the second monitoring area setting means makes it possible to set an optimum monitoring area adapted to the traveling environment, so that the object can be detected more accurately. A vehicle periphery monitoring device that can determine the risk of contact with the vehicle can be obtained.
[0070]
According to the seventh and ninth aspects of the invention, by detecting the maximum point of the luminance difference as a feature point, even if the edge of the object is not clearly imaged, a certain width can be provided along the edge. Since the width of the region having the characteristic point is not detected as the characteristic point and the width can be reduced, the image processing performed on the image composed of the characteristic points can be simplified by removing unnecessary characteristic points. Thus, it is possible to obtain an inexpensive vehicle periphery monitoring device and an image processing device that can judge a contact danger in real time at low cost.
[0071]
According to the eighth and tenth aspects of the present invention, when the same point as a feature point in one image is detected from another image, a window set for the feature point in one image is set to the other. By moving the window along the epipolar line in the image of each of the images and for each feature point existing on the epipolar line, and determining the degree of difference, the window can be scanned over the entire other image. Since the same point can be obtained without moving the window by one pixel at a time on the epipolar line in the image, the image processing relating to the same point detection can be simplified, inexpensively, and in real time. It is possible to obtain a vehicle periphery monitoring device and an image processing device capable of determining a contact danger.
[Brief description of the drawings]
FIG. 1 is a basic configuration diagram of a vehicle periphery monitoring device of the present invention.
FIG. 2 is a block diagram showing an embodiment of a vehicle periphery monitoring device incorporating the image processing device of the present invention.
FIG. 3 is a flowchart showing a processing procedure of a CPU 52 constituting the vehicle periphery monitoring device of FIG. 2;
FIG. 4 is a diagram for explaining an operation of a CPU 52 in creating and creating a feature point image in FIG. 3;
FIG. 5 is a diagram for explaining an operation of a CPU 52 in creating a parallax image in FIG. 3;
FIG. 6 is a diagram for explaining an operation of a CPU 52 in creating a projection parallax image in FIG. 3;
7 is a diagram for explaining the operation of a CPU 52 in calculating a predicted course and setting a monitoring area in FIG. 3;
8 shows a monitoring area image obtained when the monitoring area E drawn on the road surface shown in FIG. 7 is imaged by the CCDs 11R and 11L.
[Explanation of symbols]
10 Imaging means
52a predicted route calculating means
52b first monitoring area setting means
52c White line detection means
52d second monitoring area setting means
52e switching means
52f density change amount detecting means
52g feature point detecting means
52h Same point detecting means
100 danger judgment means

Claims (10)

Danger determining means for monitoring an object around the vehicle and determining a risk of contact between the object and the vehicle,
Predicted route calculating means for calculating a predicted route of the vehicle,
A first monitoring area setting means for setting a monitoring area based on the predicted course,
The perimeter monitoring device for a vehicle, wherein the danger determining means determines the danger of an object present in the monitoring area. The vehicle periphery monitoring device according to claim 1,
A vehicle surroundings monitoring device, wherein the danger determining means monitors an object in front of the vehicle and determines a risk of contact between the vehicle and an object in front of the vehicle. The vehicle periphery monitoring device according to claim 1 or 2,
The first monitoring area setting means sets an area including the predicted course as the monitoring area. The vehicle periphery monitoring device according to claim 3,
The first monitoring area setting means sets an area obtained by enlarging the predicted course in the vehicle width direction as the monitoring area. The vehicle periphery monitoring device according to any one of claims 1 to 4,
The first monitoring area setting means excludes, from the monitoring area, a range that is further from the vehicle along the predicted course and more than a distance determined according to the vehicle speed, from the monitoring area. Monitoring device. The vehicle periphery monitoring device according to any one of claims 1 to 5,
The danger determining means has an imaging means for imaging the periphery of the vehicle,
Image processing of the image captured by the imaging means, white line detection means for detecting a pair of white lines located on both sides of the own lane on which the own vehicle travels,
Second monitoring area setting means for setting a monitoring area based on the detected white line;
Switching means for switching the monitoring area used by the danger determining means for determining the risk between a monitoring area set by the first monitoring area setting means and a monitoring area set by the second monitoring area setting means And a vehicle periphery monitoring device, further comprising: The vehicle periphery monitoring device according to any one of claims 1 to 5,
The danger determining means,
Imaging means for imaging the periphery of the vehicle;
A density change amount detection unit that scans an image captured by the imaging unit and detects a density change amount between pixels existing near a scanning direction,
Characteristic point detecting means for detecting a maximum point in the scanning direction of the detected density change amount as a characteristic point,
A vehicular periphery monitoring device, wherein the danger is determined by performing image processing on an image composed of the feature points. The vehicle periphery monitoring device according to any one of claims 1 to 5,
The danger determining means,
Two image pickup means for picking up an image around a vehicle arranged at a predetermined distance from each other,
Same point detecting means for detecting the same point in the two images picked up by the two image pickup means,
The same point detecting means detects a feature point in two images picked up by the two image pickup means, and sets a window substantially centered on a feature point in one of the two images. Setting, in another image, setting a window centered on a feature point present on an epipolar line that is a straight line where the same point with respect to the feature point appears, among images in the window set in the other image And detecting a feature point constituting a substantially center of the window having the lowest difference from the image of the window set in the one image as the same point. An image processing apparatus that performs image processing on an image captured by an imaging unit mounted on a vehicle and detects a feature point in the image,
A density change amount detection unit that scans an image captured by the imaging unit and detects a density change amount between pixels existing near a scanning direction,
An image processing apparatus comprising: a feature point detecting unit configured to detect a maximum point of the detected density change amount in the scanning direction as a feature point. Image processing is performed on two images picked up by two image pickup units mounted on the vehicle and arranged at a predetermined distance from each other, and the same point in the two images picked up by the two image pickup units is detected. An image processing device,
A feature point in the two images captured by the two imaging units is detected, and a window centered on a feature point in one of the two images is set, and a window is set in the other image. Setting a window centered on a feature point present on a straight line where the same point with respect to the feature point appears, and among the images in the window set in the other image, a window set in the one image An image processing apparatus for detecting, as an identical point, a feature point constituting a substantially center of a window having the lowest degree of difference from the image.

Images