JP2004221871A - Device for monitoring periphery of vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily detect an obstacle in a traveling direction of a vehicle. <P>SOLUTION: An image from an image pickup device 1 is stored in a storage device 9 at first timing when the vehicle is to park or stop. The image given from the image pickup device 1 is compared with an image in the storage device 9 at second timing just before starting of the vehicle. The image which is picked up by the image pickup device 1 is stored in the storage device 9 in a considerably low speed state immediately after starting, and a difference between pixel values of respective sample points is operated in a prescribed curve coordinate whenever the vehicle travels by a prescribed distance. Presence or absence of the obstacle at the periphery of the vehicle is judged based on the result of comparison or the operation. Before the vehicle starts, a driver is previously and easily prevented from starting without being conscious of the obstacle. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a vehicle periphery monitoring device that captures an image of the periphery of a vehicle with an imaging device and detects an obstacle based on the captured image.
[0002]
[Prior art]
Conventionally, an image of the front of a vehicle is taken by an imaging camera, and an obstacle is detected based on the taken image.
[0003]
For example, in Japanese Patent Application Laid-Open No. 7-294251 (Prior Art 1), a three-dimensional object is extracted by acquiring a forward field of view of a vehicle using two image sensors and forming a difference image based on a stereogram principle.
[0004]
In Japanese Patent Application Laid-Open No. 7-96806 (prior art 2), instead of recognizing a front vehicle having a complicated shape and a plurality of colors and sizes, a front road surface is recognized and preset on the road surface. By determining whether there is an object other than the road (ie, a preceding vehicle) in the area and determining the safe inter-vehicle distance, the use of only one imaging device as a forward situation recognition sensor allows the necessary memory and information to be obtained. The amount of processing is greatly reduced.
[0005]
Further, in Japanese Patent Application Laid-Open No. Hei 7-264577 (prior art 3), a spotlight is radiated in the form of a grid in front of a vehicle using a light projector, and when there is an obstacle, the spotlight hits the obstacle, so Obstacle detection is performed by utilizing the fact that it is shifted as compared with the case where the object is irradiated.
[0006]
[Patent Document 1]
JP-A-7-294251 [Patent Document 2]
JP-A-7-96806 [Patent Document 3]
JP-A-7-264577
[Problems to be solved by the invention]
In general, when an image of the periphery of a vehicle is captured by an image capturing camera, the image is curved due to the characteristics of a lens used in the image capturing camera, so that it may be difficult to intuitively understand the image. In this case, for example, in the case of a back monitor, the captured image is almost the same as the image reflected on the rearview mirror. It is easy to correct the image to be intuitive. However, in the case of an image obtained by capturing the front blind spot, even if the image processing as described above is performed, the image is an image of an area that is conventionally unfamiliar. It becomes difficult to instantly understand which part is displayed.
[0008]
Further, in the above-described related art 1, obstacle detection is performed by using two imaging cameras and using two images captured at different angles at the same time. However, two imaging cameras are required. The costs are enormous.
[0009]
Furthermore, in the above-mentioned prior art 3, since a light projector is required separately from the imaging camera, the cost is also large.
[0010]
Therefore, an object of the present invention is to provide a vehicle periphery monitoring device that does not cause an increase in cost and that can easily detect an obstacle or an intruding object.
[0011]
[Means for Solving the Problems]
In order to solve the above-mentioned problem, the invention according to claim 1 is a vehicle periphery monitoring device that captures an image of the periphery of a vehicle with an imaging device and detects an obstacle based on the captured image, and captures an image of the periphery of the vehicle. An image capturing device, a storage device that stores an image captured by the image capturing device, and an image from the image capturing device that is stored in the storage device at a first timing when the vehicle attempts to park and stop. A control device that compares an image provided from the imaging device with an image in the storage device at a second timing immediately before the vehicle departs, and determines whether there is an obstacle around the vehicle based on the comparison result. Things.
[0012]
The invention according to claim 2 is the vehicle surroundings monitoring device according to claim 1, wherein the first timing is when the parking signal is switched from an off state to an on state.
[0013]
The invention according to claim 3 is the vehicle periphery monitoring device according to claim 1 or 2, wherein the second timing is when an ignition signal is switched from an off state to an on state, or when a parking signal is transmitted. Is switched from the on state to the off state.
[0014]
According to a fourth aspect of the present invention, there is provided a vehicle periphery monitoring device that captures an image of the periphery of a vehicle using an imaging device and detects an obstacle based on the captured image. A storage device for storing an image captured by the device; and determining whether or not the vehicle is in a predetermined extremely low speed state based on vehicle speed information of the vehicle. Then, a comparison operation is performed on the difference between the pixel values of the predetermined sample points in the curved coordinates corresponding to the perspective from the imaging device, and it is determined whether or not the difference is a peculiar value exceeding a predetermined reference value. And a control device for determining that an obstacle has entered the traveling direction of the vehicle when the value is a peculiar value.
[0015]
According to a fifth aspect of the present invention, there is provided the vehicle periphery monitoring device according to any one of the first to third aspects, wherein the control device operates in a predetermined extremely low speed state based on vehicle speed information of the vehicle. It is determined whether or not there is, and in the case of the extremely low speed state, every time the vehicle travels by a certain distance, the difference between the pixel values of the predetermined sample points in the curved coordinates corresponding to the perspective from the imaging device is compared and calculated. Then, it is determined whether or not the difference is a peculiar value exceeding a predetermined reference value. If the difference is a peculiar value, it is determined that an obstacle has entered in the traveling direction of the vehicle. .
[0016]
According to a sixth aspect of the present invention, in the vehicle periphery monitoring device according to the fourth or fifth aspect, the curved coordinates are changed by a steering angle of the vehicle.
[0017]
The invention according to claim 7 is the vehicle periphery monitoring device according to claim 6, wherein the control device calculates a traveling direction of the vehicle based on the steering angle, and in advance, calculates the traveling direction in accordance with the traveling direction. It selects the data of the optimal curved coordinates from the plurality of coordinate conversion tables held, generates the curved coordinates based on the selected data, and calculates the difference between each pixel value for each of the sample points. is there.
[0018]
The invention according to claim 8 is the vehicle periphery monitoring device according to claim 6, wherein the control device calculates a traveling direction of the vehicle based on the steering angle, and sequentially calculates the traveling direction according to the traveling direction. The difference between the pixel values of each sample point is calculated while calculating the curved coordinates.
[0019]
According to a ninth aspect of the present invention, in the vehicle surroundings monitoring device according to any one of the first to eighth aspects, when the control device detects the presence or approach of an obstacle, it notifies the fact. It further comprises a notifying means.
[0020]
BEST MODE FOR CARRYING OUT THE INVENTION
<< 1st Embodiment >>
FIG. 1 is a diagram showing a vehicle periphery monitoring device according to a first embodiment of the present invention. As shown in FIG. 1, the vehicle periphery monitoring device includes an imaging device 1 for photographing the front of the vehicle, an A / D conversion device 3 for converting an analog signal related to an image output from the imaging device 1 into a digital signal, A display device 5 for displaying the image captured by the imaging device 1 to the driver, a speaker 7 for outputting a warning sound, a storage device 9 for storing the image captured by the imaging device 1, A control device 11 for performing image recognition processing on the image from the D-conversion device 3 and controlling other components 5, 7, 9 based on vehicle signals such as an ignition signal 13 and a parking signal 15; Thus, the control device 11 saves a plurality of images from the A / D conversion device 3 in the storage device 9 at appropriate timing, and compares these images to detect an obstacle.
[0021]
As the imaging device 1, a general CCD camera or the like that outputs a captured image as an analog signal is used, and as shown in FIG. 2, for example, a vehicle front bumper, It is arranged forward by a grill or the like above the front bumper of the vehicle. Here, the range 17 surrounded by oblique lines in FIG. 2 is a blind spot range that cannot be seen from the driver's seating position, and the imaging range of the imaging device 1 includes the blind spot range 17 so as to include the blind spot range 17. The installation position of the imaging device 1 is set.
[0022]
The A / D conversion device 3 converts an analog signal provided from the imaging device 1 into a digital signal.
[0023]
The display device 5 is a general display device such as a liquid crystal display panel. For example, the display device 5 also serves as a display unit of a car navigation device, and receives an image captured by the imaging device 1 and displays the image. It has become.
[0024]
The speaker 7 outputs an audio signal given from the control device 11, and in particular, can output a warning sound such as an electronic sound when the control device 11 detects an obstacle. I have.
[0025]
Note that the display device 5 and the speaker 7 function as a notification unit that notifies the controller 11 of the presence of an obstacle when the controller 11 detects the presence of the obstacle.
[0026]
As the storage device 9, a non-volatile rewritable storage medium having a sufficient storage capacity to store a plurality of images as electronic data is used. For example, a flash memory or a hard disk drive is used.
[0027]
The control device 11 compares the images captured by the imaging device 1 and converted into digital signals from the A / D conversion device 3 in a time-series manner, determines the presence or absence of an obstacle in front of the vehicle, and When it is determined that there is an object, a warning is issued through the speaker 7, and a plurality of images provided from the A / D converter 3 at appropriate timing based on vehicle signals such as the ignition signal 13 and the parking signal 15. A first function of storing the image in the storage device 9 as an image of the vehicle, reading out a plurality of images stored in the storage device 9, and immediately before starting from the parked / stopped state, an image taken at the previous parking / stopping time. A second function that determines whether there is a difference between the two images by comparing the current image with the current image that is about to be started, and determines that there is an obstacle when there is a difference; When it is determined that there is , And a third function to warn that effect to the driver through the speaker 7.
[0028]
The first function of the control device 11 is that the ignition signal 13 is kept on (ie, the engine is on), and the parking signal 15 is turned on from off (ie, the vehicle is parked). At the time (first timing) when the vehicle is switched to a state in which the vehicle is about to stop, the image provided from the A / D conversion device 3 is stored in the storage device 9 as the image when the vehicle is parked and stopped. Therefore, when the ignition key is subsequently operated and the ignition signal 13 is turned off, the image at the time when the vehicle attempts to park or stop has already been stored in the storage device 9.
[0029]
The second function of the control device 11 is when the ignition signal 13 is switched from the off state to the on state (that is, when the power supply system of the vehicle is started), or when the parking signal 15 is turned off from the on state. At any time (second timing) when the vehicle is switched to the state and the vehicle is about to depart, these states are determined to be the states immediately before the vehicle starts moving from the parked / stopped state, and based on the determination result, the storage is performed. A plurality of images stored in the device 9 are read out, and the image captured at the time of the previous parking and stopping is compared with the current image about to start, and whether there is a difference between the two images is determined. Is determined, and if there is a difference, it is determined that there is an obstacle.
[0030]
Here, as a method of comparing images in the second function, a plurality of sample points are set in advance in the coordinate space of the image, and a difference value between pixel values at each sample point of both images is obtained. If any of them has a large value exceeding a certain reference value, it is recognized that some kind of obstacle exists in the image.
[0031]
Alternatively, an edge may be detected for both images, and the result may be subjected to pattern recognition. If it is determined that both images have different patterns, it may be determined that an obstacle exists.
[0032]
The third function of the control device 11 warns the driver by, for example, sounding a buzzer sound or an electronic sound with the speaker 7 when the second function determines that there is an obstacle.
[0033]
As described above, the surrounding image before departure of the vehicle is compared with the surrounding image when the vehicle is parked and stopped before, the obstacle detection process is executed based on the comparison result, and if it is determined that there is an obstacle Since the warning sound is issued to the driver to notify the driver, it is possible to prevent the driver from starting the vehicle without noticing the obstacle before the vehicle starts moving.
[0034]
In addition, the image captured by the imaging device 1 is displayed on the display device 5 and the driver visually checks the image, so that the driver can easily confirm where the obstacle is in more detail.
[0035]
<< 2nd Embodiment >>
Next, a vehicle periphery monitoring device according to a second embodiment of the present invention will be described. FIG. 3 is a block diagram showing the configuration of the vehicle periphery monitoring device. This vehicle periphery monitoring device has a configuration similar to that of the first embodiment of FIG. 1, but the vehicle periphery monitoring device of the first embodiment detects an obstacle before departure. In addition to this, in the second embodiment, even in a very low speed state immediately after the vehicle starts to depart, it is determined whether or not an intruding object exists in the traveling direction of the vehicle. It is to detect.
[0036]
For obstacles approaching from the left or right in front of the vehicle, such as when a small animal such as a child, dog, cat, etc. approaches the vehicle before departure, the driver can view the front view through the windshield. Although it is possible to easily confirm the safety visually with the naked eye, if a child or small animal jumps out of the blind spot in the left and right direction in front of the vehicle, obstacles within the blind spot range will be seen from the driver. , And it may take time to see with the naked eye. In this case, if it is possible to automatically detect an ingress from the left and right directions ahead of the vehicle, it is easy to confirm safety before departure.
[0037]
Therefore, in the vehicle periphery monitoring device, an obstacle is detected based on an image captured by the imaging device 1 in an extremely low speed state immediately after the vehicle starts to depart.
[0038]
Here, FIG. 4 is a diagram showing an array of sample points when the road surface in front of the vehicle is viewed in plan, and FIG. 5 is a diagram showing an array of sample points when the road surface in front of the vehicle is obliquely viewed from the imaging device 1. is there. Each sample point a1, a2, a3, b1, b2, b3 in FIG. 5 corresponds to each sample point A1, A2, A3, B1, B2, B3 in FIG. The sample points a1, a2, a3, b1, b2, and b3 arranged in a grid in rectangular coordinates as shown in FIG. 4 when the road surface is viewed in a plane are represented by perspective images. Thus, it looks curved as shown in FIG. When the vehicle travels in the direction P in FIG. 4, the relative movement directions of the sample points A1, A2, A3, B1, B2, and B3 around the vehicle are actually the same as those in FIG. In the image captured by the image capturing apparatus 1, the relative movement direction of each sample point a1, a2, a3, b1, b2, b3 is left and right as shown in FIG. Appears to flow to. In particular, when a wide-angle lens or the like is used in the imaging device 1, due to the characteristics of the lens, the image is more curved toward the vehicle as shown in FIG.
[0039]
Here, when the vehicle travels straight by the distance L between the time T0 and the time T1 immediately thereafter, the pixels at the positions of the sample points a1 and b1 at the time T0 are sampled at the time T1 respectively. Move to the positions of points a2 and b2. Similarly, when the vehicle travels straight by the distance L between the time T1 and the time T2 immediately after the time T1, the pixels located at the sample points a2 and b2 at the time T1 are sampled at the time T2, respectively. Move to the positions of points a3 and b3. That is, every time the vehicle travels by the distance L, the pixel that has captured the same point on the ground as a1 → a2 → a3 moves.
[0040]
Therefore, the control device 11 determines whether or not the vehicle is in a predetermined extremely low speed state (for example, less than 5 km / h) based on vehicle speed information obtained from a vehicle speed meter 19 such as a wheel speed sensor that measures the number of rotations of a tire. In the extremely low speed state, each time the vehicle travels by a certain distance L, predetermined sample points a1, a2, a3, b1, b2, and b3 in curved coordinates corresponding to a perspective from the imaging device 1 (FIG. The pixel signal of 5) is fetched, and the difference (difference value) between the pixel value and the corresponding sample point of the previously fetched image is calculated. Then, among the difference values at all the sample points a1, a2, a3, b1, b2, and b3, if there is a point showing a peculiar value exceeding a certain threshold value, the front of the vehicle with respect to the ground It is determined that there is an obstacle (an intruding object) that has not stopped, and the detection of the intruding object is notified to the driver by a warning sound or the like.
[0041]
As described above, when the vehicle is in a very low speed state immediately after the vehicle starts to depart, an obstacle is detected based on the image captured by the image capturing apparatus 1, so that an intruding object from the left and right directions ahead of the vehicle is automatically detected. If it can be detected, safety confirmation before departure becomes easy. Therefore, for example, an obstacle that cannot be detected only by the function of the first embodiment, such as an obstacle approaching from the front of the vehicle, such as a small animal such as a dog or a cat approaching the vehicle before departure. Objects can be easily detected.
[0042]
In this case, in photographing while traveling straight, by paying attention to the regularity of the movement of the captured image in front of the vehicle and fixing sample points in advance, it is possible to reliably detect an intruding object with a small amount of calculation.
[0043]
At this time, since the detection of the distance L may include a measurement error, the average value or the median of the pixels around the sample point may be used as the value of the pixel at the sample point.
[0044]
In addition, this technique detects an obstacle or an intruding object by performing the same processing not only on the image captured in front of the vehicle but also on the image captured by, for example, a camera behind the vehicle or a side camera of the vehicle. It is possible to do. In this case, for example, the traveling direction of the vehicle is calculated based on the steering angle detected by the steering angle sensor, and in accordance with the traveling direction of the vehicle, the optimal curved coordinates are calculated from a plurality of coordinate conversion tables stored in advance. Data may be selected, a curved coordinate may be generated based on the selected data, and a difference value between the sample points a1, a2, a3, b1, b2, and b3 may be calculated. Alternatively, the difference value of each sample point a1, a2, a3, b1, b2, b3 may be calculated while sequentially calculating the curved coordinates according to the traveling direction of the vehicle.
[0045]
In each of the above embodiments, the forward direction or the diagonally forward direction of the vehicle has been described as an example of the traveling direction when the vehicle departs after parking and stopping.
[0046]
Further, in the second embodiment, in addition to the function of the first embodiment (comparison between before parking and stopping and immediately before departure), the approach determination of an obstacle immediately after departure is performed. The function of the first embodiment (comparison between before parking and stopping and immediately before departure) may be omitted.
[0047]
【The invention's effect】
According to the first aspect of the present invention, the image captured by the imaging device is stored in the storage device, and the image from the imaging device is stored in the storage device at the first timing when the vehicle is about to park and stop. At a second timing immediately before the vehicle departs, the image provided by the imaging device is compared with the image in the storage device, and the presence or absence of an obstacle around the vehicle is determined based on the comparison result. At this point, it is possible to easily and easily prevent the driver from starting without noticing the obstacle.
[0048]
Here, the first timing is, for example, when the parking signal is switched from the off state to the on state, and the second timing is, for example, when the ignition is turned on. The processing timing of the image from the imaging device can be easily set by the fact that the signal is switched from the off state to the on state or that the parking signal is switched from the on state to the off state.
[0049]
According to the invention described in claim 4 or claim 5, the control device determines whether or not the vehicle is in a predetermined extremely low speed state based on the vehicle speed information of the vehicle. Every time the distance travels, the difference between the pixel values of the predetermined sample points in the curved coordinates corresponding to the perspective from the imaging device is compared and calculated, and the difference is a peculiar value exceeding a predetermined reference value. It is determined whether an obstacle has entered in the traveling direction of the vehicle if the value is a peculiar value. Therefore, it is possible to automatically detect an approaching object from the left and right direction in front of the vehicle. Therefore, it is easy to confirm safety before departure, and even if an obstacle approaches from the left or right side in front of the vehicle, for example, a small animal such as a dog or a cat approaches the vehicle before departure, it can be easily performed. Can be detected.
[0050]
According to the invention described in claims 6 to 8, since the bending coordinates are changed based on the steering angle of the vehicle, the bending coordinates are changed in accordance with the traveling direction of the vehicle, so that an accurate obstacle can be detected. The presence or absence can be determined.
[0051]
According to the ninth aspect of the present invention, when the control device detects the presence or approach of an obstacle, the fact can be notified by the notification means, so that it is possible to provide a vehicle periphery monitoring device that is easier for the occupant to recognize.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a vehicle periphery monitoring device according to a first embodiment of the present invention.
FIG. 2 is a diagram illustrating a state in which an obstacle in a traveling direction of a vehicle is detected by a vehicle periphery monitoring device.
FIG. 3 is a block diagram showing a vehicle periphery monitoring device according to a second embodiment of the present invention.
FIG. 4 is a diagram showing sample points arranged in a grid pattern on coordinates of a road surface in plan view.
FIG. 5 is a diagram showing sample points arranged on a perspective curved coordinate from the imaging apparatus.
[Explanation of symbols]
REFERENCE SIGNS LIST 1 imaging device 3 A / D conversion device 5 display device 7 speaker 9 storage device 11 control device 13 ignition signal 15 parking signal 19 vehicle speedometer

Claims (9)

A vehicle periphery monitoring device that captures an image of the periphery of a vehicle with an imaging device and detects an obstacle based on the captured image,
An imaging device for imaging the periphery of the vehicle;
A storage device for storing an image captured by the imaging device;
An image from the imaging device is stored in the storage device at a first timing when the vehicle attempts to park and stop, and an image given from the imaging device at a second timing immediately before the vehicle departs is stored in the storage device. A vehicle periphery monitoring device comprising: a control device that compares an image in the device with an image and determines whether there is an obstacle around the vehicle based on a result of the comparison. The vehicle periphery monitoring device according to claim 1,
The first timing is when the parking signal is switched from an off state to an on state. The vehicle periphery monitoring device according to claim 1 or 2,
The second timing is provided when the ignition signal is switched from an off state to an on state, or when the parking signal is switched from an on state to an off state. A vehicle periphery monitoring device that captures an image of the periphery of a vehicle with an imaging device and detects an obstacle based on the captured image,
An imaging device for imaging the periphery of the vehicle;
A storage device for storing an image captured by the imaging device;
Based on the vehicle speed information of the vehicle, it is determined whether or not the vehicle is in a predetermined extremely low speed state. In the case of the extremely low speed state, every time the vehicle travels by a certain distance, a curved coordinate corresponding to a perspective from the imaging device. The difference between the pixel values of the predetermined sample points in the comparison is calculated, and it is determined whether or not the difference is a peculiar value exceeding a predetermined reference value. And a control device for determining that an obstacle has entered the traveling direction. The vehicle periphery monitoring device according to any one of claims 1 to 3, wherein
The control device determines whether or not the vehicle is in a predetermined extremely low speed state based on the vehicle speed information of the vehicle. The difference between the pixel values of the predetermined sample points in the curved coordinates corresponding to the calculated value is compared, and it is determined whether or not the difference is a peculiar value exceeding a predetermined reference value. A vehicle periphery monitoring device that determines that an obstacle has entered the traveling direction of a vehicle in such a case. The vehicle periphery monitoring device according to claim 4 or claim 5,
A vehicle periphery monitoring device, wherein the bending coordinates are changed according to a steering angle of the vehicle. The vehicle periphery monitoring device according to claim 6,
The control device calculates a traveling direction of the vehicle based on the steering angle, and selects, based on the traveling direction, optimal curved coordinate data from a plurality of coordinate conversion tables stored in advance. A curved coordinate is generated based on the obtained data, and a difference between each pixel value for each of the sample points is calculated. The vehicle periphery monitoring device according to claim 6,
The control device calculates a traveling direction of the vehicle based on the steering angle, and calculates a difference between each pixel value of each sample point while sequentially calculating a curved coordinate according to the traveling direction. Vehicle periphery monitoring device. A vehicle periphery monitoring device according to any one of claims 1 to 8,
A vehicle periphery monitoring device further comprising a notifying means for notifying when the control device detects the presence or approach of an obstacle.

Images