JP5602354B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring device that monitors the periphery of the vehicle based on an image captured by an imaging unit mounted on the vehicle.

  Conventionally, a pedestrian is detected from an image around the host vehicle captured by the in-vehicle camera, the possibility of contact between the host vehicle and the pedestrian is judged, and the driver is warned when the contact possibility is high. A vehicle periphery monitoring device is known.

  For example, a so-called stereo camera is provided, and the parallax between the image part in the image of the right camera and the image part in the image of the left camera for the same object is calculated, and the object and the vehicle are calculated using the parallax. As a configuration for calculating the distance, a vehicle periphery monitoring device has been proposed in which the possibility of contact between the object and the vehicle is determined based on the distance between the object and the vehicle thus calculated (for example, , See Patent Document 1).

  In addition, for a time-series image captured by one camera, the image portion of the same object is searched, and the vehicle reaches the object based on the rate of change of the size of the image portion between successive images. A vehicle periphery monitoring device has been proposed in which the time until the vehicle is estimated and the possibility of contact between the object and the vehicle is determined based on the time (see, for example, Patent Document 2).

In each of the vehicle periphery monitoring devices described in Patent Documents 1 and 2, correlation operations such as SAD (Sum of Absolute Difference) are performed between different images, and the image portion of the same object is obtained. As a result of the extraction, there is a disadvantage that the amount of calculation increases.
JP 2003-230134 A JP 2007-213561 A

  The present invention has been made in view of the above background, and an object thereof is to provide a vehicle periphery monitoring device that can reduce the amount of calculation when extracting image portions of the same object between different images. .

  The present invention has been made to achieve the above object, and a first aspect of the present invention is imaged by a first imaging unit and a second imaging unit which are mounted on a vehicle and have overlapping imaging ranges. The present invention relates to an improvement in a vehicle periphery monitoring apparatus that monitors the periphery of the vehicle based on the obtained image.

Then, when an object existing within a predetermined distance from the vehicle is to be monitored, the first image captured by the first imaging unit and the second imaging unit are captured at the same time. The second image is reduced at a reduction rate that ensures the distance resolution required at the predetermined distance, the first image portion of the object is extracted from the reduced first image, and the reduced second image is obtained. Corresponding image extraction means for extracting a second image portion having a correlation with the first image portion in the image, and based on the parallax between the first image portion and the second image portion, the first image portion and An object in real space corresponding to the second image portion; and distance calculation means for calculating a distance from the vehicle, wherein the corresponding image extraction means extracts from the reduced first image. The first image portion of the image is large according to the reduction ratio. While the of the search range set in the second image after the reduction, it has a size corresponding to the reduction ratio, and the position of the first image portion of the first in the reduced image And a second image portion having a correlation with the reduced first image portion is extracted within a search range sized according to the reduction ratio. To do.

  According to the present invention, when the distance between the vehicle and the object becomes short, the objects included in the first image and the second image captured by the first imaging unit and the second imaging unit. The amount of information in the image portion increases. Therefore, even if the first image and the second image are reduced, a certain degree of distance resolution can be ensured.

  Therefore, the corresponding image search means monitors the first image and the second image picked up by the first image pickup means at the same time when an object existing within a predetermined distance from the vehicle is to be monitored. The second image captured by the imaging means is reduced at a reduction rate that ensures the distance resolution required at the predetermined distance, and the first image portion of the object is extracted from the reduced first image. Then, a second image portion having a correlation with the first image portion is extracted from the reduced second image. As a result, the accuracy of the distance between the vehicle and the object calculated by the distance calculating means can be ensured, and the object can be monitored efficiently by reducing the amount of correlation calculation.

In the present invention, the corresponding image extraction means is present in a range of the predetermined distance or more which is not ensured in the first image and the second image after the reduction in distance resolution required at a predetermined distance or more from the vehicle to the object. When the object is to be monitored, a first image portion of the object is extracted from the non-reduced first image, and a second image having a correlation with the first image portion in the non-reduced second image A part can be extracted.

  Embodiments of the present invention will be described with reference to FIGS. FIG. 1 is a configuration diagram of a vehicle surroundings monitoring apparatus according to the present invention. The vehicle surroundings monitoring apparatus according to the present invention includes an image processing unit 1. The image processing unit 1 includes an infrared camera 2R (corresponding to the first imaging means of the present invention) that can detect far infrared rays, an infrared camera 2L (corresponding to the second imaging means of the present invention), and the yaw rate of the vehicle. , A vehicle speed sensor 4 for detecting the traveling speed of the vehicle, a brake sensor 5 for detecting the amount of brake operation by the driver, a speaker 6 for alerting by voice, and infrared cameras 2R, 2L A display device 7 is connected to display an image obtained by the above and display to make the driver visually recognize an object with a high possibility of contact.

  Referring to FIG. 2, infrared cameras 2R and 2L are disposed at the front portion of vehicle 10 at a substantially target position with respect to the center portion of vehicle 10 in the vehicle width direction, and two infrared cameras 2R and 2L are arranged. The optical axes are parallel to each other, and the infrared cameras 2R and 2L are fixed at the same height from the road surface. In this case, the visual field of the infrared camera 2R and the visual field of the infrared camera 2L are substantially the same.

  The infrared cameras 2R and 2L have a characteristic that the output level increases (the luminance increases) as the temperature of the imaged object increases. Further, the display device 7 is provided so that the screen 7 a is displayed at a front position on the driver side of the front window of the vehicle 10.

  Referring to FIG. 1, an image processing unit 1 is an electronic unit composed of a microcomputer (not shown) and converts analog video signals output from infrared cameras 2R and 2L into digital data. The microcomputer has a function of performing various arithmetic processes by the microcomputer with respect to the image in front of the vehicle captured in the image memory (not shown).

  Then, by causing the microcomputer to execute the vehicle monitoring program, the microcomputer performs a real space between the first image captured by the infrared camera 2R and the second image captured by the infrared camera 2L. Corresponding image extracting means 20 for extracting the image portion of the same object, the first image portion of the object extracted from the first image by the corresponding image extracting means 20 and the second image of the object extracted from the second image Distance calculating means 21 for calculating the distance between the object and the vehicle based on the parallax with the part, and contact determining means for determining the contact possibility between the vehicle and the object based on the distance between the object and the vehicle 22 functions.

  The arrival time calculation means 23 calculates the time until the object reaches the vehicle based on the time-series image captured by one camera, and is used in the second embodiment to be described later. Is done.

  [First Embodiment] First, the first embodiment will be described with reference to FIGS. FIG. 3A shows a first image 30 captured by the infrared camera 2R (right camera) and a second image 31 captured by the infrared camera 2L. The first image 30 includes a first image portion 30a that is an image portion of a pedestrian, and the second image 31 includes a second image portion 30b that is an image portion of the same pedestrian. Yes.

  FIG. 3B shows a reduced first image 40 obtained by reducing the first image 30 shown in FIG. 3A to 1/2 and a second image 31 shown in FIG. A reduced second image 41 reduced to 2 is shown. The reduced first image 40 includes a first image portion 40a corresponding to the first image portion 30a, and the reduced second image 41 includes a second image portion 41a corresponding to the second image portion 31a. It is.

  Next, FIG. 4A shows the parallax Dx (Dx1 in FIG. 3A) when the same object is imaged by the infrared camera 2R and the infrared camera 2L, and the distance Z between the vehicle and the object. The relationship is shown with the vertical axis set to parallax Dx and the horizontal axis set to distance Z.

  FIG. 4B shows the relationship between the distance resolution by the infrared cameras 2R and 2L (resolution when calculating the distance from the image) and the distance Z between the vehicle and the object, and the vertical axis is the distance resolution. The horizontal axis is set to the distance Z. In FIG. 4B, “a” shows the relationship when the reduction is not performed as shown in FIG. 3A, and “b” is the case when the reduction is made to ½ size as shown in FIG. Showing the relationship.

  4 (a) and 4 (b), it can be seen that when the distance Z to the object is short, the parallax Dx is increased and the distance resolution is increased. Therefore, when the distance Z to the object is short, the images captured by the infrared cameras 2R and 2L can be reduced within a range in which the required distance resolution is ensured.

  For example, when the distance resolution requirement is 10 m, when calculating the distance of an object existing in a range where the distance from the vehicle is 50 m or less (corresponding to the predetermined distance of the present invention), FIG. As shown in (b), parallax can be calculated by reducing the images captured by the infrared cameras 2R and 2L to ½ size.

  Accordingly, the corresponding image extraction means 20 performs the reduction shown in FIG. 3 (a) when calculating the distance to the object assuming the object existing in the range where the distance from the vehicle exceeds 50m. The process which extracts the image part of the same target object for the 1st image 30 and the 2nd image 31 which are not performed is performed.

  That is, the corresponding image extraction unit 20 first extracts from the image portions included in the first image 30 those that may be pedestrian image portions by performing pattern matching, feature amount extraction, and the like. In FIG. 3A, the first image portion 30a is extracted. Next, the corresponding image extraction unit 20 sets a search range 32 including a position corresponding to the position of the first image portion 30 a in the first image 30 in the second image 31.

  Then, the corresponding image extraction means 20 performs a correlation operation such as SAD (Sum of Absolute Difference) with the first image portion 30a on the search range 32 of the second image 31, thereby obtaining the second image portion 31a. Extract. In addition, the distance calculation unit 21 calculates the distance Z between the object and the vehicle by using the parallax Dx1 between the first image portion 30a and the second image portion 30b according to the following equation (1). Thereby, the calculation precision of the distance with the target which exists far away can be maintained highly.

  Where Z: distance between the vehicle and the object, D: baseline length of the infrared cameras 2R, 2L, F: focal length of the infrared cameras 2R, 2L, Dx: parallax, p: pixel pitch.

  Further, when calculating the distance of the object existing in the range in which the distance from the vehicle is 50 m or less, the corresponding image extraction unit 20 performs the reduction of the half size shown in FIG. A process of extracting the image portion of the same object is performed on the one image 40 and the reduced second image 41.

  That is, the corresponding image extraction unit 20 first extracts, from the image portions included in the reduced first image 40, those that may be pedestrian image portions by performing pattern matching, feature amount extraction, or the like. . In FIG. 3B, the first image portion 40a is extracted. Next, the corresponding image extraction unit 20 sets a search range 42 including a position corresponding to the position of the first image portion 40 a in the reduced first image 40 in the reduced second image 41.

  Then, the corresponding image extraction unit 20 extracts the second image portion 41 a by performing a correlation operation with the first image portion 40 a on the search range 42 of the reduced second image 41. Further, the distance calculation means 21 calculates the distance Z between the object and the vehicle by substituting the parallax Dx2 between the first image portion 40a and the second image portion 40b into Dx in the above equation (1).

  In this way, by extracting the image portion of the same object for the reduced first image 40 obtained by reducing the first image 30 and the reduced second image 41 obtained by reducing the second image 31, the correlation calculation is performed. The amount of calculation can be reduced and the parallax can be calculated efficiently.

  The contact possibility determination unit 22 alerts the buzzer 6 when the distance between the target object calculated by the distance calculation unit 21 and the host vehicle is within an approach determination region set in advance around the host vehicle. Sound output and a warning display on the display device 7 are performed.

  [Second Embodiment] Next, a second embodiment will be described with reference to FIG. In the second embodiment, the image processing unit 1 performs correlation calculation between images captured at different points in time by a single camera and extracts image portions of the same object, so that the object is automatically detected. An arrival time calculating means 23 for estimating the own vehicle arrival time, which is an estimated time until the vehicle is reached, is provided.

  As shown in FIG. 5, the arrival time calculation means 23 extracts the first image extracted from the first image 51 imaged at t2 before t1 with respect to the second image 50 imaged at a certain time t1. The correlation calculation is performed by enlarging the portion 51a.

  That is, the arrival time calculation means 23 includes the first image portion 51a, the enlarged first image portion 52a obtained by enlarging the first image portion 51a by 1.15 times, and the enlarged first image portion obtained by enlarging the first image portion 51a by 1.25 times. 53, a correlation calculation with the second image 50 is performed for each of the enlarged first image portions 54 obtained by enlarging the first image portion 51a by 1.35 times.

  Then, the second image portion 50a is extracted from the second image 50 with the highest degree of correlation among the first image portion 51a, the enlarged first image portion 52a, the enlarged first image portion 53a, and the enlarged first image portion 54a. The own vehicle arrival time T is obtained from the expansion rate Rate (corresponding to the change rate of the present invention). For example, when the highest correlation is obtained in the enlarged first image portion 53a, the own vehicle arrival time T is calculated by the following formula (2) using Rate = 1.25.

  However, T: own vehicle arrival time, dT: imaging interval, Rate: magnification rate.

  And the arrival time calculation means 23 assumes the target which exists in the range whose distance from a vehicle is 50 m or less, when the required distance resolution is 10 m similarly to 1st Embodiment mentioned above. When performing the correlation calculation, the first image portion 51a, the enlarged first image portion 52a, the enlarged first image portion 53a, the enlarged first image portion 54a, and the second image 50 are reduced to ½ size to correlate. By performing the calculation, the host vehicle arrival time T can be efficiently calculated while reducing the amount of calculation of the correlation calculation.

  Then, the contact possibility determination means 22 outputs the warning sound by the buzzer 6 and the warning display on the display device 7 when the own vehicle arrival time calculated by the arrival time calculation means 23 becomes a predetermined time or less. I do.

  Further, when the arrival time calculation means 23 performs the correlation calculation assuming an object existing in a range where the distance from the vehicle exceeds 50 m, the first image portion 51a, the enlarged first image portion 52a, and the enlarged first image By performing correlation calculation without reducing the portion 53a, the enlarged first image portion 54a, and the second image 50, the required distance resolution can be secured and the own vehicle arrival time can be calculated.

  In the present embodiment, the configuration in which the front of the vehicle is imaged is shown. However, the possibility of contact with the object may be determined by imaging other directions such as the rear and side of the vehicle. .

  Further, in the present embodiment, the infrared cameras 2R and 2L are used as the imaging means of the present invention, but a visible camera that captures a visible image may be used.

The block diagram of the vehicle periphery monitoring apparatus of this invention. Explanatory drawing of the attachment aspect to the vehicle of the vehicle periphery monitoring apparatus shown in FIG. Explanatory drawing of the process which extracts the image part of the same target object by correlation calculation. Explanatory drawing of the change of the distance resolution by the distance of a vehicle and a target object. Explanatory drawing of the process which calculates | requires the expansion rate of the image part of a target object by the correlation calculation between 2 images.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing unit, 2R, 2L ... Infrared camera (imaging means), 3 ... Yaw rate sensor, 4 ... Vehicle speed sensor, 5 ... Brake sensor, 6 ... Speaker, 7 ... Display apparatus, 20 ... Corresponding image extraction means, 21 ... Distance calculation means, 22 ... contact possibility determination means, 23 ... arrival time calculation means

Claims (2)

A vehicle periphery monitoring device that monitors the periphery of a vehicle based on images captured by a first imaging unit and a second imaging unit that are mounted on the vehicle and have overlapping imaging ranges,
When an object existing within a predetermined distance from the vehicle is to be monitored, a first image captured by the first imaging unit and a second image captured by the second imaging unit at the same time point The image is reduced at a reduction rate that ensures the distance resolution required at the predetermined distance, the first image portion of the object is extracted from the reduced first image, and the reduced image in the second image is extracted. Corresponding image extracting means for extracting a second image portion having a correlation with the first image portion;
Based on the parallax between the first image portion and the second image portion, distance calculation for calculating the distance between the object in real space corresponding to the first image portion and the second image portion and the vehicle Means and
The corresponding image extracting means sets the size of the first image portion of the object to be extracted from the reduced first image according to the reduction ratio and is set in the reduced second image. The range is set to a range having a size corresponding to the reduction ratio and including a position corresponding to the position of the first image portion in the reduced first image, and the size corresponding to the reduction ratio A vehicle periphery monitoring device , wherein a second image portion having a correlation with the reduced first image portion is extracted within the specified search range .   The corresponding image extracting means monitors an object existing in a range of the predetermined distance or more that is not ensured in the first image and the second image after the reduction in distance resolution required at a predetermined distance or more from the vehicle to the object. When the target is selected, the first image portion of the object is extracted from the non-reduced first image, and the second image portion having a correlation with the first image portion is extracted from the non-reduced second image. The vehicle periphery monitoring device according to claim 1.

Images