JP2011131740A - Driving support device of vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an inexpensive driving support device of a vehicle having high convenience. <P>SOLUTION: This driving support device includes at least a pair of imaging means 1 and 2 mounted on a vehicle and arranged in the different direction so that the overlapping area of an imaging range exists in part in the horizontal direction and an image signal processing means 10 for performing processing of an output image signal of the imaging means 1 and 2. The image signal processing means 10 includes an object detecting part 14 for respectively detecting an object in the overlapping area from two output image signals taken in by the pair of imaging means 1 and 2, a distance calculating part 15 for calculating the distance from the vehicle to the object detected by the object detecting part 14 using the two output image signals and an overlooking image generating part 13 for generating an overlooking image by synthesizing the output image signals taken in from the imaging means 1 and 2. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a driving support apparatus for a vehicle that monitors the periphery of the host vehicle using an imaging means mounted on the vehicle.

  2. Description of the Related Art Conventionally, a vehicle driving support device has been put to practical use in which an imaging unit is mounted on a vehicle and an image captured by the imaging unit is displayed on an in-vehicle monitor to monitor the periphery of the host vehicle. Furthermore, in recent years, a technique has been proposed and put into practical use that increases the convenience and added value in monitoring the surroundings of a vehicle by performing specific image processing on an image captured by an imaging unit.

  For example, as shown in FIG. 10, the front image captured by the imaging unit 51 installed in front of the vehicle 50, the side images captured by the imaging units 52 and 53 installed on the left and right sides of the vehicle 50, and the vehicle 50 A device that obtains a bird's-eye view image around the vehicle as shown in FIG. 11 by converting the viewpoint of the rear image captured by the imaging means 54 installed behind the vehicle to an image viewed from above and combining the images. It has been. Then, by superimposing the position of the own vehicle on the obtained bird's-eye view image, it becomes possible to operate the vehicle while objectively grasping the positional relationship between the own vehicle and the other vehicles 61, 62 and the white line 60, In particular, it is suitably used for a device that supports a parking operation.

  As such a parking assistance device, in Patent Document 1 (Japanese Patent No. 4156181), a target parking lot is imaged by a side camera, and a vehicle traveling direction is imaged by a rear camera while the vehicle is moving backward, and an image of the target parking lot is obtained. And an image of the vehicle traveling direction are detected, and an apparatus for synthesizing an image of the own vehicle on the image of the target parking lot is disclosed.

  There is also known an apparatus for detecting a distance from an object using images picked up by a plurality of image pickup means. For example, Patent Literature 2 (Japanese Patent No. 3965078) discloses an out-of-vehicle monitoring device that detects a distance from an object using images captured by a plurality of stereo cameras. In order to reduce the image processing load, this out-of-vehicle monitoring device detects the running state of the vehicle with a sensor, selects one of the stereo cameras based on the sensor signal, and performs image processing.

Japanese Patent No. 4156181 Japanese Patent No. 3965078

However, in the device described in Patent Document 1, the relative positional relationship between the vehicle and the surrounding object can be grasped, but the specific distance between the vehicle and the object cannot be detected, and the surrounding obstacle In order to avoid collisions with objects, the driver had to rely on image confirmation.
On the other hand, in the apparatus described in Patent Document 2, a distance from an object around the vehicle can be detected, but an overhead image cannot be obtained. In addition, although it is possible to reduce the cost of image processing by performing image processing only on images picked up by the selected stereo camera, it is necessary to install a plurality of expensive stereo cameras, and the cost of the device itself is It was not possible to reduce.

  Therefore, an object of the present invention is to provide a low-cost and highly-convenient driving support apparatus for a vehicle that can obtain a bird's-eye view around the vehicle and can measure the distance to the object.

  In order to solve the above-described problems, a vehicle driving support device according to the present invention is mounted on a vehicle and has at least one pair arranged in different directions so that there is an overlapping region of imaging ranges in a part of the horizontal direction. An image pickup means; and an image signal processing means for processing an output image signal of the image pickup means, wherein the image signal processing means is located within the overlapping region from two output image signals captured from the pair of image pickup means. An object detection unit that detects each of the objects, a distance calculation unit that calculates a distance from the vehicle to the object detected by the object detection unit using the two output image signals, And an overhead image generation unit that generates an overhead image by synthesizing output image signals captured from the imaging unit.

According to the present invention, by providing at least one pair of imaging means arranged in different directions so that there is an overlapping area in the imaging range, an overhead image around the vehicle is obtained and the distance to the object is measured. Therefore, it is possible to provide a low-cost and highly convenient driving support device.
That is, since two image signals picked up by a pair of image pickup means have overlapping regions, it is possible to measure the distance to the object using the stereo method. In addition, since the pair of imaging units are arranged in different directions, there is a region that does not overlap the image signal. Therefore, a bird's-eye view image around the vehicle can be obtained by combining the two image signals by viewpoint conversion. Thus, since the present invention can acquire both the distance between the vehicle and the object and the bird's-eye view image around the vehicle, the convenience can be improved. In addition, since the pair of imaging units is used for both the overhead image generation and the distance measurement, the number of imaging units can be minimized and the cost can be reduced.

Moreover, it is preferable to further have a distance information addition unit that reflects the distance calculated by the distance calculation unit in the overhead image generated by the overhead image generation unit.
In this way, by reflecting the distance to the object in the overhead view image, the driver can grasp the situation around the vehicle from the overhead view image around the vehicle and at the same time grasp the specific distance from the object. This makes it possible to provide a convenient and safe driving support apparatus.

Furthermore, it is preferable that the pair of imaging means is arranged on one wall side of the vehicle body facing the overlapping area.
In this configuration, since the overlapping region exists at a position facing the one wall side of the vehicle body, the overlapping region is located at least one of the front, rear, left side, and right side of the vehicle. It is considered that an object to be particularly noted in driving operation is an object located in the front, rear, left side, and right side of the vehicle rather than the object located in the oblique direction of the vehicle body. For example, at the time of parking, it is required to accurately grasp the distance to a vehicle stop, an obstacle, a person, or the like located behind or on the side of the vehicle. Further, when the vehicle is running, it is required to accurately grasp the passing distance with other vehicles on the side of the vehicle and the inter-vehicle distance with other vehicles in front of the vehicle. Therefore, as in this configuration, a pair of imaging means is arranged on the one wall side of the vehicle body facing the overlapping area, thereby measuring the distance to the object located at the front, rear, left side, and right side of the vehicle. This makes it possible to provide a driving support device that is more convenient in driving operation.

  The imaging means preferably has a fish-eye lens, so that the overlapping area of the imaging range can be increased, and the distance detection range of the object can be increased.

Furthermore, it is preferable that the image signal processing means further includes a warning output unit that outputs a warning according to the distance of the object calculated by the distance calculation unit.
Thus, by outputting a warning according to the distance of the target object, it is possible to actively alert the driver, and a safer driving support apparatus can be obtained.

Furthermore, it is preferable that the image signal processing unit further includes a host vehicle position superimposing unit that superimposes the position of the vehicle on the overhead image generated by the overhead image generating unit.
In this way, by superimposing the position of the host vehicle on the bird's-eye view image, it becomes possible to grasp the relative positional relationship between the object around the vehicle and the host vehicle, and to make the driving support device more safe. Can do.

  According to the present invention, by providing at least one pair of imaging means arranged in different directions so that there is an overlapping area in the imaging range, an overhead image around the vehicle is obtained and the distance to the object is measured. Therefore, it is possible to provide a low-cost and highly convenient driving support device.

1 is an overall configuration diagram of a driving support apparatus according to an embodiment of the present invention. It is a schematic diagram which shows the arrangement configuration and imaging range of an imaging means. It is a schematic diagram which shows the overlapping area | region of an imaging range. It is a figure which shows the processing flow of the driving assistance device which concerns on embodiment of this invention. It is a figure explaining an image correction process. It is a figure explaining distortion correction, and is a figure showing subject P expressed on a three-dimensional coordinate system. It is a figure explaining distortion correction, and is a figure showing mapping point P 'expressed to a two-dimensional coordinate system. (A) is an image before distortion correction, and (B) is an image after distortion correction. It is a figure which shows an example of the bird's-eye view image which added distance information and the own vehicle position. It is a top view of the vehicle which shows the arrangement configuration of the conventional imaging means. It is a figure which shows the conventional bird's-eye view image around a vehicle.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the shape of the component described in this example is not intended to limit the scope of the present invention, but is merely an illustrative example.

FIG. 1 is an overall configuration diagram of a driving support apparatus according to an embodiment of the present invention. The vehicle driving support apparatus mainly includes imaging means 1 and 2 mounted on the vehicle, and an image signal processing means 10 connected to the imaging means 1 and 2.
The imaging means includes at least a pair of imaging means 1 and 2 that are mounted on the vehicle and arranged in different directions so that there is an overlapping area of the imaging range in a part in the horizontal direction. In addition, the pair of imaging means 1 and 2 is preferably arranged on one wall side of the vehicle body facing the overlapping area, thereby measuring the distance between the front, rear and left and right side objects of the vehicle. Can be made more convenient. Furthermore, it is preferable that the imaging means 1 and 2 have a fish-eye lens, whereby an overlapping area of the imaging range can be taken large and the distance detection range of the object can be enlarged.

In FIG. 2, as an example, a configuration in which four imaging units 1 to 4 are arranged at the corners of the vehicle 5 is shown. These image pickup means 1 to 4 are arranged so as to be inclined at 45 ° in the horizontal plane with respect to the direction of the vehicle body (arrow X in the figure), and to face the left and right diagonally forward and the left and right diagonally backward, respectively. Thereby, an area 6 indicated by hatching in the figure becomes an imaging area by the four imaging units 1 to 4. As described above, it is preferable to dispose the imaging means 1 to 4 at the four corners of the vehicle body by shifting the direction by 90 °, respectively, so that all directions around the vehicle can be imaged.
When these imaging means 1 to 4 are provided with fisheye lenses having a viewing angle of 180 °, as shown in FIG. 3, regions 9 located on the front, rear, left and right sides of the vehicle 5 overlap the imaging range. A region 8 located diagonally forward and diagonally rear is a region where imaging ranges do not overlap.

Returning to FIG. 1, the image signal processing means 10 mainly includes a frame memory 11, an image correction unit 12, an overhead image generation unit 13, an object detection unit 14, a distance calculation unit 15, and a display setting unit 17. Have
The frame memory 11 stores image signals input from the imaging units 1 and 2 via an interface (not shown).
The image correction unit 12 performs a process of correcting a plurality of image signals captured by the imaging units 1 and 2 into a planar image. For example, distortion correction processing and perspective projection conversion processing are performed on image signals captured by the imaging means 1 and 2 having fisheye lenses.

The bird's-eye view image generation unit 13 performs viewpoint conversion processing on the images corrected by the image correction unit 12 to form a planar image, and generates a bird's-eye view image around the vehicle viewed from above the vehicle.
The object detection unit 14 detects the same object from two image signals imaged by the pair of imaging units 1 and 2. Specifically, object extraction processing, matching, region setting, and the like are performed.
The distance calculation unit 15 calculates the distance of the object detected by the object detection unit 14 using the two image signals captured by the pair of imaging units 1 and 2.
The display setting unit 17 sets an object whose distance is to be detected.
Further, the image signal processing means 10 may include a distance information adding unit 16 that reflects the distance calculated by the distance calculating unit 15 in the overhead image generated by the overhead image generating unit 13.

Specific processing of each unit described above will be described later.
Note that the image signal processing means 10 includes a calculation device including a CPU and a storage medium in which a software program for realizing the functions of the above-described units is recorded, and the calculation device is stored in the storage medium as necessary. As a matter of course, the functions of the respective units are realized by reading and executing.

Moreover, it is preferable that the driving assistance apparatus of the present embodiment further includes an input unit 20 and a display unit 21.
The input unit 20 performs various settings for the image signal processing unit 10. For example, an object whose distance is measured from an object (including a person) around the vehicle is selected via the display setting unit 17, or whether or not a warning is output is set in a warning output unit 18 described later.
The display means 21 displays an overhead image generated by the overhead image generation unit 13 or an image obtained by adding distance information to the overhead image. As the display means 21, a car navigation monitor or the like is used.

Next, with reference to FIG. 4, the process flow of the driving assistance apparatus which concerns on embodiment of this invention is demonstrated. Here, as an example, it is assumed that the imaging units 1 to 4 include fisheye lenses having a viewing angle of 180 ° and are disposed at the four corners of the vehicle body as shown in FIG.
The output image signals of the imaging means 1 to 4 are stored in the frame memory 11 of the image signal processing means 10 and read out as needed to perform various arithmetic processes.
As shown in FIG. 5, the image signals of the imaging means 1 and 2 stored in the frame memory 11 are each subjected to distortion correction processing by the image correction unit 12 and then subjected to perspective projection conversion processing.

The distortion correction process is performed as follows, for example.
As shown in FIGS. 6 and 7, a mapping p ′ (u, v) of a subject P (x, y, z) in the xyz space onto the xy plane and the uv plane (captured image) is considered.
When the point P is represented by polar coordinates (r, θ, φ) in the three-dimensional coordinate system of FIG.

On the other hand, the image height L of the mapping point P ′ is determined by the lens projection method.
Assuming that the focal length f of the lens and the radius of the captured circular image are R, since R is equal to the image height L at φ = π / 2, the following formula 2 is established.

In the two-dimensional coordinate system shown in FIG.

From Equation 3, the following Equation 4 is obtained by using the trigonometric function.

From these, the following formula 5 can be derived.

Using Equation 5, the conversion from (x, y, z) to the coordinate point (u, v) of the image signal captured by the fisheye lens can be performed. Since the z-axis component corresponds to the zoom rate, basically, the two-dimensional two-dimensional conversion from (x, y) to (u, v) is performed.
By performing distortion correction processing in this way, a flattened image as shown in FIG. 8B is obtained from an image signal captured by a fisheye lens as shown in FIG.

The perspective projection conversion process is performed as follows, for example.
The mapping P ′ (u, v) of the subject P (x, y, z) on the xyz space onto the uv plane (captured image) parallel to the xy plane is an equal area (equal solid angle) projection. It is expressed as the following Equation 6.

As described above, by performing the distortion correction process and the perspective projection conversion process in the image correction unit 12, two plane images on the same plane can be obtained.
Next, using these two image signals, the distance between the vehicle and the object is calculated by the following processing flow.
First, the object detection unit 14 detects the same object specified from two image signals. In the detection of the object, object recognition is performed in each of the two image signals, an object estimated to correspond to the object is extracted from each image signal, and an area including the object is set. The two objects set in this region are matched, and when the two objects are the same, this is determined as the object, and the distance calculation unit 15 uses the two image signals to determine the distance from the object by the stereo method. calculate.

The details of the specific processing are shown below.
Object extraction is a process of extracting a specific object from an image signal. The object extraction method is not particularly limited. For example, when the degree of correlation with a reference image pattern registered in advance is calculated using a feature amount such as color or shape, and the degree of correlation is larger than a threshold value, the object is extracted. For example, a method for detecting
Specifically, object extraction using Haar-like features and HOG features is suitable for pedestrian recognition, and object extraction using SIFT features is used for objects such as signs. These are all known methods. The object detection unit 14 has a plurality of programs for executing object extraction processing, and may use different programs depending on the type of object to be detected, or may detect all objects with one program. .

As an example of object extraction, a method using a Haar-like feature quantity and an AdaBoost classifier will be described. The AdaBoost classifier is a strong classifier created by combining a number of weak classifiers that do not have high discrimination capability with weighting according to their importance.
The Haar-like feature value is a feature value obtained by obtaining a brightness difference between two adjacent rectangular areas, and is known as a feature value that is not easily affected by variations in illumination conditions or noise. That is, when a predetermined detection window is placed on the image, the feature amount of the image in the detection window is a value obtained by subtracting the sum of the pixel values of the white area of the rectangular feature from the sum of the pixel values of the black area of the rectangular feature. It becomes. This black and white pattern of the rectangular area is called a Haar-like feature and has a plurality of patterns in advance. A number of object detection features are used depending on the combination of the position of the rectangle in the detection window and the pattern. Each of these feature patterns serves as a weak classifier of the AdaBoost classifier. Then, the object can be detected by scanning the image with the detection window and discriminating with the AdaBoost classifier each time.

For the matching, for example, paying attention to the feature quantity of the object extracted in each image signal taken by the pair of imaging means 1 and 2, a fixed threshold value is provided, and the difference of the feature quantity is within the threshold value. For example, a method of regarding what is contained as the same object is used.
As another method for detecting an object, object recognition is performed using an output image signal of one of the pair of imaging means 1 and 2 to extract an object that is estimated to correspond to the object. Set the area including. Further, the same object as the area set object is searched (matched) from the output image signal of the other imaging means, and the area setting including the searched object is performed. Thereby, the same target object can be detected from each of the two images.
The distance calculation uses two images on the same plane corrected by the image correction unit 12 and calculates the distance to the object based on the parallax between the objects on each image, using a known stereo method. Distance method can be applied.

On the other hand, the overhead image generation unit 13 generates an overhead image using a plurality of images corrected by the image correction unit 12.
First, a plurality of images corrected by the image correction unit 12 are subjected to viewpoint conversion processing, and then combined to generate an image around the vehicle viewed from above the vehicle.
The viewpoint conversion process is a known method for converting the image signal captured by the imaging means 1 and 2 mounted on the vehicle into an image as seen from an arbitrary virtual viewpoint above the vehicle.
After converting each image into a bird's-eye view image by viewpoint conversion processing, these images are combined.
Then, the distance information with respect to the object calculated by the distance calculation unit 15 is added to the overhead image generated by the overhead image generation unit 13.
Note that the display setting unit 17 may select an object whose distance is to be displayed.

As described above, according to the present embodiment, an overhead view image around the vehicle is acquired by including at least one pair of imaging units 1 and 2 arranged in different directions so that an overlapping region exists in the imaging range. At the same time, the distance to the object can be measured, and it is possible to provide a low-cost and highly convenient driving support device.
That is, since the image signals captured by the pair of imaging units 1 and 2 have overlapping images, the distance calculation unit 15 can measure the distance to the object using the stereo method. In addition, since the pair of imaging units 1 and 2 are arranged in different directions, there is a region that does not overlap the image signal. Therefore, the bird's-eye view image around the vehicle can be acquired by converting the viewpoints of these two image signals by the overhead image generation unit 13 and combining them.

  In addition, the pair of imaging units 1 and 2 are preferably arranged on one wall side of the vehicle body facing the overlapping region, whereby the overlapping region is located at least one of the front, rear, left side, and right side of the vehicle 5. Can be measured, and the distance to the object existing in this overlapping area can be measured, so that it is possible to provide a driving assistance device that is more convenient in driving operation.

  Then, the distance information adding unit 16 reflects the distance of the object calculated by the distance calculating unit 15 on the bird's-eye view image generated by the bird's-eye view image generating unit 13, so that the driver can view the vehicle in the bird's-eye view around the vehicle. While being able to grasp the surrounding situation, it is possible to simultaneously grasp the specific distance from the object, and it is possible to provide a highly convenient and safe driving support device. In addition, since the pair of imaging units 1 and 2 are used for both the overhead image generation and the distance measurement, the number of the imaging units 1 and 2 can be minimized and the cost can be reduced.

Further, the image signal processing means 10 may have a warning output unit 18 that outputs a warning according to the distance of the object calculated by the distance calculation unit 15.
For example, the warning output unit 18 sets a threshold value for the distance to the object, and when the distance to the object approaches the threshold value, the warning means 18 displays a warning on the display means 21 or outputs a sound. A warning is output by means of means 22, and a warning is issued to the driver.
Thus, by outputting a warning by the warning output unit 18 according to the distance of the target object, it becomes possible to actively alert the driver, and a safer driving support apparatus can be obtained. .

Furthermore, the image signal processing means 10 may include a host vehicle position superimposing unit 19 that superimposes the vehicle position on the overhead image generated by the overhead image generating unit 13.
For example, the position of the host vehicle is superimposed as follows.
The overhead image generation unit 13 obtains an overhead image around the vehicle, that is, the overhead image of the portion indicated by the region 6 in FIG. indicate. The vehicle body image may be a vehicle body captured image obtained by imaging the host vehicle from above, or may be a vehicle body image image drawn by a graphic or the like. The vehicle body image is stored in advance in a storage unit (not shown). The overhead image generation unit 13 superimposes the vehicle body image on the overhead image.
The overhead image generated by the overhead image generation unit 13 is a blank area outside the imaging range of the imaging means 1 to 4 (for example, the vehicle position), but interpolation processing is performed with captured images captured at other times. May be.
In this way, by superimposing the host vehicle position on the overhead image in the host vehicle position superimposing unit 19, it is possible to grasp the relative positional relationship between the object around the vehicle and the host vehicle, and the safety is further improved. It can be set as a high driving assistance device.

FIG. 9 is a diagram illustrating an example of an overhead image to which distance information and the vehicle position are added. In the bird's-eye view image including the own vehicle 5, the distance from the own vehicle 5 to the person 31 (here, 10 m) is displayed in the balloon frame in the vicinity of the person 31 as the object. Further, a distance (here 30 m) from the own vehicle 5 to the vehicle 32 is displayed in the balloon frame in the vicinity of the target vehicle 32.
In this way, by adding distance information to the bird's-eye view image, the driver can grasp the specific distance from the target object while grasping the surrounding situation, and a convenient and safe driving support device and can do.

1-4 Imaging means 10 Image signal processing means 11 Frame memory 12 Image correction unit 13 Overhead image generation unit 14 Object detection unit 15 Distance calculation unit 16 Distance information addition unit 17 Display setting unit 18 Warning output unit 19 Own vehicle position superposition unit 20 Input means 21 Display means

Claims (6)

At least one pair of imaging means mounted on the vehicle and arranged in different directions so that there is an overlapping area of the imaging range in a part of the horizontal direction;
Image signal processing means for processing the output image signal of the imaging means,
The image signal processing means uses an object detection unit for detecting each object in the overlapping area from two output image signals captured from the pair of imaging means, and the two output image signals. A distance calculation unit that calculates a distance from the vehicle to the object detected by the object detection unit; and an overhead image generation unit that generates an overhead image by combining output image signals captured from the imaging unit; A driving support apparatus for a vehicle, comprising:   The vehicle driving support device according to claim 1, further comprising a distance information adding unit that reflects the distance calculated by the distance calculating unit in the overhead image generated by the overhead image generating unit.   The vehicle driving support device according to claim 1, wherein the pair of imaging units are arranged on one wall side of a vehicle body facing the overlapping region.   The vehicle driving support apparatus according to any one of claims 1 to 3, wherein the imaging unit includes a fisheye lens.   The said image signal processing means further has a warning output part which outputs a warning according to the distance of the said target object calculated by the said distance calculation part, The Claim 1 thru | or 4 characterized by the above-mentioned. Vehicle driving support device.   The said image signal processing means further has the own vehicle position superimposition part which superimposes the position of the said vehicle on the bird's-eye view image produced | generated by the said bird's-eye view image generation part, The one of Claim 1 thru | or 5 characterized by the above-mentioned. The vehicle driving support apparatus according to claim.

Images