JP2010191637A - Driving support device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce any labor to perform processing for searching for a corresponding point corresponding to the feature points of a reference image. <P>SOLUTION: A feature point search regions 42 for suburbs of a reference image memory 20 is divided into 8 divisions 42-1 to 42-8. A corresponding point search range 44 of a reference memory 22 is divided into 8 divisions 44-1 to 44-8. Each of the divisions is given priority corresponding to collision risks, and the search of feature points is performed in the search region division 42-1 and 42-2 of the first order (S4), and when the feature points have been extracted, the search of the corresponding points is performed in the corresponding point search divisions 44-3 and 44-4 for suburbs of the first priority of the reference memory 22 (S7). A distance L to an obstacle is calculated based on the parallax of the extracted feature points and corresponding points (S11). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle driving support device, and more particularly to a vehicle driving support device that calculates a distance to an obstacle according to the principle of triangulation.

  As one means for improving vehicle safety, a plurality of cameras arranged separately from each other are prepared, and parallax is obtained from image data captured from the plurality of cameras. A driving support device that measures a distance to an object is known (for example, Patent Documents 1 and 2).

  In Patent Document 1, parallax is obtained by detecting points and corresponding areas corresponding to each other from image data taken from cameras separated from each other on the left and right sides, and the side corresponding to the parallax and the side corresponding to the lateral position are long. It proposes distinguishing and determining a road surface and a three-dimensional object using a parallax map in which a plurality of blocks defined in the above are arranged.

  Patent Document 2 divides an image captured from a plurality of cameras separated in the horizontal direction into regions of a predetermined size, extracts a region including an edge, and the direction of the edge in the extracted region And the parallax for each area is obtained by detecting the area having the highest degree of coincidence with respect to the vertical and diagonal areas of the edge direction, and triangulation is obtained from the positional relationship between the parallax and the two left and right cameras. Based on the principle, it is proposed to calculate the distance to an obstacle existing in each area and to determine a block of areas calculated as the same distance as one obstacle.

JP 2006-236104 A JP 2000-207693 A

  As can be seen from the cited references 1 and 2, feature points are extracted from image data (hereinafter referred to as “reference image” or “reference image data”) captured from one camera (hereinafter referred to as “reference camera”). By searching for the corresponding point corresponding to the feature point from the image data (hereinafter referred to as “reference image” or “reference image data”) of the other camera (hereinafter referred to as “reference camera”), A method for obtaining the parallax is employed.

  A conventional general method of searching for corresponding points from reference image data is performed by scanning the reference image in the Y direction and repeating this scan from the top to the bottom in the X direction.

  However, in order to extract the corresponding points from the reference image, substantially the entire region of the reference image is scanned, so that there is a problem that it takes time to extract the corresponding points and calculate the parallax associated therewith.

  An object of the present invention is to provide a vehicle driving support device that can reduce the burden of executing processing for searching for corresponding points corresponding to feature points of a reference image and can reduce the time for searching for corresponding points. .

According to the present invention, the above technical problem is
The image processing apparatus includes at least first and second imaging units, searches for feature points of an obstacle from an image captured by the imaging unit, and searches for corresponding points corresponding to the feature points. In the vehicle driving support device that calculates the distance to the obstacle based on the parallax of
A reference memory for storing reference image data imaged by the first imaging means;
A reference memory for storing reference image data captured by the second imaging means;
A feature point search means for searching a feature point from a feature point search region, which is a region for searching a feature point from reference image data stored in the reference memory and limited to a part of the reference image;
A corresponding point search range for searching for a corresponding point from the reference image data stored in the reference memory corresponds to the feature point within a search range set for each search direction determined by the position of the reference image of the feature point. Corresponding point search means for searching for corresponding points;
The vehicle has a distance calculation unit that calculates a distance to an obstacle by parallax based on the feature point searched by the feature point search unit and the corresponding point searched by the corresponding point search unit. This is achieved by providing a driving assistance device. That is, the search range for corresponding points is limited to a predetermined range where obstacles may exist, and the search for corresponding points is performed only within that range. It is possible to reduce the time for searching for corresponding points as well as reducing the time.

In a preferred embodiment of the present invention,
The feature point search means for searching for feature points from the reference image data stored in the reference memory is divided into a plurality of sections, and each section is assigned a search rank, and the first rank Search for feature points in order. As a result, not only the corresponding points but also the burden of executing the process of searching for feature points can be reduced, and the feature point search processing time can be reduced.

In a preferred embodiment of the present invention,
Driving environment detection means for detecting the driving environment of the host vehicle;
Search area setting means for setting the size of the feature point search area according to the driving environment detected by the driving environment detection means. According to this, it is possible to search for feature points under a search area having a size suitable for the driving environment.

In a preferred embodiment of the present invention,
According to the driving environment detected by the driving environment detecting means, the information processing apparatus further comprises a section number setting means for setting the number of sections of the feature point search area. According to this, for example, in a complicated urban area, the detection accuracy of feature points can be improved by increasing the number of sections.

In a preferred embodiment of the present invention,
The corresponding point search area is set corresponding to the section of the feature point search area, and the search range of the corresponding point is set for each of the sections. In this way, by setting the search range of corresponding points for each category, an appropriate search range is set for each category with different search directions and different distances to obstacles. The detection processing burden can be reduced and the accuracy can be improved.

1 is a plan view of a vehicle to which an embodiment is applied. It is a figure showing the whole system of the driving support device of an example in a block diagram. It is a figure for demonstrating the method of calculating | requiring the distance to an obstruction by parallax according to the principle of triangulation using two cameras. It is a figure for demonstrating the feature point search area | region map and division employ | adopted at the time of a suburb drive. It is a figure for demonstrating the corresponding point search range map and division employ | adopted at the time of a suburb drive. It is a figure for demonstrating the feature point search area | region map and division employ | adopted at the time of driving | running | working in an urban area. It is a figure for demonstrating the corresponding point search range map and division employ | adopted at the time of urban area driving | running | working. In order to describe a specific example of control for issuing a warning when there is a danger of collision by determining the collision risk based on the distance to the obstacle by parallax following the extraction of the feature points and the corresponding points corresponding thereto. It is a flowchart of.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In FIG. 1, an automobile VC as a vehicle has left and right front wheels 1F and left and right rear wheels 1R. The vehicle VC has first and second images of the front space through the front window glass. First and second CCD cameras 2 and 4 as left and right imaging means, and the first and second cameras 2 and 4 are separated in the horizontal direction (for example, sandwiching a room mirror with the left and right sides thereof). Is arranged). The first and second first and second imaging means are not limited to CCD cameras, and may be image sensors such as CMOS cameras, for example.

  A navigation system 8 is disposed in the vehicle VC, and the navigation system 8 visually provides map information to the driver, as is well known. Further, on the front window glass, driving support information generated by the window shield display system 10 is displayed by a projector in the front part of the driver's seat 12, and the driving support information displays the obstacle and the risk of collision. An alarm display is included. The vehicle VC is equipped with a GPS reception system 14.

  FIG. 2 is a block diagram showing the entire control system including a controller U (control unit) configured by using a microcomputer constituting a part of the driving support apparatus. Imaging data is input to the controller U from the right camera 2 and the left camera 4. Here, the right camera 2 constitutes a reference camera, and the left camera 4 constitutes a reference camera. However, the right camera 2 may be used as a reference camera and the left camera 4 may be used as a reference camera. The left and right cameras 2, 4 arranged toward the front are arranged so that their optical axes are parallel to each other.

  A GPS signal received by the GPS receiving system 14 is input to the controller U, and map data from the navigation system 8 is input.

  The controller U stores first and second memories 20 and 22 that store image signals captured from the cameras 2 and 4, that is, a reference memory 20 that stores reference image signals captured from the right camera (reference camera) 2; And a reference memory 22 for storing a reference image signal captured from the left camera (reference camera) 4.

  The controller U also has a travel environment detection unit 24 that receives signals from the GPS reception system 14 and the navigation system 8 and detects the travel environment of the vehicle VC. The travel environment data detected by the travel environment detection unit 24 is included in the travel environment data. The feature point search area map and the corresponding point search range map (two types for urban driving and suburban driving, which will be described later) created based on them are stored in the third and fourth memories 26 and 28.

  A feature point search is performed based on the above-described standard image data in the standard memory 20, and a corresponding point is searched based on the reference image data in the reference memory 22. Then, the distance calculation and risk determination unit 30 calculates the distance to the object (obstacle) based on the extracted feature points and corresponding points to determine the risk of collision, and when the risk of collision is high The alarm information is displayed on the windshield by the projector of the window shield display system 10.

The basic principle of detecting the distance to the obstacle O located in front of the vehicle VC by the left and right cameras 2 and 4 will be described based on FIG. 3. The distance between the two cameras 2 and 4 that are separated from each other on the left and right L ₀ is known. Assume that the distance between the feature point of the obstacle detected by the luminance change from the image data captured by the reference camera 2 and the optical axis of the reference camera 2 is D1 (the number of pixels of the reference memory 20 corresponding to the distance D1). . When a corresponding point corresponding to the feature point is extracted from image data captured by the reference camera 4, the distance between the corresponding point and the optical axis of the reference camera 4 is D2 (in the reference memory 22 corresponding to the distance D2). (The number of pixels), the parallax is a value obtained by adding the distance D1 and the distance D2 (parallax = D1 + D2). The distance L between O and the host vehicle VC can be known.

  FIG. 4 is a diagram for explaining a search area for suburban travel used for searching for feature points in the reference image. The reference image 40 (the memory area of the reference memory 20 corresponding to this) captured by the right reference camera 2 is shown. In driving in the suburbs, in the image 40, a suburban feature point search area 42 limited to the travel lane of the host vehicle VC and its neighboring area is set, and the suburban feature point search area 42 has a total of eight up and down and left and right. It is divided into sections 42-1-8.

  FIG. 5A shows a corresponding point search area 44 corresponding to the suburban feature point search area 42 in the image captured by the left reference camera 4 (memory area of the reference memory 22 corresponding thereto). The corresponding point search area 44 is divided into eight sections 44-1 to 8-8 corresponding to the eight sections 42-1 to 8-8 of the suburban feature point search area.

  The corresponding point search area 44 is divided for each search direction, and the corresponding points are searched for in the search range set for each division. In these search ranges, there may be an obstacle that needs to be searched. It is stored as parallax data corresponding to the predicted distance range, and corresponding points are searched only within the range set by these parallax data. That is, when a feature point is extracted, as shown by a broken line in the left image in FIG. 3, only a search range set by pre-stored parallax data from a position corresponding to the feature point of the left image 22 is used. Search and extract corresponding points.

  For example, as shown in FIG. 5 (b) showing a plan view of the vehicle state corresponding to FIG. 5 (a) and FIG. 5 (c) showing a side view, up to an obstacle to be searched according to the search direction. The distance is different. That is, as shown in FIG. 5B, in the sections 44-5 and 44-6 for searching in the lane in the traveling direction of the own vehicle, the obstacle search needs to search up to a relatively long distance range. The data is set as 10 pix to 50 pix, and as shown by the broken line in FIG. 3, the search for the corresponding points starts from a position separated by 10 pix from the feature point and ends when separated by 50 pix. On the other hand, in the sections 44-7 and 44-8 for searching outside the lane in the traveling direction of the own vehicle, the obstacle search can be performed in a relatively short distance range with respect to the traveling direction of the own vehicle. The disparity data is set as 20 pix to 50 pix, and as shown by the broken line in FIG. 3, the search for the corresponding point starts from a position separated by 20 pix from the feature point and ends when separated by 50 pix.

  Similarly, as shown in FIG. 5C, in the sections 44-1 and 44-2 that search downward in the traveling direction of the own vehicle, the obstacle search searches for a relatively short distance range to the road surface. Therefore, the parallax data is set as 40 pix to 50 pix, and as shown by a broken line in FIG. 3, the search for the corresponding point starts from a position separated by 40 pix from the feature point and ends when separated by 50 pix.

  FIG. 6 is a diagram for explaining a search area used for searching for a feature point in a reference image used while traveling in an urban area. In traveling in this urban area, an urban area feature point search area 46 is set in the captured image 40 so as to spread laterally and far away. The urban area feature point search area 46 is divided into a total of 32 sections 46- 1 to 46-32.

  FIG. 7 shows a corresponding point search range 48 corresponding to the urban feature point search area 46 in the image captured by the left reference camera 4 (the corresponding memory area of the reference memory 22), and this corresponding point search. The range 48 is divided into 32 sections 48-1 to 48-32 corresponding to the 32 sections 46-1 to 46-32 of the urban feature point search area 46.

  Similar to the corresponding point search at the time of traveling in the suburbs, the corresponding point search range 48 is divided for each search direction, and the corresponding points are searched within the search range set for each division. Then, it is stored as parallax data corresponding to a distance range in which an obstacle requiring search is predicted, and corresponding points are searched only within the range set by these parallax data. That is, when a feature point is extracted, as shown by a broken line in the left image in FIG. 3, only a search range set by pre-stored parallax data from a position corresponding to the feature point of the left image 22 is used. Search and extract corresponding points.

  For example, as shown as MAP data in FIG. 7, since the distance to the obstacle to be searched differs depending on the search direction, the sections 48-22 and 48-23 for searching the lane in the traveling direction of the own vehicle. In FIG. 3, since the obstacle search needs to search up to a relatively long distance range, the disparity data is set as 15 pix to 50 pix, and as shown by the broken line in FIG. 3, the search for the corresponding point is 15 pix from the feature point. It starts from a separated position and ends when it is separated by 50 pix. On the other hand, in the sections 48-19 and 48-20 for searching outside the lane in the traveling direction of the own vehicle, the obstacle search can be performed in a relatively short distance range with respect to the traveling direction of the own vehicle. The disparity data is set as 20 pix to 50 pix, and as shown by the broken line in FIG. 3, the search for the corresponding point starts from a position separated by 20 pix from the feature point and ends when separated by 50 pix.

  Similarly, in the sections 48-6 and 48-7 for searching downward in the traveling direction of the vehicle, the obstacle search may be performed by searching for a relatively short distance range to the road surface. As shown by the broken lines in FIG. 3, the search for corresponding points starts from a position separated by 40 pix from the feature point and ends when separated by 50 pix. The same applies to other categories.

  A specific example of the corresponding point search method included in the embodiment will be described based on the flowchart of FIG. First, in step S1, input processing of signals from the left and right cameras 2, 4 and the navigation system 8 is performed, and in the next step S2, based on information from the navigation system 8, the current driving environment, that is, the urban area is determined. Whether the vehicle is traveling or traveling in the suburbs is estimated, and a feature point search area corresponding to the estimated traveling environment is set (step S3). That is, the search area 42 in FIG. 4 is set when traveling in the suburbs, and the search area 46 in FIG. 6 is set when traveling in an urban area. This step S3 constitutes a search area setting means and a section number setting means.

  In the next step S4, the search for feature points in the region is executed according to the priority order. The step S4 will be described by taking the case of traveling in the suburbs as an example. Each of the sections 42-1 to 42-8 of the feature point search area 42 is given a search rank as an attribute. As a priority order, the first order is set for two lower center sections 42-1 and 42-2 that store captured images of an area having a high collision risk, for example, a space in front of and adjacent to the vehicle. The second rank is set for the left and right sections 42-3 and 42-4 in the lower center, which is a region with a large area, and the third rank is placed in the upper center sections 42-5 and 42-6 with the next highest risk of collision. Is set, and the fourth rank is set in the sections 42-7 and 42-8 on the left and right sides of the upper center, which is considered to have the lowest collision risk (FIG. 4).

  According to this example, in step S4, the search for feature points is first executed in the two lower central sections 42-1 and 42-2 of the first rank according to the above-mentioned priority order. When the feature point exists in the lower central sections 42-1 and 42-2, the process proceeds to step S 6 as YES, and the feature point search area 42 in the suburban corresponding point search range 44. The two sections 44-1 and 44-2 that are the sections corresponding to the two sections 42-1 and 42-2 of the first rank are set as search ranges, and the two sections 44-1 and 44- The corresponding point search is executed in step 2 (step S7). This step S7 constitutes corresponding point search means. If the corresponding point cannot be extracted, it is determined as NO in step S8, and the process proceeds to step S9 to determine whether or not the search priority is lowest.

  Since the first priority is now executed, the process proceeds to step S10 as NO, and a search for feature points and corresponding points corresponding to the second priority, which is the next priority, is executed. That is, in step S10, the feature point search is executed for the second rank sections 42-3 and 42-4 of the feature point search area 42, and in step S6, the second rank feature point search section 42-. The corresponding point search is executed for the suburban corresponding point search sections 44-3 and 44-4 corresponding to the items 3 and 42-4. The search for this feature point and the corresponding point corresponding to this feature point is executed up to the last rank, that is, the lowest rank, except when the feature point does not exist.

  When there is a feature point and a corresponding point corresponding thereto, the process proceeds to step S11, and the distance L (FIG. 3) to the object (obstacle) is calculated based on the parallax. This step S11 constitutes a distance calculation means. When the calculated distance L is smaller than a predetermined threshold, that is, a distance with high collision risk, the process proceeds to step S13, and an alarm is displayed by the window shield display system 10.

  As described above, a feature point and a corresponding point corresponding thereto are searched by limiting the search area in the search of the reference memory 20 storing the reference image and the reference memory 22 storing the reference image. The burden of executing the point and corresponding point search processing can be reduced, and the search processing time can be shortened. In addition, since ranks are assigned to limited search areas, and feature point and corresponding point search processing is executed in accordance with the ranks, ranks that match the search purpose (in the above example, a collision risk alarm) are set. By assigning, it is possible to quickly extract feature points and corresponding points that match the search purpose. In addition, by changing the size of the feature point search areas 42 and 46 in the reference image according to the travel environment such as an urban area or a suburb, it is possible to perform a rational feature point extraction process corresponding to the travel environment. . Also, it is possible to set the number of pixels corresponding to different parallax for each section of the corresponding point search ranges 44 and 48 so that obstacles predicted to exist in different distance ranges depending on the search direction can be detected. It is possible to reduce the point search processing load and increase the accuracy.

  In the above specific example, priority is given to a plurality of sections obtained by dividing the feature point search areas 42 and 46, and feature points are searched for in the corresponding sections according to the priority order. With respect to the search, all the feature point search areas 42 and 46 are searched, and corresponding points are searched in the order of priorities only for searching for corresponding points corresponding to the found feature points, as in the above specific example. May be.

VC car (own vehicle)
2 First CCD camera (reference camera)
4 Second CCD camera (reference camera)
8 Navigation System 10 Window Shield Display System 20 Base Memory 22 Reference Memory 40 Captured Image 42 Suburban Feature Point Search Area 44 Corresponding Point Search Range 46 Urban Feature Point Search Area 48 Corresponding Point Search Range

Claims (6)

The image processing apparatus includes at least first and second imaging units, searches for feature points of an obstacle from an image captured by the imaging unit, and searches for corresponding points corresponding to the feature points. In the vehicle driving support device that calculates the distance to the obstacle based on the parallax of
A reference memory for storing reference image data imaged by the first imaging means;
A reference memory for storing reference image data captured by the second imaging means;
A feature point search means for searching a feature point from a feature point search region, which is a region for searching a feature point from reference image data stored in the reference memory and limited to a part of the reference image;
A corresponding point search range for searching for a corresponding point from the reference image data stored in the reference memory corresponds to the feature point within a search range set for each search direction determined by the position of the reference image of the feature point. Corresponding point search means for searching for corresponding points;
The vehicle has a distance calculation unit that calculates a distance to an obstacle by parallax based on the feature point searched by the feature point search unit and the corresponding point searched by the corresponding point search unit. Driving assistance device.   The feature point search means for searching for feature points from the reference image data stored in the reference memory is divided into a plurality of sections, and each section is assigned a search rank, and the first rank The vehicle driving support device according to claim 1, wherein the feature points are searched in order. Driving environment detection means for detecting the driving environment of the host vehicle;
The vehicle driving support apparatus according to claim 2, further comprising: search area setting means for setting a size of the feature point search area in accordance with the driving environment detected by the driving environment detection means.   The vehicle driving support device according to claim 3, further comprising: a section number setting unit configured to set the number of sections of the feature point search area according to the driving environment detected by the driving environment detection unit.   The vehicle driving support device according to claim 4, wherein the corresponding point search area is set corresponding to a section of the feature point search area, and a search range of the corresponding point is set for each of the sections.   The vehicle driving support device according to claim 4 or 5, wherein the priority order is determined in descending order of collision risk.

Images