JP2009276906A - Travelling information providing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a travelling information providing device for accurately and quickly detecting an object which suddenly appears just before a travelling vehicle to secure the safety of the travelling vehicle. <P>SOLUTION: In a drive support device 100, an inadequate parallax pixel area specifying unit 114 specifies an inadequate parallax pixel area including a pixel incapable of calculating a parallax value within a predetermined time period and a pixel obtaining a parallax value exceeding an upper limit value, an adjacent object image candidate specifying unit 115 specifies an overlap pixel area in which the inadequate parallax pixel area overlaps with an adjacent object correspondence pixel area and specifies an adjacent object image candidate on the basis of the overlap pixel area, an evocation/control unit 120 evokes the safety confirmation or controls a drive state of its own vehicle on the basis of the predetermined adjacent object image candidate. Thus, the device utilizes information (information at one timing point) of a pixel capable of obtaining an inadequate parallax value which has not been used and has been discarded to detect an object, so that the device may accurately and quickly detect the object. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a travel information providing apparatus using parallax information obtained from a stereo image.

  In recent years, in order to improve the safety of automobiles, techniques have been developed to detect obstacles such as vehicles ahead and warn the driver of the danger of collision with the obstacles. In such a technique, in-vehicle sensors such as a camera and a radar are used in order to detect an obstacle that may collide with the host vehicle.

  As a vehicle-mounted sensor, a stereo camera including a plurality of cameras may be used. In an image taken by each camera, a position where the same object image appears is specified (this process may be referred to as stereo matching), and a parallax value that is a deviation of the position is calculated. This parallax is caused by a difference in imaging position (that is, camera arrangement).

  In general, the calculated parallax value is converted into a distance from the own vehicle to the object to be imaged by a triangulation method. Object detection processing is performed using the parallax information or distance information obtained in this way. Specifically, for example, object detection is performed by a method of extracting a part where substantially the same parallax values are collected as an object image. Furthermore, by detecting this object in a plurality of images taken in time series and tracking the position of the same object image in the plurality of images, it is possible to determine whether the obstacle is dangerous for the host vehicle. And when a dangerous obstacle exists, it becomes possible to warn a driver.

  Patent Document 1 also discloses a technique for detecting an object using this parallax.

A vehicle exterior monitoring device disclosed in Patent Document 1 includes a distance detection circuit that calculates a city block distance between two images obtained by stereo imaging of an object outside the vehicle and outputs distance information. The distance detection circuit recognizes the situation outside the vehicle based on the distance information of the vehicle, collates the detection result of the current three-dimensional object based on the distance information from the distance detection circuit, and the detection result of the previous three-dimensional object, and It is determined whether or not a three-dimensional object exists near the detectable limit distance, and when it is determined that a three-dimensional object exists near the detectable limit distance of the distance detection circuit, the area set in one captured image A distance calculation program that calculates a distance to a three-dimensional object that is closer than the detectable limit distance of the distance detection circuit from the parallax by searching for the part corresponding to the other captured image from the other captured image To run.
JP 2000-207695 A

  However, as described above, there is a limit to the method of detecting an object using only the parallax value or the distance information obtained by converting the parallax value. That is, when the distance between the imaging unit such as a camera and the object to be captured is closer than a certain limit distance, an accurate parallax value cannot be obtained.

  In order to solve this problem, in Patent Document 1 described above, the distance to an object existing at a position closer to the detectable limit distance of the distance detection circuit is calculated by the closest distance calculation program.

  However, the technique disclosed in Patent Document 1 cannot detect an object that suddenly appears in the shooting area (that is, in the camera field of view). This is because the detection results at two different timings are used to determine whether or not an object is present at a close distance, so that there is a possibility that an object that appears suddenly is not reflected in the detection results at a timing far from the determination time. Because it is expensive.

  In addition, when the driver is finally alerted to the detected object, it is desired to alert the driver as soon as possible, so that the time that can be spent on the object detection is actually limited. For this reason, it is fully conceivable that the above-described conventional technology that assumes that the distance can be calculated cannot calculate the parallax value or the distance information corresponding to the parallax value within the time limit.

  As described above, the conventional technique cannot detect an object that appears suddenly in the shooting area with high accuracy and speed, and as a result, cannot sufficiently secure the safety of the traveling vehicle on which the object detection device is mounted. .

  The present invention has been made in view of the above points, and provides a travel information providing device that can detect even an object suddenly appearing immediately before a traveling vehicle more accurately and quickly and ensure the safety of the traveling vehicle. The purpose is to provide.

  The travel information providing apparatus according to the present invention includes a stereo image acquisition unit that acquires a reference image and a comparison image captured by a first camera and a second camera disposed in a host vehicle mounted thereon, the reference image, A parallax value calculation unit that calculates a parallax value with the comparison image for each pixel constituting the reference image by performing a stereo matching process using the comparison image, and a calculation result obtained by the parallax value calculation unit And means for specifying an inappropriate parallax value pixel area including a pixel for which the parallax value cannot be calculated within a predetermined time and a pixel from which a parallax value exceeding an upper limit value is obtained, and the vicinity of the first and second cameras Storage means for storing information relating to a neighboring object corresponding pixel area consisting of a group of pixels in which an image of an object to be photographed that is present in an object, and an offset between the inappropriate parallax value pixel area and the neighboring object corresponding pixel area A neighboring object image candidate identifying unit that identifies a barrier pixel region and identifies a nearby object image candidate based on the overlap pixel region, and a safety confirmation alert or a vehicle's vehicle based on the identified nearby object image candidate. A structure including arousing / controlling means for controlling the driving state is adopted.

  ADVANTAGE OF THE INVENTION According to this invention, even if it is an object which appears suddenly just before a traveling vehicle, it can detect more accurately and rapidly, and the traveling information provision apparatus which ensures the safety of a traveling vehicle can be provided.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, the driving information providing device will be described as a driving support device that supports driving by providing the driver with the appearance of an avoidance target object that has a risk of collision as driving information. In the embodiments, the same components are denoted by the same reference numerals, and the description thereof will be omitted because it is redundant.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of a driving support device (running information providing device) 100 according to Embodiment 1 of the present invention.

  In FIG. 1, the driving support device 100 is mounted on a vehicle here, and includes an object detection device 110, an awakening / control unit 120, and an information presentation unit 130.

  The object detection device 110 acquires a plurality of pieces of image information (hereinafter also referred to as “stereo image information”) taken by a plurality of cameras disposed in the vehicle. Each piece of image information is composed of pixel data groups each corresponding to a pixel (pixel).

  The object detection device 110 calculates a parallax value for each pixel using the acquired stereo image information. Then, the object detection device 110 uses the appropriate parallax value data group included in a predetermined parallax value range (hereinafter sometimes referred to as “appropriate parallax value range”) among the calculated parallax value data groups, An object existing in front is detected.

  Furthermore, the object detection device 110 includes a pixel having a parallax value that is out of the appropriate parallax value range (that is, a parallax value that is substantially zero and a parallax value that exceeds an upper limit value that ensures the reliability of the parallax value) and a predetermined period. Based on an inappropriate parallax value pixel area composed of pixels for which a parallax value could not be calculated, a collision avoidance target object existing immediately before the vehicle is detected.

  Here, the object detection device 110 includes an image acquisition unit 111, a parallax information generation unit 112, a distant object detection unit 113, an inappropriate parallax value pixel region specification unit 114, a near object image candidate specification unit 115, and a near object correspondence. An area storage unit 116 is included.

  The image acquisition unit 111 acquires stereo image information captured by two cameras (hereinafter, sometimes referred to as “stereo cameras”) disposed on the left and right sides of the vehicle. Here, for simplicity of explanation, the number of installed cameras is two, but the number is not particularly limited as long as it is two or more.

  The acquired stereo image information includes two pieces of image information taken by two cameras at the same time. Image information photographed by one camera is “reference image” information, and image information photographed by another camera is “comparative image” information. A part of the reference image is included in the comparison image. That is, the reference image and the comparison image partially overlap.

  The image acquisition unit 111 sends the acquired reference image information and comparison image information to the parallax information generation unit 112. Note that the stereo image acquired within the predetermined period is parallelized by the image acquisition unit 111 separately for the reference image and the comparison image, and the obtained information is used as the reference image information and the comparison image information to the parallax information generation unit 112. It may be sent out. This parallelization process will be described in detail later.

  The disparity information generation unit 112 calculates a disparity value between the reference image and the comparison image for each pixel by performing a stereo matching process using the reference image and the comparison image. In the stereo matching process, first, a corresponding point in the comparison image corresponding to an arbitrary point (reference point) in the reference image is specified. In other words, pixel data in a comparison image (hereinafter also referred to as “corresponding comparison pixel data”) corresponding to pixel data of the reference image (hereinafter also referred to as “reference pixel data”) is specified for each pixel. Is done.

  Then, when the entire reference image and the entire comparison image are overlaid, a distance generated between the reference point and the corresponding point is calculated as a parallax value. That is, the distance generated between the pixel of the reference pixel data and the pixel of the corresponding comparison image data is calculated as the parallax value.

  The far object detection unit 113 detects an object existing in front of the vehicle by using the appropriate parallax value data group included in the appropriate parallax value range. Specifically, the distant object detection unit 113 extracts disparity value data having substantially the same disparity value from the appropriate disparity value data group included in the appropriate disparity value range. Thereby, parallax value data can be grouped for every object image candidate. This utilizes the property that the parallax values of pixels in which an image of the same object exists are substantially equal.

  The inappropriate parallax value pixel area specifying unit 114 includes a pixel having a parallax value that is out of the proper parallax value range (that is, a parallax value that is substantially zero and a parallax value that exceeds an upper limit value that ensures the reliability of the parallax value) and a predetermined period. The pixel for which the parallax value could not be calculated is identified. Thereby, an inappropriate parallax value pixel region is specified.

  The near object image candidate specifying unit 115 specifies the image candidate of the collision avoidance target object existing immediately before the vehicle based on the inappropriate parallax value pixel area and the near object corresponding area. The near object corresponding area is an area composed of a pixel group in which an image of an imaged object existing in the vicinity of the camera appears. Information on the near object corresponding area is stored in the near object corresponding area storage unit 116.

  Specifically, the neighboring object image candidate specifying unit 115 extracts pixel identification data corresponding to an overlap portion between the inappropriate parallax value pixel area and the neighboring object corresponding area, and parallax value data corresponding thereto. Here, there is a high possibility that the overlap portion between the inappropriate parallax value pixel region and the nearby object corresponding region is a pixel obtained by copying an image of the imaged object existing in the vicinity of the host vehicle. That is, the inappropriate parallax value pixel area includes pixels whose calculated parallax values are inappropriate because the distance to the object to be imaged is too close. The overlap portion can be used as an image candidate for the collision avoidance target object.

  Based on the detection result in the object detection device 110, the arousing / control unit 120 alerts the driver to safety confirmation or controls the driving state of the vehicle (here, the own vehicle).

  The awakening / control unit 120 includes a process execution determination unit 121 and a provided information processing unit 122.

  Based on the detection result of the distant object detection unit 113, the process execution determination unit 121 alerts the driver to safety confirmation or controls the driving state of the vehicle.

  Further, the process execution determination unit 121 urges the driver to confirm safety or controls the driving state of the vehicle based on the image candidate of the collision avoidance target object specified by the nearby object image candidate specifying unit 115. . In the present embodiment, the processing execution determination unit 121 is based on the condition that the nearby object image candidate specifying unit 115 specifies the image candidate of the collision avoidance target object, that is, the presence of the overlap portion. It is determined that safety confirmation is urged or vehicle driving state control is executed.

  Provided information processing section 122, when execution determination is made by process execution determination section 121, presentation information is set / changed, and the set / changed presentation information is output to information presentation section 130.

  The presentation information set in the provided information processing unit 122 corresponds to the form presented in the information presentation unit 130. For example, when the information presentation unit 130 is a display device using a display panel or the like, the display information is set as a display image displayed on the display panel. Set as Further, in the case of a vibration device disposed on a handle or a seat, presentation information is set as vibration data.

  The information presenting unit 130 presents the information set in the provided information processing unit 122, and this presents a safety confirmation regarding the traveling of the vehicle on which the driving support device is mounted.

  Here, the information presentation part 130 is provided with the display panel (display part) mounted in a vehicle, and shall perform the display for making the driver | operator of a vehicle confirm the oversight of the target object used as safety confirmation object. In addition, this information presentation part 130 may be comprised with the sounding apparatus which alert | reports a driver | operator by emitting a warning sound, a warning sound, etc. Moreover, it is good also as an alerting | reporting apparatus which alert | reports the oversight of the driver | operator of the vehicle by which the driving assistance apparatus 100 is mounted in a pedestrian etc.

  The operation of the driving support apparatus 100 having the above configuration will be described along the flow of FIG.

[Stereo image parallelization]
In step S <b> 1001, the image acquisition unit 111 parallelizes the stereo image acquired within a predetermined period separately for the reference image and the comparison image. Here, the collimation process is a geometric correction process in which the epipolar line is parallel to the horizontal coordinate axis of each of the stereo image reference image and the comparison image. In order to perform the parallelization process, it is necessary to obtain necessary parameters in advance. As a method for obtaining this parameter, an object (for example, a checkerboard) whose shape is known and whose feature point is clear is photographed with a stereo camera, and the parameter is calculated based on the detection result of the feature point in each camera. Methods are generally known.

[Corresponding point identification processing, parallax value calculation processing]
In step S1002, the parallax information generation unit 112 identifies a corresponding point in the comparison image corresponding to an arbitrary point (reference point) in the reference image, and superimposes the parallax value (that is, the entire reference image and the entire comparison image). The distance that sometimes occurs between the reference point and the corresponding point) is calculated for each pixel.

  FIG. 3 is a diagram illustrating the relationship between the reference point of the reference image, the corresponding point of the comparison image, and the parallax value. As shown in FIG. 3, the corresponding point corresponding to the reference point on the reference image exists on the left side of the reference image in the comparison image. A difference between the position of the reference point and the position of the corresponding point is calculated as a parallax value.

  Note that any method may be used to obtain the corresponding points from the stereo image. For example, a reference region including a point around the center point is extracted around a point where the parallax is to be obtained in the reference image, a comparison region having the same size as the reference region is sequentially extracted from the comparison image, and the luminance with respect to the reference region A method of obtaining corresponding points by specifying a comparison region having the smallest sum of absolute values of differences (absolute difference sum method) may be used.

  FIG. 4 shows an image obtained by developing the parallax value obtained in this way into corresponding pixels (hereinafter, sometimes referred to as “parallax value image”) and a camera image (for example, a reference image). FIG. 4A shows a reference image, and FIG. 4B shows a parallax value image corresponding to the reference image.

  The pixel area represented by black in the parallax value image of FIG. 4B is the inappropriate parallax value image area, and the other part is a pixel area corresponding to the proper parallax value data group.

[Distant object detection processing]
In step S <b> 1003, the far object detection unit 113 detects a distant object existing far away from the host vehicle using the appropriate disparity value data group included in the appropriate disparity value range. FIG. 5 shows a parallax value image obtained by developing an appropriate parallax value data group included in an appropriate parallax value range used for detection of this distant object on corresponding pixels.

  Specifically, the distant object detection unit 113 extracts disparity value data having substantially the same disparity value from the appropriate disparity value data group included in the appropriate disparity value range. By doing so, each area surrounded by a black frame is grouped as shown in FIG. Here, three distant objects are detected.

[Inappropriate parallax value pixel area specifying process]
In step S <b> 1004, the inappropriate parallax value pixel region specifying unit 114 includes a pixel having a parallax value that is out of the proper parallax value range (that is, a parallax value that is substantially zero and a parallax value that exceeds an upper limit value that ensures the credibility of the parallax value) A pixel whose parallax value could not be calculated within a predetermined period is specified. FIG. 6 shows a parallax value image obtained by extracting only the inappropriate parallax value pixel region.

  Here, in the parallax value image of FIG. 6, the inappropriate parallax value pixel region appearing on the upper side is a region (D1 in FIG. 6) where the parallax value is substantially zero. This zero parallax portion is a portion where matching is performed at almost the same position in the reference image and the comparison image as a result of performing stereo matching. As the distance to the object to be imaged increases, the difference between the relative positions of the stereo camera with respect to the object to be imaged increases, and the parallax also decreases. Therefore, for example, the parallax value obtained in an image in which buildings, mountains, sky, or the like that are separated by a predetermined distance or more are projected is substantially zero.

  In addition, in the parallax value image of FIG. 6, an inappropriate parallax value pixel region (D2 in FIG. 6) that appears on the lower side corresponds to the image of the bumper part of the host vehicle. That is, the closer the object is to the camera, the greater the parallax of the object in stereo. Therefore, if the distance from the camera to the object to be imaged becomes too short, it is impossible to obtain the parallax by stereo matching, or even if it is obtained, the parallax value exceeds the upper limit value that ensures credibility. Become.

  Furthermore, in the parallax value image of FIG. 6, an inappropriate parallax value pixel region (D3 in FIG. 6) that is different from the bumper portion appears on the lower side.

[Nearby object image candidate identification processing]
In step S1005, the nearby object image candidate specifying unit 115 extracts pixel identification data corresponding to an overlap portion between the inappropriate disparity value pixel region and the nearby object corresponding region, and disparity value data corresponding to the pixel identification data.

  The shaded area in FIG. 6 is the above-mentioned nearby object corresponding area. Here, the nearby object corresponding region is a region determined in advance according to the installation position of the camera, and is a region existing on the lower side of the parallax value image, and an inappropriate parallax value pixel region that constantly appears such as a bumper portion. This is the area excluding.

  In this way, by using the predetermined nearby object corresponding region, the region corresponding to D1 (region where the parallax value is substantially zero) and D2 (region where the inappropriate parallax value is constantly obtained) from the overlapped portion. Can be excluded.

  There is a high possibility that the overlap portion between the inappropriate parallax value pixel area and the nearby object corresponding area obtained in this way is a pixel obtained by copying an image of the imaged object that exists in the vicinity of the host vehicle. Therefore, this overlap portion is used as an image candidate for the collision avoidance target object.

[Excitation / control execution judgment process]
In step S <b> 1006, the process execution determination unit 121 urges the driver to confirm safety or control the driving state of the vehicle based on the image candidate of the collision avoidance target object specified by the nearby object image candidate specifying unit 115. Do. In the present embodiment, the processing execution determination unit 121 is based on the condition that the nearby object image candidate specifying unit 115 specifies the image candidate of the collision avoidance target object, that is, the presence of the overlap portion. It is determined that safety confirmation is urged or vehicle driving state control is executed.

  Then, when the execution determination is made by the process execution determination unit 121, the provided information processing unit 122 sets / changes the presentation information and outputs the set / changed presentation information to the information presentation unit 130.

  Whether or not the process execution determination unit 121 executes the process for the detected distant object according to the parallax value of the object (that is, the distance to the host vehicle) and the speed of approaching the host vehicle. Determine whether. Here, any method may be used for obtaining the estimation of the position and speed of the object. For example, by calculating the degree of coincidence between the luminance information of the previously detected object and the luminance information of the currently detected object, it is determined whether or not the objects in the images taken in time series are the same object. The speed may be estimated by analyzing the time-series position change.

[Information presentation process]
In step S <b> 1007, the information presentation unit 130 presents the information set in the provided information processing unit 122, and by this presentation, the safety confirmation regarding the traveling of the vehicle on which the driving support device is mounted is invoked.

  As described above, according to the present embodiment, in the driving assistance device 100, the inappropriate parallax value pixel region specifying unit 114 selects pixels for which the parallax value cannot be calculated within a predetermined time and pixels from which the parallax value exceeding the upper limit value is obtained. An inappropriate parallax value pixel area to be included, and the neighboring object image candidate identification unit 115 identifies an overlapping pixel area between the inappropriate parallax value pixel area and the neighboring object corresponding pixel area, and a neighboring object based on the overlapping pixel area The image candidate is specified, and the awakening / control unit 120 performs safety confirmation or controls the driving state of the host vehicle based on the specified nearby object image candidate.

  By doing so, it is possible to easily detect the nearby object of the host vehicle using the inappropriate parallax value pixel region and the nearby object corresponding pixel region.

  Here, in the related art, as in the driving support device 100, although object detection is performed using the calculated parallax value, there is no viewpoint for using pixel information for which parallax value could not be obtained for object detection. Therefore, in the above prior art, it is necessary to additionally execute a close distance calculation program in order to detect a nearby object.

  On the other hand, the driving assistance device 100 detects a nearby object of the host vehicle by using pixel information that can obtain an inappropriate parallax value that has been discarded without being used for conventional object detection. Accordingly, it is not necessary to perform an additional process required in the conventional technique, so that a nearby object can be detected quickly. In addition, since a nearby object existing at that timing can be detected from the image information at one timing, even an object that suddenly appears immediately before the traveling vehicle can be detected with high accuracy.

  In this way, the driving support device 100 can detect even an object suddenly appearing immediately before the traveling vehicle with higher accuracy and speed, and based on the detection result, the driver confirms safety or controls the driving state of the host vehicle. The safety of the traveling vehicle can be ensured.

(Embodiment 2)
In the second embodiment, the shape of the overlap portion described above is analyzed, and the overlap portion is used as an image candidate for the collision avoidance target object according to the analysis result. FIG. 7 is a block diagram showing the configuration of the driving support apparatus 200 according to Embodiment 2 of the present invention.

  In FIG. 7, the driving support device 200 includes an object detection device 210. The object detection device 210 includes a near object image candidate specifying unit 211.

  The nearby object image candidate specifying unit 211 extracts pixel identification data corresponding to an overlap portion between the inappropriate disparity value pixel area and the nearby object corresponding area, and disparity value data corresponding to the pixel identification data. Specifically, the nearby object image candidate specifying unit 211 extracts both data for each closed pixel region. Thereby, the overlap portion is grouped for each closed pixel region.

  That is, in FIG. 8, the two areas indicated by D3 and D4 that are both included in the overlap portion are grouped into different groups. Hereinafter, the grouped area may be referred to as a “group area”.

  Then, the near object image candidate specifying unit 211 analyzes the shape of the overlap portion. That is, the near object image candidate specifying unit 211 analyzes at least one of the area, width, height, major axis radius, minor axis radius, convexity, and moment of the overlap portion. This analysis is performed for each group region.

  Then, when the analysis result satisfies the near object determination criterion, the near object image candidate specifying unit 211 sets the analysis target group area as an image candidate for the collision avoidance target object.

  FIG. 9 shows a shape analysis flow diagram when the area, width, and height are used as analysis parameters.

  Here, when all of the area, width, and height are equal to or greater than the threshold value (predetermined value) determined for each analysis parameter, it is determined that the analysis target group region is an image candidate for the collision avoidance target object. Is done.

  On the other hand, if at least one of the area, width, and height is less than the threshold value, it is determined that the group region to be analyzed is not an image candidate for the collision avoidance target object.

  As described above, according to the present embodiment, the near object image candidate specifying unit 211 has at least one of the area, width, height, major axis radius, minor axis radius, convexity, and moment of the overlapping pixel region. Is calculated as an analysis parameter, and when the calculated value is equal to or greater than a near object image determination threshold set for each analysis parameter, an overlap portion is specified as a near object image candidate.

  By doing this, for example, even when a parallax value calculation error occurs and an inappropriate parallax value is obtained, if there are a small number of pixels for which the inappropriate parallax value is found locally, the pixel is It can be excluded from object image candidates. Therefore, it is possible to specify the near object image candidate with higher accuracy.

(Embodiment 3)
In the third embodiment, the luminance of the overlap portion described above is evaluated, and the overlap portion is used as an image candidate for the collision avoidance target object according to the evaluation result. FIG. 10 is a block diagram showing the configuration of the driving support apparatus 300 according to Embodiment 3 of the present invention.

  In FIG. 10, the driving support device 300 includes an object detection device 310. The object detection device 310 includes a near object image candidate specifying unit 311 and a luminance evaluation unit 312.

  The nearby object image candidate specifying unit 311 extracts pixel identification data corresponding to an overlap portion between the inappropriate disparity value pixel area and the nearby object corresponding area, and disparity value data corresponding thereto. Specifically, the nearby object image candidate specifying unit 311 extracts both data for each closed pixel region. Thereby, the overlap portion is grouped for each closed pixel region.

  That is, in FIG. 11, the two areas indicated by D3 and D5, both included in the overlap portion, are grouped into different groups.

  When the result of the luminance evaluation performed by the luminance evaluation unit 312 for each group area satisfies the vicinity object determination criterion, the neighboring object image candidate specifying unit 311 selects the evaluation target group area as a collision avoidance target object image candidate. And

  The luminance evaluation unit 312 acquires stereo image information from the image acquisition unit 111 and acquires information related to the group region specified by the near object image candidate specifying unit 311. The luminance evaluation unit 312 determines the luminance of the luminance evaluation target region (see FIG. 12) in the stereo image corresponding to each group region, and evaluates the degree of variation in luminance in the luminance evaluation target region.

  Here, the luminance evaluation unit 312 applies the evaluation region frame f401 to the luminance evaluation target region, extracts a region to which the frame is applied, and evaluates the luminance of the extraction region.

  Here, when the luminance is measured for an object that is a collision avoidance target object, the degree of variation in luminance increases, whereas when the luminance is measured for a road surface or the like, the degree of variation in luminance is small.

  Therefore, the luminance evaluation unit 312 holds a luminance variation degree threshold value for determining a collision avoidance target object in advance, and when the luminance variation degree in the extraction area is equal to or greater than the threshold value, the area is the collision avoidance target object. It is evaluated that it corresponds to.

  On the other hand, when the variation degree of the luminance in the extraction area is less than the threshold, the luminance evaluation unit 312 evaluates that the group area corresponding to the extraction target does not correspond to the collision avoidance target object.

  In the lower part of FIG. 12, the luminances of the regions extracted from the respective luminance evaluation target regions are shown. As can be seen from this, in the luminance evaluation target region corresponding to D3, the luminance variation degree is large. On the other hand, in the luminance evaluation target region corresponding to D5, the luminance variation degree is small.

  That is, in this case, it is determined that the luminance evaluation target region corresponding to D5 is a region corresponding to the collision avoidance target object.

  By performing the luminance evaluation of each group area in this way, the collision avoidance target object can be detected with higher accuracy.

  As described above, according to the present embodiment, the luminance evaluation unit 312 evaluates the luminance variation degree in the overlap portion, and the neighboring object image candidate specifying unit 311 has the variation degree equal to or greater than the object determination threshold value. The overlap portion is specified as a near object image candidate.

  By doing this, even if an inappropriate parallax value is obtained for a pixel that is not a collision avoidance object, for example, an object that cannot be a collision avoidance object such as a road surface, the pixel is determined depending on the degree of variation in luminance. Can be excluded from the nearby object image candidates. Therefore, it is possible to specify the near object image candidate with higher accuracy.

  Note that the luminance evaluation processing according to the present embodiment can be applied to the second embodiment. That is, the near object image candidate specifying unit calculates at least one of the area, width, height, major axis radius, minor axis radius, convexity, and moment of the overlapping pixel region as an analysis parameter, and calculates When the value is equal to or greater than the near object image determination threshold set for each analysis parameter and the luminance variation is equal to or greater than the object determination threshold, the overlap portion may be specified as the near object image candidate.

(Embodiment 4)
In the first to third embodiments, if the near object image candidate specifying unit 115 specifies an image candidate of the collision avoidance target object, the driver is prompted to confirm safety or control the driving state of the vehicle. In contrast, in the fourth embodiment, the awakening / control unit discriminates the object type using the image candidate of the collision avoidance target object, and the driver confirms the safety confirmation or the vehicle according to the discrimination result. The control of the operation state is executed.

  That is, the process execution determination unit 121 determines the object type using the image candidate of the collision avoidance target object specified by the neighboring object image candidate specification unit 115 (211 and 311).

  For example, the process execution determination unit 121 holds a shape template (for example, a human shape) corresponding to the object type, and the image of the collision avoidance target object specified by the near object image candidate specifying unit 115 (211 or 311). The degree of coincidence between the candidate and the shape template is calculated, and when there is a certain level or more of coincidence, it is determined that the image candidate of the collision avoidance target object is an object of a type corresponding to the shape template. Then, when the object type is a collision avoidance target object, the process execution determination unit 121 determines to execute a safety confirmation for the driver or control of the driving state of the vehicle. Note that a shape template prepared in advance may be used in a fixed manner, or it may be corrected by a learning function while repeating the discrimination process.

  The travel information providing apparatus of the present invention is useful for ensuring the safety of a traveling vehicle by detecting an object suddenly appearing immediately before the traveling vehicle more accurately and quickly.

The block diagram which shows the structure of the driving assistance apparatus in Embodiment 1 of this invention. Flow chart for explaining the operation of the driving support device of FIG. The figure which shows the relationship between the reference point of a reference image, the corresponding point of a comparison image, and a parallax value A diagram for explaining correspondence between a parallax value image and a camera image The figure which shows the parallax value image obtained from an appropriate parallax value data group The figure which shows the parallax value image from which only the unsuitable parallax value pixel area was extracted. The block diagram which shows the structure of the driving assistance apparatus in Embodiment 2 of this invention. The figure with which it uses for the process description of the near object image candidate specific | specification part in the driving assistance device of FIG. Shape analysis flow diagram when area, width, and height are used as analysis parameters The block diagram which shows the structure of the driving assistance device in Embodiment 3 of this invention. The figure with which it uses for process description of the near object image candidate specific | specification part in the driving assistance device of FIG. The figure which uses for the process description of the brightness | luminance evaluation part in the driving assistance apparatus of FIG.

Explanation of symbols

100, 200, 300 Driving support device 110, 210, 310 Object detection device 111 Image acquisition unit 112 Disparity information generation unit 113 Distant object detection unit 114 Inappropriate parallax value pixel region specifying unit 115, 211, 311 Near object image candidate specifying unit 116 Near Object Corresponding Area Storage Unit 120 Arousing / Control Unit 121 Processing Execution Determination Unit 122 Provided Information Processing Unit 130 Information Presentation Unit 312 Luminance Evaluation Unit

Claims (5)

Stereo image acquisition means for acquiring a reference image and a comparison image captured by a first camera and a second camera disposed in the mounted vehicle;
A parallax value calculating means for calculating a parallax value with the comparison image for each pixel constituting the reference image by performing a stereo matching process using the reference image and the comparison image;
A specifying unit for specifying an inappropriate parallax value pixel region including a pixel for which the parallax value cannot be calculated within a predetermined time and a pixel for which a parallax value exceeding an upper limit value is obtained based on a calculation result obtained by the parallax value calculating unit; ,
Storage means for storing information relating to a neighboring object corresponding pixel region composed of a pixel group in which an image of an object to be photographed existing in the vicinity of the first and second cameras appears;
A neighboring object image candidate identifying unit that identifies an overlapping pixel area between the inappropriate parallax value pixel area and the neighboring object corresponding pixel area, and identifies a neighboring object image candidate based on the overlapping pixel area;
Based on the identified nearby object image candidate, arousal / control means for arousing safety confirmation or controlling the driving state of the host vehicle,
A travel information providing apparatus comprising: The near object image candidate specifying means calculates at least one of the area, width, height, major axis radius, minor axis radius, convexity, and moment of the overlapping pixel region as an analysis parameter, and calculates When the value is equal to or greater than a neighboring object image determination threshold set for each analysis parameter, the overlapping pixel region is identified as the neighboring object image candidate.
The travel information providing apparatus according to claim 1. Comprising a luminance evaluation means for evaluating the degree of variation in luminance in the overlap pixel region;
The nearby object image candidate specifying means specifies the overlap pixel region as the nearby object image candidate when the degree of variation is equal to or greater than an object determination threshold.
The travel information providing apparatus according to claim 1. Comprising a luminance evaluation means for evaluating the degree of variation in luminance in the overlap pixel region;
The nearby object image candidate specifying means includes:
Calculating at least one of the area, width, height, major axis radius, minor axis radius, convexity, and moment of the overlapping pixel region as an analysis parameter;
When the calculated value is equal to or greater than the near object image determination threshold set for each analysis parameter and the degree of variation is equal to or greater than the object determination threshold, the overlap pixel region is specified as the near object image candidate. ,
The travel information providing apparatus according to claim 1. The arousing / control means holds a shape template corresponding to the object type, determines an object type based on a shape matching degree between the specified nearby object image candidate and the shape template, and determines the determined object type. Depending on the safety confirmation or control of the driving state of the vehicle,
The travel information providing device according to any one of claims 1 to 4.

Images