JP5034087B2 - Vehicle travel support device, vehicle, vehicle travel support program - Google Patents

Description

  The present invention relates to an apparatus and the like that supports the traveling of a vehicle based on an image obtained through an imaging device mounted on the vehicle.

When an imaging device such as a CCD camera mounted on the vehicle is arranged to image the front area of the vehicle through the front window, sunlight irradiated on the dashboard is reflected by the front window and imaged A so-called “reflection” occurs that is sensed by the image sensor of the device. In view of this, for example, a technique has been proposed in which a polarizing filter is attached to an image pickup device, and the polarization axis of the polarizing filter is provided with an offset angle corresponding to the curvature of the front window to prevent reflection (see Patent Document 1).
JP 2001-094842 A

  However, due to a cause such as the curvature of the windshield slightly deviating from the design curvature, reflection may not be prevented even though the polarizing filter is attached to the imaging apparatus. For this reason, there is a possibility that the situation of the imaging range by the imaging device cannot be accurately recognized.

  Accordingly, the present invention provides a device and the like that can reliably eliminate the influence of reflection on an imaging device mounted on a vehicle and can accurately recognize the surrounding situation of the vehicle. To do.

According to a first aspect of the present invention, there is provided a vehicle travel support device that recognizes a travel environment of the vehicle based on an image representing a situation in a traveling direction of the vehicle obtained through an imaging device mounted on the vehicle. A histogram representing the frequency distribution of the luminance in the image is generated, the frequencies are accumulated from the lowest luminance in the histogram, and the luminance when the cumulative value of the frequency reaches the threshold is set as the designated luminance. A first processing element and a first correction unit configured to correct a luminance of the image based on the designated luminance set by the first processing element, and to recognize a situation in a traveling direction of the vehicle based on the image with the corrected luminance. Two processing elements ,
The second processing element corrects the brightness of the image so that the histogram is expanded by the reciprocal of the difference between the highest brightness and the specified brightness with reference to the highest brightness in the histogram, or a brightness domain The brightness of the image is corrected so that the histogram is expanded by the reciprocal of the difference between the upper limit brightness and the specified brightness, or the iris control value is used as a reference. The luminance of the image is corrected so that the histogram is expanded by the reciprocal of the difference between and the specified luminance .

According to the vehicle travel support device of the first invention, in the histogram representing the frequency distribution of the luminance of the image obtained through the imaging device, the luminance when the cumulative value of the frequency from the low luminance side reaches the threshold value is the designated luminance. Is set. The designated brightness represents the degree of increase in the brightness of the image due to the reflection, for example, the designated brightness increases as the overall brightness of the image increases due to the reflection on the imaging device. Therefore, by correcting the luminance of the image based on the designated luminance, a corrected image from which the influence of the reflection is eliminated can be obtained, and the situation in the traveling direction of the vehicle can be accurately recognized based on the corrected image. That is, the influence of the reflection on the imaging device mounted on the vehicle can be surely eliminated, and the surrounding situation of the vehicle can be accurately recognized.
In addition, according to the vehicle travel support device of the first invention, the maximum brightness in the histogram and the upper limit brightness of the brightness definition area are taken into account in consideration of the degree of increase in brightness of the image representing the level of the designated brightness as described above. Alternatively, the brightness of the image is corrected based on the iris control value. Accordingly, a corrected image from which the influence of the reflection is eliminated is obtained, and the situation in the traveling direction of the vehicle can be accurately recognized based on the corrected image.

  Note that the component of the present invention “recognizes” information means that the component retrieves information from a database, reads information from a storage device such as a memory, and measures information based on an output signal from a sensor or the like. It means performing any information processing necessary to prepare or prepare the information for further information processing, such as calculating, estimating, determining, storing the measured information in a memory, and the like.

A vehicle according to a second aspect of the invention includes an imaging device and a vehicle travel support device according to the first aspect of the invention.

According to the vehicle of the second aspect of the present invention, since the influence of the reflection on the imaging device mounted on the vehicle is reliably eliminated by the vehicle driving support device, the situation around the vehicle is accurately recognized. Based on the results, the operation of the in-vehicle device can be appropriately controlled from the viewpoint of safe driving of the vehicle.

According to a third aspect of the present invention, there is provided a driving support program for causing a computer mounted on a vehicle to function as the vehicle driving support apparatus of the first invention.

According to the driving support program of the third aspect of the invention, the on-vehicle computer reliably eliminates the influence of the reflection on the imaging device mounted on the vehicle, and accurately recognizes the surrounding situation of the vehicle. It can function as a device.

  DESCRIPTION OF EMBODIMENTS Embodiments of a vehicle travel support device and the like of the present invention will be described using the drawings.

  A vehicle (four-wheeled vehicle) 1 shown in FIGS. 1 and 2 is equipped with a camera (imaging device) 12 such as a CCD camera or a near infrared camera, and a vehicle travel support device 10. The camera 12 is attached to the interior space so as to capture an image of the front of the vehicle 1 through the front window FW. As shown in FIG. 2, the vehicle 1 further includes sensors such as a vehicle speed sensor 122, an acceleration sensor 124 and a yaw rate sensor 126, a steering device 14, and a braking device 16. Each of the vehicle speed sensor 122, the acceleration sensor 124, and the yaw rate sensor 126 outputs signals corresponding to the speed, acceleration, and yaw rate of the vehicle 1, respectively.

  The vehicle travel support device 10 is configured by a computer or an ECU (electronic control unit) (comprising a CPU, ROM, RAM, and electronic circuits such as an I / O circuit and an A / D conversion circuit). Output signals from the camera 12 and the speed sensor 122 are input to the vehicle travel support device 10. The “vehicle driving support program” is read from the memory by the CPU, and various processes described later are executed according to the read program. The program may be distributed or broadcast from the server to the vehicle 1 via a network or satellite at an arbitrary timing, and stored in a storage device such as a RAM of the in-vehicle computer. The vehicle travel support device 10 controls the operation of one or both of the steering device 14 and the braking device 16 to execute travel support control for assisting the travel of the vehicle 1 so that the vehicle 1 does not depart from the travel region. The vehicle travel support device 10 includes a first processing element 110 and a second processing element 120. The first processing element 110 generates a histogram representing the luminance frequency distribution in the image obtained through the camera 12, accumulates the frequencies from the lowest luminance in this histogram, and calculates the luminance when the accumulated value reaches the threshold value. Set as specified brightness. The second processing element 120 corrects the luminance of the image based on the designated luminance set by the first processing element 110, and the situation of the traveling direction of the vehicle 1 based on the luminance corrected image, specifically, the lane Recognize the position of the mark.

  Functions of the vehicle 1 and the vehicle travel support device 10 having the above-described configuration will be described.

  First, a digital image representing the situation in front of the vehicle 1 or in the traveling direction is acquired through the camera 12 (FIG. 3 / S002). Thereby, for example, an image of a road extending in front of the vehicle 1 as shown in FIG. Note that FIG. 4A shows an image with high overall brightness due to the influence of so-called “reflection” in which sunlight or the like irradiated on the dashboard is reflected by the front window FW and sensed by the image sensor of the camera 12. It is. For this reason, there is a high possibility that the extraction of the edge whose luminance greatly changes based on the video as it is and the search for the lane mark will fail.

  Therefore, the first processing element 110 generates a histogram representing the luminance frequency distribution of each pixel of the digital image (FIG. 3 / S004). Thus, for example, a histogram as shown in FIG. 5A is generated based on the image (image data) shown in FIG.

Further, the designated luminance L _C is set by the first processing element 110 according to the following procedure. That is, first, the index i representing the level of brightness is set to “1” (FIG. 3 / S006). The smaller the index i, the lower the luminance. Further, the cumulative value of the low luminance side _{Σ j = 1-i N (} L j) (= N 1 + N 2 + ‥ + N i) is calculated the frequency N (L _i) of the luminance L _i (Fig. 3 / S008). Further, it is determined whether or not the cumulative value Σ _{j = 1−i} N (L _j ) is equal to or greater than a threshold value N _th (FIG. 3 / S010). When it is determined that the cumulative value Σ _{j = 1−i} N (L _j ) is less than the threshold value N _th (FIG. 3 / S010... NO), the index i is increased by 1 (FIG. 3 / S012), and then The cumulative value Σ _{j = 1−i} N (L _j ) of the frequency N (L _i ) is newly calculated (FIG. 3 / S008), and the cumulative value Σ _{j = 1−i} N (L _j ) is the threshold value N _th. It is determined whether or not this is the case (FIG. 3 / S010). On the other hand, when it is determined that the cumulative value Σ _{j = 1−i} N (L _j ) is equal to or greater than the threshold value N _th (FIG. 3 / S010... YES), the luminance L _i determined by the index i at that time is designated. The luminance L _C is set (FIG. 3 / S014).

Further, the second processing element 120 corrects the image based on the designated luminance L _C set by the first processing element 110 (FIG. 3 / S016). Specifically, the luminance L of each pixel of the image is corrected according to the formula (1) based on the designated luminance L _C and the maximum luminance L _max of the frequency distribution range in the histogram.

L (after correction) = (L (before correction) −L _C ) · {L _max / (L _max −L _C )} (1)

The specified brightness L _C is subtracted from the brightness L before correction on the right side of the expression (1). This is because the specified brightness is entirely reduced to the lower brightness side as indicated by the arrow in FIG. This means an operation of shifting by L _C. On the right side of equation (1), the difference between the luminance L before correction and the specified luminance L _C is multiplied by a factor {L _max / (L _max −L _C )}, which is shown in FIG. This means that the histogram is extended to the high luminance side. The corrected histogram shown in FIG. 5C represents a histogram that approximates the luminance histogram of an image obtained when there is no reflection. As a result, as shown in FIG. 4B, a corrected image in which the difference in brightness is clarified or emphasized is acquired.

Note that the luminance L of the image may be corrected according to the equation (2) based on the upper limit luminance of the luminance definition region R _max (> L _max ) instead of the maximum luminance L _max .

L (after correction) = (L (before correction) −L _C ) · {R _max / (R _max −L _C )} (2)

The specified luminance L _C is subtracted from the luminance L before correction on the right side of the equation (2), which means an operation of shifting the histogram by the specified luminance L _C as a whole on the low luminance side (FIG. 5 ( b)). On the right side of the equation (2), the difference between the luminance L before correction and the designated luminance L _C is multiplied by a factor {R _max / (R _max −L _C )}, as shown in FIG. This means an operation for extending the histogram (dashed line) before correction to the high luminance side. The corrected histogram (solid line) shown in FIG. 6 represents a histogram that approximates the luminance histogram of an image obtained when there is no reflection.

Further, the luminance L of the image may be corrected according to the equation (3) based on the iris control value A (<L _max ).

L (after correction) = (L (before correction) −L _C ) · {A / (A−L _C )} (3)

The designated luminance L _C is subtracted from the luminance L before correction on the right side of the equation (3), which means an operation of shifting the histogram by the designated luminance L _C on the whole low luminance side (FIG. 5 ( b)). On the right side of Expression (3), the difference between the luminance L before correction and the designated luminance L _C is multiplied by a factor {A / (A−L _C )}. This is shown in FIG. This means an operation of extending the histogram (broken line) to the high luminance side. The corrected histogram (solid line) shown in FIG. 7 represents a histogram that approximates the luminance histogram of an image obtained when there is no reflection. Note that the luminance is corrected by expanding the uncorrected histogram on the high luminance side and the low luminance side at the same ratio (for example, {A / (A−L _C )}) with the iris control value A as a reference. Also good.

Whether the image luminance L is corrected based on the upper limit luminance of the luminance definition region R _max , the image luminance L is corrected based on the iris control value A, or the image obtained when there is no reflection A histogram that approximates the luminance histogram is obtained, and a corrected image in which the difference in luminance is clarified or emphasized is acquired (see FIG. 4B).

  Then, an edge extraction process is performed to reveal an edge whose luminance changes beyond the threshold in the image scanning direction (lateral direction) with respect to the corrected image, and a lane (white line or yellow line) or roadway ( A lane mark such as “botts” is searched (FIG. 3 / S018). As a lane mark search method, a method disclosed in Japanese Patent Laid-Open No. 2006-309605, Japanese Patent Laid-Open No. 2006-331193, Japanese Patent Laid-Open No. 2007-018154, Japanese Patent Laid-Open No. 2007-026106, or the like is employed. Can be done. Further, based on the search result of the lane mark, it is determined whether or not a travel region in which left and right are defined by a pair of lane marks can be set (FIG. 3 / S020). When it is determined that the travel region can be set (FIG. 3 / S020... YES), the vehicle 1 deviates from this travel region by controlling the operation of one or both of the steering device 14 and the braking device 16. In order to prevent this, the driving support control for supporting the driving of the vehicle 1 is executed (FIG. 3 / S022). On the other hand, when it is determined that the travel area cannot be set due to a lane mark search failure or the like (NO in FIG. 3), the travel area setting and the travel support control are omitted.

According to the vehicle travel support device 10 that exhibits the above function, in the histogram representing the frequency distribution of the luminance of the image obtained through the camera 12, the cumulative value ΣN (L _j ) of the frequency from the low luminance side is the threshold value N _th . The luminance L _{i at} the time of reaching is set as the designated luminance L _C (see FIGS. 3 / S010, S014, and FIG. 5A). The designated luminance L _C represents the degree of increase in luminance due to reflection, such as increasing as the overall luminance of the image increases due to reflection on the camera 12 (see FIG. 5A). Therefore, by correcting the brightness of the image based on the specified brightness L _C , a corrected image from which the influence of the reflection is eliminated is obtained. Based on this corrected image, the situation in the traveling direction of the vehicle 1, specifically, The position of the lane mark can be accurately recognized (see FIG. 3 / S016, FIG. 4B and FIG. 5B and FIG. 5C). That is, the influence of the reflection on the camera 12 mounted on the vehicle 1 can be surely eliminated, and surrounding conditions such as whether the travel area of the vehicle 1 can be set can be accurately recognized (see FIG. 3 / S020). And based on this recognition result, each operation | movement of the steering apparatus 14 and the braking device 16 which are mounted in the vehicle 1 can be appropriately controlled from viewpoints, such as stable running of the vehicle 1. FIG.

Configuration diagram of the vehicle of the present invention Configuration explanatory diagram of the vehicle travel support device of the present invention The flowchart which shows the function of the vehicle travel assistance device of this invention (A) Explanatory drawing about the captured image before correction (b) Explanatory drawing about the captured image after correction (A) Explanation regarding a method for generating a histogram (b) (c) Explanatory drawing regarding a method for correcting the luminance of an image (A) Explanation regarding a method for generating a histogram (b) (c) Explanatory drawing regarding a method for correcting the luminance of an image (A) Explanation regarding a method for generating a histogram (b) (c) Explanatory drawing regarding a method for correcting the luminance of an image

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Vehicle, 10 ... Vehicle travel support apparatus, 12 ... Camera (imaging device), 110 ... 1st processing element, 120 ... 2nd processing element

Claims (3)

A vehicle travel support device for recognizing a travel environment of the vehicle based on an image representing a situation in a traveling direction of the vehicle obtained through a window through an imaging device mounted on the vehicle,
A first processing element that generates a histogram representing a luminance frequency distribution in the image, accumulates the frequencies from the lowest luminance in the histogram, and sets the luminance when the cumulative value of the frequencies reaches a threshold as the designated luminance When,
A second processing element that corrects the luminance of the image based on the designated luminance set by the first processing element and recognizes the state of the traveling direction of the vehicle based on the image whose luminance has been corrected. ,
The second processing element corrects the brightness of the image so that the histogram is expanded by the reciprocal of the difference between the highest brightness and the specified brightness with reference to the highest brightness in the histogram, or a brightness domain The brightness of the image is corrected so that the histogram is expanded by the reciprocal of the difference between the upper limit brightness and the specified brightness, or the iris control value is used as a reference. And the luminance of the image is corrected so that the histogram is expanded by the reciprocal of the difference between the specified luminance and the specified luminance .   A vehicle comprising: an imaging device; and the vehicle travel support device according to claim 1.   A vehicle driving support program for causing a computer mounted on a vehicle to function as the vehicle driving support device according to claim 1.