JP2005157670A - White line detecting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a white line detecting device for accurately detecting a traffic lane even under the influence of an environment of image pickup. <P>SOLUTION: Areas VR, VC and VL for detecting the luminance of the road surface are provided on a road surface which does not include white lines RR, RL, LR or LL with the supposed minimum curvature radius on the road surface in an image obtained by image pickup of the front part of its own vehicle, and an edge threshold for detecting the white lines is set by using the luminance value in each area. Thus, it is possible to set an edge threshold for more easily discriminating a pixel value corresponding to the road surface from a pixel value corresponding to the white line in contrast to a conventional edge threshold value set on the basis of the luminance value of the whole image. As a result, it is made possible to accurately detect the position of the traffic lane even when the difference between the pixel value corresponding to the road surface and the pixel value corresponding to the traffic lane is made smaller under the influence of an environment in image pickup. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a white line detection device.

2. Description of the Related Art Conventionally, a line detection device that detects a line indicating a traveling section of a vehicle from a road image obtained by a CCD camera has been proposed (for example, see Patent Document 1). According to the line detection method disclosed in Patent Document 1, for example, a threshold value is calculated based on the maximum value and the average value of the luminance signal for each element from the CCD camera, and the luminance is calculated using the threshold value. While the signal is binarized, the left and right edge portions of the line are detected based on the change state of the binarized data.
Japanese Patent Laid-Open No. 5-289743

  For example, when the CCD camera receives backlight, it is common to adjust the output gain, shutter speed, etc. of the CCD camera so that the brightness of the image falls within a predetermined range, but when the backlight is too strong, etc. In this case, the pixel value of the image cannot be adjusted within a predetermined range, and as a result, the entire image may become too bright. In such a case, the contrast between the pixel value corresponding to the road surface in the image and the pixel value corresponding to the lane becomes small, and is based on the above-described maximum value and average value of the luminance signal for each element from the CCD camera. With the threshold value, the right and left edge portions of the line cannot be accurately detected.

  The present invention has been made in view of such a problem, and an object of the present invention is to provide a white line detection device that can accurately detect a lane even when it is affected by an environment during photographing.

  The white line detection device according to claim 1, wherein the brightness of the road surface is detected in a region of the road surface excluding the white line provided on the road surface of the image photographed by the photographing unit and an image captured by the photographing unit. Area setting means for setting an area to be used, brightness detection means for detecting the brightness of the road surface from an image photographed by the photographing means, and brightness in the area detected by the brightness detection means, on the road surface in the image Threshold setting means for setting a threshold for detecting the provided white line, and white line detection means for detecting a white line provided on the road surface in the image based on the threshold set by the threshold setting means. Features.

  As described above, the white line detection device of the present invention sets a threshold for detecting a white line using the luminance in the area set in the road surface area excluding the white line provided on the road surface. As a result, an edge threshold is set that makes it easier to distinguish between a pixel value corresponding to a road surface and a pixel value corresponding to a white line than an edge threshold set based on the pixel value of the entire image. As a result, the white line can be accurately detected even when the difference between the pixel value corresponding to the road surface and the pixel value corresponding to the lane becomes small due to the influence of the environment at the time of shooting. .

  According to the white line detection device of the second aspect, the area setting means is limited to an area in which an object on the road surface is assumed not to exist in the area of the road surface excluding the white line provided on the road surface as the area. It is characterized by setting.

  For example, when an area where there is a high possibility that an object such as a preceding vehicle is present is set as an area for detecting the brightness of the road surface, the threshold for detecting the white line includes the influence of the brightness of the object. Cannot be detected accurately. Therefore, a threshold that does not include the influence of the luminance of the object can be set by setting the area only in a region where no object is assumed to exist. According to a third aspect of the present invention, there is provided object detection means for detecting an object on the road surface of the road, and the area setting means sets an area limited to an area where no object on the road surface is assumed to exist. At this time, the setting may be made based on the position of the object on the road surface detected by the object detecting means. For example, the inter-vehicle distance measuring means for measuring the distance to the preceding vehicle is provided as an object detecting means, and the area of the road surface in front of the distance to the measured preceding vehicle is set, thereby reducing the influence of the brightness of the preceding vehicle. A threshold value not included can be set.

  According to the white line detection device of the fourth aspect, the photographing unit photographs an image including the road surface of the road in the traveling direction of the vehicle, and the area setting unit is a left located as the area on the left side in the traveling direction of the vehicle. In the area of the road surface inside the white line and the right white line located on the right side of the traveling direction of the vehicle, a central area for detecting the brightness of the road surface of this area is set, and each of the white lines of the left white line and the right white line is set. A left area and a right area for detecting the brightness of the road surface in each of these areas are set in each of the outer road surface areas. Thereby, areas for detecting the brightness of the road surface can be set on both sides of the left white line and the right white line to be detected.

  According to the white line detection device of claim 5, the area setting unit includes, as the central area, the left area, and the right area, areas indicating shapes based on a minimum curvature radius of the white line that can be detected by the white line detection unit. It is characterized by setting. In this way, by setting an area having a shape based on the minimum curvature radius of the white line that can be detected by the white line detection means, it is possible to prevent the detection of a white line having a detectable radius of curvature in each set area. In addition, an area for detecting the brightness of the road surface in the vicinity of the left and right white lines can be set.

  According to the white line detection device of claim 6, the threshold value setting means uses the luminance value of the central area, or the central area, the left area, as the threshold value when detecting the left white line and the right white line in the image. And it sets using the value of the brightness | luminance which represents each area based on the brightness | luminance of each area of right area.

  Thereby, for example, even when the difference between the pixel value of the road surface and the pixel value of the white line is small, it is possible to detect the white line provided on the road surface by using the threshold value based on the luminance contrast. As described above, the area for setting the threshold value may be only the central area that is not easily affected by headlights such as oncoming vehicles, or the central area that includes the areas on the inner and outer road surfaces of the left and right white lines. , Left area, and right area. Further, as a luminance value representing these three areas, for example, an average value of luminance of each area may be calculated, and the calculated average value of luminance may be set.

  According to the white line detecting device of the seventh aspect, the threshold value setting means sets the threshold value for detecting the left white line using the luminance values of the left area and the central area, and detects the right white line. The threshold value is set using luminance values of the right area and the central area.

  In this way, by setting a threshold value based on luminance values representing two areas that are set across the white line to be detected as a threshold value when detecting the left and right white lines, for example, the preceding The threshold based on the brightness of the road surface near the left and right white lines even if the brightness of the road surface near the left and right white lines is different due to the influence of the shadow of the parallel running car and the headlights of the oncoming vehicle. By using, the left and right white lines can be accurately detected.

  According to the white line detection device of claim 8, the threshold setting means sets the threshold value when detecting the left edge portion of the left white line using the luminance value of the left area, and sets the right edge portion of the right white line. Is set using the luminance value of the right area as a threshold when detecting the right edge, and the luminance value of the central area is used as a threshold when detecting the right edge portion of the left white line and the left edge portion of the right white line. It is characterized by setting.

  Thus, by setting the threshold value based on the luminance value of the area adjacent to the edge portion to be detected as the threshold value when detecting the edge portions of the left and right white lines, for example, the white line to be detected Even when the left and right road surfaces have different luminances, the edge of each white line can be accurately detected.

  According to the white line detection device of claim 9, the area setting means divides the central area into a left central area indicating a shape along the left white line and a right central area indicating a shape along the right white line. One area is set, an area indicating the shape along the left white line is set as the left area, and an area indicating the shape along the right white line is set as the right area. Thus, it is possible to obtain the road surface brightness by using the left white line and the portion along the right white line to be detected, and it is possible to set a threshold value suitable for each of the left and right sides.

  According to the white line detection device of claim 10, the threshold setting means represents each area based on the luminance of each of the left center area and the right center area as a threshold when detecting the left white line and the right white line. Or a luminance value representative of each area based on the luminance of each of the left center area, right center area, left area, and right area.

  Thus, even when the difference between the pixel value of the road surface and the pixel value of the white line is small, the threshold value based on the luminance of the road surface in each area set along the left and right white lines is used to provide the road surface. It is possible to detect white lines.

  According to the white line detection device of the eleventh aspect, the threshold setting means uses a luminance value representing each area based on the luminance of each area of the left area and the left center area as a threshold for detecting the left white line. And a brightness value representative of each area based on the brightness of each area of the right area and the right center area is set as a threshold when detecting the right white line.

  Thereby, for example, even if the brightness of the road surface near the left white line and the brightness of the road surface near the right white line are different due to the influence of the shadow of the parallel running vehicle and the headlight of the oncoming vehicle, It is possible to accurately detect the left and right white lines using a threshold value set using the luminance of each area set across the white line to be detected.

  According to the white line detection device of claim 12, the threshold value setting means sets the threshold value for detecting the left edge portion of the left white line using the luminance value of the left area, and sets the right edge portion of the left white line. Is set using the luminance value of the left center area as a threshold when detecting the right edge, and is set using the luminance value of the right center area as a threshold when detecting the left edge portion of the right white line. The threshold value for detecting the right edge portion of the right area is set using the luminance value of the right area.

  Thereby, for example, even if the brightness of the left and right road surfaces of the white line to be detected is different, the brightness value of the area along each white line adjacent to the edge to be detected is set as a threshold value. The edge part of each white line can be accurately detected.

  According to the white line detecting device of the thirteenth aspect, the threshold setting means is based on at least one of the brightness level of the entire image obtained from the pixel value by the brightness detecting means and the brightness change in the horizontal direction in the image. , And a change means for changing an area for obtaining a luminance to be used when setting the threshold value.

  As a result, for example, even when the brightness of the entire image is too high or too low, or when the brightness in the horizontal direction is significantly different in the image, the brightness of the road surface according to the change in the brightness in the image is reduced. The area for detection can be changed.

  The white line detection device according to claim 14 is a gain detection unit that detects an output gain of a luminance signal of an image from the imaging unit, a temperature detection unit that detects a temperature around the imaging unit, a gain detection unit, and a temperature Noise level estimating means for estimating a noise level included in the luminance signal based on a detection result of at least one of the detecting means, and the white line detecting means has a noise level estimated by the noise level estimating means equal to or higher than a predetermined level. In some cases, the detection of the white line is stopped.

  Generally, when the output gain of the luminance signal of the image output from the photographing means is increased, the level of noise included in the luminance signal also increases. Therefore, if the output gain is increased above a certain output gain, the luminance signal Noise may be erroneously detected as a white line. In such a case, accurate white line detection cannot be performed. In general photographing means, when the temperature of the photographing means rises, the noise level of the luminance signal output from the photographing means tends to increase.

  For this reason, the noise level of the luminance signal is estimated based on the output gain of the luminance signal and the temperature around the photographing means, and the detection of the white line is stopped when the estimated noise level is equal to or higher than a predetermined level. As a result, noise included in the luminance signal can be prevented from being erroneously detected as a white line.

  According to the white line detection device of claim 15, the threshold value setting means presets the lower limit value of the threshold value when detecting the white line, and the threshold value set using the luminance in the area is less than the lower limit value. In this case, the lower limit value is set as a threshold value for detecting a white line.

  For example, a lower limit value of a threshold for detecting a white line is set based on a noise level normally included in the luminance signal, and if the threshold value based on the road surface luminance is less than the lower limit value, the lower limit value is set to a final value. You may make it set as a threshold value. As a result, noise included in the luminance signal can be prevented from being erroneously detected as a lane.

  Hereinafter, embodiments of the white line detection device of the present invention will be described with reference to the drawings. Note that the white line detection device according to the present embodiment is mounted on a vehicle such as an automobile, for example, photographs a road surface of a road ahead of the host vehicle, and detects a white line indicating a travel category provided on the road surface from the captured image. The present invention is applied to a lane departure warning device that generates an alarm when the host vehicle deviates from the lane to be traveled.

(First embodiment)
FIG. 1 is a block diagram showing the overall configuration of a lane departure warning device 100 according to this embodiment. As shown in the figure, the lane departure warning device 100 includes a camera 10, an image ECU 20, a power supply device 30, a lane departure warning ECU 40, and a warning device 50.

  The camera 1 is a CCD camera or the like, for example, and is provided at a position where the front of the host vehicle can be photographed. The camera 1 is configured to be able to adjust the shutter speed, the frame rate, the gain of a digital signal output to the image ECU 20, etc., according to an instruction from the image ECU 20. A digital signal of information (hereinafter referred to as pixel value information) relating to the degree of brightness is output to the image ECU 20 together with the horizontal / vertical synchronization signal of the image. The power supply device 30 is a device for supplying power to the camera 10 and the image ECU 20.

  The lane departure warning ECU 40 is connected to, for example, a radar device (not shown) using a laser beam, a radio wave such as a microwave or a millimeter wave, an ultrasonic wave, and the like, and is located ahead of the host vehicle detected by the radar device. The preceding vehicle information such as the inter-vehicle distance from the vehicle, the relative speed, the lateral deviation amount of the preceding vehicle with respect to the host vehicle in the vehicle width direction is acquired. Note that the preceding vehicle information acquired from the radar device is output to the image ECU 20.

  The lane departure warning ECU 40 determines whether or not the host vehicle deviates from the lane to be driven based on the lane position information from the image ECU 20, and if it is determined that the vehicle departs from the lane, a warning to that effect is given. Lane departure warning processing is executed to alert the driver. The alarm device 50 is configured by, for example, a display device, a voice output device, and the like, and outputs a display and sound corresponding to the lane departure warning in accordance with an instruction from the lane departure warning ECU 40.

  The image ECU 20 is configured by, for example, a microcomputer, and includes, for example, a CPU, a ROM, a RAM, an I / O, a bus that connects these, and the like (not shown).

  The image ECU 20 controls the shutter speed, frame rate, digital signal gain, and the like of the camera 10 based on pixel value information for each pixel from the camera 10 so that the brightness of the image falls within a predetermined range. Execute brightness control processing. Further, the image ECU 20 sets an edge threshold for detecting a white line on the road surface in the image using pixel value information for each pixel of the image, and detects the white line in the image based on the set edge threshold. Then, lane detection processing for outputting lane position information based on the detected white line to the lane departure warning ECU 40 is executed.

  Next, the lane detection process in the image ECU 20 will be described using the flowchart shown in FIG. In step (hereinafter referred to as S) 10 shown in the figure, an area for measuring the luminance value of the image used when setting the edge threshold value is set. FIG. 3 shows an image diagram when areas VL, VC, and VR are set in an image ahead of the host vehicle taken by the camera 10. Lanes RL and RR in the figure indicate lanes when the host vehicle has a right curve, and lanes LL and LR indicate lanes when the host vehicle has a left curve. is there.

  Areas VL, VC, and VR shown in FIG. 3 are areas for measuring the brightness (luminance) of the road surface of the road ahead of the host vehicle, and are areas of the road surface sandwiching the left and right white lines of the host vehicle, respectively. Set. Thereby, areas for measuring the brightness of the road surface are set on both sides of the left and right white lines to be detected. In addition, each area needs to be set to an area on the road surface on the image that is assumed not to include the white line ahead of the own vehicle, the preceding vehicle, or the like when the own vehicle is traveling in the lane.

  In other words, for example, when setting each area including a road surface area where an object such as a preceding vehicle is likely to exist, an edge threshold value that includes the influence of the brightness of the object may be set. The white line cannot be detected accurately. Therefore, by setting the areas VL, VC, and VR only on the road surface area where it is assumed that no object is present, an edge threshold value that does not include the influence of the luminance of the object such as the preceding vehicle is set. Can do.

  When setting the areas VL, VC, VR not including the preceding vehicle, for example, as shown in FIG. 3, the distance between the preceding vehicles ahead of the own lane included in the preceding vehicle information from the lane departure warning ECU 40, for example, Using the distance DC, the inter-vehicle distance DR to the preceding vehicle in front of the right lane, and the inter-vehicle distance DL to the preceding vehicle in front of the left lane, the area ahead from the position where the preceding vehicle exists in the areas VL, VC, VR You may make it exclude. Thereby, the edge threshold value which does not include the influence of the brightness | luminance of a preceding vehicle can be set reliably.

  In addition, the areas VL, VC, and VR are preferably shaped (curved) based on the minimum curvature (rotation) radius of the lane to be detected, as shown in FIG. Thereby, areas VL, VC, and VR that do not include a white line with a radius of curvature greater than or equal to the minimum radius of curvature can be set.

  In S20 illustrated in FIG. 2, only the pixel value for each pixel at a preset angle of view in the image is acquired from the pixel value information for each pixel of the image from the camera 10. For example, as the angle of view from which the pixel value is to be acquired, for example, as shown in FIG. 4, the position of the horizontal line including the lane ahead of the host vehicle is set as the angle of view to be acquired, and the pixel for each pixel in this horizontal line Get only the value. In addition, you may further set a vertical line with a horizontal line as an angle of view which should acquire a pixel value. In addition, the numerical value that can be taken by the pixel value in the present embodiment is, for example, a range from 0 to 255 (256 gradations).

  In S30, as shown in FIG. 4, the pixel value for each pixel in each of the areas VL, VC, VR is extracted, the extracted pixel value is converted into a luminance value, and the converted luminance value for each area The average value (luminance average value) is calculated. The conversion from the pixel value to the luminance value is performed using the map shown in FIG.

  As described above, the map shown in FIG. 5 is used when converting the pixel value for each pixel output from the camera 10 into a luminance value. The relationship between the pixel value and the luminance value shown in this figure is determined by the characteristics of the camera 10, and a general camera has a relationship like the characteristic a. When the pixel value and the luminance value are in a linear relationship as in the characteristic b of the figure, the camera 10 is converted only by multiplying a certain constant. (Alternatively, conversion can be omitted). Note that the upper limit of the luminance value in FIG. 5 is determined by the brightness control (shutter speed, frame rate, digital signal gain, etc.) of the camera 10.

  In S40, an edge threshold value for detecting a white line from the image is set using the luminance average values of the areas VL, VC, and VR calculated in S30. In the present embodiment, the edge threshold is set using the average luminance value of the area VC or the average average luminance value obtained by further averaging the average luminance values of the areas VL, VC, and VR.

  FIG. 6A shows the detection result of pixel values in a specific horizontal line within the above-mentioned angle of view. Ideally, as shown in FIG. The greater the difference from the pixel value of the white line, the easier it is to distinguish (distinguish) between the pixel value corresponding to the road surface and the pixel value corresponding to the white line. Less detection.

  However, when the camera 10 takes a picture while the vehicle is traveling, there are many cases where an ideal detection result as shown in FIG. For example, FIG. 6B shows pixel values detected when the camera 10 receives backlight. Normally, the camera 10 performs brightness control in accordance with an instruction from the image ECU 20. However, in a situation where the camera 10 receives backlight from the front in the shooting direction, the brightness control limit may be reached. As shown in FIG. 2B, the entire image becomes too bright (the pixel value is high in the entire image), and the difference between the pixel value corresponding to the road surface and the pixel value corresponding to the white line becomes small.

  Even when the host vehicle passes near the exit of the tunnel, the difference between the pixel value corresponding to the road surface and the pixel value corresponding to the white line may be small as shown in FIG. That is, when the vehicle leaves the tunnel, the brightness change of the entire image becomes significant, so the brightness control does not respond to the change (response delay), and as a result, the pixel value corresponding to the road surface and the white line correspond to The difference from the pixel value to be reduced is reduced.

  Further, as shown in FIG. 6C, even when the host vehicle passes near the entrance of the tunnel, the brightness control does not respond to the change in the brightness of the entire image, so that the pixel value corresponding to the road surface is obtained. And the pixel value corresponding to the white line may be small.

  Therefore, in S40, the edge threshold is set using the average luminance value of the area VC or the representative average luminance value obtained by further averaging the average luminance values of the areas VL, VC, and VR. In this way, by setting the edge threshold value for detecting the white line using the luminance value of the road surface excluding the white line, the pixel value corresponding to the road surface is set compared to the edge threshold value set based on the luminance value of the entire image. And an edge threshold that makes it easy to distinguish between pixel values corresponding to white lines. As a result, even when the difference between the pixel value corresponding to the road surface and the pixel value corresponding to the white line is small, the white line can be accurately detected.

  The area for setting the edge threshold value may be only the area VC which is not easily affected by headlights such as oncoming vehicles, or the areas VL, VC, including areas on the inner and outer road surfaces of the left and right lanes. It is good also as three areas of VR.

  Further, the edge threshold value finally set in S40 is set by converting the pixel value into a pixel value using the above-described map showing the relationship between the pixel value and the luminance value. For example, as shown in FIG. 7, a luminance value having a predetermined contrast A with respect to a road surface luminance value B as a luminance average value (or representative luminance average value) is set as a lower limit luminance value of a white line to be detected. An edge threshold value is set using a pixel value corresponding to the luminance value.

  Further, as shown in FIG. 7, when the brightness value (1 + A) B ′ having contrast A with respect to the road surface brightness value B ′ exceeds the brightness value upper limit, the pixel value upper limit (255) corresponding to the brightness value upper limit. ) Is used to set the edge threshold.

  In addition, for example, the lower limit value of the edge threshold value is set based on the noise level normally included in the digital signal from the camera 10, and the edge threshold value set using the average luminance value of each area is less than the lower limit value. In this case, the lower limit value may be set as the final edge threshold value. As a result, noise included in the digital signal from the camera 10 can be prevented from being erroneously detected as a white line.

  In S50 shown in FIG. 2, edge detection is performed in which the pixel value of each pixel within the angle of view described in S20 is compared with the edge threshold value set in S40 to extract a pixel position indicating a pixel value difference equal to or greater than the edge threshold value. I do. This edge detection is repeatedly performed from the leftmost pixel to the rightmost pixel of each horizontal line, for example, from the uppermost horizontal line in the angle of view to the lowermost horizontal line. In S60, the lane position is obtained using the horizontal pixel position of the edge for each horizontal line specified by the edge detection, and is output to the lane departure warning ECU 40.

  As described above, the lane detection processing in the lane departure warning device 100 according to the present embodiment provides the areas VR, VC, and VL in the road surface area excluding the white line, and detects the white line using the luminance value of each area. Set the edge threshold. As a result, an edge threshold value that makes it easier to distinguish the luminance value corresponding to the road surface and the luminance value corresponding to the white line than the edge threshold value set based on the luminance of the entire image is set. As a result, the white line can be accurately detected even when the difference between the pixel value corresponding to the road surface and the pixel value corresponding to the white line becomes small due to the influence of the environment at the time of shooting. .

(Modification 1)
As an area used when setting the edge threshold value of the present embodiment, for example, a road surface area that does not include left and right white lines is calculated based on lane position information output to the lane departure warning ECU 40 until the previous time. As shown in FIG. 8, areas VLL and VCL indicating the shape along the left white line and areas VRR and VCR indicating the shape along the right white line may be set.

  Thus, by providing areas VLL, VCL, VRR, VCR having shapes along the left and right white lines to be detected, and setting the edge threshold for detecting the white lines using the luminance values of each area, the road surface Even when the difference between the brightness of the white line and the brightness of the white line is small, the white line can be accurately detected.

  In addition, regarding the areas VLL, VCL, VRR, and VCR in this modified example, the inter-vehicle distance DC to the preceding vehicle in front of the own lane and the preceding vehicle in front of the right lane included in the preceding vehicle information from the lane departure warning ECU 40 are also included. By using the inter-vehicle distance DR and the inter-vehicle distance DL to the preceding vehicle ahead in the left lane, the front area may be excluded from the position where the preceding vehicle exists in the areas VLL, VCL, VRR, and VCR. Thereby, the edge threshold value which does not include the influence of the brightness | luminance of a preceding vehicle can be set reliably.

  In addition, when setting an edge threshold value when detecting a white line from an image using the average brightness values of the areas VLL, VCL, VRR, and VCR, the average brightness values of the areas VCL and VCR are further averaged. The edge threshold value may be set using the representative luminance average value or the representative luminance average value obtained by further averaging the luminance average values of the areas VLL, VCL, VRR, and VCR.

(Second Embodiment)
Since the second embodiment is often in common with that according to the first embodiment, a detailed description of the common parts will be omitted below, and different parts will be described mainly. In S40 of the lane detection process in the first embodiment, the luminance average value of the area VC shown in FIG. 3 or the representative luminance average value obtained by further averaging the luminance average values of the areas VL, VC, and VR is edged. It is set as a threshold value.

  On the other hand, in S40 of the lane detection process in the present embodiment, as the edge threshold value for detecting the left white line ahead of the host vehicle, the average luminance average value obtained by further averaging the luminance average values of the areas VL and VC (or The smaller one of the luminance average values of the areas VL and VC) is used to determine the edge threshold when detecting the left white line, and the areas VR and VC are used as edge thresholds for detecting the right white line ahead of the host vehicle. A representative luminance average value obtained by further averaging the luminance average values of each area (or the smaller value of the luminance average values of the areas VC and VR) is used to obtain an edge threshold for detecting the right white line. .

  In this way, by using the luminance average value of the area provided across the white line to be detected to obtain the edge threshold when detecting the white line, when the following image is captured, the lane Demonstrates the effect of improving detection accuracy.

  FIG. 9 shows a case where the brightness of the periphery of the left white line and the right white line to be detected is different due to the shadow of the parallel running vehicle that runs ahead of the host vehicle. As shown in FIG. 11A, an image shot in such a situation has a large difference between a pixel value corresponding to the right white line and a pixel value corresponding to the surrounding road surface, whereas the left white line The difference between the pixel value corresponding to and the pixel value corresponding to the road surface around it becomes small.

  FIG. 10 shows a case where the brightness of the left and right white lines to be detected is different depending on the headlights of the oncoming vehicle. As shown in FIG. 11B, the image captured in such a situation has a large difference between the pixel value corresponding to the left white line and the pixel value corresponding to the surrounding road surface. The difference between the pixel value corresponding to the right white line and the pixel value corresponding to the road surface around it becomes small.

  In this way, when the brightness around the left and right white lines is different, by setting the edge threshold value for detecting the white line from the luminance average value of each area provided across the white line to be detected, Even when the brightness around the left and right white lines is different and the difference between the pixel value corresponding to one of the left and right white lines and the pixel value corresponding to the surrounding road surface is small, the left and right white lines are accurately detected. can do.

  In the lane detection process according to the present embodiment, the edge threshold value needs to be changed according to the white line to be detected in S50 shown in FIG. Edge detection is performed on all the pixels of the angle of view from which the pixel value is to be obtained using the edge threshold value, and from the result, the horizontal pixel position and the lane center position corresponding to the left and right white line positions (white line predicted positions) ( The horizontal pixel position corresponding to the predicted lane center position is obtained.

  From the next edge detection, the representative luminance average obtained by further averaging the luminance average values of the areas VL and VC as the edge threshold value from the leftmost horizontal pixel position of the angle of view to the horizontal pixel position corresponding to the lane center position. Area VC, VR as an edge threshold value from the horizontal pixel position corresponding to the lane center position to the rightmost horizontal pixel position using the value (or the smaller value of the luminance average value of each area of areas VL, VC). The edge detection is performed using the representative luminance average value obtained by further averaging the luminance average values (or the smaller one of the luminance average values of the areas VC and VR). This may be repeated from the uppermost horizontal line in the angle of view to the lowermost horizontal line.

(Modification 2)
As shown in FIG. 3, the areas VL, VC, and VR are used as the areas for setting the edge threshold according to the present embodiment. However, the areas shown in FIG. 8 described in the first modification of the first embodiment are used. The edge threshold value may be set using the areas VLL, VCL, VRR, and VCR.

  That is, when the edge threshold value for detecting the left white line in front of the host vehicle is obtained, the representative luminance average value obtained by further averaging the luminance average values of the areas VLL and VCL (or the luminance average values of the areas VLL and VCL). When the edge threshold value for detecting the right white line in front of the host vehicle is obtained using the smaller value of the average brightness value of each area of the area VRR and VCR, the average brightness average value (or the areas VCR and VRR) is further averaged. Is set using the smaller value of the average luminance value of each area). As a result, the brightness around the left and right white lines is different, and even if the difference between the pixel value corresponding to one of the left and right white lines and the pixel value corresponding to the surrounding road surface is small, It can be detected accurately.

  In this modification as well, in S50 shown in FIG. 2, it is necessary to change the edge threshold according to the lane to be detected. In this case, first, a predetermined edge is limited to the first edge detection. Edge detection is performed on all pixels of the angle of view from which pixel values are to be acquired using a threshold value, and the horizontal pixel position and lane center position (lane line) corresponding to the left and right white line positions (white line predicted positions) are determined based on the result. The horizontal pixel position corresponding to (center predicted position) is obtained.

  From the next edge detection, the representative luminance average obtained by further averaging the luminance average values of the areas VLL and VCL as the edge threshold value from the leftmost horizontal pixel position of the angle of view to the horizontal pixel position corresponding to the lane center position. Area VCR, as an edge threshold value from the horizontal pixel position corresponding to the lane center position to the rightmost horizontal pixel position, using the value (or the smaller value of the luminance average values of the areas VLL and VCL). Edge detection is performed using a representative luminance average value obtained by further averaging the VRR luminance average values (or a smaller value of the luminance average values of the areas VCR and VRR). This may be repeated from the uppermost horizontal line in the angle of view to the lowermost horizontal line.

(Third embodiment)
Since the third embodiment is often in common with that according to the second embodiment, a detailed description of the common parts will be omitted below, and different parts will be described mainly. In S40 of the lane detection process in the second embodiment, a representative luminance average value obtained by further averaging the luminance threshold values of the areas VL and VC shown in FIG. 3 as the edge threshold value for detecting the left white line ahead of the host vehicle. The edge threshold value for detecting the right white line ahead of the host vehicle is set using a representative luminance average value obtained by further averaging the luminance average values of the areas VR and VC.

  On the other hand, in S40 of the lane detection process in the present embodiment, an edge threshold value for detecting the left edge portion of the left white line in front of the host vehicle is set using the luminance average value of the area VL, and the left in front of the host vehicle is set. The edge threshold value for detecting the right edge portion of the white line is set using the luminance average value of the area VC. Further, the edge threshold value for detecting the left edge portion of the right white line ahead of the host vehicle is set using the luminance average value of the area VC, and the area as the edge threshold value for detecting the right edge portion of the right lane ahead of the host vehicle is set. This is set using the average luminance value of VR.

  In this way, by using the luminance average value of the area adjacent to the edge of the white line to be detected and setting the edge threshold value when detecting the edge of the white line, the following image is captured. In this case, the effect of improving the accuracy of white line detection is exhibited.

  FIG. 12 shows, for example, that the pavement state is different on both sides of the left and right white lines in concrete pavement or asphalt pavement, or the pavement state is different on both sides of the left and right white lines in old and new pavements. The case where the brightness of the road surface of each both sides of a white line differs is shown. In the image of the road surface in such a paved state, as shown in FIG. 13, the difference between the pixel value corresponding to the white line and the pixel value corresponding to the road surface on either side of the white line is small.

  Thus, when the brightness of the road surface on both sides of the left and right white lines is different, as described above, the luminance average value of the area adjacent to the edge of the white line to be detected is detected at the edge portion of the white line. Therefore, even when the difference between the pixel value corresponding to the white line and the pixel value corresponding to the left or right road surface of the white line is small, the edge portions of the left and right white lines can be accurately determined. Can be detected.

  In the lane detection process in the present embodiment, it is necessary to change the edge threshold according to the white line to be detected in S50 shown in FIG. 2 as in the second embodiment. For this purpose, first, only for the first edge detection, edge detection is performed on all pixels of the angle of view from which a pixel value is to be obtained using a predetermined edge threshold value. The horizontal pixel position and the white line width (white line predicted width) corresponding to (predicted position) are obtained.

  From the next edge detection, the luminance average value of the area VL is used as the edge threshold value from the leftmost horizontal pixel position of the angle of view to the horizontal pixel position corresponding to the right edge portion of the left white line, and the left white line The luminance average value of the area VC is used as the edge threshold value from the horizontal pixel position corresponding to the left edge portion to the horizontal pixel position corresponding to the right edge portion of the right white line, and further, the horizontal pixel corresponding to the left edge portion of the right white line Edge detection is performed using the luminance average value of the area VR as an edge threshold value from the position to the rightmost horizontal pixel position of the angle of view. This may be repeated from the uppermost horizontal line within the angle of view from which the pixel value is to be acquired to the lowermost horizontal line.

(Modification 3)
As shown in FIG. 3, the areas VL, VC, and VR are used as the areas for setting the edge threshold according to the present embodiment. However, the areas shown in FIG. 8 described in the second modification of the second embodiment are used. The edge threshold value may be set using the areas VLL, VCL, VRR, and VCR.

  That is, an edge threshold value for detecting the left edge portion of the left white line in front of the host vehicle is set using the average brightness value of the area VLL, and the brightness of the area VCL is set as an edge threshold value for detecting the right edge portion of the left white line in front of the host vehicle. Set using the average value. Further, the luminance average value of the area VCR is set as an edge threshold value for detecting the left edge portion of the right white line ahead of the host vehicle, and the luminance of the area VRR is set as an edge threshold value for detecting the right edge portion of the right white line ahead of the host vehicle. Set using the average value.

  As a result, the brightness of the road surface on both sides of the left and right white lines is different, and even if the contrast between the pixel value corresponding to the white line and the pixel value corresponding to either the left or right road surface of the white line is small, It is possible to accurately detect the edge portion of the white line.

  In this modification as well, it is necessary to change the edge threshold according to the white line to be detected in S50 shown in FIG. For this purpose, first, only for the first edge detection, edge detection is performed on all pixels of the angle of view from which a pixel value is to be obtained using a predetermined edge threshold value. The horizontal pixel position corresponding to the predicted position), the horizontal pixel position corresponding to the lane center position (lane center predicted position), and the white line width (white line predicted width) are obtained.

  From the next edge detection, the luminance average value of the area VLL is used as the edge threshold value from the leftmost horizontal pixel position of the angle of view to the horizontal pixel position corresponding to the right edge portion of the left white line, and the left white line Edge detection is performed using the luminance average value of the area VCL as an edge threshold value from the horizontal pixel position corresponding to the left edge portion to the horizontal pixel position corresponding to the lane center position.

  In addition, as the edge threshold value from the horizontal pixel position corresponding to the center position of the lane to the horizontal pixel position corresponding to the right edge portion of the right white line, the luminance average value of the area VCR is used and further corresponds to the left edge portion of the right white line. Edge detection is performed using the luminance average value of the area VRR as the edge threshold value from the horizontal pixel position to the rightmost horizontal pixel position of the angle of view. This may be repeated from the uppermost horizontal line within the angle of view from which the pixel value is to be acquired to the lowermost horizontal line.

(Fourth embodiment)
Since this embodiment is common in common with those according to the first to third embodiments, a detailed description of common portions will be omitted, and different portions will be mainly described below. In S40 of the lane detection process in the present embodiment, the edge threshold described in each of the first to third embodiments is set in accordance with the brightness level of the entire image or the level of brightness change in the image. The area for obtaining the luminance value to be used in the process is changed.

  For example, when the brightness of the entire image is high / low, as described in the first embodiment, the area VC or a combination of the areas VL, VC, and VR is set as an area for obtaining the edge threshold value. Further, when the luminance is greatly different between the left side and the right side of the image, as described in the second embodiment, the combination of the areas VL and VC (or the average luminance value of each area of the areas VL and VC is small). Is set as an area for obtaining an edge threshold for detecting the left white line, and the combination of the areas VR and VC (or the smaller value of the average luminance value of each area of the areas VR and VC) ) Is changed as an area for obtaining an edge threshold when detecting the right white line.

  Furthermore, when the luminance in the image is irregularly different in the left-right direction, as described in the third embodiment, an edge threshold value for detecting the left edge portion of the left white line in front of the host vehicle in the area VL is set. The setting is changed as an area for obtaining, and the setting is changed as an area for obtaining an edge threshold value for detecting the right edge portion of the left white line and the left edge portion of the right white line in front of the host vehicle. The setting of VR is changed as an area for obtaining an edge threshold value for detecting the right edge portion of the right white line ahead of the host vehicle. Thereby, the area for detecting the brightness of the road surface according to the change of the brightness in the image can be changed.

(Modification 4)
The process of changing the setting as the area for obtaining the edge threshold according to the present embodiment is also for changing the area for obtaining the luminance value to be used when setting the edge threshold described in each of the first to third modifications. Can be applied.

  For example, when the brightness of the entire image is high / low, as described in the first modification, the area for obtaining the edge threshold value is a combination of areas VCL and VCR or a combination of areas VLL, VCL, VCR, and VRR. Set as. Further, when the brightness is greatly different between the left side and the right side of the image, as described in the second modification, the combination of the areas VLL and VCL (or the smaller average brightness value of each area of the areas VLL and VCL) Value) is changed as an area for obtaining an edge threshold for detecting the left white line, and the combination of the areas VCR and VRR (or the smaller value of the average luminance value of each area of the areas VCR and VRR) is changed. The setting is changed as an area for obtaining an edge threshold value when detecting the right white line.

  Further, when the luminance in the image is irregularly different in the left-right direction, as described in the modification example 3, in order to obtain the edge threshold value for detecting the left edge portion of the left white line in front of the host vehicle in the area VLL. The setting is changed as an area for changing the setting of the area VCL as an area for obtaining an edge threshold value for detecting the right edge portion of the left white line in front of the host vehicle.

  Also, the area VCR is changed as an area for obtaining an edge threshold value for detecting the left edge portion of the right white line in front of the host vehicle, and the area VRR is an edge for detecting the right edge portion of the right white line in front of the host vehicle. The setting is changed as an area for obtaining a threshold value. Thereby, the area for detecting the brightness of the road surface according to the change of the brightness in the image can be changed.

(Modification 5)
Generally, when the output gain of a digital signal of pixel value information output from the camera 10 is increased, the level of noise included in the digital signal also increases. Therefore, if the output gain is increased beyond a certain output gain, Signal noise may be erroneously detected as a line. In such a case, accurate white line detection cannot be performed. In general, when the temperature of the camera 10 increases, the noise level of the output digital signal tends to increase.

  FIG. 14 is a diagram illustrating a relationship between the ambient temperature of the camera 10 and the variance value (S) of noise included in the digital signal with respect to the output gain of the digital signal output from the camera 10. As shown in the figure, the variance value (S) of noise included in the digital signal tends to increase as the output gain of the digital signal increases from 0 [db].

  As shown in the figure, when the ambient temperature of the camera 10 is low, the noise variance (S) decreases, but the variance (S) increases as the temperature increases. There is a tendency.

  Therefore, the relationship between the temperature and the noise dispersion value (S) corresponding to the output gain as shown in FIG. 14 is obtained in advance by experiments, and temperature detection means (for example, a thermistor for detecting the ambient temperature of the camera 10). Etc.), the ambient temperature of the camera 10 is detected, the gain level of the digital signal is detected, and the noise level included in the digital signal is estimated by applying this ambient temperature and gain level to the relationship shown in FIG. can do.

  Then, it is determined whether or not the estimation result is equal to or greater than the noise variance value (S1). When the estimation result is equal to or greater than the noise variance value (S1), the white line detection is stopped. Noise included in the digital signal output from the camera 10 can be prevented from being erroneously detected as a white line.

1 is a block diagram illustrating an overall configuration of a lane departure warning device 100 according to a first embodiment. It is a flowchart which shows the flow of a lane detection process concerning 1st Embodiment. It is an image figure at the time of setting area VL, VC, and VR concerning 1st Embodiment. It is an image figure at the time of extracting the pixel value for every pixel in each area of area VL, VC, and VR concerning 1st Embodiment. It is a map which shows the relationship between a pixel value and a luminance value concerning 1st Embodiment. (A) is the figure which showed the ideal detection result of the pixel value in a certain horizontal line within an angle of view, (b) shows the pixel value detected when the camera 10 receives backlight. (C) is a diagram showing pixel values detected when the vehicle passes near the entrance of the tunnel. It is the figure which showed the image at the time of calculating | requiring the edge threshold value as a pixel value difference from a luminance average value (or representative luminance average value). It is an image figure at the time of setting the areas VLL and VCR of the shape along the left white line and the areas VRR and VCR of the shape along the right white line according to the first modification of the first embodiment. The figure which shows the case where the brightness of the periphery of the left white line and the right white line which should be detected by the shadow of the parallel running vehicle which drive | works ahead of the own vehicle according to 2nd Embodiment, the shadow of a sound insulation wall, etc. differs. is there. It is a figure which shows the case where the brightness of the periphery of the left white line which should be detected differs from the headlight of an oncoming vehicle concerning 2nd Embodiment. (A) is the figure which showed the pixel value of the image when the brightness around the left white line is darker around the right white line, and (b) is an image when the brightness around the right white line is brighter than the left white line. It is the figure which showed the pixel value of. It is a figure which shows the case where a pavement state differs in the both sides of each white line on either side by the new and old of pavement, etc. concerning 3rd Embodiment. It is the figure which showed the pixel value of the image when the brightness of the road surface of each both sides of each right and left each white line concerning the 3rd Embodiment differs. It is a figure which shows the relationship with the noise dispersion value (S) contained in the digital signal with respect to the surrounding temperature of the camera 10, and the output gain of the digital signal output from the camera 10 concerning the modification 5 of 4th Embodiment. .

Explanation of symbols

10 Camera 20 Image ECU
30 Power supply device 40 Lane departure warning ECU
50 warning device 100 lane departure warning device

Claims (15)

Photographing means for photographing an image including a road surface;
Area setting means for setting an area for detecting the brightness of the road surface in an area of the road surface excluding the white line provided on the road surface of the image photographed by the photographing means;
Luminance detecting means for detecting the luminance of an image photographed by the photographing means;
Threshold setting means for setting a threshold for detecting a white line provided on the road surface in the image using the brightness in the area detected by the brightness detection means;
A white line detection device comprising: a white line detection unit configured to detect a white line provided on the road surface in the image based on a threshold set by the threshold setting unit.   The area setting means sets, as the area, only an area in which an object on the road surface is assumed not to exist among areas of the road surface excluding a white line provided on the road surface. Item 2. The white line detection device according to Item 1. An object detection means for detecting an object on the road surface of the road;
The area setting means sets the area based on the position of an object on the road surface detected by the object detection means when setting an area limited to an area where no object on the road surface is assumed to exist. The white line detection device according to claim 2. The photographing means photographs an image including a road surface of a road in a traveling direction of the vehicle,
The area setting means includes, as the area, an area of a road surface inside a left white line located on the left side in the traveling direction of the vehicle and a right white line located on the right side in the traveling direction of the vehicle. A central area for detecting the road surface, and a left area and a right area for detecting the brightness of the road surface of each of the left white line and the white line of the right white line on the outer road surface area of each of the areas. The white line detection device according to claim 1, wherein the white line detection device is set.   5. The area setting unit sets an area indicating a shape based on a minimum curvature radius of a white line that can be detected by the white line detection unit as the center area, the left area, and the right area. White line detection device.   The threshold value setting means, as a threshold value when detecting a left white line and a right white line in the image, or a luminance value of the central area, or each area of the central area, the left area, and the right area 6. The white line detection device according to claim 4, wherein the white line detection device is set using a luminance value representative of each area based on luminance. The threshold setting means includes
As a threshold when detecting the left white line, set using a luminance value representing each area based on the luminance of each area of the left area and the central area,
6. The threshold value for detecting the right white line is set using a luminance value representative of each area based on the luminance of each area of the right area and the central area. White line detector. The threshold setting means includes
As a threshold when detecting the left edge portion of the left white line, set using the luminance value of the left area,
As a threshold when detecting the right edge portion of the right white line, set using the luminance value of the right area,
6. The white line detection according to claim 4, wherein the threshold value for detecting the right edge portion of the left white line and the left edge portion of the right white line is set using a luminance value of the central area. apparatus. The area setting means includes
As the central area, two areas divided into a left central area indicating a shape along the left white line and a right central area indicating a shape along the right white line are set,
As the left area, set an area showing the shape along the left white line,
The road white line detection device according to claim 4, wherein an area indicating a shape along the right white line is set as the right area.   The threshold setting means, as the threshold when detecting the left white line and the right white line, a luminance value representing each area based on the luminance of each area of the left central area and the right central area, or The white line detection according to claim 9, wherein the white line detection is set using a luminance value representative of each area based on the luminance of each of the left center area, the right center area, the left area, and the right area. apparatus. The threshold setting means includes
As a threshold when detecting the left white line, set using a luminance value representing each area based on the luminance of each area of the left area and the left center area,
10. The white line detection according to claim 9, wherein a brightness value representative of each area based on the brightness of each area of the right area and the left center area is set as a threshold for detecting the right white line. apparatus. The threshold setting means includes
As a threshold when detecting the left edge portion of the left white line, set using the luminance value of the left area,
As a threshold when detecting the right edge portion of the left white line, using the luminance value of the left center area,
As a threshold when detecting the left edge portion of the right white line, using the luminance value of the right center area,
The white line detection device according to claim 9, wherein the threshold value for detecting a right edge portion of the right white line is set using a luminance value of the right area.   The threshold setting unit determines a luminance to be used when setting the threshold based on at least one of the luminance level of the entire image detected by the luminance detection unit and the luminance change in the horizontal direction in the image. The white line detection device according to claim 4, further comprising a changing unit that changes an area to be obtained. Gain detecting means for detecting an output gain of a luminance signal of an image from the photographing means;
Temperature detecting means for detecting the ambient temperature of the photographing means;
Noise level estimation means for estimating a noise level included in the luminance signal based on a detection result of at least one of the gain detection means and the temperature detection means,
14. The white line detection unit according to claim 1, wherein when the noise level estimated by the noise level estimation unit is equal to or higher than a predetermined level, the white line detection unit stops detecting the white line. White line detection device.   The threshold value setting means presets a lower limit value of a threshold value when detecting the white line, and when the threshold value set using the luminance in the area is less than the lower limit value, the lower limit value is set as the threshold value. The white line detection apparatus according to claim 1, wherein the white line detection apparatus is set as a threshold value for detecting a white line.

Images