JP2009096274A - White line detector, parking and stop assist device, and white line detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a white line detector capable of extracting a white line even from an image in which an edge part is hardly detected. <P>SOLUTION: A picked-up image of the periphery of a vehicle by an image pickup unit A is prepared with the converted image subjected to the viewpoint conversion by a conversion image preparation unit C01b of an image edition unit C01 being an objective image, and saved in a memory unit B02. Then, an edge candidate area estimated to be a white line candidate area from the frequency of appearance of the brightness value of each pixel of the objective image is extracted by an edge part extraction unit C02 and an edge brightness value histogram preparation unit C03, the brightness value in a vicinity of the peak brightness value of the edge candidate area is determined as the binary threshold by a binary threshold determination unit C04. The objective image is converted into the binary image by a white line calculating unit C05 using the binary threshold, and a white line part is extracted therefrom, and saved in the data memory unit B02. Not the converted image, but the picked-up image or the compressed image with the pixel number being thinned from the picked-up image by a compressed image preparation unit C01a may be used for the objective image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a white line detection device, a parking / stop support device, and a white line detection method.

As an existing technique for detecting a white line on a road surface around a vehicle, there is a technique described in Japanese Patent Application Laid-Open No. 2004-118757 “Road Surface Traveling Lane Detection Device”. In this patent document 1, in order to detect the traveling lane of a vehicle, an edge whose luminance value changes sharply is detected from an image acquired by a camera mounted on the vehicle, and a plurality of detected edges are detected. A point is extracted as a feature point of a white line candidate. By applying a method applying RANSAC (Random Sample Consensus) to the extracted feature point for detecting a driving lane, a straight line fit is performed. The configuration of detecting a white line is employed.
JP 2004-118757 A

  However, since the technique as described in Patent Document 1 is based on the premise that the edge portion is detected, such as an image captured in a state in which the edge portion becomes distorted, the edge portion becomes dull. When extraction of the edge portion itself becomes difficult, a sufficient number of white line candidate feature points cannot be extracted, and white lines may be erroneously detected or may not be detected.

  The present invention has been made in view of such circumstances, and appropriately sets a binarization threshold value for performing binarization of an image based on a luminance value of an image to be processed obtained by imaging around the vehicle. An object of the present invention is to provide a white line detection device, a parking / stop support device, and a white line detection method capable of extracting a white line even if it is difficult to detect an edge portion of a white line part by determining It is said.

  In order to solve the above-described problem, the present invention acquires a captured image around a vehicle as a processing target image, and is estimated as a white line candidate region based on the appearance frequency of the luminance value of each pixel of the processing target image. By obtaining a luminance value near the peak luminance value in the edge candidate region and determining a binarization threshold for binarizing the processing target image, the binarized image converted by the binarization threshold is obtained. It is characterized by detecting a white line by using.

  According to the white line detection device, the parking / stop support device, and the white line detection method of the present invention, the binarization threshold is determined based on the appearance frequency of the luminance value of each pixel of the processing target image, and the processing target image 2 Since it is configured to detect the white line using the result of the digitization, non-detection or false detection of the white line that occurs when the white line is detected using only the edge portion that can be detected. Even if an image whose luminance value changes slowly and it is difficult to detect the edge part of the white line part is used as the processing target image, the white line can be reliably detected. The effect of becoming can be produced.

  DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of a white line detection device, parking / stop support device, and white line detection method according to the present invention will be described below in detail with reference to the drawings.

  Prior to the description of the embodiments of the present invention, the main features of the white line detection device, parking / stop support device and white line detection method according to the present invention will be described first. The present invention determines a binarization threshold based on the appearance frequency of the luminance value of each pixel of a processing target image, and detects a white line using the result of binarization of the processing target image. It is the main feature. Thus, even for an image in which the luminance value of the edge portion changes slowly and it is difficult to extract the edge portion, non-detection and false detection of the white line are reduced, and the white line is reliably detected. It is possible to detect.

  That is, the present invention is characterized in that a white line is detected based on the following procedure. First, an image captured around the vehicle is acquired as a processing target image to be processed, and the frequency of appearance of the luminance value of each pixel of the processing target image is created as a luminance value histogram. Based on this, an edge candidate area estimated as a white line candidate area is detected.

  Thereafter, the luminance value in the vicinity of the peak luminance value in the detected edge candidate region is set, for example, in the vicinity of the detected edge candidate region, and the luminance value in advance is higher than the peak luminance value in the edge candidate region. A luminance value that is lower by a predetermined value is determined as a binarization threshold value for binarizing the processing target image. Next, the processing target image is converted into a binarized image using the binarization threshold.

  After that, by detecting a part forming a white straight line as a white line from the converted binarized image, even if the processing target image is an image with a slanted edge portion, the white line is surely obtained. Can be detected.

  Depending on the mounting position of the imaging unit that captures an image around the vehicle on the vehicle, the captured image itself around the vehicle may be used as the processing target image, or the viewpoint is converted in a predetermined direction. The converted image may be created, and the converted image may be used as the processing target image. As the converted image, for example, an overhead image in which the viewpoint is placed in the upward direction of the vehicle is used. It is also good.

  Further, a compressed image obtained by extracting and compressing each pixel arranged linearly in the longitudinal direction and / or the lateral direction of the vehicle at a predetermined interval with respect to the captured image around the vehicle is processed as described above. It may be made as a target image, or, with respect to a captured image around the vehicle, a predetermined interval from one or a plurality of predetermined centers or two predetermined focal points inside or around the vehicle. A compressed image obtained by extracting and compressing each pixel arranged concentrically or elliptically as a representative pixel may be created as the processing target image.

  Further, when creating a compressed image, the luminance value of the representative pixel may be used as it is without being changed, or a weight including a luminance value of a pixel that is thinned out around the representative pixel is included. The averaged luminance value may be used as the luminance value of the representative pixel.

  Further, when creating a converted image as the processing target image as described above, when creating a compressed image, not only the captured image around the vehicle, but also a converted image obtained by converting the viewpoint of the compressed image. You may make it create as.

  Further, when creating a converted image as the processing target image, each pixel position of the captured image around the vehicle and the converted image can be created as quickly as possible from the captured image or compressed image around the vehicle. You may make it use the conversion map which shows the correspondence with each pixel position.

  Furthermore, when creating the luminance value histogram as described above, the processing target is not targeted for the entire area of the processing target image by referring to the change state of the luminance value of each pixel in the processing target image. An edge candidate region estimated as a white line candidate region may be extracted from the image, and a luminance value histogram related to the extracted region including the edge candidate region may be created as an edge portion luminance value histogram.

  In addition, when the converted image is used as the processing target image, information indicating the conversion level for converting each pixel of the captured image around the vehicle into each pixel of the converted image can be referred to, and the conversion level is set. Accordingly, it is divided into one or a plurality of image areas, and for each divided image area, the edge extraction filter used for extracting the edge candidate area is changed, and the edge portion luminance value histogram of each image area is changed. You may make it produce by dividing | segmenting. As a result, it is possible to more reliably detect the white line in a form that takes image distortion accompanying conversion into consideration.

  Further, when creating the edge portion luminance value histogram, the average value of the luminance value of each pixel in the edge candidate region, the average value of the luminance value of each pixel in the peripheral region of the edge candidate region, the edge bright portion of the edge candidate region, and The processing target image is divided into one or a plurality of image regions based on one or more of the appearance position with the edge dark portion and the luminance difference between the edge bright portion and the edge dark portion of the edge candidate region. One or a plurality of edge portion luminance value histograms corresponding to the respective image regions may be created. As a result, the white line can be detected more reliably in consideration of the influence of sunlight and shade.

  In addition, a plurality of imaging units (cameras) are mounted at a plurality of locations on the vehicle, a plurality of vehicle surroundings captured by each imaging unit are integrated into one integrated image, and the integrated image is processed as a processing target image. And the white line may be detected in a wider range of the vehicle.

(Configuration of the first embodiment of the present invention)
First, an example of the configuration of the white line detection device according to the present invention will be described. A configuration example of the main part of the white line detection device according to the present invention in the first embodiment is shown in FIG. FIG. 1 shows only the components directly related to the present invention. As shown in FIG. 1, the white line detection device of the present invention includes at least an imaging unit A, a storage unit B, and a calculation unit C mounted on a vehicle.

  The imaging unit A includes a camera (an imaging device such as a CCD camera) provided with an imaging element such as a CCD, and constitutes an image acquisition unit that captures an image around the vehicle and stores it in the storage unit B as a vehicle surrounding captured image. For example, as shown in FIG. 2, it is installed at the rear of the vehicle in order to capture the rear of the vehicle as a vehicle surrounding captured image. 2A and 2B are layout diagrams showing an example of the installation location of the imaging unit A of the white line detection device according to the present invention. FIG. 2A is a side view and FIG. 2B is a plan view.

  However, the installation location of the imaging unit A in the present invention is not limited to such a case, and it may be installed on a vehicle side portion such as a lower part of a door mirror in order to image the side of the vehicle, or the front of the vehicle is imaged. In order to do so, it may be installed at the front of the vehicle. Further, the number of imaging units A is not limited to one, and a plurality of imaging units A may be installed.

  The storage unit B includes an image memory unit B01 that stores a captured image around the vehicle and a data memory unit B02 that stores various data for detecting a white line.

  The image memory unit B01 is configured by a storage device such as a RAM (Random Access Memory), and stores the vehicle periphery captured image captured by the imaging unit A. The data memory unit B02 is configured by a storage device such as a RAM, and temporarily stores various data output by arithmetic processing or extraction processing performed to detect a white line on the road surface.

  That is, the data memory unit B02 transfers and stores the vehicle surrounding captured image itself stored in the image memory unit B01 as a processing target image for image processing, or predetermines a vehicle surrounding captured image as the processing target image. A compressed image in which the number of pixels is thinned out so as to include only pixels at every interval is stored, or a converted image converted into an image from a viewpoint position in a predetermined direction is stored as the processing target image.

  Further, the data memory unit B02 calculates the appearance frequency of the luminance value of the processing target image, thereby storing the appearance frequency of each luminance value as a luminance value histogram, or extracted based on the appearance frequency of the luminance value histogram. Among the edge candidate areas, the brightness value near the peak brightness value level, which is calculated by using the edge candidate area estimated as the white line candidate area, is stored as a binarization threshold. To do. Further, the data memory unit B02 converts the processing target image into a binarized image using the binarization threshold value and stores it, extracts a white line from the binarized image and stores it, or relates to the extracted white line Edit and store information in a form that can be output to the outside.

  The calculation unit C is configured by a known information processing device such as a processor, and includes an image editing unit C01, an edge part extraction unit C02, an edge portion luminance value histogram creation unit C03, a binarization threshold value determination unit C04, and a white line calculation unit. C05 and the result output unit C06 are included.

  Here, the image editing unit C01 is a part that edits a captured vehicle surrounding captured image as a processing target image, and the edge part extracting unit C02 selects an edge candidate region based on a change in luminance value of the processing target image. This is the part to be extracted. The edge part luminance value histogram creation unit C03 is a part that creates an edge part luminance value histogram indicating the appearance frequency of the luminance value of each pixel in the edge candidate area estimated as the white line candidate area among the edge candidate areas.

  Further, the binarization threshold value determination unit C04 determines a luminance value near the peak luminance value in the edge candidate region estimated as the white line candidate region as a binarization threshold value for binarizing the processing target image. The white line calculation unit C05 is a part that converts a processing target image into a binarized image using a binarization threshold and detects a white line based on the converted binarized image. The result output unit C06 is a part that edits and outputs information related to the detected white line to a format that can be output to the outside.

  In other words, the calculation unit C executes various processing programs by an internal CPU (Central Processing Unit) to extract the functions of the image editing unit C01 that realizes the image editing unit according to the present invention, and edge portion extraction. The function of the edge part extraction unit C02 that realizes the means, the function of the edge portion luminance value histogram creation unit C03 that realizes the edge portion luminance value histogram creation unit, and the binarization threshold value determination unit C04 that realizes the binarization threshold value determination unit The function of the white line calculation part C05 which implement | achieves a function, a white line detection means, and the function of the result output part C06 which implement | achieves a result output means are implement | achieved.

  The image editing unit C01 can output the vehicle surrounding captured image as it is as a processing target image, or can integrate a plurality of vehicle surrounding captured images into one integrated image and output it as a processing target image. Further, a compressed image creation unit C01a that creates a compressed image obtained by thinning out the number of pixels of the vehicle surrounding captured image or the integrated image and outputs it as a processing target image, or a viewpoint position in a predetermined direction for the vehicle surrounding captured image or the integrated image It further includes a converted image creation unit C01b that converts the image into a converted image viewed from above and outputs it as a processing target image.

  Here, the compressed image creating unit C01a that realizes the compressed image creating unit linearly extends in the longitudinal direction and / or the lateral direction of the vehicle at predetermined intervals from, for example, the vehicle periphery captured image and the integrated image. Extract only the arranged pixels, or concentric circles or ellipses at predetermined intervals as seen from one or more predetermined centers or two predetermined focal points inside or around the vehicle In this case, only the pixels arranged in are extracted, and a compressed image obtained by thinning out the number of pixels is stored in the data memory unit B02 as a processing target image. The compressed image creation unit C01a is not necessarily required depending on the storage capacity of the storage unit B and the white line detection speed, but in order to further improve the white line detection speed, the compressed image creation unit C01a It is desirable to have.

  Further, the converted image creating unit C01b that realizes the converted image creating means is not necessarily required depending on the mounting position of the imaging unit A on the vehicle or the number of the mounted imaging units A, but as shown in FIG. In addition, when only one imaging unit A is mounted for imaging the rear of the vehicle, for example, in the upward direction of the vehicle, the white line detection process on the road surface can be performed more reliably. In order to convert to a bird's-eye view image looking down from above, it is desirable to include a converted image creation unit C01b.

  Further, the function of the edge part extraction unit C02 that realizes the edge part extraction unit and the edge unit luminance value histogram creation unit C03 that realizes the edge unit luminance value histogram creation unit also include the mounting position of the imaging unit A on the vehicle and the imaging unit. Depending on the number of mounted A and the image quality of the captured image, the image is not limited to the edge candidate region, and is created by converting the captured image around the vehicle or the compressed image created by compressing the captured image or the viewpoint position of the captured image. The luminance value of each pixel may be calculated for the entire area of the converted image, and the luminance value histogram creating means for creating the appearance frequency of each luminance value as a luminance value histogram may be configured.

  Details of the functions of the image editing unit C01, the edge part extraction unit C02, the edge portion luminance value histogram creation unit C03, the binarization threshold value determination unit C04, the white line calculation unit C05, and the result output unit C06 will be described later.

(Operation of the first embodiment)
Next, an example of the operation of the white line detection device of FIG. 1 shown as the first embodiment of the present invention will be described using the flowchart of FIG. FIG. 3 is a flowchart for explaining the first embodiment of the operation of the white line detection apparatus according to the present invention, and shows an example of the white line detection method according to the present invention. In the flowchart of FIG. 3, as the image editing unit C01, the converted image creation unit C01b is used to convert the captured image around the vehicle into an overhead view image looking down from above the vehicle, and as a brightness value histogram creation unit. An edge portion indicating the frequency of appearance of the luminance value of each pixel in the region including the edge candidate region estimated as the white line candidate region by the edge portion luminance value histogram creation unit C03 by extracting the edge candidate region by the edge part extraction unit C02 The case where the brightness value histogram is created is shown.

  That is, the flowchart of FIG. 3 shows a processing target image created by converting a captured image around the vehicle into a bird's-eye view viewed from above the vehicle according to a predetermined threshold value based on a change in luminance value of each pixel. An edge portion that indicates the appearance frequency of the luminance value of each pixel by extracting an area having a large change as an edge candidate area and limiting the extracted edge candidate area to an edge candidate area estimated as a white line candidate area. By creating a luminance value histogram, a binarization threshold for converting the processing target image into a binarized image is determined, and the processing target image is converted into a binarized image using the binarization threshold. An example of the operation for extracting a white line from the above is described.

  In the flowchart of FIG. 3, first, an image around the vehicle imaged by the imaging unit A (image acquisition means) mounted on the rear side of the vehicle is acquired and stored in the image memory unit B01 (step D01). Here, instead of using the picked-up image around the vehicle as the processing target image as it is, the compressed image creating unit C01a of the image editing unit C01 performs pixels at predetermined intervals from the processing target image stored in the image memory unit B01. A compressed image obtained by extracting only the image may be temporarily created as a processing target image and stored in the data memory unit B02.

  Next, the obtained captured image itself around the vehicle or a compressed image in which the number of pixels is thinned out is converted into a converted image in which the viewpoint position is converted in a predetermined direction, for example, an overhead image as shown in FIG. The converted image creating unit C01b converts the image into an image looking down from above (step D02). As a result, a converted image as shown in FIG. 5 (that is, an overhead image behind the vehicle) is created as a processing target image. FIG. 4 is a schematic diagram showing a state in which the vehicle surroundings image captured by the imaging unit A is converted into a bird's-eye view image, and FIG. 5 is converted as a converted image by the converted image creation unit C01b of the white line detection device of the present invention. It is an image figure which shows an example of the made bird's-eye view image.

  As an image conversion method of the converted image creation unit C01b, a method using Affine conversion is generally known. However, with the aim of reducing the amount of calculation for conversion, FIG. Thus, for example, the conversion location for each pixel between each pixel position of the vehicle surrounding captured image F01 (or a compressed image with the number of pixels thinned out) acquired by the imaging unit A and each pixel position of the overhead image F02 after conversion May use the conversion map F03 registered in advance. FIG. 4B shows a specific image of the conversion map F03 for converting the captured vehicle surrounding captured image.

  Next, based on the converted image (overhead image) created as the processing target image in step D02, an edge candidate region whose luminance value changes beyond a predetermined threshold is determined using the edge part extraction unit C02. Obtain (step D03). For example, when the luminance value from the point G to the point G ′ is observed along the line G01 on the bird's-eye view image in FIG. 5, the change in the luminance value is as shown by a curve H01 in FIG. The result of the first derivative of the curve H01 is the curve H02, and the result of the second derivative is the curve H03.

  Here, FIG. 6 is a graph showing the luminance value of the converted image (overhead image) of FIG. 5 and its primary differential value and secondary differential value. The horizontal axis indicates the position on the G01 line of FIG. On the axis, the luminance values at the respective positions and the primary differential values and secondary differential values thereof are plotted by circles, x marks, and ■ marks. In FIG. 6, the luminance value of the curve H01 exceeds a predetermined luminance value threshold value H01a, and a predetermined predetermined value corresponding to one of the primary and secondary differential values of the curves H02 and H03, respectively. A portion having a value exceeding the differential luminance value thresholds H02a and H03a is extracted as an edge candidate region having a large change in luminance value. As a result, the edge candidate region includes a region estimated as a white line candidate region.

  Note that, as a technique used for extraction of edge candidate regions, the case where primary and secondary differential values are used has been described here. However, the method is not limited to such a case, and is generally used for edge detection in the image processing field. Of course, other methods such as the Sobel method that is commonly used may be used.

  Further, as shown in the lower half of FIG. 7, among the edge candidate areas of the entire image shown in FIG. 6, using the brightness value of the image of the edge candidate part having a large change in brightness value obtained in step D03. The edge portion luminance value histogram creation unit C03 creates an edge portion luminance value histogram in which the appearance frequency of luminance values limited to only the edge candidate region estimated as the white line candidate region is plotted, and is stored in the data memory unit B02 (step S22). D04).

  FIG. 7 is a graph showing a histogram (edge portion luminance value histogram) regarding the luminance value of an area extracted as an edge candidate area estimated as a white line candidate area among the edge candidate areas in FIG. A value (darker as it goes to the left side of the figure and brighter as it goes to the right side) is shown, and the vertical axis shows the appearance frequency of each luminance value. FIG. 7 shows a histogram i01 related to the luminance value of the entire image in the upward direction, and a histogram i02 related to the luminance value of only the edge candidate portion in the downward direction.

  For example, when the histogram of the entire image is calculated, a white line portion region i01a that is estimated to be a white line candidate region having a high brightness value (bright) and existing in a narrow range with a small area, as indicated by a curve i01 in FIG. And a brightness value histogram i01 having a peak brightness value in a road surface area i01b that has a relatively low brightness value (relatively dark), exists in a wide range, and has a different aspect from a white line candidate area having a large area. can get.

  Next, in the luminance value histogram i01, it is estimated as a white line candidate region out of two luminance value regions having a peak luminance value and a high appearance frequency, and corresponds to a portion of the white line portion region i01a showing a higher luminance value in a narrow range. The edge portion luminance value histogram i02 is generated by paying attention only to the edge candidate region.

  When the edge portion luminance value histogram i02 is created only for the edge candidate region estimated as the white line candidate region, as shown by the curve i02 in FIG. 7, the bright part (bright part) and the dark part (dark part) of the edge part are shown. ) And a luminance value histogram having peaks. This is because, in the edge portion luminance value histogram i02 of the edge portion, more frequency value voting is performed as a histogram on the bright portion (bright portion) and the dark portion (dark portion) of the edge portion. In the case of the curve i02, as described above, the frequency value becomes higher as the frequency value is reversed with respect to the histogram i01 of the entire image and proceeds downward in FIG.

  As described above, in the case of the edge portion luminance value histogram i02 focusing only on the edge portion, the luminance values corresponding to the luminance value corresponding to the white line portion i02a and the luminance value corresponding to the white line peripheral portion (for example, road surface) i02b, respectively. The value is output as a high appearance frequency. That is, histograms having respective peak luminance values i03 and i04 are obtained in the bright and dark portions of the edge candidate region. This edge portion luminance value histogram i02 is a histogram that is not affected by any area other than the edge candidate area, and it is possible to determine a threshold value for binarizing the processing target image in a stable state. ing.

  Next, using the edge portion luminance value histogram i02 as shown in FIG. 7 of the edge candidate region estimated as the white line candidate region obtained in step D04, the binarization threshold value determination unit C04 estimates the white line candidate region. The luminance value near the peak luminance value in the edge candidate area to be processed is determined as a binarization threshold for binarizing the processing target image, and stored in the data memory unit B02 (step D05).

  That is, the edge portion luminance value histogram i02 of the edge portion uses the primary differential value and the secondary differential value of the luminance value so as to observe the luminance value of the part where the luminance value changes beyond a predetermined threshold value. Therefore, as shown in FIG. 7, the peak is a peak luminance value i03 of a part estimated to be a white line as a bright part and a part estimated to be a periphery (for example, a road surface) adjacent to the white line as a dark part. Appear at two locations with the peak luminance value i04. Using this characteristic, it is estimated to be a white line between the peak luminance value i03 and the peak luminance value i04, and is determined in advance than the peak luminance value i03 indicating the peak value at a higher luminance value. The luminance value i05 shown in FIG. 7 that is smaller by the predetermined value is determined as the binarization threshold value i05 for monochrome determination for binarization.

  Next, in the white line calculation unit C05, using the binarized threshold value i05 obtained in step D04, the converted image (overhead image) created as the processing target image in step D02 is as shown in FIG. By converting to a binarized image and searching for an area having a white value in the binarized image from the binarized image, a white line is extracted and stored in the data memory unit B02 (step D06). FIG. 8 is an image diagram illustrating an example of a binarized image obtained by binarizing a region including a white line candidate region with respect to a converted image (overhead image) obtained by converting a vehicle surrounding captured image.

  Here, as a white line extraction method, for example, processing for extracting a linear component of a white line by performing Hough conversion on a binarized image in which a white line candidate area exists, using the outline portion of the area as a white line outline candidate, and the like. You should do it. Finally, by using the result output unit C06, the information on the part of the white line extracted in step D06 is edited into contents that can be output to the outside, and stored in a predetermined area of the data memory unit B02. Output.

  By determining the binarization threshold value for creating the binarized image for extracting the white line using the luminance value of each pixel of the processing target image by the procedure as shown in FIG. This is also the case for an image in a vehicle surrounding area that is 4 to 5 m away from the position, where there is a portion where the distortion is large and the edge portion is difficult to extract, or a cloudy image with no clear shadow and an unclear image. However, the white line and its peripheral part can be separated by binarization irrespective of the detection of the edge part where the luminance value changes sharply, and the non-detection / false detection of the white line can be reduced. It becomes possible.

  In other words, the configuration is such that the binarization threshold is determined based on the appearance frequency of the luminance value of each pixel of the processing target image, and the white line is detected using the binarization result of the processing target image. Therefore, it is possible to reduce white line non-detection and false detection that occur when white line detection is performed using only the edge portion that can detect a sharp change in luminance value. Even when an image in which the change in luminance value is gradual and it is difficult to detect the edge portion of the white line portion is used as the processing target image, the white line can be reliably detected.

  Furthermore, by creating an edge portion luminance value histogram limited to the edge candidate region estimated as the white line candidate region, the luminance value estimated to have the white line and the luminance value estimated to have the region around the white line And an appropriate binarization threshold can be set according to the luminance value of the image of the white line candidate area, and the white line area can be extracted more stably. Become.

  That is, when determining the binarization threshold, the binarization threshold is more appropriate by targeting only a part of the image area where the white line candidate area exists, instead of handling the entire area to be processed. Therefore, it is possible to more reliably prevent erroneous detection of white lines. Furthermore, since the amount of data to be processed can be reduced as compared with the case where the entire processing target image is targeted, the white line detection processing speed can be increased.

  As described above, the vehicle imaged by the imaging unit A without generating a converted image whose viewpoint has been converted by the converted image creation unit C01b, depending on the mounting position and the number of the imaging unit A mounted on the vehicle. The surrounding captured image itself or a compressed image obtained by thinning the number of pixels from the vehicle surrounding captured image by the compressed image creation unit C01a may be used as the processing target image.

  Further, as described above, the edge portion luminance value histogram limited to the edge candidate region estimated as the white line candidate region is not generated depending on the mounting position and the number of the imaging unit A mounted on the vehicle and the image quality of the captured image. Thus, the binarization threshold for creating the binarized image may be determined using the luminance image histogram regarding the vehicle surrounding captured image or the compressed image obtained by thinning the number of pixels from the vehicle surrounding captured image. .

(Configuration of second embodiment of the present invention)
Next, a configuration example of the white line detection device according to the present invention, which is different from FIG. 1, will be described. FIG. 9 shows a configuration example in the second embodiment of the main part of the white line detection device according to the present invention. FIG. 9 shows only components that are directly related to the present invention. Also in FIG. 9, as in the case of FIG. 1, the white line detection device of the present invention includes at least an imaging unit A, a storage unit B, and a calculation unit J mounted on the vehicle, and the calculation unit C of FIG. 1. Instead of the above, the calculation unit J is used. Here, the imaging unit A and the storage unit B in FIG. 9 are the same as those in FIG. 1, and only the calculation unit J will be described below.

  As in the case of FIG. 1, the calculation unit J is configured by a known information processing device such as a processor, and includes an image editing unit J01, an edge part extraction unit J02, an edge portion luminance value histogram creation unit J03, and a binarization threshold value. At least a determination unit J04, a white line calculation unit J05, and a result output unit J06 are included. The image editing unit J01 includes a compressed image creating unit J01a that creates a compressed image obtained by compressing a vehicle surrounding captured image, and a converted image creating unit J01b that creates a converted image by performing viewpoint conversion of the vehicle surrounding captured image or the compressed image. It consists of.

  The image editing unit J01, the edge part extraction unit J02, the edge portion luminance value histogram creation unit J03, the binarization threshold value determination unit J04, the white line calculation unit J05, and the result output unit J06 basically have the components shown in FIG. 1 implements substantially the same function, but for each component other than the result output unit J06, the converted image creation unit C01, edge part extraction unit C02, edge portion brightness value histogram creation unit C03 of FIG. Each of the determination unit C04 and the white line calculation unit C05 has a different function.

  That is, the calculation unit J executes various programs by an internal CPU (Central Processing Unit), and unlike the case of FIG. 1, the calculation target J is processed in the converted image creation unit J01b of the image editing unit J01. Depending on the degree of conversion of each pixel of the image or depending on the distribution of the luminance value of each pixel, the edge part extraction unit J02 and the edge luminance value histogram creation unit J03 select one or more processing target images. Is divided into image areas.

  Thereafter, the binarization threshold value determination unit J04 determines a binarization threshold value to be applied to each image region based on the edge portion luminance value histogram of each divided image region, thereby obtaining a white line calculation unit J05. 2 realizes a function of converting each image area into a binarized image and extracting a white line.

  As described above, the calculation unit J has a function of an image editing unit J01 that realizes an image editing unit different from the case of FIG. 1, a function of an edge part extraction unit J02 that realizes an edge unit extracting unit, and an edge portion luminance value histogram creation. The function of the edge portion luminance value histogram creation unit J03 that realizes the means, the function of the binarization threshold value determination unit J04 that realizes the binarization threshold value determination unit, and the function of the white line calculation unit J05 that realizes the white line detection unit In addition, the function of the result output unit J06 that implements the result output means is executed in the same manner as in the case of the result output unit C06 of FIG.

  As in the case of the image editing unit C01 in FIG. 1, the image editing unit J01 can output the vehicle surrounding captured image as it is as a processing target image, or can integrate a plurality of vehicle surrounding captured images into one integrated image. Then, it can be output as a processing target image, and further, a compressed image in which the number of pixels of the vehicle surrounding captured image and the integrated image is compressed can be generated by the compressed image generating unit J01a and output as the processing target image. Then, the converted image creation unit C01b can create a converted image obtained by viewing the vehicle periphery captured image and the integrated image from the viewpoint position in a predetermined direction, and output the converted image as a processing target image.

  Here, the compressed image creating unit J01a that realizes the compressed image creating unit is similar to the compressed image creating unit C01a in FIG. 1, for example, at a predetermined interval from the vehicle surrounding captured image or the integrated image. Extracting only pixels arranged in a straight line in the vertical direction and / or the horizontal direction, or by looking at one or more predetermined centers or two predetermined focal points inside or around the vehicle, Only the pixels arranged concentrically or elliptically are extracted at predetermined intervals, and a compressed image obtained by thinning out the number of pixels is stored in the data memory unit B02 as a processing target image.

  The compressed image creation unit J01a is not necessarily indispensable depending on the storage capacity of the storage unit B and the white line detection speed, but in order to further improve the white line detection speed, the compressed image creation unit J01a It is desirable to have.

  Also, the converted image generating unit J01b that realizes the converted image generating means converts the captured image around the vehicle and the integrated image into a converted image viewed from a viewpoint position in a predetermined direction and stores it as a processing target image in the data memory unit B02. However, unlike the converted image creation unit C01b of FIG. 1, the degree of conversion of each pixel when converted into a converted image is also output together with the converted image. The converted image creation unit J01b is not necessarily required depending on the mounting position of the imaging unit A in the vehicle or the number of the imaging units A mounted, but as shown in FIG. In the case where only the imaging unit A is mounted, for example, in order to convert to a bird's-eye view looking down from above the vehicle so that the white line detection process on the road surface can be performed more reliably In addition, it is desirable to include a converted image creation unit J01b.

  In addition, regarding the function of the edge part extraction unit J02 that realizes the edge part extraction unit and the edge unit luminance value histogram creation unit J03 that realizes the edge portion luminance value histogram creation unit, the mounting position of the imaging unit A on the vehicle and the imaging unit Depending on the number of A mounted and the image quality of the captured image, as in the case of the edge portion extraction unit C02 and the edge portion luminance value histogram creation unit C03 in FIG. Calculate the luminance value of each pixel for the entire area of the compressed image created by compressing the surrounding captured image or the converted image created by converting the viewpoint position of the captured image, and the frequency of appearance of each luminance value May be configured in the form of a brightness value histogram creating means for creating a brightness value histogram.

  Details of the functions of the image editing unit J01, the edge part extraction unit J02, the edge portion luminance value histogram creation unit J03, the binarization threshold value determination unit J04, the white line calculation unit J05, and the result output unit J06 will be described later.

(Operation of Second Embodiment)
Next, an operation example of the white line detection device of FIG. 9 shown as the second embodiment of the present invention will be described using the flowchart of FIG. FIG. 10 is a flowchart for explaining the second embodiment of the operation of the white line detection apparatus according to the present invention, and shows an example different from FIG. 3 of the white line detection method according to the present invention. Note that the flowchart of FIG. 10 performs basically the same operation as the flowchart of FIG.

  However, in the case of the flowchart shown in FIG. 10, unlike the flowchart of FIG. 3, for example, depending on the degree of conversion (degree of conversion) when the converted image is created as the processing target image, the processing target image is changed. An example in which an edge extraction filter used for extracting an edge candidate area is detected by dividing the image area into one or a plurality of image areas, detecting the magnitude of change in the luminance value of each pixel in each divided image area Show.

  Further, among the edge candidate regions extracted by the edge extraction filter, an edge portion luminance value histogram indicating the appearance frequency of the luminance value of each pixel in the edge candidate region estimated as the white line candidate region is divided into one or a plurality. An example of creating each image area is shown. Thereafter, a binarization threshold value to be applied to each image region is determined based on each edge portion luminance value histogram, and binarization is performed using an appropriate binarization threshold value that is different for each image region. An example is shown in which white lines are extracted based on a binarized image that is optimal in accordance with the degree of image conversion by generating a digitized image.

  In the flowchart of FIG. 10, Step K01 and Step K03 perform the same processing as Step D01 and Step D02 shown in FIG. 3, respectively. That is, first, an image around the vehicle imaged by the imaging unit A (image acquisition unit) mounted on the rear side of the vehicle is acquired and stored in the image memory unit B01 (step K01). Here, instead of using the captured image around the vehicle as the processing target image as it is, the compressed image creation unit J01a of the image editing unit J01 uses the pixel for each predetermined interval from the processing target image stored in the image memory unit B01. A compressed image obtained by extracting only the image may be temporarily created as a processing target image and stored in the data memory unit B02.

  Next, the acquired captured image itself around the vehicle or the compressed image in which the number of pixels is thinned out is converted into a converted image in which the viewpoint position is converted in a predetermined direction, for example, a converted image (overhead image as shown in FIG. 4 described above). ) Is converted by the converted image creation unit J01b (step K03). As a result, a converted image (that is, a bird's-eye view image behind the vehicle) as shown in FIG. 5 is created as a processing target image.

  However, in the case of FIG. 10, instead of the conversion map F03 shown in FIG. 4, a pixel around the vehicle surrounding captured image or the vehicle surrounding captured image is used as a conversion map to be referred to when converting into a converted image (overhead image) in step K03. It is assumed that a conversion map K02 having information indicating the conversion degree (degree of conversion) of each pixel when converting the thinned compressed image into a converted image (overhead image) is used. Note that the information regarding the degree of conversion is not limited to information held for each pixel in the conversion map K02, and information regarding a different degree of conversion may be held for each divided image area. You may make it memorize | store in the area | region different from K02.

  Next, for example, by using information regarding the degree of conversion of each pixel included in the conversion map K02, the edge extraction filter to be used in the edge part extraction unit J02 is changed, and a threshold value with a predetermined luminance value is set. Edge candidate regions that have changed in excess are extracted (step K04).

  For example, in the converted image (overhead image) after conversion shown in FIG. 5, when the vicinity of the point G near the vehicle and the vicinity of the point G ′ far from the vehicle are compared along the line G01, In the vicinity of the point G ′ where the degree of conversion of the image becomes large when converting to an image (overhead image), the image is distorted and the edge portion becomes duller than the vicinity of the point G where the degree of conversion of the image is small.

  In the converted image (overhead image) in such a state, a luminance value that rises in a substantially rectangular shape is observed from a white line portion near the point G as shown by a curve L01 shown in FIG. The differentiation can be calculated as a sharp edge shape that changes sharply in a narrow range like a curve L03, and a white line candidate region can be easily extracted from the calculated edge portion.

  On the other hand, with respect to the white line portion in the vicinity of the point G ′ where the image is distorted and the edge is dull, the luminance of a gentle curve shape such as the curve L02 in FIG. The edge portion that is observed as a change in value and is the second derivative thereof is also dull as shown by the curve L04. Therefore, using the same edge extraction filter as that near the point G makes it difficult to extract the edge portion.

  As described above, for example, the filter size for extracting the edge portion is properly used according to the degree of conversion of the image so that an image of a region having a large distortion can be dealt with. That is, as shown in FIG. 11A, in the case of edge extraction in an image area in the vicinity of a vehicle with a small degree of conversion and small distortion, as shown in FIG. 12A, the size is 3 × 3 pixels. On the contrary, if the image area is far away from the vehicle with a large degree of conversion and a large image distortion as shown in FIG. 11B, using an edge extraction filter, as shown in FIG. The edge extraction filter is changed in accordance with the degree of image conversion, such as using a large edge extraction filter having a size of 9 × 9 pixels.

  FIG. 11 is a graph for explaining an example of a change in luminance value near the edge portion. FIG. 11A shows a change in luminance value in the vicinity of the edge portion in an overhead image with a small degree of image conversion. FIG. 11B shows a change in the luminance value in the vicinity of the edge portion in the overhead image with a high degree of image conversion.

  FIG. 12 is a schematic diagram for explaining an example of the filter size of the edge extraction filter used for extracting the edge candidate region. FIG. 12A is a 3 × 3 pixel size, and FIG. FIG. 12B shows an example of a size of 5 × 5 pixels, and FIG. 12C shows an example of a size of 9 × 9 pixels. By switching the filter size of the edge extraction filter according to the degree of conversion of the image region, it is more reliable to detect the edge candidate region even in an image with a dull edge portion.

  Next, in the edge portion luminance value histogram creation unit J03 and the binarization threshold value determination unit J04, basically, the same processing as the processing performed in step D04 and step D05 in FIG. An edge portion luminance value histogram relating to luminance values around the edge portion estimated as a white line candidate region is created (step K05), and a binarization threshold value for binarizing the converted image (overhead image) is determined. Each is stored in the data memory unit B02 (step K06).

  However, the edge portion luminance value histogram created in the edge portion luminance value histogram creating unit J03 and the binarization threshold value determined in the binarization threshold value determining unit J04 are converted images (overhead images) to be processed images. Depending on the degree of conversion of each pixel at the time of creation, the filter size of the edge extraction filter, the condition of the luminance value in the processing target image, and which part of the edge it corresponds to, it is divided into not only one but also multiple To create.

  Specifically, for example, when the filter size of the edge extraction filter is switched according to the degree of conversion, a separate edge portion luminance value histogram is created for each filter size, and each edge portion Based on the luminance value histogram, a binarization threshold value to be applied to each image region is determined.

  As a result, an appropriate binarization threshold value to be applied to each converted image area is determined separately according to the degree of conversion, and the determined one or more binarization threshold values are set in each image area. By applying and converting the processing target image into a binarized image, it becomes possible to more reliably detect the white line in a form that takes into account image distortion caused by the conversion.

  In addition, if the bright and dark portions of the edge candidate area are created as separate edge portion luminance value histograms, how well the edge bright and edge dark portions that should have been separated can be separated. Can be observed. That is, when the peaks of the luminance values of the bright portion histogram and the dark portion histogram appear in a mixed manner, the image has a bright portion and a dark portion due to the influence of sunlight and the like. Since it is understood that it is inappropriate to apply only one binarization threshold to the entire image, a plurality of binarization thresholds such as a bright edge portion and an edge dark portion in each edge candidate region may be used. It is desirable to set a binary threshold value.

  Further, for example, a plurality of histograms may be created according to the average luminance value of the peripheral pixels in the edge portion. Thereby, for example, it is possible to calculate separate binarization threshold values for the shadow portion of the vehicle caused by the influence of sunlight and the sunny portion, thereby reducing the influence of disturbance. In addition, when creating a plurality of histograms according to the average luminance value of the image area estimated as the edge candidate area, or when creating a plurality of histograms according to the luminance difference between the bright part and the dark part of the edge candidate area, Similar effects can be obtained.

  In this way, by observing the distribution of the luminance values of the edge portion and its peripheral portion, an appropriate binarization threshold value to be applied to each region of the processing target image in consideration of the influence of sunlight and shade. By applying the determined binarization threshold value to each image region and converting the processing target image into a binarized image, the white line can be detected more reliably. Will be able to.

  Next, the white line calculation unit J05 created the processing target image in step K03 using the one or more binarization threshold values obtained in step K06 corresponding to one or more image regions. For the converted image (overhead image), after creating a binarized image for each image area, the binarized image of each image area is integrated, for example, to convert the binarized image as described above with reference to FIG. A white line is extracted and stored in the data memory unit B02 (step K07).

  Here, as a method for extracting a white line, as in the case of the first embodiment, for example, for a binarized image in which a white line candidate area exists, Hough conversion is performed using the outline portion of the area as a white line outline candidate. Then, a process for extracting the straight line component of the white line may be performed. Finally, by using the result output unit J06, the information on the part of the white line extracted in step K07 is edited into contents that can be output to the outside, and stored in a predetermined area of the data memory unit B02. Output.

  According to the procedure as shown in FIG. 10, in the present embodiment, unlike the first embodiment, the luminance value information of each pixel of the processing target image is used and the degree of conversion of the processing target image is set. If the edge extraction filter is changed, or the average brightness value of each pixel in the edge candidate area, the average brightness value of each pixel in the peripheral area of the edge candidate area, the edge bright part, the edge dark part The image is divided into one or a plurality of image regions based on one or more of the appearance position, the luminance difference between the edge bright portion and the edge dark portion of the edge candidate region, and the like.

  Thereafter, an edge portion luminance value histogram corresponding to each of the image regions divided into one or more is created, and a binary image is created from the image of each image region using each edge portion luminance value histogram. An appropriate binarization threshold is determined.

  As a result, a white line can be detected more reliably from a binarized image created using an appropriate binarization threshold for each image region. Thus, for example, in an area around the vehicle 4 to 5 m away from the vehicle position, an image area where distortion is large and the edge portion is difficult to extract, or is cloudy and has no shadow, Even in an environment where the amount of reflection is small, or in an environment where the sun's shadow is strong and the shadow of the vehicle extends, the white line and the surrounding area are separated by binarization. Thus, non-detection / false detection of white lines can be further reduced.

  In other words, by creating multiple histograms according to the conversion level of the image to be processed and the distribution of luminance values, the effects of disturbances such as image distortion and solar radiation can be further reduced and white line detection performance improved. It becomes possible to plan.

  Furthermore, according to the degree of conversion of the processing target image, in other words, by changing the edge extraction filter that extracts the edge candidate region according to the degree of distortion of the processing target image, the distortion is large and the edge portion is Even for an image that has become dull, it is possible to extract an edge candidate area estimated as a white line candidate area, and stably create an edge portion luminance value histogram used for calculating a binarization threshold value. It becomes possible.

  In addition, by generating a plurality of edge portion luminance value histograms used for calculating the binarization threshold value according to the analysis result of the characteristic of the luminance value in the edge candidate region, the binarization threshold value is appropriately set for each. It is possible to set an appropriate binarization threshold value even in the case of handling an image showing a plurality of characteristics in one processing target image due to the influence of sunlight and shadows. The white line can be reliably extracted from the binarized image created by using the image.

  In this embodiment as well, as in the case of the first embodiment, a converted image in which the viewpoint is converted by the converted image creation unit J01b is generated depending on the mounting position of the imaging unit A on the vehicle and the number of mounted units. Instead, a vehicle surrounding captured image itself captured by the imaging unit A or a compressed image obtained by thinning the number of pixels from the vehicle surrounding captured image by the compressed image creating unit J01a may be used as the processing target image.

(Other embodiments)
In any of the embodiments described above, by limiting the search range of the edge part that is a white line candidate region, it is possible to stably detect a white line and at the same time reduce the amount of data to be processed. It is possible to improve the processing speed. Here, as shown in FIG. 1 and FIG. 9 of each of the above-described embodiments, the compressed image creation unit C01a or the compressed image creation unit J01a compresses the number of pixels of the vehicle surrounding captured image obtained by imaging the vehicle periphery. By using the compressed image as the processing target image, it is possible to further reduce the amount of data to be processed. Hereinafter, a specific creation example for creating the compressed image will be described more specifically.

  First, for example, as shown by a dotted line around the vehicle in FIG. 13, when a captured image around the vehicle captured by the imaging unit A of the image acquisition unit is edited as a processing target image in the image editing units C01 and J01, Using compressed image creation units C01a and J01a, pixels arranged along one or more straight lines in the vehicle lateral direction and / or longitudinal direction are extracted as representative pixels at predetermined intervals. By setting each pixel of the target image, a compressed image having the number of pixels thinned out from the captured image around the vehicle is created as a processing target image, and an edge candidate area is searched based on the compressed image. You may do it.

  FIG. 13 is a schematic diagram illustrating an example of a case where a converted image (overhead image) obtained by performing viewpoint conversion from a compressed image obtained by compressing a vehicle surrounding captured image, and is a compression in which the number of pixels in the vertical and horizontal directions of the vehicle is thinned out. An example when an image is converted into a bird's-eye view image is shown.

  When such a bird's-eye view image (converted image) is used as a processing target image, the target is a compressed image obtained by extracting each pixel arranged in the vertical and horizontal directions of the vehicle at predetermined intervals from a captured image around the vehicle. As a processing target image converted to a bird's-eye view image, the amount of data handled can be reduced as compared to the original vehicle surrounding captured image, and the white line detection processing speed can be improved. .

  When a converted image like this is used as a processing target image, when scanning pixels existing on a straight line along the front-rear direction (vertical direction) of the vehicle, a white line appearing in the lateral direction of the vehicle is used. The target edge portion can be extracted. At this time, by using a filter having directionality (for example, a Sobel filter) as an edge extraction filter, for example, by using a filter for extracting a lateral edge, a white line in the vehicle lateral direction is detected, and a target direction is detected. Edge portions appearing in directions other than can be eliminated.

  Similarly, when extracting edges by scanning pixels that exist on a straight line along the lateral direction of the vehicle, a filter with directionality is used as an edge extraction filter, and a filter that extracts lateral edges is used. By doing so, it becomes possible to extract the white line of the vehicle front-back direction (vertical direction) which appears before and after the vehicle. In addition, when a directional filter is used as an edge extraction filter, an edge in a direction other than the target direction can be similarly removed by using a filter that extracts a vertical edge or a diagonal edge instead of a horizontal edge. Is possible.

  It should be noted that when extracting only representative pixels by thinning out the number of pixels in the compressed image creation units C01a and J01a, the luminance value of the representative pixel may be used as it is, or it is subject to thinning in the vicinity. The luminance value obtained by performing the weighted average including the luminance value of each pixel may be used as the luminance value of the representative pixel.

  Further, in the above description, in the image editing units C01 and J01 constituting the image editing means, the vehicle surroundings captured images captured by the image capturing unit A are compressed by the compressed image creating units C01a and J01a in the longitudinal direction and / or the horizontal direction of the vehicle. Although an example of reducing the amount of data to be processed by outputting a compressed image obtained by extracting and compressing each pixel arranged at a predetermined interval in the direction as a processing target image has been described. Is not limited to the case of using a compressed image compressed with respect to the processing target image, and any method may be used as long as the processing amount required for white line detection can be reduced and the processing speed can be improved.

  For example, instead of extracting each pixel arranged at a predetermined interval in the longitudinal direction and / or the lateral direction of the vehicle in the processing target image, the white line calculation units C05 and J05 use the binarization threshold value to perform the processing target image. Is converted into a binarized image, each pixel arranged in the vertical direction and / or the horizontal direction of the vehicle is extracted from the processing target image at predetermined intervals to create a binarized image. You may do it. By creating a binarized image based on the comparison result between the extracted luminance value of each pixel and the binarization threshold, it is possible to reduce the amount of data to be processed and create a binarized image. The white line detection process can be speeded up.

  Alternatively, in some cases, when the white line calculation units C05 and J05 extract a region corresponding to a white line from the binarized image, the binarized image is subjected to a vehicle longitudinal direction and / or a horizontal direction at predetermined intervals. An area corresponding to a white line may be extracted from each pixel arranged in the direction.

  As described above, by limiting the number of pixels to be processed, it is possible to reduce the amount of data to be processed, and it is possible to speed up the white line detection process.

  Next, as shown by a dotted line around the vehicle in FIG. 14, compression is performed when a captured image around the vehicle captured by the imaging unit A of the image acquisition unit is edited as a processing target image in the image editing units C01 and J01. Using the image creating units C01a and J01a, they are arranged concentrically or elliptically at predetermined intervals when viewed from one or more predetermined centers or two predetermined focal points inside or around the vehicle. The extracted image is extracted as a representative pixel and set as each pixel of the processing target image, thereby creating a compressed image having the number of pixels thinned out from the captured image around the vehicle as the processing target image. The edge candidate area may be searched based on the above.

  FIG. 14 is a schematic diagram showing an example different from FIG. 13 in the case where a converted image (overhead image) obtained by converting a viewpoint from a compressed image obtained by compressing a captured image around the vehicle, and is determined in advance inside or around the vehicle. View a compressed image from which the number of pixels is thinned by using, as representative pixels, pixels arranged concentrically or elliptically at predetermined intervals as viewed from one or more centers or two predetermined focal points. An example when converted into an image is shown.

  When such a bird's-eye view image (converted image) is used as a processing target image, a compressed image obtained by extracting pixels arranged concentrically or elliptically at predetermined intervals from a captured image around the vehicle is targeted. Thus, it is converted into a bird's-eye view image, and the amount of data handled can be reduced as compared with the original captured image around the vehicle, and the white line detection processing speed can be improved.

  Furthermore, by using an edge search method such as searching for an edge portion along an arc-shaped or elliptical curve using the converted image as a processing target image, a white line at a position close to the vehicle is removed from the vehicle. For example, when it is desired to search with priority over a white line at a farther position, an edge portion can be efficiently extracted.

  In the case of FIG. 14 as well, when the compressed image creation units C01a and J01a thin out the number of pixels and extract only the representative pixel, the luminance value of the representative pixel may be used as it is. You may make it make the luminance value of the result of having performed the weighted average including the luminance value of each pixel used as the thinning | decimation object of a neighborhood be a luminance value of a representative pixel.

  In the above description, in the image editing units C01 and J01 constituting the image editing means, the vehicle surroundings captured image captured by the imaging unit A is determined in advance in a concentric or elliptical shape by the compressed image creating units C01a and J01a. The example of reducing the amount of data to be processed by outputting a compressed image obtained by extracting and compressing each pixel arranged at a predetermined interval as a processing target image has been described. The present invention is not limited to the case where a compressed image compressed with respect to an image is used.

  For example, in the processing target image, each pixel arranged concentrically or elliptically at a predetermined interval when viewed from one or more predetermined centers or two predetermined focal points inside or around the vehicle. When the processing target image is converted into a binarized image using the binarization threshold value by the white line calculation units C05 and J05, the processing target image is concentrically formed at predetermined intervals. Alternatively, each pixel arranged in an elliptical shape may be extracted to create a binarized image.

  By creating a binarized image based on the comparison result between the extracted luminance value of each pixel and the binarization threshold, it is possible to reduce the amount of data to be processed and create a binarized image. The white line detection process can be speeded up, and the white line near the vehicle can be detected more efficiently.

  Alternatively, in some cases, when the white line calculation unit C05, J05 extracts a region corresponding to the white line from the binarized image, the binarized image may have one or a plurality of predetermined centers inside or around the vehicle. An area corresponding to a white line may be extracted from each pixel arranged concentrically or elliptically at predetermined intervals when viewed from two predetermined focal points.

  Thus, by limiting the pixels to be processed to concentric or elliptical positions, it is possible to speed up the white line detection process and prioritize white line candidates related to a specific position of the vehicle, for example, a nearby position. Therefore, the white line detection process can be further speeded up.

  Next, a configuration example different from the case of FIG. 2 of the imaging unit A configuring the image acquisition unit will be described. That is, in the present invention, as the image acquisition means, the image capturing means (camera) to be mounted on the vehicle may be mounted not only on one unit but on a plurality of arbitrary positions on the vehicle. In the case where a plurality of image capturing means (cameras) are installed, the vehicle surroundings captured images obtained from the plurality of image capturing means (cameras) are integrated as one integrated image, and the integrated images are By editing the image to be processed, it is possible to perform white line detection processing similar to that in each of the above-described embodiments.

  When white line detection processing is performed based on a single integrated image obtained by integrating a plurality of vehicle periphery captured images obtained from a plurality of image capturing units, for example, only one image capturing unit (camera) is mounted. Compared to the case, it is possible to detect a white line in a wider range around the vehicle, and a white line detection process with a wide field of view can be realized.

  Furthermore, for example, as a parking / stop support device that supports the operation of parking / stopping a vehicle, a parking / stop support device including at least the white line detection device or the device that realizes the white line detection method according to any of the above embodiments is configured. By doing so, when the vehicle is parked or stopped, information indicating the white line of the parking / stopping position can be stably output regardless of the distortion of the vehicle surrounding image captured around the vehicle or the presence or absence of sunlight. As a result, the driver can grasp the position where the vehicle should be parked and stopped more reliably, and further expands to a wider area by using a white line detection device equipped with a plurality of image capturing means (cameras). , Parking assistance can be provided.

It is a block block diagram which shows the structure in 1st Embodiment of the principal part of the white line detection apparatus by this invention. It is a layout showing an example of an installation location of an imaging unit of the white line detection device according to the present invention. It is a flowchart for demonstrating 1st Embodiment of operation | movement of the white line detection apparatus by this invention. It is a schematic diagram which shows a mode that the vehicle periphery picked-up image imaged by the imaging part is converted into a conversion image (overhead image). It is an image figure which shows an example of the conversion image (overhead image) converted by the conversion image preparation part of the white line detection apparatus of this invention. It is a graph which shows the luminance value of the converted image (overhead image) of FIG. 5, its primary differential value, and a secondary differential value. It is a graph which shows the histogram regarding the luminance value of the area | region extracted as an edge candidate area | region of FIG. It is an image figure which shows an example of the binarized image which binarized the area | region containing a white line candidate area | region regarding the converted image (overhead image) which converted the vehicle periphery picked-up image. It is a block block diagram which shows the structure in 2nd Embodiment of the principal part of the white line detection apparatus by this invention. It is a flowchart for demonstrating 2nd Embodiment of operation | movement of the white line detection apparatus by this invention. It is a graph for demonstrating an example of the change of the luminance value of the edge part vicinity. It is a schematic diagram for demonstrating an example of the filter size of the filter for edge extraction used in order to extract an edge candidate area | region. It is a schematic diagram which shows an example at the time of producing the conversion image (overhead image) which carried out viewpoint conversion from the compression image which compressed the surrounding vehicle captured image. It is a schematic diagram which shows the example different from FIG. 13 at the time of producing the conversion image (overhead image) which carried out viewpoint conversion from the compression image which compressed the surrounding vehicle captured image.

Explanation of symbols

A ... Imaging unit, B ... Storage unit, B01 ... Image memory unit, B02 ... Data memory unit, C ... Calculation unit, C01 ... Image editing unit, C01a ... Compressed image creation unit, C01b ... Conversion image creation unit, C02 ... Edge Part extraction unit, C03 ... edge portion luminance value histogram creation unit, C04 ... binarization threshold value determination unit, C05 ... white line calculation unit, C06 ... result output unit, F01 ... vehicle surrounding captured image, F02 ... overhead image, F03 ... conversion Map, H01a ... Luminance value threshold, H02a, H03a ... Differential luminance value threshold, i01 ... Luminance value histogram, i02 ... Edge portion luminance value histogram, i01a ... White line region, i01b ... Road surface region, i03, i04 ... Peak luminance value, i05: Binarization threshold, J: Calculation unit, J01: Image editing unit, J01a: Compressed image creation unit, J01b: Conversion image creation unit, J02: Edge unit Extractor, J03 ... edge intensity histogram creation unit, J04 ... 2 binarization threshold value determining unit, J05 ... white line calculator, J06 ... result output unit, K02 ... conversion map.

Claims (15)

  In a white line detection device that detects a white line on a road surface around a vehicle, an image acquisition unit that outputs a vehicle periphery captured image acquired by capturing an image of the vehicle periphery as a processing target image, and the output from the image acquisition unit Based on the brightness value histogram creation means for calculating the brightness value of each pixel of the processing target image and creating the appearance frequency of each brightness value as a brightness value histogram, and the brightness value histogram created by the brightness value histogram creation means Binarization threshold value determining means for determining a luminance value in the vicinity of the peak luminance value in the edge candidate region estimated as a white line candidate region as a binarization threshold value for binarizing the processing target image; Using the binarization threshold determined by the binarization threshold determination means, the processing target image is converted into a binarized image, and the converted binarized image is used as a basis. There are white line detection apparatus characterized by comprising at least a white line detecting means for detecting a white line, a.   In a white line detection device that detects a white line on a road surface around a vehicle, an image acquisition unit that captures an image around the vehicle and acquires the image as a vehicle periphery captured image, and edits the captured image around the vehicle acquired by the image acquisition unit An image editing means for outputting the editing result as a processing target image; and calculating a luminance value of each pixel of the processing target image output from the image editing means, and calculating an appearance frequency of each luminance value as a luminance value histogram A brightness value histogram creating means to create a brightness value near a peak brightness value in an edge candidate area estimated as a white line candidate area based on the brightness value histogram created by the brightness value histogram creating means A binarization threshold value determining unit that determines a binarization threshold value for binarizing the target image; and the binarization threshold value determining unit And a white line detecting means for converting the processing target image into a binarized image using the binarized threshold and detecting a white line based on the converted binarized image. A characteristic white line detector.   3. The white line detection device according to claim 2, wherein each of the pixels that are linearly arranged in the longitudinal direction and / or the lateral direction of the vehicle is extracted and compressed with respect to the captured image around the vehicle at predetermined intervals. A compressed image creating unit configured to create a compressed image as the processing target image; and outputting the compressed image created by the compressed image creating unit when the edited image to be processed is output by the image editing unit. A white line detection device characterized by:   In the white line detection device according to claim 2, with respect to the vehicle surrounding captured image, when viewed from one or more predetermined centers or two predetermined focal points inside or around the vehicle, at predetermined intervals, Compressed image creation means for creating a compressed image obtained by extracting and compressing each pixel arranged concentrically or elliptically as the processing target image, and outputting the edited processing target image by the image editing means A white line detecting device that outputs the compressed image created by the compressed image creating means.   5. The white line detection device according to claim 2, wherein a converted image in which a viewpoint is converted in a predetermined direction with respect to the vehicle surrounding captured image or the compressed image is generated as the processing target image. A white line detection apparatus comprising: a creating unit, wherein the converted image created by the converted image creating unit is output when the processed image edited by the image editing unit is output.   6. The white line detecting device according to claim 5, wherein the converted image created by the converted image creating means is a bird's-eye view image with a viewpoint position in an upward direction of the vehicle.   The white line detection device according to claim 5 or 6, wherein the converted image creating means uses a conversion map indicating a correspondence relationship between each pixel position of the vehicle surrounding captured image and each pixel position of the converted image. A white line detection apparatus that converts a captured image around a vehicle or the compressed image into the converted image.   8. The white line detection device according to claim 1, wherein the brightness value histogram creating unit determines an image area in which a change in brightness value of a pixel is greater than a predetermined threshold with respect to the processing target image as an edge candidate area. As an edge portion luminance value histogram, an edge portion extracting means for extracting as an edge portion and a luminance value histogram relating to an area including the edge candidate region estimated as a white line candidate region among the edge candidate regions extracted by the edge portion extracting means Edge portion luminance value histogram creating means, and when determining the binarization threshold value by the binarization threshold value determining means, the edge portion luminance value histogram creating means is used as the luminance value histogram. The binarization threshold value is determined using the edge portion luminance value histogram created by White line detection apparatus according to claim Rukoto.   9. The white line detection device according to claim 8, wherein when the binarization threshold value is determined by the binarization threshold value determination unit, in the edge candidate area estimated as a white line candidate area in the edge portion luminance value histogram. A white line detection apparatus, wherein a luminance value smaller than a luminance value indicating a peak by a predetermined value is determined as the binarization threshold value.   10. The white line detection device according to claim 8, wherein the processing target image is created by converting the viewpoint around the vehicle surrounding captured image or the compressed image obtained by compressing the vehicle surrounding captured image in a predetermined direction. When the converted image is used, the processing target image is converted into one or a plurality of image regions according to the degree of conversion for converting each pixel of the vehicle surrounding captured image or the compressed image into each pixel of the converted image. For each of the divided image regions, the edge extraction filter used for extracting the edge candidate region by the edge portion extraction unit is changed, and the edge portion luminance value corresponding to each of the divided image regions is changed One or a plurality of histograms are created by the edge portion luminance value histogram creating means. .   11. The white line detection device according to claim 8, wherein for the processing target image, an average value of luminance values of each pixel in the edge candidate region, and a luminance value of each pixel in a peripheral region of the edge candidate region. The processing target based on one or more of an average value, an appearance position of an edge bright part and an edge dark part of the edge candidate area, and a luminance difference between an edge bright part and an edge dark part of the edge candidate area An image is divided into one or a plurality of image regions, and the edge portion luminance value histogram corresponding to each of the divided image regions is created by one or a plurality of edge portion luminance value histogram creating means. Detection device.   12. The white line detection apparatus according to claim 10 or 11, wherein the binarization threshold value determination unit is based on each of the one or more edge portion luminance value histograms created by the edge portion luminance value histogram creation unit. The binarization threshold to be applied to the image area is determined, and the white line extraction means applies one or more of the binarization thresholds determined by the binarization threshold determination means to each image area. A white line detection apparatus, wherein the processing target image is converted into a binarized image.   13. The white line detection device according to claim 1, wherein the image acquisition unit includes a plurality of image imaging units that capture the captured image around the vehicle, and the plurality of vehicles captured by each of the image imaging units. A white line detection apparatus, wherein a surrounding captured image is integrated into one integrated image and output as the processing target image.   A parking / stop support device for supporting parking / stopping of a vehicle, comprising at least the white line detection device according to claim 1.   A white line detection method for detecting a white line on a road surface around a vehicle, wherein a captured surrounding image of the vehicle is acquired as a processing target image, and based on an appearance frequency related to a luminance value of each pixel in the processing target image, a white line candidate By determining a luminance value in the vicinity of a peak luminance value in an edge candidate region estimated as a region as a binarization threshold for binarizing the processing target image, the processing using the binarization threshold is performed. A white line detection method, wherein a white line is detected based on a binarized image obtained by converting a target image.

Images