JP4939137B2 - Lane recognition device - Google Patents

Description

  The present invention relates to a lane recognition device, and more particularly to a lane recognition device that detects a lane from a captured image in front of a vehicle.

  In recent years, with the aim of improving driving safety of automobiles, automatic control of vehicles, etc., image processing is performed on in-vehicle stereo cameras and monocular cameras to improve safety, avoid lane tracking, and avoid lane departure Development of a lane recognition device for recognizing a lane on a road, which is a precondition for automatic control such as the above, is underway (see, for example, Patent Documents 1 to 4).

  In these lane recognition devices, the lane is determined in the portion of the captured image area close to the host vehicle, and the lane is tracked in a further distant portion based on that, or if no lane is detected, it is detected before that It has been proposed to estimate the current lane position based on lane position information.

  However, even if a lane is detected, if it is erroneously detected, the vehicle is automatically controlled based on the erroneous information. In the worst case, accidents may occur.

Therefore, in the lane recognition device described in Patent Document 4, the detected lane is expressed by a lane model, and the number of data detected for each section by dividing the lane model into a plurality of sections in the distance direction ahead of the host vehicle. It is proposed to calculate the reliability indicating how reliable the detected lane is based on the number of data and the continuity of the lane position detected in the past.
JP-A-9-73545 Japanese Patent Laid-Open No. 10-49672 JP 11-86199 A JP-A-2001-92970

  However, the lane recognition device described in Patent Document 4 shows how reliable the detected lane position or lane model is as a reliability, but eventually provides the lane position and lane model to other recognition processes, Alternatively, the range of the detection area in the captured image in the next sampling cycle is changed based on the lane position and lane model.

  Therefore, whether the detected lane should be used for automatic control such as safety improvement, lane tracking, lane departure avoidance, etc., or which part of the detected lane should be used for such control even if it should be used, etc. There was a problem that a detection result that provides clear information about was not necessarily obtained.

  Further, in a conventional lane recognition device such as the lane recognition device described in Patent Document 4, when a preceding vehicle approaches and only a few left and right lanes of the host vehicle are detected, it is difficult to calculate reliability in the first place. There was also a problem of being.

  The present invention has been made in view of such circumstances, and clearly indicates whether or not the detected lane should be used for automatic control such as avoiding lane departure, and which part of the detected lane is reliable. An object of the present invention is to provide a lane recognition device capable of accurately indicating whether the degree is high.

In order to solve the above problem, the first invention provides:
In the lane recognition device,
Imaging means for imaging the front of the host vehicle at a predetermined sampling period and outputting a pair of images having a luminance value for each pixel;
Image processing means for calculating a distance in real space for each pixel of at least one image based on the pair of captured images;
Lane position detection means for detecting a pixel corresponding to one edge portion of the lane as a lane candidate point based on the luminance value and the distance for the one image, and detecting a lane position based on the detected lane candidate point; ,
And lane position estimation unit to estimate a lane position in the far region where the lane position based on the detected lane marking location has not been detected by the lane location detection means,
The detected lane position and the estimated lane position are combined to calculate the current lane position for each imaging sampling by the imaging means , and at a forward position away from the host vehicle, the current lane position The current lane position is determined to be valid when the absolute value of the difference from the lane position calculated in the past continues for a predetermined number of times that is equal to or less than the threshold value set for the forward position. And a judging means.

  According to a second aspect of the present invention, in the lane recognition device according to the first aspect, the determination unit divides the current lane position and the lane position calculated in the past into a plurality of corresponding sections and sets the number of sampling times. When the absolute value of the difference between the current lane position and the previously calculated lane position is not more than each first threshold value set for each of the categories, continuously for the first predetermined number of times or more, It is determined that the classification of the lane position is valid, and the classification other than the classification is judged invalid.

  According to a third aspect of the present invention, in the lane recognition device according to the second aspect of the present invention, the determining means includes only the lane position estimated by the lane position estimating means among the sections of the current lane position determined to be valid. And there is a section in which the lane candidate point has been detected by the lane position detecting means in the past, it is determined that each section far from the section is invalid, and the section and It is determined that each section of the current lane position closer to the vehicle than the section is valid.

  According to a fourth aspect of the present invention, in the lane recognition device according to the second aspect of the present invention, the determination means includes only the lane position estimated by the lane position estimation means among the sections of the current lane position that are determined to be valid. And there is a section exceeding the second predetermined number of times when the number of sampling times in which the lane candidate points are not continuously detected by the lane position detecting means is set, Each section far from the section is determined to be invalid, and each section of the current lane position closer to the vehicle than the section is determined to be valid.

  According to a fifth aspect of the present invention, in the lane recognition device according to any one of the second to fourth aspects, the determination means includes the current lane position and the current lane position among the sections of the current lane position determined to be invalid. When there is a segment in which the absolute value of the difference with the lane position calculated in the past is equal to or less than a second threshold set to a value smaller than the first threshold and the sampling count is continuous for a third predetermined count or more, It is determined that the current lane position is valid.

  According to a sixth aspect of the present invention, in the lane recognition device according to any one of the first to fifth aspects of the present invention, the determination means includes the current lane position and the past in all of a plurality of forward positions separated by a predetermined distance from the host vehicle. Only when the absolute value of the difference from the calculated lane position is equal to or less than each third threshold value set to a value greater than the first threshold value for each of the plurality of forward positions, continues for a fourth predetermined number of times or more. The validity or invalidity is determined.

  According to a seventh aspect of the present invention, in the lane recognition device according to the sixth aspect of the present invention, the determination means includes the current lane position and the lane position calculated in the past at at least one front position among the plurality of front positions. When the absolute value of the difference exceeds the third threshold value, it is determined that the entire current lane position is invalid.

  According to an eighth aspect of the present invention, in the lane recognition device according to the seventh aspect of the present invention, when the determination means determines that the entire current lane position is invalid, the current lane position is discarded, and has been validated in the past. A comparison with the current lane position in the next sampling cycle is performed using the lane position.

  According to a ninth aspect of the present invention, in the lane recognition device according to the eighth aspect of the present invention, when the determination means performs a comparison using the lane positions that have been validated in the past, the number of sampling times that is determined to be invalid is the fifth. If the predetermined number of times is exceeded, the previously valid lane position is discarded, the current lane position determined to be invalid this time is validated, and the current lane position in the next sampling cycle is used by using it. It is characterized by comparing with.

  According to a tenth aspect of the present invention, in the lane recognition device according to any of the sixth to ninth aspects, the third threshold value is changed according to a steering angle or a turning curvature of the host vehicle.

  According to the first invention, the lane position is detected by detecting the lane candidate point from the captured image, and the lane position is estimated based on the detected lane position in the far region where the lane candidate point is not detected. The left and right lane positions are detected. Then, the number of samplings in which the absolute value of the difference between the current lane position detected in the current sampling process and the lane position detected in the past is equal to or less than a threshold value is counted at a front position that is a predetermined distance away from the host vehicle. Then, when the count number continues for a predetermined number of times or more, it is determined that the current lane position is valid.

  Thus, not only the continuity between the lane position detected from the captured image and the lane position detected in the past, but also the current lane position on the basis that the continuity is continuous for a predetermined number of times of sampling. It is possible to accurately indicate whether the reliability of the calculated current lane position is high or low, and it is possible to accurately indicate the effectiveness.

  Therefore, it is clear whether or not automatic control can be performed based on the detected current lane position, that is, whether or not the detected current lane position should be used for automatic control such as lane departure avoidance. It becomes possible to show.

  According to the second invention, the current lane position and the lane position calculated in the past are divided into a plurality of corresponding sections, and the reliability of each section is determined and whether or not there is validity is determined. As a result, it is possible to more accurately indicate the reliability of the current lane position, and the effect of the first invention is more accurately demonstrated, and at the same time, which part of the detected lane position has higher reliability. Can be accurately shown.

  According to the third aspect of the present invention, the current lane position which is considered to be effective in the second aspect of the present invention is classified into only the lane position estimated by the lane position estimating means without detecting the lane candidate point. Is low in reliability, but if the lane position detection means has detected a lane candidate point in the past, the reliability of that category is high, so only that category and the category closer to the vehicle are valid By doing so, it becomes possible to accurately determine the validity of each section of the current lane position that is considered to be effective in the second invention, and the effect of each of the inventions can be exhibited more accurately. The

  According to the fourth aspect of the present invention, among the sections of the current lane position that are considered to be effective in the second aspect, the lane candidate point is not detected and only the lane position estimated by the lane position estimating means is used. Even if there is a lane candidate point detected in the past by the lane position detecting means, the reliability of the classification is high. However, after a certain amount of time, the reliability of the segment decreases, so by invalidating the segment and each segment farther away, each of the current lane positions that are considered valid in the second invention It becomes possible to determine the effectiveness of the classification more strictly, and the effects of the respective inventions can be exhibited more accurately.

  According to the fifth aspect of the present invention, among the sections of the current lane position determined to be invalid in each of the above inventions, the stricter condition set for the difference between the current lane position and the lane position calculated in the past is set. The effect of each of the above-described inventions can be exhibited more accurately by making the section that satisfies a considerable number of times effective.

  According to the sixth and seventh inventions, as a premise for the determination of each invention, the inventions are not made until the conditions set gently for the difference between the current lane position and the lane position calculated in the past are satisfied. If the above conditions are not met, the entire lane position is invalidated, and the current lane position with low reliability is detected such that the lane position varies and is detected at each sampling period. It is possible to prevent use in automatic control such as avoiding lane departure, and the effects of the above-described inventions can be exhibited more accurately.

  According to the eighth aspect of the present invention, when the entire current lane position is invalidated, the determination in each of the above inventions is made by using the lane position validated in the past instead of the detected current lane position. The reliability of the invention can be improved, and the effects of the inventions described above can be exhibited more accurately.

  According to the ninth invention, in the eighth invention, when the lane position that has been validated in the past is incorrect rather than the current lane position, the lane position that has been validated in the past is discarded. By determining that the current lane position determined to be invalid this time is valid, it becomes possible to improve the reliability of the determination, and the effects of the above-described inventions can be exhibited more accurately.

  According to the tenth invention, when the road shape is greatly curved, the determination can be made more accurately by changing the third threshold value according to the steering angle or turning curvature of the host vehicle. Thus, the effects of the respective inventions are more accurately exhibited.

  Hereinafter, embodiments of a lane recognition device according to the present invention will be described with reference to the drawings.

  As shown in FIG. 1, the lane recognition device 1 according to the present embodiment mainly includes an imaging unit 2, a conversion unit 3, an image processing unit 6, and a detection unit 9.

  The imaging means 2 is for imaging the periphery of the vehicle, and is configured to capture a landscape including a road ahead of the vehicle at a predetermined sampling period and output a pair of images. In the present embodiment, a stereo camera including a pair of main camera 2a and sub-camera 2b each incorporating a synchronized image sensor such as a CCD or CMOS sensor is used. In the present embodiment, CCD cameras are used for the main camera 2a and the sub camera 2b.

  The main camera 2a and the sub camera 2b are attached, for example, in the vicinity of a rearview mirror with a predetermined interval in the vehicle width direction. Of the pair of stereo cameras, a camera closer to the driver, as will be described later, a main camera 2a that captures an image serving as a basis for detecting distances and detecting lanes for each pixel, and a camera farther from the driver Is a sub-camera 2b that captures an image to be compared in order to obtain the distance or the like.

  A / D converters 3a and 3b as conversion means 3 are connected to the main camera 2a and the sub camera 2b, respectively. In the A / D converters 3a and 3b, each of the pair of analog images output from the main camera 2a and the sub camera 2b has a luminance value of a predetermined luminance gradation such as a gray scale of 256 gradations for each pixel. It is configured to be converted into an image.

  From the A / D converter 3a, a digital image converted from an image which is captured by the main camera 2a, the distance is calculated for each pixel and the lane is detected is output as a reference image, and the A / D converter 3b is output. The digital image captured and converted by the sub camera 3b is output as a comparison image.

  An image correction unit 4 is connected to the A / D converters 3a and 3b. In the image correction unit 4, the main camera 2a and the comparison image are output from the A / D converters 3a and 3b. Image correction such as correction of a luminance value including deviation due to an error in the mounting position of the sub camera 2b, noise removal, and the like is performed using affine transformation or the like.

  Note that the reference image T is, for example, image data including luminance values of 512 pixels in the horizontal direction and 200 pixels in the vertical direction as shown in FIG. 2, and the comparison image is also image data having a luminance value in each pixel. Each is configured to be output from the image correction unit 4. In addition, the reference image T in FIG. 2 is taken while traveling on a snowy expressway, and the pixel portion P in the figure is a pier portion of a road bridged above the expressway, and the pixel portion Pt is The tunnel-like portion below the bridge and the pixel portion S are pixel portions corresponding to the accumulated snow. Also, the broken line in the figure is the boundary between the portion D where snow is stepped on the road surface and the asphalt is visible and imaged relatively dark and the portion W where the snow remains on the road surface and imaged relatively brightly. Represents.

  An image data memory 5 is connected to the image correction unit 4, and the respective image data of the reference image T and the comparison image are stored in the image data memory 5 and simultaneously transmitted to the detection means 9. ing.

  An image processing means 6 is connected to the image correction unit 4. The image processing means 6 is mainly composed of an image processor 7 and a distance data memory 8.

  In the image processor 7, each setting area composed of each pixel or a block composed of a plurality of pixels of the reference image T based on the digital data of the reference image T and the comparison image output from the image correction unit 4 by the stereo matching process and the filtering process. The parallax dp for calculating the distance in the real space is calculated. The calculation of the parallax dp is described in detail in Japanese Patent Application Laid-Open No. 5-1114099 previously filed by the applicant of the present application.

  The image processor 7 divides the reference image T into pixel blocks of 4 × 4 pixels, for example, and calculates one parallax dp for each pixel block by stereo matching processing.

  Specifically, as described above, the luminance values p1ij of 0 to 255 are assigned to the 16 pixels constituting one pixel block, and the luminance value p1ij of the 16 pixels is a luminance value characteristic unique to the pixel block. Is forming. The subscripts i and j of the luminance value p1ij represent the i and j coordinates of a pixel when the lower left corner of the image plane of the reference image T is the origin, the horizontal direction is the i coordinate axis, and the vertical direction is the j coordinate axis. For the comparison image, the i-coordinate and the j-coordinate are similarly taken with the pixel previously associated with the origin of the reference image T as the origin.

The image processor 7 divides the comparison image into four-pixel-wide horizontal lines extending in the horizontal direction, takes out one pixel block of the reference image T, and horizontally in the horizontal direction of the corresponding comparison image one pixel at a time. That is, while shifting in the i direction, the absolute values of the differences between the luminance values p1ij of the 16 pixels in the pixel block of the reference image T and the luminance values p2ij of the 16 pixels in the corresponding comparison image are totaled as follows ( The pixel block on the horizontal line where the city block distance CB obtained by the equation (1) is minimum, that is, the pixel block on the comparative image having the luminance value characteristic closest to the pixel block of the reference image T is searched. .
CB = Σ | p1ij−p2ij | (1)

  The image processor 7 calculates the amount of deviation between the pixel block on the comparison image specified in this way and the pixel block on the reference image T, and uses the amount of deviation as the parallax dp to the pixel block on the reference image T. It is designed to be assigned. The reference image T to which the parallax dp is assigned in this manner is distinguished from the reference image T and is hereinafter referred to as a distance image. Note that the coordinates (i, j) on the distance image correspond to the coordinates (i, j) on the reference image T.

  The parallax dp is a relative displacement amount in the horizontal direction with respect to the mapping position of the same object in the reference image T and the comparison image derived from the main camera 2a and the sub camera 2b separated by a certain distance. The distance from the center position of the camera 2b to the object and the parallax dp can be associated based on the principle of triangulation.

Specifically, in real space, a point on the road surface directly below the center of the main camera 2a and the sub camera 2b is set as the origin, and the vehicle width direction of the own vehicle, that is, the X axis in the left-right direction, the Y axis in the vehicle height direction, When the Z axis is taken in the vehicle length direction, that is, the distance direction, the coordinate conversion from the point (i, j) on the distance image to which the parallax dp is assigned to the point (X, Y, Z) in the real space is as follows ( This is performed based on the equations (2) to (4).
X = CD / 2 + Z * PW * (i-IV) (2)
Y = CH + Z × PW × (j−JV) (3)
Z = CD / (PW × (dp−DP)) (4)
Here, CD is the distance between the main camera 2a and the sub camera 2b, PW is the viewing angle per pixel, CH is the mounting height of the main camera 2a and the sub camera 2b, and IV and JV are infinity points in front of the host vehicle. I-coordinate and j-coordinate, and DP on the distance image represent vanishing point parallax.

  That is, the center position of the main camera 2a and the sub camera 2b, more precisely, the distance L from the point on the road surface just below the center to the object and the parallax dp are obtained by setting Z in the above equation (4) as the distance L. Uniquely associated. Further, by substituting Z obtained from the parallax dp based on the equation (4) into the equations (2) and (3), it can be obtained in association with the i coordinate and the j coordinate on the distance image.

  Further, for the purpose of improving the reliability of the parallax dp, the image processor 7 performs a filtering process on the parallax dp thus obtained, and outputs only the valid parallax dp. That is, for example, even if a 4 × 4 pixel block consisting of only a road image is scanned on a horizontal line having a width of 4 pixels of the comparison image, all the portions of the comparison image where the road is imaged are correlated. Even if the corresponding pixel block is specified and the parallax dp is calculated, the reliability of the parallax dp is low. Therefore, such a parallax dp is invalidated in the filtering process, and 0 is output as the value of the parallax dp.

  Accordingly, the distance L between the pixels of the reference image T output from the image processor 7, that is, the parallax dp for calculating the distance in the real space for each pixel block of the reference image T is usually in the left-right direction of the reference image T. The data has an effective value only for a so-called edge portion in which the difference in luminance value p1ij is large between adjacent pixels.

  The parallax dp of each pixel block of the reference image T calculated by the image processor 7 is converted to Z, that is, the distance L based on the above equation (4), and is stored in the distance data memory 8 of the image processing means 6 as the above-described distance image. It is to be stored. In the processing in the detection means 9 described below, one pixel block on the distance image is treated as 4 × 4 pixels, and 16 pixels belonging to one pixel block are independent with the same distance L. It is processed as a pixel (i, j). Hereinafter, the distance L of the pixel (i, j) on the distance image is expressed as Lij. It is also possible to configure the processing in the detection means 9 not to process one pixel at a time, but to process a pixel block as one unit.

  The detection means 9 is composed of a microcomputer in which a CPU, ROM, RAM, input / output interface and the like (not shown) are connected to a bus. The detection means 9 is connected to a vehicle speed sensor A, a steering angle sensor B for measuring the steering angle of the steering wheel, and the like.

  In this embodiment, the detection means 9 includes a lane position detection means 91, a lane position estimation means 92, and a determination means 93.

  The lane position detection means 91 of the detection means 9 is based on the brightness value p1ij of each pixel (i, j) of the reference image T and the distance Lij of each pixel (i, j) of the distance image on the reference image T. The left and right lane positions are detected. In addition, the detection method of the lane position in the lane position detection means 91 is not limited to this embodiment, For example, it can also detect by other methods, such as the method as described in the said cited documents 1-4.

  Specifically, the lane position detecting means 91 follows a one-pixel wide horizontal line extending in the horizontal direction of the reference image T input from the image correction unit 4 from the lower side upward according to the basic flow shown in FIG. A search is performed while sequentially shifting one line at a time, and pixels that satisfy the conditions described below are detected as pixels that may represent lanes, that is, lane candidate points, and the left and right sides of the host vehicle are detected based on the detected lane candidate points. The lane position is detected. Hereinafter, since the j coordinate of each pixel (i, j) is the same on one horizontal line, the horizontal line in which the j coordinate of each pixel on the line is j is represented as a horizontal line j.

  In the present embodiment, the search for lane candidate points is performed by setting the areas around the right lane position LRlast and the left lane position LLlast detected in the previous sampling period as search areas Sr and Sl as shown in FIG. The lane candidate point is searched only within the lane.

  The lane position detection means 91 uses the horizontal line j to be searched, the left end of the right lane side search area Sr, and the right end of the left lane side search area S1 as intersection pixels as search start points is of the respective areas, and on the right lane side. The lane candidate points are searched for up to the search end point ie while offsetting one pixel each on the horizontal line j to the right and on the left lane side to the left on the horizontal line j.

  In the lane candidate point detection conversion process (step S10 in FIG. 3), which is the first process, the lane position detection means 91 detects lane candidate points according to the flowchart shown in FIG. Since detection of lane candidate points for the left and right search areas Sr and S1 is performed in the same manner, a case where the right lane side search area Sr is searched will be described below. The search for the left lane side search area S1 is configured to be performed simultaneously with the search for the right lane side search area Sr for the same horizontal line j.

  The lane position detection unit 91 first sets a search start point is and a search end point ie for the search region Sr on the horizontal line j of the reference image T (step S101), and offsets the pixel to be searched to the right ( Step S102), it is determined whether or not the search pixel satisfies the following first start point condition (Step S103).

[First start point condition]
Condition 1: The luminance value p1ij of the search pixel is greater than the road surface luminance value proad by the first start point luminance threshold pth1 and the edge intensity Eij represented by the luminance differential value is greater than or equal to the first start point edge intensity threshold Eth1. There is.
Condition 2: A point on the real space corresponding to the search pixel is on the road surface.
Here, the road surface luminance value proad is the appearance of the pixel luminance value histogram on the four horizontal lines j-1 to j-4 where the search is performed immediately below the horizontal line j currently being searched. The luminance value with the highest frequency is calculated for each horizontal line j.

  As shown in FIGS. 6A and 6B, the condition 1 is that the luminance value p1ij increases from the road surface luminance value proad to the threshold value pth1 or more and the edge intensity Eij as the luminance differential value is the threshold value Eth1 or more. This is a condition for finding a search pixel corresponding to one edge portion as a starting point that can be a lane candidate point. Condition 2 is a condition for excluding search pixels which are body parts such as pillars and bumpers of the preceding vehicle above the road surface and edge parts such as guardrails and telephone poles from the start point.

  When a pixel satisfying the condition 1 appears, the lane position detection unit 91 reads the distance Lij for the pixel from the distance data memory 8 and determines whether or not a point in the real space corresponding to the pixel is on the road surface. It is determined whether or not the condition 2 is satisfied.

  When the lane position detection unit 91 determines that the search pixel satisfies the first start point condition (step S103 in FIG. 5: YES), the lane position detection unit 91 sets the search pixel as the start point Ps on the RAM and sets the first lane width threshold value. Wth1 is set (step S104). The first lane width threshold value Wth1 is a threshold value for securing a search range when the start point Ps is detected.

  When the lane position detection unit 91 sets the start point Ps and the first lane width threshold Wth1, the lane position detection unit 91 continues to search rightward on the horizontal line j, and whether or not the search pixel satisfies the following second start point condition. Is determined (step S105). This is the pixel Psold corresponding to the end of the old lane Lold where the pixel set as the start point Ps has actually disappeared as shown in FIG. 7, and is newly overlapped with the old lane Lold. When the repainted lane Lnew is drawn, the set of the original start point Ps etc. is canceled, and the pixel Psnew corresponding to the end of the repainted lane Lnew is newly set as the start point Ps. It is a criterion for doing this.

[Second starting point condition]
Condition 3: The luminance value p1ij of the search pixel is larger than the road surface luminance value proad by the second starting point luminance value threshold value pth2, and the edge strength Eij represented by the luminance differential value is equal to or larger than the second starting point edge strength threshold value Eth2. Be. However, pth2> pth1.
Condition 4: The difference between the average luminance value from the start point Ps to the pixel adjacent to the left of the search pixel and the road surface luminance value proad is equal to or less than the first lane average luminance value threshold Ath1.
Condition 5: A point on the real space corresponding to the search pixel is on the road surface.

  If the search pixel satisfies the second start point condition (step S105: YES), the lane position detection unit 91 cancels the set of the original start point Ps and the first lane width threshold Wth1 and The search pixel is reset as a new start point Ps, and the first lane width threshold value Wth1 is also reset (step S106).

  FIG. 8A is a diagram for explaining the second start point luminance threshold value pth2 of the second start point condition, and FIG. 8B is a diagram for explaining the second start point edge intensity threshold Eth2. If the difference between the average luminance value of the region K from the original start point to the pixel adjacent to the left of the current search pixel and the road surface luminance value proad is larger than the threshold value Ath1 in the condition 4, the region K is corresponded. Since it is considered that the lane is currently marked as a living lane, there is no need to reset the starting point Ps.

The lane position detection means 91 continues searching rightward on the horizontal line j, and determines whether or not the search pixel satisfies the following end point condition (step S107 in FIG. 5).
[End condition]
Condition 6: The edge intensity Eij represented by the luminance differential value of the search pixel is equal to or less than the end point edge intensity threshold −Eth2, or the luminance value of the search pixel is smaller than the luminance value at the start point Ps.

  Although not shown, the end point Pe is a point corresponding to the edge portion on the opposite side of the lane, from the high luminance pixel corresponding to the lane to the low luminance value pixel corresponding to the road surface. Represents. In the present embodiment, the absolute value of the end point edge intensity threshold −Eth2 is set to be the same as the absolute value of the second start point edge intensity threshold Eth2.

  If a point satisfying the end point condition, that is, the end point Pe is not detected (step S107: NO), the lane position detecting unit 91 performs a search until the first lane width threshold value Wth1 is reached, and sets the first lane width threshold value Wth1. If the end point Pe is not detected even after reaching (step S108: YES), the setting of the start point Ps and the first lane width threshold Wth1 is canceled.

  If the search end point ie has not been reached (step S102: NO), the search on the horizontal line j is continued and the determination from the determination whether the search pixel satisfies the first start point condition (step S103). Repeat the above process. As can be seen from this flowchart, even if the search continues to the right from the start point Ps and exceeds the search end point ie, the horizontal line is not reached until the first lane width threshold Wth1 is reached. The search on j continues.

When the lane position detection unit 91 detects the end point Pe (step S107: YES), the average luminance value from the start point Ps to the pixel adjacent to the left of the end point Pe satisfies the following first average luminance value condition: Whether or not (step S109).
[First average luminance value condition]
Condition 7: The difference between the average luminance value from the start point Ps to the pixel adjacent to the left of the end point Pe and the road surface luminance value proad is equal to or less than the first lane average luminance value threshold Ath1.

  In the present embodiment, the first lane average luminance value threshold Ath1 in the condition 7 is the same value as the first lane average luminance value threshold Ath1 in the condition 4 of the second start point condition. As described above, the first lane average luminance value threshold value Ath1 is a threshold value that divides the average luminance value corresponding to the old lane Lold that has disappeared and the average luminance value corresponding to the newly repainted lane Lnew.

  The lane position detection means 91 corresponds to a lane in which the average luminance value from the start point Ps to the pixel adjacent to the left of the end point Pe satisfies the first average luminance value condition (step S109: YES), that is, the average luminance value is low. If there is a possibility that there is a newly painted lane near this lane, the lane candidate point is not already stored in the storage means (not shown) (step S110: YES). The coordinates (i, j) of the start point Ps are temporarily stored as lane candidate points (step S111), and the search on the horizontal line j is continued.

  On the other hand, the lane position detection unit 91 does not satisfy the first average luminance value condition in the average luminance value from the start point Ps to the pixel adjacent to the left of the end point Pe (step S109: NO). If it is determined that the average luminance value corresponds to a high lane, if there is a lane candidate point corresponding to a lane with a low average luminance value already saved in the search on the same horizontal line j (step S112: YES), The lane candidate point is deleted (step S113), the start point Ps having the higher average luminance value is stored as the lane candidate point (step S114), and the search on the same horizontal line j is terminated.

  In the present embodiment, as described above, when the search on the horizontal line j ends, when a lane with a high average luminance value is found and the search is terminated, only the lane with a low average luminance value is found and the search is completed. When the end point ie is reached, there are three cases where the lane is not found and the search end point ie is reached. Then, if a live lane is found with a high average luminance value, priority is given to the lane, and among the pixels satisfying the first start point condition or the second start point condition, the pixel closest to the host vehicle is set as the lane candidate point. To detect.

  When the search on the horizontal line j is completed, the lane position detection unit 91 determines whether or not the lane candidate points are stored (step S115). If the lane candidate point is not stored (step S115: NO), it is determined whether or not the horizontal line j has reached the 200th line (step S116). If the lane candidate point has not reached the 200th line (step S116: NO) Until the 200th line is reached, the above processing procedure is repeated for the horizontal line j + 1 that is one pixel above and transmitted from the image correction unit 4.

  If the lane candidate point is stored (step S115: YES), the lane position detection means 91 is a pixel of the edge portion between the snow and the road surface where the start point Ps stored as the lane candidate point has accumulated on the road. Determine whether or not. Specifically, paying attention to the fact that the average luminance of the pixels corresponding to snow is generally smaller than the average luminance of the pixels corresponding to the lane, the high luminance pixel portion is the portion corresponding to snow. Whether or not there is a possibility is determined based on the following determination conditions such as snow.

[Conditions for snow etc.]
Condition 8: The luminance differential value at the start point Ps corresponding to the detected lane candidate point is smaller than the third start point edge intensity threshold Eth3, and the number of pixels from the start point Ps to the end point Pe is the second lane threshold Wth2. Be bigger. However, Eth3> Eth1 and Wth2 <Wth1.

  Here, at the start point corresponding to snow, the luminance differential value at the start point corresponding to the lane is small, and as shown in FIG. 9, the luminance value often increases slowly without clearly rising at the start point. For this reason, a relatively large third start point edge intensity threshold Eth3 corresponding to the luminance differential value at the start point corresponding to the lane is set, and it is first sieved depending on whether the luminance differential value at the start point Ps is larger or smaller than the threshold value. Call it.

  The width of the pixel portion corresponding to snow usually appears larger than the width of the pixel portion corresponding to the lane. Therefore, the second lane width threshold value Wth2 is set to be relatively narrow so as to be about 20 cm in real space corresponding to the width of the normal lane, and the number of pixels from the start point Ps to the end point Pe is the second lane width. Further sieving is performed depending on whether the threshold value Wth2 is larger or smaller.

  If the lane position detection unit 91 determines that the start point Ps corresponding to the lane candidate point satisfies the determination condition such as snow and the high-luminance pixel portion may be a portion corresponding to snow (step S117: YES). ), The second lane average luminance threshold value Ath2 which is the basis of the determination of the second average luminance condition (step S119) described later is reset to a value higher than the current value (step S118). If the start point Ps corresponding to the lane candidate point does not satisfy the judgment condition such as snow (step S117: NO), the current value is used as the value of the second lane average luminance threshold Ath2.

Subsequently, the lane position detection unit 91 determines whether or not the average luminance value from the lane candidate point to the pixel adjacent to the left of the end point Pe satisfies the following second average luminance value condition (step S119).
[Second average luminance value condition]
Condition 9: The difference between the average luminance value from the lane candidate point to the left adjacent pixel of the corresponding end point Pe and the road surface luminance value proad is equal to or greater than the second lane average luminance value threshold Ath2.

  In the present embodiment, as described above, the first lane average luminance value threshold Ath1 defines the average luminance value corresponding to the old lane Lold that has disappeared and the average luminance value corresponding to the newly repainted lane Lnew. The second lane average luminance value threshold value Ath2 in condition 8 defines the difference between the minimum required average luminance value as the lane and the road surface luminance value proad, and is the first lane average luminance. A value smaller than the value threshold Ath1 is set as appropriate.

If the lane position detection unit 91 determines that the lane candidate point does not satisfy the second average luminance value condition (step S119: NO), the lane candidate point is deleted (step S120). On the other hand, if it is determined that the lane candidate point satisfies the second average luminance value condition (step S119: YES), the lane candidate point is left without being deleted. Thus, as shown in FIG. 10, the lane candidate point (I _j , J _j ) is finally detected on the horizontal line j.

Subsequently, the lane position detection unit 91 performs Hough transformation on the lane candidate points detected on the horizontal line j as described above (step S121). In the present embodiment, a known method is used for the Hough transform. Specifically, for example, the detected lane candidate point is a straight line on the reference image T i = aj + b (8)
Assuming that it exists above, I _j and J _j are
I _j = aJ _j + b (9)
And the formula (9) is
b = −J _j × a + I _j (10)
And can be transformed. As can be seen from the equation (10), when a lane candidate point (I _j , J _j ) is detected on the horizontal line j in the lane candidate point detection process, −J _j , I _j is set as a slope and a b-intercept. One straight line can be drawn on the ab plane which is a plane.

  The ab plane is divided into cells having a predetermined size, and when a straight line represented by the equation (10) is drawn, the count value of the cell through which the straight line passes is increased by one. It should be noted that since the predetermined a and b values correspond to each cell on the ab plane, selecting the cell on the ab plane means selecting the corresponding a and b values, This is synonymous with selecting a straight line having an inclination a and an i-intercept b on the reference image T as expressed by equation (8).

In the above description, the case of searching the right lane side search region Sr has been described. However, in the case of searching the left lane side search region S1 as well, the Hough transform is performed on the detected lane candidate point (I _j , J _j ). The ab plane is created separately from the case of the right lane side search area Sr, and a count value is added to each cell of the ab plane.

  The lane position detection means 91 searches the horizontal line j rightward and leftward while shifting the horizontal line upward by one pixel on the reference image T, and each time a lane candidate point is detected on each horizontal line. The Hough transform is performed, and the count values of the cells on the ab plane are added.

  Then, when the search is finally finished up to the top 200 horizontal line of the reference image T (step S116 in FIG. 5: YES), the lane position detecting means 91 then follows the lane straight line as the second process. The process proceeds to the detection process (step S20 in FIG. 3).

  In the lane straight line detection process (step S20), the lane position detecting means 91 has a single count value that is large from each ab plane of the right lane side search area Sr and the left lane side search area S1 obtained by the Hough transform or Each of the plurality of squares is extracted, and the extracted plurality of peak lines that meet the following selection conditions are rejected, and selection is performed for each of the right search and the left search until the peak line is narrowed to one. It is like that.

[Selection condition]
Condition 10: The position of the peak straight line at a predetermined forward distance exists inside the position of the vehicle width converted from the center of the host vehicle. However, when the lane position is continuously detected, the condition 10 is not applied if the amount of change from the previously detected lane position is within a predetermined threshold.
Condition 11: Parallelism with the estimated trajectory Lest of the host vehicle is larger than a certain threshold value.
Condition 12: A peak straight line not less than a specified value far_th when there is a peak straight line having a left-right difference distance from the center of the host vehicle at a predetermined forward distance that is more than a specified value far_th and a peak straight line that is not more than a specified value near_th. However, far_th> near_th.
Condition 13: Leave a peak straight line closest to the lane estimated center position estimated from the previously detected lane position and width.

  The lane position detection means 91 in FIG. 11 shows left and right peak straight lines r1 and l1 representing straight lines suitable as lane positions from the peak straight lines extracted in the right lane side search area Sr and the left lane side search area S1 in this way. The lane straight lines r1 and l1 as shown are selected one by one on the reference image T, and the lane straight line detection process (step S20) in FIG. 3 ends.

  Subsequently, the lane position detection means 91 proceeds to a lane position detection process (step S30 in FIG. 3) which is a third process. In the lane position detection process, linear or curved lanes existing on the left and right of the host vehicle are detected with reference to the lane straight lines r1 and l1 obtained in the lane straight line detection process.

  Specifically, the lane position detection unit 91 performs processing while scanning rightward and leftward on a horizontal line j having a single pixel width with a constant j coordinate in the reference image T, similarly to the lane candidate point detection conversion processing. Do. First, the right lane straight line r1 or the left lane straight line l1 is used as a reference, and lane candidate points whose difference in i direction from each straight line is within a threshold are recorded as lane positions until a predetermined number is reached. Lane candidate points whose i-direction difference from the right lane straight line r1 or the left lane straight line l1 is larger than a threshold value are excluded.

  The lane position detection means 91 records the predetermined number of lane positions in the vicinity of the lane straight lines r1 and l1, and the horizontal line j above the lane position detection means 91 uses the lane position detected last as a reference regardless of the lane straight lines r1 and l1. In the following manner, the lane is followed and the lane position is detected by following the curve even when the lane is curved.

  That is, when a predetermined number of lane positions are recorded, the lane position detecting means 91 is the i direction between the lane candidate point detected next on the horizontal line j above and the predetermined number of the last detected lane positions. , It is determined whether the displacement in the j direction is within a specified value. When it is determined that the value is within the specified value, the lane candidate point is recorded as a lane position.

  Thereafter, similarly, as shown in FIG. 12A, when a lane candidate point is detected on the horizontal line j, whether or not the displacement in the i direction and the j direction with respect to the previously detected lane position a is within a specified value. If it is determined that it is within the specified value, the lane candidate point detected this time is recorded as the lane position b.

  In addition, when a lane candidate point is detected on the horizontal line j and the displacement in the i direction and the j direction with respect to the previously detected lane position is not within the specified value, a straight line obtained by extending the previously detected lane position, That is, in FIG. 12B, if the displacement in the i direction from the straight line connecting the lane position b and the lane position a is within a specified value, the currently detected lane candidate point is recorded as the lane position c, d,. .

  The lane position detecting means 91 sequentially detects the lane position while shifting the horizontal line j above the reference image T while confirming the space in the real space between the left and right lanes, that is, the width of the traveling lane, and finally FIG. As shown in FIG. 3, the lane positions LRdec and LLdec are detected on the left and right of the host vehicle, and the lane position detection process (step S30) in FIG. 3 is terminated.

  The lane position detecting means 91 stores information such as the coordinates of each lane position detected as described above and the inclinations and intercepts of the lane straight lines r1 and l1 in a storage means such as a RAM (not shown).

  The lane position estimation means 92 of the detection means 9 estimates the lane position in the far region where the lane position was not detected based on the lane positions LRdec and LLdec detected by the lane detection means 91.

  In the present embodiment, as shown in FIG. 14, the lane position estimating means 92 divides the lane position LRdec detected by the lane detecting means 91 into two equal parts, and the lane position A closest to the host vehicle and the lane position B farthest from the host vehicle. The coordinates of the intermediate point C are calculated from these coordinates, and a line segment having the same length and the same inclination as the line segment AC and the line segment CB is extended far away from the farthest point B. For example, from the own vehicle to 100 m ahead By repeating the operation, the lane position LRinf in the far region is estimated. The lane position LLdec is similarly extended to estimate the lane position LLinf.

  Note that the method of estimating the lane position in the lane position estimating unit 92 is not limited to the present embodiment, and any other method can be employed as long as the lane position in the far region can be estimated.

  The lane position estimating means 92 stores information such as the inclination and intercept of each lane position LRinf and LLinf estimated as described above in a storage means such as a RAM (not shown).

  The determination means 93 of the detection means 9 calculates the current lane positions LRpre and LLpre from the detection results and the estimation results by the lane position detection means 91 and the lane position estimation means 92, and whether the current lane positions LRpre and LLpre are valid or invalid. To come to judge. Since processing is performed in the same way for the left and right lane positions LRpre and LLpre, a case where processing is performed for the right lane position LRpre will be described below. Processing is performed simultaneously for the left and right lane positions LRpre and LLpre.

  The determining means 93 calculates the current lane position LRpre by combining the lane position LRdec detected by the lane position detecting means 91 and the lane position LRinf estimated by the lane position estimating means 92 as shown in FIG. . Then, based on the vehicle speed of the host vehicle and the steering angle of the steering wheel input from the vehicle speed sensor A and the steering angle sensor B, the current position of the lane position LRlast calculated in the previous sampling cycle is calculated.

  Next, as shown in FIG. 15, the judging means 93 divides the current lane position LRpre and the previous lane position LRlast into a plurality of sections at intervals of 10 m, for example, as the distance from the host vehicle. In the position closest to the host vehicle, that is, for example, in a section of 10 m to 20 m, the absolute value of the difference Δn in the vehicle width direction between both lane positions at the position of 10 m is calculated.

  At this stage, the determination means 93 performs the following processing as the processing (1).

  In the process (1), the determination means 93 has an absolute value of the difference Δn between the current lane position LRpre and the previous lane position LRlast at at least one of the plurality of forward positions separated from the host vehicle by a predetermined distance. When the third threshold value shift_th3 set for the forward position is exceeded, it is determined that the entire current lane position LRpre detected and estimated this time is invalid.

  Specifically, for example, when the third threshold shift_th3 is set to, for example, 300 mm, 400 mm, and 1000 mm for the forward positions 10 m, 20 m, and 50 m away from the host vehicle, If even one of the absolute values of the difference Δn between the lane position LRpre and the previous lane position LRlast exceeds the third threshold shift_th3, the current lane position LRpre is determined to be invalid because the reliability is low. .

  For example, in the reference image T shown in FIG. 2, the snow on the road surface is stepped on the tire and the asphalt can be seen and the portion D that is imaged relatively dark and the snow remains on the road surface and is imaged relatively brightly. A boundary with the portion W may be erroneously detected as an edge portion of the lane. In such a case, for example, as shown in FIG. 16, the boundary part is detected as the right lane position LRpre for each sampling process, or the edge part of the actual lane is detected as the right lane position LRpre. . In such a case, in the process (1), the current lane position LRpre detected and estimated is determined to be invalid because the reliability is low.

  In the present embodiment, when the determination unit 93 determines that the current lane position LRpre is invalid, the determination unit 93 discards the current lane position LRpre, and the current position of the lane position LRlast calculated in the previous sampling cycle is more current. It is determined that the reliability representing the actual lane position is high, and the current position of the previous lane position LRlast is stored in the storage means as the current lane position LRpre and used for processing in the next sampling cycle. It has become.

  However, in the case where such discarding is continuously performed, there is a possibility that the lane position that has newly begun to appear is the correct lane position. Therefore, in the present embodiment, the determination means 93 determines that the current lane position LRpre is invalid and is discarded when the number of samplings exceeds the fifth predetermined number cnt_th5, that is, the fifth time this time is the previous time. The current position of the current lane position LRlast is discarded as having low reliability, the invalidated current lane position LRpre is stored as valid in the storage means, and is used for processing in the next sampling cycle. It has become. For example, 4 times is set as the fifth predetermined number of times cnt_th5.

  In the present embodiment, as can be seen from the way of estimating the lane position LRinf in the far region in the lane position estimating means 92 described above, particularly when the road shape is a large curve, it is particularly farthest from the own vehicle. Since the absolute value of the difference Δn between the current lane position LRpre and the previous lane position LRlast becomes large at the forward position, the current lane position LRpre is frequently discarded and it becomes practically difficult to detect the lane position. There is a case.

  Therefore, in the present embodiment, the determination unit 93 is configured to change each of the third threshold values shift_th3 according to the steering angle of the steering wheel input from the steering angle sensor B or the turning curvature calculated based on the steering angle. ing.

  On the other hand, the determination means 93 has a fourth sampling count at which the absolute value of the difference Δn is equal to or less than each third threshold value shift_th3 at all of a plurality of forward positions separated from the host vehicle by a predetermined distance, for example, 10 m, 20 m, and 50 m. Only when the predetermined number of times cnt_th4 is continued, the validity / invalidity of the current lane position LRpre described in the following processing (2) is determined. The fourth predetermined number of times cnt_th4 is set to 15 times, for example.

  As described above, the reason for determining whether the current lane position LRpre is valid or invalid only when the number of samplings continues for the fourth predetermined number of times cnt_th4 or more is that the third threshold value shift_th3 set to a relatively large value is stabilized. This is because the reliability can be accurately determined based on the current lane position LRpre that can be obtained for the first time in a clear state.

  As described above, the determination means 93 divides the vehicle into a plurality of sections at intervals of 10 m as the distance from the host vehicle, and calculates the absolute difference Δn in the vehicle width direction between the current lane position LRpre and the previous lane position LRlast for each section. When each value is calculated, the following process (2) is performed in parallel with the process (1).

  In the process (2), the judging means 93 compares the absolute value of the difference Δn calculated as shown in Table 1 below, for example, with each first threshold value shift_th1 set in advance for each section, and compares the difference Δn. If the absolute value of is less than or equal to the first threshold value shift_th1, the count number of the section is incremented as shown in Table 2 below.

  The first threshold shift_th1 is set to a smaller value than the third threshold shift_th3. Further, Table 1 shows the case where the first threshold shift_th1 is set to the same value in each section, but the first threshold shift_th1 can be changed for each section and is set as appropriate.

  The number of counts in Table 2 is incremented by 1 for each sampling process if the absolute value of the difference Δn in that section is less than or equal to the first threshold shift_th1. In addition, when the absolute value of the difference Δn exceeds a first threshold shift_th1 in a certain section, the count number of the section is immediately cleared to zero.

  The determining means 93 continuously determines the respective divisions in which the absolute value of the difference Δn between the current lane position LRpre and the previous lane position LRlast is equal to or less than each first threshold shift_th1 for the first predetermined number of times cnt_th1 set for the number of samplings. Each section of the current lane position LRpre whose count number in Table 2 is equal to or greater than the first predetermined number of times cnt_th1 is determined to be highly reliable and output. The first predetermined number of times cnt_th1 is set to 5 times, for example.

  Specifically, the determination means 93 advances the increment in each section that meets the above condition, and when the count number of each section is as shown in Table 3 below, the determination means 93 determines from the own vehicle in the process (1). If the number of samplings in which the absolute value of the difference Δn is equal to or less than each third threshold shift_th3 at a plurality of front positions separated by a distance continues for a fourth predetermined number of times cnt_th4 or more, the lane position LRpre is 10 m to 60 m. Each category is judged to be valid. In the case of Table 3 below, it is determined that each segment from 10 m to 70 m is effective for the left lane position LLpre.

  As described above, when the absolute value of the difference Δn exceeds the first threshold value shift_th1 in a certain section, the count number of the section is immediately cleared to 0, so that the count exceeds the fourth predetermined number of times cnt_th4. It doesn't make sense. Therefore, in the present embodiment, as shown in Table 3, when the count number of each section reaches the fourth predetermined number of times cnt_th4, no further addition is performed.

  On the other hand, even if the current lane positions LRpre and LLpre are determined to be valid in the process (2), the lane position detection means 91 does not detect lane candidate points in the current sampling process, and the lane The degree of reliability is not necessarily high with respect to the section composed only of the lane positions LRinf and LLinf estimated by the position estimating means 92. For this reason, it is required to determine the validity / invalidity more strictly for a section including only the estimated lane positions LRinf and LLinf.

  Various cases can be considered for the section composed only of the lane positions LRinf and LLinf estimated by the lane position estimating means 92.

  In a category in which there is no preceding vehicle in front of the host vehicle, and no lane candidate point is detected by the lane position detecting unit 91 according to the performance of the apparatus, and the lane positions LRinf and LLinf are estimated by the lane position estimating unit 92, Since the degree is low, it is easy to invalidate.

  On the other hand, for example, as shown in FIG. 17, when a preceding vehicle is present ahead of the host vehicle due to a lane change or the like, lane candidate points are detected by the lane position detecting means 91 in the sampling processing up to that point. The lane positions LRinf and LLinf may be estimated by the lane position estimating means 92 without detecting the lane candidate points in the current sampling process.

  In such a case, of the classification of the lane positions LRinf and LLinf estimated by the lane position estimation means 92 in the current sampling process, the lane position detection means 91 has detected the lane candidate points in the previous sampling process. If the absolute value of the difference Δn with respect to the previous lane position LRlast is small, it can be said that the reliability of the classification is high, so that the classification may be determined to be effective.

  Therefore, the processing (3) includes only the lane positions LRinf and LLinf estimated by the lane position estimation means 92 among the respective sections of the current lane positions LRpre and LLpre that are valid in the processing (2). If there is a classification and a lane candidate point has been detected by the lane position detection means 91 in the past, the classification up to that classification, that is, the classification and the current lane that is closer to the vehicle than the classification Each section of the position can be determined to be valid, and each section farther than the section can be determined to be invalid.

  However, in the present embodiment, in order to make a more precise determination, the determination means 93, as the process (3), out of the respective sections of the current lane positions LRpre and LLpre that are valid in the process (2). Second, the lane position is estimated by the lane position estimating means 92 and is a segment composed only of the lane positions LRinf and LLinf, and the lane position detecting means 91 sets the number of sampling times in which the lane candidate points are not continuously detected. If there is a division exceeding the predetermined number of times cnt_th2, it is judged that the division and each division far from the division are invalid, and each division of the current lane positions LRpre and LLpre closer to the own vehicle than the division is valid. It comes to judge that there is.

  Specifically, as shown in Table 4 below, the determination unit 93 counts the number of times the lane candidate points are not continuously detected by the lane position detection unit 91 for each section. The count is incremented by 1 if a lane candidate point is not detected by the lane position detecting means 91 in the sampling process, and is cleared to 0 when a lane candidate point is detected. The second predetermined number of times cnt_th2 is set to, for example, 10 times. As shown in Table 4, when the count number of each section reaches the second predetermined number of times cnt_th2, no further addition is performed.

  For example, in the example of Table 3, 30 m to 40 m, 40 m to 50 m, 50 m to 60 m among the respective sections from 10 m to 60 m of the current right lane position LRpre determined to be effective in the process (2). It is assumed that the lane candidate point is not detected by the lane position detecting unit 91 in the current sampling process, and the segment is composed only of the lane position LRinf estimated by the lane position estimating unit 92.

  In this case, of the two sections, the lane candidate points are not detected five times in succession after the lane candidate points are detected in the past. However, since the second predetermined number of times cnt_th2 has not been reached, it is determined in the process (3) that the division of 30 to 40 m is effective. On the other hand, since the lane candidate points have not been continuously detected 10 times or more after the lane candidate points have been detected in the past in the two categories of 40 m to 50 m and 50 m to 60 m, the second predetermined number of times cnt_th2 has been reached. In the process (3), it is determined that these two sections are invalid, and a section of 60m to 100m farther than that is also determined to be invalid.

  In the present embodiment, the determination unit 93 further performs the processing (2) and (3) as described above to invalidate the current lane positions LRpre and LLpre, that is, the current right lane position. In the example of LRpre, the absolute value of the difference Δn between the current lane position LRpre and the previous lane position LRlast is a stricter value than the first threshold shift_th1 as the process (4) for each section of 40m to 100m. That is, if there is a segment in which the number of samplings that is less than or equal to the second threshold shift_th2 set to a smaller value is equal to or greater than the third predetermined number of times cnt_th3, it is determined that the current lane positions LRpre and LLpre are valid for that segment. It is supposed to be.

  Specifically, as shown in Table 1, the second threshold shift_th2 is set for each section, for example, 150 mm smaller than the first threshold shift_th1, which is set to 250 mm, for example, and the third predetermined number of times cnt_th3 is set to 5 times, for example Is set.

  In this state, the determination means 93 compares the absolute value of the difference Δn with each second threshold value shift_th2 in the same manner as in Tables 1 and 2, and the absolute value of the difference Δn is less than or equal to the second threshold value shift_th2. If so, the count is incremented by 1 for each sampling process. In the example of the current right lane position LRpre, the count number is 5 times, that is, the third predetermined number of times cnt_th3 or more in the 40m to 50m section, for example, of the two sections 40m to 50m and 50m to 60m. For example, it is determined that the division of 40 to 50 m once invalidated in the processes (2) and (3) is valid again in the process (4).

  The determination means 93 performs the processes (1) to (4) described above, and outputs the current lane positions LRpre and LLpre determined to be valid.

  Specifically, for example, when the determination unit 93 determines that the entire lane position is invalid in the process (1) in the above example for the right lane position LRpre, the determination unit 93 does not output the current lane position LRpre. Further, when it is not determined that the entire lane position is invalid in the process (1), the sections from 10 m to 60 m are valid among the sections of the current right lane position LRpre in the process (2). In the process (3), it is determined that two sections of 40 m to 60 m are invalid, and in the process (4), it is determined that the section of 40 m to 50 m is valid again. Therefore, in the above example, it is finally determined that each section of 10 m to 50 m is valid for the right lane position LRpre and is output.

  As described above, according to the lane recognition device 1 according to the present embodiment, the lane candidate point is detected from the reference image T that is the captured image to detect the lane position, and is detected in the far region where the lane candidate point is not detected. The left and right lane positions of the host vehicle are detected by estimating the lane position based on the determined lane position. Then, the number of samplings in which the absolute value of the difference between the current lane position detected in the current sampling process and the lane position detected in the past is equal to or less than a threshold value is counted at a front position that is a predetermined distance away from the host vehicle. Then, when the count number continues for a predetermined number of times or more, it is determined that the current lane position is valid.

  In this way, not only the continuity between the lane position detected from the current reference image T and the lane position detected in the past, but also the continuity of the lane position detected in the past is based on the fact that the continuity is continuous for a predetermined number of times. By determining the reliability of the current lane position, it is possible to accurately indicate whether the calculated reliability of the current lane position is high or low, and it is possible to accurately indicate the effectiveness.

  Therefore, it is clear whether or not automatic control can be performed based on the detected current lane position, that is, whether or not the detected current lane position should be used for automatic control such as lane departure avoidance. It becomes possible to show.

  In addition, if the current lane position is divided into a plurality of sections and the reliability of each section is determined to determine whether it is valid or not, the reliability of the current lane position can be shown more accurately. It becomes possible to accurately indicate which part of the detected lane position has high reliability. In addition, even if there is a category with low reliability, it is judged that the category with high reliability is valid without invalidating the entire detected lane position. By performing automatic control such as this, it is possible to increase the control sustainability of automatic control.

  Furthermore, if a relatively gentle threshold is set for each of the current lane position and the lane position detected in the past at a plurality of forward positions that are a predetermined distance away from the host vehicle, By determining that the reliability of the current lane position detected by the sampling process is low and invalidating the entire current lane position, the reliability of the determination result of the determination unit can be improved.

  In addition, when the lane on the road surface is marked so as to mark the broken line inside the lane and make the width of the driving lane narrow, it is also possible to increase the threshold value shift_th. .

It is a block diagram which shows the structure of the lane recognition apparatus which concerns on this embodiment. It is a figure which shows an example of a reference | standard image. It is a flowchart which shows the procedure of the process performed by a lane position detection means. It is a figure explaining the search area | region limited to the circumference | surroundings of the lane detected last time. It is a flowchart which shows the procedure of a lane candidate point detection conversion process. It is a figure explaining (A) 1st starting point brightness | luminance threshold value pth1 and (B) 1st starting point edge intensity | strength threshold value Eth1. It is a figure explaining the new lane partially overlapped with the old lane. It is a figure explaining (A) 2nd starting point brightness | luminance threshold value pth2 and (B) 2nd starting point edge intensity | strength threshold value Eth2. It is a graph showing the brightness | luminance change at the time of imaging the snow on a road surface. It is a figure which shows the lane candidate point detected by the search on a horizontal line. It is the figure which represented the peak straight line on the reference | standard image. It is a figure explaining the conditions which record a lane candidate point as a lane position. It is a figure explaining the detected left and right lane position. It is a figure explaining the lane position of the estimated far field. It is a figure explaining the difference of the present lane position and the lane position calculated in the past. It is a figure explaining the lane position detected by dispersion | variation. It is a figure explaining the lane position estimated when a preceding vehicle exists.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lane recognition apparatus 2 Imaging means 6 Image processing means 91 Lane position detection means 92 Lane position estimation means 93 Determination means
cnt_th1 First predetermined number of times
cnt_th2 Second predetermined number of times
cnt_th3 Third predetermined number of times
cnt_th4 4th specified number of times
cnt_th5 5th predetermined number of times Δ Difference Lij Distance LRinf, LLinf Estimated lane position LRdec, LLdec Detected lane position LRpre, LLpre Current lane position LRlast, LLlast Previously calculated lane position pnij Luminance value
shift_th1 first threshold
shift_th2 second threshold
shift_th3 Third threshold T Reference image (one image)

Claims (10)

Imaging means for imaging the front of the host vehicle at a predetermined sampling period and outputting a pair of images having a luminance value for each pixel;
Image processing means for calculating a distance in real space for each pixel of at least one image based on the pair of captured images;
Lane position detection means for detecting a pixel corresponding to one edge portion of the lane as a lane candidate point based on the luminance value and the distance for the one image, and detecting a lane position based on the detected lane candidate point; ,
And lane position estimation unit to estimate a lane position in the far region where the lane position based on the detected lane marking location has not been detected by the lane location detection means,
The detected lane position and the estimated lane position are combined to calculate the current lane position for each imaging sampling by the imaging means , and at a forward position away from the host vehicle, the current lane position The current lane position is determined to be valid when the absolute value of the difference from the lane position calculated in the past continues for a predetermined number of times that is equal to or less than the threshold value set for the forward position. A lane recognition device, comprising: a determination unit that performs the determination.   The determination means divides the current lane position and the previously calculated lane position into a plurality of corresponding sections, and continuously determines the current lane position for a first predetermined number of times set for the number of samplings. When the absolute value of the difference from the lane position calculated in the past is equal to or less than each first threshold set for each section, it is determined that the section of the current lane position is valid, and the section The lane recognition device according to claim 1, wherein it is determined that a division other than that is invalid.   The determination means is a section composed only of the lane position estimated by the lane position estimation means among the sections of the current lane position determined to be valid, and the lane position detection means in the past. If there is a segment in which the lane candidate point is detected, it is determined that each segment far from the segment is invalid, and each segment of the current lane position that is closer to the vehicle than the segment and the segment is determined. The lane recognition device according to claim 2, wherein the lane recognition device is determined to be effective.   The determination means is a section consisting only of the lane position estimated by the lane position estimation means among the sections of the current lane position determined to be valid, and the lane position detection means If there is a division exceeding the second predetermined number of times when the number of sampling points where the lane candidate points were not detected continuously is determined, it is determined that the division and each division far from the division are invalid, and the division The lane recognition device according to claim 2, wherein each section of the current lane position closer to the host vehicle is determined to be valid.   The determination means is a value in which an absolute value of a difference between the current lane position and the previously calculated lane position is smaller than the first threshold among the sections of the current lane position determined to be invalid. When there is a segment in which the number of times of sampling that is equal to or less than the second threshold set in (3) is a third predetermined number of times or more, it is determined that the segment of the current lane position is valid. The lane recognition device according to any one of claims 2 to 4.   The determination means determines whether the absolute value of the difference between the current lane position and the previously calculated lane position is the first forward position for each of the plurality of front positions at all of the plurality of front positions separated from the host vehicle by a predetermined distance. 6. The valid / invalid determination is performed only when the number of samplings equal to or less than each third threshold set to a value greater than the threshold continues for a fourth predetermined number of times or more. The lane recognition device according to claim 1.   The determination means, when an absolute value of a difference between the current lane position and the lane position calculated in the past exceeds the third threshold at at least one of the plurality of front positions, The lane recognition device according to claim 6, wherein the entire current lane position is determined to be invalid.   When the determination means determines that the entire current lane position is invalid, the current lane position is discarded, and the current lane position in the next sampling period is discarded using the lane position that has been validated in the past. The lane recognition device according to claim 7, wherein comparison is performed.   In the case where the determination means performs the comparison using the lane positions that have been validated in the past, if the number of sampling times judged to be invalid exceeds a fifth predetermined number of times, the lane that has been validated in the past The position is discarded, the current lane position determined to be invalid this time is validated, and the current lane position is compared with the current lane position in the next sampling period using the current lane position. Lane recognition device.   The lane recognition device according to any one of claims 6 to 9, wherein the third threshold value is changed according to a steering angle or a turning curvature of the host vehicle.

Images