JP6132807B2 - Lane mark recognition device - Google Patents

Description

  The present invention relates to a lane mark recognition device for recognizing a lane mark provided on a road based on a captured image of a camera mounted on a vehicle.

  2. Description of the Related Art Conventionally, a lane mark recognition device that recognizes a lane mark provided on a road from an image of a road ahead of the vehicle captured by an in-vehicle camera is known.

  In this type of lane mark recognition apparatus, it is common to detect a lane mark by extracting a high-luminance image portion from an image. However, if the road surface is wet due to puddles, the image portion at the wet location has high luminance, and there is a possibility that the location is erroneously detected as a lane mark.

  In order to solve such a problem, a plurality of surrounding images taken at different positions are compared, and the scattered reflection light and the specular reflection light included in the surrounding image are identified based on a change between the images, and the specular reflection is performed. There has been proposed a photographing object recognition device including a removing unit that removes only light from the surrounding image and a recognizing unit that recognizes the photographing target that is scattered and reflected from the surrounding image from which specular reflection light is removed (see Patent Document 1). ).

  While the reflected light of the lane mark is scattered reflected light, the reflected light from rainwater or the like is specularly reflected light. According to the technique of Patent Document 1, the specularly reflected light reflected by rainwater or the like can be removed, so that the object to be photographed Thus, it is possible to reliably recognize only the scattered reflected light reflected with high brightness and accurately extract the object to be photographed.

Japanese Patent No. 4901275

  However, the technique of Patent Document 1 has room for improvement from the viewpoint of reducing calculation costs. For example, if the road is wet extensively due to rain, a large amount of calculation cost may be required for the process of removing the image portion of the specular reflection light.

  The present invention has been made in view of such a background, and an object of the present invention is to provide a lane mark recognition apparatus capable of appropriately reducing the calculation cost in view of road conditions in lane mark recognition.

  The present invention has been made to achieve the above object, and a lane mark recognition device for recognizing a lane mark provided on a road based on an image around the vehicle imaged by a camera mounted on the vehicle, The present invention relates to a vehicle and a lane mark recognition method.

And the lane mark recognition device of the present invention,
The ratio of the region where the corresponding real space position is an image portion that is more than a predetermined distance away from the camera and the luminance is equal to or higher than the predetermined luminance or the luminance change amount at the boundary is equal to or higher than the predetermined change amount is predetermined. A specific candidate exclusion unit that excludes from the image at least one of a first image portion that is greater than or equal to the ratio and a second image portion that is greater than or equal to a predetermined proportion of the specific color area;
A lane mark recognizing unit for recognizing a lane mark based on a region of the image in which the position in the real space corresponding to the image portion excluded by the specific candidate excluding unit is on the camera side. To do.

  When there is rain or the like, there is a high probability that the road surface is wet over a wide area. If the road surface is wet over a wide area, performing a lane mark recognition process or a wet road surface exclusion process on the entire image of the road surface will result in a calculation even though the lane mark recognition accuracy will not improve. Costs can be significant. In addition, when there is a pedestrian crossing or the like, there is a high probability that there is an intersection or the like where no lane mark exists in the distance. When the lane mark recognition process or the wet road surface exclusion process is performed on such an image portion such as an intersection, the calculation cost may increase even though the lane mark recognition accuracy is not improved.

  In particular, when a wet road surface, a pedestrian crossing, or the like exists far from the camera, the amount of information obtained is reduced in a region farther from them. For this reason, when the lane mark recognition process is performed in the area, the lane mark recognition process is performed based on a smaller amount of information on a wet road surface or an intersection with low lane mark recognition accuracy. As a result, although the lane mark recognition accuracy is not improved, the calculation cost may increase.

  According to the lane mark recognizing apparatus having the configuration configured by paying attention to these points, the corresponding real space position is an image portion separated by a predetermined distance or more from the camera, and the luminance is equal to or higher than the predetermined luminance. Or, the first image part in which the ratio of the area where the luminance change amount at the boundary is greater than or equal to the predetermined change quantity is greater than or equal to the predetermined ratio, and the second image part in which the ratio of the specific color area is greater than or equal to the predetermined ratio At least one of them is excluded. Thereby, an image portion of a road surface different from a normal road surface such as a road surface or a pedestrian crossing wetted by rainwater or the like is excluded. The lane mark is recognized based on the image portion before the excluded image portion. As a result, according to the lane mark recognition apparatus of the present invention, the calculation cost can be appropriately reduced in view of road conditions.

In the lane mark recognition apparatus of the present invention,
An edge image generation unit that generates an edge image obtained by extracting an edge point whose luminance change amount with respect to the surrounding portion is a first predetermined value or more from a road image captured by the camera;
An edge part creation unit that creates an edge part by combining the adjacent edge points;
The predetermined amount of change is set to a second predetermined value larger than the first predetermined value;
The specific candidate excluding unit is configured such that the number of the edge portions is equal to or greater than a predetermined number, and the ratio of the area where the luminance change amount at the boundary is equal to or greater than the second predetermined value is equal to or greater than the predetermined ratio. The first image portion satisfying at least one, and the number of the edge portions is a predetermined number or more, and the amount of change in luminance with respect to the peripheral portion is created by a plurality of edge points that are the second predetermined value or more. It is preferable that at least one of the second image portions satisfying at least one of including the edge portion is excluded.

  According to the lane mark recognition apparatus having the above configuration, the probability that the image portion to be excluded is not a lane mark increases. As a result, it is possible to improve both the recognition accuracy of the lane mark and the calculation cost.

In the lane mark recognition apparatus of the present invention,
If the specular reflection component is included in the first image part by a predetermined ratio or more, the specific candidate exclusion unit is farther from the camera than the position corresponding to the image part in the corresponding real space position. It is preferable to be configured so as to exclude the image portion that becomes.

  An image portion containing a lot of components of specular reflection light has a high probability of being an image portion such as a puddle or snow.

  According to the lane mark recognizing device having the configuration configured by paying attention to this point, an image portion far from an image portion having a high probability of being an image portion such as a puddle or snow is excluded. As a result, the calculation cost can be appropriately reduced in view of the road conditions.

  In the lane mark recognition apparatus of the present invention, the specific candidate excluding unit recognizes the image portion as an image portion of a pedestrian crossing when the first image portion includes a diffuse reflection light component in a predetermined ratio or more. It is preferable that it is comprised.

  An image portion containing a large amount of diffuse reflection light is highly likely to be an image portion such as a pedestrian crossing.

  According to the lane mark recognizing apparatus having the configuration configured by paying attention to this point, a pedestrian crossing is appropriately recognized. As a result, the calculation cost can be appropriately reduced in view of the road conditions.

  In the lane mark recognition apparatus having the above configuration, the lane mark recognition unit has a distance from the camera to a position in the real space corresponding to the image portion recognized as the image portion of the pedestrian crossing is equal to or less than the predetermined distance. In this case, the lane mark is recognized including the image portion whose position in the corresponding real space is farther from the camera than the image portion of the pedestrian crossing.

  As described above, when there is an intersection in the distance, there is little information to be obtained. Therefore, it is difficult to perform a highly reliable lane mark recognition process based on the intersection and its surrounding image portions. On the other hand, since there is a lot of information to be obtained at an intersection near the vehicle, a highly reliable lane mark recognition process can be performed based on the intersection and its surrounding image portions.

  According to the lane mark recognition apparatus having the above configuration, when the distance between the vehicle and the pedestrian crossing is equal to or less than a predetermined distance, the lane mark recognition process is performed including the image portion of the road surface ahead of the pedestrian crossing. As a result, when the distance between the vehicle and the pedestrian crossing is a predetermined distance or more, that is, when there is a high probability that highly reliable lane mark recognition processing cannot be performed, an image part such as an intersection far away from the pedestrian crossing However, if the distance between the vehicle and the pedestrian crossing is within a predetermined distance, that is, if there is a high probability that a reliable lane mark recognition process can be performed, an intersection far from the pedestrian crossing Etc. are not excluded. As a result, according to the lane mark recognition apparatus of the present invention, the calculation cost can be appropriately reduced in view of road conditions.

  In these lane mark recognition apparatuses, the specific candidate exclusion unit is configured to perform specular reflection on the first image portion based on a change in luminance of an image portion corresponding to the first image portion in a plurality of images captured in the past. It is preferable to be configured to determine whether the light or the diffuse reflected light is different.

  While the specular reflection light has a large change in luminance according to the observation angle of the object, the diffuse reflection light has a small change in luminance according to the observation angle of the object. According to the lane mark recognizing device having the configuration configured in view of this point, the observation angle of the object changes as the vehicle moves, and the luminance of the image portion of the object according to the change in the observation angle is increased. By determining based on the change, it is possible to accurately determine the difference between the specular reflection light and the diffuse reflection light.

  In the lane mark recognition apparatus of the present invention, the specific candidate excluding unit divides the image into image portions of a plurality of regions for each distance from the camera to a corresponding position in real space, and It is preferable that an image portion exclusion process is performed for each region.

  Puddles, pedestrian crossings, etc. have a high probability of spreading to some extent in a direction orthogonal to the traveling direction of the vehicle. According to the lane mark recognition device of the configuration, the image portion of the traveling direction of the vehicle is The image portion is excluded by being divided into a plurality of regions for each distance to the position in the real space corresponding to the image portion. As a result, puddles, pedestrian crossings, and the like that are spread to some extent in a direction orthogonal to the traveling direction of the vehicle can be collectively included in some of the plurality of regions. By performing the image part exclusion process for each divided area, the entire puddle or the entire pedestrian crossing is determined rather than determining whether each image part such as a puddle or pedestrian crossing within a predetermined range is individually excluded. It becomes easy to determine whether or not to be excluded collectively. Therefore, according to the lane mark recognition apparatus having the configuration, the calculation cost can be appropriately reduced in view of the road condition.

  In the lane mark recognition apparatus of the present invention, when a part of the image portion is excluded by the specific candidate exclusion unit, a process of changing the predetermined distance longer and a threshold value for starting control of the steering of the vehicle than usual. It is preferable to include a steering control unit that performs at least one of a process of setting a low value and a process of notifying the driver of the vehicle.

  According to the lane mark recognizing device having the configuration, even if the area targeted for the lane mark recognition process is narrowed by removing a part of the image portion by the specific candidate excluding unit, the predetermined distance is increased. Since at least one process of changing, setting a threshold value for avoidance control lower than usual, or notifying the driver of the vehicle is performed, the uncomfortable feeling given to the driver can be reduced or eliminated.

The vehicle of the present invention
The ratio of the region where the corresponding real space position is an image portion that is more than a predetermined distance away from the camera and the luminance is equal to or higher than the predetermined luminance or the luminance change amount at the boundary is equal to or higher than the predetermined change amount is predetermined. A specific candidate exclusion unit that excludes from the image at least one of a first image portion that is greater than or equal to the ratio and a second image portion that is greater than or equal to a predetermined proportion of the specific color area;
A lane mark recognizing unit for recognizing a lane mark based on a region of the image in which the position in the real space corresponding to the image portion excluded by the specific candidate excluding unit is on the camera side.

  According to the vehicle of the present invention, the same operational effects as those of the lane mark recognition apparatus of the present invention described above can be obtained.

The lane mark recognition method of the present invention is
The ratio of the region where the corresponding real space position is an image portion that is more than a predetermined distance away from the camera and the luminance is equal to or higher than the predetermined luminance or the luminance change amount at the boundary is equal to or higher than the predetermined change amount is predetermined. A specific candidate excluding step of excluding at least one of the first image portion that is equal to or higher than the ratio and the second image portion where the ratio of the specific color area is equal to or greater than a predetermined ratio;
A lane mark recognition step for recognizing a lane mark based on a region of the image in which the position in the real space corresponding to the image portion excluded by the specific candidate exclusion step is the camera side. To do.

  By applying the lane mark recognition method of the present invention to a vehicle and implementing it, the same operational effects as those of the lane mark recognition device of the present invention described above can be obtained.

The block diagram of a lane mark recognition apparatus. The flowchart of a lane mark recognition process. The flowchart of a lane mark recognition process. The figure explaining the division | segmentation of the captured image, edge image, and image part in case a pedestrian crossing is ahead, (a) is a figure explaining a captured image, (b) is a figure explaining an edge image, (c) is Explanatory drawing of the division | segmentation of an image part. FIG. 6 is a diagram illustrating division of a captured image, an edge image, and an image portion when a puddle is ahead, (a) is a diagram illustrating a captured image, (b) is a diagram illustrating an edge image, and (c) is an image. Explanatory drawing of a division | segmentation of a part. It is a figure explaining the relationship between recognition of a lane mark and steering control, (a) is a figure when the distance with a pedestrian crossing is more than predetermined, (b) is the case when the distance with a pedestrian crossing is less than predetermined. Figure.

  An embodiment of a lane mark recognition apparatus of the present invention will be described with reference to FIGS.

(Configuration of lane mark recognition device)
Referring to FIG. 1, a lane mark recognition device 10 is mounted on a vehicle 1 (corresponding to a vehicle of the present invention) including a camera 2 (color camera), a speaker 5, a display 6, and a steering mechanism 7. Yes.

  The lane mark recognition device 10 is an electronic unit that includes a CPU, a memory, various interface circuits, and the like (not shown), and a lane mark recognition and steering control program held in the memory is executed by the CPU, thereby taking a captured image. It functions as an acquisition unit 11, an edge image generation unit 12, a candidate image part extraction unit 13 (including the functions of an edge part creation unit and a specific candidate exclusion unit of the present invention), a lane mark recognition unit 14, and a steering control unit 15. . Further, the lane mark recognition apparatus 10 implements the lane mark recognition method of the present invention.

(Lane mark recognition process)
Hereinafter, processing for recognizing a lane mark provided on a road by the lane mark recognition apparatus 10 will be described with reference to the flowcharts shown in FIGS. 2 (a) and 2 (b). The lane mark recognizing device 10 executes processing according to the flowcharts shown in FIGS. 2A and 2B for each predetermined control cycle, and the lane mark (continuous line lane) of the road on which the vehicle 1 is traveling. Mark and broken line lane mark).

  FIG. 2A / STEP 1 is processing by the captured image acquisition unit 11. The captured image acquisition unit 11 inputs a video signal output from the camera 2 in front of the vehicle 1 (corresponding to the periphery of the vehicle of the present invention), and color components (R value, G value, B value) of the video signal. ) To obtain a color captured image 21 having R value, G value, and B value as data of each pixel. The acquired data of the captured image 21 is held in the image memory 20.

  Next, FIG. 2A / STEP2 is processing by the edge image generation unit 12.

  The edge image generation unit 12 performs a process of converting the color component of each pixel of the captured image 21 into luminance, and generates a grayscale image (multi-valued image). Then, the edge image generation unit 12 is a pixel whose luminance difference (luminance change amount) with respect to the edge point (the surrounding pixel (image portion)) is equal to or greater than a first predetermined value from the grayscale image. Edge image 22 (see FIG. 1) is generated by extracting positive edge points that change and negative edge points whose luminance changes from light to dark.

  Note that when the camera 2 is a monochrome camera, a grayscale captured image is obtained from the luminance of each pixel, and thus the above-described processing for generating a grayscale image from a color captured image is not necessary.

  Here, Im1 shown in FIG. 3A is an example of a grayscale image generated from the captured image 21, and an image of the left dashed lane mark that divides the lane in which the vehicle 1 (own vehicle) is traveling. The portions 51a to 52a and the image portions 61a and 62a of the solid line lane mark on the right side are included. Im1 also includes image portions 71a and 81a of crosswalk marks far away from the lane, image portions 53a to 54a of left dashed lane marks, and image portion 63a of right dashed lane marks far away from the lane. .

  Im2 illustrated in FIG. 3B is an edge image generated by performing edge extraction processing on the grayscale image Im1. In the edge image Im2, the left broken line lane mark image parts 51c to 54c, the right solid line lane mark image parts 61c to 63c, and the pedestrian crossing mark image parts 71c and 81c are linearly continuous edges. The image portion includes many points.

  Im4 shown in FIG. 4A is another example of a grayscale image generated from the captured image 21, and is an image of a solid line lane mark on the right side that divides the lane in which the vehicle 1 (own vehicle) is traveling. Portions 64a and 65a are included. Also, Im4 includes a puddle image portion 91a in front of the lane.

  Im5 shown in FIG. 4B is an edge image generated by performing edge extraction processing on the grayscale image Im4. In the edge image Im5, the right solid lane mark image portions 64c and 65c are image portions including many linearly continuous edge points. On the other hand, the image portion 91c of the puddle is an image portion including many curved edge points.

  Next, FIG. 2A / STEP 3 to FIG. 2A / STEP 17 are processes performed by the candidate image portion extraction unit 13. The processing steps STEP7 to STEP14 performed by the candidate image portion extraction unit 13 correspond to the specific candidate exclusion step in the lane mark recognition method of the present invention.

  The candidate image portion extraction unit 13 detects an edge pair that divides one area from the edge image 22 (FIG. 2A / STEP3). For example, when the edge image 22 is divided in the horizontal direction, adjacent positive edge points and negative edge points can be one edge pair.

  The candidate image portion extraction unit 13 then labels the edge image 22 based on edge pairs, labeling a group of edge points (hereinafter referred to as edge portions), and adjacent edge portions. Clustering is performed that associates image portions of the same object (FIG. 2 (a) / STEP4).

  The back projection conversion from the camera coordinates to the real space coordinates is performed on the image portions associated in FIG. 2A / STEP 4 to calculate the position of the object in the real space corresponding to each candidate image portion ( FIG. 2 (a) / STEP5). Note that an image portion whose size is equal to or smaller than a predetermined size (a size that is not assumed to be an image portion of a lane mark) may be excluded from the candidate image portions.

  The candidate image portion extraction unit 13 divides the road surface ahead of the vehicle 1 in the edge image Im2 into a plurality of regions (hereinafter referred to as divided regions) Z1 to Z5 having a horizontal width x and a vertical width y in the corresponding real space. (FIG. 2 (a) / STEP6). At this time, the candidate image portion extraction unit 13 sets the center C in the left-right direction of the vehicle 1 as the center in the horizontal direction of the divided area. Here, the lateral width x is a lateral width (a width slightly larger than the lane) necessary for recognizing the lane mark on the basis of the lateral width of the lane determined by the laws and regulations of each country. The vertical width y is a length that does not allow excessive noise to exclude a pedestrian crossing or a puddle.

  Im3 in FIG.3 (c) has shown the image when the road surface ahead of the vehicle 1 is divided | segmented into division area Z1-Z5 in edge image Im2.

  In addition, Im6 in FIG. 4C shows an image when the road surface in front of the vehicle 1 is divided into divided regions Z6 to Z9 in the edge image Im5.

  The candidate image part extraction unit 13 selects a divided area closer to the vehicle 1 from among the divided areas that are not subjected to the processing of FIG. 2A / STEP7 to FIG. 2A / STEP16. (FIG. 2 (a) / STEP7). Hereinafter, the selected divided region is referred to as “current divided region”.

  For example, among the divided areas Z1 to Z5 (see FIG. 3C), when none of the divided areas Z1 to Z5 is the processing target, the divided area Z1 closest to the vehicle 1 is selected. Also, among the divided areas Z1 to Z5 of the divided areas (see FIG. 3C), when the divided area Z1 has been processed and the other divided areas Z2 to Z5 have not been processed, Thus, the divided region Z2 closest to the vehicle 1 is selected.

  In addition, among the divided areas Z6 to Z9 (see FIG. 4C), when none of the divided areas Z6 to Z9 is the processing target, the divided area Z6 closest to the vehicle 1 is selected. Moreover, among the divided areas Z6 to Z9 (see FIG. 4C), when the divided area Z6 has been processed and the other divided areas Z7 to Z9 have not been processed, the vehicle 1 is in the unprocessed state. The closest divided area Z7 is selected.

  The candidate image portion extraction unit 13 determines whether or not the number of edge portions is a predetermined number for the current divided region (FIG. 2A / STEP 8). If the determination result is affirmative (FIG. 2 (a) / STEP8... YES), the process proceeds to FIG. 2 (a) / STEP10.

  If the determination result is negative (FIG. 2 (a) / STEP8... NO), the candidate image portion extraction unit 13 determines whether or not the luminance change amount of the edge portion included in the current divided region is equal to or greater than a second predetermined value. (FIG. 2 (a) / STEP9). The second predetermined value is a value larger than the first predetermined value in the edge point extraction of FIG. 2A / STEP2.

  If the determination result is affirmative (FIG. 2 (a) / STEP9... YES), the process proceeds to FIG. 2 (a) / STEP10. If the determination result is negative (FIG. 2 (a) / STEP9... YES), the process proceeds to FIG. 2 (a) / STEP16.

  The candidate image portion extraction unit 13 obtains the width of the high-intensity image portion based on the edge pair created in FIG. 2A / STEP 3 for the current divided region (FIG. 2A / STEP 10). More specifically, the candidate image portion extraction unit 13 scans the current divided region in the horizontal direction from a positive edge point (an edge point at which the luminance changes from dark to bright) to a negative edge point (the luminance becomes bright). To the edge point that changes darkly). By doing in this way, the ratio (area or breadth) which the high-intensity image part occupies in this division area is calculated | required.

  The candidate image part extraction unit 13 determines whether or not the total width of the obtained high-brightness image parts in the current divided region is equal to or greater than a predetermined width (FIG. 2A / STEP 11). The predetermined width is a value obtained by adding a predetermined threshold to the width of the lane mark defined by the laws and regulations of each country. When the total width of the high-brightness image portion is equal to or larger than the predetermined width, the current divided region is highly likely to be an image portion where a lane mark cannot be detected, such as a wet road surface, a puddle, or a pedestrian crossing. Instead of or in addition to this, the candidate image portion extraction unit 13 may determine whether the ratio of the obtained high-luminance image portion in the current divided region is equal to or higher than a predetermined ratio.

  When the determination result is affirmative (FIG. 2 (a) / STEP11... YES), the candidate image portion extraction unit 13 sets the luminance in the divided region to the luminance of the image portion corresponding to the divided region in the past plural frames. To determine whether or not the high-brightness image portion is due to specular reflection light (FIG. 2 (a) / STEP 12).

  More specifically, as the vehicle 1 moves, the observation angle and the observation distance from the camera 2 with respect to the high-luminance image portion change. Here, the reflected light is generally classified into specular reflected light whose luminance varies depending on the observation angle, or diffuse reflected light (scattered reflected light) having substantially the same luminance at any observation angle. In general, the brightness of the reflected light does not change depending on the observation distance.

  For this reason, when the luminance of the portion corresponding to the high-luminance image portion has changed in the past plural frames, the high-luminance image portion is estimated to be due to the specular reflection light component. On the other hand, when the luminance of the portion corresponding to the high-luminance image portion has not changed substantially in the past plural frames, the high-luminance image portion is estimated to be due to the diffuse reflected light component.

  If the determination in FIG. 2 (a) / STEP 12 is positive (FIG. 2 (a) / STEP 12... YES), the candidate image portion extraction unit 13 has a puddle, snow, or the like that is not a pedestrian crossing in the current divided region. Then, it is determined (FIG. 2A / STEP 13).

  When the determination of FIG. 2 (a) / STEP 12 is negative (FIG. 2 (a) / STEP 12... NO), the candidate image portion extraction unit 13 determines that a crosswalk exists in the current divided region (FIG. 2). (A) / STEP 14). This is because the paint used for road signs contains many components (solid color pigments) that emit diffusely reflected light.

  After the processing of FIG. 2 (a) / STEP 13 or FIG. 2 (a) / STEP 14, it is determined whether or not the position in the real space corresponding to the current divided area is more than a predetermined distance from the vehicle 1 (camera 2). Determination is made (FIG. 2A / STEP 15).

  If there are puddles, snow, etc. in the current divided area, there is a high possibility that puddles, snow, etc. are present in the image part farther from the vehicle 1 (camera 2) than the current divided area. If there is no information, it may be difficult to detect the lane mark. In addition, when there is a pedestrian crossing in the current divided area, there is a possibility that an intersection is farther from the vehicle 1 (camera 2) than the current divided area. In this case, there is no lane mark. there is a possibility.

  For this reason, when the current divided area is a long distance (FIG. 2 (a) / STEP15, if the determination result is affirmative, FIG. 2 (a) / STEP15... YES), it is far from the current divided area. It is difficult to obtain sufficient information for the image portion. For this reason, the candidate image portion extraction unit 13 does not perform the processing of FIG. 2A / STEP7 to FIG. 2A / STEP17 for the segmented region farther than the current segmented region, instead of FIG. The process proceeds to STEP18 and subsequent steps.

  When there are two pedestrian crossings in the divided area, the pedestrian crossing (first pedestrian crossing) 71c as viewed from the camera 2 indicates one end of the intersection, but the pedestrian crossing far away from the camera 2 (second crossing) The sidewalk 81c often indicates the other end of the intersection. In this case, it is unlikely that there is an intersection far from the pedestrian crossing (second pedestrian crossing) 81c as viewed from the camera 2. Therefore, even if the candidate image portion extraction unit 13 determines that the divided area is the image portion of the pedestrian crossing in FIG. 2A / STEP 14, if it is the second pedestrian crossing, FIG. a) The determination of STEP 15 may be omitted, and the processing of FIG. 2A / STEP 7 to FIG. 2A / STEP 17 may be performed on further distant regions.

  On the other hand, when the current divided area is a short-distance divided area (when the determination result in FIG. 2 (a) / STEP 15 is negative, FIG. 2 (a) / STEP 15... NO), However, sufficient information can be obtained for the far-division areas as viewed from the camera 2. For this reason, the candidate image portion extraction unit 13 performs the processing of FIG. 2A / STEP 7 to FIG. 2A / STEP 17.

  In addition, when the determination result of FIG. 2A / STEP 11 is negative (FIG. 2A / STEP 11... NO), the candidate image partial extraction unit 13 is included in the current divided region and is also shown in FIG. a) An image portion having line characteristics (for example, an image portion in which the ratio of linearly continuous edge points is equal to or higher than a predetermined level) among image portions extracted by clustering in / STEP 4 is an image portion of a lane mark. Is extracted as a candidate image portion that is a candidate for (FIG. 2A / STEP 16).

  Then, the candidate image portion extraction unit 13 determines whether or not there is a divided region that has not been subjected to the processing of FIG. 2A / STEP 7 to FIG. 2A / STEP 16 among the divided regions (FIG. 2 ( a) / STEP 17).

  If the determination result is affirmative (FIG. 2 (a) / STEP17... YES), the candidate image portion extraction unit 13 executes the processing of FIG. 2 (a) / STEP7 and subsequent steps. On the other hand, if the determination result is affirmative (FIG. 2 (a) / STEP17... NO), the process proceeds to FIG. 2 (b) / STEP18.

  Next, FIG. 2B / STEP 18 is processing by the lane mark recognition unit 14. The processing step by the lane mark recognition unit 14 corresponds to the lane mark recognition step in the lane mark recognition method of the present invention.

  The lane mark recognition unit 14 is set with the length in the corresponding real space obtained in FIG. 2 (a) / STEP 5 as a reference (the length of the broken lane mark (specified length) defined by the law). In this case, a candidate for a broken line lane mark is detected based on candidate image portions that are continuous in a certain direction.

  Further, the lane mark recognition unit 14 uses a known method described in, for example, Japanese Patent Application Laid-Open No. 11-219435 when the lane is divided by continuous line lane marks (white line, yellow line, etc.). Detect candidates for.

  The lane mark recognition unit 14 recognizes a lane mark based on these lane mark candidates.

  For example, as shown in FIG. 5A, left lane marks 51b to 52b are recognized on the left side of the lane in which the vehicle 1 is traveling, and right lane marks 61b and 62b are recognized on the right side.

  Here, when the distance d1 to the pedestrian crossing 71b is equal to or greater than the predetermined distance, the processing of FIG. 2A / STEP15 is YES, so that the lane mark recognition process is not performed for the distance from the pedestrian crossing 71b. .

  On the other hand, as shown in FIG. 5 (b), when the distance d2 to the pedestrian crossing is less than the predetermined distance, the processing of FIG. Mark recognition processing is performed. As a result, in addition to the left lane marks 51b to 52b on the left side of the lane in which the vehicle 1 is traveling and the right lane marks 61b and 62b on the right side, lane marks 53b, 54 and 63b far from the pedestrian crossing 71b can be recognized.

  In FIG. 2B / STEP 19, the lane mark recognition unit 14 recognizes the center position of the lane based on the positions of the left and right lane marks. In the example of FIG. 5A, the center position Xc1 between the inner position XL1 of the left lane marks 51b to 52b on the left side and the inner position XR1 of the right lane mark 61b on the right side is recognized. In the example of FIG. 5B, the lane mark recognizing unit 14 adds a far position in addition to the center position Xc1 between the inner position XL1 of the left lane marks 51b to 52b on the left side and the inner position XR1 of the right lane mark 61b on the right side. Based on the left lane marks 53b and 54b and the right lane mark 63b, a center position Xc2 between the inner position XL2 of the left lane marks 53b to 54b on the left side and the inner position XR2 of the right lane mark 63b on the right side is recognized.

  Next, FIG. 2B / STEP 20 to FIG. 2B / STEP 27 are processes by the steering control unit 15.

  The steering control unit 15 determines whether or not there is an unprocessed image portion in FIG. 2A / STEP 7 to STEP 16 (FIG. 2B / STEP 20). If the determination result is negative (FIG. 2 (b) / STEP20... NO), the steering control unit 15 proceeds to the process of FIG. 2 (b) / STEP27.

  On the other hand, if the determination result is affirmative (FIG. 2 (b) / STEP20... YES), the steering control unit 15 detects the front part recognized by FIG. 2 (a) / STEP7 to FIG. 2 (a) / STEP17. Based on the road surface, it is determined whether or not to expand the search range (FIG. 2B / STEP 21). If the determination result is affirmative (FIG. 2B / STEP 21... YES), the steering control unit performs a search range expansion process (FIG. 2B / STEP 22). For example, when there is a setting for expanding the search range in the traveling direction of the vehicle 1, when there is a pedestrian crossing in front of the vehicle 1 and the winker is in an operating state, the steering control unit 15 determines whether the steering control unit 15 in FIG. The predetermined distance is changed to another longer distance, and the image portion in the direction in which the blinker is operating is set to be added to the target of the next lane mark recognition process.

  When the determination result is negative (FIG. 2 (b) / STEP21... NO) or after the processing of FIG. 2 (b) / STEP22, in the following FIG. 2 (b) / STEP23, the steering control unit 15 Based on the road surface ahead recognized by FIG. 2A / STEP 7 to FIG. 2A / STEP 17, it is determined whether or not the threshold value for avoidance control is to be lowered. If the determination result is affirmative (FIG. 2 (b) / STEP 23... YES), the steering control unit 15 performs a process of lowering the avoidance control threshold (FIG. 2 (b) / STEP 24). For example, when there is a puddle in front of the vehicle 1 and the setting is made to lower the avoidance control threshold when there is a puddle, the steering control unit 15 lowers the avoidance control threshold and performs early steering control. Accordingly, the steering control unit 15 can perform steering control such as, for example, regarding a puddle as an obstacle and avoiding the puddle within a range not departing from the lane.

  Further, when there is a pedestrian crossing in front of the vehicle 1, the steering control unit 15 may lower the threshold value for avoidance control.

  When the determination result of FIG. 2 (b) / STEP23 is negative (FIG. 2 (b) / STEP23... NO) or after the processing of FIG. 2 (b) / STEP24, the steering control unit 15 It is determined whether or not to notify the driver based on the road surface in front recognized by a) / STEP 7 to FIG. 2 (a) / STEP 17 (FIG. 2 (b) / STEP 25). If the determination result is affirmative (FIG. 2 (b) / STEP 25... YES), the steering control unit 15 indicates that there is a puddle, snow, or a pedestrian crossing in front of at least one of the speaker 5 and the display 6. The driver is notified via (FIG. 2 (b) / STEP 26).

  When the determination result of FIG. 2B / STEP 25 is negative (FIG. 2B / STEP 25... NO), or after the processing of FIG. 2B / STEP 26, the steering control unit 15 b) Proceed to step 27.

  The steering control unit 15 activates the steering mechanism 7 to assist the driving operation of the driver of the vehicle 1 so that the vehicle 1 travels in the vicinity of the center position Xc in the lane (FIG. 2 (b) / STEP 27). In addition, the steering control unit 15 alerts the driver of the vehicle 1 with at least one of the speaker 5 and the display 6 as necessary.

  Here, the steering control unit 15 increases the deviation between the vehicle 1 and the center position Xc, and the interval WR between the vehicle 1 and the right lane mark 61b or the interval WL between the vehicle 1 and the left lane marks 51b to 54b is increased. When the value is equal to or less than a predetermined determination value, the steering mechanism 7 is operated to control the vehicle 1 closer to the center position Xc.

(Operation and effect of this embodiment)
When there is rain or the like, there is a high probability that the road surface is wet over a wide area. When the road surface is wet over a wide area, the lane mark recognition accuracy is improved by performing the lane mark recognition process or the wet road surface exclusion process on the image portions Z1 to Z5 or Z6 to Z9 of the road surface. In spite of not doing so, there is a possibility that the calculation cost becomes large. In addition, when there is a pedestrian crossing or the like, there is a high probability that there is an intersection or the like where no lane mark exists in the distance. If the lane mark recognition process or the wet road surface exclusion process is performed on the image portion Z3 such as an intersection, the calculation cost may increase even though the lane mark recognition accuracy is not improved.

  In particular, if a wet road surface, a pedestrian crossing, or the like exists far away from the camera, the amount of information obtained is reduced in regions Z3 to Z5 and Z8 to Z9 farther from them. Therefore, when the lane mark recognition process is performed in the area, the lane mark recognition process is performed based on a smaller amount of information in the image portions Z3 to Z5 and Z8 to Z9 such as wet road surfaces or intersections with low lane mark recognition accuracy. Will be done. As a result, although the lane mark recognition accuracy is not improved, the calculation cost may increase.

  According to the lane mark recognizing device 10 having the configuration configured focusing on these points, the corresponding real space positions are image portions Z2 to Z5 and Z7 to Z9 that are separated by a predetermined distance or more, By removing at least one of the image portions Z2 and Z7 having a luminance equal to or higher than a predetermined luminance and the image portion Z2 having a white hue equal to or larger than a predetermined width, a normal road surface such as a road surface or a pedestrian crossing wetted by rainwater or the like The image portions Z2 and Z7 on the road surface different from the above are excluded. The lane marks 51b, 52b, 61b, and 62b are recognized based on the image portions Z1 and Z6 before the excluded image portion. As a result, according to the lane mark recognition apparatus 10 of the present invention, the calculation cost can be appropriately reduced in view of road conditions.

  Further, the image portion Z7 containing a large amount of specular reflection light has a high probability of being an image portion such as a puddle or snow.

  According to the lane mark recognizing device 10 having the configuration configured by paying attention to this point, image portions Z8 to Z9 far from the image portion Z7 having a high probability of being an image portion such as a puddle or snow are excluded. As a result, the calculation cost can be appropriately reduced in view of the road conditions.

  The image portion Z2 containing a large amount of diffusely reflected light is highly likely to be an image portion such as a pedestrian crossing.

  According to the lane mark recognizing device 10 having the configuration configured focusing on this point, the pedestrian crossing 71b is appropriately recognized. As a result, the calculation cost can be appropriately reduced in view of the road conditions.

  As described above, when there is an intersection in the distance, there is little information to be obtained, so it is difficult to perform a highly reliable lane mark recognition process based on the intersection and the surrounding image portions Z3 to Z5. On the other hand, since there is a lot of information to be obtained at an intersection near the vehicle, a highly reliable lane mark recognition process can be performed based on the intersection and its surrounding image portions.

  According to the lane mark recognition apparatus having the configuration, when the distance d2 between the vehicle 1 and the pedestrian crossing is equal to or less than a predetermined distance, the lane mark recognition process is performed including the image portion of the road surface far from the pedestrian crossing 71b. . Thereby, when the distance d1 between the vehicle 1 and the pedestrian crossing is equal to or greater than a predetermined distance, that is, when there is a high probability that highly reliable lane mark recognition processing cannot be performed, an intersection or the like located farther from the pedestrian crossing 71b When the distance d2 between the vehicle 1 and the pedestrian crossing is within a predetermined distance, that is, when there is a high probability that highly reliable lane mark recognition processing can be performed, Image portions such as intersections that are far away are not excluded. As a result, according to the lane mark recognition apparatus 10 of the present invention, the calculation cost can be appropriately reduced in view of road conditions.

  While the specular reflection light has a large change in luminance according to the observation angle of the object, the diffuse reflection light has a small change in luminance according to the observation angle of the object. According to the lane mark recognition device 10 having the configuration configured in view of this point, the observation angle of the object changes as the vehicle 1 moves, and the image portion Z2 of the object according to the change in the observation angle. By determining based on the change in the brightness of Z7, it is possible to accurately determine whether the light is specularly reflected or diffusely reflected.

  Puddles, pedestrian crossings, and the like are likely to be spread to some extent in a direction orthogonal to the traveling direction of the vehicle 1. According to the lane mark recognition device 10 having the configuration, an image portion in the traveling direction of the vehicle 1 is displayed. For each distance y from the vehicle 1 to the position in the real space corresponding to the image portion, the image portion is divided into a plurality of regions Z1 to Z5 and Z6 to Z9, and the image portion is excluded. As a result, puddles, pedestrian crossings, and the like that are spread to some extent in the direction orthogonal to the traveling direction of the vehicle 1 are grouped into some areas Z2 and Z7 of the plurality of areas Z1 to Z5 and Z6 to Z9. May be included. By performing the image part exclusion process for each divided area, the entire puddle or the entire pedestrian crossing is determined rather than determining whether each image part such as a puddle or pedestrian crossing within a predetermined range is individually excluded. It becomes easy to determine whether or not to be excluded collectively. Therefore, according to the lane mark recognition apparatus 10 having the configuration, the calculation cost can be appropriately reduced in view of the road condition.

  According to the lane mark recognition apparatus having the configuration, the specific candidate exclusion unit (candidate image portion extraction unit 13) excludes some of the image portions Z2 and Z7 and the subsequent processing target area of the lane mark recognition process is narrow. Even when the vehicle has become, at least one process of changing the predetermined distance longer, setting a threshold value for avoidance control lower than usual, or notifying the driver of the vehicle 1 is performed. The discomfort given to the person can be reduced or eliminated.

(Modification)
The processing in FIG. 2A / STEP 12 to FIG. 2A / STEP 14 may be omitted. The processes of FIG. 2B / STEP 20 to FIG. 2B / STEP 26 may be omitted.

  In addition to or instead of the processing of FIG. 2A / STEP 11 to FIG. 2B / STEP 12, whether or not the ratio of the area where the current divided area has a specific color (for example, white) is equal to or greater than a predetermined ratio May be determined. In this case, for example, when the determination result is affirmative, it is recognized that the current divided region includes a pedestrian crossing, while when the determination result is negative, lane mark recognition is performed. The processing may be performed.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle (own vehicle), 2 ... Camera, 7 ... Steering mechanism, 10 ... Lane mark recognition apparatus, 11 ... Captured image acquisition part, 12 ... Edge image generation part, 13 ... Candidate image part extraction part, 14 ... Lane mark Recognizing unit, 15 ... steering control unit, 20 ... image memory, 21 ... captured image, 22 ... edge image.

Claims (10)

A lane mark recognition device for recognizing a lane mark provided on a road based on an image around the vehicle imaged by a camera mounted on the vehicle,
The ratio of the region where the corresponding real space position is an image portion that is more than a predetermined distance away from the camera and the luminance is equal to or higher than the predetermined luminance or the luminance change amount at the boundary is equal to or higher than the predetermined change amount is predetermined. A specific candidate exclusion unit that excludes from the image at least one of a first image portion that is greater than or equal to the ratio and a second image portion that is greater than or equal to a predetermined proportion of the specific color area;
A lane mark recognizing unit for recognizing a lane mark based on a region of the image in which the position in the real space corresponding to the image portion excluded by the specific candidate excluding unit is on the camera side. Lane mark recognition device. The lane mark recognition apparatus according to claim 1,
An edge image generation unit that generates an edge image obtained by extracting an edge point whose luminance change amount with respect to the surrounding portion is a first predetermined value or more from a road image captured by the camera;
An edge part creation unit that creates an edge part by combining the adjacent edge points;
The predetermined amount of change is set to a second predetermined value larger than the first predetermined value;
The specific candidate excluding unit is configured such that the number of the edge portions is equal to or greater than a predetermined number, and the ratio of the area where the luminance change amount at the boundary is equal to or greater than the second predetermined value is equal to or greater than the predetermined ratio. The first image portion satisfying at least one, and the number of the edge portions is a predetermined number or more, and the amount of change in luminance with respect to the peripheral portion is created by a plurality of edge points that are the second predetermined value or more. A lane mark recognition apparatus configured to exclude at least one of the second image portions satisfying at least one of including the edge portion. In the lane mark recognition apparatus according to claim 1 or 2,
If the specular reflection component is included in the first image part by a predetermined ratio or more, the specific candidate exclusion unit is farther from the camera than the position corresponding to the image part in the corresponding real space position. A lane mark recognizing device configured to exclude an image portion to be In the lane mark recognition apparatus according to any one of claims 1 to 3,
The specific candidate excluding unit is configured to recognize the image portion as an image portion of a pedestrian crossing when the first image portion includes a component of diffuse reflection light at a predetermined ratio or more. Lane mark recognition device. In the lane mark recognition apparatus according to claim 4,
When the distance from the camera to the position in the real space corresponding to the image part recognized as the image part of the pedestrian crossing is equal to or less than the predetermined distance, the lane mark recognition unit starts from the image part of the pedestrian crossing. The lane mark recognition apparatus is configured to recognize a lane mark including an image portion whose corresponding position in real space is far from the camera. In the lane mark recognition apparatus according to any one of claims 3 to 5,
The specific candidate excluding unit distinguishes between specular reflection light and diffuse reflection light based on a change in luminance of an image portion corresponding to the first image portion in a plurality of images captured in the past for the first image portion. A lane mark recognizing device configured to make a determination. In the lane mark recognition apparatus according to any one of claims 1 to 6,
The specific candidate exclusion unit divides the image into image portions of a plurality of regions for each distance from the camera to a corresponding position in real space, and performs image portion exclusion processing for each of the divided regions. A lane mark recognition apparatus configured to perform In the lane mark recognition apparatus according to any one of claims 1 to 7,
When a part of the image portion is excluded by the specific candidate excluding unit, a process of changing the predetermined distance longer, a process of setting a threshold value for starting control of steering of the vehicle lower than usual, A lane mark recognizing device comprising a steering control unit that performs at least one of a process of notifying a driver. A vehicle for recognizing a lane mark provided on a road based on an image around the vehicle imaged by a camera mounted on the vehicle,
The ratio of the region where the corresponding real space position is an image portion that is more than a predetermined distance away from the camera and the luminance is equal to or higher than the predetermined luminance or the luminance change amount at the boundary is equal to or higher than the predetermined change amount is predetermined. A specific candidate exclusion unit that excludes from the image at least one of a first image portion that is greater than or equal to the ratio and a second image portion that is greater than or equal to a predetermined proportion of the specific color area;
A lane mark recognizing unit for recognizing a lane mark based on a region of the image in which the position in the real space corresponding to the image portion excluded by the specific candidate excluding unit is on the camera side. Vehicle. A lane mark recognition method for recognizing a lane mark provided on a road based on an image around the vehicle imaged by a camera mounted on a vehicle,
The ratio of the region where the corresponding real space position is an image portion that is more than a predetermined distance away from the camera and the luminance is equal to or higher than the predetermined luminance or the luminance change amount at the boundary is equal to or higher than the predetermined change amount is predetermined. A specific candidate excluding step of excluding at least one of the first image portion that is equal to or higher than the ratio and the second image portion where the ratio of the specific color area is equal to or greater than a predetermined ratio;
A lane mark recognition step for recognizing a lane mark based on a region of the image in which the position in the real space corresponding to the image portion excluded by the specific candidate exclusion step is the camera side. Lane mark recognition method.