JP4735530B2 - Road marking line recognition device - Google Patents

Description

  The present invention relates to a road lane marking recognition device that recognizes road lane markings by analyzing a captured image of an imaging means that images the periphery of a vehicle.

  In recent years, a control system that performs automatic steering control so as to travel while maintaining a traveling lane is known under a name such as LKA (Lane Keeping Assist) (for example, see Non-Patent Document 1). An important point in such a control system is to accurately recognize road lane markings (lane markers) that divide the traveling lane and grasp the positional relationship between the traveling lane and the host vehicle.

In relation to this, an invention has been disclosed concerning an apparatus that detects a white line drawn on a road by capturing an image of the front of the vehicle with a video camera or the like, and performing edge extraction processing or binarization processing on the captured image. (For example, refer to Patent Document 1). In this apparatus, the width of the search range in the captured image (corresponding to the road width direction) is changed based on the result of the edge extraction process and the steering angle from the viewpoint of improving the processing speed. For example, the search range is narrowed at a position on the image corresponding to a position far from the vehicle, and the search range is widened when the steering angle is large.
JP-A-35198 Toyota Motor Corporation, "Crown Majesta New Model Car Description (Part No. 7109100)", Toyota Motor Corporation Service Department, July 5, 2004, Chapter 10 Body & Electrical, p10-287-10-306

  However, the road lane marking includes a straight road lane marking represented by a solid white line and a broken line, and a dotted line lane marking such as Botts Dots and Cat's Eye. Here, the botsdots are ceramic discs having a diameter of about 10 cm, which are mainly used in North America and are embedded in the road at intervals. The cat's eye is a reflector embedded in the road at an interval, and has a characteristic of reflecting incident light in the same direction.

  The apparatus described in Patent Document 1 takes into consideration changing the search range based on the circumstances peculiar to such a dotted line of road lane markings, particularly changing the size of the search range in the depth direction. Not done. Therefore, there may be a case where the position of the road marking line cannot be accurately recognized on the road partitioned by the point-string road marking line.

  The present invention is for solving such problems, and a main object of the present invention is to provide a road lane marking recognition apparatus that can more accurately recognize the position of a road lane marking.

  One aspect of the present invention for achieving the above object is a road lane marking recognition device that includes imaging means for imaging the periphery of a vehicle and recognizes a road lane marking by analyzing a captured image of the imaging means. When recognizing a reflective target with a reflective material and recognizing a road marking, compared to a non-reflecting target recognizing a non-reflective target that recognizes a non-reflective target different from the reflective target Then, the image analysis area in the captured image of the imaging means is expanded in the depth direction. Here, the “target” is a concept that does not include a solid line and a broken line drawn with a white line or a yellow line.

  According to this aspect of the present invention, when recognizing a reflective target that is recognizable on an image even at a relatively long distance and recognizing a road marking line, the target different from the reflective target is recognized. Since the image analysis area in the captured image of the imaging means is expanded in the depth direction, compared to when recognizing a road marking line, a virtual straight line (or curve) connecting the targets can be generated more accurately. As a result, the road lane marking can be recognized more accurately.

  Note that such enlargement control of the image analysis region may be performed only on a road on which both a reflective target and a non-reflective target are laid.

  In one aspect of the present invention, at the time of reflecting target recognition, it is desirable to expand the image analysis region in the picked-up image of the image pickup means to such an extent that a plurality of reflecting targets laid along the road are included.

  Further, in one aspect of the present invention, a reflective target detection means for detecting that the reflective target is laid on the road is provided, and the reflective target detection means detects that the reflective target is laid on the road. In such a case, it is desirable to recognize a road marking line by recognizing a reflective target.

  Further, in one aspect of the present invention, the projector includes a light projecting unit that projects light toward the front of the vehicle, and when the target detecting unit detects that the reflective target is laid on the road, the light projecting unit It is desirable to project light. In this case, it is more desirable to control the light projecting means so that light is projected along the traveling lane divided by the road marking line.

  In one embodiment of the present invention, vehicle speed detection means for detecting a vehicle speed may be provided, and an image analysis region in a captured image of the imaging means may be enlarged in accordance with an increase in the vehicle speed detected by the vehicle speed detection means.

  ADVANTAGE OF THE INVENTION According to this invention, the road lane marking recognition apparatus which can recognize the position of a road lane marking more correctly can be provided.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the accompanying drawings.

[First embodiment]
Hereinafter, the road lane marking recognition apparatus 1 according to the first embodiment of the present invention will be described. FIG. 1 is a diagram illustrating an example of the overall configuration of a road lane marking recognition device 1. FIG. 2 is a diagram schematically showing the arrangement positions of the respective components and the operating status of the present apparatus.

  The road lane marking recognition device 1 has, as main components, a near infrared camera 10, a light ECU (Electronic Control Unit) 20, a near infrared projector 25, a vehicle speed sensor 30, an LKA ECU 40, a main And a switch 50. Further, the steering device 60 is illustrated as one that uses the output of the present device. In addition, the arrow in a figure shows the flow of the main information communication in this apparatus via a multiple communication line etc. The communication is performed using an appropriate communication protocol such as CAN (Controller Area Network), BEAN, AVC-LAN, or FlexRay.

  The near-infrared camera 10 is, for example, a camera that is disposed in the upper center of the windshield and uses an image sensor such as a CCD or CMOS whose sensitivity distribution is extended to the near-infrared. An infrared filter that blocks visible light is used as necessary. The near-infrared camera 10 has an optical axis that faces obliquely downward in front of the vehicle, and images a road in front of the vehicle. The captured image of the near-infrared camera 10 is transmitted to the LKA ECU 40 as an image signal.

  The light ECU 20 is a computer unit in which, for example, a CPU, a ROM, a RAM, and the like are connected to each other via a bus. / O port, timer, counter etc. are provided. The ROM stores programs and data executed by the CPU. The light ECU 20 centrally controls the irradiation of headlights, tail lamps, and the like, and transmits the irradiation state of the headlights to the LKA ECU 40.

  Further, a near infrared projector 25 is connected to the light ECU 20. The near-infrared projector 25 projects near-infrared light having a wavelength that can be imaged by the near-infrared camera 10 in a direction substantially coincident with the optical axis of the near-infrared camera 10. The near-infrared projector 25 is disposed, for example, inside the headlight ASSY. Thus, by setting it as the structure which projects near infrared rays, it can suppress that a pedestrian, the driver of an oncoming vehicle, etc. are dazzled by irradiation of unnecessary visible light.

  The vehicle speed sensor 30 includes, for example, a wheel speed sensor attached to each wheel and a skid control computer. The skid control computer converts a wheel speed pulse signal output from the wheel speed sensor into a vehicle speed rectangular wave pulse signal (vehicle speed signal). To the LKA ECU 40.

  The LKA ECU 40 is, for example, a computer unit having the same hardware configuration as the light ECU 20. The LKA ECU 40 is activated, for example, by a user operation on the main switch 50 disposed beside the steering, and transmits a steering signal to the steering device 60 based on the captured image analysis of the near-infrared camera 10. Specific contents of the steering signal transmission will be described later.

  The LKA ECU 40 includes an image recognition unit 42 and a steering signal generation unit 44 as main functional blocks that function when the CPU stores (executes) a program stored in the ROM and executes it. .

  The image recognition unit 42 analyzes the image signal transmitted from the near-infrared camera 10 and recognizes the positional relationship between the road lane marking and the host vehicle (recognizes the road lane marking). Recognition processing by image analysis is performed based on different analysis regions and methods depending on the type of road marking line. In the present embodiment, the road is partitioned by a road lane marking in which the road is partitioned by a white line, a yellow solid line, a broken line, or the like (hereinafter simply referred to as “white line etc.”), and a bots dot or a cat's eye other than the white line. It is the structure which can respond to the road lane marking of an aspect.

[1. When roads are marked with white lines]
First, the image recognition unit 42 sets an analysis region in the lower region in the captured image of the near-infrared camera 10. FIG. 3 is a diagram illustrating an analysis region set on a captured image of the near-infrared camera 10 when recognizing a white line or the like. The lower end portion of the analysis area corresponds to the front of about several [m] to several tens [m] of the own vehicle, and the upper end corresponds to the front of about several tens [m] of the own vehicle. The former is set to a position on the image corresponding to the point where the straight line connecting the position where the near-infrared camera 10 is disposed and the vehicle nose touches the road, and the latter is set so that the road marking line can be accurately recognized. It is set at a position with sufficient depth. Further, the reason for narrowing the left and right end portions toward the center from the end portion of the image is to reduce the influence of noise elements that are unnecessary in image recognition, such as pedestrians and buildings.

  When an analysis area is set, a point whose luminance change in the horizontal direction of the image in the analysis area is greater than or equal to a threshold value is extracted as a feature point, and among the feature points arranged in a straight line or a curved line using a straight line extraction method such as Hough transform Those whose length is greater than or equal to a predetermined value are recognized as road marking lines. Then, through the conversion process from the image coordinate system to the real coordinate system (actual coordinate system viewed from the sky), and in the case of curved roads, the curvature estimation process, etc. (For example, expressed by an inclination angle and an offset). Here, the inclination angle is a deviation angle between the traveling direction of the traveling lane and the traveling direction of the host vehicle, and the offset is a deviation of the center of the host vehicle from the traveling lane center line (see FIG. 4). .

  Although detailed explanation is omitted because it is not the core part of the present invention, in order to improve the speed of image analysis, the feature points are not extracted by scanning all over the analysis region, but the image is scanned at an appropriate interval. It is realistic to set a plurality of directional scanning lines and extract feature points on the scanning lines. In order to more accurately recognize road marking lines on curved roads, it is realistic to divide the analysis area into an upper stage and a lower stage and perform a process of combining them after performing image analysis for each.

[2. When the road is partitioned by botsdots or catseye]
In this case, the image recognition unit 42 performs image analysis in an analysis region smaller than the case of [1] in principle, and recognizes road lane markings. FIG. 5 is a diagram illustrating an analysis region set on the captured image of the near-infrared camera 10 in this case. The lower end of the analysis region is set to a position on the image corresponding to a point where a straight line connecting the position where the near infrared camera 10 is disposed and the vehicle nose is in contact with the road, as in [1] above. On the other hand, the upper end is set to a position on the image corresponding to the vicinity of the limit distance that can be recognized by the cat's eye that is not illuminated by a botsdot or a headlight. Compared to FIG. 3 and FIG. 5, the front side is set. This is because a cat's eye that is not irradiated with botsdots or light has a lower contrast with a road surface (such as an asphalt surface) than a white line or the like, and it becomes difficult to discriminate it on an image as the distance increases. As described above, the processing speed can be improved and the processing load can be reduced by not making the region on the image that cannot be discriminated an analysis target.

  When the analysis region is set, processing such as pattern matching processing and morphological operation is performed in the analysis region to recognize botsdots and cat's eyes. Then, among the image elements recognized as botsdots, a virtual straight line (or a curved line) connecting image elements arranged in a straight line or curved line extracted through Hough transform or voting processing is recognized as a road lane marking. To do. The subsequent recognition of the positional relationship between the travel lane and the host vehicle is the same as in the case of [1].

  However, if the analysis area is set narrower than [1], the recognition accuracy of the road lane marking will be lowered. The degree of reduction in the recognition accuracy is particularly large in the curvature estimation process for curved roads. Therefore, in this embodiment, when recognizing the road lane marking by the cat's eye, the analysis area is enlarged, and if necessary, the near-infrared projector 25 performs control to project light ahead of the vehicle. did. This is because the cat's eye that reflects the light emitted from the near-infrared projector 25 and has high brightness on the image can be sufficiently recognized even when the distance from the vehicle is long.

  FIG. 6 is a flowchart showing a flow of characteristic processing repeatedly executed by the image recognition unit 42. This flow is started when a white line or the like is lost during recognition (specifically, feature points longer than a predetermined length are no longer extracted).

  First, an analysis region is set as shown in FIG. 5 (S100).

  Then, it is determined whether or not the cat's eye is laid on the road (S110). This determination is performed based on, for example, a result of performing processing such as pattern matching processing and morphological calculation on the captured image of the near-infrared camera 10. Alternatively, the presence of the cat's eye may be detected by a radar device that searches forward of the vehicle or a communication device that communicates with the road-side infrastructure.

  When it is determined that the cat's eye is not laid on the road, the road marking line is recognized by recognizing the bots dot in the analysis region set in S100 (S150).

  On the other hand, when it is determined that the cat's eye is laid on the road, it is determined whether or not a sufficient amount of light is applied to recognize the cat's eye (S120). Specifically, referring to the irradiation state of the headlight transmitted from the light ECU 20, when the headlight is irradiated with a low beam, it is estimated that a sufficient amount of light has been irradiated to recognize the cat's eye. be able to. Alternatively, the determination may be made based on the luminance distribution in the captured image of the near-infrared camera 10.

  If it is determined that a sufficient amount of light for recognizing the cat's eye is not irradiated, the near-infrared projector 25 projects light forward of the vehicle (S130).

  Then, the analysis area is expanded (S140), and the road marking line is recognized by recognizing the cat's eye in the expanded analysis area (S150). It should be noted that a bottling dot may be further recognized in the expanded analysis region (especially the region on the near side), and a road lane marking may be recognized in consideration of this.

  Here, it is desirable to expand the analysis area to such an extent that a plurality of cat's eyes are included (see FIG. 7). By doing so, the virtual straight line (or curve) generated by connecting the recognized positions of the cat's eyes can be made more accurate.

  As described above, according to the road lane marking recognition apparatus 1 of the present embodiment, when there is a cat's eye that can be recognized on the image even at a relatively long distance, the analysis area is recognized as the bots dot. Since the image is enlarged as compared with the case, the virtual straight line (or curve) can be generated more accurately. As a result, the road lane marking can be recognized more accurately.

  In addition, since the near-infrared projector 25 performs control to project light forward of the vehicle as necessary, the cat's eye can be recognized more reliably.

  In addition to the above control, as shown in the flowchart of FIG. 8, when the vehicle speed is high, control for further expanding the analysis region may be performed. This flow is executed, for example, as part of the processing of S140 in the flowchart of FIG.

  First, it is determined whether or not the vehicle speed transmitted from the vehicle speed sensor 30 is equal to or greater than a threshold value (S141). If the vehicle speed is less than the threshold value, no processing is performed.

  If the vehicle speed is greater than or equal to the threshold, the analysis area is further expanded (S142). By doing so, it is possible to recognize the road lane markings more accurately when traveling at high speed. Note that the enlargement is not intended to be expanded indefinitely as the flowchart of FIG. 8 is repeatedly executed. If the vehicle speed is equal to or higher than the threshold value, the image after the enlargement set in the flowchart of FIG. This means that the analysis region is enlarged by a predetermined multiple in the image depth direction.

  Such control may be performed independently of the flowchart of FIG. For example, as shown in FIG. That is, an analysis region is set (S200), and it is determined whether the vehicle speed is equal to or higher than a threshold value (S210). If the vehicle speed is less than the threshold value, the bottling dot is recognized in the initially set analysis region to recognize the road lane marking (S230). On the other hand, if the vehicle speed is equal to or higher than the threshold value, the analysis area is expanded (S220), the cat's eye (and botsdots) is recognized, and the road lane marking is recognized (S230). If the control of FIGS. 8 and 9 is not performed, the vehicle speed sensor 30 is not required.

[Use of recognized road lane markings]
The positional relationship between the traveling lane and the host vehicle derived by the image recognition unit 42 is output to the steering signal generation unit 44. The steering signal generator 44 issues a warning by a buzzer when it is predicted that the vehicle will deviate from the driving lane after a predetermined time (for example, 0 comma number [sec] to several [sec]) based on the input positional relationship. At the same time, a steering signal is generated so as to output a small auxiliary steering force for a short time, and is transmitted to the steering device 60 (lane departure warning control). In addition, a steering signal is generated and transmitted to the steering device 60 so as to continuously output a small auxiliary steering force so that the host vehicle can stably travel in the vicinity of the center of the traveling lane (lane keeping support control). By these controls, it is possible to suppress deviation of the host vehicle from the traveling lane.

  Strictly speaking, the steering signal generation unit 44 is a functional block that exceeds the category of “road lane marking recognition device” in the claims, and the road lane marking recognition device 1 of this embodiment In other words, “steering control system including line recognition device”.

  The steering device 60 is, for example, an electric power steering device, and includes a steering angle sensor, a torque sensor, an assist motor, a controller, and the like. During normal times when the steering signal is not transmitted from the LKA ECU 40, the controller of the steering device 60 generates torque necessary for steering the vehicle based on the steering torque signal from the torque sensor and other vehicle state signals (vehicle speed, yaw rate, etc.). A control signal is output to the drive circuit of the assist motor so as to output. When the steering signal is transmitted from the LKA ECU 40, the assist motor is controlled based on the steering signal from the LKA ECU 40 in addition to (or instead of) the above-described normal assist motor control.

[Second Embodiment]
The road marking line recognition device 2 according to the second embodiment of the present invention will be described below. FIG. 10 is a diagram illustrating an example of the overall configuration of the road lane marking recognition device 2. In addition, about the duplication part with 1st Example, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The road lane marking recognition device 2 includes a visible light camera 15 in place of the near infrared camera 10 included in the road lane marking recognition device 1 of the first embodiment, and excludes the near infrared projector 25. Yes.

  The visible light camera 15 is, for example, a camera that uses an image sensor such as a CCD or a CMOS disposed in the upper center of the windshield, and has an optical axis directed obliquely downward in front of the vehicle, and the road ahead of the vehicle. Take an image. The captured image of the visible light camera 15 is transmitted to the LKA ECU 40 as an image signal generated by an interlace method such as NTSC (National Television Standards Committee).

  The light projection control in this embodiment is performed by the visible light of the headlight. That is, the headlight is irradiated when it is determined that it is difficult to recognize the cat's eye even during the daytime. Further, it is conceivable to increase the irradiation light of the headlight when it is determined that the irradiation light is insufficient even at night.

  For this reason, it is desirable that the headlight included in the vehicle on which the present embodiment is mounted includes an actuator that allows the irradiation direction (particularly the horizontal irradiation direction) to be freely adjusted to some extent. Then, the irradiation direction of the headlight is controlled not to be the direction of the vehicle central axis but to be the direction along the traveling lane (road marking line). For the adjustment of the irradiation angle at this time, an inclination angle obtained by road lane marking recognition may be used. If it does in this way, it can control that a pedestrian, a driver of an oncoming vehicle, etc. are dazzled.

  For other effects, such as the effect of recognizing the road lane markings more accurately by expanding the analysis area and recognizing the cat's eye, the function and significance of each component, etc. It is the same.

  The best mode for carrying out the present invention has been described above with reference to the embodiments. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the scope of the present invention. And substitutions can be added.

  For example, the light ECU 20 may be omitted in the first embodiment, and the near infrared projector 25 may be directly controlled by the LKA ECU 40.

  It is also possible to omit the light projection control. In this case, the expansion of the analysis area on the road that is not partitioned by a white line or the like may be performed only at night when the headlights will be irradiated. The detection of nighttime may be made by referring to the irradiation state of the headlight and the value of the internal timer.

  The near-infrared projector 25 may be always operated.

  Further, the expansion of the analysis region according to the vehicle speed is not limited to the two-step control based on the threshold illustrated in the embodiment, and may be expanded in more steps.

  The present invention can be used in the automobile manufacturing industry, the automobile parts manufacturing industry, and the like.

It is a figure which shows an example of the whole structure of the road lane marking recognition apparatus 1 which concerns on 1st Example of this invention. It is a figure which shows typically the arrangement | positioning position of each component, and the operating condition of this apparatus. It is a figure showing the analysis region set on the picked-up image of near infrared camera 10 when recognizing a white line etc. It is a figure which shows an inclination angle and an offset. It is a figure showing the analysis area | region set on the picked-up image of the near-infrared camera 10 in principle, when recognizing a bots dot and a cat's eye. It is a flowchart which shows the flow of the characteristic process which the image recognition part 42 performs repeatedly. It is a figure which shows the preferable expansion degree of an analysis area | region. It is a figure which shows the flow of control which expands an analysis area | region further when vehicle speed is large. It is a figure which shows the flow of control which expands an analysis area | region when a vehicle speed is large. It is a figure which shows an example of the whole structure of the road lane marking recognition apparatus 2 which concerns on 2nd Example of this invention.

Explanation of symbols

1, 2 Road marking line recognition device 10 Near infrared camera 15 Visible light camera 20 Light ECU
25 Near infrared projector 30 Vehicle speed sensor 40 ECU for LKA
42 Image Analysis Unit 44 Steering Signal Generation Unit 50 Main Switch 60 Steering Device

Claims (6)

A road lane marking recognition device that includes an imaging means for imaging a vehicle periphery and recognizes a road lane marking by analyzing a captured image of the imaging means,
When recognizing a road lane marking in a manner that divides the road with a white line or a yellow solid line or a broken line, a first analysis region is set in the captured image of the imaging means,
When recognizing road lane markings that divide the road with targets other than solid white lines, yellow lines, and broken lines,
When recognizing a non-reflective target that recognizes a non-reflective target that does not have a light-reflecting material and recognizes a road marking line, the captured image of the imaging unit is displayed on the front side of the vehicle compared to the first analysis region Setting a second analysis region that is biased and narrower than the first analysis region;
At the time of reflective target recognition for recognizing a road marking by recognizing a reflective target having a light reflecting material, a third analysis area obtained by enlarging the second analysis area in the depth direction is displayed on the captured image of the imaging means. It is characterized by setting,
Road marking line recognition device. The road marking line recognition device according to claim 1,
An apparatus for recognizing road lane markings, wherein the image analysis area in the captured image of the imaging means is expanded to the extent that a plurality of the reflective targets laid along the road are included during the reflection target recognition. A road lane marking recognition device according to claim 1 or 2,
A reflecting target detecting means for detecting that the reflecting target is laid on a road;
A road lane marking recognition, characterized in that, when the reflection target detection means detects that the reflection target is laid on a road, the road marking is recognized by recognizing the reflection target. apparatus. The road marking line recognition device according to claim 3,
A light projecting means for projecting toward the front of the vehicle,
A road lane marking recognition apparatus characterized in that when the target detection means detects that the reflection target is laid on a road, the light projection means projects light. The road lane marking recognition device according to claim 4,
A road lane marking recognition apparatus, wherein the light projecting means is controlled so as to project light along a traveling lane partitioned by a road lane marking. A road lane marking recognition device according to any one of claims 1 to 5,
Vehicle speed detection means for detecting the vehicle speed,
A road lane marking recognition apparatus that expands an image analysis area in a captured image of the imaging means in accordance with an increase in vehicle speed detected by the vehicle speed detection means.

Images