JP2004310522A - Vehicular image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicular image processor that increases recognition accuracy for a road with a plurality of white lines drawn, in a vehicular image processor for recognizing a traffic lane on which one's own vehicle is traveling by image recognition. <P>SOLUTION: Road division lines of a traffic lane on which one's own vehicle is traveling is detected by edge extraction from an image acquired by an imaging means mounted on the vehicle. In this case, if by the edge extraction a plurality of road division line candidates are detected on one side of the traffic lane, edge periods of the respective candidates are compared and the candidate that gives a closest continuous line is determined as a road division line. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a vehicular image processing device for recognizing a lane in which a vehicle travels.
[0002]
[Prior art]
Development of automatic driving equipment that automatically runs on the set lane, technology to warn of inadvertent departure from the lane of the vehicle, and to activate a system that protects occupants by predicting collision after departure Is being promoted. In such a technique, it is necessary to accurately recognize the lane in which the host vehicle should travel. As a technique for this, an image in the traveling direction of the vehicle is acquired, and a boundary line (white line) that separates the lane is detected by image recognition. There is known an apparatus for performing such processing (for example, see Patent Document 1).
[0003]
This technology detects an edge point that changes from light to dark and an edge point that changes from dark to light in a road surface image, and performs a Hough transform on them to obtain peaks corresponding to each of these edge points from the Hough space. Are detected as a pair to detect a white line. This describes that the position of a white line can be accurately detected even if the road has a complicated shape, for example, a double white line on a road.
[0004]
[Patent Document 1]
JP-A-11-85999 (paragraphs 0056 to 0059, FIG. 3)
[0005]
[Problems to be solved by the invention]
However, in the technique of Patent Document 1, although the accuracy of detecting white lines on a road is improved, when a plurality of white lines are detected, how to distinguish them, and which white line should be recognized as a road division line. Is not listed.
[0006]
Accordingly, the present invention provides a vehicular image processing device that recognizes a lane in which a vehicle is traveling by image recognition and that has improved recognition accuracy when a plurality of white lines are drawn on a road. That is the task.
[0007]
[Means for Solving the Problems]
In order to solve the above-described problems, a vehicular image processing apparatus according to the present invention provides a vehicular image processing device that detects a road lane marking of a lane in which a vehicle travels by extracting edges from an image acquired by an imaging unit mounted on the vehicle. In the processing apparatus, when a plurality of candidates are detected on one side of a lane by edge extraction, the edge cycle is compared, and a candidate closest to a continuous line is determined as a road division line. apparatus.
[0008]
If the lane in which the vehicle is traveling is a double white line, one of the lanes is the original lane line, and the other is an auxiliary line such as a gaze guidance line. The auxiliary line is a discontinuous line such as a broken line so that the auxiliary line and the original division line can be visually distinguished. Therefore, by selecting the line closest to the continuous line from the plurality of lines (candidates) detected by the edge detection, the original division line can be selected.
[0009]
It is preferable to determine that the candidate having the shortest edge interval in the predetermined cycle is closest to the continuous line. Alternatively, it is preferable to compare the dashed line periods and determine that the candidate with the longest dashed line period is closest to the continuous line. The closer the line is to the continuous line, the shorter the edge interval and the longer the broken line period. Therefore, the line closest to the continuous line can be selected from the plurality of lines by comparing the edge interval or the broken line period.
[0010]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate understanding of the description, the same components are denoted by the same reference numerals as much as possible in each drawing, and redundant description is omitted.
[0011]
FIG. 1 is a block diagram of a vehicle control device including a vehicle image processing device according to the present invention, and FIG. 2 is a perspective view showing a vehicle equipped with the same. The vehicle control device 100 includes a front camera 11 that acquires an image in front of the vehicle, and a white line recognition ECU 1 that recognizes a white line on the front course of the vehicle by image processing from the image acquired by the front camera 11 (the vehicle according to the present invention). Image processing device). Output signals of the yaw rate sensor 12, the steering angle sensor 13, and the vehicle speed sensor 14 are input to the white line recognition ECU 1, and the recognition result of the white line recognition ECU 1 is transmitted to the PCS (Pre-Crash Safety) system 3, via the in-vehicle LAN 2, It is sent to the departure warning system 4, the lane keeping system 5, and the like. Details of the PCS system 3, the departure warning system 4, and the lane keeping system 5 will be described later. The functions may be integrated in the same ECU without using the in-vehicle LAN 2.
[0012]
The front camera 11 is disposed above a front window of the vehicle 200 (for example, behind a rearview mirror) as shown in FIG. 2, and an image in front of the vehicle 200, that is, an image of a traveling lane 300 in front of the vehicle (section). (Including the line 301). The front camera 11 may be provided at any position on the vehicle body as long as the front image can be captured.
[0013]
FIGS. 3A and 3B are diagrams illustrating examples of images acquired by the front camera 11. The traveling lane 300 shown in FIG. 3A is divided on both sides by demarcation lines 301R and 301L, and gaze guidance lines 302R and 302L are provided inside the lane so that the driver can easily recognize the lane markings 301R and 301L. It is provided in parallel with the dividing lines 301R and 301L. The traveling lane 300 shown in FIG. 3B is also divided on both sides by the dividing lines 301R and 301L, but the merging line 303 indicating that the vehicle joins from the right side inside the dividing line 301R. Are provided in parallel. In any case, the original division lines 301R and 301L are continuous lines, and the line-of-sight guidance lines 302R and 302L and the merging line 303 are discontinuous lines. Since these are usually painted white, hereinafter, the painted portion is referred to as a white line, but the white line also includes a case where the color is painted yellow other than white and a case where it is partitioned by tacks, blocks, or the like. Note that the division lines 301R and 301L may be discontinuous lines, but also in this case, the line-of-sight guidance lines 302R and 302L and the merging line 303 (hereinafter, collectively referred to as white lines on other roads). ), The length of the white line portions on the division lines 301R and 301L is longer than the white lines on other roads, and the interval between the white line portions is set shorter than the white lines on other roads. . In other words, the original division line has a shape closer to a continuous line than other white lines.
[0014]
Hereinafter, the process of recognizing the positions of the division lines 301R and 301L from the acquired image by image processing will be specifically described. FIG. 4 is a processing flow of this image processing, and FIGS. 5 to 8 are diagrams for explaining the processing contents. The processing described below is performed by the white line recognition ECU 1 unless otherwise specified.
[0015]
First, an image as shown in FIG. 5 is obtained by the front camera 11 (step S1). This image includes left and right division lines 301L and 301R and gaze guidance lines 302L and 302R provided inside the division lines.
[0016]
The acquired image is subjected to AD conversion to obtain a predetermined number of pixels (for example, the horizontal direction in FIG. 5 is the x direction, the vertical direction is the y direction, and x × y is 320 × 240 pixels) (for example, monochrome gray scale). The image data is converted into digital image data (8 gradations) and sent to the white line recognition ECU 1 (step S2). Of course, a digital camera capable of directly outputting digital data having a predetermined number of pixels and gradation may be used for the front camera 11, and the analog output of the front camera 11 may be AD-converted inside the white line recognition ECU 1. The converted digital image is substantially identical to the analog image shown in FIG.
[0017]
Next, the white line recognition ECU 1 extracts a white line candidate from the received digital image by performing edge extraction processing on the image (step S3). For example, a pixel whose luminance on the left side is lower than the threshold value and whose luminance on the right side pixel is equal to or higher than the threshold value is extracted as an edge, using the average luminance in the image as a threshold value. As a result, a pixel position corresponding to the left boundary of the white line (including both the division lines 301L and 301R and the line-of-sight guidance lines 302L and 302R) is extracted. The edge extraction is not limited to this. For example, a pixel whose luminance difference from the left pixel is equal to or larger than a threshold may be extracted as an edge. These were to extract the pixel position at the position corresponding to the left boundary line of the white line by extracting the edge that changes from dark to light. Conversely, by extracting the edge that changes from light to dark, the white line was extracted. May be extracted as the pixel position corresponding to the right boundary line of.
[0018]
FIG. 6 shows the extracted edge position superimposed on the original image. Black circles indicate the extracted edge positions, and broken lines indicate pixel positions in the Y direction. Hereinafter, an explanation will be given of the edge of the area surrounded by the hatched circle, that is, the extracted edge pixels corresponding to the division line 301L and the visual line 302L.
[0019]
The acquired image of the traveling lane 300 is obtained by projecting the traveling lane 300 on the image plane of the front camera 11. On the other hand, the positions of the lane markings 301L and 301R that are actually desired are positions with respect to the vehicle on the traveling lane 300. Therefore, by means of geometric transformation, with the vehicle as the origin (0, 0), the horizontal direction in the X-axis direction (the right direction as viewed from the vehicle is positive) is changed to the distance from the vehicle in the Y-axis direction (the larger the numerical value, the larger the value). A position corresponding to the edge pixel position on the traveling plane, which is expressed by taking forward (that is, being away from the vehicle), is obtained (step S4). This geometric transformation can be performed based on the mounting position of the front camera 11, the imaging direction, and the imaging area. Note that the image pickup direction and the like may be corrected by detecting the inclination of the vehicle body and the like with a plurality of vehicle height sensors. In addition, it is preferable that the image distortion caused by the optical aberration of the imaging system of the front camera 11 is also corrected. FIG. 7 shows the positions of the extracted edge pixels corresponding to the division line 301L and the line-of-sight guidance line 302L obtained by the geometric transformation in this way with respect to the vehicle.
[0020]
Next, straight line extraction is performed by performing Hough transformation on the result of the geometric transformation (step S5). Specifically, when the position of the extracted edge point on the traveling plane is represented by (x, y), the inclination θ is changed from 0 degree to 180 degrees for each edge point at a predetermined angle Δθ, Ρ corresponding to each θ is obtained as ρ = xcos θ + ysin θ. Then, the (ρ, θ) obtained in this manner is searched for in which of the areas divided at predetermined intervals in the Huff space, and a vote is made on the searched area. In other words, the number of votes in each area indicates the number of Hough transform results of edge points in this area. This voting is preferably performed separately for the left side and the right side of the traveling lane 300. The center line of the driving lane 300 required to perform the left and right division of the driving lane 300 can be grasped by referring to the lane recognition results, the traveling direction of the vehicle, the yaw rate, the steering angle, and the like. it can. FIG. 8 shows a voting result in the Hough transform area on the left side of the traveling lane 300 obtained in this manner.
[0021]
A (ρ, θ) region having a plurality of votes is shown as a straight line extracted from an edge point (even if the region is intermittent, it is also extracted as one line). This straight line corresponds to y = ρ / sin θ−x × cos θ / sin θ on the (x, y) plane. Here, the larger the voting result (ρ, θ), the closer the edge point is to a straight line. Considering the influence of noise and the like, a straight line having the number of votes equal to or more than a threshold value (3, preferably) is set as an extraction result (white line candidate) (step S6). Then, the number of white line candidates existing on one side of the traveling lane 300 is checked (step S7). If there is only one white line candidate, position information is output using that candidate as a dividing line (step S11). If there are a plurality of white line candidates on one side of the traveling lane 300, the process proceeds to the lane line selection processing within the white line candidates.
[0022]
Here, since the division line 301L (301R) and the line-of-sight guidance line 302L (302R) provided inside are provided in parallel, the inclination θ of the straight line extracted from both edges is the same. Furthermore, since more edge points are detected closer to the continuous line, this means that the edge period is shorter in the Hough transform target area. The number of votes increases. Therefore, it can be determined that the white line candidate with the same inclination and the largest number of votes is closest to the continuous line among the white line candidates running in parallel. Therefore, when there are a plurality of white line candidates, the number of votes and the inclination of the extracted white line candidates are compared, and among the groups whose difference in inclination falls within the threshold value, the group with the largest number of votes is determined as the white line candidate. , And the others are excluded from white line candidates (step S8). Then, the number of remaining white line candidates is checked again (step S9). When there is one group and one remaining white line candidate, position information is output using the candidate as a dividing line (step S11).
[0023]
When there are a plurality of groups, it is considered that a line that can intersect with the division line is recognized. Selection of a white line candidate from such intersecting lines is performed by comparing with a recognition result up to that time, and comparing with a traveling direction of a vehicle or lane line information on the opposite side of the traveling lane 300 (usually, a section on both sides of the traveling lane 300). The lines 301R and 301L are also arranged in parallel.) (Step S10). When the number of white line candidates is reduced to one in this manner, the process proceeds to step S11, and position information is output using the candidates as demarcation lines.
[0024]
According to the present invention, even when another white line is provided in parallel with the original marking line, the marking line can be automatically selected and recognized, so that the recognition accuracy of the marking line is improved. For this reason, various vehicle controls using the lane information can be performed with high accuracy. Further, even when the lane markings are not continuous lines, or when the lane markings that are continuous during edge extraction are recognized as discontinuous lines, there is an advantage that the lane marking selection accuracy can be maintained.
[0025]
In the above description, the Hough transform is performed on all the geometrically transformed areas, but the area where the Hough transform is performed may be limited to a predetermined area in front of the vehicle (see FIG. 9). Here, the target area where the Hough transform is performed is called a control cycle. The interval between the edges corresponds to the edge period. This control cycle may be, for example, an area where the vehicle can reach after t1 to t2 seconds based on the output of the vehicle speed sensor 14. By limiting the region to be subjected to the Hough transform in this manner, the edge cycle can be reliably compared, and accurate detection can be performed even when a plurality of white line candidates exist only at positions close to the vehicle. be able to.
[0026]
In the above description, an example in which the white line candidate with the largest number of votes is selected as the division line from the parallel white line candidates has been described, but the method of selecting the white line candidate is not limited to this.
[0027]
FIG. 10 is a processing flow showing another form of the selection processing, and FIG. 11 is an explanatory diagram of the processing. In this process, the process of step S8 of the process shown in FIG. 4 is replaced. First, in step S21, the result of the Hough transform is projected on the (x, y) plane, and this is geometrically transformed into the projection on the image plane of the front camera 11 (since the geometric transformation is the reverse of the geometric transformation in step S4). , Hereinafter referred to as an inverse geometric transformation) to obtain an image as shown in FIG. This corresponds to an image in which a white line candidate line is drawn in the edge extraction image shown in FIG.
[0028]
Next, for each edge point on the same white line candidate line (including an edge point that is close to a predetermined range), an edge cycle (interval between adjacent edge points) is determined. Specifically, a distance in the Y coordinate direction, that is, a difference between adjacent edge points of the Y coordinate is obtained (step S22). When edges are extracted from a continuous line, the distance between edge points in the Y coordinate direction is one pixel. On the other hand, in the case of a discontinuous line, since there is no edge point in a portion where the line is interrupted, the distance in the Y coordinate direction between adjacent edge points on the same white line candidate, that is, the difference is large. There exists a place. This difference increases as the distance between the coating portions increases, and the frequency at which the difference increases increases as the length of the coating portion decreases. Therefore, the number of differences (for example, the distance in the Y coordinate direction between adjacent edges is 3 pixels or more) between the white line candidates (the difference occurrence frequency) and the magnitude of the difference (the maximum value in the Y coordinate direction between adjacent edges) Or the average value), and when the frequency of occurrence of the difference is low and the frequency of occurrence is almost the same, it is determined that the white line candidate having a smaller difference is close to a continuous line, and the other is excluded from the white line candidates (step S23). . When the number of white line candidates is reduced to one in this way, the process proceeds to step S8, and position information is output using the candidates as partition lines.
[0029]
Also in this case, as in the case described above, the area for comparing the differences may be limited to a predetermined range. In addition, a region sandwiched between regions having a large difference from an adjacent edge is regarded as a painted portion, and the number of edge points (corresponding to the length of the painted portion) included in the painted portion is compared to form a continuous line. May be narrowed down from the white line candidates. In this case, when there is no region where the difference is large within the predetermined range of the white line candidate (when the region is a continuous line), the number of all the edges included in the predetermined range is included in the painted portion in the white line candidate. Calculated as the number of edges to be processed.
[0030]
Next, vehicle control in the vehicle control device according to the present invention using the white line information thus obtained will be described. 12 to 14 are block diagrams of the PCS system 3, the departure warning system 4, and the lane keeping system 5 shown in FIG.
[0031]
First, the PCS system 3 shown in FIG. 12 will be described. The PCS system includes a braking system 32 for controlling a braking force, a steering assist motor 33 for applying an assisting force to a driver's steering force, a seat belt pretensioner 34 for winding up a seat belt, and an airbag 35 for occupant protection. And a PCS / ECU 31 for performing these controls. The PCS / ECU 31 has a laser radar 16 for detecting a preceding vehicle and an obstacle, a yaw rate sensor 12, a steering angle sensor 13, a vehicle speed sensor 14, and a brake pedal. The output of the brake pedal switch 15 for detecting the operation or non-operation state is input.
[0032]
The PCS / ECU 31 detects a preceding vehicle and an obstacle on the course of the own vehicle based on lane information (white line information) sent from the lane recognition ECU 1 and outputs of the sensors 12 to 15 and the laser radar 16. Predict the likelihood of a collision. When it is determined that a collision is inevitable, the braking system 32 is instructed to switch to a setting in which the assist force applied during the brake operation is increased in advance, so that when the driver senses a danger and operates the brake pedal, a large force is applied. Make sure that braking force is obtained.
[0033]
Further, the assist steering force by the steering assist motor 33 is increased so that the driver can perform steering with a weaker force. If necessary, steer automatically to avoid collision.
[0034]
Further, by winding the extra length of the seat belt in advance by the seat belt pretensioner 34 before the collision, the occupant is more strongly restrained by the seat and the damage at the time of the collision is reduced. In addition, by operating the airbag at an optimal timing before and after the collision, not after detecting the collision, damage to the occupant at the time of the collision is reduced. With these controls, even when a collision cannot be avoided, it is possible to suppress serious damage to the vehicle and the occupant and reduce the shock of the collision.
[0035]
Next, the departure warning system 4 shown in FIG. 13 will be described. The departure warning system 4 includes a departure warning ECU 41, a monitor device 42 for display, and a speaker 43 for sound output. The departure warning ECU 41 has lane information (white line information), vehicle speed information, yaw rate, and the like. Information has been entered.
[0036]
The departure warning ECU 41 predicts the course of the host vehicle from the vehicle speed / yaw rate information and compares this with the lane information. When the vehicle is expected to depart from the lane, a message indicating that the vehicle is expected to deviate from the lane is displayed on the monitor device 42, and an alarm is generated by the speaker 43 to generate an alarm sound or sound output. Notify to the effect.
[0037]
Subsequently, the lane keeping system 5 shown in FIG. 14 will be described. The lane keeping system 5 includes a lane keeping ECU 51, a braking system 32 for controlling a braking force, a steering assist motor 33 for applying an assisting force to a driver's steering force, and a speed changing means 52 for changing a transmission shift state. The lane keeping ECU 51 includes a laser radar 16 for detecting a preceding vehicle and an obstacle, a yaw rate sensor 12, a steering angle sensor 13, a vehicle speed sensor 14, and operation of a brake pedal. The outputs of a brake pedal switch 15 for detecting a non-operation state and an accelerator opening sensor 17 for detecting an operation amount of an accelerator pedal are input.
[0038]
The lane keeping system 5 travels while maintaining the position of the vehicle in the traveling lane 300 currently traveling. Specifically, by predicting the course of the host vehicle from the outputs of the yaw rate sensor 12, the steering angle sensor 13, and the vehicle speed sensor 14, and comparing this with the lane information, it is determined whether traveling in the traveling lane 300 has been maintained. Is determined. Then, based on the front lane information, the steering assist motor 33 is controlled so that the own vehicle travels within the white line from the position of the own vehicle and the white line, so that the traveling in the traveling lane 300 is maintained.
[0039]
As described above, by detecting the lane marking of the traveling lane of the vehicle using the image processing apparatus for a vehicle according to the present invention, the recognition accuracy of the lane marking, and thus the traveling lane, is improved, and various types of vehicles using the lane marking are improved. Can be controlled accurately.
[0040]
In the above description, only the case where a straight line is detected by Hough transform has been described as an example. However, if a known curve detection technique using Hough transform is used, a traveling lane including a curve can be detected. Also in this case, since the plurality of white lines are drawn in parallel, the division lines can be selected by the same method as described above.
[0041]
【The invention's effect】
As described above, according to the present invention, when a plurality of candidates are detected as demarcation lines based on an edge extraction value, a candidate determined to be closest to a continuous line by comparing the cycle of edges is determined as a demarcation line. Since the determination is made, even when another white line is drawn in parallel with the original division line, the original division line can be accurately recognized.
[Brief description of the drawings]
FIG. 1 is a block diagram of a vehicle control device including a vehicle image processing device according to the present invention.
FIG. 2 is a perspective view showing a vehicle equipped with the vehicle control device of FIG. 1;
FIGS. 3A and 3B are diagrams showing examples of images acquired by the front camera 11 of FIGS. 1 and 2, wherein FIG. 3A shows a lane with a line-of-sight guidance line, and FIG. 3B shows a lane with a merging line; .
FIG. 4 is a processing flow of image processing in the image processing apparatus according to the present invention.
FIG. 5 is an example of an image acquired by a front camera.
FIG. 6 is a diagram showing edge positions extracted by image processing.
FIG. 7 is a diagram showing positions of extracted edge pixels obtained by geometric transformation with respect to a vehicle.
FIG. 8 is an explanatory diagram showing a voting result in the Hough transform area of FIG. 7;
FIG. 9 is an explanatory diagram of a target area when the target area of the Hough transform is limited.
FIG. 10 is a processing flow showing another form of the selection processing in the processing of FIG. 4;
11 is a diagram illustrating the inverse geometric transformation in the process of FIG.
FIG. 12 is a block diagram of the PCS system shown in FIG. 1;
FIG. 13 is a block diagram of the departure warning system shown in FIG. 1;
FIG. 14 is a block diagram of the lane keeping system shown in FIG. 1;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... White line recognition ECU, 2 ... In-vehicle LAN, 3 ... PCS (Pre-Crash Safety) system, 4 ... Departure warning system, 5 ... Lane keep system, 11 ... Front camera, 12 ... Yaw rate sensor, 13 ... Steering angle sensor, 14: vehicle speed sensor, 15: brake pedal switch, 16: laser radar, 17: accelerator opening sensor, 31: PCS / ECU, 32: braking system, 33: steering assist motor, 34: seat belt pretensioner, 35: air Bag, 41: departure warning ECU, 42: monitoring device, 43: speaker, 51: lane keeping ECU, 52: transmission means, 53: electronic control throttle, 100: vehicle control device, 200: vehicle, 300: traveling lane, 301 … Compartment line, 302… gaze guidance line, 303… merge line.

Claims (3)

An image processing device for a vehicle that detects a road lane marking of a lane in which the vehicle travels by extracting an edge from an image acquired by an imaging unit mounted on the vehicle,
When a plurality of road lane marking candidates are detected on one side of a lane by edge extraction, an edge cycle is compared, and a candidate closest to a continuous line is determined as a road lane marking. . 2. The vehicular image processing apparatus according to claim 1, wherein a candidate having the shortest edge interval in a predetermined cycle is determined to be closest to a continuous line. 2. The image processing device for a vehicle according to claim 1, wherein the dashed line cycle is compared to determine that the candidate having the longest dashed line cycle is closest to the continuous line.

Images