JP6259239B2 - Vehicle white line recognition device - Google Patents

Description

  The present invention relates to a vehicle white line recognition device that recognizes a white line based on an image captured by an in-vehicle camera.

  In recent years, various driving support systems have been developed in the automobile field with the development of electronics. In this type of driving support device, in general, in order to realize a lane departure prevention function or the like, a white line is recognized based on a captured image or the like in front of the host vehicle, and the own vehicle traveling lane is estimated based on the white line or the like. Done.

  That is, in white line recognition using an image, for example, a change in luminance is examined from the inside to the outside in the vehicle width direction for each of a plurality of search lines extending in the horizontal direction with respect to the search area set on the captured image. Thus, an edge point whose luminance changes from dark to bright by a predetermined value or more is detected. Then, a white line is recognized by approximating the detected edge point group with a quadratic curve or the like using a least square method or the like.

  By the way, the white line on the actual road is a double line in which an auxiliary lane line such as a line of sight guide line is provided inside the lane line in addition to a white line (single white line) constituted by a single lane line. There is a white line. When recognizing such a double white line, in order to avoid the influence of dirt, puddles, or grass that is often present near the shoulders on the road, the edge point group of the auxiliary lane markings existing on the inner side in the vehicle width direction is used. It is desirable to approximate the white line using a curve. On the other hand, when the double white line is recognized based on the edge points of the auxiliary lane markings, the point where the white line on the road switches from the double white line to the single white line (double white line exit), or the single white line The edge point group shifts to the outside in the vehicle width direction or the inside in the vehicle width direction at the point where the white line is switched to the double white line (double white line entrance). And if the edge point group fluctuates in this way and the continuity of the edge point group is temporarily impaired, temporary disturbance occurs in the white line approximate line, preventing lane departure using alarm control, steering control, etc. May affect function.

  To cope with this, for example, Patent Document 1 obtains parameter coefficients of a lane estimation formula from the least squares method based on a point group (edge point group) of lane candidate points, and sets a time interval before the set time width based on the time of this calculation. In the white line recognition that sets the prediction parameter after the set time advance from the parameter coefficient point sequence at, when the double white line entrance / exit is detected, the white line approximate line is sharpened by setting the above set time width long. A technique for suppressing fluctuation is disclosed.

JP2012-58984A

  However, in the technique disclosed in Patent Document 1 described above, for example, when the double white line exit and the curve entrance appear on the road at substantially the same timing, the influence of past data (past parameter coefficients, etc.) Therefore, it is difficult to recognize the curve shape with high responsiveness, and the controllability such as the lane departure prevention function may be deteriorated.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a vehicular white line recognition device capable of recognizing a road shape with good responsiveness without reducing recognition accuracy even at the exit of a double white line. To do.

A vehicle white line recognition device according to an aspect of the present invention is configured such that a plurality of search lines extending in the horizontal direction within each left and right white line detection region on an image obtained by imaging the front side of the host vehicle traveling path are arranged from the vehicle width inner side to the outer side. The luminance change in the direction is examined, and when there is an edge point where the luminance changes from dark to bright over a predetermined value, the first edge point is detected as the first edge point and the next edge point is detected as the second edge point in order. edge point detecting means, wherein on the same of the search line first, when the second edge points are detected in pairs, the first white line based on the distance between the second edge point are two An offset amount calculating means for calculating an offset amount between the lane line and the auxiliary lane line in the case of a heavy white line, and a white line that divides the vehicle traveling path based on the point cloud of the first and second edge points. Single white line consisting of a single lane line Alternatively, it is determined that the white line is a double white line by the white line state determining means for determining whether the white line is a double white line with an auxiliary lane line inside the lane line, and the white line state determining means. A white line candidate point extracting means for extracting a white line candidate point defining the auxiliary partition line from the point group of the edge points detected by the edge point detecting means, and the coordinates of the white line candidate point extracted before the previous frame And a white line based on a point group of the white line candidate point and the back white line candidate point. White line approximating means, and the white line state determining means determine that the white line has switched from a double white line to a single white line, and the white line candidate point and the rear white line candidate point are offset to the outside in the vehicle width direction. A candidate point shifting means for shifting at, those having a.

  According to the white line recognition device for a vehicle of the present invention, it is possible to recognize the road shape with high responsiveness without reducing the recognition accuracy even at the exit of the double white line.

Schematic configuration diagram of a vehicle driving support device Flow chart showing white line recognition routine (part 1) Flow chart showing white line recognition routine (part 2) Flow chart showing white line recognition routine (part 3) Flowchart showing white line recognition routine (4) Schematic diagram showing an example of edge point groups detected from the image of the double white line entrance Explanatory drawing which shows an example of the edge point group of the double white line entrance coordinate-transformed on real space Schematic diagram showing an example of the edge point group detected from the image of the double white line exit Explanatory drawing which shows an example of the edge point group of the double white line exit by which coordinate conversion was carried out on real space Explanatory drawing which shows an example of the change of the brightness | luminance on a search line Explanatory drawing of coordinate conversion when moving the previous representative position to the current representative position Explanatory drawing which shows the estimated position of the edge point accompanying the movement of the own vehicle Explanatory drawing which shows an example of the edge point group coordinate-converted on real space, and a back white line candidate point Explanatory drawing showing the calculation method of Hough transform Explanatory drawing showing Hough space (A) is explanatory drawing which shows an example of a Hough straight line and a white line candidate point extraction area | region, (b) is explanatory drawing which shows an example of the extracted white line candidate point (A) is explanatory drawing which shows an example of an edge point inside a white line candidate point extraction area | region immediately before determined as a double white line entrance, (b) is a white line candidate point when it determines with a double white line entrance Illustration showing an example (A) is explanatory drawing which shows an example of the relationship between a white line candidate point and a back white line candidate point when it determines with a double white line entrance, (b) is a back white line candidate point shifted inside a vehicle width direction. Explanatory drawing showing an example Explanatory drawing which shows an example of the white line approximate curve recognized from the white line candidate point and the back white line candidate point, and a white line search area | region. (A) is explanatory drawing which shows an example of the white line candidate point extracted based on the 1st, 2nd edge point pair, (b) is an example of the white line candidate point extracted based on the division line space | interval. Illustration showing (A) is explanatory drawing which shows an example of the relationship between a white line candidate point and a back white line candidate point when it determines with a double white line exit, (b) is a vehicle width direction outer side when it determines with a double white line exit Explanatory drawing which shows an example of the white line candidate point and back white line candidate point shifted to Explanatory drawing which shows an example of the white line candidate point and back white line candidate point shifted to the vehicle width direction outer side in the setting frame after determining with a double white line exit

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The drawings relate to an embodiment of the present invention, FIG. 1 is a schematic configuration diagram of a vehicle driving support device, FIGS. 2 to 5 are flowcharts showing a white line recognition routine, and FIG. 6 is detected from an image of a double white line entrance. FIG. 7 is a schematic diagram illustrating an example of an edge point group, FIG. 7 is an explanatory diagram illustrating an example of an edge point group at a double white line entrance that has been coordinate-transformed in real space, and FIG. 8 is an edge detected from an image at the double white line exit. FIG. 9 is a schematic diagram illustrating an example of a point group, FIG. 9 is an explanatory diagram illustrating an example of an edge point group at the exit of a double white line coordinate-converted in real space, and FIG. FIG. 11 is an explanatory diagram of coordinate conversion when moving the previous representative position to the current representative position, FIG. 12 is an explanatory diagram showing an estimated position of an edge point accompanying the movement of the host vehicle, and FIG. 13 is in real space Shows an example of edge point group and rear white line candidate point whose coordinates are converted to FIG. 14 is an explanatory diagram showing the calculation method of the Hough transform, FIG. 15 is an explanatory diagram showing the Hough space, and FIG. 16A is an explanatory diagram showing an example of the Hough straight line and white line candidate point extraction region (b) ) Is an explanatory diagram showing an example of extracted white line candidate points, and FIG. 17A is an explanatory diagram showing an example of edge points inside the white line candidate point extraction region immediately before being determined as a double white line entrance. FIG. 18B is an explanatory diagram showing an example of a white line candidate point when it is determined as a double white line entrance, and FIG. 18A shows a white line candidate point and a back white line candidate point when it is determined as a double white line entrance. It is explanatory drawing which shows an example of a relationship, (b) is explanatory drawing which shows an example of the back white line candidate point shifted inside a vehicle width direction, FIG. 19 is the white line approximation curve recognized from the white line candidate point and the back white line candidate point FIG. 20A is an explanatory diagram showing an example of a white line search area, and FIG. FIG. 21B is an explanatory diagram illustrating an example of white line candidate points extracted based on a pair of points, FIG. 21B is an explanatory diagram illustrating an example of white line candidate points extracted based on a lane marking interval, and FIG. It is explanatory drawing which shows an example of the relationship between a white line candidate point and a back white line candidate point when it determines with a heavy white line exit, (b) is the white line shifted outside a vehicle width direction when it determines with a double white line exit FIG. 22 is an explanatory diagram showing an example of candidate points and rear white line candidate points, and FIG. 22 shows an example of white line candidate points and rear white line candidate points that are shifted outward in the vehicle width direction in the setting frame after the double white line exit is determined. It is explanatory drawing.

  In FIG. 1, reference numeral 1 denotes a vehicle such as an automobile (own vehicle), and a driving support device 2 is mounted on the vehicle 1. The driving support device 2 includes, for example, a stereo camera 3, a stereo image recognition device 4, a control unit 5, and the like, and a main part is configured.

  The host vehicle 1 is connected to a vehicle speed sensor 11 that detects the host vehicle speed V, a yaw rate sensor 12 that detects the yaw rate γ, a main switch 13 that performs ON-OFF switching of various functions of driving support control, and the like, and a steering wheel. A steering angle sensor 14 that detects the steering angle θst and that is opposed to the steering shaft, an accelerator opening sensor 15 that detects an accelerator pedal depression amount (accelerator opening) θacc by the driver, and the like are provided.

  The stereo camera 3 is constituted by a set of CCD cameras using a solid-state imaging device such as a charge coupled device (CCD) as a stereo image optical system. These left and right CCD cameras are each mounted at a certain distance in front of the ceiling in the vehicle interior, take a stereo image of an object outside the vehicle from different viewpoints, and output image data to the stereo image recognition device 4. In the following description, one image (for example, the right image) of the stereo images is referred to as a reference image, and the other image (for example, the left image) is referred to as a comparative image.

  First, the stereo image recognition device 4 divides the reference image into small regions of 4 × 4 pixels, compares the luminance or color pattern of each small region with the comparison image, finds the corresponding region, and creates the entire reference image. Find the distance distribution across. Further, the stereo image recognition device 4 examines the luminance difference between each pixel on the reference image and an adjacent pixel, extracts those whose luminance difference exceeds a threshold value as an edge point, and extracts the extracted pixel (edge A distance image (a distribution image of edge points having distance information) is generated by giving distance information to (point). Then, the stereo image recognition device 4 performs a well-known grouping process on the generated distance image, and compares it with a pre-stored three-dimensional frame (window) so that a white line, a side wall, Recognizes three-dimensional objects.

  Here, the white line to be recognized in the present embodiment is, for example, a white line (single white line) configured with a single lane line or a white line (double line with an auxiliary lane line inside the lane line) (White line) is a general term for lines extending on the road and defining the vehicle traveling lane, and the form of each line includes a solid line, a broken line, and the like, and further includes a yellow line and the like. In the white line recognition of the present embodiment, even if the white line that exists on the road is a double white line, it is recognized by approximating it with a single approximate line on each of the left and right sides.

  In this white line recognition, the stereo image recognition device 4 sets left and right white line detection areas A (Al, Ar) on the image, for example, as shown in FIGS. For each of a plurality of existing search lines L, a change in luminance is examined from the inside to the outside in the vehicle width direction. Then, the stereo image recognition device 4 determines the first edge point as the first edge point when there is an edge point on the search line L in each white line detection area A where the luminance changes from dark to bright over a predetermined level. While detecting as the edge point P1, the next edge point is sequentially detected as the second edge point P2.

  Further, the stereo image recognition device 4 uses the first edge point P1 and the second edge point P2 detected in pairs on the same search line L as the second edge point with respect to the first edge point P1. The offset amount W_12 (W_12l, W_12r) of P2 is calculated.

  Further, the stereo image recognition device 4 determines whether the white line that divides the vehicle traveling path is a single white line or a double white line based on the detected point group of the first and second edge points P1 and P2. judge.

  If it is determined that the white line is a single white line, the stereo image recognition device 4 calculates a Hough straight line H (Hl, Hr) based on the point group of the first edge point P1, and based on the Hough straight line H. The first and second edge points P1 and P2 existing in the white line candidate point extraction area Ad (Adl, Adr) determined as described above are extracted as white line candidate points Ps.

  On the other hand, when it is determined that the white line is a double white line, the stereo image recognition device 4 extracts a white line candidate point Ps that defines the auxiliary partition line from the detected point group of the first edge points P1. That is, the stereo image recognition device 4 basically has the first edge point P1 on the search line L where the second edge point P2 is detected (that is, the second edge point on the same search line L). A first edge point P1) detected as a pair with P2 is extracted as a white line candidate point Ps. Furthermore, the stereo image recognition device 4 calculates the interval between the left and right lane markings (lane marking line width W_out) based on the edge point P on the search line L where the second edge point P2 is detected. Then, the stereo image recognition device 4 obtains an interval (edge point interval W_P1) between the first edge points P1 that exist in a left-right pair on the search line L in which the second edge point P2 is not detected. When the point interval W_P1 coincides with the lane line width W_out within the setting error range, the first edge point P1 is extracted as the white line candidate point Ps and shifted inward in the vehicle width direction by the offset amount W_12.

  Further, the stereo image recognition device 4 is located behind the vehicle within a set distance (for example, 20 to 30 [m]) based on the coordinates of the white line candidate point Ps extracted before the previous frame and the movement amount of the vehicle. A white line candidate point Ps_pre is estimated.

  Then, the stereo image recognition device 4 approximates the white line based on the point group of the extracted white line candidate point Ps and the estimated rear white line candidate point Ps_pre.

  Here, when it is determined that the white line is a single white line, the stereo image recognition device 4 determines the number N1_n of white line candidate points Ps extracted in the current frame and the white line candidate point extraction area Ad one frame before. When the difference from the number N0_n-1 of edge points P existing on the inner side in the vehicle width direction is equal to or greater than a preset threshold value Nth, the white line is switched from a single white line to a double white line (that is, the host vehicle 1 Is approaching the double white line entrance). In addition, in the determination of switching from the single white line to the double white line, the state of the Hough straight line H or the like can be further added as a determination condition. When the switching from the single white line to the double white line is determined, the stereo image recognition device 4 shifts the rear white line candidate point Ps_pre inwardly in the vehicle width direction by the offset amount W_12.

  On the other hand, when it is determined that the white line is a double white line, the stereo image recognition device 4 occupies the white line candidate point Ps offset by the offset amount W_12 out of the currently extracted white line candidate points Ps. When the ratio is equal to or greater than the set threshold value Rth, it is determined that the white line has been switched from a double white line to a single white line. When the switching from the double white line to the single white line is determined, the stereo image recognition device 4 shifts the white line candidate point Ps and the rear white line candidate point Ps_pre to the outside in the vehicle width direction by the offset amount W_12. Note that the white line candidate point Ps and the back white line candidate point Ps_pre are preferably shifted in stages over the set number of frames.

  As described above, in this embodiment, the stereo image recognition device 4 includes an edge point detection unit, an offset amount calculation unit, a white line state determination unit, a white line candidate point extraction unit, a rear white line candidate point estimation unit, a white line approximation unit, a candidate point. Each function as a shift means and a lane division line width calculation means is realized.

  The control unit 5 receives driving environment information in front of the host vehicle 1 recognized by the stereo image recognition device 4. Further, the vehicle speed V from the vehicle speed sensor 11, the yaw rate γ from the yaw rate sensor 12, and the like are input to the control unit 5 as travel information of the host vehicle 1, and the operation input information by the driver from the main switch 13. An operation signal, a steering angle θst from the steering angle sensor 14, an accelerator opening θacc from the accelerator opening sensor 15, and the like are input.

  For example, when an execution of an ACC (Adaptive Cruise Control) function which is one of the functions of the driving support control is instructed through the operation of the main switch 13 by the driver, the control unit 5 recognizes the stereo image recognition device 4. A three-dimensional object such as a vehicle is read and whether or not a preceding vehicle to be followed is traveling on the traveling path of the vehicle is identified.

  As a result, when the preceding vehicle to be tracked is not detected, the constant speed traveling control for maintaining the vehicle speed V of the host vehicle 1 at the set vehicle speed set by the driver through the opening / closing control (engine output control) of the throttle valve 16. Execute.

  On the other hand, when a preceding vehicle that is a tracking target vehicle is detected and the vehicle speed of the preceding vehicle is equal to or lower than the set vehicle speed, the following traveling control is performed in which the following distance is converged to the target inter-vehicle distance. Executed. During this follow-up running control, the control unit 5 basically converges the inter-vehicle distance to the target inter-vehicle distance through the opening / closing control of the throttle valve 16 (engine output control). Further, when it is determined that sufficient deceleration cannot be obtained only by controlling the throttle valve 16 due to sudden deceleration of the preceding vehicle, the control unit 5 controls the output hydraulic pressure from the active booster 17 (automatic brake intervention control). ) To converge the inter-vehicle distance to the target inter-vehicle distance.

  Further, when execution of the lane departure prevention function, which is one of the functions of the driving support control, is instructed through the operation of the main switch 13 by the driver, the control unit 5 will, for example, the left and right white lines that define the own vehicle traveling lane. An alarm determination line is set based on (approximate white line recognized by the stereo image recognition device 4), and the own vehicle travel path is estimated. Then, for example, when the control unit 5 determines that the host vehicle travel route crosses either the left or right alarm determination line within a set distance (for example, 10 to 16 [m]) ahead of the host vehicle, It is determined that the host vehicle 1 is likely to depart from the current host vehicle lane, and a lane departure warning is issued.

  Next, the white line recognition executed by the stereo image recognition device 4 will be described in detail according to the white line recognition routine shown in FIGS. This routine is repeatedly executed every set time. When the routine starts, the stereo image recognition device 4 first performs edge processing from an image (for example, a reference image) captured by the stereo camera 3 in step S101. A point P is detected. That is, the stereo image recognition device 4 changes the luminance from the inner side to the outer side in the vehicle width direction for each search line L (see FIGS. 6 and 8) in the white line detection area A (Al, Ar) set on the image. , And search for an edge point P where the luminance of the pixel on the outer side in the vehicle width direction is relatively higher than the luminance of the inner pixel and the differential value of the luminance indicating the amount of change is equal to or greater than the set threshold value on the positive side. (See FIG. 10). Then, when a single edge point P exists in the white line detection area A, the stereo image recognition device 4 detects the edge point P as the first edge point P1. Alternatively, when there are two or more edge points P in the white line detection area A, for example, the first edge point P is detected as the first edge point P1, and the next edge point P is detected as the second edge point P2. To do. Furthermore, the stereo image recognition device 4 converts the coordinates of the edge points P1 and P2 into coordinates in the real space by giving distance information on the distance image to the detected edge points P1 and P2 (FIG. 7). , 9). In the present embodiment, the white line detection area A set on the image is set in the process of step S115 or step S133 described later.

  In subsequent step S102, the stereo image recognition device 4 extracts edge points P1 and P2 that exist in pairs on the same search line L in the left and right white line detection areas Al and Ar. For example, these edge points P1 , P2 is calculated as an offset amount W_12 (W_12l, W_12r).

In subsequent step S103, the stereo image recognition device 4 estimates the rear white line candidate point Ps_pre based on the coordinates of the white line candidate point Ps extracted before the previous frame and the movement amount of the host vehicle 1. Here, the movement amount of the host vehicle 1 can be obtained by using, for example, the host vehicle speed V and the yaw angle θ (an angle calculated from the yaw rate (dθ / dt)). For example, as shown in FIG. 11, assuming that the frame rate is Δt, the movement amounts Δx and Δz that the vehicle moves in the X-axis direction and the Z-axis direction in one frame are expressed by the following equations (1) and (2). It is calculated by.
Δx = V · Δt · sin θ (1)
Δz = V · Δt · cos θ (2)
Therefore, for example, as shown in FIG. 12, the coordinates of the white line candidate point Ps_old detected in the previous frame are set to (xold, zold), and the white line candidate point (rearward) estimated to have moved in the current frame The coordinates (xpre, zpre) of the white line candidate point) Ps_pre are obtained by the following equations (3) and (4). That is, the coordinates (xpre, zpre) are obtained by subtracting the vehicle movement amounts Δx, Δz from the coordinates (xold, zold) and then converting the coordinates to the vehicle fixed coordinate system (X ′, Z ′) in the current frame. Required by doing.
xpre = (Xold · Δx) · cos θ− (zold−Δz) · sin θ (3)
zpre = (xold · Δx) · sin θ + (zold−Δz) · cos θ (4)
Thereby, for example, as shown in FIG. 13, the rear white line candidate point Ps_pre within the set distance behind the host vehicle is estimated.

  When the process proceeds from step S103 to step S104, the stereo image recognition device 4 checks whether the white line state is determined to be a double white line as the state of the white line of interest (the right white line or the left white line). It is checked whether or not a single white line transition flag F, which will be described later, is set to “1”. Here, in this embodiment, the process after step S104 is performed separately with respect to the left and right white lines, for example. In the following description, FIGS. 16 to 22 will be described with reference to the processing for the left white line.

  If it is determined in step S104 that the white line state is not a double white line and it is determined that the single white line transition flag F is cleared to “0”, the stereo image recognition device 4 determines whether the white line state is cleared to “0”. Then, the calculation of the Hough straight line H using the point group of the first edge point P1 is performed. That is, for example, as shown in FIG. 14, the stereo image recognition device 4 increases the slope of the straight line h passing through the first edge point P1 from 0 ° with respect to each first edge point P1 constituting the point group. The distance from the origin O to the straight line h at each θ (perpendicular length) ρ is obtained based on the following equation (5) by changing the angle up to 180 ° for each predetermined angle Δθ.

ρ = x · cos θ + z · sin θ (5)
Then, the stereo image recognition device 4 votes (projects) the relationship between each θ and ρ obtained for each point P1 as a frequency on a corresponding location on the Hough plane (θ, ρ) shown in FIG. Further, the stereo image recognition device 4 extracts a combination of θ and ρ that has the largest frequency on the Hough plane (θ, ρ), and uses the θ and ρ to define a straight line (Hough) defined by the equation (5). A straight line H) is obtained.

  When the process proceeds from step S105 to step S106, the stereo image recognition device 4 sets a white line candidate point extraction area Ad having a predetermined width with the calculated Hough straight line H as a reference (see FIG. 16A). Here, the white line candidate point extraction region Ad is configured by, for example, a region in which a general lane line width is added to both sides of the Hough straight line H.

  When the process proceeds from step S106 to step S107, the stereo image recognition device 4 extracts the first and second edge points P1 and P2 existing in the white line candidate point extraction area Ad as white line candidate points Ps (FIG. 16B). In step S108, the number N1 of white line candidate points Ps (hereinafter referred to as white line candidate point number N1) is calculated.

  When the process proceeds from step S108 to step S109, the stereo image recognition device 4 determines the number N0 of edge points P existing in the vehicle width direction inner side than the white line candidate point extraction area Ad (hereinafter referred to as the number of inner edge points N0). After the calculation, the process proceeds to step S110.

  Here, the processing from step S110 to step S112 is for determining the transition from the single white line to the double white line. When the process proceeds from step S109 to step S110, the stereo image recognition device 4 calculates the current frame. It is checked whether or not the difference between the number N1_n of white line candidate points and the number N0_n−1 of inner edge points calculated in the previous frame is equal to or larger than a preset threshold value Nth.

  That is, for example, as shown in FIG. 17A, immediately before the switching to the double white line (the entrance of the double white line) is determined, the number of the first edge points P1 distributed on the auxiliary lane line is determined. Since the number of the first edge points P1 distributed on the lane line is large, the Hough straight line H is set along the lane line. At this time, basically, the number of first edge points P1 distributed on the auxiliary lane marking is calculated as the number N0 of inner edge points. On the other hand, as shown in FIG. 17B, at the timing when switching to the double white line (double white line entrance) is determined, the number of first edge points P1 distributed on the auxiliary lane line and the lane line Since the number of the first edge points P1 distributed on the line is reversed, the Hough straight line H is set along the auxiliary partition line. At this time, basically, the number of first edge points P1 distributed on the auxiliary lane markings is calculated as the number N1 of white line candidate points. The number of white line candidate points N1 calculated at this timing is generally larger than the inner edge point number N0 calculated in the previous frame by a predetermined number or more.

  Therefore, when it is determined in step S110 that the difference between the number N1_n of white line candidates and the number N0_n-1 of inner edge points is smaller than the threshold value Nth, the stereo image recognition device 4 switches the white line state from a single white line to a double white line. The process proceeds to step S115.

  On the other hand, when it is determined in step S110 that the difference between the number N1_n of white line candidate points and the number N0_n-1 of inner edge points is equal to or greater than the threshold Nth (for example, see FIGS. 17A and 17B), the stereo image recognition device. 4 determines that there is a high possibility that the white line state is switched from the single white line to the double white line, and the process proceeds to step S111.

  When the process proceeds from step S110 to step S111, the stereo image recognition device 4 determines the difference between the X coordinate of the Hough straight line H in the previous frame and the X coordinate of the Hough straight line H in the current frame at the vehicle front distances z1 and z2. That is, it is checked whether or not the movement amounts ΔX (z1) and ΔX (z2) in the vehicle width direction are within a preset threshold value ΔXth.

  That is, as described above, at the timing when the white line state is switched from the single white line to the double white line, the Hough straight line H may be substantially translated from the state along the lane line to the state along the auxiliary lane line. is assumed. In this case, the movement amount in the vehicle width direction between the frames of the Hough straight line H is within a predetermined range (for example, within a range obtained by adding the width of the auxiliary lane line to the interval between the lane line and the auxiliary lane line). It is common to stop.

  Therefore, when it is determined in step S111 that the movement amounts ΔX (z1) and ΔX (z2) in the vehicle width direction of the Hough straight line H are not within the threshold value ΔXth, the stereo image recognition device 4 determines that the Hough straight line H is in the lane. It is determined that there is a high possibility of movement due to factors other than the shift from the lane line to the auxiliary lane line (for example, the influence of noise or the like), and the process proceeds to step S115.

  On the other hand, if it is determined in step S111 that the movement amounts ΔX (z1) and ΔX (z2) in the vehicle width direction of the Hough straight line H are within the threshold value ΔXth, the stereo image recognition device 4 proceeds to step S112, It is examined whether or not the absolute value of the difference between the movement amounts Δx (z1) and Δx (z2) in the vehicle width direction of the Hough straight line H at the vehicle front distances z1 and z2 is equal to or less than a preset threshold value Xth.

  That is, as described above, at the timing when the white line state is switched from the single white line to the double white line, the Hough straight line H may be substantially translated from the state along the lane line to the state along the auxiliary lane line. is assumed. In this case, the difference between the movement amounts Δx (z1) and Δx (z2) is generally a very small value.

  Therefore, when it is determined in step S112 that the absolute value of the difference between the movement amounts ΔX (z1) and ΔX (z2) in the vehicle width direction of the Hough straight line H is not within the threshold value ΔXth, the stereo image recognition device 4 It is determined that there is a high possibility that the straight line H has moved due to other factors (for example, the influence of noise or the like) other than the shift from the lane line to the auxiliary lane line side, and the process proceeds to step S115.

  On the other hand, if it is determined in step S112 that the absolute value of the difference between the movement amounts ΔX (z1) and ΔX (z2) in the vehicle width direction of the Hough straight line H is within the threshold value ΔXth, the stereo image recognition device 4 In S113, it is determined that the white line state has shifted from a single white line to a double white line.

  In step S114, the stereo image recognition device 4 shifts the rear white line candidate point Ps_pre estimated in step S103 inward in the vehicle width direction by the offset amount W_12 calculated in step S102 (FIG. 18 ( a) and (b)), the process proceeds to step S115.

When the process proceeds from step S110, step S111, step S112, or step S114 to step S115, the stereo image recognition device 4 uses the least square method based on the point group of the white line candidate point Ps and the back white line candidate point Ps_pre. After calculating a white line approximation line (white line approximation formula: X = AZ ² + BZ + C) (see FIG. 19), and further setting a white line detection area A having a preset width ΔE on the left and right sides of this white line approximation line, Exit the routine.

  If it is determined in step S104 that the white line state is a double white line, or if it is determined that the single white line transition flag F is set to “1”, the stereo image recognition device 4 Advances to step S116, and a search line in which the first and second edge points P1 and P2 are detected on the same search line L is extracted.

  In step S117, the stereo image recognition device 4 extracts the first edge point P1 existing on the search line L extracted in step S116 as the white line candidate point Ps.

  Further, when the process proceeds from step S117 to step S118, the stereo image recognition device 4 calculates the distance between the left and right lane lines (lane line width W_out) based on the edge point P on the search line L extracted in step S116. To do. The lane line width W_out is obtained, for example, by calculating an average value (edge point interval W_P1) of the intervals between the first edge points P1 that exist in left and right pairs on the extracted search line L. It is possible to calculate by adding the left and right offset amounts W_12 calculated in step S102. Alternatively, the lane line width W_out is obtained, for example, by calculating an average value (edge point interval W_P2) of the second edge points P1 that exist in left and right pairs on the extracted search line L, and this edge point interval W_P2 Can be set as the lane line width W_out as it is.

  When the process proceeds from step S118 to step S119, the stereo image recognition device 4 has not yet processed the search line L for only the first edge point P1 that has not been extracted in step S116, with respect to the processes in steps S120 to S123 described later. It is checked whether or not the search line L exists.

  If it is determined in step S119 that there is no unprocessed search line L for only the first edge point P1, the stereo image recognition device 4 proceeds to step S124.

  On the other hand, if it is determined in step S119 that there is an unprocessed search line L with only the first edge point P1, the stereo image recognition device 4 proceeds to step S120 and selects a new unprocessed search line L.

  In subsequent step S121, the stereo image recognition device 4 calculates the interval (edge point interval W_P1) between the first edge points P1 that exist in a left-right pair on the unprocessed search line L selected in step S120.

  Then, when the process proceeds from step S121 to step S122, the stereo image recognition device 4 matches the edge point interval W_P1 calculated in step S121 with the lane marking line width W_out calculated in step S118 within a predetermined error 2α. (That is, whether or not W_out−α ≦ W_P1 ≦ W_out + α).

  If it is determined in step S122 that the edge point interval W_P1 does not match the lane marking line width W_out within a predetermined error range, the stereo image recognition device 4 directly returns to step S119.

  On the other hand, when it is determined in step S122 that the edge point interval W_P1 matches the lane marking W_out within a predetermined error range (see FIG. 20A), the stereo image recognition device 4 proceeds to step S123, One edge point P1 is extracted as a white line candidate point Ps, and the extracted white line candidate point Ps is shifted inward in the vehicle width direction by the offset amount W_12 calculated in step S102, and then the process returns to step S119.

  When the process proceeds from step S119 to step S124, the stereo image recognition apparatus 4 sets the single white line transition flag F indicating that the white line recognition control is shifting from the one for the double white line to the one for the single white line is “1”. Check if it is set to.

  If it is determined in step S124 that the single white line transition flag F is set to “1”, the stereo image recognition device 4 proceeds to step S128.

  On the other hand, if it is determined in step S124 that the single white line transition flag F is not set to “1” (that is, if it is determined that the single white line transition flag F is cleared to “0”), stereo. In step S125, the image recognition apparatus 4 sets the number of white line candidate points Ps shifted inward in the vehicle width direction in the processes of steps S119 to S123 with respect to the total number of white line candidate points Ps currently extracted. Check if the percentage is over.

  If it is determined in step S125 that the number of white line candidate points Ps shifted inward in the vehicle width direction is less than the set ratio, the stereo image recognition device 4 proceeds to step S133.

  On the other hand, if it is determined in step S125 that the number of white line candidate points Ps shifted inward in the vehicle width direction is equal to or greater than the set ratio, the stereo image recognition device 4 changes the white line state from a double white line to a single white line. It is determined that the switching has been performed, and the process proceeds to step S125, where the single white line transition flag F is set to “1”.

  Then, when the process proceeds from step S126 to step S127, the stereo image recognition device 4 calculates the shift amount ΔW per frame by dividing the offset amount W_12 calculated in step S102 by the preset number of transition frames C1. At the same time, after the counter C indicating the elapsed time (number of frames) since the single white line transition flag F is set is cleared to “0”, the process proceeds to step S128.

  Then, when the process proceeds from step S124 or step S127 to step S128, the stereo image recognition device 4 increments the counter C (C ← C + 1), and then proceeds to step S129 to move the white line candidate point Ps outward in the vehicle width direction. The shift is performed by the shift amount (ΔW × C) (see FIG. 21B and FIG. 22), and in the subsequent step S130, the rear white line candidate point Ps_pre is shifted by the shift amount ΔW outward in the vehicle width direction (FIG. 21 (b), see FIG.

  Then, when the process proceeds from step S130 to step S131, the stereo image recognition device 4 checks whether or not the counter C has reached the number of transition frames C1, and if it is determined that the number of transition frames has not been reached, step S133 Proceed to

  On the other hand, if it is determined in step S131 that the counter C has reached the number of transition frames C1, the stereo image recognition device 4 proceeds to step S132, clears the single white line transition flag F to “0”, and then proceeds to step S132. The process proceeds to S133.

Then, when the process proceeds from step S125, step S131, or step S132 to step S133, the stereo image recognition device 4 uses the least-squares method based on the point group of the white line candidate point Ps and the back white line candidate point Ps_pre. An approximate line (white line approximation formula: X = AZ ² + BZ + C) is calculated (see FIG. 19), and a white line detection region A having a preset width ΔE is set on both the left and right sides of the white line approximate line, and then a routine is executed. Exit.

  According to such an embodiment, not only the point group of white line candidate points Ps is extracted from the edge points (first and second edge points P1, P2) detected in the current frame, but also extracted before the previous frame. Based on the coordinates of the white line candidate point Ps_old and the movement amount (Δx, Δz) of the own vehicle, the rear white line candidate point Ps_pre existing behind the own vehicle is estimated, and the point of the white line candidate point Ps and the rear white line candidate point Ps_pre By performing white line approximation based on the group, in the unlikely event that noise or the like is erroneously extracted as the white line candidate point Ps, the white line that divides the vehicle traveling path is greatly affected by the noise or the like. It can be recognized stably and accurately.

  On that basis, the white line that divides the vehicle traveling path based on the point group of the first and second edge points P1 and P2 is a single white line composed of a single lane line, or the lane line It is determined whether it is a double white line with an auxiliary lane line inside, and when it is determined that the white line is a double white line, a white line candidate point that defines the auxiliary lane line is extracted from the edge point group, When it is determined that the white line is switched from the double white line to the single white line, the offset calculated based on the first and second edge points P1 and P2 detected as a pair on the same search line By shifting the white line candidate point Ps and the rear white line candidate point Ps_pre outward in the vehicle width direction by the amount W_12, the road shape can be recognized with high responsiveness without deteriorating the recognition accuracy even at the exit of the double white line. . That is, by shifting the white line candidate point Ps and the rear white line candidate point Ps_pre to the outside in the vehicle width direction, the white line candidate point Ps and each of the rear line candidate points Ps_pre can be changed even during a transition in which the white line switches from a double white line to a single white line. Continuity can be ensured, and the road shape can be recognized with good responsiveness without lowering the recognition accuracy.

  Further, when it is determined that the white line is a single white line, a Hough straight line H is calculated based on the point group of the first edge points P1, and the white line candidate point extraction area Ad determined based on the Hough straight line H is calculated. Edge points P1 and P2 are extracted as white line candidate points Ps, and when it is determined that the white line is switched from a single white line to a double white line, the first detected as a pair on the same search line. , The rear white line candidate point Ps_pre is shifted inward in the vehicle width direction by the offset amount W_12 calculated based on the second edge points P1 and P2, thereby reducing the recognition accuracy even at the entrance of the double white line. The shape can be recognized with good responsiveness. That is, by shifting the rear white line candidate point Ps_pre inward in the vehicle width direction, the continuity of the white line candidate point Ps and the rear white line candidate point Ps_pre is ensured even during a transition in which the white line switches from a single white line to a double white line. The road shape can be recognized with good responsiveness without degrading the recognition accuracy.

  The present invention is not limited to the embodiment described above, and various modifications and changes are possible, and these are also within the technical scope of the present invention.

1 ... Vehicle (own vehicle)
2 ... Driving assistance device 3 ... Stereo camera 4 ... Stereo image recognition device (edge point detection means, offset amount calculation means, white line state determination means, white line candidate point extraction means, rear white line candidate point estimation means, white line approximation means, candidate point Shift means, lane line width calculation means)
DESCRIPTION OF SYMBOLS 5 ... Control unit 11 ... Vehicle speed sensor 12 ... Yaw rate sensor 13 ... Main switch 14 ... Steering angle sensor 15 ... Accelerator opening sensor 16 ... Throttle valve 17 ... Active booster

Claims (4)

The brightness change from the inside to the outside of the vehicle width is examined for each of the plurality of search lines extending in the horizontal direction in the left and right white line detection areas on the image taken in front of the vehicle traveling path, and the brightness is changed from dark to bright. Edge point detecting means for sequentially detecting the first edge point as the first edge point and the next edge point as the second edge point when there is an edge point that changes by a predetermined amount or more;
The first in the same of the search on the line, when the second edge points are detected in pairs, when the white line is a double white line based on the first interval of the second edge point An offset amount calculating means for calculating an offset amount between the lane line and the auxiliary lane line ;
The white line that divides the vehicle traveling path based on the point cloud of the first and second edge points is a single white line composed of a single lane line, or an auxiliary lane line inside the lane line A white line state determining means for determining whether or not a double white line is attached;
A white line that extracts a white line candidate point that defines the auxiliary division line from the point group of the edge points detected by the edge point detection unit when the white line state determination unit determines that the white line is a double white line Candidate point extraction means;
Back white line candidate point estimating means for estimating a back white line candidate point within a set distance behind the host vehicle based on the coordinates of the white line candidate point extracted before the previous frame and the movement amount of the host vehicle;
White line approximation means for approximating a white line based on a point group of the white line candidate point and the back white line candidate point;
Candidate point shift for shifting the white line candidate point and the rear white line candidate point to the outside in the vehicle width direction by the offset amount when it is determined by the white line state determining means that the white line is switched from a double white line to a single white line. And a vehicle white line recognizing device.   The candidate point shift means, when the white line state determination means determines that the white line has switched from a double white line to a single white line, shifts the white line candidate point and the rear white line candidate point over a set frame. The white line recognition device for a vehicle according to claim 1. When the white line state determination means determines that the white line is a double white line, the distance between the left and right lane markings based on the edge point on the search line where the second edge point is detected A lane line width calculating means for calculating a lane line width indicating
The white line candidate point extracting means, when the white line state determining means determines that the white line is a double white line, the first edge point on the search line from which the second edge point is detected Are extracted as white line candidate points, and the interval between the first edge points forming a left-right pair on the search line where the second edge point is not detected matches the lane line width within a set error range 3. The vehicle white line recognition device according to claim 1, wherein the first edge point is extracted as a white line candidate point and shifted inward in the vehicle width direction by the offset amount.   When the white line state determination unit determines that the white line is a double white line, among the white line candidate points extracted by the white line candidate point extraction unit, the white line candidate points shifted by the offset amount 4. The vehicle white line recognition device according to claim 3, wherein when the occupying ratio is equal to or greater than a set ratio, it is determined that the white line is switched from a double white line to a single white line.

Images