JP2013120458A - Road shape estimating device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely estimate a road shape from a photographed image of a road surface, without being affected by auxiliary lines due to degradation in resolution, even in the case of a multiple-lane road having the auxiliary lines painted in parallel with lane division lines.SOLUTION: A road shape estimating device includes: an edge detecting section 911 for detecting a plurality of edge lines from a photographed image of a road surface of a road to generate composite edge lines by combining one or more edge lines out of the detected edge lines; a road shape estimating section 912 for obtaining road shape parameters of the road, including a crosswise offset of a camera having photographed the road surface, and a road width, on the basis of information on the inner side composite edge lines that are closest to a center of the road; a correction amount estimating section 914 for obtaining a correction amount on the basis of a distance between the inner side composite edge lines and the outer side composite edge lines positioned on the outer side of the inner side composite edge lines, and storing the correction amount in a storage section 915; and a correcting section 913 for correcting the crosswise offset and the road width by employing the correction amount.

Description

  The present invention relates to a road shape estimation device and a program.

  Mount the camera in the direction of travel of the vehicle or in the direction opposite to the direction of travel, detect the lane markings included in the captured image, determine the positional relationship between the driving lane and the vehicle from the lane marking information, and determine the road shape The estimation method has been researched and developed for a long time.

  For example, from a road surface image taken in front of a vehicle photographed by a camera, images are scanned in a plurality of places in the vertical direction in the horizontal direction, the edges of the lane markings are detected as luminance (shading) change points, and the vehicle travels based on the edge information. A method for estimating the positional relationship between a lane and a vehicle is known. In this method, the lane markings are identified by connecting the edge points detected from the image in the vertical direction, and the position of the lane markings on the road surface is determined by comparing the road shape model with the position of the lane markings. The

  On the road surface of the curved road, an auxiliary line (deceleration auxiliary marker) for promoting deceleration may be painted in parallel with the lane line. As this auxiliary line, for example, a row of broken line blocks shorter than the broken line used for the lane line is used.

  When edges of multiple lane marking candidates are detected by edge detection from the left or right side of the traveling lane in the road image of such a multiple track, the edge period is compared and the candidate closest to the continuous line is determined. A method for determining a lane marking is known. In this method, even if a lane division line and an auxiliary line are painted on the lane in which the vehicle is traveling, the auxiliary line is recognized as a discontinuous line consisting of the edges of a short broken line block. By selecting the candidate closest to the continuous line among the line candidates, the original lane marking can be recognized.

JP 2004-310522 A

Shin Takahashi and Yoshiki Ninomiya, “A Robust Lane Marking Candidate Selection Method for Driving Lane Recognition”, IEICE Transactions D-II Vol. J81-D-II No. 8 pp. 1904-1911, August 1998

The conventional lane marking recognition technology described above has the following problems.
In the conventional lane marking recognition technology, as shown in FIG. 1, the lane markings 101 and 102 and the auxiliary are shown in all areas from the vicinity of the vehicle (the lower side of the image) to the far side (the upper side of the image). It is necessary that lines 103 and 104 can be distinguished. Further, it is necessary that auxiliary lines adjacent in the vertical direction are separated in the image.

  However, because the resolution of the road surface image is limited, the resolution of an object reflected in the distance may be reduced, and in the distance, lane markings and auxiliary lines may be merged or auxiliary lines adjacent in the vertical direction may be merged. .

  FIG. 2 shows a first example in which lane markings and auxiliary lines are fused. In the first example, the lane markings 201 and 202 are solid lines, and the auxiliary lines 203 and 204 are broken lines. However, at a distance, due to a decrease in resolution, the lane markings 201 and 203 are merged and the lane markings 202. And the auxiliary line 204 are fused. Furthermore, fusion of broken line blocks of the auxiliary line 203 and fusion of broken line blocks of the auxiliary line 204 also occur. In this case, since the candidates closest to the continuous line are the edge lines 205 and 206 of the merged block, the road shape is estimated based on the edge lines having these erroneous shape characteristics.

  FIG. 3 shows a second example in which lane markings and auxiliary lines are fused. In the second example, the lane markings 301 and 302 are broken lines, and the auxiliary lines 303 and 304 are also broken lines, but the auxiliary lines 303 and 304 have shorter broken line lengths. In the distance, due to a decrease in resolution, the lane division line 301 and the auxiliary line 303 are merged and the lane division line 302 and the auxiliary line 304 are merged. The broken line blocks of the auxiliary line 303 and the broken line blocks of the auxiliary line 304 are merged. Has also occurred. In this case, since the candidate closest to the continuous line is the edge lines 305 and 306 of the merged block, as in the case of FIG. 2, the road shape is estimated based on the edge line having an incorrect shape characteristic. become.

  The problem of the present invention is that an image obtained by photographing the road surface of a road without being affected by the auxiliary line due to a decrease in resolution, even if the auxiliary line is painted in parallel with the lane marking. It is to estimate the road shape with high accuracy.

The road shape estimation device includes an edge detection unit, a road shape estimation unit, a correction amount estimation unit, a storage unit, and a correction unit.
The edge detection unit detects a plurality of edge lines from an image obtained by photographing a road surface of a road, and generates a composite edge line by connecting one or more edge lines among the edge lines. The road shape estimation unit obtains road shape parameters of the road including the lateral offset of the camera that captured the road surface and the road width based on the information of the inner composite edge line closest to the center of the road.

  The correction amount estimation unit obtains a correction amount based on the distance between the inner composite edge line and the outer composite edge line located outside the inner composite edge line, and the storage unit stores the correction amount. The correction unit corrects the lateral offset and the road width using the correction amount, and outputs a road shape parameter including the corrected lateral offset and the road width.

  According to the road shape estimation device described above, even in a multiple track where an auxiliary line is painted in parallel with the lane marking, the road surface of the road is not affected by the auxiliary line due to a decrease in resolution. The road shape can be accurately estimated from the captured image.

It is a figure which shows a lane division line and an auxiliary line. It is a figure which shows the 1st example which fusion of a lane division line and an auxiliary line arises. It is a figure which shows the 2nd example which fusion of a lane division line and an auxiliary line arises. It is a figure which shows a road shape model. It is a figure which shows the observation system model of a camera. It is a figure which shows the inner boundary line of an auxiliary line. It is a figure which shows the shift amount. It is a figure which shows the correction method of road width and offset. It is a functional block diagram of a 1st road shape estimation apparatus. It is a flowchart of a 1st road shape estimation process. It is a functional block diagram of the 2nd road shape estimation apparatus. It is a flowchart of a 2nd road shape estimation process. It is a flowchart of an edge detection process. It is a figure which shows a digital image. It is a figure which shows an edge point. It is a figure which shows an edge line. It is a figure which shows the area | region on either side containing a lane marking. It is a figure which shows an inner edge line. It is a flowchart (the 1) of a composite edge line production | generation process. It is a flowchart (the 2) of a composite edge line production | generation process. It is a figure which shows a composite edge line. It is a flowchart of a candidate line determination process. It is a flowchart of an auxiliary line determination process. It is a flowchart (the 1) of a correction amount estimation process. It is a flowchart (the 2) of a correction amount estimation process. It is a block diagram of a lane departure warning system. It is a block diagram of a vehicle steering system. It is a block diagram of information processing apparatus.

Hereinafter, embodiments will be described in detail with reference to the drawings.
FIG. 4 shows an example of a road shape model. Here, let us consider a model in which the radius of curvature of the lane center line Lc is R, and the road width (lane width) is always W measured in the horizontal direction (X-axis direction) at each position of the lane. In this model, the lane width in the direction orthogonal to the center line Lc is not W, so it is different from the actual lane, but the error is small in the range relatively close to the observation point by the camera, and analytical handling is easy. Become.

  As shown in FIG. 5, the center of the camera is the origin O, the X axis of the road surface coordinate system is taken from the origin O in the horizontal direction, and the Z axis of the road surface coordinate system is taken in the vertical direction. In this case, the direction perpendicular to the XZ plane is the Y axis of the road surface coordinates. A point where the origin O is vertically projected on the road surface is defined as RO. The meaning of each symbol in the road shape model of FIG. 4 is as follows.

Lc: lane center line Lr: lane line on the right side of lane Ll: lane line on the left side of lane W: road width R: radius of curvature e: distance from point RO to center line Lc P: lane line Lr or Point of interest on L1 A: Intersection of lane marking Lr and XY plane B: Point of interest point P projected perpendicularly to XY plane C: Center of curvature D: Distance from XY plane to point of interest P

  The distance e represents the distance from the camera to the center of the road (the lateral offset of the camera), and the distance D represents the distance in the depth direction from the camera to the point of interest P. At this time, the X coordinate Px and the Z coordinate Pz of the point of interest P (X, Z) are given by the following equations.

  Formula (1) is a basic formula that gives the position of the lane on the road surface. However, k is a constant for distinguishing the lane markings Lr and Ll. When the point of interest P is on the right lane marking Lr, k = 1, and when it is on the left lane marking Ll, k = -1. The radius of curvature R is a positive value when the right curve, that is, the center of curvature C is in the positive section of the X axis, and is a negative value when the left curve, that is, the center of curvature C is in the negative section of the X axis.

  Since the basic formula (1) includes a square root, it is difficult to handle analytically. Therefore, the basic equation (1) is approximated assuming a normal traveling time in which the radius of curvature R is not extremely small. For example, if up to the first term of the Taylor expansion is used as the first order approximation of the square root, the following equation is obtained.

  Using equation (2), basic equation (1) is rewritten as follows.

  Equation (3) is the basic equation in the first order approximation.

  Next, a model of the position of the lane marking when the lane marking on the road surface is observed with a camera is derived.

  FIG. 5 shows an example of a camera observation system model. The center of the camera is located at a height H from the road surface, and the camera is installed at a pitch angle φ, a yaw angle θ, and a roll angle 0 with respect to the traveling direction of the vehicle. The pitch angle φ and the yaw angle θ take the transformation of the coordinate system from the camera coordinate system to the road surface coordinate system in the positive rotation direction. Therefore, from the road surface coordinate system to the camera coordinate system, the transformation of the coordinate points becomes the positive direction of rotation. The focal length of the camera is f. At this time, coordinate conversion from road surface coordinates (X, Y, Z) to camera coordinates (xc, yc, zc) is considered.

  When the coordinates (X, Y, Z) of the point P in the road surface coordinate system are expressed by the coordinates (xc, yc, zc) in the camera coordinate system having the pitch angle φ and the yaw angle θ, the following expression is obtained.

  If this point is taken from the center of the image observed by the camera, the x-axis is directed to the right, the y-axis is directed downward, and the image coordinates are (x, y), the image coordinates (px, py) of the point of interest P are as follows: Given.

  Substituting (Px, Pz) in the approximate basic equation (3) into X and Z in Equation (5) and substituting H into Y, the following equation is obtained as the positional relationship of the lane markings on the image.

  If the parameter D is removed from the equation (6), a relational expression between px and py is obtained. Actually, it is difficult to derive a relational expression because D cannot be easily solved in the form of Expression (6). Therefore, approximation of equation (6) is performed in consideration of the order of each term in equation (6).

  Specifically, each rotation angle φ and θ is about 20 degrees at most, and is not so large, so the value of the sin system is about 0.1. Assuming that the unit of D, R, W, H, and e is meters, in the denominator of px and py in equation (6), the value multiplied by cos (φ) sin (θ) of the first term is about 2 Therefore, the first term has a value of about 0.2. Also, since the value of H in the second term is about 2, the second term also has a value of about 0.2. On the other hand, the value of D in the third term is about 30, and since the third term is a cosine product, it has a value of about 30. Therefore, in the denominator, the third term is dominant and the first and second terms have little influence. The same applies to the py molecule. Therefore, the following equation is obtained as the first approximate equation.

  When D is obtained from the py expression of Expression (7), is substituted into the px expression, and (px, py) is rewritten as (x, y), the following expression is obtained.

  Expression (8) is a relational expression between the x-coordinate and the y-coordinate of the point on the lane marking line in the image, and is a basic expression in the image coordinate system. Here, assuming that the rotation angles φ and θ are very small and sin φ = φ, cos φ = 1, sin θ = θ, cos θ = 1, the following equation is obtained.

  Expression (9) is a final approximation expression representing the road shape model. The road curvature ρ in equation (9) corresponds to the reciprocal of the curvature radius R. It is also possible to use another parameter representing the relative distance in the lateral direction between the camera and the reference position on the road as the lateral offset e of the camera.

  If the position (x, y) of the lane marking is detected from the image (frame) at each time in the video, the road shape parameters (ρ, W, e, φ, θ) are estimated using equation (9). Can do. Thereby, the road shape can be estimated by obtaining the relationship between the shape of the lane marking on the road surface and the position and direction of the vehicle.

  In the following description, the origin of the image coordinate system is translated from the center of the image to the upper left point of the image, and the position (x, y) of the lane marking is described.

When auxiliary lines are painted inside the left and right lane markings of the driving lane, the following points are focused on in order to correctly estimate the road shape formed by the lane markings without being affected by the auxiliary lines.
(1) The auxiliary line exists in parallel with the lane marking.
(2) The auxiliary line exists on the side close to the center of the road (inside the lane marking).
(3) The auxiliary lines are broken lines, and the lengths and intervals of the broken line blocks that are continuously arranged are shorter than the lengths and intervals of the broken line blocks used in the lane markings.

  Considering the above (2) and (3), even if the lane markings and auxiliary lines merge, or even if the auxiliary lines merge in the depth direction of the road, as shown in FIG. It can be seen that the boundary lines 601 to 606 are always observed stably without being damaged.

  In consideration of the above (1), when the road shape parameters are estimated using the inner boundary lines 601 to 606 of the auxiliary line, correct values are required for the road curvature ρ, the pitch angle φ, and the yaw angle θ. I understand. Therefore, in the embodiment, when there is an auxiliary line, the road shape is estimated using the inner boundary line of the auxiliary line regardless of whether or not the lane division line and the auxiliary line are merged.

  However, for the road width W and the offset e, incorrect values are obtained when the inner boundary lines 601 to 606 are regarded as road ends. The error of the road width W and the offset e is a value resulting from the distance from the position of the inner boundary line of the auxiliary line to the position of the lane line.

  Next, we consider the fusion of lane markings and auxiliary lines. As shown in FIG. 7, in a region 701 where the lane markings 711 and 712 and the auxiliary lines 721 and 722 are in the distance, it is not possible to separate the fused line segments due to insufficient resolution. On the other hand, in the area 702 in which the line segments are in the vicinity of the vehicle, each line segment is shown in large detail in detail, so that the lane division lines 711 and 712 and the auxiliary lines 721 and 722 are not merged. Therefore, if the positional relationship of the detected line segments is examined focusing on the area 702 near the lower end of the image, the positional errors between the auxiliary lines 721 and 722 and the lane marking lines 711 and 712 can be obtained.

The shift amount DX _{L in} FIG. 7 represents the distance from the inner boundary line 741 of the left auxiliary line 721 to the inner boundary line 731 of the lane line 711 that is spaced outward by the width of the auxiliary line. The shift amount DX _R represents the distance from the inner boundary 742 of the right auxiliary line 722, to the inner boundary 732 of the lane marker 712 by the width of the auxiliary line spaced outwardly. As long sought shift amount DX _L and DX _R, using them to correct the error of the road width W and the offset e, we are possible to estimate the correct road shape based on the lane dividing line.

FIG. 8 shows a method for correcting the road width W and the offset e. The road width W1 before correction represents the distance between the position 801 of the inner boundary line 741 and the position 803 of the inner boundary line 742, and the offset e1 before correction is from the camera position 804 to the position 802 of the road center before correction. Represents the distance. The corrected road width W2 represents the distance between the position 811 of the inner boundary line 731 and the position 813 of the inner boundary line 732, and the corrected offset e2 is from the camera position 804 to the corrected center position 812 of the road. Represents the distance. At this time, the corrected road width W2 and offset e2 are obtained by the following equations.
W2 = W1 + (DX _L + DX _R )
e2 = e1 + (− DX _L + DX _R ) / 2 (21)

  However, when the lane line is a broken line, there is a gap with no corresponding paint between the broken line blocks. Therefore, when attention is paid to the area 702 near the lower end of the image, a plurality of auxiliary lines and lane line are always formed. A line segment does not always exist. For example, on a general highway, the length of a broken line block (white line portion) is 8 m, and the length of a gap (non-white line portion) between broken line blocks is 12 m.

  Therefore, the region 702 near the lower end of the image is always monitored to determine whether or not there are a plurality of line segments, and the positional shift between the auxiliary line and the lane division line is performed in consideration of the presence of the plurality of line segments. Calculate the amount. Further, when there are a plurality of line segments, the shift amount is detected every time, so that it is possible to cope with a change in the interval between the auxiliary line and the lane marking line.

  FIG. 9 shows a functional configuration example of the road shape estimation apparatus. The road shape estimation apparatus 901 in FIG. 9 includes an edge detection unit 911, a road shape estimation unit 912, a correction unit 913, a correction amount estimation unit 914, and a storage unit 915.

  FIG. 10 is a flowchart showing an example of road shape estimation processing by the road shape estimation apparatus 901 of FIG. First, the edge detection unit 911 detects a plurality of edge lines from an image obtained by photographing a road surface of a road (step 1001), and generates a composite edge line by connecting one or more of the edge lines. (Step 1002).

  Next, the road shape estimation unit 912 obtains road shape parameters including the road width and offset based on the information of the inner composite edge line closest to the center of the road (step 1003). In addition, the correction amount estimation unit 914 obtains a correction amount based on the distance between the inner composite edge line and the outer composite edge line located outside the inner composite edge line, and stores the correction amount in the storage unit 915 (step 915). 1004). Steps 1003 and 1004 may be performed in order or in parallel.

  Next, the correction unit 913 corrects the road width and offset using the correction amount (step 1005), and outputs a road shape parameter including the corrected road width and offset (step 1006).

  According to such a road shape estimation device, even if it is a multiple track in which an auxiliary line is painted in parallel with the lane marking, the road surface of the road is not affected by the auxiliary line due to a decrease in resolution. The road shape can be estimated with high accuracy from the image obtained by capturing the image.

  FIG. 11 shows another functional configuration example of the road shape estimation apparatus. 11 includes a video input unit 1101, an edge detection unit 1102, a candidate line determination unit 1103, a road shape estimation unit 1104, a correction unit 1105, an auxiliary line determination unit 1106, a correction amount estimation unit 1107, and a storage unit. 1108 is included. A camera 1100 connected to the video input unit 1101 is attached so that a road surface can be photographed in front of or behind the vehicle.

  FIG. 12 is a flowchart illustrating an example of road shape estimation processing by the road shape estimation apparatus of FIG. 11. First, the video input unit 1101 performs video input processing (step 1201), the edge detection unit 1102 performs edge detection processing (step 1202), and the candidate line determination unit 1103 performs candidate line determination processing (step 1203). Next, the auxiliary line determination unit 1106 performs auxiliary line determination processing (step 1204), and the correction amount estimation unit 1107 performs correction amount estimation processing (step 1205). The road shape estimation unit 1104 performs road shape estimation processing (step 1206). Then, the correction unit 1105 performs road shape correction processing (step 1207).

  In the video input process of step 1201, the video input unit 1101 receives a video shot by the external camera 1100 and outputs it to the edge detection unit 1102. At this time, if the input video is an analog video, it is converted to a digital image, and if it is a color video, it is converted to a monochrome grayscale image before being output to the edge detection unit 1102. The output digital image is, for example, two-dimensional array information having a predetermined size vertically and horizontally.

  In the edge detection process of step 1202, the edge detection unit 1102 detects edge point groups that constitute the boundary line of the region from each portion including the left and right lane marking lines in the input digital image. Next, edge points are detected by connecting edge points in the vertical direction of the image. The detected edge lines correspond to edge lines such as the whole solid lane line, one broken line block of the broken lane line, and one broken line block of the auxiliary line.

  Then, the edge lines are connected in the vertical direction of the image to generate a composite edge line, which is output to the candidate line determination unit 1103 and the correction amount estimation unit 1107. The generated composite edge line corresponds to edge lines such as the whole solid lane line, the whole broken lane line, the whole auxiliary line, and the like.

  In the candidate line determination process in step 1203, the candidate line determination unit 1103 selects the composite edge line closest to the center of the road (innermost) from the input composite edge lines for the left and right portions. The candidate line used for shape estimation is determined. Then, the composite edge line is output to the road shape estimation unit 1104, the auxiliary line determination unit 1106, and the correction amount estimation unit 1107.

  In the auxiliary line determination processing in step 1204, the auxiliary line determination unit 1106, for the composite edge line of the input candidate line, based on the length of the edge line included in the composite edge line on the road surface, Is determined to correspond to the auxiliary line. Then, the determination result is output to the correction amount estimation unit 1107.

  In the correction amount estimation processing in step 1205, the correction amount estimation unit 1107, for each of the left and right portions, when the input determination result indicates an auxiliary line, the composite edge line positioned outside the candidate line by the width of the auxiliary line Check for the presence or absence. At this time, the composite edge line input from the edge detection unit 1102 is used as a check target.

  If there is a corresponding outer composite edge line, the distance from the candidate line to the outer composite edge line is obtained and stored in the storage unit 1108 as a correction amount. As the correction amount, for example, the distance between the candidate line and the outer composite edge line in the direction orthogonal to the traveling direction on the road surface is used. On the other hand, if the corresponding outer composite edge line does not exist, the correction amount stored in the storage unit 1108 is not updated.

When the input determination result indicates that it is not an auxiliary line, 0 is stored in the storage unit 1108 as the correction amount.
For example, as shown in FIG. 3, when the broken lane line 301 and the auxiliary line 303 are merged, there is no composite edge line outside the auxiliary line 303 by the width of the auxiliary line. Will not be updated. Thereafter, when the distant lane line 301 approaches the vehicle after several frames, the image resolution is ensured and the lane line 301 and the auxiliary line 303 are separated. Desired.

  In the road shape estimation processing in step 1206, the road shape estimation unit 1104 obtains road shape parameters using the image coordinates (x, y) of the edge points included in the composite edge line of the input candidate lines, and the correction unit 1105 is output.

  In the road shape correction process in step 1207, the correction unit 1105 corrects the road width and offset of the input road shape parameters using the correction amount stored in the storage unit, and the corrected road width and offset Other road shape parameters are output to the outside. When the candidate line is an auxiliary line, the road corresponding to the actual position of the lane marking line is added by appropriately adding the correction amount corresponding to the shift amount from the auxiliary line to the lane marking line to the estimated road width and offset. Width and offset are obtained. On the other hand, when the candidate line is not an auxiliary line, since the correction amount is 0, the estimated road width and offset are not changed.

  In this way, by using the innermost composite edge line as a candidate line around each of the left and right lane marking lines, if there is no auxiliary line, the edge line of the lane marking line is extracted as a candidate line. The If an auxiliary line exists inside, an edge line of the auxiliary line is extracted as a candidate line. In either case, there is no edge line inside the candidate line, and the inner boundary of the candidate line does not merge with other line segments, so the shape of the road can be estimated stably. it can.

  However, if the candidate line is an auxiliary line, an error is included in the estimated road width and offset. Therefore, the error is corrected by adding the correction amount from the auxiliary line calculated separately to the lane marking, and always correct. The road shape can be estimated. By calculating the correction amount when the lane line and the auxiliary line are separated, that is, when the lane line exists in the vicinity of the vehicle and the resolution becomes high, the correction amount can be obtained stably.

  Therefore, even for multiple tracks where auxiliary lines are painted in parallel with the lane markings, the road shape is accurately obtained from an image of the road surface without being affected by the auxiliary lines due to a decrease in resolution. Can be estimated.

  FIG. 13 is a flowchart illustrating an example of the edge detection process of FIG. First, the edge detection unit 1102 applies a differential operator such as a Sobel operator to the input digital image to perform a differential process in the horizontal direction, and detects an edge point whose luminance change is equal to or greater than a threshold ( Step 1301).

  For example, a sequence of edge points arranged in the vertical direction as shown in FIG. 15 is obtained from a digital image as shown in FIG. In FIG. 15, the position of the detected edge point is schematically shown. In practice, differentiation processing is performed on the pixels in each row of the image in FIG. 14, and the edge point is detected from each row.

  Next, the detected edge point is subjected to a labeling process using connectivity of surrounding pixels, and an edge line is detected by integrating a plurality of edge points (step 1302). Thereby, edge lines corresponding to the outer shape of the entire solid lane line, one broken line block of the broken lane line, one broken line block of the auxiliary line, and the like are obtained. For example, an edge line as shown in FIG. 16 is detected from the edge points in FIG.

  Next, the edge line is narrowed down (step 1303). In this narrowing-down process, first, as shown in FIG. 17, left and right regions RL and RR including lane markings are provided in the digital image, and edge lines that exist outside each region are excluded.

  Next, the luminance values of the left and right pixels sandwiching each edge line in the regions RL and RR are examined. Since the edge line in the left region RL is related to the left lane line, the edge line is left when the luminance value of the left pixel is large and the luminance value of the right pixel is small, and the edge line is otherwise Is excluded. Also, since the edge line in the right region RR is related to the right lane line, the edge line is left when the luminance value of the left pixel is small and the luminance value of the right pixel is large, and otherwise. Exclude edge lines.

  As a result, only the inner edge line is left among the edge lines of the entire solid lane line, one broken line block of the broken lane line, one broken line block of the auxiliary line, and the outer edge line is Excluded. For example, when the outer edge line is excluded from the edge lines in FIG. 17, only the inner edge line is left as shown in FIG.

Next, the length of each edge line on the road surface is examined, and the line types are classified into a solid line, a broken line, an auxiliary line, and a noise line type in order from the longest, and very short edge lines classified as noise are excluded. When the image coordinates of the upper end point of the edge line are (x1, y1), the image coordinates of the lower end point are (x2, y2), and the coordinates of the image center are (cx, cy), the length LL of the edge line is Is given by:
LL = Z2-Z1
Z2 = f.H / (f..phi. + Y2-cy)
Z1 = f.H / (f..phi. + Y1-cy) (31)

However, unlike FIG. 5, (x1, y1), (x2, y2), and (cx, cy) are described with the upper left point of the image as the origin of the image coordinate system. Line types can be classified as follows using the LL of the formula (31) and the threshold values ThL1 to ThL3.
(1) LL is greater than or equal to ThL1: solid line lane marking (2) LL is less than ThL1 and greater than or equal to ThL2: dashed lane segmentation line (3) LL is less than ThL2 and greater than or equal to ThL3: dashed auxiliary line (4) LL is equal to ThL3 Less than: Noise

Here, it is assumed that the length of the broken line block of the auxiliary line is shorter than the length of the broken line block of the lane marking line. Exclude sorted edge line noise, remaining NL L number of edge lines in the left region RL of the edge ^line, the number of edge line in the right region RR and NL ^R. Then, the edge line in the region RL is described as L ^L _i (i = 1, 2,..., NL ^L ), and the edge line in the region RR is described as L ^R _i (i = 1, 2,. , NL ^R ).

  Next, the edge lines are connected in the vertical direction of the image to generate a composite edge line (step 1304). In this composite edge line generation process, all combinations of edge lines are examined for each of the regions RL and RR. If both the distance in the X direction and the distance in the Z direction between the end points on the near side of the two edge lines are within the threshold value, the edge lines are integrated.

19 and 20 are flowcharts illustrating an example of the composite edge line generation process in the left region RL. First, the edge detection unit 1102 creates a list F indicating processed edge lines {0, 0,. . . , 0} (step 1901). The number of elements in the list F is NL ^L , and when the i-th element F [i] is 1, this indicates that the processing of the i-th edge line has ended.

Next, 0 is set to the variable i indicating the i-th edge line in the region RL (step 1902), and 0 is set to the variable m indicating the number of generated composite edge lines (step 1903). Then, i is incremented by 1 (step 1904), and i is compared with NL ^L (step 1905).

If i is equal to or less than NL ^L (step 1905, NO), then the value of F [i] is checked (step 1906). If F [i] is 1 (step 1906, YES), the processing after step 1904 is repeated. If F [i] is not 1 (step 1906, NO), 1 is set to F [i] (step 1907), and m is incremented by 1 (step 1908).

Next, the edge line L ^L _i is added to the integrated list CL ^L _m representing the set of edge lines included in the mth composite edge line (step 1909). Then, the line type of the edge line L ^L _i is set as the line type T ^L _m of the m-th composite edge line (step 1910).

Then, set the i variable n indicating the n-th edge line in the region RL (step 2001), n is incremented by 1 (step 2002), comparing the n and NL ^L (step 2003). If n is equal to or less than NL ^L (step 2003, NO), the edge line L ^L _n is selected as one edge line c1 included in the combination of edge lines (step 2004). As the other edge line c2, an edge line closest to the end point on the side closer to the edge line c1 is selected from the edge lines in the integrated list CL ^L _m (step 2005).

Next, a distance DZ in the Z-axis direction (depth direction) and a distance DX in the X-axis direction (direction perpendicular to the depth direction) between the end points of the edge line c1 and the edge line c2 that are close to each other are Calculation is performed using an equation (step 2006).
DZ = | Z2-Z1 |
Z1 = f.H / (f..phi. + Y1-cy)
Z2 = f.H / (f..phi. + Y2-cy) (41)
DX = | X2-X1 |

  (X1, y1) represents the image coordinate of the end point of the edge line existing on the upper side of the image, and (x2, y2) represents the image coordinate of the end point of the edge line existing on the lower side. Expressions (41) and (42) are obtained from the geometric relationship between the road surface coordinate system and the image coordinate system.

  Next, DX is compared with a threshold ThDX (step 2007), and DZ is compared with a threshold ThDZ (step 2008). If DX is equal to or less than ThDX (step 2007, YES) and DZ is equal to or less than ThDZ (step 2008, YES), the line types of the edge lines c1 and c2 are determined (step 2009). Here, of the edge lines c1 and c2, the line type that exists on the lower side of the image is t1, and the line type that exists on the upper side of the image is t2.

Next, it is checked whether or not the line type t1 is an auxiliary line (step 2010). If the line type t1 is not an auxiliary line (step 2010, NO), t1 and t2 are compared (step 2011). If t1 and t2 match (step 2011, YES), the edge line c1 is added to the integrated list CL ^L _m (step 2012), and 1 is set to F [n] (step 2013). The processes after step 2002 are repeated.

  On the other hand, when the line type t1 is an auxiliary line (step 2010, YES), the processing after step 2012 is performed. If DX is larger than ThDX (step 2007, NO), if DZ is larger than ThDZ (step 2008, NO), or if t1 and t2 do not match (step 2011, NO), the processing after step 2002 is performed. repeat.

Next, when n exceeds NL ^L (step 2003, YES), the processing after step 1904 is repeated. When i is greater than NL ^L (step 1905, YES), by setting m to the number NC ^L of generated composite edge line (step 1911), the process ends.

In the integrated list CL ^L _m (m = 1, 2,..., NC ^L ) obtained in this way, one or more edge lines included in the left m-th composite edge line are recorded, and the line type T ^{In L} _m , the line type of the m-th composite edge line on the left is recorded. The generated composite edge line corresponds to edge lines such as the whole solid lane line, the whole broken lane line, the whole auxiliary line, and the like.

In FIG. 19 and FIG. 20, when the subscript L is replaced with R, a flowchart of a process for generating a composite edge line in the right region RR is obtained. Consolidated list _{^{CL R m (m = 1,2,}} ..., NC R) , the one or more edge lines included in the right side of the m-th composite edge line is recorded, the line type ^T _{R m} The line type of the m-th composite edge line on the right side is recorded.

  For example, as shown in FIG. 21, two composite edge lines 2101 and 2102 on the left side and two composite edge lines 2103 and 2104 on the right side are generated from the plurality of edge lines in FIG.

FIG. 22 is a flowchart illustrating an example of processing for determining a candidate line from the left composite edge line in the candidate line determination processing 1203 of FIG. First, the candidate line determination unit 1103 sets 0 to a variable m indicating a composite edge line corresponding to the integrated list CL ^L _m (step 2201). Then, it sets the -1 to a variable S _L showing the composite edge line that is determined candidate line (step 2202) and sets a sufficiently large value XG variable Xmin represents the minimum X-coordinate of the road surface coordinate system (Step 2203). When the unit of distance on the road surface is meters, for example, a value of about 100,000 can be used as XG.

Then, m is incremented by 1 (step 2204), it is compared with the NC ^L m (step 2205). If m is less than NC ^L image coordinates (step 2205, NO), (endpoint most on the lower side in the image) close endpoints p most vehicles of end points of the edge line included in the integrated list CL ^L _m It is set to (x, y) (step 2206). Then, the X coordinate of the end point p in the road surface coordinate system is calculated by the following equation (step 2207).

Next, the absolute value | X | of X is compared with Xmin (step 2208). If | X | is smaller than Xmin (step 2208, YES), | X | is set to Xmin (step 2209). Then, by setting m to _{S L} (step 2210) and repeats the step 2204 and subsequent steps.

On the other hand, if | X | is equal to or greater than Xmin (step 2208, NO), the processing after step 2204 is repeated. When m exceeds NC ^L (step 2205, YES), the process ends. Thus, the number of composite edge line as a left candidate lines is recorded on S _L.

If the subscript L is replaced with R in FIG. 22, a flowchart of processing for determining a candidate line from the composite edge line on the right side can be obtained. The number of composite edge line on the right side of the candidate lines is recorded in S _R.

FIG. 23 is a flowchart illustrating an example of processing for determining the line type of the left candidate line in the auxiliary line determination processing 1204 of FIG. First, the auxiliary line determination unit 1106 determines whether or not the line type T ^L _m (m = S _L ) of the left S _L- th composite edge line is an auxiliary line (step 2301). If the line type T ^L _m (m = S _L ) is an auxiliary line (step 2301, YES), a flag Q _L indicating the line type of the left candidate line is set to 1 (step 2302), and the line type T ^L _{If m} (m = S _L ) is not an auxiliary line (step 2301, NO), the flag Q _L is set to 0 (step 2303).

If the subscript L is replaced with R in FIG. 23, a flowchart of the process for determining the line type of the right candidate line is obtained. Then, 1 or 0 is set in the flag Q _R indicating the line type of the right candidate lines.

24 and 25 are flowcharts showing an example of processing for obtaining the correction amount for the left candidate line in the correction amount estimation processing 1205 of FIG. First, the correction amount estimation unit 1107 checks the flag Q _L (step 2401). If Q _L = 0 (step 2401, YES), it can be determined that the candidate line is not an auxiliary line but a lane line, so the correction amount DX _L is set to 0 (step 2413), and the process ends. To do.

On the other hand, if Q _L = 1 (step 2401, NO), since the candidate line is an auxiliary line, the presence / absence of a composite edge line located outside the candidate line by the width of the auxiliary line is checked. Therefore, to set the -1 flag _{U L} indicating the presence or absence of such a condition is satisfied outside the composite edge line (step 2402). When the value of the flag U _L is -1, satisfy represents that the outer composite edge line is not present, when the value of the flag U _L is 1 or more, indicating that there is satisfies outer composite edge line.

Next, among the end points of the edge line included in the S _L th composite edge line is a candidate line, sets the image coordinates of the closest to the vehicle end point to (xs, ys) (step 2403). Then, the X coordinate Xs in the road surface coordinate system of the end point is calculated by the following equation (step 2404).

Next, the minimum value Xmin indicating the range of the X coordinate of the outer composite edge line that satisfies the condition is calculated by the following equation (step 2405).
Xmin = | Xs | + Mmin (62)

Further, the maximum value Xmax of the range is calculated by the following equation (step 2406).
Xmax = | Xs | + Mmax (63)

  Mmin and Mmax are threshold values determined based on the width of the auxiliary line, and Mmax is larger than Mmin. If a composite edge line exists in the range from Xmin to Xmax, the composite edge line corresponds to the outer composite edge line that satisfies the condition.

Then, 0 is set to the variable m indicating the m-th composite edge line of the left (step 2407), it is incremented by 1 m (step 2408) and compares m with NC ^L (step 2409). If m is less than NC ^L (step 2409, NO), then, the m is compared with _{S L} (step 2410). If m = S _L (step 2410, YES), the m-th composite edge line matches the candidate line, so the process returns to step 2408 to select the next composite edge line as a processing target.

On the other hand, m is _{S L} unless (step 2410, NO), among the end points of the edge line included in the m-th composite edge line, sets the image coordinates of the closest to the vehicle end point to (xc, yc) ( Step 2411). Then, the X coordinate Xc and the Z coordinate Zc in the road surface coordinate system of the end point are calculated by the following equations (step 2412).

Next, the absolute value | Xc | of Xc is compared with Xmin (step 2501). If | Xc | is equal to or greater than Xmin (step 2501, YES), | Xc | is compared with Xmax (step 2502). | Xc | if is less than Xmax (step 2502, YES), then, to check the _{U L} (step 2503). If U _L = -1 (step 2503, YES), satisfies the condition outside the composite edge line so that the first time is detected, setting the m to _{U L} (step 2504).

  Next, | Xc | is set to the absolute value | Xu | of Xu (step 2505), Zc is set to Zu (step 2506), and the processing after step 2408 is repeated. Xu and Zu represent the X coordinate and the Z coordinate of the end point closest to the vehicle among the end points of the edge line included in the outer composite edge line that satisfies the condition.

On the other hand, unless U _L -1 (step 2503, NO), satisfies the condition by an outer composite edge line so that the newly detected and compared with Zu that has already been calculated Zc (step 2507). If Zc is smaller than Zu (step 2507, YES), it can be seen that the newly detected outer composite edge line is located closer to the vehicle than the previously detected outer composite edge line. Therefore, the processing after step 2504 is performed to update the information U _L , | Xu |, and Zu of the outer composite edge line that satisfies the condition.

  On the other hand, if Zc is equal to or greater than Zu (step 2507, NO), the newly detected outer composite edge line is located farther from the vehicle than the previously detected outer composite edge line, or the previously detected outer composite edge line. It can be seen that it exists at the same distance as the composite edge line. In this case, the processing after step 2408 is repeated without updating the information of the outer composite edge line that satisfies the condition.

  In addition, when | Xc | is smaller than Xmin (step 2501, NO) or | Xc | is larger than Xmax (step 2502, NO), it can be seen that the m-th composite edge line does not satisfy the condition. Therefore, returning to the processing of step 2408, the next composite edge line is selected as a processing target.

When m exceeds NC ^L (step 2409, YES), checks the _{U L} (step 2414). If U _L = −1 (step 2414, YES), the outer composite edge line that satisfies the condition has not been detected, and the process ends. Otherwise U _L -1 (step 2414, NO), because the condition is satisfied outside the composite edge line is detected, the correction amount DX _L is calculated by the following equation (step 2415).
DX _L = | Xu | − | Xs | (66)

By replacing accompanied by letter L in FIGS. 24 and 25 to R, to obtain a flowchart of processing for obtaining the correction amount for the right of candidate lines, the correction amount DX _R is obtained. Correction amount DX _L and DX _R obtained is stored in the storage unit 1108. These correction amounts represent shift amounts in which the direction from the road center to the outside is a positive direction.

In the road shape estimation processing 1206 in FIG. 12, the road shape estimation unit 1104, and S _L th composite edge line is a left candidate lines, on the basis of the S _R th composite edge line is a right candidate lines, roads Estimate shape parameters. The road shape parameter can be estimated using, for example, the above equation (9). When the equation (9) is rewritten with the upper left point of the image as the origin of the image coordinate system, the following equation is obtained.

  Since the equation (71) is a nonlinear equation, it is approximated so as to be in a linear form with respect to the road shape parameters (W, ρ, e, θ, φ), and each parameter is determined by the least square method using a linear approximation equation. The calculation method to obtain can be considered. In this calculation method, the estimated values (Wr, ρr, er, θr, φr) of the road shape parameters are obtained by so-called repetitive calculation that repeats the calculation until the error becomes equal to or less than the threshold value.

The image coordinates of the j-th edge point (j = 1, 2,..., NE) included in the left or right candidate line are set to (x _j , y _j ), and the right side of equation (71) is F (W , Ρ, e, θ, φ), and each parameter is described as follows.
W = W0 + ΔW
ρ = ρ0 + Δρ
e = e0 + Δe
θ = θ0 + Δθ
φ = φ0 + Δφ (72)

  NE represents the number of edge points included in the candidate line, (W0, ρ0, e0, θ0, φ0) represents an initial value of each parameter, and (ΔW, Δρ, Δe, Δθ, Δφ) represents each parameter. Represents a small amount of change. F (W, ρ, e, θ, φ) is a non-linear function. A function J obtained by linearizing the function using the equation (72) is given by the following equation.

Here, the residual s _j is defined as follows.

Let ½ of the result obtained by integrating the square value of the residual s _j at the edge points (j = 1, 2,..., NE) on all lane markings in the frame be the total residual S.

  Since the optimum parameter is a value that minimizes S, the result of partial differentiation of S with respect to each parameter is set to 0. For example, if the result of partial differentiation of S with W is set to 0, the following equation is obtained.

  By performing the same partial differential operation for other parameters, the following equation is obtained.

  The simultaneous equations of equations (77) and (78) can be described as the following equations.

  By solving this simultaneous linear equation, (ΔW, Δρ, Δe, Δθ, Δφ) is obtained. Then, until the value of each parameter becomes sufficiently small, the calculation is repeated by performing the following substituting operation.

  The final road shape parameters (Wr, ρr, er, θr, φr) are obtained by such repeated calculation.

Here, the image coordinates of the edge point e ^L _j (j = 1, 2,..., NEL) included in the left S _L- th compound edge line are (x ^L _j , y ^L _j ), and the right side edge point contained in S _R th composite edge line _{^{e R j (j = 1,2,}} ..., NER) is the image coordinates of the _{^{(x R j, y R j}} ). NEL represents the number of edge points included in the S _L th composite edge line, NER represents the number of edge points included in the S _R th composite edge line.

The set of edge points E _L included in the S _L th composite edge line is {e ^L ₁ , e ^L ₂ ,. . . , ^E _{L NEL}} and is described, the set _{E R} of the edge points included in the _{S R} th composite edge line is ^{_{{e R 1, e R 2}} ,. . . , E ^R _NER }. At this time, the procedure of the road shape estimation process is as follows.

1. (W0, ρ0, e0, θ0, φ0) is initialized to the current value.
2. R11 = R12 = R13 = R14 = R15 = R16 = 0
R21 = R22 = R23 = R24 = R25 = R26 = 0
R31 = R32 = R33 = R34 = R35 = R36 = 0
R41 = R42 = R43 = R44 = R45 = R46 = 0
R51 = R52 = R53 = R54 = R55 = R56 = 0
3. tgt = -1
4). tgt = tgt + 1
5. if tgt == 0 then
Edge set E = E _L , NE = NEL, k = −1
else if tgt == 1 then
Edge set E = E _R , NE = NER, k = 1
else goto 13 endif
6). j = 0
7). j = j + 1
8). if j> = NE then goto 4 endif
9. The position (x _j , y _j ) of the edge point that is the j-th element is acquired from the edge set E.
10. From (73) and (75), (a _j , b _j , c _j , d _j , e _j , f _j ) is calculated.
11. _{R11 + = a j a j,} R12 + = a j b j, R13 + = a j c j, R14 + = a j d j,
_{R15 + = a j e j,} R16 + = a j f j,
_{R21 + = b j a j,} R22 + = b j b j, R23 + = b j c j, R24 + = b j d j,
_{R25 + = b j e j,} R26 + = b j f j,
_{R31 + = c j a j,} R32 + = c j b j, R33 + = c j c j, R34 + = c j d j,
R35 + = c _j e _j , R36 + = c _j f _j ,
R41 + = d _j a _j , R42 + = d _j b _j , R43 + = d _j c _j , R44 + = d _j d _j ,
R45 + = d _j e _j , R46 + = d _j f _j ,
_{R51 + = e j a j,} R52 + = e j b j, R53 + = e j c j, R54 + = e j d j,
R55 + = e _j e _j , R56 + = e _j f _j
12 goto 7
13. The solution of the simultaneous equations (79) is obtained by numerical calculation.
14 Check the convergence of the solution.
if | ΔW |> ThR or | Δρ |> ThR or
| Δe |> ThR or | Δθ |> ThR or | Δφ |> ThR
then
goto 2
else
Wr = W0, ρr = ρ0, er = e0, θr = θ0, φr = φ0
endif
15. Finish

  ThR is a minute threshold for evaluating the convergence of the solution. By such road shape estimation processing, road shape parameters (Wr, ρr, er, θr, φr) are obtained.

In the road shape correction process 1207 in FIG. 12, the correction unit 1105, by using the correction amount DX _L and DX _R stored in the storage unit 1108, a road shape parameter according to the following equation (Wr, ρr, er, θr , φr) Correct.
Wr = Wr + (DX _L + DX _R )
er = er + (− DX _L + DX _R ) / 2 (91)

  The road shape parameter calculation formulas (1) to (91) described above are merely examples, and different formulations may be performed based on different coordinate systems, different preconditions, and the like. Further, the flowcharts shown in FIGS. 10, 12, 13, 19, 20, and 22 to 25 are merely examples, and some processes are performed depending on the configuration and conditions of the road shape estimation device. It may be omitted or changed.

  FIG. 26 shows a configuration example of a lane departure warning system using a road shape estimation device. The lane departure warning system of FIG. 26 includes a camera 1201, a road shape estimation device 2601, a distance calculation device 2602, a departure determination device 2603, and a departure warning device 2604. The operation of the road shape estimation apparatus 2601 is as described above.

  The distance calculation device 2602 calculates the distance between the vehicle and the left and right lane markings based on the road shape parameter output from the road shape estimation device 2601. The departure determination device 2603 compares the calculated distance with a threshold value to determine whether or not the vehicle has deviated from the lane. The departure warning device 2604 outputs a warning when the vehicle departs from the lane. As a result, the driver can immediately steer the vehicle and return it to the lane.

  FIG. 27 shows a configuration example of a vehicle steering system using the road shape estimation device. The vehicle steering system in FIG. 27 includes a camera 1201, a road shape estimation device 2601, a distance calculation device 2602, a wobble determination device 2701, and a vehicle steering device 2702.

  The wobble determination device 2701 compares the distance calculated by the distance calculation device 2602 with a threshold value and determines whether or not the vehicle is wobbling. When the vehicle is wobbling, the vehicle steering device 2702 automatically steers the vehicle so as to maintain traveling on the lane, thereby eliminating the vehicle wobble. This makes it possible to prevent deviation from the lane.

The road shape estimation apparatus shown in FIGS. 9, 11, 26, and 27 can be realized using, for example, an information processing apparatus (computer) as shown in FIG.
28 includes a video input unit 1101, a central processing unit (CPU) 2801, a memory 2802, an output unit 2803, an external storage device 2804, a medium drive device 2805, and a network connection device 2806. These are connected to each other by a bus 2807.

  The memory 2802 is a semiconductor memory such as a read only memory (ROM), a random access memory (RAM), or a flash memory, and stores programs and data used for processing. For example, the CPU 2801 performs processing of the road shape estimation device by executing a program using the memory 2802. The memory 2802 can also be used as the storage unit 915 in FIG. 9 or the storage unit 1108 in FIG.

  The video input unit 1101 takes in a video signal from the camera and stores it in the memory 2802 as a time-series image. The output unit 2803 is, for example, an interface with an external device, and is used for outputting a processing result including a road shape parameter.

  The external storage device 2804 is, for example, a magnetic disk device, an optical disk device, a magneto-optical disk device, a tape device, or the like. The external storage device 2804 includes a hard disk drive. The information processing apparatus can store programs and data in the external storage device 2804 and load them into the memory 2802 for use.

  The medium driving device 2805 drives the portable recording medium 2808 and accesses the recorded contents. The portable recording medium 2808 is a memory device, a flexible disk, an optical disk, a magneto-optical disk, or the like. The portable recording medium 2808 includes a compact disk read only memory (CD-ROM), a digital versatile disk (DVD), a universal serial bus (USB) memory, and the like. A user or an operator can store programs and data in the portable recording medium 2808 and load them into the memory 2802 for use.

  As described above, computer-readable recording media for storing programs and data used in various processes include physical (non-temporary) such as the memory 2802, the external storage device 2804, and the portable recording medium 2808. ) A recording medium is included.

  The network connection device 2805 is a communication interface that is connected to a communication network and performs data conversion accompanying communication. The information processing apparatus can also receive a program and data from an external apparatus via the network connection apparatus 2805 and load them into the memory 2802 for use.

  Note that the information processing apparatus does not have to include all the components illustrated in FIG. 28, and some of the components may be omitted depending on applications and conditions. For example, when the information processing apparatus does not access the portable recording medium 2808, the medium driving apparatus 2805 may be omitted, and when the information processing apparatus is not connected to the communication network, the network connection apparatus 2805 may be omitted. .

  Although the disclosed embodiments and their advantages have been described in detail, those skilled in the art can make various modifications, additions and omissions without departing from the scope of the present invention as explicitly set forth in the claims. Let's go.

101, 102, 201, 202, 301, 302, 711, 712 Lane lane markings 103, 104, 203, 204, 303, 304, 721, 722 Auxiliary lines 205, 206, 305, 306 Edge lines 601-606, 731, 732, 741, 742 Inner boundary lines 701, 702 Regions 801-804, 811-813 Positions 901, 2601 Road shape estimation device 911, 1102 Edge detection unit 912, 1104 Road shape estimation unit 913, 1105 Correction unit 914, 1107 Correction amount Estimating unit 915, 1108 Storage unit 1100 Camera 1101 Video input unit 1103 Candidate line determination unit 1106 Auxiliary line determination unit 2101, 1022, 2103, 2104 Compound edge line 2602 Distance calculation device 2603 Deviation determination device 2604 Deviation warning device 2701 Variability determination device 2702 Steering system 2801 CPU
2802 Memory 2803 Output unit 2804 External storage device 2805 Medium drive device 2806 Network connection device 2807 Bus 2808 Portable recording medium

Claims (6)

An edge detection unit that detects a plurality of edge lines from an image of a road surface of a road and generates a composite edge line by connecting one or more of the plurality of edge lines;
Based on the information of the inner composite edge line closest to the center of the road, including a lateral offset of the camera that captured the road surface and the road width, a road shape estimation unit for obtaining road shape parameters of the road;
A correction amount estimation unit for obtaining a correction amount based on a distance between the inner composite edge line and an outer composite edge line located outside the inner composite edge line;
A storage unit for storing the correction amount;
A road shape estimation apparatus comprising: a correction unit that corrects the lateral offset and the road width using the correction amount, and outputs a road shape parameter including the corrected lateral offset and road width.   The correction amount estimation unit includes a first edge line that is present in the vicinity of the camera in the image among edge lines included in the inner composite edge line, and the edge line included in the outer composite edge line. The road shape estimation apparatus according to claim 1, wherein the correction amount is obtained based on a distance from a second edge line existing in the vicinity of the camera in an image.   The correction amount estimation unit checks whether the outer composite edge line exists outside the inner composite edge line when the length of the edge line included in the inner composite edge line is smaller than a threshold, When the outer composite edge line exists, the correction amount is obtained based on the distance between the inner composite edge line and the outer composite edge line, and the length of the edge line included in the inner composite edge line is larger than the threshold value. The road shape estimation apparatus according to claim 1, wherein the correction amount is set to zero. The road shape estimation device according to any one of claims 1 to 3,
A departure determination device that determines whether the vehicle has deviated from the lane based on the road shape parameter output from the road shape estimation device;
A lane departure warning system comprising a departure warning device that outputs a warning when the vehicle departs from the lane. The road shape estimation device according to any one of claims 1 to 3,
Based on the road shape parameter output from the road shape estimation device, a stagger determination device that determines whether the vehicle is staggered,
A vehicle steering system comprising: a vehicle steering device that steers the vehicle so as to maintain traveling on a lane when the vehicle is staggered. Detect multiple edge lines from the image of the road surface,
Connecting one or more edge lines of the plurality of edge lines to generate a composite edge line;
Based on the information of the inner composite edge line closest to the center of the road, including the lateral offset of the camera that captured the road surface and the road width, to determine the road shape parameters of the road,
Obtaining a correction amount based on the distance between the inner composite edge line and the outer composite edge line located outside the inner composite edge line,
Correcting the lateral offset and the road width using the correction amount;
A program for causing a computer to execute a process of outputting road shape parameters including a corrected lateral offset and road width.