JP4207935B2 - Travel path recognition device - Google Patents

Description

  The present invention relates to a travel path recognition device that is mounted on a vehicle such as an automobile and recognizes the type of multiple lane markings on the travel path on which the vehicle travels.

As a conventional travel path recognition device, for example, as described in Patent Document 1, each sign constituting the left multiple division line based on the captured image of the left multiple division line and the right multiple division line Detects the distance between the line (white line, driving guidance line, etc.) and each marking line that forms the right multiple division line, and uses the right and left pair of marking lines that are most appropriate in time series for driving the vehicle. What is set as the lane boundary to be used is known.
JP 2003-168198 A

  However, in the above prior art, the lane boundary is determined only by the distance between the left and right marking lines based on time-series estimation. For example, when the left and right multiple division lines are double lines, Misrecognitions such as a line outside the double line as a lane boundary and a line inside the double line as a lane boundary on the right side are likely to occur. Therefore, in order to obtain an accurate lane boundary, it is necessary to correctly recognize the types of multiple dividing lines on the left and right sides.

  An object of the present invention is to provide a travel path recognition device capable of accurately recognizing types of multiple division lines on a travel path.

  The present invention provides a travel path recognition apparatus for recognizing a multiple segment line composed of a plurality of label lines provided on a travel path based on a captured image of a travel path on which a vehicle travels. Based on the captured image, a storage means for storing in advance a plurality of types of matching patterns consisting of a template that is combination information and line type information that is line type information of each marking line that constitutes multiple division lines Multiple division line extracting means for extracting multiple division lines, determination means for determining the line type of each marking line constituting the extracted multiple division lines, and determination means for determining the type of template corresponding to the extracted multiple division lines And an identification means for identifying the type of the multiple lane markings based on the comparison between the determination result of the determination means, the determination result of the determination means, and the matching pattern.

  In this way, when identifying the types of multiple lane marking lines, in addition to the combination of each marking line constituting the multiple lane marking lines, the line type (solid line or broken line) of each marking line is also considered, As a result of the above verification, the recognition rate of the types of multiple lane markings can be improved. For example, there are double lines having different thicknesses and double lines having the same thickness among the double lines as the multiple dividing lines. In the double lines having different thicknesses, the marking line on one side is a solid line or a broken line, and the marking line on the other side is a broken line. In the double line having the same thickness, one of the marking lines is one of a solid line and a broken line, and the other marking line is the other of the solid line and the broken line. Here, in recognition of multiple division lines, a certain amount of extraction margin is provided in consideration of the imaging accuracy of the imaging camera and the like. For this reason, when a double line on the traveling road is imaged, it may be determined that the double line has a different thickness and is also a double line having the same thickness. However, the double lines having different thicknesses and the double lines having the same thickness have different types of line types as described above. Or the broken line, it is possible to accurately determine which double line it is.

  Preferably, the identification means identifies the types of multiple dividing lines in the left and right regions independently of the traveling direction of the vehicle. The combination and line type of each marking line constituting the multiple dividing line may be different between the left region and the right region with respect to the traveling direction of the vehicle. For this reason, the type of multiple division lines on the road can be accurately recognized on both the left and right sides by independently comparing the discrimination result of the discrimination means, the determination result of the determination means, and the pattern for collation in the left and right regions. Can do.

  Preferably, the template is information on the number of marking lines constituting the multiple dividing lines and the thickness of each marking line. As the multiple dividing line, there are a double line with two marking lines and a triple line with three marking lines. Some double lines have different marking lines and others have the same thickness. Therefore, it is possible to recognize these types of multiple dividing lines.

  At this time, preferably, the storage means stores a matching pattern corresponding to a triple line in which the number of marking lines is three and the line type of each marking line is a broken line. In this case, the triple line on the traveling road can be accurately recognized.

  Further, the storage means has two marking lines, the line type of one of the marking lines is a solid line or a broken line, the line type of the other marking line is a broken line, and the thickness thereof is one. A matching pattern corresponding to a double line with a different width that is thicker than the side marking line may be stored. In this case, the right thick double line and the left thick double line on the traveling road can be accurately recognized.

  At this time, according to whether the identification means is identifying the type of the multiple division lines in the left or right region with respect to the traveling direction of the vehicle, the identification pattern has a matching pattern whose line type of the one side marking line is a solid line. It is preferable not to perform collation. In the double line of different width such as the right thick double line and the left thick double line, the line type of the narrower marking line is always a broken line depending on whether it exists on the left or right side with respect to the traveling direction of the vehicle. There is. In this case, since it is not necessary to collate the pattern for collation in which the line type of the one side marking line is a solid line, the processing can be simplified.

  Further, the storage means has two marking lines, the line type of one of the marking lines is one of a solid line or a broken line, and the line type of the other marking line is the other of the broken line. You may memorize | store the pattern for a collation corresponding to the double line of the same width which has substantially the same thickness as the side marking line. In this case, it is possible to accurately recognize both the thick double lines and the both thin double lines on the road.

  In addition, the storage means stores a matching pattern corresponding to a hollow double line in which the number of marking lines is two and the line type of each marking line is a solid line. In the identification of the multiple division lines in the left region with respect to the traveling direction, the matching with the matching pattern corresponding to the hollow double line may not be performed. Among multiple lane markings, there is a so-called provisional service section that tentatively shares either an up line or a down line as a two-lane section with two-way traffic. This provisional service part exists only in the right region with respect to the traveling direction of the vehicle, and has a hollow double line structure in which the two marking lines are both solid lines. Therefore, in identifying the type of multiple division lines in the right region with respect to the traveling direction of the vehicle, by preparing a matching pattern corresponding to the hollow double line, a triple line and a different width double line are prepared. In addition to basic multiple division lines such as double lines of the same width, provisional service parts can be reliably recognized. Further, in identifying the type of multiple division lines in the left region with respect to the traveling direction of the vehicle, it is not necessary to collate a pattern for collation that is a hollow double line.

  Preferably, the apparatus further includes means for setting a lane boundary used for traveling of the vehicle based on the type of the multiple division line identified by the identification means. Thus, for example, by performing vehicle steering control so that the vehicle travels according to the lane boundary of the multiple division lines, it is possible to keep the travel position of the vehicle or prevent the vehicle from departing from the lane.

  According to the present invention, it is possible to accurately recognize the type of multiple lane markings on a travel path on which a vehicle travels. Thus, for example, in vehicle steering control, it is possible to correctly determine whether or not the vehicle can be changed to the left lane.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a preferred embodiment of a travel path recognition apparatus according to the invention will be described in detail with reference to the drawings.

  FIG. 1 is a schematic block diagram showing a steering control system including an embodiment of a travel path recognition apparatus according to the present invention. In the figure, a steering control system 1 is mounted on a vehicle such as an automobile. The steering control system 1 includes a travel path recognition device 2, a steering assist ECU (Electronic Control Unit) 3, and an electric power steering ECU 4.

  The travel path recognition apparatus 2 is an apparatus that recognizes the type of multiple lane markings drawn on the travel path on which the vehicle travels. The traveling path recognition device 2 detects a lane boundary (white line) in the multiple division lines used for traveling of the vehicle based on the front camera 5 that images the multiple division lines ahead of the vehicle and the image captured by the front camera 5. The white line detection ECU 6 that performs

  The steering assist ECU 3 controls the electric power steering ECU 4 to assist the steering of the vehicle based on the lane boundary detected by the white line detection ECU 6 and the detected values of the vehicle speed sensor 7, the yaw rate sensor 8, and the rain sensor 9. . The electric power steering ECU 4 drives the steering device 10 in accordance with a control signal from the steering assist ECU 3.

  FIG. 2 is a functional block diagram of the white line detection ECU 6. In the figure, a white line detection ECU 6 includes a multiple lane marking extraction unit 11, a solid line broken line determination unit 12, a multiple lane marking section determination unit 13, a multiple lane marking type identification unit 14, a lane boundary setting unit 15, and a storage. Part 16.

The multiple lane marking extraction unit 11 acquires a captured image of the front camera 5, performs image processing such as binarization processing and edge extraction, and performs projective conversion of the image data into bird's-eye image data as shown in FIG. Multiple division lines are extracted based on the bird's-eye view image data. A portion surrounded by a broken line in the bird's-eye image data is the processing region P. The processing area P is divided into a far area Pa and a neighboring area Pb as shown in FIG. A distance I from the front camera 5 to the imaging position is, for example, 7 m. The distance N between the far region Pa and the neighboring region Pb is, for example, 10 m.

  Specifically, as shown in FIG. 5, the multiple lane marking extraction unit 11 first extracts the edge of the processing area P of the bird's-eye image data for each line and extracts the edge points extracted in the far area Pa and the neighboring area Pb. Multiple division lines Q are extracted by performing Hough transform and performing linear approximation. This process is performed separately on both the left and right sides of the areas Pa and Pb. That is, the extraction of the multiple lane markings Q is performed in the left far region, the left neighboring region, the right far region, and the right neighboring region of the bird's-eye image data.

  Here, as shown in FIG. 6, there are a triple line, a right thick double line, a left thick double line, a double thick double line, and a double thin double line. The right thick double line and the left thick double line are different width double lines, and both the thick double lines and both thin double lines are the same width double lines.

  As shown to Fig.6 (a), the triple line 20 consists of the marking lines 21a-21c. The marking lines 21a to 21c are all broken lines. The marking lines 21a and 21c on both sides are travel guiding lines and are thicker than the middle marking line 21b.

  As shown in FIG. 6B, the right thick double line 22 is composed of marking lines 23a and 23b, and the right marking line 23b is thicker than the left marking line 23a. The thin marking line 23a on the left side is a broken line, and the thick marking line 23b on the right side is a broken line because it is a travel guide line. Another right thick double line 24 is composed of marking lines 25a and 25b, and the right marking line 25b is thicker than the left marking line 25a. The left thin marking line 25a is a solid line, and the right thick marking line 25b is a broken line because it is a travel guide line.

  As shown in FIG. 6 (c), the left thick double line 26 is composed of marking lines 27a and 27b, and the left marking line 27a is thicker than the right marking line 27b. The thin marking line 27b on the right is a broken line, and the thick marking line 27a on the left is a broken line because it is a travel guide line. Another left thick double line 28 is composed of marking lines 29a and 29b, and the left marking line 29a is thicker than the right marking line 29b. The thin marking line 29b on the right side is a solid line, and the thick marking line 29a on the left side is a broken line because it is a travel guide line.

  As shown in FIG. 6D, both thick double lines 30 are composed of marking lines 31a and 31b, and the thicknesses of the marking lines 31a and 31b are substantially the same. The left marking line 31a is a solid line, and the right marking line 31b is a broken line because it is a travel guide line. The other thick double line 32 includes marking lines 33a and 33b, and the marking lines 33a and 33b have substantially the same thickness. Since the left marking line 33a is a travel guide line, it is a broken line, and the right marking line 33b is a solid line.

  As shown in FIG. 6 (e), both fine double lines 34 are formed of marking lines 35a and 35b, and the thicknesses of the marking lines 35a and 35b are substantially the same. The left marking line 35a is a solid line, and the right marking line 35b is a broken line because it is a travel guide line. The other two thin double lines 36 consist of marking lines 37a and 37b, and the thicknesses of the marking lines 37a and 37b are substantially the same. Since the left marking line 37a is a travel guidance line, it is a broken line, and the right marking line 37b is a solid line.

  The solid line broken line discriminating unit 12 is based on the multiple segment lines extracted by the multiple segment line extracting unit 11 (hereinafter, extracted data of the multiple segment lines), and the line type (solid line or broken line) of each marking line constituting the multiple segment line. ). Specifically, when the voting value of a straight line subjected to Hough transform at the edge points on all the lines Pa and Pb is 100%, when the voting value is larger than a reference value (for example, 80%), the straight line Is determined to be a solid line; otherwise, the straight line is determined to be a broken line.

  The multiple lane marking section determination unit 13 determines whether or not the multiple lane markings on the traveling road are continuous (multi-segment line section) based on the extracted data of the multiple lane markings. This determination process is performed using a plurality of types of matching patterns stored in advance in the storage unit 16. The verification pattern includes a multi-line template that is information on the combination (number and thickness) of each marking line that constitutes the multiple dividing line, and line type information that is information on the line type of each marking line that constitutes the multiple dividing line. And information on the type of multiple dividing lines. Then, the multiple lane marking section determination unit 13 determines the type of the multiple line template corresponding to the extracted data of the multiple lane markings.

  Specifically, as the multi-line template, as shown in FIGS. 7A to 7E, a triple line template, a right thick double line template, a left thick double line template, and both thick double lines are used. There are line templates and double thin double line templates. The triple line template is a template corresponding to the triple line 20 shown in FIG. The right thick double line template is a template corresponding to the right thick double lines 22 and 24 shown in FIG. 6 (b), and the left thick double line template is the left thick double line shown in FIG. 6 (c). 26 and 28. Both thick double line templates are templates corresponding to both thick double lines 30 and 32 shown in FIG. 6 (d), and both thin double line templates are both thin double lines shown in FIG. 6 (e). This is a template corresponding to 34 and 36.

Using a multi-line template corresponding to such multiple multi-segment lines, the multi-line template extraction data (a combination of edge lines as a result of linear approximation by Hough transform of edge points) and multi-line template are collated. As described above, this is performed on each of the left and right sides of the distant area and the vicinity area of the bird's-eye image data. In this case, the collation of the example 3 doublets template, as shown in FIG. 8, the width W ₁ of the left marking _line, the width W ₂ of the marking line in the _middle, the width W ₃ of the right marking _line, left signs spacing D ₁ of the the marking lines of the line and the _middle, spacing D ₂ of the marking line and the right marking line in the _middle, the difference in inclination between both edge line at the left side of the marking line, the slope of both edges line in marking line in the middle And the difference between the slopes of both edge lines in the right marking line are used as parameters.

  Also, among the multiple division lines, there is also a double line called a temporary service section as shown in FIG. The provisional service section is, for example, a two-lane section for two-lane construction of an expressway, and when only one of the up-line or the down-line is completed, This is a section in which either one of the uplink and the downlink is provisionally used. The provisional service section 38 shown in FIG. 9A is provided with orange lines 41 via white lines 40 on both sides of a special center line 39 such as a block. The provisional service part 42 shown in FIG. 9B is provided with orange lines 41 on both sides of the center line 39 with a space therebetween. That is, in the provisional service part 42, the area between the center line 39 and the orange line 41 is asphalt. Note that the white line 40 and the orange line 41 are both solid lines.

  As a template for the provisional service section, for example, a hollow double line as shown in FIG. 10 is used. The hollow double line template shown in FIG. 10 (a) corresponds to the provisional service section 38 shown in FIG. 9 (a), and the width of each marking line X is the width of the white line 40 and the width of the orange line 41. It is thick corresponding to the total of. The hollow double line template shown in FIG. 10B corresponds to the provisional service section 42 shown in FIG. 9B, and the width of each marking line X is narrower corresponding to the width of the orange line 41 alone. It has become.

  The multiple lane marking section determination unit 13 performs the determination of such multiple lane marking sections independently in the far and near areas of the bird's-eye image data, and regards the multiple lane marking sections as the multiple lane marking sections when the left and right determination results are complete. Specifically, in the vicinity region of the bird's-eye image data, after the extracted data of the multiple lane markings coincides with any one of the multiple-line templates continuously for a predetermined number of times (for example, three times), If the extracted data of the lane markings coincides with the same multiplex line template continuously for a predetermined number of times (for example, 3 times), it is determined that it is a multiplex lane marking section. In addition, after the extracted data of the multiple lane markings coincides with any of the multiple line templates in the distant area of the bird's-eye image data for a predetermined number of times, the extracted data of the multiple lane markings is determined for a predetermined number of times in the vicinity of the bird's-eye image data. Even when the same multi-line template is continuously matched, it is determined that it is a multi-section line section. At this time, after the extracted data of the multiple lane markings coincides with any one of the multiple line templates in both the near area and the far area of the bird's-eye image data, and further matches with any one of the multi-line templates, further in the near area and the far area of the bird's-eye image data. Of course, the extracted data of multiple division lines may coincide with the same multiple line template continuously for a predetermined number of times.

  By determining the continuity of multiple division lines in this way, even if a multiple division line of a type different from the actual multiple division line is instantaneously determined due to noise or the like, the determination result is cut. . Accordingly, it is possible to prevent erroneous determination as a multiple lane marking section even though it is not actually a multiple lane marking section.

  Here, after the extracted data of the multiple segment lines coincides with the multiple line template continuously for a predetermined number of times, a mismatch between the extracted data of the multiple segment lines and the multiple line template occurs at predetermined times (near and far areas of the bird's-eye image data). For example, if it continues, it is temporarily determined that the multiple lane marking section has ended. This provisional determination is performed independently in the left and right regions in the distant region and the vicinity region of the bird's-eye view image data, and is valid only on the side on which provisional determination is made that the multiple lane marking section has ended on both the left and right sides.

  If the mismatch between the extracted data of the multiple lane markings and the multiple line template continues for a predetermined number of times (for example, 3 times), it is determined that the multiple lane marking section is officially completed. The determination of the end of the multiple lane marking section is performed independently in the far and near areas of the bird's-eye image data, and if it is determined that the multiple lane marking section has ended in either of the left and right sides, the multiple lane marking sections are multiplexed The lane marking section is considered complete. In addition, when the extracted data of the multi-segment line matches the multi-line template in the vicinity region or the distant region of the bird's-eye image data before the mismatch between the extracted data of the multi-segment line and the multi-line template reaches the number of times, The provisional determination that the multiple lane marking section has ended is cancelled, and the multiple lane marking section is regarded as continuing.

  The multiple lane marking type identification unit 14 uses the determination result of the solid line broken line determination unit 12 and the determination result of the multiple lane marking line determination unit 13 to determine the type of multiple lane marking lines (triple triple) extracted based on the bird's-eye image data. Line, right thick double line, left thick double line, both thick double lines, both thin double lines, and provisional service section). The multiple segment line type identification unit 14 performs identification based on a continuous match with individual multiple line templates. However, the number of times of coincidence between the extracted data of the multiple lane markings and the multiple line template may be smaller than that at the time of the determination of the multiple lane marking sections by the multiple lane marking section determination unit 13 described above. Any one of the regions may be matched, for example, four times in succession.

  The identification of the type of the multiple lane markings by the multiple lane marking type identification unit 14 is performed independently of the determination of the multiple lane marking line segments by the multiple lane marking segment determination unit 13 described above. It becomes effective. Further, the type of multiple lane markings is identified independently on the left and right sides in the far and near areas of the bird's-eye image data. Hereinafter, the concept of identifying the types of multiple lane markings will be described.

  Here, the lane boundary of the triple line (see FIG. 6A) is the middle marking line. The lane boundary of the right thick double line (see FIG. 6B) is the left marking line. The lane boundary of the left thick double line (see FIG. 6C) is the marking line on the right side. The lane boundary of both thick double lines (see FIG. 6 (d)) is the left marking line when it is on the left side with respect to the traveling direction of the vehicle, and the right side when it is on the right side with respect to the traveling direction of the vehicle. It is a marking line. The lane boundary of both thin double lines (see FIG. 6E) is the same as the lane boundary of both thick double lines. The lane boundary of the temporary service section (see FIG. 9) is a marking line (orange line) on both the left and right sides. The lane boundary is a white line that defines a lane (also referred to as a travel lane) for the vehicle to travel.

The triple line is only at the boundary between the lane and the lane, and is not at the boundary between the lane and the road shoulder. For this reason, the lane boundary (the middle marking line) of the triple line is always a broken line. Therefore, as a condition for discriminating the triple line section,
(Consecutive coincidence with the triple line template in the distance or in the vicinity several times) AND (lane boundary candidate is determined by broken line)
It becomes. The reason why the lane boundary candidate is selected is that the lane boundary is not yet officially determined at this point.

The right thick double line is never on the right shoulder. For this reason, when the right thick double line is in the right region with respect to the traveling direction of the vehicle, it is at the boundary between the lane and the lane. Line) becomes a broken line. Therefore, as a condition for discriminating the right thick double line section,
Left area: (Following the right thick double line template in the distance or near several times)
Right side area: (Consecutive matches with the thick double line template in the distance or near several times) AND (lane boundary candidate is determined by broken line)
It becomes.

The left thick double line is never on the left shoulder. For this reason, when the left thick double line is in the left region with respect to the traveling direction of the vehicle, it is at the boundary between the lane and the lane, so the lane boundary of the left double line (the thin sign on the right side) Line) becomes a broken line. Therefore, as a condition for discriminating the left thick double line section,
Left side area: (Consecutive matches with the left double line template in the distance or near several times) AND (lane boundary candidate is determined by broken line)
Right side area: (Consecutive matches with the left thick double line template in the distance or nearby)
It becomes.

Both thick double lines are only at the boundary between the shoulder and the lane. For this reason, when both thick double lines are in the left region with respect to the traveling direction of the vehicle, the lane boundary (the left marking line) becomes a solid line, and both thick double lines correspond to the traveling direction of the vehicle. In the right region, the lane boundary (right marking line) is a solid line. In addition, in the left region with respect to the traveling direction of the vehicle, even if the lane boundary (left marking line) is not a solid line, if it is not determined as a left thick double line, even if it overlaps with a right thick double line, Lane boundaries can be set. In addition, even if the lane boundary (right marking line) is not a solid line in the right region with respect to the traveling direction of the vehicle and is not determined to be a right thick double line, even if it overlaps with a left thick double line, Lane boundaries can be set. Therefore, as a condition for discriminating both thick double line sections,
Left region: (matching with both thick double line templates in the distance or in the vicinity several times in succession) AND {(lane boundary candidate is a solid line determination) OR (not to identify the left thick double line section in the past)}
Right area: (matching with both thick double line templates in the distance or in the vicinity several times) AND {(lane boundary candidate is a solid line determination) OR (not to identify right thick double line section in the past)}
It becomes.

Since both thin double lines are only at the boundary between the shoulder and the lane, they can be considered in the same way as the double thick double lines. Therefore, as a condition for discriminating both thin double line sections,
Left region: (matching with both thin double-line templates in the distance or in the vicinity several times in succession) AND {(lane boundary candidate is a solid line determination) OR (in the past, it is not a left-thick double line section determination)}
Right area: (Consecutive matches with both thin double line templates in the distance or in the vicinity several times) AND {(lane boundary candidate is a solid line determination) OR (in the past, right thick double line section determination)}
It becomes.

Since the provisional service section is a face-to-face section, it is limited to the right region with respect to the traveling direction of the vehicle. Further, in the face-to-face section, the lane boundary of the provisional service section (the marking lines on both outer sides) is a solid line. Therefore, as a condition for determining the provisional service section,
(Sequential matches with the hollow double line template in the far or near area in the right area several times in succession) AND (lane boundary candidate is a solid line judgment)
It becomes.

  That is, the identification of the triple line, the right thick double line, the left thick double line, and the provisional service unit is performed by identifying the determination result of the solid line broken line determination unit 12, the determination result of the multiple lane marking section determination unit 13, and the matching pattern. This is done based on the collation. Further, the identification of both the thick double lines and the both thin double lines is performed based on the comparison result of the determination result of the solid line broken line determination unit 12, the determination result of the multiple lane marking section determination unit 13, and the verification pattern. Or a comparison result between the multiple lane markings identified in the past (previous) and the type of multiple lane markings stored in the storage unit 16 in advance, the determination result of the multiple lane marking section determination unit 13, and a matching pattern This is based on the collation.

  Details of the processing procedure of the multiple lane marking type identification unit 14 based on this concept are shown in FIGS. FIG. 11 shows a processing procedure for the left region with respect to the traveling direction of the vehicle.

  In FIG. 11, first, the extracted data of multiple division lines is collated with a triplet template, and it is determined whether or not the coincidence between both has been continued a predetermined number of times (procedure 101). At this time, if the coincidence between the extracted data of the multiple lane markings and the triplet template continues for a predetermined number of times, the lane boundary candidate (the middle lane) of the extracted data of the multiple lane markings is determined based on the discrimination result of the solid line dashed line discriminating unit 12. It is determined whether the marking line is a broken line (procedure 102). When the lane boundary candidate is a broken line, it is identified as a triple line section (procedure 103). When it is determined in step 101 that the match between the extracted data of the multiple lane markings and the triplet template is not continuous a predetermined number of times, and in step 102, if it is determined that the lane boundary candidate is not a broken line, a triplet section Otherwise (step 104).

  In parallel with the processing of steps 101 to 104, the extracted data of the multiple dividing lines is collated with the right thick double line template, and it is determined whether or not the matching between both has been continued a predetermined number of times (step 105). At this time, when the match between the extracted data and the right bold double line template continues for a predetermined number of times, it is identified as a right thick double line section (procedure 106), and the match between the two does not continue for a predetermined number of times Is identified as not being a right thick double line section (step 107).

  Further, in parallel with the processing of steps 101 to 104, it is determined whether the lane boundary candidate (left side marking line) of the extracted data of the multiple segment lines is a broken line based on the determination result of the solid line broken line determination unit 12 (procedure) 108). At this time, when the lane boundary candidate is a broken line, the extracted data of the multiple dividing lines is collated with the left thick double line template, and it is determined whether or not the match between both is continued a predetermined number of times (procedure 109). When the match between the left thick double line template and the left thick double line template continues for a predetermined number of times, it is identified as a left thick double line section (step 110).

  If it is determined in step 109 that the match between the extracted data and the left thick double line template is not continued a predetermined number of times, the extracted data of the multiple dividing lines is checked against both the thin double line templates. It is determined whether or not it has continued for a predetermined number of times (procedure 111). When the match between the extracted data and both fine double line templates continues for a predetermined number of times, it is identified as a double thin line segment (procedure 112), and when the match between both does not continue for a predetermined number of times , It is discriminated that it is not a both-double-line section (step 113).

  In parallel with the processing of steps 111 to 113, the extracted data of the multiple dividing lines is collated with both thick double line templates, and it is determined whether or not the coincidence between both has been continued a predetermined number of times (step 114). When the match between the extracted data and both thick double line templates continues for a predetermined number of times, the extracted data is identified as a double line segment (procedure 115). Then, it is identified that it is not a double-thick double line section (procedure 116).

  When it is determined in step 108 that the lane boundary candidate of the extracted data of the multiple lane markings is not a broken line, the extracted data of the multiple lane markings is collated with both fine double-line templates, and whether or not the coincidence of both is continued a predetermined number of times. Is determined (procedure 117). When the match between the extracted data and both fine double line templates continues for a predetermined number of times, it is identified as a both thin double line section (procedure 118). , It is discriminated that it is not a double thin line section (procedure 119).

  In parallel with the processing of steps 117 to 119, the extracted data of the multiple dividing lines is collated with both thick double line templates, and it is determined whether or not the matching between both has been continued a predetermined number of times (step 120). When the match between the extracted data and the thick double line template is continued for a predetermined number of times, it is identified as a double thick double line section (procedure 121). , It is identified that it is not a double-thick double line section (procedure 122).

  FIG. 12 shows a processing procedure for the right region with respect to the traveling direction of the vehicle. In FIG. 12, first, the extracted data of multiple lane markings is collated with a triplet template, and it is determined whether or not a match between both is continued a predetermined number of times (procedure 131). At this time, if the match between the extracted data and the triple line template continues for a predetermined number of times, the lane boundary candidate (middle marking line) of the extracted data of the multiple segment lines is based on the determination result of the solid line broken line determination unit 12. It is determined whether it is a broken line (procedure 132). When the lane boundary candidate is a broken line, it is identified as a triple line section (procedure 133). When it is determined in step 131 that the match between the extracted data of the multiple lane markings and the triplet template is not continuous a predetermined number of times, and in step 132, if it is determined that the lane boundary candidate is not a broken line, a triplet section (Step 134).

  In parallel with the processing of steps 131 to 134, the extracted data of the multiple dividing lines is collated with the left thick double line template, and it is determined whether or not the matching between both has been continued a predetermined number of times (step 135). At this time, when the match between the extracted data and the left thick double line template continues for a predetermined number of times, it is identified as the left thick double line section (procedure 136), and the match between both does not continue for a predetermined number of times. Identifies that it is not a left thick double line section (procedure 137).

  Further, in parallel with the processing of steps 131 to 134, it is determined whether the lane boundary candidate (right marking line) of the extracted data of the multiple segment lines is a broken line based on the determination result of the solid line broken line determination unit 12 (procedure) 138). At this time, when the lane boundary candidate is a broken line, the extracted data of the multiple division lines is collated with the right thick double line template, and it is determined whether or not a predetermined number of matches between them has been continued (procedure 139). And the right thick double line template are identified as being a right thick double line section (step 140).

  If it is determined in step 139 that the extracted data does not match the right thick double line template a predetermined number of times, the extracted data of the multiple segment lines is checked against both thin double line templates. It is determined whether or not it has continued for a predetermined number of times (procedure 141). When the match between the extracted data and both fine double line templates continues for a predetermined number of times, it is identified as a both thin double line section (procedure 142). , It is discriminated that it is not the both thin double line section (procedure 143).

  In parallel with the processes of steps 141 to 143, the extracted data of the multiple segmentation lines is collated with both thick double line templates, and it is determined whether or not the match between both is continued a predetermined number of times (procedure 144). Then, when the match between the extracted data and both thick double line templates continues for a predetermined number of times, it is identified as a double thick double line section (procedure 145), and when both matches do not continue for a predetermined number of times Then, it is identified that it is not a double-thick double line section (procedure 146).

  If it is determined in step 138 that the lane boundary candidate of the extracted data of the multiple lane markings is not a broken line, the extracted data of the multiple lane markings is collated with both thin double-line templates, and whether or not the matching between both is continued a predetermined number of times Is determined (procedure 147). When the match between the extracted data and both fine double line templates continues for a predetermined number of times, it is identified as a double thin line segment (procedure 148). , It is discriminated that it is not the both thin double line section (procedure 149).

  In parallel with the processing of steps 147 to 149, the extracted data of the multiple dividing lines is checked against both thick double line templates, and it is determined whether or not the matching between both has been continued a predetermined number of times (step 150). Then, when the match between the extracted data and both thick double line templates continues for a predetermined number of times, it is identified as a both thick double line section (procedure 151), and when both matches do not continue for a predetermined number of times Then, it is identified that it is not a double-thick double line section (procedure 152).

  Further, in parallel with the processing of procedures 131 to 134, the extracted data of the multiple segment lines is collated with the hollow double template, and it is determined whether or not the matching between both has been continued a predetermined number of times (procedure 153). When the match between the extracted data and the hollow double-line template continues for a predetermined number of times, the extracted data is identified as a provisional service unit (procedure 154). It is identified that it is not a part (step 155).

  Note that, in such a multi-segment line type identifying process by the multi-segment line type identifying unit 14 as well, there are provisional determinations regarding the end of various multi-line segments, similar to the above-described multi-line segment determining process.

  In addition, since the determination as to whether or not it is a provisional service unit is only in the processing for the right region with respect to the traveling direction of the vehicle, the processing for the left region with respect to the traveling direction of the vehicle can be simplified.

  The lane boundary setting unit 15 sets the lane boundary of the multiple lane marking determined by the multiple lane marking type identification unit 14 using the determination result of the multiple lane marking section determination unit 13. The lane boundaries of the triple line, right thick double line, left thick double line, both thin double lines, both thick double lines, and the provisional service section are as described above.

  Here, the triplet template includes a doublet template. For this reason, triple lines and double lines (right thick double line, left thick double line, both thin double lines, both thick double lines) may overlap, and the setting of the lane boundary may become unstable. is there. Therefore, when it is determined that the triple line section and the double line section overlap each other, the lane boundary candidate in the triple line section is preferentially determined as the lane boundary. Further, when it is determined that the triple line section overlaps with the other multiple line sections, the lane boundary candidate in the triple line section is similarly given priority and determined as the lane boundary.

  Next, the recognition of the type of multiple lane markings by the travel path recognition device 2 configured as described above will be described with an example. Here, as shown in FIG. 13, the road on which the vehicle is traveling has a right thick double line in the left region with respect to the traveling direction of the vehicle, and a triple line in the right region with respect to the traveling direction of the vehicle. There shall be.

  The collation result between the extracted data of the multiple segment lines obtained from the captured image of the front camera 5 and the multiple line template is as shown in the table in FIG. As can be seen from this table, in the left region of the road, naturally, there are many matches between the extracted data of the multiple division lines and the right thick double line template. In the right region of the road, as a matter of course, there are many matches between the extracted data of the multiple lane markings and the triplet template. However, since the triple line has a shape including the right thick double line, in the right region of the road, there is a slight coincidence between the extracted data of the multiple segment lines and the right thick double line template.

The determination process of the multiple line segment by the multiple segment line segment determination unit 13 of the white line detection ECU 6 is performed as follows. That is, in the left area of the road, after the coincidence between the extracted data of the multiple lane markings and the right thick double line template continues three times in the far and near areas, the extracted data of the multiple lane markings and the right thick 2 in the neighboring area. in the timing T ₁ match with heavy line template has continued three times, multi-segment line on the left is determined to be a multi-line section. In the right area of the road, the match between the extracted data of the multiple segment line and the triplet template is continued three times in the neighboring region, and then the match between the extracted data of the multiple segment line and the triplet template is performed three times in the remote region. Subsequently timing T _2, the right side of the multi-lane marking is determined to be a multi-line section. In the determination processing of the multi-section line section, since it is determined that the multi-segment line section where uniform determination in the region of the left and right sides as described above, at the timing T _2, multiple division line of the road on which the vehicle is traveling Is determined to be a multiline section.

  The process of determining the line type of the multiple segment lines by the solid line broken line determination unit 12 of the white line detection ECU 6 is performed as follows. That is, as described above, the lane boundary of the right thick double line is the left marking line, and the lane boundary of the triple line is the middle marking line. For this reason, a thin line that is a lane boundary candidate for multiple division lines in the left region is determined to be a solid line, and a thin line that is a lane boundary candidate for multiple division lines in the right region is determined to be a broken line.

The identification process of the multiple lane marking sections by the multiple lane marking type identification unit 14 of the white line detection ECU 6 is performed as follows. That is, in the left region of the road, at the timing T ₃ match the extracted data and MigiFutoshi double line template multiple marking line was followed four times in the far region or neighboring region, multiple dividing line on the left MigiFutoshi double It is determined that it is a line section.

The right region of the road, matches the extracted data and the triplets template multiple partial lines followed four times in the far region or neighboring region at the timing T _4. In addition, the lane boundary candidate of the triple line is determined as a broken line as described above. Thus, at the timing T ₄ above, right multi lane marking is determined to be a triple line section.

Further, in the right region of the road, matches the extracted data and MigiFutoshi double line template multiplex section line in the far region or neighboring region at the timing T ₅ has continued four times, moreover lane of MigiFutoshi double line The candidate is determined as a broken line as described above. Thus, at the timing T ₅ above, right multi lane marking is determined to be a double line MigiFutoshi. That is, in the timing T _5, so that the right side of the multi-lane marking is determined to overlap with the triplet and MigiFutoshi double lines.

However, thereafter, since the mismatch between the extracted data and MigiFutoshi double line template multiplex section line in the far region and the adjacent region at the timing T ₆ it has continued twice, in the timing T _6, the right of the multiple lane marking It is tentatively determined that it is not the right thick double line section. At timing T ₇ , since the disparity between the extracted data of the multiple segment line and the right thick double line template continues three times in the far and near regions, the right multiple segment line at the timing T ₇ . Is clearly not a right thick double line section.

The lane boundary setting process by the lane boundary setting unit 15 is performed as follows. That is, until the timing T ₂ of multiplex dividing line is determined to be a multi-line section, the setting method of the lane boundary when it is determined not to be multi-line section (for details omitted) lane boundary is determined by. If it is determined that the lane marking is a lane segment, the lane boundary of the right double line is selected for the lane marking in the left area of the road, and the triple line is selected for the lane marking in the right area of the road. The lane boundary is selected. In the multiple division line in the right region, there is a part where the triple line and the right thick double line are overlapped and discriminated, but in this case, the lane boundary is determined with priority on the triple line. The lane boundary of the heavy line will be selected.

  By the way, in general, in white line recognition, since the edge of the white line is extracted with an error from the edge extraction accuracy of the image data, the imaging accuracy of the camera, etc., the thickness of the white line as the edge extraction data has a certain amount of error. Become. For this reason, in order to reliably recognize the white line, it is not necessary to extract only the white line of the original thickness, but to extract the white line that is slightly thicker than the original white line or that is slightly thinner than the original white line. It is necessary to consider the detection margin. Therefore, depending on how the detection margin is set, when a white line having an intermediate thickness between a thick white line and a thin white line is extracted, it may be determined that the white line is a thick white line or a thin white line. Thereby, for example, the multiple division line on the left side of the road may be discriminated by being overlapped with the left thick double line and the both thick double lines. In addition, the double line that is the original multiple division line may be identified as a triple line due to noise or the like.

  On the other hand, in the present embodiment, a line type that is information of a line type (solid line and broken line) of a multiple line template that is information on the combination (number and thickness) of each mark line that constitutes a multiple division line. A plurality of types of matching patterns including information are prepared in advance. Then, based on the extracted data of the multiple division lines obtained from the captured image of the front camera 5, the line type of each marking line constituting the multiple division line is determined, and whether or not it is a multiple division line section is determined. Therefore, the type of multiple division lines can be accurately identified.

  For example, even when there is a possibility that the multiple division line on the left side of the road is overlapped with the left thick double line and both thick double lines, it is possible to determine which of the multiple lines by determining whether the lane boundary is a solid line or a broken line. It is possible to accurately identify whether it is a dividing line. In addition, by determining whether the lane boundary is a solid line or a broken line, if the multiple division line is a triple line or a double line, and if the multiple division line is a double line, any type of double line is used. Whether it is a line or the like can be accurately identified.

  As a result, the lane boundary setting unit 15 can correctly set the lane boundary serving as a reference for lane keeping and lane departure prevention. In addition, for example, it is possible to correctly determine whether the route can be changed to the left lane. As a result, it becomes possible to effectively carry out steering assistance of the vehicle.

  In the above embodiment, the multiple lane marking section determination unit 13 and the multiple lane marking line type identification unit 14 use the multiple lane line template and the multiple lane line template in order to improve the accuracy of the multiple lane marking line segment determination and the multiple lane marking line type identification. Although it is determined whether or not the match is continued a plurality of times, the present invention is not particularly limited to this method, and the determination of the multi-segment line segment may be performed from the result of the match with the multi-line template only once.

  Moreover, although the travel path recognition apparatus 2 of the said embodiment is applied to the steering control system 1, this invention is applicable also to the other travel assistance system etc. which are mounted in a vehicle.

It is a schematic block diagram which shows the steering control system provided with one Embodiment of the traveling path recognition apparatus concerning this invention. It is a functional block diagram of white line detection ECU shown in FIG. It is a figure which shows the projective transformation of the image data implemented with the image process part shown in FIG. It is a conceptual diagram which shows the processing area | region of the bird's-eye image data shown in FIG. It is a figure which shows the concept which performs a solid line or a broken line discrimination | determination by the solid line broken line discrimination | determination part shown in FIG. It is a figure which shows the specific example of the kind of the multiple division line provided on the road. It is a figure which shows the various multi-line templates corresponding to the kind of multiple division line shown in FIG. It is a figure which shows the parameter at the time of collating with the triplet | line template shown in FIG. It is a figure which shows the provisional service part provided on the road. It is the schematic which shows the hollow double line template corresponding to the provisional service part shown in FIG. It is a flowchart which shows the detail of the discrimination | determination process procedure (left side area) by the multiple lane marking type discrimination | determination part shown in FIG. It is a flowchart which shows the detail of the discrimination | determination processing procedure (right side area | region) by the multiple lane marking type discrimination | determination part shown in FIG. It is a figure which shows an example of the kind of multiple division line drawn on the right-and-left both sides on the road. It is a table | surface and timing chart which show an example of the operation | movement which recognizes the kind of multiple division line with the traveling path recognition apparatus shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 2 ... Traveling path recognition apparatus, 6 ... White line detection ECU, 11 ... Multiple division line extraction part (multiple division line extraction means), 12 ... Solid line broken line determination part (determination means), 13 ... Multiple division line section determination part (determination means) ), 14 ... Multiple lane marking type identification unit (identification means), 15 ... Lane boundary setting unit (means for setting the lane boundary), 16 ... Storage unit (storage means).

Claims (9)

In the travel path recognition device for recognizing a multiple division line composed of a plurality of marking lines provided on the travel path based on a captured image of a travel path on which the vehicle travels,
A plurality of types of matching patterns including a template that is information of a combination of each marking line that constitutes the multiple dividing line and line type information that is information of the line type of each marking line that constitutes the multiple dividing line are stored in advance. Storage means to keep,
Multiple lane marking extraction means for extracting the multiple lane marking based on the captured image;
Discriminating means for discriminating the line type of each marking line constituting the extracted multiple segment line,
Determining means for determining the type of template corresponding to the extracted multiple dividing line;
A travel path recognition apparatus comprising: an identification unit that identifies a type of the multiple lane markings based on a discrimination result of the discrimination unit, a determination result of the determination unit, and a collation pattern. .   2. The travel path recognition apparatus according to claim 1, wherein the identification unit independently identifies the types of the multiple division lines in the left and right regions with respect to the traveling direction of the vehicle.   3. The travel path recognition apparatus according to claim 1, wherein the template is information on the number of marking lines constituting the multiple dividing lines and the thickness of each marking line.   4. The storage means stores a matching pattern corresponding to a triple line in which the number of the marking lines is three and the line type of each marking line is a broken line. The travel path recognition apparatus described.   The storage means has two marking lines, the line type of one marking line is a solid line or a broken line, the line type of the other marking line is a broken line, and the thickness thereof is 4. The travel path recognition apparatus according to claim 3, wherein a matching pattern corresponding to a double line having a different width that is thicker than the one-side marking line is stored.   The identification means is for the collation in which the line type of the one side marking line is a solid line depending on whether the type of the multiple dividing line is identified in the left or right region with respect to the traveling direction of the vehicle The travel path recognition apparatus according to claim 5, wherein pattern matching is not performed.   The storage means has two marking lines, the line type of one of the marking lines is one of a solid line or a broken line, the line type of the other marking line is the other of the broken line, and 4. The travel path recognition apparatus according to claim 3, wherein a matching pattern corresponding to a double line having the same width as the one side marking line is stored. The storage means stores a matching pattern corresponding to a hollow double line in which the number of the marking lines is two and the line types of the marking lines are both solid lines,
Said identifying means, according to claim 3, characterized in that in the identification of the multi-dividing line in the left area with respect to the traveling direction of the vehicle and does not match the collation pattern corresponding to the in missing double line The travel path recognition apparatus described. The travel path according to any one of claims 1 to 8, further comprising means for setting a lane boundary used for travel of the vehicle based on the type of the multiple lane marking identified by the identification means. Recognition device .

Images