JP2013186660A - Lane mark detection device and lane mark search range setting method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a "lane mark detection device and a lane mark search range setting method" for surely detecting a lane mark without using any information from the outside other than a photographic image even when the crosswise moving amounts of the lane mark projected in the photographic image are increased.SOLUTION: On the basis of the change of the position of a lane mark detected in the past from a photographic image input by an imaging device 20, the moving amounts per unit time of the lane mark are calculated, and on the basis of the position of the lane mark detected in the past and the moving amounts per unit time of the lane mark, the next position of the lane mark is estimated, and by using the estimated position as a reference, a search range in the case of detecting the lane mark is set so that even when the moving amounts of the lane mark projected in the photographic image are increased, the search range of the lane mark is set at a position under the consideration of the moving amounts, and the lane mark can be surely detected from the search range of the photographic image.

Description

  The present invention relates to a lane mark detection apparatus and a lane mark search range setting method, and is particularly suitable for use in an apparatus that detects a lane mark on a road by processing an image photographed by a camera.

  2. Description of the Related Art Conventionally, there has been provided a lane mark detection device that detects a lane mark (lane boundary line) on a road by processing an image captured by a camera. In this type of lane mark detection apparatus, generally, a certain search range is set in a captured image, and a lane mark is detected from the set search range. Various methods for setting the search range in this case have been proposed. For example, there is a technique for setting a range of a predetermined width on the left and right sides from the detected position as a reference range for the next timing based on the position of the lane mark detected at a certain timing.

  However, with such a technique, when the amount of lateral movement of the lane mark appearing in the captured image increases, the lane mark may not be detected within the search range of the next timing, and the lane mark may not be detected. There was a problem. As a case where the amount of movement of the lane mark in the lateral direction becomes large, the vehicle may change the lane at high speed. Even if the vehicle is changing lanes little by little, if a CPU with low processing capability is used, the time interval for capturing frames of captured images becomes long, and the amount of lateral movement of the lane mark between frames Becomes larger.

  In order to solve such a problem, it is conceivable to expand the search range. However, if the search range is expanded, a plurality of lane mark candidates are included in the search range, and the possibility of erroneous detection of lane marks increases. In addition, the processing time becomes longer due to the expanded search range. On the other hand, several techniques for setting the search range using a method different from setting the next search range based on the detected lane mark position have been proposed (see, for example, Patent Documents 1 to 3).

  In the technique described in Patent Document 1, in the setting of a window for performing image processing for white line detection, the moving amount of the vehicle is detected by a wheel speed sensor or the like, and the next window is set based on the moving amount of the vehicle. Yes. In the technique described in Patent Document 2, the horizontal position of the window set for white line detection is set to the position of a travel lane line (predicted travel trajectory) predicted based on past data, vehicle speed, and steering angle. ing. In the technique described in Patent Document 3, the search range in white line detection is arbitrarily changed according to road shape information such as a straight road or a curved road output from the navigation device.

Japanese Patent Laid-Open No. 11-339012 JP-A-6-225308 JP-A-9-35198

  However, the technique described in Patent Document 1 requires a wheel speed sensor or the like to set the search range. In the technique described in Patent Document 2, a steering angle sensor or the like is required to set a search range. Moreover, in the technique described in Patent Document 3, a navigation device is required to set a search range. For this reason, in the commercial lane mark detection device retrofitted to the vehicle, the output information of the wheel speed sensor, the steering angle sensor, and the navigation device cannot be used, and the techniques described in Patent Documents 1 to 3 can be applied. There was a problem that I could not.

  The present invention has been made to solve such a problem, and does not use external information other than the photographed image even when the lateral movement amount of the lane mark appearing in the photographed image becomes large. Another object of the present invention is to make it possible to reliably detect lane marks.

  In order to solve the above-described problems, in the present invention, a search range is set in a captured image input from an imaging device, and a lane mark detection device detects a lane mark from the set search range, by processing the captured image. Based on the obtained past lane mark detection result, the movement amount of the lane mark per unit time is calculated. The search range for detecting the lane mark is set based on the position of the lane mark detected in the past and the movement amount of the lane mark per unit time.

  According to the present invention configured as described above, even when the movement amount per unit time in the horizontal direction of the lane mark appearing in the captured image becomes large, the lane mark search range is set at a position that takes the movement amount into consideration. Therefore, the lane mark can be reliably detected from within the search range of the photographed image. In addition, since the movement amount is obtained based on the detection result of the past lane mark obtained by the processing of the captured image, the lane mark can be detected without using external information other than the captured image.

It is a block diagram which shows the function structural example of the lane mark detection apparatus by this embodiment. It is a figure for demonstrating the outline of operation | movement of the lane mark detection apparatus by this embodiment. It is a flowchart which shows the operation example of the lane mark candidate extraction part by this embodiment. It is a figure for demonstrating the extraction process of the slice performed by the lane mark candidate extraction part of this embodiment. It is a figure for demonstrating the grouping process of the slice performed by the lane mark candidate extraction part of this embodiment. It is a figure for demonstrating the extraction process of the line segment performed by the lane mark candidate extraction part of this embodiment. It is a flowchart which shows the initialization process of the lane mark detection part performed when starting the lane mark detection apparatus of this embodiment. It is a figure which shows an example of the lane mark information by this embodiment. It is a flowchart which shows the lane mark detection process by the lane mark detection part of this embodiment. It is a flowchart which shows the processing time calculation process performed by the lane mark detection part of this embodiment. It is a flowchart which shows the lane change process performed by the lane mark detection part of this embodiment. It is a flowchart which shows the matching process of the lane mark performed by the lane mark detection part of this embodiment. It is a flowchart which shows the lane mark selection process performed by the lane mark detection part of this embodiment. It is a figure for demonstrating an example of the search range setting process performed by the lane mark detection part of this embodiment. It is a flowchart which shows the operation example of the lane mark position estimation part by this embodiment. It is a flowchart which shows the estimation process of the lane mark performed by the lane mark position estimation part of this embodiment.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a functional configuration example of a lane mark detection apparatus 10 according to the present embodiment. As shown in FIG. 1, the lane mark detection device 10 of this embodiment is connected to an imaging device 20 such as a camera. The imaging device 20 is a rear camera installed behind the vehicle, for example.

  The lane mark detection apparatus 10 includes a lane mark candidate extraction unit 1, a lane mark detection unit 2, and a lane mark position estimation unit 3 as its functional configuration. Each of these functional blocks 1 to 3 can be realized by any of a hardware configuration, a DSP, and software. For example, when realized by software, each of the functional blocks 1 to 3 is actually configured by including a computer CPU or MPU, RAM, ROM, and the like, and can be realized by operating a program stored in the RAM or ROM. .

  The lane mark candidate extraction unit 1 extracts lane mark candidates from the captured image input from the imaging device 20. The lane mark candidate is specified as a portion having a possibility of a lane mark based on the difference in luminance or shape of the subject in the captured image. Details regarding the lane mark candidate extraction process will be described later.

  The lane mark detection unit 2 sets a search range in the captured image, and detects a lane mark from the set search range. Specifically, the lane mark detection unit 2 detects a lane mark from lane mark candidates within the search range set in the captured image. In the present embodiment, the lane mark detection unit 2 sets a search range based on the position of the lane mark estimated by the lane mark position estimation unit 3, and detects a lane mark from within the search range.

  In addition, when the lane mark candidate is within the search range, the lane mark detection unit 2 detects the lane mark candidate as a lane mark, and when the lane mark candidate is not within the search range, the lane mark position is detected. It is assumed that a lane mark is detected at the lane mark position estimated by the estimation unit 3. The details of the lane mark detection processing by the lane mark detection unit 2 will be described later.

  The lane mark position estimation unit 3 corresponds to the position estimation unit in the claims, and includes a movement amount calculation unit in the claims. The movement amount calculation unit calculates a movement amount (movement speed in the Y-axis direction) per unit time in the horizontal direction of the lane mark based on the detection result of the past lane mark by the lane mark detection unit 2. The lane mark position estimation unit 3 determines the next unit time based on the past lane mark position detected by the lane mark detection unit 2 and the movement amount per unit time of the lane mark calculated by the movement amount calculation unit. Estimate the lane mark position later. The details of the lane mark position estimation process will be described later.

FIG. 2 is a diagram for explaining the outline of the operation of the lane mark detection apparatus 10 according to the present embodiment. FIG. 2A shows the position of the lane mark detected from the captured image at a certain time t. The distance from the vehicle center to the left lane mark at time t is D _Lt , and the distance from the vehicle center to the right lane mark is D _Rt . In this embodiment, the position of the lane mark is represented by a distance from the vehicle center.

FIG. 2B shows the position of the lane mark (distance from the vehicle center) detected from the captured image at time t + d (d: unit time) after the next unit time. The distance from the vehicle center to the left lane mark at time t + d is D _{Lt + d} , and the distance from the vehicle center to the right lane mark is D _{Rt + d} . FIG. 2C shows the position of the lane mark detected from the captured image at time t + 2d after the next unit time.

When the lane mark detection device 10 detects a lane mark at time t + 2d, the movement amount calculation unit of the lane mark position estimation unit 3 detects the lane mark position D _Lt , D detected by the lane mark detection unit 2 at time t. _{Based on Rt} and the lane mark positions D _{Lt + d} and D _{Rt + d} detected at time t + d, the movement amount per unit time in the lateral direction of the lane mark is calculated. In this case, the amount of movement of the left lane mark per unit time is (D _Lt −D _{Lt + d} ) / {(t + d) −t} in the direction in which the vehicle approaches the left lane mark. On the other hand, the movement amount of the right lane mark per unit time is (D _{Rt + d} −D _Rt ) / {(t + d) −t} in the direction in which the vehicle moves away from the right lane mark.

The lane mark position estimation unit 3 includes the lane mark positions D _{Lt + d} and D _{Rt + d} detected by the lane mark detection unit 2 at time t + d, and the lane mark movement amount per unit time calculated by the movement amount calculation unit (D _{Based on Lt−} D _{Lt + d} ) / {(t + d) −t} and (D _{Rt + d} −D _Rt ) / {(t + d) −t}, the lane mark position at time t + 2d is estimated.

Specifically, the amount of movement of the left lane mark per unit time (D _Lt −D _{Lt + d} ) / {(t + d) −t} × d is subtracted from the position D _{Lt + d} of the left lane mark detected at time t + d. The position obtained by this is the estimated position of the left lane mark at time t + 2d. Further, it is obtained by adding the movement amount (D _{Rt + d} −D _Rt ) / {(t + d) −t} × d per unit time of the right lane mark to the position D _{Rt + d} of the right lane mark detected at time t + d. This position is the estimated position of the right lane mark at time t + 2d. In FIG. 2C, 21 _L is the estimated position of the left lane mark, and 21 _R is the estimated position of the right lane mark.

The lane mark detection unit 2 sets a search range for the captured image input from the imaging device 20 at time t + 2d with reference to the lane mark positions 21 _L and 21 _R estimated by the lane mark position estimation unit 3, and the search is performed. A lane mark is detected from within the range. Specifically, as shown in FIG. 2 (c), to set the search range ₂₂ L, 22 _R of a predetermined width around the estimated position ₂₁ L, 21 _R of the lane mark, the search range ₂₂ L, 22 _R Left and right lane marks are detected from within. In the present embodiment, the lane mark detection unit 2 changes the size of the predetermined width set as the search range in accordance with the amount of movement of the lane mark per unit time calculated by the movement amount calculation unit. (This will be described in detail later).

FIG. 2D is a conventional operation example shown for comparison with the operation example of the present embodiment shown in FIG. As shown in FIG. 2 (d), conventionally, setting the search range ₂₃ L, 23 _R of a predetermined width around the detection position _{D Lt + d, D Rt +} d of the lane mark at time t + d, the search range ₂₃ L, 23 Left and right lane marks were detected in _R. Therefore, if the amount of movement of the lane mark per unit time (D _Lt −D _{Lt + d} ) / {(t + d) −t}, (D _{Rt + d} −D _Rt ) / {(t + d) −t} is large, the set search range In some cases, no lane mark is entered in 23 _L and 23 _R , and the lane mark cannot be detected.

On the other hand, according to the present embodiment shown in FIG. 2C, the amount of movement per unit time in the horizontal direction of the lane mark (D _Lt −D _{Lt + d} ) / {(t + d) −t}, (D _{Rt + d} − Even if D _Rt ) / {(t + d) −t} becomes large, the lane mark search range 22 _L , 22 _R is set at a position that takes the movement amount into consideration, so the captured image search range 22 _L , 22 Lane marks can be reliably detected from within _R.

<Processing contents of lane mark candidate extraction unit 1>
Below, the processing content by the lane mark candidate extraction part 1 is demonstrated concretely.

  FIG. 3 is a flowchart illustrating an operation example of the lane mark candidate extraction unit 1 according to the present embodiment. First, the lane mark candidate extraction unit 1 generates a grayscale luminance image from the captured image input from the imaging device 20 (step S1). Then, as shown in FIG. 4, the lane mark candidate extraction unit 1 scans the luminance image in the horizontal direction, and extracts the horizontal portion where the luminance equal to or higher than the threshold continues for a certain interval as a line (step S2). This is performed on the entire captured image. Each extracted line is called a “slice”.

  Next, as shown in FIG. 5, the lane mark candidate extraction unit 1 has a plurality of coordinate values in the horizontal direction that overlap each other in the slices adjacent in the vertical direction with respect to each extracted slice. The slices are grouped (step S3). In the example shown in FIG. 5, three groups 1 to 3 of slices are extracted from the captured image.

  Further, as shown in FIG. 6, the lane mark candidate extraction unit 1 projects the slices of the groups 1 to 3 onto the road surface (Z = 0) of the three-dimensional coordinates, and then uses the least square method from the midpoint of each slice. In step S4, the linear expression Y = aX + b (Z = 0) of the line segment as the lane mark candidate is extracted. Here, a is the slope of the line segment, and b is the Y intercept of the line segment. The projection of the three-dimensional coordinates onto the road surface means that the captured image input from the imaging device 20 is converted into an image from directly above viewed from a virtual viewpoint above the vehicle. The origin of the three-dimensional coordinates is the position of the vehicle center. In addition, the front of the vehicle is positive on the X axis, the left side of the vehicle is positive and the right side is negative on the Y axis.

  The line expression extracted from group 1 of the slice is Y = a1X + b1, the line expression extracted from group 2 is Y = a2X + b2, and the line expression extracted from group 3 is Y = a3X + b3. is there.

  Here, the lane mark candidate extraction unit 1 includes a plurality of linear expressions that fall within a certain range of values of the inclination a and the Y intercept b among the linear expressions calculated for the groups 1 to 3 as described above. If there are, groups corresponding to those linear expressions are combined into one. In the example of FIG. 6, group 2 and group 3 are combined into one group. Then, the lane mark candidate extraction unit 1 again extracts one line segment from the information of each slice included in the grouped group. As a result, the linear expression of the combined line segments is Y = a4X + b4.

  The lane mark candidate extraction unit 1 extracts line segments from the left end and right end of each slice by the least square method in the same manner as the line segment is extracted from the midpoint of each slice. Thereby, a line segment parallel to the line segment extracted from the midpoint of the slice is drawn on both sides, and the slice width is calculated. In the example of FIG. 6, the slice width is calculated for the line segment Y = a1X + b1 of the group 1, and the slice width is calculated for the line segment Y = a4X + b4 of one group in which the groups 2 and 3 are combined.

Next, the lane mark candidate extraction unit 1 calculates the reliability indicating the certainty as the lane mark for each line segment extracted as described above (step S5). There are various methods for calculating the reliability, but here, the reliability is calculated from the variation of the midpoint of each slice. That is, the line segment reliability (value of 0 to 10,000) is calculated by the following equation.
Reliability = 10000− (average of absolute value of distance from straight line formula of line segment to midpoint of each slice / threshold of distance error) × 10000

  As a method for calculating the reliability, it is also possible to use the aspect ratio of the lane mark, the variation of the left and right ends of the slice, the number of slices of the group, and the like, or a combination thereof.

<Processing contents of lane mark detection unit 2>
Next, the processing content by the lane mark detection unit 2 will be specifically described.

  FIG. 7 is a flowchart showing an initialization process of the lane mark detection unit 2 executed when the lane mark detection apparatus 10 is started. In the initialization process, the lane mark detection unit 2 sets the left lane mark information and the right lane mark information to “none” (steps S11 and S12). The lane mark information is information including a linear expression Y = aX + b indicating the position of the lane mark, the width of the lane mark, and the reliability of the lane mark, as shown in FIG. In the present embodiment, it is assumed that the lane mark information stores detection results for up to the past 10 times.

  Further, the lane mark detection unit 2 clears the final time and the processing time used for measuring the processing time to 0 (steps S13 and S14). Note that the processing time refers to a unit time (d) for processing a captured image input from the imaging device 20. The final time is the latest time when the lane mark is detected last.

  FIG. 9 is a flowchart showing lane mark detection processing by the lane mark detection unit 2. This lane mark detection process is performed using the left and right lane mark information detected in the past and the lane mark candidate line segment extracted by the lane mark candidate extraction unit 1. In FIG. 9, the lane mark detection unit 2 first calculates the processing time (step S21). Details of the processing time calculation processing in step S21 are shown in FIG.

  In FIG. 10, the lane mark detection unit 2 first acquires the current time (step S2101). Next, it is determined whether or not the last time at which the lane mark was last detected is set (step S2102). If set, the processing time is calculated by subtracting the final time from the current time (step S2103). . Then, the setting of the final time is updated with the current time (step S2104), and the processing time calculation process ends. On the other hand, if the final time has not yet been set immediately after the initialization process of FIG. 7 is performed, the current time is set as the final time without performing the process of step S2103 (step S2104).

  In FIG. 9, after performing the processing time calculation process of step S21, the lane mark detection unit 2 performs a lane change process (step S22). In the lane change process, the left lane mark information and the right lane mark information are exchanged when a lane change occurs in which the vehicle moves across the lane mark. For example, when the vehicle changes lanes to the left lane, the lane mark on the left side of the vehicle before the lane change is positioned on the right side of the vehicle after the lane change. Therefore, it is necessary to change the left lane mark information to the right lane mark information. Details of such a lane change process are shown in FIG.

  In FIG. 11, the lane mark detection unit 2 first determines the Y intercept b ′ of the estimation result Y = a′X + b ′ of the left lane mark (obtained by the lane mark position estimation unit 3 as described later). It is determined whether the value is negative (step S2201). If the Y intercept b 'is a negative value, the estimated position of the left lane mark is on the right side of the vehicle center, so it is determined that the vehicle has changed the lane to the left lane. In this case, the lane mark detection unit 2 resets the current left lane mark information as the right lane mark information (step S2202), sets the left lane mark information to “none” (step S2203), and performs the lane change process. finish.

  On the other hand, when it is determined that the Y intercept b ′ of the next position of the left lane mark to be estimated is not a negative value, the lane mark detection unit 2 estimates Y of the next position of the right lane mark Y = a′X + b ′. It is determined whether the intercept b ′ is a value greater than or equal to 0 (step S2204). If the Y intercept b 'is 0 or more, the estimated position of the right lane mark is on the left side of the vehicle center, so it is determined that the vehicle has changed to the right lane. In this case, the lane mark detection unit 2 resets the current right lane mark information as the left lane mark information (step S2205), sets the right lane mark information to “none” (step S2206), and performs lane change processing. finish.

  Note that the Y intercept b ′ of the estimated position of the left lane mark is not a negative value (the vehicle has not changed the lane to the left lane), and the Y intercept b ′ of the estimated position of the right lane mark is not 0 or more. When the vehicle has not changed the lane to the right lane, the lane mark detection unit 2 ends the lane change process without performing any replacement of the left and right lane mark information.

  In FIG. 9, after finishing the lane change process of step S22, the lane mark detection unit 2 determines the presence or absence of left and right lane mark information (steps S23, S26, S29), and the lane mark information according to the determination result. Update processing is performed. That is, the lane mark detection unit 2 first determines whether or not there is both left and right lane mark information (step S23). If it is determined that both are present, a line of lane mark candidates that match the left lane mark information. The minute is searched as the left lane mark (step S24), and the line segment of the lane mark candidate matching the right lane mark information is searched as the right lane mark (step S25).

  If it is determined that both the left and right lane mark information are not present, the lane mark detection unit 2 next determines whether there is only the left lane mark information (step S26). When it is determined that only the left lane mark information exists, the lane mark detection unit 2 searches the left lane mark for the line segment of the lane mark candidate that matches the left lane mark information (step S27), and the left lane mark information The right lane mark is selected from the line segments of the lane mark candidates in consideration of the reference lane width based on (Step S28). The reference lane width is a predetermined value, for example, 3.5 m in the case of an expressway.

  On the other hand, when determining that there is no left lane mark information, the lane mark detection unit 2 determines whether there is only the right lane mark information (step S29). When it is determined that only the right lane mark information exists, the lane mark detection unit 2 searches the right lane mark information for a lane mark candidate line segment that matches the right lane mark information (step S30), and the right lane mark information. The left lane mark is selected from the lane mark candidate line segments in consideration of the reference lane width based on (step S31).

  If it is determined that neither the right lane mark information nor the left lane mark information is present, the lane mark detection unit 2 selects the left and right lane marks from the lane mark candidate line segments (steps S32 and S33). That is, in step S32, the lane mark detection unit 2 selects a left lane mark from line segments of lane mark candidates included in a certain range on the left side of the vehicle center.

  When the left lane mark information can be set in step S32, the lane mark detection unit 2 sets the right lane mark to the lane mark in the next step S33 in consideration of the reference lane width based on the set left lane mark information. Select from candidate line segments. On the other hand, if the left lane mark information is not set in step S32, the lane mark detection unit 2 in the next step S33, the right lane from the line segments of the lane mark candidates included in a certain range on the right side of the vehicle center. Select a mark.

  After the lane mark information update process as described above, the lane mark detection unit 2 outputs the lane mark information (step S34). Thus, the lane mark detection shown in the flowchart of FIG.

FIG. 12 is a flowchart showing details of the lane mark matching process performed in steps S24, S25, S27, and S30 shown in FIG. In FIG. 12, the lane mark detection unit 2 first calculates a search width that determines the size of the search range (two search ranges 22 _L and 22 _{R in} the example of FIG. 2C) set in the captured image. (Step S41).

  In the present embodiment, the lane mark detection unit 2 calculates the movement calculated from the movement speed (movement amount per unit time) of the lane mark in the Y-axis direction and the processing time (unit time d) obtained in step S2103 in FIG. The search width is calculated by adding a predetermined search width reference value to the quantity. For example, the lane mark detection unit 2 calculates a search width by a calculation formula of Y-axis direction moving speed × processing time × coefficient + search width reference value. Thus, in this embodiment, the size of the search range is changed according to the moving speed of the lane mark in the Y-axis direction.

Next, the lane mark detection unit 2 searches for a line segment satisfying a predetermined condition from the line segments of the lane mark candidate extracted by the lane mark candidate extraction unit 1 (step S42). In other words, the lane mark detection unit 2 uses a linear Y-intercept b ′ ± search width search range (in the example of FIG. 2C) where the linear Y-intercept b of the lane mark candidate represents the estimated position of the lane mark. The line segments having the highest reliability are searched for in the search ranges 22 _L and 22 _R ) set with the search widths on both sides centering on the estimated positions 21 _L and 21 _R of the left and right lane marks.

  The lane mark detection unit 2 determines whether or not the corresponding line segment is found by the search in step S42 (step S43). If there is a corresponding line segment, it is a lane mark. Therefore, a straight line formula Y = aX + b is added to the past lane mark position of the lane mark information (step S44), and the line width of the line segment is added to the past lane mark width. The line segment reliability is added to the past reliability (step S46). When adding these pieces of information, if the storage destination is full, the oldest data is deleted and added.

  On the other hand, when the corresponding line segment is not found in the search in step S42, the lane mark detection unit 2 adds 0 to the past reliability of the lane mark information (step S47). Then, the lane mark detection unit 2 calculates an average value of the data stored in the past reliability and determines whether it is 0 (step S48). When the average value of the past reliability is 0, it means that the lane mark has not been found ten times in succession, so the lane mark detection unit 2 sets the lane mark information to “none” (step S51).

  When the average value of the past reliability is not 0, the lane mark detection unit 2 regards the position estimated by the lane mark position estimation unit 3 as the lane mark position, and sets the lane mark information based on the estimated position. That is, the lane mark detection unit 2 adds a straight line expression Y = aX ′ + b ′ representing the estimated position of the lane mark to the past lane mark position of the lane mark information (step S49), and the lane mark information A value (latest value) last set for the past lane mark width is added to the past lane mark width (step S50).

  FIG. 13 is a flowchart showing details of the lane mark selection process performed in steps S28, S31, S32, and S33 shown in FIG. In FIG. 13, the lane mark detection unit 2 first checks whether there is lane mark information on the opposite side (step S61). That is, when the right lane mark is selected, it is confirmed whether there is left lane mark information. When the left lane mark is selected, it is confirmed whether there is right lane mark information.

  As shown in FIG. 14, when there is lane mark information on the opposite side, the search range is determined based on this information. In other words, the lane mark detection unit 2 adds a predetermined search width reference value to the movement amount calculated from the movement speed in the Y-axis direction of the lane mark obtained from the lane mark information on the opposite side and the processing time. Is calculated (step S62). In this case, the search width reference value is preferably set to a value larger than the search width reference value used in step S41. This is because the search width is calculated from the lane mark information on the opposite side, which may include an error.

Next, the lane mark detection unit 2 uses the Y-intercept of the lane mark indicated by the lane mark information on the opposite side (the linear Y-intercept b added in step S44 of FIG. 12 or the linear-intercept added in step S49). minus the reference lane width W of the absolute value of the Y intercept b ') is obtained since the estimated position of the lane mark (24 R in FIG. _14), from the search range (estimated position 24 _R of the position ± search width Search range 1 of the vehicle closer, the vehicle from the estimated position 24 _R for setting search range 2) on the opposite side (step S63, S64).

  On the other hand, if it is determined in step S61 that there is no lane mark information on the opposite side, the lane mark detection unit 2 takes the reference range 1 from the position to the vehicle centering on a position at a predetermined distance from the vehicle center. The search range 1 near the vehicle is set (step S65), and the reference range 2 (reference range 1 <reference range 2) is set on the opposite side of the vehicle from the position to set the search range 2 on the opposite side (step S65). S66). Note that the values of the reference range 1 and the reference range 2 may be arbitrarily determined, for example, 1.1 m and 3.0 m.

  After setting the search range in any of steps S62 to S64 or steps S65 to S66, the lane mark detection unit 2 determines whether the lane mark information to be obtained is information related to the right lane mark (step S67). In the case of the information related to the right lane mark, the Y coordinate of the three-dimensional coordinate is negative, so the lane mark detection unit 2 sets the search range to a negative value (steps S68 and S69).

  Next, the lane mark detection unit 2 searches for a line segment satisfying a predetermined condition from the line segments of the lane mark candidate extracted by the lane mark candidate extraction unit 1 (step S70). That is, the lane mark detection unit 2 searches for a line segment having the highest reliability with the linear Y-intercept b of the lane mark candidate between the search range 1 and the search range 2.

  The lane mark detection unit 2 determines whether the corresponding line segment is found by the search in step S70 and the reliability of the line segment is equal to or higher than the selection reference value (step S71). If there is a corresponding line segment and the reliability of the line segment is equal to or greater than the selection reference value, the lane mark detection unit 2 selects it as a new lane mark. That is, the lane mark detection unit 2 newly sets a straight line formula Y = aX + b at the past lane mark position of the lane mark information (step S72), and newly sets the line width of the line segment as the past lane mark width ( In step S73), the reliability of the line segment is newly set in the past reliability (step S74).

  On the other hand, when the corresponding line segment is not found in the search in step S70, or when the reliability of the line segment is less than the selection reference value, the lane mark detection unit 2 sets “None” as the lane mark information to be obtained. (Step S75). The lane mark selection process shown in FIG. 13 is terminated by the process of steps S72 to S74 or step S75.

<Processing contents of lane mark position estimation unit 3>
Below, the processing content by the lane mark position estimation part 3 is demonstrated concretely.

  FIG. 15 is a flowchart showing an operation example of the lane mark position estimation unit 3 according to the present embodiment. The lane mark position estimation unit 3 calculates the moving speed in the Y-axis direction based on the lane mark information that is the detection result of the lane mark by the lane mark detection unit 2, and also determines the next position of the lane mark (linear equation Y = a 'X + b') is estimated.

  Specifically, as shown in FIG. 15, the moving speed in the Y-axis direction is calculated based on the left lane mark information that is the detection result of the lane mark by the lane mark detection unit 2, and the next position of the left lane mark Is estimated (step S81). The lane mark position estimation unit 3 calculates the moving speed in the Y-axis direction based on the right lane mark information that is the detection result of the lane mark by the lane mark detection unit 2, and estimates the next position of the right lane mark. (Step S82).

  FIG. 16 is a flowchart showing details of the processes in steps S81 and S82. First, the lane mark position estimation unit 3 determines whether there is no lane mark information (step S91). If there is no lane mark information, it is impossible to estimate the next position of the lane mark or to calculate the moving speed in the Y-axis direction, so the processing of the flowchart shown in FIG.

  On the other hand, when there is lane mark information, the lane mark position estimation unit 3 determines whether the number of data set in the past lane mark position included in the lane mark information is one (step S92). If the number of data is 1, this means that the lane mark detection unit 2 has newly detected a lane mark. In this case, the lane mark position estimation unit 3 initializes the moving speed of the lane mark information in the Y-axis direction to 0 (step S93), and also shows a linear inclination a ′ representing the estimated next position of the lane mark and The Y intercept b ′ is initialized to a linear slope a and Y intercept b representing the newly detected lane mark (steps S94 and S95).

  If the number of data set in the past lane mark position is not one, the lane mark position estimation unit 3 determines whether the number of data is two (step S96). When the number of data is two, the lane mark position estimation unit 3 calculates the movement speed of the lane mark in the Y-axis direction based on the Y intercept b of the two past lane mark positions and the processing time (step S97). ). Specifically, the movement speed of the lane mark in the Y-axis direction is calculated by the calculation of (Y intercept b of the latest past lane mark position-Y intercept b of the previous lane mark position immediately before) / processing time.

  Further, the lane mark position estimation unit 3 calculates a change rate of the inclination based on the inclinations a of the two past lane mark positions (step S98). More specifically, the rate of change of the lane mark inclination is calculated by the calculation of (the latest inclination of the previous lane mark position a-1 the inclination of the previous lane mark position immediately before) / processing time.

  Next, the lane mark position estimation unit 3 estimates the next position of the lane mark (linear equation Y = a′X + b ′) using the values obtained in steps S97 and S98. Specifically, the Y intercept b 'of the next position of the estimated lane mark is calculated by the calculation of the movement speed of the latest past lane mark position in the Y-intercept b + Y-axis direction x the processing time (step S101). Further, the inclination a 'of the next position of the estimated lane mark is calculated by the calculation of the latest past lane mark position inclination a + inclination change rate x processing time (step S102).

  If it is determined in step S96 that the number of data set in the past lane mark position is not two, it means that three or more data are stored in the past lane mark position. In this case, the lane mark position estimation unit 3 calculates the change rate of the moving speed and the inclination in the Y-axis direction based on the latest data of the past lane mark position, the oldest data, and the processing time.

  Specifically, the lane mark position estimation unit 3 calculates the lane mark by calculating (Y intercept b of the latest past lane mark position−Y intercept b of the oldest past lane mark position) / (processing time × n). A moving speed in the Y-axis direction is calculated (step S99). Note that n = the number of data of the past lane mark position−1. Further, the lane mark position estimation unit 3 calculates the rate of change of the lane mark inclination by a calculation of (latest past lane mark position inclination a−oldest past lane mark position inclination a) / (processing time × n). Calculate (step S100).

  Thereafter, the lane mark position estimation unit 3 estimates the next position of the lane mark (linear equation Y = a′X + b ′) using the values obtained in steps S99 and S100 (steps S101 and S102). Thus, the lane mark estimation process shown in FIG. Even when three or more pieces of data are stored at the past lane mark position, the processing of steps S97 and S98 may be performed instead of the processing of steps S99 and S100.

  As described above in detail, in the present embodiment, a search range is set in a captured image input from the imaging device 20, and the lane mark detection device 10 that detects a lane mark from the set search range detects from the captured image. Based on the change in the position of the lane mark detected in the past, the movement amount of the lane mark per unit time (movement speed in the Y-axis direction) is calculated. The next position of the lane mark is estimated based on the calculated movement speed in the Y-axis direction and the position of the lane mark detected in the past, and the search range is set based on the estimated position.

  According to the lane mark detection device 10 of the present embodiment configured as described above, even when the driver moves the steering wheel suddenly, for example, even when the movement amount per unit time of the lane mark that appears in the captured image increases, Since the lane mark search range is set at a position in consideration of the movement amount, the lane mark can be reliably detected from within the captured image search range. In addition, since the movement amount is obtained based on the detection result of the past lane mark obtained by the processing of the captured image, the lane mark can be detected without using external information other than the captured image.

  In the above embodiment, the lane mark detection unit 2 estimates the next lane mark position from the previously detected lane mark position, and sets the search range based on the estimated position. Is not limited to this. For example, the lane mark detection unit 2 temporarily sets the search range based on the position of the lane mark detected in the past. Thereafter, the position of the temporarily set search range may be corrected based on the movement amount per unit time of the lane mark calculated by the movement amount calculation unit. Even in this case, the search range finally set is the same as the case where the estimated position of the lane mark is set as a reference.

  In the above-described embodiment, the lane mark candidate extraction unit 1 extracts line segments that are lane mark candidates from the entire captured image input from the imaging device 20. Then, the lane mark detection unit 2 detects the lane mark candidate within the search range set from the estimated lane mark position calculated by the lane mark position estimation unit 3 and the search width calculated by the lane mark detection unit 2. Although an example in which a lane mark is detected from a line segment is described, the present invention is not limited to this. For example, the lane mark candidate extraction unit 1 includes a lane mark candidate within a search range set from the estimated lane mark position calculated by the lane mark position estimation unit 3 and the search width calculated by the lane mark detection unit 2. A line segment may be extracted.

  Moreover, in the said embodiment, although the line segment used as a lane mark candidate is calculated by the linear equation (Y = aX + b), this invention is not limited to this. For example, the lane mark may be expressed by a quadratic expression in consideration of the fact that there is a lane mark drawn on a curved curve of the road.

  In addition, each of the above-described embodiments is merely an example of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the gist or the main features thereof.

DESCRIPTION OF SYMBOLS 1 Lane mark candidate extraction part 2 Lane mark detection part 3 Lane mark position estimation part 10 Lane mark detection apparatus 20 Imaging device

Claims (6)

A lane mark detection unit configured to set a search range in a captured image input from the imaging apparatus and detect a lane mark from the set search range;
A movement amount calculation unit that calculates a movement amount per unit time of the lane mark based on the detection result of the past lane mark by the lane mark detection unit;
The lane mark detection unit detects the lane mark when detecting the lane mark based on the position of the lane mark detected in the past and the movement amount of the lane mark per unit time calculated by the movement amount calculation unit. A lane mark detection device that sets a range. Based on the previous lane mark position detected by the lane mark detection unit and the movement amount per unit time of the lane mark calculated by the movement amount calculation unit, the lane mark position after the next unit time is determined. A position estimation unit for estimation;
The lane mark detection device according to claim 1, wherein the lane mark detection unit sets the search range based on a next lane mark position estimated by the position estimation unit. A lane mark candidate extraction unit that extracts lane mark candidates from the captured image;
The lane mark detection unit is configured to detect the lane mark from the lane mark candidates within the search range set in the captured image.
When the lane mark candidate is within the search range, the lane mark detection unit detects the lane mark candidate as the lane mark, whereas when the lane mark candidate is not within the search range, The lane mark detection apparatus according to claim 2, wherein the lane mark is regarded as being detected at the lane mark position estimated by the position estimation unit. The lane mark detection unit temporarily sets the search range when the lane mark is detected after the next unit time on the basis of the position of the lane mark detected in the past, and is calculated by the movement amount calculation unit. The lane mark detection apparatus according to claim 1, wherein the position of the temporarily set search range is corrected based on a movement amount of the lane mark per unit time. 2. The lane mark according to claim 1, wherein the lane mark detection unit changes a size of the search range according to a movement amount of the lane mark per unit time calculated by the movement amount calculation unit. Mark detection device. A search range setting method for a lane mark in a lane mark detection device for setting a search range in a captured image input from an imaging device and detecting a lane mark from the set search range,
The movement amount calculation unit of the lane mark detection device calculates a movement amount per unit time of the lane mark based on a past detection result of the lane mark by the lane mark detection unit of the lane mark detection device. Steps,
The search range setting unit of the lane mark detection device is based on the position of the lane mark detected in the past by the lane mark detection unit and the movement amount per unit time of the lane mark calculated by the movement amount calculation unit. And a second step of determining the position of the search range and setting the search range at the determined position.

Images