TWI689433B - Lane tracking method and system for autonomous driving vehicles - Google Patents

Abstract

In a lane tracking method and system for self-driving cars, the processing unit estimates the yaw rate and lateral acceleration of the confidence level according to the measurement results of the inertial measurement unit, and the pre-stored and current time points for the vehicles driving in the lane Estimated current position data, estimate and store the vehicle's future position data at a future unit time point, and then estimate the vehicle's self-history time point based on the pre-stored historical reference position data and future position data estimated by the vehicle at the historical time point Total longitudinal displacement, total lateral displacement, and total azimuth offset to the future unit time point, as well as pre-stored reference lane line data generated based on the image of the lane sensed at the historical time point and using coordinate conversion to estimate correspondence Information on future lane lines at future unit time points.

Description

Lane tracking method and system for autonomous driving vehicles

The present invention relates to the control of autonomous driving vehicles, in particular to a lane tracking method and system for autonomous driving vehicles.

According to the L1~L5 level of automatic driving technology developed by the International Society of Automated Engineers (SAE), for higher levels of automatic driving technology, the robustness and reliability of the active control system of autonomous vehicles will face more High challenge.

However, in view of the current camera's limitations on the processing time required for the captured image from processing to output, the update frequency of lane detection information (eg, 10-20 Hz) is too low, and the output information of lane detection generally exists The problem of time synchronization with vehicle motion control, for example, the lane line information provided to the ECU of the vehicle control system is usually information before 100~200ms. More specifically, the update frequency of the lane sensing information of the vehicle control system should preferably be 5 to 10 times the control frequency, so as to avoid errors in related control caused by non-real-time lane sensing information. On the other hand, if the control frequency is too low due to, for example, a low lane sensing information update frequency, this may result in a reduction in the resolution and accuracy of the control commands. In particular, for high-curvature sections or when the lateral offset speed is large, the vehicle trajectory tracking effect is not good.

In addition, the existing roads often have lane line stains, blurs, or abnormal color contrast of the lane lines caused by changes in light, or even the lane lines do not exist at all (such as at intersections), which may lead to lane detection. The image recognition error of the module or the lane detection function is abnormal, and even the lane line detection function cannot be provided at all.

Therefore, how to develop a lane tracking technology that can solve the above problems has become an important issue.

Therefore, an object of the present invention is to provide a lane tracking method for an autonomous vehicle, which can overcome at least one of the disadvantages of the above-mentioned conventional techniques.

Therefore, the present invention provides a lane tracking method for an autonomous driving vehicle, which is implemented by a processing unit and includes the following steps: (A) a lane detection module for a vehicle traveling in a lane The reference lane line data generated from the image of the lane sensed at a historical time point, the previously estimated estimated lane line data corresponding to the current time point, and a previously estimated reference point of the vehicle relative to the lane The historical reference position data and the current position data of the position at the historical time point and the current time point, respectively, and the history of the position of each historical unit time point within a historical period from the historical time point to the current time point The location data is stored in a storage unit, wherein the historical reference location data, the historical location data and Each of the current position data contains a longitudinal position value, a lateral position value and an azimuth angle; (B) according to the angular velocity and acceleration of the vehicle measured at the current time point by an inertial measurement unit provided in the vehicle, Obtain the estimated yaw rate and estimated lateral acceleration of the vehicle at the current time, and at least according to the steering wheel and all wheels related to the vehicle sensed at the current time by a dynamic sensing unit provided at the vehicle Vehicle dynamic data to obtain the reference yaw rate and reference lateral acceleration of the vehicle at the current time point; (C) when determining the estimated yaw rate and the estimated lateral acceleration and the reference yaw rate and the reference respectively When the similarity of the lateral acceleration reaches a predetermined confidence level, the longitudinal displacement, lateral displacement, and azimuth deviation of the vehicle in the future unit time are estimated based on the estimated yaw rate and the estimated lateral acceleration (D) estimate the reference point of the vehicle based on the current position data stored in the storage unit and the longitudinal displacement, the lateral displacement and the azimuth deviation estimated in step (C) Relative to the future position data of the lane at a future unit time point, and store the future position data in the storage unit, wherein the future position data contains longitudinal position value, lateral position value and azimuth; (E) according to the Historical reference position data and the future position data, estimate the total longitudinal displacement, total lateral displacement and total azimuth deviation of the vehicle from the historical time point to the future unit time point; and (F) according to the storage The unit stores the reference lane line data, the total longitudinal displacement, the total lateral displacement, and the total azimuth offset and uses coordinate conversion to estimate the future lane line data corresponding to the future unit time point, and The future lane line data is stored in the storage unit.

Therefore, another object of the present invention is to provide a lane tracking system for self-driving vehicles, which can overcome at least one of the disadvantages of the above-mentioned conventional techniques.

Thus, the present invention provides a lane tracking system for autonomous vehicles, which includes a lane detection module, an inertial measurement unit, a dynamic sensing unit, a storage unit, and a processing unit.

The lane detection module is set on a vehicle driving in a lane, and is used to continuously sense the image of the lane at a sensing frequency, so as to generate corresponding lane line data according to each sensed image.

The inertial measurement unit is installed in the vehicle, and used to sense the inertia of the vehicle to generate the angular velocity and acceleration of the vehicle.

The dynamic sensing unit is installed in the vehicle, and is used to sense the motion state of the vehicle itself and its steering wheel and all wheels to generate vehicle dynamic data.

The storage unit stores reference lane line data generated by the lane detection module based on the image of the lane sensed at a historical time point, and a previously estimated reference point of the vehicle is at the historical time relative to the lane The historical reference position data and the current position data of the position of the point and the current time point and the corresponding historical position data of the position of each historical unit time point within a historical period from the historical time point to the current time point, wherein Each of the historical reference position data, the historical position data, and the current position data contains a longitudinal position value, a lateral position value, and an azimuth angle.

The processing unit is electrically connected to the lane detection module, the inertial measurement unit, the dynamic sensing unit and the storage unit, and is used to perform the following operations: according to the angular velocity generated from the inertial measurement unit at the current time point And the acceleration, the estimated yaw rate and the estimated lateral acceleration of the vehicle at the current time point are obtained, and at least according to the vehicle dynamic data generated from the dynamic sensing unit at the current time point, the vehicle The reference yaw rate and the reference lateral acceleration at the current time point; when it is determined that the estimated yaw rate and the estimated lateral acceleration are similar to the reference yaw rate and the reference lateral acceleration, respectively, a predetermined confidence is reached When leveling, according to the estimated yaw rate and the estimated lateral acceleration, the longitudinal displacement, lateral displacement and azimuth deviation of the vehicle in the future unit time are estimated; according to the current stored in the storage unit Position data, as well as the estimated longitudinal displacement, the lateral displacement and the azimuth offset, estimate the future position data of the reference point of the vehicle relative to the lane at a future unit time point, and convert the Future position data is stored in the storage unit, where the future position data contains longitudinal position values, lateral position values, and azimuth angles; based on the historical reference position data and the future position data, the vehicle is estimated from the historical time point to the future The total longitudinal displacement, the total lateral displacement and the total azimuth offset per unit time point; and according to the reference lane line data stored by the storage unit, the total longitudinal displacement, the total lateral displacement and the The total azimuth offset and the coordinate conversion are used to estimate the future lane line data corresponding to the future unit time point, and store the future lane line data in the storage unit.

The effect of the present invention is to further confirm the credibility of the yaw rate and lateral acceleration estimated according to the measurement results of the inertial measurement unit (ie, angular velocity and acceleration) by using the vehicle dynamic data; using the estimated future position data With reference to the lane line data, the future lane line data at the future unit time point is estimated to increase the update frequency of the lane line data used for lateral trajectory tracking; and the lane detection function in the lane detection module can be effectively compensated The lane information during the short-term failure, thereby improving the accuracy and robustness of the lateral control.

100: Lane tracking system

1: Lane detection module

11: CCD image sensor

12: Image processor

2: Inertial measurement unit

21: Three-axis gyroscope

22: Three-axis accelerometer

3: Dynamic sensing unit

31: Steering wheel angle sensor

32: Vehicle speed sensor

33: Wheel speed sensor group

4: storage unit

5: Processing unit

401-413: Steps

Other features and functions of the present invention will be clearly presented in the embodiment with reference to the drawings, in which: FIG. 1 is a schematic diagram illustrating an example of a vehicle driving in a lane; FIG. 2 is a block diagram, Illustratively illustrate an embodiment of the lane tracking system of the present invention for an autonomous driving vehicle; FIG. 3 is a schematic diagram illustrating the contents of data stored in a storage unit of the embodiment; FIG. 4 is a flowchart illustrating an example Explain how a processing unit of this embodiment implements the lane tracking method for autonomous vehicles of the present invention; FIG. 5 is a schematic diagram illustrating the contents of data stored in the storage unit after the processing unit performs step 406; FIG. 6 Is a schematic diagram illustrating after the processing unit executes step 409 The data content stored in the storage unit; FIG. 7 is a schematic diagram illustrating the data content stored in the storage unit after the processing unit performs step 413; and FIG. 8 is a schematic diagram illustrating the embodiment according to the lane The tracking method estimates the future lane line data at a future unit time point.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are denoted by the same numbers.

Referring to FIGS. 1 and 2, an embodiment of the lane tracking system 100 for automatic driving vehicles of the present invention is applied to a vehicle 300 driving in a lane 200, which can provide an automatic driving function. In this embodiment, both sides of the lane 200 have lane lines that are dashed lines, for example, but not limited to this example. In other embodiments, the lane lines may also be solid lines or double solid lines. The lane tracking system 100 includes a lane detection module 1, an inertial measurement unit 2, a dynamic sensing unit 3, a storage unit 4, and a processing unit 5.

The lane detection module 1 is installed in the vehicle 300, and includes, for example, a CCD image sensor 11 installed on the front windshield of the vehicle 300, and an image processing electrically connected to the CCD image sensor 11器12. 12. 器12. The CCD image sensor 11 is used to continuously sense the image of the lane 200 at a sensing frequency (for example, 10/sec), and the image processor 12 is based on each sensed by the CCD image sensor 11 An image and use known image processing algorithms to generate corresponding lane line data. It is worth noting that the lane line data provided by the lane detection module 1 itself is also updated at a frequency of, for example, 10/sec. In other words, the image of each lane line data from the lane 200 is captured by the CCD image sensor 11 It is generated and output by the image processor 12 100 ms after the start of the sensing time. In this embodiment, the generated lane line data includes, for example, the left lane line equation (as shown in Equation 1 below) and the right lane line equation (as shown in Equation 2 below): y _L = f _L ( x ) = A _L x ³ + B _L x ² + C _L x + D _L Formula 1

y _R = f _R ( x ) = A _R x ³ + B _R x ² + C _R x + D _R Equation 2 where y _L represents the lateral position value of the left lane line when the longitudinal position value is x , and y _R represents the lateral position value of the right lane line when the longitudinal position value is x .

The inertial measurement unit (Inertial Measurement Unit) 2 is installed in the vehicle 300, and includes, for example, a triaxial gyroscope (Triaxial Gyroscope) and a triaxial accelerometer (Triaxial Accelerometer), but not limited to this example, and used The angular velocity and acceleration of the vehicle 300 in three-dimensional space are measured to produce a measurement result containing the measured angular velocity and acceleration.

The dynamic sensing unit 3 is installed in the vehicle 300, and is used to sense the motion state of the vehicle 300, its steering wheel, and all wheels to generate vehicle dynamic data. In this embodiment, the dynamic sensing unit 3 includes, for example, a steering wheel angle sensor 31 for sensing the turning angle (Steering Angle) of the steering wheel of the vehicle 300, a A vehicle speed sensor 32 for sensing the driving speed (Longitudinal Velocity) of the vehicle 300, and a wheel speed sensor group 33 for sensing the rotation speeds of all wheels of the vehicle 300, but not as an example limit. Therefore, the vehicle dynamic data generated by the dynamic sensing unit 3 includes, for example, at least the steering wheel angle, the vehicle speed, the rear right wheel speed, and the rear left wheel speed.

It is worth noting that, ideally, the inertial measurement unit 2 and the dynamic sensing unit 3 can be designed to have the same output update rate. The output update rate is, for example, the reciprocal of the unit time (10 ms) (that is, 100/ sec), and is ten times the output update rate of the lane detection module 1 (for example, 10/sec).

N

10×P，其中P為一預定正整數，但不以此例為限。 Referring to FIG. 3, in this embodiment, the storage unit 4 stores the lane detection module 1 according to the image of the lane 200 sensed at a historical time point t _0-N (for example, after t _0-N 100ms) generated reference lane line data, the estimated lane line data previously estimated and corresponding to the current time t ₀ , and a reference point 301 of the vehicle 300 previously estimated (for example, as shown in FIG. 8 of the vehicle 300 (Center of gravity) the historical reference position data and the current position data with respect to the position of the lane 200 at the historical time point t _0-N and the current time point t ₀ and from the historical time point t _0-N to the The corresponding historical position data of the position of each historical unit time point t _0-1 , ..., t _0-(N-1) within the historical period of the current time point t ₀ . In this embodiment, each of the reference lane line data and the estimated lane line data includes, for example, a left lane line equation and a right lane line equation, and the historical reference position data, the historical position data, and the current position data Each contains, for example, a longitudinal position value, a lateral position value, and an azimuth angle. For example, this unit time is 10ms, and 10

N

10×P, where P is a predetermined positive integer, but not limited to this example.

The processing unit 5 is electrically connected to the lane detection module 1, the inertial measurement unit 2, the dynamic sensing unit 3 and the storage unit 4 to receive all lane line data from the lane detection module 1, from the The measurement result of the inertial measurement unit 2 and the vehicle dynamic data from the dynamic sensing unit 3. It is worth noting that all the data stored in the storage unit 4 is obtained through the processing unit 5 repeatedly executing the lane tracking method for autonomous vehicles of the present invention, and the processing unit 5 stores it to the storage unit 4.

The following will refer to FIGS. 2 to 7 to illustrate in detail how the processing unit 5 estimates the future lane of the vehicle 300 at a future unit time t ₀₊₁ by executing the lane tracking method at the current time t ₀ Line data, the lane tracking method includes the following steps 401 ~ 413 (Figure 4).

In step 401, the processing unit 5 in accordance with the current time point t ₀ generated (output) of the inertial measurement unit 2, the measurement result (i.e., the angular velocity and the acceleration) to obtain the vehicle 300 t at the point of the current time ₀ estimated yaw rate and estimated lateral acceleration. More specifically, here, the processing unit 5 serves as a Kalman filter (Kalman Filter), so as to first filter out the noise of the angular velocity and the acceleration by the Kalman filtering method, and then according to the filtered angular velocity And the acceleration and use the Kalman estimation method to obtain the estimated yaw rate (Yaw Rate) and the estimated lateral acceleration. Since the details of the Kalman filter are well-known to those skilled in the art, they will not be repeated here.

表示)及該參考側向加速度(以

表示)，且該參考偏航率

及該參考側向加速度

分別能由以下式3及式4計算出：

On the other hand, at step 402, the processing unit 5 in accordance with at least the dynamics data of the vehicle at the current time point t ₀ generated, to obtain the acceleration of the vehicle 300 at the current time point t ₀ reference yaw rate and a reference lateral . More specifically, the processing unit 5 is based on the steering wheel angle (denoted by δ _sw ) and the vehicle speed (denoted by V _x ) contained in the vehicle dynamic data, and the steering ratio of the vehicle 300 (steering wheel angle and wheel steering angle The ratio of N, and the understeer coefficient (in _Kus ) and wheelbase (Wheel Base; that is, the distance from the front axle to the rear axle, and expressed in L) to obtain the reference yaw rate (in

Indicated) and the reference lateral acceleration (in

Indicates), and the reference yaw rate

And the reference lateral acceleration

It can be calculated by the following formula 3 and formula 4, respectively:

其中，δ _f 為前輪轉向角，且δ _f =δ _sw /N。或者，該處理單元5亦能根據該車輛動態資料所含的該車速V _x 、該後右車輪輪速(以v _rr 表示)及該後左車輪輪速(以v _rl 表示)，以及該車輛300的後輪輪距(Rear Track Width；且以S _r 表示)，獲得該參考偏航率

及該參考側向加速度

，且該參考偏航率

及該參考側向加速度

分別能由以下式5及式6計 算出：

Where δ _f is the steering angle of the front wheels, and δ _f = δ _sw / N. Alternatively, the processing unit 5 can also based on the vehicle speed V _x, the right wheel speed (expressed in v _rr) and the rear left wheel speed (expressed in v _rl), the rear of the vehicle and vehicle dynamic data contained in the The rear track width of 300 (Rear Track Width; and expressed as S _r ), to obtain the reference yaw rate

And the reference lateral acceleration

And the reference yaw rate

And the reference lateral acceleration

It can be calculated by the following formula 5 and formula 6 respectively:

附帶一提的是，若該車輛動態資料還包含前右車輪輪速(以v _fr 表示)及該前左車輪輪速(以v _fl 表示)，該處理單元5還能根據該前右車輪輪速v _fr 及該前左車輪輪速v _fl 、該車輛300的前輪輪距(Front Track Width；且以S _f 表示)及該前輪轉向角δ _f 獲得該參考偏航率

及該參考側向加速度

，且該參考偏航率

及該參考側向加速度

分別能由以下式7及式8計算出：

Incidentally, if the vehicle dynamic data further includes the front right wheel speed (indicated by v _fr ) and the front left wheel speed (indicated by v _fl ), the processing unit 5 can also use the front right wheel The speed v _fr and the front left wheel speed v _fl , the front wheel width of the vehicle 300 (Front Track Width; and denoted by S _f ), and the front wheel steering angle δ _f obtain the reference yaw rate

And the reference lateral acceleration

And the reference yaw rate

And the reference lateral acceleration

It can be calculated by the following formula 7 and formula 8 respectively:

After step 401 and step 402, the processing unit 5 determines the estimated yaw rate and the estimated lateral acceleration with the reference yaw rate and the reference side by using, for example, the Two Sample T-Test method. Whether the similarity of the accelerations has reached a predetermined confidence level. In this embodiment, the predetermined confidence level is 95, for example. If the determination result is affirmative, this means that the measurement result of the inertial measurement unit 2 has passed the verification of the credibility as available data, and the flow proceeds to step 405. Conversely, if the determination result is negative, this means that the processing unit 5 has detected that the measurement result of the inertial measurement unit 2 has been verified as unavailable data. In this case, the processing unit 5 can output an instruction The warning signal of the abnormality of the inertial measurement unit 2 is output to An external control system (not shown) (step 404), for the external control system to perform related subsequent processing.

表示)。然後，該處理單元5根據該儲存單元1所儲存的該當前位置資料，以及所估算的該縱向位移量Δs _x 、該側向位移量Δs _y 及該方位角偏移量Δ

，估算該車輛300的該參考點301相對於該車道在一未來單位時間點t₀₊₁的未來位置資料，並將該未來位置資料儲存於該儲存單元4(步驟406)，其中該未來位置資料含有縱向位置值、側向位置值及方位角，如圖5所示。 In step 405, based on the estimated yaw rate and the estimated lateral acceleration, the processing unit 5 estimates the longitudinal displacement (indicated by Δ s _x ) of the vehicle 300 in the future unit time (for example, 10 ms), side Amount of displacement (in Δ s _y ) and azimuth offset (in Δ

Means). Then, the processing unit 5 is based on the current position data stored in the storage unit 1 and the estimated longitudinal displacement Δ s _x , the lateral displacement Δ s _y and the azimuth deviation Δ

, Estimate the future position data of the reference point 301 of the vehicle 300 relative to the lane at a future unit time point t ₀₊₁ , and store the future position data in the storage unit 4 (step 406), wherein the future position The data contains longitudinal position value, lateral position value and azimuth, as shown in Figure 5.

It should be noted that the processing unit 5 will first confirm whether it has received the lane line data newly generated from the lane detection module 1 at the current time point t ₀ (step 407). If the processing unit 5 does not receive any lane line data from the lane detection module 1 at the current time point t ₀ , the process proceeds to step 412. On the contrary, if the processing unit 5 receives the lane line data newly generated from the lane detection module 1 at the current time point t ₀ (more specifically, the lane line data is the image processor 12 based on the CCD The image sensor 11 senses the image of the lane 200 at a historical unit time point (for example, t _0-10 ) in the historical period and generates it at the current time point t ₀ ), and the processing unit 5 according to the The estimated lane line data stored in the storage unit 4 and the predetermined reference conditions related to the image sensing (resolution) specifications of the CCD image sensing module 11 of the lane detection module 1 determine the received Whether the lane line data is available (step 408), thereby confirming whether the lane detection function of the lane detection module 1 is normal.

In this embodiment, the image sensing specification defines, for example, a predetermined shortest effective longitudinal distance and a predetermined longest effective longitudinal distance, and the predetermined reference condition includes, for example, the predetermined shortest effective longitudinal distance and the predetermined longest effective longitudinal distance, respectively. A predetermined first width difference threshold and a predetermined second width difference threshold related to the width of the lane 200, and the centerline of the lane 200 at the predetermined shortest effective longitudinal distance and the predetermined longest effective longitudinal distance, respectively A predetermined first lateral offset threshold and a predetermined second lateral offset threshold. For example, the predetermined shortest effective longitudinal distance and the predetermined longest effective longitudinal distance are 15 meters and 25 meters, respectively, the predetermined first width difference threshold and the predetermined second width difference threshold are both 0.5 meters, and the The predetermined first lateral offset threshold and the predetermined second lateral offset threshold are both 0.1 meters, but not limited to this example.

Thus, in step 408, the processing unit 5 obtains the first width value representing the width of the lane 200 at the predetermined shortest effective longitudinal distance and the lateral position representing the center line of the lane 200 according to the estimated lane line data The first position value, and the second width representing the width value of the lane 200 at the predetermined longest effective longitudinal distance and the second position value representing the lateral position of the centerline of the lane 200, and according to the The lane line data obtains a third width value representing the width value of the lane 200 at the predetermined shortest effective longitudinal distance and a third position value representing the lateral position of the centerline of the lane, and at the predetermined longest effective longitudinal distance A fourth width value representing the width of the lane 200 and a fourth position value representing the lateral position of the centerline of the lane. In this embodiment, the processing unit 5 determines whether the first difference between the first width value and the third width value is not greater than the predetermined first width difference threshold, the second width value and the first Whether the second difference between the four width values is not greater than the predetermined second width difference threshold, and whether the third difference between the first position value and the third position value is not greater than the predetermined first lateral offset The amount threshold and whether the fourth difference between the second position value and the fourth position value is not greater than the predetermined second lateral offset threshold determines whether the lane line data is available. When the processing unit 5 determines that the first difference, the second difference, the third difference and the fourth difference are not greater than the predetermined first width difference threshold, the predetermined second width difference threshold, the When the first lateral offset threshold and the predetermined second lateral offset threshold are predetermined, the lane line data is determined to be available by the processing unit 5, and thus, the lane detection of the lane detection module 1 The function is confirmed to be normal. Then, the flow will proceed to step 409. Otherwise, the process will proceed to step 410.

及該右車道線方程式

分別被表成以下式9至式12：y' _L1=f' _L1(x)=A' _L1 x ³+B' _L1 x ²+C' _L1 x+D' _L1 式9 The left lane and the left lane marking line equation of the equation y _{'L 1} and right lane lines of the equation y' _{R 1,} and the lane line profile includes an example, the estimated lane line information contained

And the right lane line equation

They are expressed as the following formula 9 to formula 12: y ' _{L 1} = f ' _{L 1} ( x ) = A ' _{L 1} x ³ + B ' _{L 1} x ² + C ' _{L 1} x + D ' _{L 1} formula 9

y ' _{R 1} = f ' _{R 1} ( x ) = A ' _{R 1} x ³ + B ' _{R 1} x ² + C ' _{R 1} x + D ' _{R 1} Formula 10

於是，依循前述範例，將x=15代入式9至式12，能獲得該第一寬度值W1、該第三寬度值W2、該第一位置值Y1及該第三位置值Y3，其中W1=f' _L1(15)-f' _R1(15)，

，Y1=(f' _L1(15)+f' _R1(15))/2，及

；而將x=25代入式9至式12，能獲得該第二寬度值W2、該第四寬度值W4、該第二位置值Y2及該第四位置值Y4，其中W2=f' _L1(25)-f' _R1(25)，

，Y2=(f' _L1(25)+f' _R1(25))/2，及

。然後，該第一差值D1、該第二差值D2、該第三差值D3及該第四差值D4能被獲得，其中D1=|W1-W3|，D2=|W2-W4|，D3=|Y1-Y3|，及D4=|Y2-Y4|。值得注意的是，前述所有值均以公尺為單位。若D1

0.5，D2

0.5，D3

0.1且D4

0.1，則該車道線資料被判定為可利用的，相反地，若前述四個關係式其中的任一者未被滿足時，該車道線資料被判定為不可利用的，此意謂該車道偵測模組1可能因擷取到含有已汙損、模糊不清或因光線變化所導致的車道線顏色對比異常的車道線或者根本不存在有車道線的車道影像而導致車道偵測功能的短暫失效。

Therefore, following the aforementioned example, substituting x=15 into Equations 9 to 12, the first width value W1, the third width value W2, the first position value Y1 and the third position value Y3 can be obtained, where W1= f ' _{L 1} (15)- f ' _{R 1} (15),

, Y1=( f ' _{L 1} (15)+ f ' _{R 1} (15))/2, and

; And substituting x=25 into Equation 9 to Equation 12, the second width value W2, the fourth width value W4, the second position value Y2 and the fourth position value Y4 can be obtained, where W2= f ′ _{L 1} (25)- f ' _{R 1} (25),

, Y2=( f ' _{L 1} (25)+ f ' _{R 1} (25))/2, and

. Then, the first difference D1, the second difference D2, the third difference D3 and the fourth difference D4 can be obtained, where D1=| W 1- W 3|, D2=| W 2- W 4|, D3=| Y 1- Y 3|, and D4=| Y 2- Y 4|. It is worth noting that all the aforementioned values are in meters. If D1

0.5, D2

0.5, D3

0.1 and D4

0.1, the lane line data is judged to be available. Conversely, if any of the above four relational expressions are not satisfied, the lane line data is judged to be unavailable, which means the lane detection The detection module 1 may cause a short-term lane detection function because it captures a lane line that has been stained, blurred, or has abnormal color contrast of the lane line due to light changes, or there is no lane image with a lane line at all. Failure.

If the processing unit 5 determines in step 408 that any one of the first to fourth difference values is greater than the predetermined first width difference threshold, the predetermined second width difference threshold, the When a corresponding one of the first lateral offset threshold and the predetermined second lateral offset threshold is predetermined, the lane line data is determined to be unavailable by the processing unit 5 (that is, the lane detection mode The lane detection function of group 1 is confirmed to be abnormal). In this case, the processing unit 5 will further confirm whether the accumulated number of unavailable lane line data has exceeded the threshold number of times (for example, 7 times) (step 410). Then, if the processing unit 5 determines that the accumulated number of times does not exceed the threshold number of times, the flow proceeds to step 412. Conversely, if the processing unit 5 determines that the cumulative number of times exceeds the threshold number of times, the processing unit 5 outputs a warning signal indicating that the lane detection module 1 is abnormal to the external control system (step 411) for The external control system performs subsequent related control.

In step 409, the processing unit 5 updates the reference lane line data and the historical reference position data stored in the storage unit 4 to the lane line data and the historical position at the historical unit time point t _0-10 , respectively data. In this case, the content of the location data stored in the storage unit 4 is updated to only store the historical location data from the time point t _0-10 of the historical unit, as shown in FIG. 6.

In step 412, the processing unit 5 estimates the vehicle 300 from the historical time point t _0-N (t _0-10 ) to the future unit time point t ₀₊₁ based on the historical reference position data and the future position data The total longitudinal displacement S _x , the total lateral displacement S _y and the total azimuth deviation ψ . More specifically, after the processing unit 5 confirms that the newly generated lane line data is not received in step 407, or confirms that the newly generated lane line data is received in step 407 but determines the lane line data in step 408 Is unavailable and after the determination that the lane line data is unavailable in step 410 does not exceed the threshold, the historical reference position data and the future position data shown in FIG. 5 are used, and the historical time The point adopts t _0-N ; and the processing unit 5 confirms that the newly generated lane line data is received in step 407 and when step 409 is executed, the historical reference position data and the future position shown in FIG. 6 are used Data, and the historical time point adopts t _0-10 .

Finally, in step 413, the processing unit 5 according to the reference lane line data stored by the storage unit 4, the total longitudinal displacement S _x , the total lateral displacement S _y and the total azimuth offset ψ and Using coordinate conversion, the future lane line data corresponding to the future unit time point t ₀₊₁ is estimated. In this case, the future lane line data will be regarded as the estimated lane line data corresponding to the future unit time point t ₀₊₁ . In addition, the processing unit 5 also stores the future lane line data (for example, in a rewrite manner that covers the previously stored estimated lane line data, but not limited to this) in the storage unit 4, as shown in FIG. 7, For subsequent processing.

For example, referring to FIG. 8, the left lane line equation y _{L 1} and the right lane line equation y _{R 1} included in the estimated lane line data are respectively expressed as the following equations 13 to 14: y _{L 1} = f _{L 1} ( x ) = A _{L 1} x ³ + B _{L 1} x ² + C _{L 1} x + D _{L 1} Equation 13

將式15經過反座標轉換後，獲得以下式16：

於是，從式16可獲得x=g(x',y')且y=h(x',y')。由於y _L1=f _L1(x)且y _R1=f _R1(x)，於是最後可獲得該未來車道線資料所含的左車道線方程式y _L2及右車道線方程式y _R2，並可表示成以下式17及式18：y _L2=f _L1(g(x',y'))=A _L2 x ³+B _L2 x ²+C _L2 x+D _L2 式17 y _{R 1} = f _{R 1} ( x ) = A _{R 1} x ³ + B _{R 1} x ² + C _{R 1} x + D _{R 1} Formula 14 In this case, the reference point 301 of the vehicle 200 is in the future unit The coordinate relationship of the time point t ₀₊₁ with respect to the historical time point t _0-N (t _0-10 ) can be expressed as the following formula 15:

After transforming Equation 15 through inverse coordinates, the following Equation 16 is obtained:

Thus, from equation 16, x = g ( x ', y ') and y = h ( x ', y ') can be obtained. Since y _{L 1} = f _{L 1} ( x ) and y _{R 1} = f _{R 1} ( x ), the left lane line equation y _{L 2} and the right lane line equation y _{R 2} contained in the future lane line data are finally obtained , And can be expressed as the following formula 17 and formula 18: y _{L 2} = f _{L 1} ( g ( x ', y ')) = A _{L 2} x ³ + B _{L 2} x ² + C _{L 2} x + D _{L 2} Formula 17

y _{R 2} = f _{R 1} ( g ( x ', y ')) = A _{R 2} x ³ + B _{R 2} x ² + C _{R 2} x + D _{R 2} Equation 18 So far, the processing unit 5 has completed the lane Tracking method. The processing unit 5 can output the estimated future lane line data to the external control system for use as a basis for the lateral control of the vehicle 300. It is worth noting that when the lane tracking method of the present invention is used and the unit time is 10 ms, the lane detection system 100 can equivalently increase the update rate of the lane line data to the lane detection module 1 Ten times the update rate of lane line information provided by itself.

In summary, the lane tracking system 100 of the present invention uses the vehicle dynamic data to further confirm the reliability of the yaw rate and lateral acceleration estimated based on the measurement results (ie, angular velocity and acceleration) of the inertial measurement unit 2 And use the estimated The position data and the reference lane line data are used to estimate the future lane line data at the future unit time point, so as to increase the update frequency of the lane line data used for lateral trajectory tracking, and can effectively compensate for the lane detection module 1 Lane information during the short-term failure of the lane detection function (that is, the newly generated lane line data is determined to be unavailable), thereby improving the accuracy and robustness of the lateral control. Therefore, the purpose of cost invention can indeed be achieved.

However, the above are only examples of the present invention, and the scope of implementation of the present invention cannot be limited by this, any simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the content of the patent specification are still classified as Within the scope of the invention patent.

100: Lane tracking system

1: Lane detection module

11: CCD image sensor

12: Image processor

2: Inertial measurement unit

21: Three-axis gyroscope

22: Three-axis accelerometer

3: Dynamic sensing unit

31: Steering wheel angle sensor

32: Vehicle speed sensor

33: Wheel speed sensor group

4: storage unit

5: Processing unit

Claims (10)

A lane tracking method for autonomous vehicles, implemented by a processing unit, includes the following steps: (A) The lane detection module of a vehicle driving in a lane will be sensed based on a historical time point The reference lane line data generated by measuring the image of the lane, and the historical reference position data and the current position data of the position of the reference point of the vehicle previously estimated at the historical time point and the current time point with respect to the lane respectively and in A historical position data of the position of each historical unit time point in the historical period from the historical time point to the current time point is stored in a storage unit, wherein the historical reference position data, the historical position data and the current position data Each of them contains longitudinal position value, lateral position value and azimuth angle; (B) according to the angular velocity and acceleration of the vehicle measured by an inertial measurement unit provided at the vehicle at the current time The estimated yaw rate and the estimated lateral acceleration at the current time, and at least according to vehicle dynamic data related to the steering wheel and all wheels of the vehicle sensed by a dynamic sensing unit provided at the vehicle at the current time, Obtain the reference yaw rate and reference lateral acceleration of the vehicle at the current time point; (C) After determining the estimated yaw rate and the estimated lateral acceleration, respectively, the reference yaw rate and the reference lateral acceleration When the similarity reaches a predetermined confidence level, based on the estimated yaw rate and the estimated lateral acceleration, the longitudinal displacement, lateral displacement, and azimuth deviation of the vehicle in the future unit time are estimated; (D) According to the current position data stored in the storage unit, and the longitudinal displacement, the lateral displacement, and the azimuth deviation estimated in step (C), estimate the reference point of the vehicle relative to The future position data of the lane at a future unit time point, and store the future position data in the storage unit, wherein the future position data includes longitudinal position value, lateral position value and azimuth angle; (E) according to the historical reference Position data and the future position data, estimate the total longitudinal displacement, total lateral displacement and total azimuth deviation of the vehicle from the historical time point to the future unit time point; and (F) according to the storage unit The stored reference lane line data, the total longitudinal displacement, the total lateral displacement, and the total azimuth offset are converted using coordinates to estimate the future lane line data corresponding to the future unit time point, and the future The lane line data is stored in the storage unit. The lane tracking method for an autonomous driving vehicle according to claim 1, wherein in step (A), the processing unit also stores the estimated lane line data previously estimated and corresponding to the current time point in the storage unit Between step (A) and step (E), the following steps are also included: (G) When the lane detection module is received at the current time point, it is sensed according to a historical unit time point in the historical period When the image of the lane is newly generated lane line data, the received lane line data stored in the storage unit and the predetermined reference conditions related to the image sensing specifications of the lane detection module are used to determine the received Whether the lane line data is available; and (H) When it is determined that the lane line data is available, the storage slip The reference lane line data and the historical reference position data stored in the yuan are updated to the lane line data and the historical position data at the time point of the historical unit, respectively. The lane tracking method for an automatic driving vehicle according to claim 2, wherein in step (G): the image sensing specification defines a predetermined shortest effective longitudinal distance and a predetermined longest effective longitudinal distance; the predetermined reference The conditions include a predetermined first width difference threshold and a predetermined second width difference threshold corresponding to the width of the lane at the predetermined shortest effective longitudinal distance and the predetermined longest effective longitudinal distance, respectively, and respectively at the predetermined shortest effective longitudinal distance And a predetermined first lateral offset threshold and a predetermined second lateral offset threshold related to the center line of the lane at the predetermined maximum effective longitudinal distance; the processing unit obtains the The first width value representing the width of the lane at the predetermined shortest effective longitudinal distance and the first position value representing the lateral position of the centerline of the lane, and the first value representing the width of the lane at the predetermined longest effective longitudinal distance The second width value and the second position value representing the lateral position of the center line of the lane, and the third width value representing the width of the lane at the predetermined shortest effective longitudinal distance and the The third position value of the lateral position of the center line, and the fourth width value representing the width of the lane at the predetermined longest effective longitudinal distance and the fourth position value representing the lateral position of the center line of the lane; the processing The unit determines the first width value and the third width value by Whether the first difference between them is not greater than the predetermined first width difference threshold, whether the second difference between the second width value and the fourth width value is not greater than the predetermined second width difference threshold, the first Whether the third difference between the position value and the third position value is not greater than the predetermined first lateral offset threshold, and whether the fourth difference between the second position value and the fourth position value is not Greater than the predetermined second lateral offset threshold to determine whether the lane line data is available; and when the processing unit determines the first difference, the second difference, the third difference and the first When the four differences are not greater than the predetermined first width difference threshold, the predetermined second width difference threshold, the predetermined first lateral offset threshold, and the predetermined second lateral offset threshold, the lane line data is The processing unit determines that it is available. The lane tracking method for an autonomous driving vehicle according to claim 1, wherein in step (B), the processing unit first uses Kalman filtering to filter out the noise of the angular velocity and the acceleration, and then according to the filtering After the angular velocity and the acceleration, the estimated yaw rate and the estimated lateral acceleration are obtained using the Kalman estimation method. The lane tracking method for an autonomous driving vehicle according to claim 1, wherein in step (B): the vehicle dynamic data includes at least a steering wheel angle, a vehicle speed, a rear right wheel speed and a rear left wheel speed; and The processing unit is based on the steering wheel angle and the vehicle speed contained in the vehicle dynamic data, as well as the vehicle's steering ratio, understeer coefficient and wheelbase, or the vehicle speed and the rear right wheel wheels included in the vehicle dynamic data Speed and the rear left wheel speed, and the rear wheel track of the vehicle, the reference yaw rate and the reference lateral acceleration are obtained. A lane tracking system for an autonomous driving vehicle includes: a lane detection module, set in a vehicle driving in a lane, and used to continuously sense the image of the lane at a sensing frequency, so as to be based on the sensed Each image of the corresponding lane line data is generated; an inertial measurement unit is provided in the vehicle and used to sense the inertia of the vehicle to generate the angular velocity and acceleration of the vehicle; a dynamic sensing unit is provided in the vehicle and used To sense the motion status of the vehicle itself and its steering wheel and all wheels to generate vehicle dynamic data; a storage unit stores the reference lane generated by the lane detection module based on the image of the lane sensed at a historical time point Line data, as well as the historical reference position data and current position data of a previously estimated position of a reference point of the vehicle relative to the lane at the historical time point and the current time point and from the historical time point to the current time The historical position data of the position of each historical unit time point within the historical period of the point, where each of the historical reference position data, the historical position data, and the current position data each contains a longitudinal position value, a lateral position value, and an orientation Angle; and a processing unit, electrically connected to the lane detection module, the inertial measurement unit, the dynamic sensing unit and the storage unit, and used to perform the following operations according to the inertial measurement unit and generated at the current time point The angular velocity and the acceleration of the vehicle at the current time Estimated yaw rate and estimated lateral acceleration, and based on at least the vehicle dynamic data generated from the dynamic sensing unit at the current time point, the reference yaw rate and reference side of the vehicle at the current time point are obtained Acceleration, when it is determined that the similarity of the estimated yaw rate and the estimated lateral acceleration to the reference yaw rate and the reference lateral acceleration respectively reach a predetermined confidence level, according to the estimated yaw rate and the estimated Lateral acceleration, estimate the longitudinal displacement, lateral displacement and azimuth offset of the vehicle in the future unit time, according to the current position data stored in the storage unit and the estimated longitudinal displacement, The lateral displacement and the azimuth offset, estimate the future position data of the reference point of the vehicle relative to the lane at a future unit time point, and store the future position data in the storage unit, wherein the future The position data includes longitudinal position value, lateral position value and azimuth angle. Based on the historical reference position data and the future position data, the total longitudinal displacement and total lateral direction of the vehicle from the historical time point to the future unit time point are estimated. Displacement and total azimuth offset, and according to the reference lane line data stored in the storage unit, the total longitudinal displacement, the total lateral displacement and the total azimuth offset and using coordinate conversion, estimate the corresponding The future lane line data at the future unit time point, and store the future lane line data in the storage unit. The lane tracking system for autonomous vehicles as described in claim 6, wherein: The storage unit also stores previously estimated lane line data corresponding to the current time point; when the processing unit receives the lane detection module at the current time point and based on a history in the historical period When the image of the lane is sensed at a unit time and the lane line data is newly generated, the processing unit sets a reference based on the estimated lane line data stored in the storage unit and the image sensing specifications related to the lane detection module Conditions, determine whether the received lane line data is available; and when the processing unit determines that the lane line data is available, the processing unit is performing the total longitudinal displacement and total lateral displacement of the vehicle Before the estimation of the total azimuth offset, the reference lane line data and the historical reference position data stored in the storage unit are updated to the lane line data and the historical position data at the time point of the historical unit, respectively. The lane tracking system for an autonomous driving vehicle according to claim 7, wherein: the image sensing specification defines a predetermined shortest effective longitudinal distance and a predetermined longest effective longitudinal distance; the predetermined reference condition includes respectively at the predetermined shortest A predetermined first width difference threshold and a predetermined second width difference threshold at the effective longitudinal distance and the predetermined longest effective longitudinal distance relative to the width of the lane, and at the predetermined shortest effective longitudinal distance and the predetermined longest effective longitudinal distance, respectively A predetermined first lateral offset threshold and a predetermined second lateral offset threshold related to the centerline of the lane; The processing unit obtains, according to the estimated lane line data, a first width value representing the width of the lane at the predetermined shortest effective longitudinal distance and a first position value representing the lateral position of the center line of the lane, and At the effective longitudinal distance, a second width value representing the width value of the lane and a second position value representing the lateral position of the center line of the lane, and obtaining the representative at the predetermined shortest effective longitudinal distance according to the lane line data The third width value of the width and the third position value representing the lateral position of the centerline of the lane, and the fourth width value representing the width of the lane at the predetermined longest effective longitudinal distance and the centerline representing the lane The fourth position value of the lateral position of the; the processing unit determines whether the first difference between the first width value and the third width value is not greater than the predetermined first width difference threshold and the second width value Whether the second difference between the fourth width value is not greater than the predetermined second width difference threshold, and whether the third difference between the first position value and the third position value is not greater than the predetermined first side The threshold of the lateral offset, and whether the fourth difference between the second position value and the fourth position value is not greater than the predetermined second lateral offset threshold to determine whether the lane line data is available; And when the processing unit determines that the first difference, the second difference, the third difference and the fourth difference are not greater than the predetermined first width difference threshold, the predetermined second width difference threshold, the When the first lateral offset threshold and the predetermined second lateral offset threshold are predetermined, the lane line data is determined to be available by the processing unit. The lane tracking system for autonomous vehicles as described in claim 6, which In the process, the processing unit first uses the Kalman filtering method to filter out the noise of the angular velocity and the acceleration, and then uses the Kalman estimation method to obtain the estimated yaw rate and the estimated side according to the filtered angular velocity and the acceleration Directional acceleration. The lane tracking system for an autonomous driving vehicle according to claim 6, wherein: the dynamic sensing unit includes a steering wheel angle sensor for sensing the turning angle of the steering wheel of the vehicle, and a sensor for sensing the A vehicle speed sensor for the driving speed of the vehicle, and a wheel speed sensor group for sensing the rotation speeds of all wheels of the vehicle, so that the vehicle dynamic data generated by the dynamic sensing unit includes at least the steering wheel angle and the vehicle speed , The rear right wheel speed and the rear left wheel speed; and the processing unit according to the steering wheel angle and the vehicle speed contained in the vehicle dynamic data, as well as the vehicle's steering ratio, understeer coefficient and wheelbase, or according to the vehicle The vehicle speed, the rear right wheel speed and the rear left wheel speed included in the dynamic data, and the rear wheel track of the vehicle obtain the reference yaw rate and the reference lateral acceleration.

Images