TWI726810B - Obstacle tracking system and method combined with image data - Google Patents

Abstract

An obstacle tracking system and method combined with map data, which is installed on a vehicle, uses a vehicle dynamic positioning module to obtain the position of the vehicle, and uses a map data module to obtain map information path information near the vehicle; The position of the vehicle is compared with the route information of the map to find out multiple candidate routes in the forward direction of the vehicle. Another environmental sensing module detects the environmental information on the road in front of the car, determines whether there is an obstacle, and tracks it. After the system receives the tracking result of an obstacle, it determines which candidate path the obstacle is on, and corrects the obstacle's moving path. Finally, the system estimates the position of the obstacle and outputs it based on the obstacle's moving path. Since the present invention combines the path information of the map and can obtain information such as the curvature and slope of the road ahead, the position of the obstacle can be corrected more accurately, and a more stable obstacle detection result can be obtained.

Description

Obstacle tracking system and method combined with image data

The present invention relates to the technical field of vehicle assisted driving, in particular to an obstacle tracking system and method combined with image data.

The driving assistance system of the vehicle can be used to determine the obstacle in front, and use a filtering mechanism to estimate the current obstacle position based on the past time point. However, there are still shortcomings in the project of following the car on the curve. For example, when the current car is not entering the corner or has entered the corner, the system will not be sure at which time the obstacle (ie the vehicle in front) will enter the corner or re-enter the corner. Turning, and then the estimated current position has a larger offset from its actual position, resulting in an unstable result of detecting obstacles in the curve, for example, the obstacle will disappear and reappear.

At present, in the detection of obstacles in bends, the factors that cause instability mainly include the following three factors: First, the camera's limited viewing angle and visible distance affect the recognition of lane lines and obstacles; Second, the radar is vulnerable to iron rods The influence of the guardrail causes misjudgment; 3. The appearance deformation of the obstacle detected by (Lidar) on the curve is larger than the shape change of the straight road section.

In view of this, the present invention proposes an obstacle tracking system and method combined with map data to effectively solve the above-mentioned problems. The specific architecture and implementation methods will be described in detail below:

The main purpose of the present invention is to provide an obstacle tracking system and method combined with map data, which uses map data path information to estimate the position of obstacles, and can correct the accuracy of using Kalman filter to detect the current position of obstacles ahead. Achieve a more stable obstacle detection effect on bends.

Another object of the present invention is to provide an obstacle tracking system and method combined with map data, which combines dynamic vehicle positioning with map data path information to obtain a map of the location of the vehicle and its vicinity, and find out which vehicles can use next Candidate path, accuracy can include multiple lanes on the same road.

Another object of the present invention is to provide an obstacle tracking system and method combined with map data, which even when the obstacle exceeds the region of interest (ROI) preset by the sensor, by predicting the obstacle ( The vehicle ahead) plans the driving route, and the obstacle can still be matched with the candidate path ahead, and the moving path of the obstacle can be estimated without the problem of system dissociation.

To achieve the above objective, the present invention provides an obstacle tracking system combined with map data, which is installed on a vehicle and includes: a vehicle dynamic positioning module to obtain the position of the vehicle; and a map data module to use To obtain the map data path information near the location of the vehicle; a road path module compares the location of the vehicle obtained by the vehicle dynamic positioning module with the map data path information obtained by the map data module Yes, find out a plurality of candidate paths of the vehicle; an obstacle path module, connected to the road path module, receive the tracking result of at least one obstacle, determine one of the candidate paths where the obstacle is located, and predict Estimate the movement path of the obstacle; and an obstacle information filtering module, connected to the obstacle path module, estimate the position of the obstacle and output it according to the movement path of the obstacle.

According to the embodiment of the present invention, the vehicle dynamic positioning module can further obtain the state of the vehicle, including speed, orientation, position, and direction of travel.

According to an embodiment of the present invention, the map data path information includes the centerline, direction, lane line, signs, road curvature, and slope of the road.

According to an embodiment of the present invention, it further includes at least one environment sensing device that captures road information in front of the vehicle to determine whether there is an obstacle in front of the vehicle.

According to an embodiment of the present invention, the tracking result of the obstacle includes obstacle parameters such as the longitudinal position of the obstacle, the direction of travel, and the speed of the obstacle.

According to an embodiment of the present invention, the obstacle path module further includes: a path matching module that receives the tracking result of the obstacle, and matches the obstacle tracking result with the candidate paths to find the obstacle An obstacle path among the candidate paths where the object is located; and a path estimation module that uses an optimal path equation to predict the obstacle's moving path on the obstacle path according to the tracking result of the obstacle .

According to the embodiment of the present invention, an obstacle tracking module is further included, which is connected to the path matching module to provide a tracking result of the obstacle.

According to an embodiment of the present invention, the path estimation module provides the estimation result of the obstacle's movement path to the obstacle information filtering module, and the obstacle information filtering module uses the estimation result as the filtering parameter. To estimate the location of the obstacle.

According to the embodiment of the present invention, a coordinate conversion module is further included, which is connected to the vehicle dynamic positioning module, the map data module and the road path module, and the coordinate system of the position of the vehicle and the map data path information Convert from world coordinates to vehicle coordinates.

According to the embodiment of the present invention, it further includes a coordinate conversion module connected to the environment sensing device for converting the coordinate system of the sensing result of the at least one environment sensing device from world coordinates to vehicle coordinates, and outputting In an obstacle judgment module, it is judged whether there is the obstacle ahead, or whether an object in front is the obstacle, and then the judgment result is transmitted to the obstacle information filtering module.

The present invention also provides an obstacle tracking method combined with map data, including the following steps: using a vehicle dynamic positioning module to obtain the location of the vehicle, and using a map data module to obtain map information near the location of the vehicle; Compare the position of the vehicle obtained by the vehicle dynamic positioning module with the path information of the map data obtained by the map data module to find a plurality of candidate paths of the vehicle; receive the tracking result of an obstacle, It is judged that the obstacle is in one of the candidate paths, and the movement path of the obstacle is estimated; and the position of the obstacle is estimated and output according to the movement path of the obstacle.

The present invention provides an obstacle tracking system and method combined with image data, which is installed on a vehicle, and obtains information such as the curvature and slope of the curve in front of the road through image data integration, so as to correct the obstacle originally estimated by Kalman The current position makes the estimated result more in line with the actual position of the obstacle.

Please refer to Figure 1, which is a block diagram of the obstacle tracking system combined with image data of the present invention. The system 10 of the present invention includes a pre-processing module 11, an obstacle determination module 13, an obstacle tracking module 17, an obstacle path module 18, and an obstacle information filtering module 20. The pre-processing module 11 includes a vehicle dynamic positioning module 12, a map data module 14 and a road path module 16 for finding multiple candidate paths of the vehicle.

The vehicle dynamic positioning module 12 includes a satellite positioning system (GPS) to obtain the position of the vehicle, especially the longitude and latitude coordinates; further speaking, the vehicle dynamic positioning module 12 can obtain the state of the vehicle, including speed and direction. The dynamic information of the vehicle such as the direction; the map data module 14 is used to obtain the map data path information near the position of the vehicle. The map data is the point cloud image of the three-dimensional image captured by the camera in advance, so the road map data path information will be included in the map data path information. Information about the centerline, direction, lane line, signs, road curvature, gradient, etc.; the road path module 16 will dynamically position the vehicle position obtained by the vehicle dynamic positioning module 12 with the map data path information obtained by the map data module 14 After the comparison, find out the multiple candidate paths of the vehicle. In more detail, the road path module 16 finds the paths that can be used near the vehicle (including multiple lanes of the same road), and then finds candidate paths based on the vehicle's dynamic information (such as the direction of travel); obstacle paths The module 18 is connected to the road path module 16, receives the tracking result of an obstacle, determines which candidate path the obstacle is on, and predicts the movement path of the obstacle; and an obstacle information filtering module 20, which connects the obstacle The object path module 18 estimates the position of the obstacle according to the movement path of the obstacle, and outputs the position information.

The obstacle tracking system 10 combined with image data of the present invention further includes at least one environment sensing device 22, which can be at least one of a camera, a radar, a LiDAR, or a combination thereof. The environment sensing device 22 is connected to the obstacle determination module 13, and after capturing the road information in front of the vehicle, it is provided to the obstacle determination module 13 to determine whether there is an obstacle in front, or whether the object in front is an obstacle, this part The method for the obstacle judging module 13 to judge whether the object detected in the front is an obstacle is not a technical feature of the present invention, and will not be repeated here. The obstacle judgment module 13 is connected to an obstacle tracking module 17. The obstacle tracking module 17 receives the preceding obstacle determined by the obstacle determination module 13 and performs obstacle tracking. The tracking result includes obstacle parameters such as the longitudinal position of the obstacle, the direction of travel, and the speed. This part of the method for the obstacle tracking module 17 to track the trajectory of the obstacle ahead is not a technical feature of the present invention, and will not be repeated here. The obstacle tracking result generated by the obstacle tracking module 17 is provided to the obstacle path module 18.

The application process of the block diagram in Fig. 1 of the present invention is shown in the flowchart in Fig. 2. First, in step S10, a vehicle dynamic positioning module 12 is used to obtain the position of the vehicle, which is the longitude and latitude coordinates located by satellites, and a map information module 14 is used to obtain map information near the location of the vehicle; step S12 In the example, the road path module 16 compares the position of the vehicle obtained by the vehicle dynamic positioning module 12 with the map information path information obtained by the map information module 14 to find out which multiple candidate paths can be used near the vehicle ; Then in step S14, receive the tracking result of an obstacle, determine which candidate path the obstacle is on according to the tracking result of the obstacle, and estimate the movement path of the obstacle, such as when the obstacle enters the curve, The possible movement path in the curve, etc.; finally, in step S16, the obstacle information filtering module estimates and outputs the position of the obstacle according to the movement path of the obstacle.

The road path module 16 finds a schematic diagram of the road path, please refer to Figure 3A. According to the map information, there are three road paths 32 on the road near the vehicle 30, that is, there are three lanes. The three road paths are determined after judgment. 32 are all candidate paths. Next, referring to Figure 3B, the obstacle path module 18 uses the known position of the obstacle 34 to pair with the road path 32 to create an obstacle path 36, that is, the path the obstacle 34 is currently traveling on, and estimate the obstacle The moving path 38 of the object 34, so the moving path 38 of the obstacle 34 in Figure 3B is also the estimated future position of the obstacle.

Please further refer to Figure 4 for the system of the present invention, which is a detailed block diagram of the obstacle tracking system of the present invention combined with map data. The system 10 of the present invention further includes a coordinate conversion module 15 which is connected to the vehicle dynamic positioning module. Group 12, map data module 14 and road path module 16, convert the coordinate system of the vehicle position provided by the vehicle dynamic positioning module 12 and map data path information provided by the map data module 14 from world coordinates to the vehicle The coordinates are used for the road path module 16 to use the vehicle as the center to find candidate paths that are available near the vehicle. The coordinate conversion module 15 is further connected to the environment sensing device 22 to convert the coordinate system of the sensing result of the environment sensing device 22 from world coordinates to vehicle coordinates.

The obstacle path module 18 further includes a path matching module 182 and a path estimation module 184. After receiving the obstacle tracking result, the path matching module 182 matches the tracking result with the candidate path to find Find the candidate path where the obstacle is located, and set the candidate path as an obstacle path. The path estimation module 184 uses an optimal path equation to predict the movement path of the obstacle on the obstacle path based on the obstacle tracking result, and provides the obstacle information filter module 20 to the path. The obstacle information filtering module 20 uses the estimation result as a filtering parameter to estimate the position of the obstacle.

The sensing result of the environment sensing device 22 is converted into vehicle coordinates by the coordinate conversion module 15 and then provided to the obstacle judgment module 13 for judgment, and the obstacle tracking module generates the obstacle tracking result. The obstacle tracking module 17 is connected to the path matching module 182 and provides the obstacle tracking result to the path matching module 182.

Figure 5 is a detailed flowchart of finding candidate paths in the obstacle tracking method combined with map data of the present invention. First, the process of step S20 starts, and the pre-processing module 11 is activated. In step S22, the road path module 16 reads information from the vehicle dynamic positioning module 12, including the position of the vehicle and the state of the vehicle. In step S24, the road path module 16 reads the map data path information from the map data module 14. In step S26, the coordinate conversion module 15 converts the coordinate system of the vehicle position provided by the vehicle dynamic positioning module 12 and the map data path information provided by the map data module from world coordinates to three-dimensional vehicle coordinates. Then, as described in steps S28 to S38, the road path module starts to operate and judges the road paths found one by one. First, in step S28, initially assume that the road path i is 1, and then determine whether i is less than the total number of road paths in step S30, if not, the road path is 0, and then the road information is output in step S32. Conversely, if it is determined in step S30 that i is less than the total number of road paths, it means that this path does exist. Then, in step S36, it is determined whether the distance R between the road path (i-th PATH) and the vehicle (host) is less than a predetermined value. Set value. This default value is usually a constant, that is, the width of the road. Assuming that the road width is 2 meters, if the distance R is less than 2 meters, it means that the road path is a path that can be used by vehicles. In step S38, the road path is set as a candidate path. If step S36 determines that the distance R between the road path and the vehicle is greater than the preset value, it means that the road path is too far away from the vehicle and is not a path that can be used by the vehicle. Therefore, if the judgment result of step S36 is no, go to step S34, i+1=2, start to judge the second road path, and repeat steps S30~S36. Until it is determined in step S30 that i is greater than the total number of paths, that is, all paths have been determined to be completed. For example, assuming that the total number of road paths is 3, and i=4, i>4, it means that all three road paths are judged to be completed, and it is convenient to output the information of the three road paths in step S32.

Figure 6 is a detailed flow chart of the path matching and path estimation in the obstacle tracking method combined with map data of the present invention to obtain a better obstacle position detection result. Step S40 starts. In step S42, the front obstacle information detected by the environment sensing device 22 is read, and in step S44, the coordinate conversion module 15 performs a three-dimensional coordinate conversion to convert it into vehicle coordinates. In step S46, the obstacle information is provided to the obstacle judgment module 13 for obstacle judgment. Step S48 starts from the first obstacle, and judges each obstacle one by one. First, in step S50, it is judged whether the obstacle j is less than the number of obstacles. If not, it means that the judgment of all obstacles has been completed, and the result is output in step S52. If the determination result of step S50 is yes, then step S54 is performed, and the obstacle tracking module 17 determines whether the j-th obstacle is being tracked, and if not, it is set as a new obstacle, as described in step S56. Conversely, if the j-th obstacle is already being tracked, the tracking result of the obstacle is provided to the path matching module 182. Next, in steps S58~S60, the path matching module 182 reads the candidate path in the road path module 16, and performs path matching with the tracking result of the j-th obstacle to find the obstacle path where the obstacle is. . Then in step S62, the obstacle path is added to the path estimation module 18 to estimate the movement path of the obstacle. Finally, in step S64, the movement path of the obstacle is added to the obstacle information filtering module 20 for filtering, and the position of the obstacle is estimated. At this point, the information filtering of the jth obstacle has been completed, go to step S66 to add 1 to j, and repeat steps S50~S64 to perform path matching, path estimation and filtering for the j+1th obstacle, and find the jth +1 the location of obstacles. Steps S50 to S66 are repeated until step S50 confirms that all obstacles are judged to be completed, and the result is output, and the output result is the obstacle position obtained by filtering.

進行最佳路徑比對，

，其中

為最佳路徑方程式，

為最大可能路徑，

為路徑方程式。換言之，本發明選取在障礙物縱向位置、行進方向、速度…等各項參數皆最符合之路徑作為路徑配對的結果，將配對成功之路徑設為障礙物路徑，並將之加入為障礙物參數。此外，利用障礙物路徑及障礙物之動態進行障礙物路徑預估，並將預估結果作為濾波參數。最終，障礙物資訊濾波模組20會利用障礙物參數及濾波參數，計算出較佳的偵測結果，使預估的障礙物位置更符合實際位置。 Obstacle path pairing will use an optimal path equation. Tracking results by obstacles

Compare the best path,

,among them

Is the best path equation,

Is the maximum possible path,

Is the path equation. In other words, the present invention selects the path that best matches the obstacle longitudinal position, direction of travel, speed, etc. as the result of path pairing, sets the path that is successfully paired as the obstacle path, and adds it as the obstacle parameter . In addition, the obstacle path is estimated by using the obstacle path and the dynamics of the obstacle, and the estimated result is used as the filter parameter. Finally, the obstacle information filtering module 20 uses the obstacle parameters and the filtering parameters to calculate a better detection result, so that the estimated obstacle position is more in line with the actual position.

In summary, the obstacle tracking system and method combined with map data provided by the present invention combines dynamic vehicle positioning with map data path information to obtain a map of the location of the vehicle and its vicinity, and find out which vehicles can be used next. Candidate path. The accuracy of the candidate path can include multiple lanes on the same road. The present invention uses the map data path information to estimate the position of the obstacle, finds the path (lane) where the obstacle is, and can correct the accuracy of the Kalman filter to detect the current position of the obstacle ahead, and further enable the obstacle detection on the curve. The test effect is more stable. Furthermore, by applying the system and method of the present invention, even when the obstacle exceeds the preset detection range (ROI) of the sensor, by predicting the obstacle’s planned movement route, the obstacle can still be separated from the candidate. The paths are paired to estimate the movement path of obstacles without the problem of system dissociation.

Only the above are only preferred embodiments of the present invention, and are not used to limit the scope of implementation of the present invention. Therefore, all equivalent changes or modifications made in accordance with the characteristics and spirit of the application scope of the present invention shall be included in the patent application scope of the present invention.

10: Obstacle tracking system combined with picture materials 11: Pre-processing module 12: Vehicle dynamic positioning module 13: Obstacle judgment module 14: Graphics module 15: Coordinate conversion module 16: Road path module 17: Obstacle tracking module 18: Obstacle Path Module 182: Path matching module 184: Path Estimation Module 20: Obstacle Information Filtering Module 22: Environmental Sensing Module 30: Vehicle 32: road path 34: Obstacles 36: Obstacle Path 38: Movement path of obstacles

Figure 1 is a block diagram of an obstacle tracking system combined with image data of the present invention. Figure 2 is a flowchart of the obstacle tracking method combined with image data of the present invention. Figure 3A is a schematic diagram of the road path module of the present invention finding a road path. FIG. 3B is a schematic diagram of an obstacle path and an estimated movement path of the obstacle generated by the obstacle path module of the present invention. Figure 4 is a detailed block diagram of the obstacle tracking system combined with image data of the present invention. Figure 5 is a detailed flowchart of finding candidate paths in the obstacle tracking method combined with map data of the present invention. Figure 6 is a detailed flow chart of the path matching and path estimation in the obstacle tracking method combined with map data of the present invention to obtain a better obstacle position detection result.

10: Obstacle tracking system combined with picture materials

11: Pre-processing module

12: Vehicle dynamic positioning module

13: Obstacle judgment module

14: Graphics module

15: Coordinate conversion module

16: Road path module

17: Obstacle tracking module

18: Obstacle Path Module

182: Path matching module

184: Path Estimation Module

20: Obstacle Information Filtering Module

22: Environmental sensing device

Claims (19)

An obstacle tracking system combined with map data, which is installed on a vehicle and includes: A vehicle dynamic positioning module for obtaining the location of the vehicle; A map data module for obtaining map data path information near the location of the vehicle; A road path module, which compares the position of the vehicle obtained by the vehicle dynamic positioning module with the path information of the map data obtained by the map data module to find multiple candidate paths of the vehicle; An obstacle path module connected to the road path module, receiving the tracking result of at least one obstacle, determining one of the candidate paths where the obstacle is located, and predicting the moving path of the obstacle; and An obstacle information filtering module is connected to the obstacle path module, and the position of the obstacle is estimated and output according to the movement path of the obstacle. The obstacle tracking system combined with map data as described in claim 1, wherein the vehicle dynamic positioning module can further obtain the state of the vehicle, including speed, orientation, position and direction of travel. The obstacle tracking system combined with map data as described in claim 1, wherein the map data path information includes the centerline, direction, lane line, signs, road curvature and slope of the road. The obstacle tracking system combined with map data as described in claim 1 further includes at least one environmental sensing device that captures road information in front of the vehicle to determine whether there is an obstacle in front of the vehicle. The obstacle tracking system combined with map data according to claim 1, wherein the tracking result of the obstacle includes obstacle parameters such as the longitudinal position, travel direction, and speed of the obstacle. The obstacle tracking system combined with map data as described in claim 1, wherein the obstacle path module further includes: A path matching module, receiving the tracking result of the obstacle, and matching the tracking result of the obstacle with the candidate paths to find an obstacle path among the candidate paths where the obstacle is located; and A path estimation module uses an optimal path equation to predict the movement path of the obstacle on the obstacle path according to the tracking result of the obstacle. The obstacle tracking system combined with map data as described in claim 6 further includes an obstacle tracking module connected to the path matching module to provide a tracking result of the obstacle. The obstacle tracking system combined with map data according to claim 6, wherein the path estimation module provides an estimation result of the movement path of the obstacle to the obstacle information filtering module, and the obstacle information filtering The module uses the estimated result as a filter parameter to estimate the position of the obstacle. The obstacle tracking system combined with map data as described in claim 1, further comprising a coordinate conversion module, which connects the vehicle dynamic positioning module, the map data module and the road path module to determine the position of the vehicle And the coordinate system of the route information of the map is converted from world coordinates to vehicle coordinates. According to claim 4, the obstacle tracking system combined with map data further includes a coordinate conversion module connected to the environment sensing device to convert the sensing result of the at least one environment sensing device from the coordinate system The world coordinates are converted into vehicle coordinates and output to an obstacle judgment module to judge whether there is an obstacle in front or whether an object in front is the obstacle, and then the judgment result is sent to the obstacle information filtering module group. An obstacle tracking method combined with image data includes the following steps: Use a vehicle dynamic positioning module to obtain the location of a vehicle, and use a map data module to obtain map data path information near the location of the vehicle; Compare the position of the vehicle obtained by the vehicle dynamic positioning module with the information on the route of the map obtained by the map module to find out a plurality of candidate routes for the vehicle; Receiving the tracking result of an obstacle, judging that the obstacle is in one of the candidate paths, and predicting the moving path of the obstacle; and According to the movement path of the obstacle, estimate the position of the obstacle and output it. The obstacle tracking method combined with map data according to claim 11, wherein the vehicle dynamic positioning module can further obtain the state of the vehicle, including speed, orientation, position, and direction of travel. The obstacle tracking method combined with map data according to claim 11, wherein the map data path information includes the centerline, direction, lane line, signs, road curvature and slope of the road. According to claim 11, the obstacle tracking method combined with map data further includes using at least one environmental sensing device to capture road information in front of the vehicle to determine whether there is the obstacle in front of the vehicle. The obstacle tracking method combined with map data according to claim 11, wherein the tracking result of the obstacle includes obstacle parameters such as the longitudinal position, travel direction, and speed of the obstacle. The obstacle tracking method combined with map data according to claim 11, wherein the step of estimating the movement path of the obstacle further includes: After receiving the tracking result of the obstacle, match the tracking result of the obstacle with the candidate paths to find an obstacle path among the candidate paths where the obstacle is located; and Then, according to the tracking result of the obstacle, an optimal path equation is used to estimate the moving path of the obstacle on the obstacle path. The obstacle tracking method combined with map data as described in claim 16 further includes using an obstacle tracking module to provide a tracking result of the obstacle. The obstacle tracking method combined with map data according to claim 16, wherein the estimated result of the moving path of the obstacle is used as a filter parameter to estimate the position of the obstacle. The obstacle tracking method combined with map data as described in claim 11 further includes using a coordinate conversion module to convert the position of the vehicle and the coordinate system of the map data path information from world coordinates to vehicle coordinates.

Images