TWI774543B - Obstacle detection method - Google Patents

Abstract

An obstacle detection method includes: (A) after receiving a plurality of point cloud data and a plurality of images, for each point cloud data, obtaining a to-be-fused image from the images according to the point cloud data; (B) For each image to be fused, obtain the first N th image to be synthesized earlier than the image to be fused; (C) for each point cloud data, according to the corresponding image to be fused and the previous N th image to be synthesized, obtain an integrated image; (D) for each point cloud data, align and project the point cloud data to the corresponding integrated image; and (E) for each point cloud data, according to the aligned projected point cloud data and the integrated image, use An obstacle identification model identifies the obstacle type corresponding to each obstacle located in the front.

Description

Obstacle detection method

The present invention relates to an obstacle detection method, in particular to an obstacle detection method integrating different sensors.

Mobile vehicles have introduced more and more sensors and intelligent algorithms to continuously enhance their environmental perception and flexible movement capabilities. In various fields, factories use the AMR (Autonomous Mobile Robot) industry for warehousing and handling and self-driving cars run on the road. Obstacle detection, stop or avoidance requires the use of multiple sensors, such as radar, lidar, and cameras to detect and avoid obstacles. When the mobile vehicle travels on any path, it is the most important requirement to be able to avoid obstacles and identify the type of obstacles.

Each sensing element has its own advantages, and it is now mostly used on various vehicle platforms with heterogeneous sensing fusion. However, the sensing data and sampling time of each sensor are different. For example, the radar system obtains a radar data every 50ms, the lidar system obtains a point cloud data every 100ms, and the camera system captures an image every 33ms. When the sensors are very different, there is a problem that time and space cannot be synchronized, and it is impossible to know what the obstacles are. It is necessary to propose a solution.

Therefore, the object of the present invention is to provide an obstacle detection method for synchronizing space and time and recognizing the type of obstacles when fusing multiple sensors.

Therefore, the obstacle detection method of the present invention is implemented by a computing device, the computing device is electrically connected to an image capturing device and a lidar module disposed on a mobile carrier, and the image capturing device is used for capturing images. A series of images of a plurality of obstacles located in front of the mobile vehicle, the lidar module is used to obtain a series of point cloud data located in front of the mobile vehicle and including the obstacles, and the obstacle detection method includes the following step:

(A) After receiving the point cloud data of the lidar module and the images of the image capture device, for each point cloud data, according to the point cloud data, the images captured by the image capture device Obtain a pair of images to be fused corresponding to the point cloud data in the image;

(B) For each image to be fused, obtain the Nth image to be synthesized earlier than the image to be fused, where N≧3;

(C) For each point cloud data, obtain a pair of integrated images corresponding to the point cloud data according to the corresponding image to be fused and the first N image to be synthesized;

(D) for each point cloud data, align the projection of the point cloud data to the corresponding integrated image; and

(E) For each point cloud data, according to the point cloud data and the integrated image after the alignment and projection in step (D), an obstacle identification model is used to identify each obstacle located in front of the mobile vehicle corresponding to obstacle category.

The effect of the present invention lies in: obtaining the image to be fused corresponding to the point cloud data from the images captured by the image capturing device by the computing device, and according to the corresponding image to be fused and the previous N th image to be synthesized The integrated image is obtained from the image, and the point cloud data is aligned and projected to the corresponding integrated image, so as to achieve spatial and temporal synchronization when fusing the sensing data of the lidar module and the image capture device. According to the integrated image and point cloud data after alignment and projection, the obstacle identification model is used to identify the obstacle type corresponding to each obstacle, so as to achieve the purpose of identifying the obstacle.

Referring to FIG. 1 , an embodiment of the obstacle detection method of the present invention is implemented by an obstacle detection system disposed on a mobile vehicle. The obstacle detection system includes an image capturing device 11 , a lidar module 12 , a radar module 13 and an electrical connection to the image capturing device 11 , the lidar module 12 and the The computing device 14 of the radar module 13 .

The image capturing device 11 is used for continuously capturing a series of images of obstacles located in front of the moving vehicle. In this embodiment, the image capturing device 11 is, for example, a camera, and captures an image every 33 ms.

The lidar module 12 is used to continuously obtain a series of point cloud data in front of the moving vehicle and including the obstacles. In this embodiment, the lidar module 12 is, for example, a lidar sensor, and obtains a point cloud data every 100ms. The point cloud data obtained every 100 ms is the point cloud data scanned from right to left by the lidar module 12 within a preset period from a first scan time point to an Nth scan time point.

The radar module 13 is used to continuously obtain a series of radar data in front of the moving vehicle and including the obstacles. In this embodiment, the radar module 13 is, for example, a radar sensor, and obtains a piece of radar data every 50 ms.

The computing device 14 is, for example, a processor or a microprocessor and other chips capable of performing computing functions.

The operation of each element in the obstacle detection system will be described below by means of an embodiment of the obstacle detection method of the present invention, which includes a radar fusion process and an image fusion process.

Referring to FIG. 1 and FIG. 2 , the radar fusion process of the embodiment of the obstacle detection method of the present invention includes the following steps.

In step 21, after the computing device 14 receives the point cloud data of the lidar module 12 and the radar data of the radar module 13, for each point cloud data, according to the point cloud data, automatically A pair of radar data to be fused corresponding to the point cloud data is obtained from the radar data obtained by the radar module 13 .

It is worth mentioning that step 21 also includes the following sub-steps (refer to FIG. 3 ).

In sub-step 211, after the computing device 14 receives the point cloud data of the lidar module 12 and the radar data of the radar module 13, for each point cloud data, according to the point cloud data, A first candidate radar data and a second candidate radar data are extracted from the radar data obtained by the radar module 13, and the acquisition time of the first candidate radar data is earlier than the acquisition time of the point cloud data, and is the same as that of the point cloud data. The point cloud data is obtained with the shortest time interval, and the second candidate radar data is the previous radar data of the first candidate radar data.

In sub-step 212, for each point cloud data, the computing device 14 obtains, according to the first candidate radar data corresponding to the point cloud data, the corresponding one of the obstacles located in front of the moving vehicle. a first specific obstacle parameter. In this embodiment, the first specific obstacle parameter is a first specific obstacle distance between the mobile vehicle and the specific obstacle. However, in other embodiments, the first specific obstacle parameter may also be A first specific obstacle position of the specific obstacle or a first specific obstacle speed of the specific obstacle is not limited thereto.

In sub-step 213, for each point cloud data, the computing device 14 obtains a second specific obstacle parameter corresponding to the specific obstacle according to the second candidate radar data corresponding to the point cloud data. In this embodiment, the second specific obstacle parameter is a second specific obstacle distance between the mobile vehicle and the specific obstacle. However, in other embodiments, the second specific obstacle parameter may also be A second specific obstacle position of the specific obstacle or a second specific obstacle speed of the specific obstacle is not limited thereto.

、該點雲資料所對應之第一候選雷達資料的獲得時間

及第二候選雷達資料的獲得時間

、該第一特定障礙物參數

以及該第二特定障礙物參數

，利用以下公式(1)計算該特定障礙物在該點雲資料之獲得時間t的一推測障礙物參數

。在本實施方式中，該推測障礙物參數為該移動載具與該特定障礙物的一推測障礙物距離，然而，在其他實施方式中，該推測障礙物參數亦可為該特定障礙物的一推測障礙物位置，或是該特定障礙物的一推測障礙物速度，並不以此為限。

…(1) In sub-step 214, for each point cloud data, the computing device 14 obtains the time according to the point cloud data

, the acquisition time of the first candidate radar data corresponding to the point cloud data

and the acquisition time of the second candidate radar data

, the first specific obstacle parameter

and the second specific obstacle parameter

, use the following formula (1) to calculate an estimated obstacle parameter of the specific obstacle at the acquisition time t of the point cloud data

. In this embodiment, the estimated obstacle parameter is an estimated obstacle distance between the mobile vehicle and the specific obstacle. However, in other embodiments, the estimated obstacle parameter may also be a value of the specific obstacle. The estimated obstacle position, or an estimated obstacle speed of the specific obstacle, is not limited to this.

…(1)

為一誤差值。 in,

is an error value.

In sub-step 215 , for each point cloud data, the computing device 14 converts the estimated obstacle parameter into an estimated obstacle parameter according to the estimated obstacle parameter and a first conversion rule of the obstacle parameter such as the distance to the obtained interval interval. It is worth noting that, the first conversion rule first uses a first conversion function of an obstacle parameter to time according to the estimated obstacle parameter to calculate a presumed acquisition time corresponding to the estimated obstacle parameter, and then, According to the estimated acquisition time and the plurality of radar sampling intervals, the radar sampling interval in which the estimated acquisition time falls is used as the estimated acquisition interval. The first conversion function is obtained according to the acquisition time of each piece of radar data and the indicated obstacle parameter of the specific obstacle, and the acquisition time point of each radar data to the acquisition time point of the next radar data , but does not include the time interval defined by the acquisition time point of the next piece of radar data, which constitutes a radar sampling interval. FIG. 4 illustrates the first conversion function 41 and the radar sampling intervals 42 .

In sub-step 216, for each point cloud data, the computing device 14 obtains the radar data corresponding to the estimated interval from the radar data obtained by the radar module 13 according to the estimated interval, as the to-be-fused Radar data. The radar data corresponding to the estimated acquisition interval is the radar data corresponding to the acquisition time at the starting time point of the estimated acquisition interval.

It is worth noting that, since the radar module 13 obtains one piece of radar data every 50ms, the lidar module 12 obtains one piece of point cloud data every 100ms, plus the radar module 13 and the lidar module 12 start There may be errors in the starting time of data acquisition, so the time when the radar module 13 and the lidar module 12 acquire data may not be synchronized. The acquisition time of the point cloud data is as a criterion to obtain the corresponding radar data to be fused, so that the obtained radar data to be fused can be synchronized in time with the corresponding point cloud data as much as possible.

In step 22, for each point cloud data, the computing device 14 uses a one-directional bounding box algorithm to locate the obstacles in the point cloud data. In this embodiment, the directional bounding box algorithm is based on principal component analysis to locate the obstacles.

In step 23, the computing device 14 uses a Kalman filter to track the located obstacles in each point cloud data to obtain the mobile vehicle corresponding to each obstacle located in front of the mobile vehicle obstacle distance.

In step 24, for each radar data to be fused, the computing device 14 uses the directional bounding box algorithm to locate the obstacles in the radar data to be fused.

In step 25, the computing device 14 uses the Kalman filter to track the obstacles located in each radar data to be fused to obtain the obstacle velocity corresponding to each obstacle in front of the moving vehicle .

Referring to FIG. 1 and FIG. 5 , the image fusion process of the embodiment of the obstacle detection method of the present invention includes the following steps.

In step 51, after the computing device 14 receives the point cloud data of the lidar module 12 and the images of the image capturing device 11, for each point cloud data, according to the point cloud data, from the A pair of to-be-fused images corresponding to the point cloud data are obtained from the images captured by the image capturing device 11 .

It is worth mentioning that step 51 includes the following sub-steps (refer to FIG. 6 ).

In sub-step 511, after receiving the point cloud data of the lidar module 12 and the images of the image capturing device 11, the computing device 14, for each point cloud data, according to the point cloud data, automatically A first candidate image and a second candidate image are taken out of the images captured by the image capturing device 11 , and the capturing time of the first candidate image is earlier than the acquisition time of the point cloud data, and it is the same as the point cloud data. The time interval of the acquisition time is the shortest, and the second candidate image is the previous image of the first candidate image.

In sub-step 512, for each point cloud data, the computing device 14 obtains, according to the first candidate image corresponding to the point cloud data, a corresponding one of the specific obstacles among the obstacles located in front of the moving vehicle. The third specific obstacle parameter. In this embodiment, the third specific obstacle parameter is a third specific obstacle distance between the mobile vehicle and the specific obstacle. However, in other embodiments, the third specific obstacle parameter may also be A third specific obstacle position of the specific obstacle or a third specific obstacle speed of the specific obstacle is not limited thereto. Wherein, when the third specific obstacle parameter is the third specific obstacle distance or the third specific obstacle speed, the computing device 14 also needs to obtain the third specific obstacle distance according to the focal length of the image capturing device 11 or the third specific obstacle speed.

In sub-step 513, for each point cloud data, the computing device 14 obtains a fourth specific obstacle parameter corresponding to the specific obstacle according to the second candidate image corresponding to the point cloud data. In this embodiment, the fourth specific obstacle parameter is a fourth specific obstacle distance between the mobile vehicle and the specific obstacle. However, in other embodiments, the fourth specific obstacle parameter may also be A fourth specific obstacle position of the specific obstacle or a fourth specific obstacle speed of the specific obstacle is not limited thereto. Wherein, when the fourth specific obstacle parameter is the fourth specific obstacle distance or the fourth specific obstacle speed, the computing device 14 also needs to obtain the fourth specific obstacle distance according to the focal length of the image capturing device 11 or the fourth specific obstacle speed.

、該點雲資料所對應之第一候選影像的拍攝時間

及第二候選影像的拍攝時間

、該第三特定障礙物參數

以及該第四特定障礙物參數

，利用以下公式(2)計算該特定障礙物在點雲資料之獲得時間t的另一推測障礙物參數

。在本實施方式中，該另一推測障礙物參數為該移動載具與該特定障礙物的另一推測障礙物距離，然而，在其他實施方式中，該另一推測障礙物參數亦可為該特定障礙物的另一推測障礙物位置，或是該特定障礙物的另一推測障礙物速度，並不以此為限。

…(2) In sub-step 514, for each point cloud data, the computing device 14 obtains the time according to the point cloud data

, the shooting time of the first candidate image corresponding to the point cloud data

and the shooting time of the second candidate image

, the third specific obstacle parameter

and the fourth specific obstacle parameter

, use the following formula (2) to calculate another estimated obstacle parameter of the specific obstacle at the acquisition time t of the point cloud data

. In this embodiment, the other estimated obstacle parameter is another estimated obstacle distance between the mobile vehicle and the specific obstacle. However, in other embodiments, the other estimated obstacle parameter can also be the other estimated obstacle parameter. Another estimated obstacle position of the specific obstacle, or another estimated obstacle speed of the specific obstacle, is not limited thereto.

…(2)

為一誤差值。 in,

is an error value.

In sub-step 515, for each point cloud data, the computing device 14 converts the other estimated obstacle parameter according to the other estimated obstacle parameter and an obstacle parameter such as a second conversion rule of distance to shooting interval to a presumed shooting interval. It is worth noting that, the second conversion rule first uses a second conversion function of an obstacle parameter to time according to the other estimated obstacle parameter to calculate a speculative shot corresponding to the other estimated obstacle parameter. Then, according to the estimated shooting time and a plurality of image sampling intervals, the image sampling interval in which the estimated shooting time falls is regarded as the estimated shooting interval. Wherein, the second conversion function is obtained according to the shooting time of each image and the indicated obstacle parameter of the specific obstacle, from the shooting time point of each image to the shooting time point of the next image , but does not include the time interval defined by the shooting time point of the next image, which constitutes an image sampling interval. FIG. 7 illustrates the second conversion function 71 and the image sampling intervals 72 .

In sub-step 516 , for each point cloud data, the computing device 14 obtains an image corresponding to the estimated shooting interval from the images captured by the image capturing device 11 according to the estimated shooting interval, as the image to be fused. The image corresponding to the estimated shooting interval is the image corresponding to the shooting time at the start time point of the estimated shooting interval.

It is worth noting that, since the image capturing device 11 captures an image every 33ms, the lidar module 12 obtains a point cloud data every 100ms, adding the image capturing device 11 and the lidar module 12 to start There may be errors in the starting time of data acquisition, so the time when the image capturing device 11 and the lidar module 12 acquire data may not be synchronized. The acquisition time of the point cloud data is based on the acquisition of the corresponding image to be fused, so that the obtained image to be fused and the corresponding point cloud data can be synchronized in time as far as possible.

In step 52 , for each image to be fused, the computing device 14 obtains the first N-th image to be synthesized that is captured earlier than the image to be fused, where N≧3. In this embodiment, N is 3.

In step 53, for each point cloud data, the computing device 14 obtains a pair of integrated images corresponding to the point cloud data according to the corresponding image to be fused and the previous N th image to be synthesized. In this embodiment, for each point cloud data, the computing device 14 compares a part of the corresponding image to be fused (ie, the left half) with a part of the previous N-th image to be synthesized (ie, the right half of the image) half) to perform image synthesis to obtain the integrated image corresponding to the point cloud data. Since the lidar module 12 scans each time from right to left to obtain each point cloud data, the obtained point There is a scan delay between the right half of the scene in front of the moving vehicle and the left half of the scene in front of the moving vehicle in the cloud data (that is, the part of the obtained point cloud data that corresponds to the left half of the scene It is relatively new, and the part corresponding to the right half of the scene in the obtained point cloud data is older), so by integrating the left half of the corresponding image to be fused with the right half of the previous N-th image to be synthesized, the synthesis can be achieved. The latter integrated image is closer to the scanning condition of the lidar module 12, thereby achieving spatial synchronization.

In step 54, for each point cloud data, the computing device 14 aligns and projects the point cloud data to the corresponding integrated image.

It is worth mentioning that step 54 includes the following sub-steps (refer to FIG. 8 ).

In sub-step 541, for each point cloud data, the computing device 14 converts the point cloud coordinate system according to the coordinate system conversion parameter set related to the point cloud coordinate system of the point cloud data and the coordinate system of the pixel coordinate system of the integrated image to the point cloud data. Convert cloud data to 2D point cloud data. It is worth mentioning that the coordinate conversion parameter set includes an external parameter matrix and an internal parameter matrix, and the external parameter matrix is obtained according to the installation position of the lidar module 12 and the installation position of the image capturing device 11 . The internal parameter matrix is obtained according to the focal distance in pixels of the image capturing device 11 and the image center coordinates. Since the focus of the present invention is not on how to convert the point cloud data into 2D point cloud data, the details of the calculation can be found in the website http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/parameters.html description, and details are not repeated here.

In sub-step 542, for each 2D point cloud data, the computing device 14 obtains edge features of each object in the 2D point cloud data.

In sub-step 543, for the integrated image corresponding to each point cloud data, the computing device 14 obtains the edge feature of each object in the integrated image.

In sub-step 544, for each point cloud data, the computing device 14 calculates the corresponding edge feature of each object in the corresponding two-dimensional point cloud data and the corresponding edge feature of each object in the integrated image. The 2D point cloud data is aligned with the corresponding integrated image.

In step 55, for each point cloud data, the computing device 14 uses an obstacle recognition model to identify each point cloud data in front of the mobile vehicle according to the point cloud data and the integrated image after the alignment and projection in step 54. An obstacle category corresponding to an obstacle. It is worth mentioning that the obstacle recognition model is trained using a machine learning algorithm based on multiple training data marked with obstacle positions and obstacle categories. Each training data includes a training point cloud after alignment and projection. The training point cloud data and training image after alignment and projection are marked with obstacle positions and obstacle categories.

To sum up, in the obstacle detection method of the present invention, the computing device 14 obtains the to-be-fused image corresponding to the point cloud data from the images captured by the image capturing device 11 , and obtains the image to be fused corresponding to the point cloud data from the radar module. The radar data to be fused corresponding to the point cloud data is obtained from the radar data obtained by 13, so as to synchronize the data obtained by different sensors. The integrated image is obtained from the Nth image to be synthesized, and the point cloud data is aligned and projected to the corresponding integrated image, so that the point cloud data and the integrated image are spatially fused. The integrated image and point cloud data are used to identify the obstacle type corresponding to each obstacle by using the obstacle identification model, so as to achieve the effect of identifying obstacles, so the object of the present invention can indeed be achieved.

However, the above are only examples of the present invention, and should not limit the scope of implementation of the present invention. Any simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the patent specification are still included in the scope of the present invention. within the scope of the invention patent.

11: Video shooting device 12: LiDAR module 13: Radar module 14: Computing device 41: The first conversion function 42: Radar sampling interval 71: Second conversion function 72: Image sampling interval 21~25: Steps 51~55: Steps 211~216: Substeps 511~516: Substeps 541~544: Substeps

Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: 1 is a block diagram illustrating an obstacle detection system implementing an embodiment of the obstacle detection method of the present invention; 2 is a flow chart illustrating a radar fusion procedure of an embodiment of the obstacle detection method of the present invention; 3 is a flowchart illustrating how a computing device obtains a radar data to be fused corresponding to a point cloud data; 4 is a schematic diagram illustrating a first transfer function and a plurality of radar sampling intervals; 5 is a flow chart illustrating an image fusion process of an embodiment of the obstacle detection method of the present invention; 6 is a flowchart illustrating how the computing device obtains an image to be fused corresponding to the point cloud data; FIG. 7 is a schematic diagram illustrating a second transfer function and a plurality of image sampling intervals; and FIG. 8 is a flowchart illustrating how the computing device aligns and projects the point cloud data to the corresponding integrated image.

51~55: Steps

Claims (8)

An obstacle detection method is implemented by a computing device, the computing device is electrically connected with an image capturing device and a lidar module arranged on a mobile carrier, and the image capturing device is used for continuously capturing images located in the A series of images of a plurality of obstacles in front of the moving vehicle, the lidar module is used to continuously obtain a series of point cloud data in front of the moving vehicle and including the obstacles, and the obstacle detection method includes the following steps : (A) After receiving the point cloud data of the lidar module and the images of the image capture device, for each point cloud data, according to the point cloud data, the image captured by the image capture device Obtain a pair of images to be fused corresponding to the point cloud data in the other images; (B) for each image to be fused, obtain the Nth image to be synthesized earlier than the image to be fused, where N≧3; (C) For each point cloud data, obtain a pair of integrated images corresponding to the point cloud data according to the corresponding image to be fused and the previous N-th image to be synthesized; (D) for each point cloud data, align the point cloud data Projecting to the corresponding integrated image, wherein step (D) includes the following sub-steps, (D-1) for each point cloud data, according to a point cloud coordinate system related to the point cloud data and the pixel coordinates of the integrated image The coordinate conversion parameter group of the coordinate system conversion of the system, and the point cloud data is converted into a two-dimensional point cloud data; (D-2) For each two-dimensional point cloud data, obtain the edge features of each object in the two-dimensional point cloud data; (D-3) For the integrated image corresponding to each point cloud data, obtain the integrated image The edge feature of each object; and (D-4) for each point cloud data, according to the edge feature of each object in the corresponding two-dimensional point cloud data and the edge feature of each object in the corresponding integrated image, the The corresponding two-dimensional point cloud data is aligned with the corresponding integrated image; and (E) for each point cloud data, using a The obstacle identification model identifies the obstacle type corresponding to each obstacle located in front of the mobile vehicle. The obstacle detection method according to claim 1, wherein step (A) includes the following sub-steps: (A-1) after receiving the point cloud data of the lidar module and the image capturing device After waiting for the images, for each point cloud data, according to the point cloud data, a first candidate image and a second candidate image are extracted from the images captured by the image capturing device, and the first candidate image was captured earlier At the acquisition time of the point cloud data, and the time interval with the acquisition time of the point cloud data is the shortest, the second candidate image is the previous image of the first candidate image; (A-2) For each point cloud data , according to the point cloud data corresponding to the For the first candidate image, obtain a first specific obstacle parameter corresponding to a specific obstacle among the obstacles located in front of the mobile vehicle; (A-3) For each point cloud data, according to the corresponding point cloud data obtain a second specific obstacle parameter corresponding to the specific obstacle; (A-4) For each point cloud data, according to the acquisition time of the point cloud data, the first parameter corresponding to the point cloud data The shooting time of a candidate image and a second candidate image, the first specific obstacle parameter and the second specific obstacle parameter, calculate an estimated obstacle parameter of the specific obstacle at the time of obtaining the point cloud data; (A- 5) For each point cloud data, according to the conversion rule of the estimated obstacle parameter and an obstacle parameter to the shooting interval, convert the estimated obstacle parameter to an estimated shooting interval; and (A-6) For each point cloud data , according to the estimated shooting interval, an image corresponding to the estimated photographing interval is obtained from the images captured by the image photographing device as the to-be-fused image. The obstacle detection method according to claim 2, wherein, in the sub-step (A-4), the estimated obstacle parameter f _I (t) is also calculated according to the following formula:

Among them, t is the acquisition time of the point cloud data, t _-1 is the shooting time of the first candidate image corresponding to the point cloud data, t _-2 is the shooting time of the second candidate image corresponding to the point cloud data, f _I (t _-1 ) is the first specific obstacle parameter, f _I (t _-2 ) is the second specific obstacle parameter, and ε is an error. The obstacle detection method according to claim 1, wherein, in step (C), for each point cloud data, a part of the corresponding image to be fused and a part of the previous N th image to be synthesized are compared Image synthesis is performed to obtain the integrated image corresponding to the point cloud data. The obstacle detection method according to claim 1, further comprising the following steps: (F) after receiving the point cloud data of the lidar module, for each point cloud data, use a unidirectional bounding box algorithm , locate the obstacles in the point cloud data; and (G) use a Kalman filter to track the obstacles located in each point cloud data to obtain the mobile vehicle and the location in the mobile vehicle The obstacle distance corresponding to each obstacle ahead. The obstacle detection method according to claim 1, wherein the computing device is further electrically connected to a radar module disposed on the mobile vehicle, and the radar module is used to continuously obtain information that is located in front of the mobile vehicle and includes A series of radar data of the obstacles, the obstacle detection method further includes the following steps: (H) after receiving the point cloud data of the lidar module and the radar data of the radar module, for For each point cloud data, according to the point cloud data, a pair of radar data to be fused corresponding to the point cloud data is obtained from the radar data obtained by the radar module; (1) for each radar data to be fused, use a a directional bounding box algorithm to locate the obstacles in the radar data to be fused; and (J) Using a Kalman filter to track the obstacles located in each radar data to be fused to obtain the obstacle velocity corresponding to each obstacle in front of the moving vehicle. The obstacle detection method according to claim 6, wherein step (H) comprises the following sub-steps: (H-1) after receiving the point cloud data of the lidar module and the data of the radar module After waiting for the radar data, for each point cloud data, according to the point cloud data, extract a first candidate radar data and a second candidate radar data from the radar data obtained by the radar module, the first candidate radar data The acquisition time of the data is earlier than the acquisition time of the point cloud data, and the time interval with the acquisition time of the point cloud data is the shortest, and the second candidate radar data is the previous radar data of the first candidate radar data; (H- 2) For each point cloud data, according to the first candidate radar data corresponding to the point cloud data, obtain a first specific obstacle parameter corresponding to a specific obstacle in the obstacles located in front of the mobile vehicle; (H-3) For each point cloud data, obtain a second specific obstacle parameter corresponding to the specific obstacle according to the second candidate radar data corresponding to the point cloud data; (H-4) For each point cloud data , according to the acquisition time of the point cloud data, the acquisition time of the first candidate radar data and the second candidate radar data corresponding to the point cloud data, the first specific obstacle parameter and the second specific obstacle parameter, calculate the The specific obstacle when the point cloud data is obtained (H-5) For each point cloud data, according to the conversion rule of the estimated obstacle parameter and an obstacle parameter to the obtained interval, convert the estimated obstacle parameter to a predicted obtained interval; and (H-6) for each point cloud data, obtain radar data corresponding to the inferred interval from the radar data obtained by the radar module according to the inferred interval, as the radar data to be fused. According to the obstacle detection method described in claim 7, in sub-step (H-4), the estimated obstacle parameter f _R (t) is also calculated according to the following formula:

Among them, t is the acquisition time of the point cloud data, t _-1 is the acquisition time of the first candidate radar data corresponding to the point cloud data, and t _-2 is the acquisition time of the second candidate radar data corresponding to the point cloud data Time, f _R (t _-1 ) are the first specific obstacle parameter, and f _R (t _-2 ) is the second specific obstacle parameter, and ε is an error value.

Images