WO2021134296A1

WO2021134296A1 - Obstacle detection method and apparatus, and computer device and storage medium

Info

Publication number: WO2021134296A1
Application number: PCT/CN2019/130114
Authority: WO
Inventors: 吴伟; 何明; 叶茂盛; 邹晓艺; 许双杰; 许家妙; 曹通易
Original assignee: 深圳元戎启行科技有限公司
Priority date: 2019-12-30
Filing date: 2019-12-30
Publication date: 2021-07-08
Also published as: CN113424079A

Abstract

An obstacle detection method and apparatus (500), and a computer device (104) and a storage medium. The method comprises: obtaining a current point cloud image frame and a previous point cloud image frame (202); by performing point cloud matching on the current point cloud image frame and the previous point cloud image frame, determining each first point cloud in the current point cloud image frame and matched second point clouds in the previous point cloud image frame (204); calculating a displacement vector of each second point cloud with respect to the matched first point cloud (206); clustering the second point clouds with the displacement vector greater than a threshold to obtain at least one group of target point clouds (208); and determining each group of target point clouds as an obstacle (210). Thus, the accuracy of obstacle detection can be improved.

Description

Obstacle detection method, device, computer equipment and storage medium

Technical field

This application relates to an obstacle detection method, device, computer equipment and storage medium.

Background technique

Self-driving cars, also known as self-driving cars, computer-driven cars, or wheeled mobile robots, rely on artificial intelligence, visual computing, radar, monitoring devices, and global positioning equipment to work together to allow computers to work without any human initiative. Under operation, the smart car of the motor vehicle is automatically operated. In the process of unmanned driving, the most important thing is to identify the moving objects in the driving area. In the traditional way, obstacle detection is mainly performed through machine learning models. However, the machine learning model can only detect the target obstacle categories that have participated in the training, such as people, cars, etc., for the obstacle categories that are not involved in the training in the driving area (such as animals, cones, etc.), the machine learning model cannot be passed. Perform correct recognition, thereby increasing the probability of unmanned vehicle safety accidents due to failure to correctly recognize moving objects.

Summary of the invention

According to various embodiments disclosed in the present application, an obstacle detection method, device, computer equipment, and storage medium are provided.

An obstacle detection method includes:

Obtain the point cloud image of the current frame and the point cloud image of the previous frame;

By performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, it is determined that each first point cloud in the point cloud image of the current frame matches the second point cloud in the point cloud image of the previous frame;

Calculate the displacement vector of each second point cloud relative to the matched first point cloud;

Clustering the second point cloud whose displacement vector is greater than the threshold to obtain at least one set of target point cloud;

Each group of target point cloud is judged as an obstacle.

A point cloud-based target tracking device includes:

The point cloud matching module is used to obtain the point cloud image of the current frame and the point cloud image of the previous frame; by performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, each point cloud image in the current frame is determined A first point cloud that matches a second point cloud in the previous frame of point cloud image;

The point cloud clustering module is used to calculate the displacement vector of each second point cloud relative to the matched first point cloud; clustering the second point clouds whose displacement vector is greater than the threshold to obtain at least one set of target points cloud;

The judging module is used to judge each group of target point clouds as an obstacle.

A computer device, including a memory and one or more processors, the memory stores computer readable instructions, and when the computer readable instructions are executed by the processor, the one or more processors execute The following steps:

By performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, determining that each first point cloud in the point cloud image of the current frame matches the second point cloud in the point cloud image of the previous frame;

Each group of target point cloud is judged as an obstacle.

One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors perform the following steps:

Each group of target point cloud is judged as an obstacle.

The details of one or more embodiments of the present application are set forth in the following drawings and description. Other features and advantages of this application will become apparent from the description, drawings and claims.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. A person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

Fig. 1 is an application scenario diagram of an obstacle detection method in an embodiment;

Figure 2 is a schematic flowchart of an obstacle detection method in an embodiment;

3A is a schematic diagram of a bird's-eye view of the spatial coordinate system in an embodiment;

Fig. 3B is a three-dimensional schematic diagram of a space coordinate system in an embodiment;

FIG. 4 is a schematic flowchart of a step of performing point cloud matching according to point features according to an embodiment;

Figure 5 is a block diagram of an obstacle detection device in an embodiment;

Figure 6 is a block diagram of a computer device in one embodiment.

Detailed ways

In order to make the technical solutions and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

The obstacle detection method provided in this application can be applied in a variety of application environments. For example, it can be applied to the application environment of automatic driving as shown in FIG. 1, and it can include a laser sensor 102 and a computer device 104. The computer device 104 can communicate with the laser sensor 102 via a network. The laser sensor 102 can collect multi-frame point cloud images of the surrounding environment, and the computer device 104 can acquire the point cloud image of the previous frame and the point cloud image of the current frame collected by the laser sensor 102, and use the above obstacle detection method to analyze the point cloud image of the previous frame. The image and the point cloud image of the current frame are processed to realize the detection of obstacles. The laser sensor 102 may be a sensor carried by an automatic driving device, and may specifically include a laser radar, a laser scanner, and the like.

In one of the embodiments, as shown in FIG. 2, an obstacle detection method is provided. Taking the method applied to the computer device 104 in FIG. 1 as an example for description, the method includes the following steps:

Step 202: Obtain a point cloud image of the current frame and a point cloud image of the previous frame.

Among them, the laser sensor may be mounted by a device capable of automatic driving. For example, it can be carried by an unmanned vehicle, or it can be carried by a vehicle including an autonomous driving model. Laser sensors can be used to collect environmental data within the visual range.

Specifically, a laser sensor can be set up on the unmanned vehicle in advance, and the laser sensor emits a detection signal to the driving area according to a preset time frequency, and compares the signal reflected by objects in the driving area with the detection signal to obtain the surrounding environment data , And generate corresponding point cloud images based on environmental data. The point cloud image refers to the object in the scanning environment recorded in the form of points, and the collection of point clouds corresponding to multiple points on the surface of the object. The point cloud may specifically include a variety of information such as the three-dimensional space position coordinates, laser reflection intensity, and color of a single point on the surface of the object in the environment in the space coordinate system. The spatial coordinate system may be a Cartesian coordinate system. As shown in Figure 3, the spatial coordinate system takes the center point of the laser sensor as the origin, the horizontal plane horizontal to the laser sensor as the reference plane (that is, the xoy plane), and the axis horizontal to the moving direction of the unmanned vehicle is the Y axis; The axis in the datum plane that passes through the origin and is transversely perpendicular to the origin is the X axis; the axis that passes through the origin and is perpendicular to the datum plane is the Z axis. FIG. 3A is a schematic diagram of a bird's-eye view of the spatial coordinate system in an embodiment. Fig. 3B is a three-dimensional schematic diagram of the spatial coordinate system in an embodiment.

The laser sensor embeds a time stamp in the collected point cloud image, and sends the point cloud image with the embedded time stamp to the computer device. In one of the embodiments, the laser sensor can send the locally stored point cloud images collected within a preset time period to the computer device at one time. The computer equipment sorts the point cloud images according to the timestamps in the point cloud images, and determines the point cloud image of the previous order among the two point cloud images adjacent in time interval as the point cloud image of the current frame. The point cloud image is determined as the point cloud image of the previous frame.

Step 204: By performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, it is determined that each first point cloud in the point cloud image of the current frame matches the second point cloud in the point cloud image of the previous frame. .

Specifically, the point cloud image of the current frame and the point cloud image of the previous frame are input into the trained matching model. The matching model can extract the spatial position information of the second point cloud point from the point cloud image of the previous frame, and filter out the point cloud image of the current frame that the distance from the spatial position of the second point cloud is less than the preset distance threshold. The first cloud. For example, when the distance threshold is q and the spatial position coordinates of the second point cloud are (x2, y2, z2), the first point cloud coordinates filtered out at this time are (x1, y1, z1), where x1∈x2± q, y1∈y2±q, z1∈z2±q. The matching model may specifically be a neural network model, a dual path network model (DPN, DualPath Network), a support vector machine, or a logistic regression model.

Further, the matching model extracts the first point feature from the first point cloud, extracts the second point feature from the second point cloud, and performs similarity matching between the first point feature and the second point feature, and the similarity The largest first point cloud is determined as the point cloud matching the second point cloud. The first point cloud and the corresponding second point cloud at this time may be point cloud data collected for the same point on the surface of the same object at different times. The matching model traverses each second point cloud in the point cloud image of the previous frame until each second point cloud matches the corresponding first point cloud. Since the laser sensor can collect multiple point cloud images within 1 second, within the adjacent collection time interval, the position coordinates of the moving obstacles in the driving area in the adjacent two frames of images are not much different, so the matching model As long as the first point cloud and the second point cloud separated by a distance less than the threshold are matched for similarity, the first point cloud corresponding to the second point cloud can be matched.

In another embodiment, if the feature matching results are all less than the preset threshold, the matching model increases the distance threshold correspondingly, and filters out the corresponding first point cloud from the current frame point cloud image according to the increased distance threshold. Then point cloud matching is performed based on the re-screened first point cloud.

In another embodiment, the training step of the matching model includes: collecting a large number of point cloud images, and dividing the collected point cloud images into a plurality of image pairs according to the collection time of the point cloud images. The image pair includes the point cloud image of the current frame and the point cloud image of the previous frame. The point cloud image of the current frame and the point cloud of the previous frame can be marked for matching points, and then the marked point cloud image of the current frame and the point cloud image of the previous frame are input into the matching model, and the matching training model is adjusted according to the matching point markings. Parameters in the model.

In another embodiment, simulation software may be used to generate multiple current point cloud images with matching point markers and a point cloud image of the previous frame.

Step 206: Calculate the displacement vector of each second point cloud relative to the matched first point cloud.

Among them, the displacement vector refers to a directed line segment that takes the coordinates of the moving mass point at the current moment in the space coordinate system as the starting point, and the coordinates of the moving mass point at the next moment in the space coordinate system as the end point.

Specifically, the computer device extracts the spatial position coordinates of the point from the second point cloud, extracts the spatial position coordinates of the point from the first point cloud matching the second point cloud, and calculates the spatial position of the second point cloud The coordinates are taken as the starting point, and the spatial position coordinates of the matched first point cloud are taken as the end point, so as to obtain the displacement vector of the second point cloud relative to the matched first point cloud. Since the first point cloud and the second point cloud are based on the point cloud data collected by the same laser sensor, the spatial position coordinates contained in the first point cloud and the spatial position coordinates contained in the second point cloud are also based on the same spatial coordinates The coordinate value calculated by the system, so the computer device only needs to directly connect the space position coordinates of the first point cloud with the space position coordinates of the corresponding second point cloud in the space coordinate system to draw the displacement vector.

Step 208: Perform clustering on the second point cloud whose displacement vector is greater than the threshold to obtain at least one set of target point cloud.

Specifically, the computer device calculates the absolute value of the displacement vector corresponding to each second point cloud based on the preset absolute value calculation formula, and determines the absolute value of the displacement vector as a single point on the surface of the object in the driving area. The movement amplitude of the movement within the collection interval of the sensor. For example, in the above example, when the spatial position coordinates of the first point cloud are (x1, y1, z1) and the coordinates of the second point cloud are (x2, y2, z2), the corresponding absolute value calculation formula is:

Further, the computer device filters out the second point cloud whose absolute value is greater than the preset threshold value from the previous frame of point cloud image (for the convenience of description, the second point cloud whose absolute value of the displacement vector is greater than the preset threshold value will be recorded as pending in the following. Classification point cloud), and use the to-be-classified point cloud as the point cloud data collected on the surface of the moving object in the driving area, and then the computer device clusters the to-be-classified point cloud data to obtain at least one set of target point clouds. The computer equipment can cluster the point cloud to be classified in a variety of ways. For example, the computer device may determine the spatial position coordinates of each point cloud to be classified, and divide the point cloud to be classified into a group of target point clouds whose distance between adjacent spatial positions is less than a threshold. For another example, the computer device may group the point clouds to be classified based on clustering algorithms such as k-means clustering and Mean-Shift clustering.

Step 210: Determine each group of target point clouds as an obstacle.

Specifically, the computer device judges different groups of target point clouds as point cloud data collected for different obstacles. At the same time, the computer equipment obtains the spatial position coordinates of the target point cloud in the same group and the corresponding displacement vector, and determines the direction pointed by the displacement vector as the movement direction of the target point cloud. Space position coordinates and movement direction, determine the movement direction and position coordinates of the corresponding obstacle.

In another embodiment, the computer device determines the movement speed of the corresponding obstacle according to the collection interval time of the laser sensor and the absolute value of the displacement vector, and predicts the space of the obstacle after a preset time period according to the movement speed and movement direction. Position information, so as to generate obstacle avoidance instructions corresponding to the predicted position information. For example, when the computer equipment predicts that the obstacle is moving in a straight line at the current direction of motion and the speed of motion, and it may collide with an unmanned vehicle after a preset time, the computer equipment generates a braking instruction corresponding to the prediction result to make the unmanned The car stopped running.

In another embodiment, if the point cloud image of the previous frame is the image data collected by the laser sensor at time t-1, the point cloud image of the current frame is the image data collected at time t. The computer equipment determines the movement speed of the point cloud A according to the collection interval time of the laser sensor and the absolute value of the displacement vector corresponding to the point cloud A at time t-1, and predicts the point cloud at t+ according to the movement speed and movement direction. 2 Time location information. Therefore, when performing point cloud matching on the point cloud image collected at time t+1 and the point cloud image collected at time t+2, the predicted spatial position information of the point cloud A at time t+2 can be used to assist the point cloud matching .

More specifically, the spatial position information of point cloud A is extracted from the point cloud image collected at time t+1 based on the matching model, and the point cloud image collected at time t+2 is filtered according to the spatial position information of point cloud A When multiple first point clouds separated by a distance less than a preset distance threshold are obtained, the computer device obtains the spatial position coordinates of each first point cloud that has been filtered out, and subtracts the predicted point cloud from the spatial position coordinates of the first point cloud The spatial position coordinates of A at t+2 time, the coordinate difference is obtained, and the absolute value of the coordinate difference is calculated to obtain the absolute difference. After that, the computer device further filters out the point cloud data whose absolute difference is less than the preset difference threshold from the plurality of first point clouds. The difference threshold is less than the distance threshold.

Since the collection interval of the laser sensor is generally 10ms, it can be considered that from t-1 to t+2, the movement speed and direction of the obstacle remain unchanged, so the movement direction of the obstacle at t-1 and Motion speed, the estimated spatial position coordinates at t+2 have a high degree of confidence, so it can be considered that the first point cloud matching the second point cloud is near the estimated spatial position coordinates, so it can be based on The estimated spatial coordinates further filter multiple point cloud data, thereby reducing the amount of calculation for the subsequent matching model to calculate the point cloud, thereby improving the efficiency of obstacle detection.

In this embodiment, by performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, it is possible to determine the first point cloud corresponding to the same point on the surface of the same object in the two previous and next point cloud images. And the second point cloud; by calculating the displacement vector of the matched first point cloud and the second point cloud, the movement displacement of the object in the driving area within the laser sensor collection interval can be determined, so that the movement displacement can be calculated Points greater than the threshold are determined as moving points, and points with displacement less than the threshold are determined as stationary points, and then by clustering the moving points, all moving obstacles in the driving area can be identified. Compared with the traditional obstacle detection of known categories through the machine learning model, this solution does not need to identify the obstacle category. It only needs to judge whether the obstacle in the driving area is a movement obstacle, so as to avoid movement obstacles. The purpose of things.

In one embodiment, by performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, it is determined that each first point cloud in the point cloud image of the current frame matches the first point cloud image of the previous frame. Two point clouds include:

Step 302: Acquire the point feature corresponding to the second point cloud extracted from the previous frame of point cloud image, and the point feature corresponding to the first point cloud extracted from the current point cloud image;

Step 304: Perform similarity matching on the point feature corresponding to the second point cloud and the point feature corresponding to the first point cloud;

Step 306: Determine the first point cloud and the second point cloud whose similarity matching results meet the conditions as matching point cloud pairs.

Specifically, the computer device can input the collected point cloud image into the matching model, and the matching model will rasterize the collected point cloud image, and divide the three-dimensional space corresponding to the point cloud image into a plurality of columnar grids. , And determine the grid to which the point cloud belongs according to the spatial position coordinates in the point cloud. The matching model calculates the point cloud in the grid based on the preset convolution kernel, thereby extracting high-dimensional point features from the point cloud. The matching model can be one of a variety of neural network models. For example, the matching model may be a convolutional neural network model. The matching model performs feature matching on the point features extracted from the second point cloud with the point features extracted from multiple first point clouds, and determines the first point cloud with the largest matching degree as the second point cloud Matching point cloud.

In this embodiment, because the matching model is a pre-trained machine learning model, the current frame of point cloud image and the previous frame of point cloud image can be accurately matched based on the matching model, so that subsequent computer equipment can be based on successful matching. The point cloud judges the movement state of the obstacle.

In one embodiment, acquiring the point feature corresponding to the second point cloud extracted from the point cloud image of the previous frame includes: performing structural processing on the point cloud image of the previous frame to obtain the processing result; The second point cloud in the cloud image is encoded to obtain the point feature corresponding to the second point cloud.

Specifically, the matching model may perform structured case processing on the point cloud image of the previous frame, and obtain the processing result after the structured processing. For example, the matching model can perform rasterization processing on the point cloud image of the previous frame, and can also perform voxelization processing on the point cloud image of the previous frame. Taking rasterization processing as an example, the computer equipment can rasterize the plane with the laser sensor as the origin, and divide the plane into multiple grids. The structured space after the structuring process can be a columnar space, and the points can be distributed in the columnar space corresponding to the vertical axis of the grid, that is, the abscissa and ordinate of the points in the columnar space are in the corresponding grid coordinates, and each columnar The space may include at least one point. After that, the matching model can encode each second point cloud according to the structured processing result to obtain the point feature corresponding to the second point cloud.

It is easy to understand that the point feature extraction method described above can also be used to extract the point feature of each first point cloud in the point cloud image of the current frame.

In this embodiment, by structuring the point cloud image, the corresponding processing result can be obtained; by encoding the processing result, the point feature can be extracted from the point cloud data. Since the point cloud data will not be affected by illumination, target movement speed, etc., the matching model suffers less interference when extracting point features, thereby ensuring the accuracy of feature extraction.

In one embodiment, calculating the displacement vector of each second point cloud relative to the matched first point cloud includes: acquiring the spatial position information of the first point cloud and the spatial position information of the matched second point cloud; Based on the spatial position information of the first point cloud and the matched spatial position information of the second point cloud, the displacement vector of the second point cloud relative to the corresponding first point cloud is determined.

Among them, the point cloud is composed of point cloud data, and the point cloud data includes information such as the three-dimensional coordinates of the point in the space coordinate system and the laser reflection intensity.

Specifically, the computer device respectively extracts the three-dimensional coordinates of the corresponding point in the space coordinate system from the first point cloud and the matched second point cloud, and uses the three-dimensional coordinates extracted from the second point cloud as the starting point of the displacement vector , The three-dimensional coordinates extracted from the first point cloud are used as the end point of the displacement vector, and the displacement vector containing the start point and the end point is inserted into the point cloud data corresponding to the second point cloud to obtain for example ((x1, y1, z1)

) Point cloud data.

In another embodiment, if the point cloud image of the previous frame and the point cloud image of the current frame are collected by two laser sensors installed on the left and right sides of the unmanned vehicle, the computer device also needs to perform the point cloud image and The previous frame of point cloud image is coordinated, and the point cloud images collected based on different coordinate systems are converted into two frames of point cloud images in the same coordinate system, so as to carry out the displacement vector based on the two frames of point cloud images in the same coordinate system Calculation. Specifically, the computer equipment obtains the point cloud images collected by the two laser sensors installed on the left and right sides of the unmanned vehicle at the same time, and records the point cloud image collected based on the laser sensor installed on the left side of the unmanned vehicle as the left point cloud image , The point cloud image collected by the laser sensor installed on the right side of the unmanned vehicle is recorded as the right point cloud image. The computer equipment extracts the space coordinates in the left point cloud collected for the object point A from the left point cloud image, and extracts the space coordinates in the right point cloud collected for the same object point A from the right point cloud image, and based on the left The space coordinates of the point cloud and the space coordinates of the right point cloud are coordinate conversion of the point cloud image of the previous frame or the point cloud image of the previous frame, so that the point cloud image of the current frame and the point cloud image of the previous frame are in the same spatial coordinate system.

In this embodiment, by extracting the spatial position information of the point cloud from the matched point cloud pair, accurate displacement vector determination can be made based on the extracted spatial position coordinates, so that subsequent displacement vectors can be derived from multiple point clouds based on the displacement vector. Filter out the motion point cloud.

In one embodiment, clustering the second point cloud whose displacement vector is greater than the threshold value to obtain at least one set of target point cloud includes: filtering out the second point cloud whose displacement vector is greater than the threshold value from the point cloud image of the previous frame, denoted as Point cloud to be classified; obtain the spatial position information of the point cloud to be classified; determine the movement direction of the point cloud to be classified based on the displacement vector; cluster the point cloud to be classified with the similar movement direction and the distance between adjacent spatial positions less than the threshold to obtain at least A set of target point clouds.

Specifically, when the displacement vector of each second point cloud relative to the corresponding first point cloud is calculated, the computer calculates the absolute value of the displacement vector to obtain each second point cloud and the corresponding first point cloud The separation distance between, that is, the movement amplitude of the points on the surface of the object within the collection interval can be obtained. The computer screens out the point cloud to be classified whose motion amplitude is greater than the preset motion threshold from the previous frame of point cloud image, and extracts the spatial position coordinates and the corresponding displacement vector from the point cloud to be classified. Among them, the movement threshold can be set according to actual needs. For example, when the collection interval of the laser sensor is 10 ms, the movement threshold can be set to 0.05 meters.

Further, the computer determines the current movement direction of the unmanned vehicle through the direction of the compass installed in the unmanned vehicle, and determines each of the spatial coordinates established with the center point of the laser sensor as the origin according to the current movement direction of the unmanned vehicle. The direction represented by each axis. For example, after the computer determines that the current unmanned vehicle is moving north according to the direction of the compass, the computer determines the positive Y axis in the three-dimensional space coordinate system as north, the positive X axis as east, and the negative Y axis as south, and The negative X axis is judged to be West. The computer projects the displacement vector to the XOY plane, and calculates the included angle between the projected displacement vector and the X-axis and Y-axis based on the start and end coordinates of the displacement vector, and then calculates the included angle and the X-axis and Y-axis according to the calculation. The corresponding direction determines the movement direction of the point cloud to be classified corresponding to the displacement vector.

Further, the computer clusters the point clouds to be classified according to the movement direction and spatial position coordinates of the point clouds to be classified, and divides the point clouds to be classified into the same group of point clouds that have similar movement directions and whose distance between adjacent spatial positions is less than a threshold.

In this embodiment, since the calculated movement direction and spatial position information will not be affected by the ambient light and the shape of the obstacle, compared to the traditional clustering of obstacles relying on simple geometric shape features and environmental information, this solution It can effectively reduce the environmental impact, thereby greatly improving the accuracy of clustering.

In an embodiment, the above obstacle detection method further includes: calculating the motion parameters of the corresponding obstacle according to the displacement vector of each target point cloud in the same group of target point clouds; generating corresponding obstacle avoidance instructions according to the motion parameters of the obstacle; Control unmanned vehicles to drive based on obstacle avoidance commands.

Among them, the motion parameter refers to the information value of information such as the speed of the object and the direction of the object.

Specifically, the computer device obtains the acquisition frequency of the laser sensor, and calculates the movement speed of the corresponding point cloud based on the acquisition frequency and the displacement vector of each point cloud in the same group. The computer device performs a weighted average calculation on the movement speeds of all point clouds in the same group to obtain the average speed, and judges the average speed as the movement speed of the corresponding obstacle. At the same time, the computer equipment obtains the displacement vector of each point cloud in the same group, determines the movement direction of each point cloud based on the displacement vector, and calculates the movement direction of all point clouds in the same group to obtain the movement of the corresponding obstacle direction.

Further, the computer equipment determines the area where the corresponding obstacle is located according to the spatial coordinates of each point cloud in the same group, and compares the area where the obstacle is not located with the area where the unmanned vehicle is located, so as to determine whether the obstacle is different from that of the unmanned vehicle. People and vehicles are in the same lane. If the obstacle and the unmanned vehicle are in the same lane, the computer equipment obtains the separation distance between the obstacle and the unmanned vehicle, and calculates the unmanned vehicle and the obstacle according to the current speed, maximum braking deceleration and separation distance of the unmanned vehicle The probability of collision. When the collision probability is greater than the threshold, the computer equipment generates a lane change instruction and controls the unmanned vehicle to perform lane change processing based on the lane change instruction; if the collision probability is less than the threshold, the computer equipment generates a deceleration instruction and controls the unmanned vehicle based on the deceleration instruction Slow down.

In this embodiment, because the computer device synthesizes the space position coordinates of the obstacle, the movement speed, and the current vehicle speed to generate the obstacle avoidance instruction, the generated obstacle avoidance instruction has a high degree of confidence, so that the unmanned vehicle can be based on the obstacle avoidance instruction Carrying out correct driving greatly improves the safety of unmanned driving.

In one embodiment, calculating the motion parameters of the corresponding obstacle according to the displacement vector of each target point cloud in the target point cloud includes: determining the motion direction of each target point cloud in the same group based on the displacement vector; and counting the corresponding motion directions The number of point clouds of the target point cloud; the motion direction of the target point cloud with the largest number of point clouds is determined as the motion direction of the corresponding obstacle.

Specifically, the computer device obtains the displacement vector of each target point cloud in the same group, and determines the movement direction of each target point cloud based on the displacement vector and the axis direction of the three-dimensional coordinate axis. The computer device counts the number of target point clouds corresponding to each different motion direction, and determines the motion direction with the largest number of target point clouds as the motion direction of the corresponding obstacle.

In this embodiment, since the movement directions of different components of the same obstacle are roughly the same, the movement direction of the most numerous target point cloud can be directly determined as the movement direction of the corresponding obstacle. In addition, because the quantity statistics belong to simple superposition calculations, this solution only needs to be based on simple calculations to obtain the movement direction of the corresponding obstacles, which not only saves the computing resources of the computer, but also improves the detection rate of obstacles.

In one embodiment, calculating the motion parameters of the corresponding obstacle according to the displacement vector of each target point cloud in the target point cloud includes: obtaining the acquisition frequency of the laser sensor; and determining the displacement value of each target point cloud in the same group based on the displacement vector ; Determine the movement speed of each target point cloud in the same group according to the acquisition frequency and displacement value; perform a mixed calculation on the movement speed of each target point cloud in the same group to obtain the movement speed of the corresponding obstacle.

Specifically, after obtaining the displacement vector of each target point cloud in the same group, the computer device performs an absolute value calculation on the displacement vector, and determines the calculation result after the absolute value calculation as the displacement value of the target point cloud. The computer equipment divides the displacement value by the collection interval time of the laser acquisition sensor to obtain the movement speed of the target point cloud, and performs a mixed calculation on the movement speed of each target point cloud in the same group to obtain the movement speed of the corresponding obstacle. The computer equipment can comprehensively calculate the movement speed of each target point cloud based on a variety of hybrid computing algorithms. For example, the computer equipment can calculate the average value of the movement speed of all target point clouds in the same group, and calculate the average speed obtained by the average value. Determine the movement speed of the corresponding obstacle. For another example, the computer device may remove a certain percentage of the target point cloud with the maximum motion speed and the target point cloud with the minimum motion speed in advance, and then perform an average calculation on the remaining point clouds.

In this embodiment, the computer device can determine the movement speed of the target point cloud according to the acquisition frequency and displacement vector of the laser sensor, and determine the movement speed of the corresponding obstacle according to the movement speed of each target point cloud, which is helpful for the computer equipment to root the obstacle. The movement speed of the object prompts or controls the unmanned equipment.

It should be understood that although the steps in the flowcharts of FIGS. 2 and 4 are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated in this article, there is no strict order for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in Figures 2 and 4 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. These sub-steps or stages The execution order of is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

In one of the embodiments, as shown in FIG. 5, an obstacle detection device 500 is provided, including: a point cloud matching module 502, a point cloud clustering module 504, and a determination module 506, wherein:

The point cloud matching module 502 is used to obtain the point cloud image of the current frame and the point cloud image of the previous frame; by performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, each point cloud image in the current frame is determined The first point cloud matches the second point cloud in the point cloud image of the previous frame.

The point cloud clustering module 504 is used to calculate the displacement vector of each second point cloud relative to the matched first point cloud; clustering the second point clouds whose displacement vector is greater than the threshold to obtain at least one set of target point clouds .

The determination module 506 is used to determine each group of target point clouds as an obstacle.

In one of the embodiments, the above-mentioned point cloud matching module 502 is also used to obtain the point feature corresponding to the second point cloud extracted from the point cloud image of the previous frame, and the point feature corresponding to the first point cloud extracted from the current point cloud image. Point features; similarity matching is performed on the point features corresponding to the second point cloud and the point features corresponding to the first point cloud; the first point cloud and the second point cloud whose similarity matching results meet the conditions are determined as matching point clouds Correct.

In one of the embodiments, the above-mentioned point cloud matching module 502 is further configured to perform structural processing on the point cloud image of the previous frame to obtain the processing result; and encode the second point cloud in the point cloud image of the previous frame based on the processing result , Get the point feature corresponding to the second point cloud.

In one of the embodiments, the above-mentioned point cloud clustering module 504 is also used to obtain the spatial position information of the first point cloud and the spatial position information of the matched second point cloud; based on the spatial position information and the relative spatial position information of the first point cloud The spatial position information of the matched second point cloud determines the displacement vector of the second point cloud relative to the corresponding first point cloud.

In one of the embodiments, the above-mentioned point cloud clustering module 504 is further configured to filter the second point cloud whose displacement vector is greater than the threshold value from the previous frame of point cloud image, and record it as the point cloud to be classified; to obtain the space of the point cloud to be classified Position information; determine the moving direction of the point cloud to be classified based on the displacement vector; cluster the point clouds to be classified with similar moving directions and the distance between adjacent spatial locations less than a threshold to obtain at least one set of target point clouds.

In one of the embodiments, the obstacle detection device 500 includes an obstacle avoidance instruction generating module 508, which is used to calculate the motion parameters of the corresponding obstacle according to the displacement vector of each target point cloud in the same group of target point clouds; The motion parameters generate corresponding obstacle avoidance instructions; based on the obstacle avoidance instructions, the unmanned vehicle is controlled to drive.

In one of the embodiments, the obstacle avoidance instruction generation module 508 is further configured to determine the movement direction of each target point cloud in the same group based on the displacement vector; count the number of point clouds of the target point cloud corresponding to each movement direction; The movement direction of the target point cloud with the largest number of point clouds is determined as the movement direction of the corresponding obstacle.

In one of the embodiments, the obstacle avoidance command generation module 508 is also used to obtain the acquisition frequency of the laser sensor; determine the displacement value of each target point cloud in the same group based on the displacement vector; determine the displacement value of each target point cloud in the same group according to the acquisition frequency and displacement value. The movement speed of each target point cloud; the mixed calculation is performed on the movement speed of each target point cloud in the same group to obtain the movement speed of the corresponding obstacle.

For the specific limitation of the obstacle detection device, please refer to the above limitation of the obstacle detection method, which will not be repeated here. Each module in the above obstacle detection device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules may be embedded in the form of hardware or independent of the processor in the computer equipment, or may be stored in the memory of the computer equipment in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 6. The computer equipment includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. The database of the computer equipment is used to store detection data. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer readable instruction is executed by the processor to realize an obstacle detection method.

Those skilled in the art can understand that the structure shown in FIG. 6 is only a block diagram of part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

A computer device includes a memory and one or more processors. The memory stores computer readable instructions. When the computer readable instructions are executed by the processor, the one or more processors execute the above method embodiments. step.

One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute A step of.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through computer-readable instructions. The computer-readable instructions can be stored in a non-volatile computer. In a readable storage medium, when the computer-readable instructions are executed, they may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered as the range described in this specification.

The above-mentioned embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims

An obstacle detection method includes:

Obtain the point cloud image of the current frame and the point cloud image of the previous frame;

By performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, determining that each first point cloud in the point cloud image of the current frame matches the second point cloud in the point cloud image of the previous frame;

Calculate the displacement vector of each second point cloud relative to the matched first point cloud;

Clustering the second point cloud whose displacement vector is greater than the threshold to obtain at least one set of target point cloud;

Each group of target point cloud is judged as an obstacle.
The method according to claim 1, wherein the point cloud image of the current frame and the point cloud image of the previous frame are matched to determine that each first point cloud in the current frame of point cloud image and The second point cloud that matches the point cloud image in the previous frame includes:

Acquiring the point feature corresponding to the second point cloud extracted from the previous frame of point cloud image, and the point feature corresponding to the first point cloud extracted from the current point cloud image;

Performing similarity matching on the point feature corresponding to the second point cloud and the point feature corresponding to the first point cloud;

The first point cloud and the second point cloud whose similarity matching result meets the condition are determined as the matched point cloud pair.
The method according to claim 2, wherein said acquiring the point feature corresponding to the second point cloud extracted from the point cloud image of the previous frame comprises:

Performing structural processing on the point cloud image of the previous frame to obtain a processing result;

Encoding the second point cloud in the point cloud image of the previous frame based on the processing result to obtain the point feature corresponding to the second point cloud.
The method according to claim 1, wherein the calculating the displacement vector of each second point cloud relative to the matched first point cloud comprises:

Acquiring the spatial location information of the first point cloud and the matching spatial location information of the second point cloud;

Based on the spatial position information of the first point cloud and the matched spatial position information of the second point cloud, a displacement vector of the second point cloud relative to the corresponding first point cloud is determined.
The method according to claim 1, wherein the clustering the second point cloud whose displacement vector is greater than the threshold to obtain at least one set of target point cloud comprises:

The second point cloud whose displacement vector is greater than the threshold is selected from the point cloud image of the previous frame and recorded as the point cloud to be classified;

Acquiring spatial position information of the point cloud to be classified;

Determining the movement direction of the point cloud to be classified based on the displacement vector;

Clustering the to-be-classified point clouds whose motion directions are similar and the distance between adjacent spatial positions is less than a threshold value to obtain at least one set of target point clouds.
The method according to claim 1, wherein the method further comprises:

Calculate the motion parameters of the corresponding obstacle according to the displacement vector of each target point cloud in the same group of target point clouds;

Generate a corresponding obstacle avoidance instruction according to the motion parameters of the obstacle;

The unmanned vehicle is controlled to run based on the obstacle avoidance instruction.
The method according to claim 6, wherein the motion parameter includes a motion direction; and the calculation of the motion parameter of the corresponding obstacle according to the displacement vector of each target point cloud in the target point cloud comprises:

Determine the movement direction of each target point cloud in the same group based on the displacement vector;

Count the number of point clouds of the target point cloud corresponding to each movement direction;

The movement direction of the target point cloud with the largest number of point clouds is determined as the movement direction of the corresponding obstacle.
The method according to claim 6, wherein the motion parameters include motion speed; the point cloud image is collected by a laser sensor; and the calculation is based on the displacement vector of each target point cloud in the target point cloud The motion parameters of the corresponding obstacles include:

Acquiring the acquisition frequency of the laser sensor;

Determine the displacement value of each target point cloud in the same group based on the displacement vector;

Determine the movement speed of each target point cloud in the same group according to the acquisition frequency and the displacement value;

The mixed calculation is performed on the movement speed of each target point cloud in the same group to obtain the movement speed of the corresponding obstacle.
An obstacle detection device includes:

The point cloud matching module is used to obtain the point cloud image of the current frame and the point cloud image of the previous frame; by performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, each point cloud image in the current frame is determined A first point cloud that matches a second point cloud in the previous frame of point cloud image;

The point cloud clustering module is used to calculate the displacement vector of each second point cloud relative to the matched first point cloud; clustering the second point clouds whose displacement vector is greater than the threshold to obtain at least one set of target points cloud;

The judging module is used to judge each group of target point clouds as an obstacle.
The device according to claim 9, wherein the point cloud matching module is further configured to:

Acquiring the point feature corresponding to the second point cloud extracted from the previous frame of point cloud image, and the point feature corresponding to the first point cloud extracted from the current point cloud image;

Performing similarity matching on the point feature corresponding to the second point cloud and the point feature corresponding to the first point cloud;

The first point cloud and the second point cloud whose similarity matching result meets the condition are determined as the matched point cloud pair.
The device according to claim 9, wherein the point cloud matching module is further configured to:

Performing structural processing on the point cloud image of the previous frame to obtain a processing result;

Encoding the second point cloud in the point cloud image of the previous frame based on the processing result to obtain the point feature corresponding to the second point cloud.
A computer device includes a memory and one or more processors. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the one or more processors, the one or more Each processor performs the following steps:

Obtain the point cloud image of the current frame and the point cloud image of the previous frame;

By performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, it is determined that each first point cloud in the point cloud image of the current frame matches the second point cloud in the point cloud image of the previous frame;

Calculate the displacement vector of each second point cloud relative to the matched first point cloud;

Clustering the second point cloud whose displacement vector is greater than the threshold to obtain at least one set of target point cloud;

Each group of target point cloud is judged as an obstacle.
The computer device according to claim 12, wherein the processor further executes the following steps when executing the computer-readable instruction:

Acquiring the point feature corresponding to the second point cloud extracted from the previous frame of point cloud image, and the point feature corresponding to the first point cloud extracted from the current point cloud image;

Performing similarity matching on the point feature corresponding to the second point cloud and the point feature corresponding to the first point cloud;

The first point cloud and the second point cloud whose similarity matching result meets the condition are determined as the matched point cloud pair.
The computer device according to claim 13, wherein the processor further executes the following steps when executing the computer-readable instruction:

Performing structural processing on the point cloud image of the previous frame to obtain a processing result;

Encoding the second point cloud in the point cloud image of the previous frame based on the processing result to obtain the point feature corresponding to the second point cloud.
The computer device according to claim 12, wherein the processor further executes the following steps when executing the computer-readable instruction:

Acquiring the spatial location information of the first point cloud and the matching spatial location information of the second point cloud;

Based on the spatial position information of the first point cloud and the matched spatial position information of the second point cloud, a displacement vector of the second point cloud relative to the corresponding first point cloud is determined.
One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the following steps:

Obtain the point cloud image of the current frame and the point cloud image of the previous frame;

By performing point cloud matching on the point cloud image of the current frame and the point cloud image of the previous frame, it is determined that each first point cloud in the point cloud image of the current frame matches the second point cloud in the point cloud image of the previous frame;

Calculate the displacement vector of each second point cloud relative to the matched first point cloud;

Clustering the second point cloud whose displacement vector is greater than the threshold to obtain at least one set of target point cloud;

Each group of target point cloud is judged as an obstacle.
The storage medium according to claim 16, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Acquiring the point feature corresponding to the second point cloud extracted from the previous frame of point cloud image, and the point feature corresponding to the first point cloud extracted from the current point cloud image;

Performing similarity matching on the point feature corresponding to the second point cloud and the point feature corresponding to the first point cloud;

The first point cloud and the second point cloud whose similarity matching result meets the condition are determined as the matched point cloud pair.
18. The storage medium according to claim 17, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Performing structural processing on the point cloud image of the previous frame to obtain a processing result;

Encoding the second point cloud in the point cloud image of the previous frame based on the processing result to obtain the point feature corresponding to the second point cloud.
The storage medium according to claim 16, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Acquiring the spatial location information of the first point cloud and the matching spatial location information of the second point cloud;

Based on the spatial position information of the first point cloud and the matched spatial position information of the second point cloud, a displacement vector of the second point cloud relative to the corresponding first point cloud is determined.
The storage medium according to claim 16, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

The second point cloud whose displacement vector is greater than the threshold is selected from the point cloud image of the previous frame and recorded as the point cloud to be classified;

Acquiring spatial position information of the point cloud to be classified;

Determining the movement direction of the point cloud to be classified based on the displacement vector;

Clustering the to-be-classified point clouds whose motion directions are similar and the distance between adjacent spatial positions is less than a threshold value to obtain at least one set of target point clouds.