WO2021056499A1

WO2021056499A1 - Data processing method and device, and movable platform

Info

Publication number: WO2021056499A1
Application number: PCT/CN2019/108847
Authority: WO
Inventors: 吴显亮; 陈进; 赖镇洲
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-09-29
Filing date: 2019-09-29
Publication date: 2021-04-01
Also published as: CN112154455A

Abstract

A data processing method and device, and a movable platform. The method comprises: acquiring target sensor data and fusion data (S301); performing object-on-road point cloud clustering processing on point cloud data to obtain a point cloud cluster, and determining the state information of the point cloud cluster (S302); determining whether the state information of the point cloud cluster and the state information of a target object in the fusion data satisfy consistency conditions (S303); and if not, determining the probability of false detection of the state information of the target object according to the observable scope of a sensor in the environment where a movable platform is located, the probability being used to indicate whether the movable platform performs an obstacle avoidance operation (S304), so that the checking accuracy of the consistency conditions is higher, and the probability obtained is more in line with the objective reality, so as to more accurately instruct whether the movable platform performs an obstacle avoidance operation or not, thus ensuring the safety of the movable platform during movement.

Description

Data processing method, equipment and movable platform

Technical field

The embodiments of the present application relate to the field of automatic driving technology, and in particular, to a data processing method, equipment, and movable platform.

Background technique

During the driving of the autonomous vehicle, the autonomous vehicle recognizes, tracks, and merges dynamic or static objects in its environment to obtain fusion data. The fusion data includes the status information of the identified objects. And according to the state information of these objects, navigation planning, and control the driving of autonomous vehicles. Among them, the state information of the object may include, for example, information such as object attributes, position, speed, orientation, acceleration, and so on. For example, if the autonomous vehicle estimates that there is a stopped vehicle ahead, the autonomous vehicle can perform a deceleration operation to ensure driving safety. In the process of obtaining the above-mentioned fusion data, there is more or less a certain probability of failure, which leads to inaccurate state information of the object, which in turn affects the driving of the autonomous vehicle.

Summary of the invention

The embodiments of the present application provide a data processing method, equipment, and a movable platform, which are used to determine the accuracy of the state information of objects in the fusion data, so as to guide and control the movement of the movable platform and ensure the safety of the movable platform.

In the first aspect, an embodiment of the present application provides a data processing method, including:

Acquire target sensor data and fusion data, where the fusion data is obtained based on data fusion of multiple sensors, and the sensor is used to collect data on the environment in which the movable platform is located, and the fusion data includes the Status information of the detected target in the environment, where the target sensor data includes point cloud data;

Performing road object point cloud clustering processing on the point cloud data to obtain a point cloud cluster, and determining the state information of the point cloud cluster;

Judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition;

If it does not, determine the probability of false detection of the status information of the target according to the observable range of the sensor in the environment where the movable platform is located, and the probability is used to indicate the Whether the mobile platform performs obstacle avoidance operations.

In a second aspect, an embodiment of the present application provides a data processing device, including: multiple sensors and processors;

The processor is configured to acquire target sensor data and fusion data, where the fusion data is obtained by fusion of the data of the multiple sensors, and the sensor is used for data collection of the environment in which the movable platform is located , The fusion data includes the status information of the detected target in the environment, the target sensor data includes point cloud data; performing road surface object point cloud clustering processing on the point cloud data to obtain point cloud clusters, And determine the state information of the point cloud cluster; determine whether the state information of the point cloud cluster and the state information of the target meet the consistency condition; if not, then according to the location of the sensor on the movable platform The observable range in the environment in which it is located determines the probability of false detection of the status information of the target, and the probability is used to indicate whether the movable platform performs an obstacle avoidance operation.

In a third aspect, an embodiment of the present application provides a movable platform, including: a movable platform body and the data processing device according to the embodiment of the present application in the second aspect, wherein the data processing device is installed on the movable platform. On the platform body.

In a fourth aspect, an embodiment of the present application provides a readable storage medium on which a computer program is stored; when the computer program is executed, it realizes the data described in the embodiment of the present application in the first aspect. Approach.

In a fifth aspect, an embodiment of the present application provides a program product, the program product includes a computer program, the computer program is stored in a readable storage medium, and at least one processor of a removable platform can download from the readable storage medium The computer program is read, and the at least one processor executes the computer program to enable the mobile platform to implement the data processing method described in the embodiment of the present application in the first aspect.

The data processing method, equipment, and movable platform provided by the embodiments of the application acquire target sensor data and fusion data, perform point cloud clustering processing on the point cloud data of the target sensor data to obtain point cloud clusters, and determine all The state information of the point cloud cluster, if the state information of the point cloud cluster and the state information of the target in the fusion data do not meet the consistency condition, then according to the sensor’s availability in the environment where the movable platform is located The observation range determines the probability of false detection of the status information of the target, and the probability is used to indicate whether the movable platform performs an obstacle avoidance operation. Since the state information of the point cloud cluster is obtained from the point cloud data, the accuracy of the consistency check on the state information of the target in the fusion data through the state information of the point cloud cluster is higher. If the consistency check fails , Through the observable range of the sensor in the environment where the movable platform is located, the probability of false detection of the target's status information is obtained, which is more in line with the objective reality, so as to more accurately guide whether the movable platform is executed Obstacle avoidance operation to ensure the safety of the movement process of the movable platform.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Fig. 1 is a schematic architecture diagram of an autonomous driving vehicle according to an embodiment of the present application;

Figure 2 is a schematic diagram of an application scenario provided by an embodiment of the application;

FIG. 3 is a flowchart of a data processing method provided by an embodiment of the application;

4 is a schematic diagram of controlling the decelerating movement of a movable platform provided by an embodiment of the application;

FIG. 5 is a corresponding diagram of the recommended comfortable dynamic object maintaining distance dist _dynamic at different speeds, the motorized static object maintaining distance dist _static , and the buffer distance dist _{margin provided} by an embodiment of the application;

FIG. 6 is a schematic structural diagram of a data processing device provided by an embodiment of this application;

FIG. 7 is a schematic structural diagram of a movable platform provided by an embodiment of this application;

FIG. 8 is a schematic structural diagram of a movable platform provided by another embodiment of the application.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of this application. The terms used in the specification of the application herein are only for the purpose of describing specific embodiments, and are not intended to limit the application. The term "and/or" as used herein includes any and all combinations of one or more related listed items.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

The embodiments of the present application provide a data processing method, equipment, and a movable platform, where the movable platform may be an unmanned aerial vehicle, an unmanned vehicle, an unmanned boat, a robot, or an autonomous vehicle, etc.

The following description of the mobile platform of the present application uses an autonomous vehicle as an example. Fig. 1 is a schematic architecture diagram of an autonomous driving vehicle according to an embodiment of the present application.

The autonomous vehicle 100 may include a sensing system 110, a control system 120, and a mechanical system 130.

Among them, the perception system 110 is used to measure the state information of the autonomous vehicle 100, that is, the perception data of the autonomous vehicle 100. The perception data may represent the position information and/or state information of the autonomous vehicle 100, for example, position, angle, speed, Acceleration and angular velocity, etc. The perception system 110 may include, for example, a visual sensor (for example, including multiple monocular or binocular vision devices), lidar, millimeter wave radar, inertial measurement unit (IMU), global navigation satellite system, gyroscope, ultrasonic sensor At least one of sensors such as, electronic compass, and barometer. For example, the global navigation satellite system may be the Global Positioning System (GPS).

After the sensing system 110 obtains the sensing data, it can transmit the sensing data to the control system 120. Among them, the control system 120 is used to make decisions on how to control the autonomous driving vehicle 100 based on the perception data, for example: how much speed to travel, or how much braking acceleration to brake, or whether to change lanes, or, Turn left/right, etc. The control system 120 may include, for example, a computing platform, such as a vehicle-mounted supercomputing platform, or at least one device having processing functions such as a central processing unit and a distributed processing unit. The control system 120 may also include a communication link for various data transmission on the vehicle.

The control system 120 may output one or more control commands to the mechanical system 130 according to the determined decision. Among them, the mechanical system 130 is used to control the autonomous vehicle 100 in response to one or more control commands from the control system 120 to complete the above-mentioned decision. For example, the mechanical system 130 can drive the wheels of the autonomous vehicle 100 to rotate so as to be automatic. The driving vehicle 100 provides power for driving, wherein the rotation speed of the wheels can affect the speed of the self-driving vehicle. Wherein, the mechanical system 130 may include, for example, at least one of a mechanical body engine/motor, a controlled wire control system, and the like.

It should be understood that the aforementioned naming of the components of the unmanned vehicle is only for identification purposes, and should not be understood as a limitation to the embodiments of the present application.

Figure 2 is a schematic diagram of an application scenario provided by an embodiment of the application. As shown in Figure 2, the autonomous vehicle can drive on the road, and the autonomous vehicle can drive on the road in the current environment (for example, by The aforementioned perception system 110) collects perception data, which can include point cloud data, image data, radar data, etc., and then obtains the fusion data based on the perception data, and how to process the fusion data after obtaining the fusion data You can refer to the descriptions of the following embodiments of this application.

The embodiments of the present application can be applied to dynamic scenes in which the movable platform is moving, and the dynamic or static objects in the environment where the movable platform is located are identified, tracked, and fused to obtain the state estimation of these objects, thereby guiding the relevant Navigation planning and control tasks; however, the processing methods such as the identification, tracking and fusion of these objects all have a certain failure probability, that is, the correct state estimation information cannot be obtained. In this case, the implementation of this application can be used Examples of the scheme to identify these failure modes, so as to actively evade processing, improve the safety performance of the movable platform.

In related technologies, the estimation failure of the object state can be roughly divided into false detection, missed detection, inaccurate state estimation (for example, the position, speed, heading of the vehicle, inaccurate category information, etc.), and related information (for example, when the object is different Whether the above time series are the same object) is not accurate and so on. Among them, for false detections and missed detections, positive and negative types are often defined, corresponding to the presence or absence of detection, respectively, and false positive and false negative types (False Negative) correspond to errors respectively. Inspection and missed inspection.

Generally speaking, the state estimation of an object is usually divided into several steps: first process the original sensor to obtain the basic data of the object state estimation. These processing methods can include image processing, point cloud processing, etc.; then Next, start the object detection, such as training with a deep neural network to obtain more accurate detection results; then, the detected objects are data associated on the time series, and the detection results of the same object at different times are associated together This correlation process is usually used in combination with tracking algorithms to obtain a more stable detection result in time sequence. If there are multiple observations for each object, such as the overlap of different camera angles or the collection of data by multiple different types of sensors, you need Combining these observations to obtain a final object state estimation involves the technology of multiple information fusion.

However, all of the above processing procedures have a certain probability of causing failure, and the cumulative failure of each module will cause the failure rate of the final system to greatly exceed the design requirements. Therefore, each module controls its own module in time. There is no guarantee that the failure rate of the entire system can be greatly reduced. Therefore, for the ultimate safety goal, the system should be able to automatically identify some common failure modes and actively avoid them instead of relying only on the internal components of each module. Failure detection.

In the current system construction of mobile platforms (such as drones and unmanned vehicles), these safety indicators are often assigned to each module, and each module is used to detect and avoid failure rates, such as in object detection. In the module, there are related technical methods that can be used to reduce false detections and missed detections, such as improving sensor accuracy, setting more detailed sampling rules, and so on.

In the multi-information fusion module, many systems will use this system as the final output of the entire system's perceptual information. However, relevant technologies have not been able to propose an effective solution for verifying fusion data.

FIG. 3 is a flowchart of a data processing method provided by an embodiment of this application. As shown in FIG. 3, the method of this embodiment may include:

S301. Obtain target sensor data and fusion data.

In this embodiment, the fusion data is acquired, where the fusion data is obtained based on data fusion of multiple sensors of the movable platform, and the sensor is used for data collection of the environment in which the movable platform is located. If the sensor is For the image sensor, the image sensor collects image data of the environment where the movable platform is located; if the sensor is a laser sensor, the image sensor collects point cloud data of the environment where the movable platform is located. The above-mentioned fusion data includes the status information of the detected target in the environment. Taking the mobile platform as an autonomous vehicle as an example, when the self-driving vehicle is driving on the road, the above-mentioned fusion data may include the environment Status information of other vehicles that have been detected in. Among them, how to obtain the fusion data according to the data fusion of multiple sensors can refer to the description in the related technology, which will not be repeated here.

In addition, this embodiment also acquires target sensor data. The target sensor may be, for example, a sensor among the above-mentioned multiple sensors. The target sensor data includes point cloud data, and the target sensor may be, for example, a laser sensor.

Optionally, the state information of the aforementioned target may include any one or more of the following parameter information: object attributes, position, orientation, speed, acceleration. Wherein, the speed may include at least one of the following: linear velocity and angular velocity. The object attribute can be, for example, a vehicle, or a person, and so on.

S302: Perform road surface object point cloud clustering processing on the point cloud data to obtain a point cloud cluster, and determine state information of the point cloud cluster.

In this embodiment, the point cloud data in the above-mentioned target sensor data is subjected to road surface object point cloud clustering processing to obtain point cloud clusters, and the obtained state information of each point cloud cluster is determined.

Optionally, the state information of the point cloud cluster may include any one or more of the following parameter information: object attributes, position, orientation, velocity, acceleration.

S303: Determine whether the state information of the point cloud cluster and the state information of the target in the fusion data meet the consistency condition.

S304. If not, determine the probability of false detection of the status information of the target according to the observable range of the sensor in the environment where the movable platform is located, and the probability is used to indicate the State whether the movable platform performs obstacle avoidance operations.

In this embodiment, after obtaining the state information of the point cloud cluster, it is determined whether the obtained state information of the point cloud cluster and the state information of the target in the fusion data meet the consistency condition. If the state information of the point cloud cluster and the state information of the target in the fusion data meet the consistency condition, it means that the detection of the state information of the target in the fusion data is correct. If the state information of the point cloud cluster and the state information of the target in the fusion data do not meet the consistency condition, it means that the state information of the target in the fusion data may be detected incorrectly. Then, according to the observable range of the sensor in the environment where the movable platform is located, the probability of false detection of the status information of the target in the fusion data is determined, and the probability is used to indicate whether the movable platform performs obstacle avoidance operations.

In this embodiment, the target sensor data and the fusion data are acquired, the point cloud data of the target sensor data is clustered by the road object point cloud to obtain the point cloud cluster, and the state information of the point cloud cluster is determined. If the state information of the point cloud cluster and the state information of the target in the fusion data do not meet the consistency condition, then according to the observable range of the sensor in the environment where the movable platform is located, it is determined that the occurrence of the target is The probability of false detection of status information, the probability being used to indicate whether the movable platform performs an obstacle avoidance operation. Since the state information of the point cloud cluster is obtained from the point cloud data, the accuracy of the consistency check of the state information of the target in the fusion data through the state information of the point cloud cluster is higher. If the consistency check fails , Through the observable range of the sensor in the environment where the movable platform is located, the probability of false detection of the target's status information is obtained, which is more in line with the objective reality, so as to more accurately guide whether the movable platform is executed Obstacle avoidance operation to ensure the safety of the movement process of the movable platform.

In some embodiments, the environment is further classified into multiple environmental categories according to the observable range of the sensor in the environment; for example, the environment may be classified into multiple environmental categories according to the observable range of at least one of the multiple sensors in the environment. For another example, the environment may be divided into multiple environmental categories according to the observable range of the target sensor (such as a laser sensor) in the environment. The multiple environmental categories are, for example, urban roads with buildings around, highways with mountainous areas, highways with flat terrain, highways with tunnels, etc. The present embodiment is not limited to this.

Correspondingly, a possible implementation manner of the foregoing S304 may include S3041-S3043:

S3041. Obtain environmental probability information of the environment in which the mobile platform is located and belong to various environmental categories.

S3042. Acquire a priori probability information of false detection of the sensor in the environment category.

S3043. Determine the probability of erroneous detection of the state information of the target according to the environmental probability information and the prior probability information.

In this embodiment, the environment probability information of the environment where the mobile platform is currently located belongs to each of the multiple environment categories obtained by the above division. In addition, the priori probability information of the sensor's error detection in each of the above-mentioned environmental categories is also obtained. Then, according to the environmental probability information of the environment in which the mobile platform is located in each of the above environmental categories and the prior probability information of the sensor (for example, the target sensor) in each of the above environmental categories, the error detection of the target in the fusion data is determined. The probability of false detection of the state information of the object.

For example: According to the observable range of the sensor in the environment, the environment is divided into N environmental categories, which are the first environmental category, the second environmental category, ..., the Nth environmental category. Obtain the environmental probability information of the environment where the mobile platform is located in the first environment category is probability P(A1), and the environmental probability information of the environment where the mobile platform is located in the second environment category is the probability P(A2),..., The environmental probability information of the environment in which the mobile platform is located belongs to the Nth environmental category is the probability P(AN); the prior probability information of the sensor's false detection in the environment of the first environmental category is the probability P(B1), and the sensor is in the first environment. 2 The prior probability information of false detection in the environment of the environment category is the probability P(B2),..., the prior probability information of the sensor's false detection in the environment of the Nth environmental category is the probability P(BN); then, determine The probability of erroneous detection of the status information of the target is: P(A1)*P(B1)+P(A2)*P(B2)+...+P(AN)*P(BN). Therefore, based on the obtained probability of erroneous detection of the status information of the target in the fusion data, the possibility of erroneous detection of the status information of the target can be more accurately evaluated.

Optionally, a possible implementation manner of the foregoing S3041 is: according to the point cloud distribution density in the point cloud data, it is determined that the environment in which the movable platform is located belongs to the environmental probability information of each environment category. For example: if the point cloud distribution density is dense, it means that the environment where the movable platform is located belongs to an urban road with buildings; if the point cloud distribution density is sparse, it means that the movable platform is flat The environment is more likely to be a highway with flat ground.

Since the point cloud data is data with higher reliability, the accuracy of the environmental probability information of the environment in which the mobile platform is obtained from the mobile platform belongs to each environmental category is higher.

In some embodiments, after performing the above S304, this embodiment also determines whether the probability determined in S304 is greater than the preset probability. If the probability is greater than the preset probability, it means that an erroneous detection of the status information of the target has occurred. If the probability is greater, it indicates that the movable platform needs to perform obstacle avoidance operations. If the probability is less than or equal to the preset probability, it indicates that the possibility of false detection of the status information of the target is less likely to occur, then the movable platform is indicated There is no need to perform obstacle avoidance operations. Then find the point cloud cluster corresponding to the target in the point cloud data, the state information of the point cloud cluster can more truly reflect the actual state information of the target, and then obtain the movement of the point cloud cluster corresponding to the target Parameters (such as speed, acceleration, movement direction, etc.), according to the movement parameters of the point cloud cluster of the target to control the movable platform to perform obstacle avoidance operations, and the movement parameters of the point cloud cluster of the target can also be used as the target The movement parameters of the object. Optionally, controlling the movable platform to perform obstacle avoidance operations may be, for example, controlling the movable platform to decelerate movement, or controlling the movable platform to move (for example, change the orientation), or controlling the movable platform to decelerate and steer. , In order to make the movable platform avoid the target through these operations and ensure the safety of the movable platform.

In some embodiments, a possible implementation manner of controlling the decelerated movement of the movable platform may be: calculating when the movable platform moves from the current position to the first position where the point cloud cluster is currently located, The first distance of movement of the movable platform; according to the movement parameters of the point cloud clusters and the movement parameters of the movable platform, predicting the intersection of the movement trajectory of the movable platform and the movement trajectory of the point cloud cluster Two positions; calculating the second distance of the movable platform when the movable platform moves to the second position; if the distance difference of the second distance minus the first distance is a positive number, The movable platform is controlled to perform a decelerating movement on the movement trajectory of the distance difference.

As shown in Figure 4, the current position of the movable platform is O, and the current position of the point cloud cluster is called the first position (that is, C). Assuming that the target corresponding to the point cloud cluster is stationary, the calculation is movable The distance from the movement of the platform to the position of the point cloud cluster (ie, the first position C) is the first distance d1. Then according to the motion parameters of the point cloud clusters and the motion parameters of the movable platform, Figure 4 shows the point cloud cluster and the movable platform moving linearly in the same direction as an example to predict the motion trajectory of the movable platform and the point cloud cluster The intersecting position is called the second position (that is, D), that is, it is predicted that the point cloud cluster and the movable platform will continue to move according to the corresponding motion parameters, and it is estimated that the target corresponding to the point cloud cluster will collide with the movable platform. The position is the second position D. Then calculate the second distance d2 that the movable platform moves when the movable platform moves from the current position O to the second position D. Then subtract the first distance d1 from the second distance d2 to obtain the distance difference △d=d2-d1. If △d is greater than 0, control the movable platform to perform deceleration on the trajectory of △d from the current position. Exercise to determine the safety of the movable platform.

In some embodiments, a possible implementation manner of controlling the movable platform to perform decelerating motion on the motion trajectory of the distance difference may be: calculating the decelerating motion of the movable platform from the current position to The first position, and the first acceleration in the process when the velocity of the first position is zero; controlling the movable platform to perform decelerating motion at the second acceleration on the motion trajectory of the distance difference, the first The absolute value of the second acceleration is smaller than the absolute value of the first acceleration.

Referring to Figure 4, if Δd is greater than 0, calculate the acceleration during the decelerating movement of the movable platform from the current position O to the first position C, and when the movable platform moves to the first position C when the speed is reduced to 0 Is the first acceleration, and then control the movable platform to perform deceleration movement at the second acceleration whose absolute value is less than the first acceleration on the trajectory of Δd, so that the movable platform decelerates from the current position O at the second acceleration to the second acceleration At a position C, the speed of the movable platform is greater than 0, that is, the movement of the movable platform from the current position O to the first position C is a slower deceleration movement, which not only ensures the movement safety of the movable platform, but also The discomfort caused by the rapid deceleration to the user is avoided, and the user experience is improved.

Optionally, after controlling the movable platform to decelerate to the first position C at the second acceleration, the movable platform may also be controlled to move at the third acceleration/deceleration rate, wherein the absolute value of the third acceleration is greater than the absolute value of the second acceleration Value, the third acceleration may be equal to the first acceleration, for example.

If the target is also continuing to move, it is possible that the movable platform decelerates and moves the trajectory of Δd at the second acceleration. At this time, the new Δd is still greater than 0, and the movable platform can continue to perform slower deceleration movement.

In some cases, for example, the case of missed detection is not limited to this case. Generally speaking, it can be detected by point cloud data with a high probability. As a form of point cloud, it lacks necessary status information and cannot express dynamics. Situation, if an object with speed estimation is the result of its missed detection, it is likely to degenerate into a point cloud without speed. Therefore, follow the car or predict the lane change on a movable platform (such as an autonomous vehicle) At the time, it is necessary to consider that the vehicle in front may suddenly degenerate into the form of a point cloud, and use the following safe distance (dist) calculation formula:

Among them, v _r and v _f are the instant speeds of the next car (that is, the mobile platform) and the preceding car, _{respectively, a r} and a _f are the instant accelerations of the following car and the preceding car, and a _{brake means that} it can be received. The braking acceleration of the following car, t _resp is the reaction time of the following car. Among them, in the task of car following, you can assume a _brake = 0.1g～0.2g to plan the comfortable dynamic object keeping the distance dist _dynamic (for example, the second distance in the above embodiment), but considering the degeneration into a point cloud, this When v _f = 0, so, then use a _brake = 0.5g to plan a static obstacle. The motorized static object keeps the distance dist _static (for example, the first distance in the above-mentioned embodiment). It is usually necessary to ensure that dist _dynamic > dist _static , so , Get a buffer distance dist _margin = dist _dynamic- _{dist static} (for example, the distance difference in the above embodiment), so that even if the vehicle in front suddenly degenerates into a point cloud, _{the interval of dist margin} can be used to use the comfort of 0.1g～0.2g The braking acceleration is decelerated. If the buffer distance is exceeded, the motor braking acceleration a _brake = 0.2g～0.5g is adopted. If it is determined that the object suddenly appears within the emergency braking distance, and the object is large, it may not be a misdetection. Emergency braking a _brake ＞0.5g. For example, as shown in Figure 5, the recommended comfortable dynamic object maintains the distance dist _{dynamic at} different speeds, the motorized static object maintains the distance dist _static , and the corresponding relationship diagram of the buffer distance dist _margin. If the dynamic object degenerated into a point cloud is driving forward, the distance from the next frame to the actual object will not be shortened, that is, it will not exceed the buffer distance dist _margin , and there will be a high probability of less maneuvering braking. Don't enter emergency braking, just make comfortable braking, so as to avoid the hazards of dynamic objects degenerating into point clouds while ensuring user experience. That is, when the vehicle behind maintains the distance dist _dynamic from the vehicle in front according to the comfortable dynamic object, the speed of the vehicle in front is known. If the vehicle in front degenerates to a point cloud with no speed or underestimated speed at a certain moment, it will be It is predicted that motorized static objects that need to maneuver to avoid static objects maintain a distance of dist _static . At this time, dist _dynamic is often greater than dist _static . The difference between these two distances is the buffer distance dist _margin , that is, the vehicle can first perform comfortable acceleration _{at the buffer distance dist margin} Braking, after the buffer distance dist _{margin is} exceeded, the dynamic object has already moved forward at the next moment during motor braking or even emergency braking, so the buffer distance dist _margin will be updated and longer at the next moment, resulting in a high probability that the following car will not Exceed the buffer distance, thereby improving comfort while ensuring safety.

In some embodiments, the consistency conditions described in the foregoing embodiments may include at least one of the following items 1)-3):

1) There are point cloud clusters corresponding to the target object in the point cloud data. That is, it is judged whether there is a point cloud cluster corresponding to the target object (that is, whether there is a misdetection) in the point cloud cluster by performing road object point cloud clustering processing on the point cloud data, and if it exists, it is determined The state information of the point cloud cluster meets the condition of consistency with the state information of the target object (that is, there is no misdetection). If it does not exist, it is determined that the state information of the point cloud cluster is different from the state information of the target object. Meet the consistency condition (that is, there is a false detection).

For example, for false detections, it is necessary to combine the environment to distinguish whether the false detection is a "false detection generated out of thin air"; for example, on a flat highway, without any obstruction, if the false detection suddenly appears, it is likely to be true It is a false detection. If it is not a false detection, it means that it is not visible for a long time for a reason. If there is an intersection or other obstructions nearby, the sudden appearance can be attributed to the sudden appearance of the intersection or other visual blind spots If there is no visual blind zone, it is considered to be caused by the previous missed detection. If the possibility of missed detection is very small, it can be considered that the high probability of the false detection is really a false detection. You can do an event is defined as follows: E ^FP is false detection, E ^TP is really detect, E ^Open observed previously defined for this position may now belong to visually observable range, E ^N is defined as the period of time before detection was not detected . then

Which can be assessed P (E ^Open) approximate range, P depending on the environment ^{^{(E FP | E N, E}} Open) represents the environment without blind spots, and under conditions not previously detected, probability of detection in error detection, and should set a Larger value

Represents a scene under no dead zone of the detected detection probability of false detection, a compromise should be a value of about 0.5, ^{^{Similarly, P (E TP | E N}} , E Open) = 1-P (E FP |E ^N ,E ^Open ) should be smaller,

Compromise, and then evaluated separately P (E ^FP | E ^N) according to the above formula and P (E ^TP | E ^N) in the approximate range, the object is to determine whether the probability of false detection. In this way, if it is determined P (E ^Open) is large, it can be ^{^{P (E FP | E N)}} >> P (E TP | E N), is determined to be a high probability false detection.

2) There is state information of the target corresponding to any one of the point cloud clusters in the fusion data. That is, it is determined whether there is any state information of the target object of the point cloud cluster in the fusion data, and if the state information of the target object corresponding to at least one point cloud cluster does not exist in the fusion data, the state of the point cloud cluster is determined The information does not meet the condition of consistency with the state information of the target object (that is, there is a missed detection). If the state information of the target object corresponding to any point cloud cluster exists in the fusion data, the state information of the point cloud cluster is determined The state information of the target object meets the consistency condition (that is, there is no missed detection).

3) The parameter information of the target corresponding to the point cloud cluster is consistent with the parameter information of the point cloud cluster. For example: to determine whether at least one parameter information of the position, orientation, velocity, and acceleration of each point cloud cluster is consistent with at least one parameter information of the position, orientation, velocity, and acceleration of the target corresponding to the point cloud cluster in the fusion data , If all the parameter information determined are consistent, it is determined that the state information of the point cloud cluster and the state information of the target object meet the consistency condition, and if at least one parameter information in all the determined parameter information is inconsistent, it is determined The state information of the point cloud cluster and the state information of the target object do not meet the consistency condition.

In some embodiments, if the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition, the parameter information of the point cloud cluster corresponding to the target object is used as the parameter of the target object information.

In some embodiments, the target sensor data further includes image data, and the target sensor further includes an image sensor. Correspondingly, a possible implementation of determining whether there is a point cloud cluster corresponding to the target object in the point cloud data may be : Determine whether there is a point cloud cluster corresponding to the target object in the point cloud data according to the intensity of the pixels in the image data. If it does not exist, it is determined that the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition; if it exists, the state information of the point cloud cluster and the state information of the target object are determined The information meets the consistency conditions. The intensity of the pixels in the image data is used to assist in judging whether there are point cloud clusters corresponding to the target in the point cloud data, which can improve the accuracy of the judgment result, especially when the point cloud distribution density is relatively sparse in the point cloud data , Can guarantee the accuracy of the judgment result. For example, for an object of a specific color, such as black, the distribution of the point cloud corresponding to the black object may be sparse, and the intensity of the pixel corresponding to the black object in the image data is larger, the image data can also be used to assist in determining the point There are point cloud clusters corresponding to the black object in the cloud data.

In some embodiments, the point cloud cluster is based on a laser point cloud point clustering that does not conform to a plane or does not conform to a target curved surface, and the target curved surface is a curved surface with a curvature lower than a preset curvature. This ensures that the obtained point cloud clusters correspond to the target objects above the road surface. Since the target objects above the road surface may cause safety hazards to the movable platform, the focus is on these point cloud clusters corresponding to the target objects above the road surface. The point cloud data of these point cloud clusters are useful point cloud data, and the other point cloud data does not need to be used to determine whether the consistency conditions are met, which improves the processing efficiency.

In some embodiments, the fusion data includes the position of the target object, and by evaluating the position of the target object, it is determined whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition, and accordingly, the point cloud cluster is determined A possible realization of whether the state information of and the state information of the target meets the consistency condition may be: judging whether the position of the target in the fusion data is consistent with the position of the point cloud cluster corresponding to the target, where, The position of the point cloud cluster is determined by the point cloud data. If it is consistent, it means that the position of the target in the fusion data is accurate. It is determined that the state information of the point cloud cluster and the state information of the target meet the consistency condition. If it is inconsistent, It means that the position of the target in the fusion data is not accurate, and it is determined that the state information of the point cloud cluster and the state information of the target do not meet the consistency condition. In this embodiment, the current position of the point cloud cluster is determined by the point cloud data, which can truly reflect the current actual position of the target of the point cloud cluster, so the accuracy of judging whether the consistency condition is met is improved.

In some embodiments, the fusion data includes the speed of the target, and the speed of the target is evaluated to determine whether the state information of the point cloud cluster and the state information of the target meet the consistency condition.

Optionally, a possible implementation method for judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition may be: determining the target according to the historical speed parameter corresponding to the target object in the fusion data The current predicted position of the object, and then determine whether the current position of the point cloud cluster corresponding to the target object is consistent with the predicted position. The current position of the point cloud cluster can be determined based on the current point cloud data. If it is consistent, the target The speed of the object is accurate. It is determined that the state information of the point cloud cluster and the state information of the target object meet the consistency condition. If it is inconsistent, it means that the speed of the target object in the fusion data may be inaccurate. The state information of the target object does not meet the consistency condition. In this embodiment, the current position of the point cloud cluster is determined by the point cloud data, which can truly reflect the current actual position of the target of the point cloud cluster, so the accuracy of judging whether the consistency condition is met is improved.

Optionally, another possible implementation manner for judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition may be: acquiring the point cloud cluster corresponding to the target object in the first frame The position of the point cloud cluster in the second frame, where the time of the second frame is later than the time of the first frame, and the position of the point cloud cluster in the first frame is determined based on the point cloud data of the first frame Yes, the position of the point cloud cluster in the second frame is determined according to the point cloud data of the second frame, and then according to the position of the point cloud cluster in the first frame and the position of the point cloud cluster in the second frame, The predicted speed of the point cloud cluster is determined, and the predicted speed refers to the predicted speed of the point cloud cluster from the position of the first frame to the position of the second frame within the time period from the first frame to the second frame. Then judge whether the above predicted speed is consistent with the speed of the target in the fusion data. If they are consistent, the speed of the target is accurate. It is determined that the state information of the point cloud cluster and the state information of the target meet the consistency condition. It means that the speed of the target in the fusion data may be inaccurate, and it is determined that the state information of the point cloud cluster and the state information of the target do not meet the consistency condition. In this embodiment, the predicted speed is determined according to the position of the point cloud cluster in different first and second frames, which can truly reflect the actual speed of the target object of the point cloud cluster, so it improves the judgment of whether the point cloud cluster is consistent. The accuracy of sexual conditions.

Optionally, the target sensor data also includes radar data, and the target sensor also includes radar. Another possible implementation manner for judging whether the state information of the point cloud cluster and the state information of the target meet the consistency condition may be : Determine the predicted speed of the point cloud cluster corresponding to the target based on the radar data. The predicted speed can be obtained by performing a differential processing on the radar data, for example, and then judge whether the predicted speed is consistent with the speed of the target in the fusion data. If they are consistent, it means that the speed of the target object is accurate, and it is determined that the state information of the point cloud cluster and the state information of the target object meet the consistency condition. The status information of the cluster and the status information of the target do not meet the consistency condition. In this embodiment, the predicted speed is determined based on radar data, which can more accurately reflect the actual speed of the target of the point cloud cluster, so the accuracy of judging whether the consistency condition is met is further improved. The radar data is, for example, millimeter wave radar data. It should be noted that the sensor data used to obtain the speed is not limited to radar data, and may also be other sensor data.

In some embodiments, the fusion data includes the acceleration of the target, and the acceleration of the target is evaluated to determine whether the state information of the point cloud cluster and the state information of the target meet the consistency condition.

Optionally, another possible implementation manner for judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition may be: determining the point cloud corresponding to the target object according to the point cloud data The predicted acceleration of the cluster. The predicted acceleration can be obtained by performing a differential processing on the point cloud data, and then determine whether the predicted acceleration is consistent with the acceleration of the target in the fusion data. If they are consistent, the acceleration of the target is accurate. The state information of the point cloud cluster and the state information of the target object meet the consistency condition. If they are inconsistent, the acceleration of the target object in the fusion data may be inaccurate. Determine the state information of the point cloud cluster and the state information of the target object Does not meet the consistency conditions. In this embodiment, the predicted acceleration is determined based on the point cloud data, which can more accurately reflect the actual speed of the target object of the point cloud cluster, so the accuracy of judging whether the consistency condition is met is improved.

Optionally, the target sensor data also includes radar data, and the target sensor also includes radar. Another possible implementation manner for judging whether the state information of the point cloud cluster and the state information of the target meet the consistency condition may be : Determine the predicted acceleration of the point cloud cluster corresponding to the target object based on the radar data. The predicted acceleration can be obtained by performing secondary differential processing on the radar data, for example, and then determine whether the predicted acceleration is consistent with the velocity of the target in the fusion data If they are consistent, it means that the acceleration of the target object is accurate. It is determined that the state information of the point cloud cluster and the state information of the target object meet the consistency conditions. The state information of the cloud cluster does not meet the consistency condition with the state information of the target object.

In some embodiments, the fusion data includes the orientation of the target, and the orientation of the target is evaluated to determine whether the status information of the point cloud cluster and the status information of the target meet the consistency condition.

Optionally, a possible implementation manner for judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition may be: judging the orientation of the target object in the fusion data and the corresponding target object Whether the orientation of the point cloud clusters is consistent, where the orientation of the point cloud cluster is determined by the distribution of the point cloud in the point cloud cluster. If they are consistent, it means that the orientation of the target in the fusion data is accurate, and the state of the point cloud cluster is determined The information meets the condition of consistency with the status information of the target. If it is inconsistent, it means that the orientation of the target in the fusion data may be inaccurate. It is determined that the status information of the point cloud cluster does not meet the condition of consistency with the status information of the target. . Among them, the orientation of the point cloud cluster is determined by the point cloud data, which can truly reflect the actual orientation of the target corresponding to the point cloud cluster, so the accuracy of judging whether the consistency condition is met is improved.

Optionally, a possible implementation manner for judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition may be: obtaining the speed of the point cloud cluster corresponding to the target object, and the point cloud The speed of the cluster can be determined according to the point cloud data or radar data, the direction of the point cloud cluster is determined according to the speed direction of the point cloud cluster, and then it is judged whether the orientation of the target in the fusion data is consistent with the orientation of the point cloud cluster corresponding to the target. If they are consistent, it means that the orientation of the target in the fusion data is accurate, and it is determined that the status information of the point cloud cluster and the status information of the target meet the consistency condition. If they are inconsistent, it means that the orientation of the target in the fusion data may be inaccurate. It is determined that the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition. The speed direction of the point cloud cluster can also truly reflect the actual orientation of the target corresponding to the point cloud cluster, so the accuracy of judging whether the consistency condition is met is improved.

In some embodiments, if the object attribute in the status information of the target in the fusion data does not match the parameter information, then it can be determined that an erroneous detection of the status information of the target has occurred, for example: the object attribute of the target is pedestrian and The moving speed of the target is 120km/h, the object attribute of the target is a vehicle, and the height of the target is 5m. If the target object in the fusion data is back-projected to the image data with scene segmentation, and it is inconsistent with the corresponding pixel label, it can be determined that an error detection of the status information of the target object has occurred. If the frame used to identify the vehicle is inside other static objects, it can be determined that an erroneous detection of the status information of the target has occurred.

In some embodiments, it is also possible to judge whether the timing correlation meets the consistency condition: if it is for a single object, it is basically equivalent to the judgment of speed consistency, but for multiple objects, it is necessary to consider the correlation of different objects. Correlation, that is, the object A and object B at a certain moment are all related to the object A at the next moment. In this case, a single match may be within the correlation threshold and no abnormality can be seen, but consider the global correlation After that, that is, at the next moment, the object B has no correlation. At this time, it should be considered that the above correlation is unreliable, which means that the timing correlation does not meet the consistency condition.

In addition, for the state information of the point cloud clusters and the state information of the target object do not meet the consistency condition, if the position of the target object is detected incorrectly, what can actually be done is to replace the target object with the point cloud (for example, change the target object's The state information is used as the state information of the target object), and the speed of the target object is used as a priori of these point clouds for prediction. If the speed of the target object is detected incorrectly, the speed of the target object can be treated as 0, so , You can use the degraded point cloud processing method to deal with, that is, define the buffer distance of the brake, and follow the car with a conservative strategy to ensure the user experience while ensuring safety.

If the direction of the target is wrong, in this case, if the speed information of the target is available, the above-mentioned target speed detection error will be handled in the same way, and it will be treated as if the speed is zero. All detection errors of position, speed, and orientation will define the data interval in which these parameters may be located, and then consider whether all the states in this possible interval will cause potential collision hazards or planning difficulties for the movable platform , If there are no potential dangers and difficulties, the fault can be determined not to be dealt with. Then, the weight of these parameters participating in the motion control of the movable control platform will be adjusted.

Of course, the introduction of scenes is also very necessary. Obstacles such as lane lines and static guardrails can be used to evaluate whether other vehicles can affect the illumination of movable platforms (such as autonomous vehicles). For example, vehicles on the opposite side of the guardrail can be ignored. As well as vehicles that are separated by 3 lanes, the impact can be considered small.

It is worth noting that the above-mentioned embodiments are only examples, and without departing from the inventive concept of the present disclosure, the following improvements can also be made to the solution:

In some implementations, in the perception system, the fault detection module at the system level (for example, the module used to determine whether the above-mentioned solutions in this application meets the consistency condition (or the above-mentioned probability) is not necessarily It runs as an independent module. It can also be inside a certain functional module of the mobile platform but performs system-level fault diagnosis and detection, such as in the fusion module (for example, the module used to obtain the above-mentioned fusion data) At the same time, access the original data stream or other types of perception information to determine the consistency of the system level. The fault detection at the system level (such as the global scope) or the module level (such as a single module) mainly depends on whether the input has been The function realization of this module is not very relevant, but the system level considers how to judge whether it meets the consistency conditions.

In some embodiments, although it is proposed to use the data obtained by the original laser or millimeter wave radar as the reference information for determining whether the consistency condition is met, it does not mean that the image information cannot be used as the reference information. For example, the system has functional degradation (laser millimeter wave radar). Radar failure), the original image information can be used as the most reliable information of all information. At this time, the state information of the object of the perception algorithm can be back-projected to the image, and the consistency of the image can be judged to diagnose the fault type of the detection error in the system ; It can also be judged together with the data obtained by laser, millimeter wave and other sensors, and the principle of large numbers is used to determine the source of the fault; therefore, in principle, the information obtained by TOF, ultrasound, etc. can also be used as potential raw sensing data to determine whether the consistency is consistent Sexual condition determination;

In addition to processing the original sensor data, other preprocessing and even the final perception results can be cross-validated to determine whether the inconsistency conditions are met, but each input is not the original input, so it will be affected by the algorithm. Low reliability, but in the implementation process, if the relevant input information is sufficient, the principle of large numbers can be used to locate the fault source and determine the fault type.

Actually, when dealing with these faults, it is often based on some common-sense prior principles. For example, the vehicle cannot walk sideways, the vehicle cannot be visible in the obstructed place, the vehicle cannot appear or disappear out of thin air, etc.; these The principle of common sense can be considered as a combination of scenarios and specific requirements to further characterize the fault and evaluate its potential impact.

The system-level fault detection module uses high-reliability original sensor (such as laser) data and sensing algorithm processing results (such as the above-mentioned fusion data) to compare with each other, so as to detect the conflict between the sensing algorithm results and the original sensor data. If the problem of the original sensor is ruled out at a higher confidence level, the inconsistency can be used to evaluate the type of failure that occurs in the result of the perception algorithm, such as false detection, missed detection, parameter information detection error, correlation matching error, etc. At the same time, the potential impact of these types of failures is evaluated according to specific system requirements, so as to serve as a reference for subsequent decision-making. Through fault diagnosis, although the failure rate cannot be reduced by itself, it can effectively convert the failure rate of the original algorithm framework from an unknown state to a most known state. This provides a basis for the subsequent post-processing algorithm to detect the type of failure. Based on the use of post-processing to further reduce the failure rate.

FIG. 6 is a schematic structural diagram of a data processing device provided by an embodiment of this application. As shown in FIG. 6, the data processing device 600 of this embodiment may include: multiple sensors 601 and a processor 602.

The processor 602 is configured to acquire target sensor data and fusion data in a plurality of sensors 601, wherein the fusion data is obtained by fusion of the data of the plurality of sensors 601, and the sensor is used to Data collection is performed on the environment in which the platform is located, the fusion data includes the status information of the detected target in the environment, the target sensor data includes point cloud data; the point cloud data is performed on the road surface object point cloud The clustering process obtains the point cloud cluster, and determines the state information of the point cloud cluster; judges whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition; The observable range of the sensor 601 in the environment where the movable platform is located, determines the probability of false detection of the status information of the target, and the probability is used to indicate whether the movable platform performs obstacle avoidance operations .

Wherein, the target sensor includes a laser sensor, and correspondingly, the plurality of sensors 601 includes a laser sensor.

In some embodiments, the processor 602 is further configured to classify the environment into multiple environmental categories according to the observable range of the sensor 601 in the environment;

When the processor 602 determines the probability of an erroneous detection of the status information of the target object according to the observable range of the sensor 601 in the environment, it is specifically configured to:

Acquiring environmental probability information of the environment in which the mobile platform is located belongs to each environmental category;

Acquiring a priori probability information of false detection of the sensor 601 in the environment category;

According to the environmental probability information and the prior probability information, the probability of the occurrence of an erroneous detection of the state information of the target object is determined.

In some embodiments, the processor 602 is specifically configured to:

According to the point cloud distribution density in the point cloud data, it is determined that the environment in which the movable platform is located belongs to the environment probability information of each environment category.

In some embodiments, the processor 602 is further configured to:

If the probability is greater than the preset probability, searching for a point cloud cluster corresponding to the target object in the point cloud data;

Acquiring the motion parameter of the point cloud cluster corresponding to the target;

According to the motion parameters, the movable platform is controlled to perform obstacle avoidance operations.

In some embodiments, the processor 602 is specifically configured to:

Control the decelerating movement and/or steering movement of the movable platform.

In some embodiments, the processor 602 is specifically configured to:

Calculating the first distance that the movable platform moves when the movable platform moves from the current position to the first position where the point cloud cluster is currently located;

According to the motion parameters of the point cloud clusters and the motion parameters of the movable platform, predict a second position where the motion trajectory of the movable platform and the motion trajectory of the point cloud cluster intersect;

Calculating the second distance that the movable platform moves when the movable platform moves to the second position;

If the distance difference of the second distance minus the first distance is a positive number, the movable platform is controlled to perform a decelerating movement on the movement track of the distance difference.

In some embodiments, the processor 602 is specifically configured to:

Calculating the first acceleration when the movable platform decelerates from the current position to the first position, and when the speed of the first position is zero;

The movable platform is controlled to perform a deceleration movement with a second acceleration on the movement track of the distance difference, and the absolute value of the second acceleration is smaller than the absolute value of the first acceleration.

In some embodiments, the state information includes any parameter information of an object attribute, position, orientation, speed, acceleration, and the consistency condition includes at least one of the following:

There are point cloud clusters corresponding to the target object in the point cloud data;

There is state information of the target object corresponding to any one of the point cloud clusters in the fusion data;

The parameter information of the target corresponding to the point cloud cluster is consistent with the parameter information of the point cloud cluster.

In some embodiments, the processor 602 is specifically configured to:

Determine the current predicted position of the target object according to the historical speed parameter corresponding to the target object;

Judging whether the current position of the point cloud cluster corresponding to the target object is consistent with the predicted position;

If they are inconsistent, it is determined that the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.

In some embodiments, the target sensor data further includes image data; the target sensor further includes an image sensor, and the plurality of sensors 601 further include an image sensor.

The processor 602 is specifically configured to:

Determining whether there is a point cloud cluster corresponding to the target object in the point cloud data according to the intensity of the pixels in the image data;

If it does not exist, it is determined that the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.

In some embodiments, the point cloud cluster is based on a laser point cloud point clustering that does not conform to a plane or does not conform to a target curved surface, and the target curved surface is a curved surface with a curvature lower than a preset curvature.

In some embodiments, the processor 602 is specifically configured to:

Determine the predicted speed of the point cloud cluster according to the position of the point cloud cluster corresponding to the target in the first frame and the position of the point cloud cluster in the second frame;

If the predicted speed is inconsistent with the speed in the state information of the target, the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.

In some embodiments, the target sensor data further includes radar data, the target sensor further includes a radar, and the plurality of sensors 601 further include a radar, and the radar is, for example, a millimeter wave radar.

The processor 602 is specifically configured to:

Determine the predicted speed of the point cloud cluster according to the radar data;

The processor 602 is specifically configured to:

Determine the predicted acceleration of the point cloud cluster according to the radar data or point cloud data;

If the predicted acceleration is inconsistent with the acceleration in the state information of the target object, the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition.

In some embodiments, the processor 602 is further configured to:

If the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition, the parameter information of the point cloud cluster corresponding to the target object is used as the parameter information of the target object.

Optionally, the data processing device 600 of this embodiment may further include: a memory (not shown in the figure) for storing program codes. The memory is used for storing program codes. When the program codes are executed, the data processing device 600 can implement the above-mentioned technical solutions.

The data processing device of this embodiment can be used to implement the technical solutions of FIG. 3 and the corresponding method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.

FIG. 7 is a schematic structural diagram of a movable platform provided by an embodiment of this application. As shown in FIG. 7, the movable platform 700 of this embodiment may include: a plurality of sensors 701 and a processor 702.

The processor 702 is configured to acquire target sensor data and fusion data from a plurality of sensors 701, where the fusion data is obtained by fusion of the data of the plurality of sensors 701, and the sensor is used to Data collection is performed on the environment in which the platform 700 is located, the fusion data includes the status information of the detected target in the environment, the target sensor data includes point cloud data, and the road surface object points are performed on the point cloud data. The cloud clustering process obtains the point cloud cluster, and determines the state information of the point cloud cluster; judges whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition; The observable range of the sensor 701 in the environment where the movable platform 700 is located determines the probability of false detection of the status information of the target, and the probability is used to indicate whether the movable platform 700 performs Obstacle avoidance operation.

Wherein, the target sensor includes a laser sensor, and accordingly, the plurality of sensors 701 includes a laser sensor.

In some embodiments, the processor 702 is further configured to classify the environment into multiple environmental categories according to the observable range of the sensor 701 in the environment;

The processor 702 is specifically configured to: when determining the probability of false detection of the status information of the target object according to the observable range of the sensor 701 in the environment:

Acquiring environmental probability information of the environment in which the mobile platform 700 is located belongs to each environmental category;

Acquiring a priori probability information of the sensor 701 detecting an error in the environment category;

In some embodiments, the processor 702 is specifically configured to:

According to the point cloud distribution density in the point cloud data, it is determined that the environment in which the movable platform 700 is located belongs to the environment probability information of each environment category.

In some embodiments, the processor 702 is further configured to:

According to the motion parameters, the movable platform 700 is controlled to perform obstacle avoidance operations.

In some embodiments, the processor 702 is specifically configured to:

The movable platform 700 is controlled to decelerate and/or steer.

In some embodiments, the processor 702 is specifically configured to:

Calculating the first distance that the movable platform 700 moves when the movable platform 700 moves from the current position to the first position where the point cloud cluster is currently located;

According to the motion parameters of the point cloud clusters and the motion parameters of the movable platform 700, predict a second position where the motion trajectory of the movable platform 700 and the motion trajectory of the point cloud cluster intersect;

Calculating the second distance that the movable platform 700 moves when the movable platform 700 moves to the second position;

If the distance difference of the second distance minus the first distance is a positive number, the movable platform 700 is controlled to perform a decelerating movement on the movement track of the distance difference.

In some embodiments, the processor 702 is specifically configured to:

Calculating the first acceleration when the movable platform 700 decelerates from the current position to the first position, and when the speed of the first position is zero;

The movable platform 700 is controlled to perform a deceleration motion at a second acceleration on the motion trajectory of the distance difference, and the absolute value of the second acceleration is smaller than the absolute value of the first acceleration.

In some embodiments, the processor 702 is specifically configured to:

In some embodiments, the target sensor data further includes image data; the target sensor further includes an image sensor, and the plurality of sensors 701 further include an image sensor.

The processor 702 is specifically configured to:

In some embodiments, the processor 702 is specifically configured to:

In some embodiments, the target sensor data further includes radar data, the target sensor further includes a radar, and the plurality of sensors 701 further includes a radar, and the radar is, for example, a millimeter wave radar.

The processor 702 is specifically configured to:

In some embodiments, the processor 702 is further configured to:

Optionally, the movable platform 700 of this embodiment may further include: a memory (not shown in the figure) for storing program codes, the memory is used for storing program codes, and when the program codes are executed, the movable platform 700 can implement the above-mentioned technical solutions.

The movable platform of this embodiment can be used to implement the technical solutions of FIG. 3 and the corresponding method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.

FIG. 8 is a schematic structural diagram of a movable platform provided by another embodiment of this application. As shown in FIG. 8, the movable platform 800 of this embodiment may include: a movable platform body 801 and a data processing device 802.

Wherein, the data processing device 802 is installed on the movable platform body 801. The data processing device 802 may be a device independent of the movable platform body 801.

The data processing device 802 may adopt the structure of the device embodiment shown in FIG. 6, and correspondingly, it may execute the technical solutions of FIG. 3 and its corresponding method embodiments. The implementation principles and technical effects are similar, and will not be repeated here.

A person of ordinary skill in the art can understand that all or part of the steps in the above method embodiments can be implemented by a program instructing relevant hardware. The foregoing program can be stored in a computer readable storage medium. When the program is executed, it is executed. Including the steps of the foregoing method embodiment; and the foregoing storage medium includes: read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks, etc., which can store program codes Medium.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. range.

Claims

A data processing method, characterized in that the method includes:

Acquire target sensor data and fusion data, where the fusion data is obtained based on data fusion of multiple sensors, and the sensor is used to collect data on the environment in which the movable platform is located, and the fusion data includes the Status information of the detected target in the environment, where the target sensor data includes point cloud data;

Performing road object point cloud clustering processing on the point cloud data to obtain a point cloud cluster, and determining the state information of the point cloud cluster;

Judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition;

If it does not, determine the probability of false detection of the status information of the target according to the observable range of the sensor in the environment where the movable platform is located, and the probability is used to indicate the Whether the mobile platform performs obstacle avoidance operations.
The method according to claim 1, further comprising:

Divide the environment into multiple environmental categories according to the observable range of the sensor in the environment;

The determining the probability of erroneous detection of the status information of the target according to the observable range of the sensor in the environment includes:

Acquiring environmental probability information of the environment in which the mobile platform is located belongs to each environmental category;

Acquiring a priori probability information of false detection of the sensor in the environment category;

According to the environmental probability information and the prior probability information, the probability of the occurrence of an erroneous detection of the state information of the target object is determined.
The method according to claim 2, wherein said obtaining environmental probability information of the environment in which the mobile platform is located belongs to each environmental category, comprising:

According to the point cloud distribution density in the point cloud data, it is determined that the environment in which the movable platform is located belongs to the environment probability information of each environment category.
The method according to any one of claims 1-3, further comprising:

If the probability is greater than the preset probability, searching for a point cloud cluster corresponding to the target object in the point cloud data;

Acquiring the motion parameter of the point cloud cluster corresponding to the target;

According to the motion parameters, the movable platform is controlled to perform obstacle avoidance operations.
The method according to claim 4, wherein the controlling the movable platform to perform an obstacle avoidance operation comprises:

Control the decelerating movement and/or steering movement of the movable platform.
The method according to claim 5, wherein said controlling said movable platform to decelerate movement comprises:

Calculating the first distance that the movable platform moves when the movable platform moves from the current position to the first position where the point cloud cluster is currently located;

According to the motion parameters of the point cloud clusters and the motion parameters of the movable platform, predict a second position where the motion trajectory of the movable platform and the motion trajectory of the point cloud cluster intersect;

Calculating the second distance that the movable platform moves when the movable platform moves to the second position;

If the distance difference of the second distance minus the first distance is a positive number, the movable platform is controlled to perform a decelerating movement on the movement track of the distance difference.
The method according to claim 6, wherein the controlling the movable platform to perform a decelerating movement on the movement path of the distance difference comprises:

Calculating the first acceleration when the movable platform decelerates from the current position to the first position, and when the speed of the first position is zero;

The movable platform is controlled to perform a deceleration movement with a second acceleration on the movement track of the distance difference, and the absolute value of the second acceleration is smaller than the absolute value of the first acceleration.
The method according to any one of claims 1-7, wherein:

The state information includes any parameter information among object attributes, position, orientation, speed, and acceleration, and the consistency condition includes at least one of the following:

There are point cloud clusters corresponding to the target object in the point cloud data;

There is state information of the target object corresponding to any one of the point cloud clusters in the fusion data;

The parameter information of the target corresponding to the point cloud cluster is consistent with the parameter information of the point cloud cluster.
The method according to claim 8, wherein the judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition comprises:

Determine the current predicted position of the target object according to the historical speed parameter corresponding to the target object;

Judging whether the current position of the point cloud cluster corresponding to the target object is consistent with the predicted position;

If they are inconsistent, it is determined that the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.
The method according to claim 8, wherein the target sensor data further comprises image data;

The determining whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition includes:

Determining whether there is a point cloud cluster corresponding to the target object in the point cloud data according to the intensity of the pixels in the image data;

If it does not exist, it is determined that the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.
The method according to claim 8, wherein the point cloud cluster is based on clustering of laser point cloud points that do not conform to a plane or a target surface, and the target surface is a curved surface with a curvature lower than a preset curvature.
The method according to claim 8, wherein the judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition comprises:

Determine the predicted speed of the point cloud cluster according to the position of the point cloud cluster corresponding to the target in the first frame and the position of the point cloud cluster in the second frame;

If the predicted speed is inconsistent with the speed in the state information of the target, the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.
The method according to claim 8, wherein the target sensor data further includes radar data, and the judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition comprises:

Determine the predicted speed of the point cloud cluster according to the radar data;

If the predicted speed is inconsistent with the speed in the state information of the target, the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.
The method according to claim 8, wherein the target sensor data further includes radar data, and the judging whether the state information of the point cloud cluster and the state information of the target object meet the consistency condition comprises:

Determine the predicted acceleration of the point cloud cluster according to the radar data or point cloud data;

If the predicted acceleration is inconsistent with the acceleration in the state information of the target object, the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition.
The method according to claim 8, further comprising:

If the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition, the parameter information of the point cloud cluster corresponding to the target object is used as the parameter information of the target object.
A data processing device, which is characterized by comprising: a plurality of sensors and processors;

The processor is configured to acquire target sensor data and fusion data, where the fusion data is obtained by fusion of the data of the multiple sensors, and the sensor is used for data collection of the environment in which the movable platform is located , The fusion data includes the status information of the detected target in the environment, the target sensor data includes point cloud data; performing road surface object point cloud clustering processing on the point cloud data to obtain point cloud clusters, And determine the state information of the point cloud cluster; determine whether the state information of the point cloud cluster and the state information of the target meet the consistency condition; if not, then according to the location of the sensor on the movable platform The observable range in the environment in which it is located determines the probability of false detection of the status information of the target, and the probability is used to indicate whether the movable platform performs an obstacle avoidance operation.
The device according to claim 16, wherein the processor is further configured to classify the environment into multiple environmental categories according to the observable range of the sensor in the environment;

When the processor determines the probability of false detection of the status information of the target object according to the observable range of the sensor in the environment, it is specifically configured to:

Acquiring environmental probability information of the environment in which the mobile platform is located belongs to each environmental category;

Acquiring a priori probability information of false detection of the sensor in the environment category;

According to the environmental probability information and the prior probability information, the probability of the occurrence of an erroneous detection of the state information of the target object is determined.
The device according to claim 17, wherein the processor is specifically configured to:

According to the point cloud distribution density in the point cloud data, it is determined that the environment in which the movable platform is located belongs to the environment probability information of each environment category.
The device according to any one of claims 16-18, wherein the processor is further configured to:

If the probability is greater than the preset probability, searching for a point cloud cluster corresponding to the target object in the point cloud data;

Acquiring the motion parameter of the point cloud cluster corresponding to the target;

According to the motion parameters, the movable platform is controlled to perform obstacle avoidance operations.
The device according to claim 19, wherein the processor is specifically configured to:

Control the decelerating movement and/or steering movement of the movable platform.
The device according to claim 20, wherein the processor is specifically configured to:

Calculating the first distance that the movable platform moves when the movable platform moves from the current position to the first position where the point cloud cluster is currently located;

According to the motion parameters of the point cloud clusters and the motion parameters of the movable platform, predict a second position where the motion trajectory of the movable platform and the motion trajectory of the point cloud cluster intersect;

Calculating the second distance that the movable platform moves when the movable platform moves to the second position;

If the distance difference of the second distance minus the first distance is a positive number, the movable platform is controlled to perform a decelerating movement on the movement track of the distance difference.
The device according to claim 21, wherein the processor is specifically configured to:

Calculating the first acceleration when the movable platform decelerates from the current position to the first position, and when the speed of the first position is zero;

The movable platform is controlled to perform a deceleration movement with a second acceleration on the movement track of the distance difference, and the absolute value of the second acceleration is smaller than the absolute value of the first acceleration.
The device according to any one of claims 16-22, characterized in that:

The state information includes any parameter information among object attributes, position, orientation, speed, and acceleration, and the consistency condition includes at least one of the following:

There are point cloud clusters corresponding to the target object in the point cloud data;

There is state information of the target object corresponding to any one of the point cloud clusters in the fusion data;

The parameter information of the target corresponding to the point cloud cluster is consistent with the parameter information of the point cloud cluster.
The device according to claim 23, wherein the processor is specifically configured to:

Determine the current predicted position of the target object according to the historical speed parameter corresponding to the target object;

Judging whether the current position of the point cloud cluster corresponding to the target object is consistent with the predicted position;

If they are inconsistent, it is determined that the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.
The device according to claim 23, wherein the target sensor data further comprises image data;

The processor is specifically used for:

Determining whether there is a point cloud cluster corresponding to the target object in the point cloud data according to the intensity of the pixels in the image data;

If it does not exist, it is determined that the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.
The device according to claim 23, wherein the point cloud cluster is based on a laser point cloud point clustering that does not conform to a plane or does not conform to a target surface, and the target surface is a surface with a curvature lower than a preset curvature.
The device according to claim 23, wherein the processor is specifically configured to:

Determine the predicted speed of the point cloud cluster according to the position of the point cloud cluster corresponding to the target in the first frame and the position of the point cloud cluster in the second frame;

If the predicted speed is inconsistent with the speed in the state information of the target, the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.
The device according to claim 23, wherein the target sensor data further comprises radar data, and the processor is specifically configured to:

Determine the predicted speed of the point cloud cluster according to the radar data;

If the predicted speed is inconsistent with the speed in the state information of the target, the state information of the point cloud cluster and the state information of the target do not meet the consistency condition.
The device according to claim 23, wherein the target sensor data further comprises radar data, and the processor is specifically configured to:

Determine the predicted acceleration of the point cloud cluster according to the radar data or point cloud data;

If the predicted acceleration is inconsistent with the acceleration in the state information of the target object, the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition.
The device according to claim 23, wherein the processor is further configured to:

If the state information of the point cloud cluster and the state information of the target object do not meet the consistency condition, the parameter information of the point cloud cluster corresponding to the target object is used as the parameter information of the target object.
A movable platform, comprising: a movable platform body and the data processing device according to any one of claims 16-30, wherein the data processing device is installed on the movable platform body.
The movable platform according to claim 31, wherein the movable platform comprises an unmanned aerial vehicle, an unmanned vehicle, an unmanned boat, a robot, or an autonomous vehicle.
A readable storage medium, wherein a computer program is stored on the readable storage medium; when the computer program is executed, the data processing method according to any one of claims 1-15 is realized.