CN114528941A - Sensor data fusion method and device, electronic equipment and storage medium - Google Patents
Sensor data fusion method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114528941A CN114528941A CN202210156864.7A CN202210156864A CN114528941A CN 114528941 A CN114528941 A CN 114528941A CN 202210156864 A CN202210156864 A CN 202210156864A CN 114528941 A CN114528941 A CN 114528941A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- determining
- boundary
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000007500 overflow downdraw method Methods 0.000 title abstract description 18
- 238000000034 method Methods 0.000 claims abstract description 46
- 230000004927 fusion Effects 0.000 claims abstract description 34
- 238000012545 processing Methods 0.000 claims description 16
- 230000003068 static effect Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 12
- 238000001914 filtration Methods 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 5
- 239000000126 substance Substances 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 8
- 230000008447 perception Effects 0.000 abstract description 7
- 230000000875 corresponding effect Effects 0.000 description 42
- 238000004364 calculation method Methods 0.000 description 13
- 238000010586 diagram Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 6
- 230000001133 acceleration Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 102100024633 Carbonic anhydrase 2 Human genes 0.000 description 2
- 101000760643 Homo sapiens Carbonic anhydrase 2 Proteins 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Traffic Control Systems (AREA)
Abstract
The disclosure provides a sensor data fusion method and device, electronic equipment and a storage medium, and relates to the technical field of automatic driving and target detection. The method comprises the following steps: acquiring image data and point cloud data corresponding to a plurality of targets in the surrounding environment of the vehicle through a sensor, and fusing the image data and the point cloud data to obtain feature data of each target; acquiring boundary data of a drivable area of a vehicle, and clustering the boundary data to obtain position data of each target; based on the feature data and the position data, attribute information of each object is determined. According to the technical scheme, the attribute information corresponding to each target can be determined based on the data obtained by fusion and the data obtained by clustering, so that the environment perception accuracy and the scene generalization capability of the intelligent vehicle are improved.
Description
Technical Field
The present disclosure relates to the field of automatic driving technologies, and in particular, to a sensor data fusion method and apparatus, an electronic device, and a storage medium.
Background
In the fields of automatic driving and mobile robots, the perception of the environment is important, and the detection of the boundary of a travelable area (freespace) of the surrounding environment by using vision or laser radar is an implementation mode. However, the detection result of the boundary detection of the travelable area can only describe the boundary of the travelable area in the form of point coordinates, and it is impossible to distinguish whether these boundary points are detected moving traffic participants or static objects such as buildings, and it is impossible to determine which target level information each point belongs to, which results in poor environmental perception accuracy of the smart car and limited scene generalization capability.
Disclosure of Invention
The disclosure provides a sensor data fusion method, a sensor data fusion device, an electronic device and a storage medium.
According to an aspect of the present disclosure, there is provided a sensor data fusion method, including:
acquiring image data and point cloud data corresponding to a plurality of targets in the surrounding environment of the vehicle through a sensor, and fusing the image data and the point cloud data to obtain feature data of each target;
acquiring boundary data of a drivable area of a vehicle, and clustering the boundary data to obtain position data of each target;
based on the feature data and the position data, attribute information of each object is determined.
According to another aspect of the present disclosure, there is provided a sensor data fusion apparatus including:
the fusion module is used for acquiring image data and point cloud data corresponding to a plurality of targets in the surrounding environment of the vehicle through a sensor, and fusing the image data and the point cloud data to obtain feature data of each target;
the clustering module is used for acquiring boundary data of a drivable area of the vehicle and clustering the boundary data to obtain position data of each target;
and the determining module is used for determining the attribute information of each target based on the characteristic data and the position data.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method according to any one of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform a method in any of the embodiments of the present disclosure.
According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method in any of the embodiments of the present disclosure.
The invention provides a sensor data fusion method, a sensor data fusion device, electronic equipment and a storage medium.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a flow chart of a method for sensor data fusion in an embodiment of the present disclosure;
FIG. 2 is a schematic diagram illustrating fusion of position data of a target and position feature data of the target according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of calculating a standard deviation of boundary points according to an embodiment of the disclosure;
FIG. 4 is a schematic diagram of a grid map according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of a sensor data fusion method according to an embodiment of the disclosure;
FIG. 6 is a flow chart of a method for sensor data fusion in an embodiment of the present disclosure;
FIG. 7 is a schematic diagram of a sensor data fusion apparatus according to an embodiment of the present disclosure;
FIG. 8 is a block diagram of an electronic device for implementing a sensor data fusion method of an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a flowchart of a sensor data fusion method according to an embodiment of the present disclosure, and the method may be applied to a sensor data fusion device, for example, the device may perform sensor data fusion and the like when deployed in a server, a terminal device, or other processing devices. In some possible implementations, the method may also be implemented by a processor calling computer readable instructions stored in a memory. As shown in fig. 1, includes:
step S101, acquiring image data and point cloud data corresponding to a plurality of targets in the surrounding environment of the vehicle through a sensor, and fusing the image data and the point cloud data to obtain feature data of each target;
the execution main body of the embodiment of the present disclosure may be a vehicle-mounted terminal. The intelligent automobile obtains image data and point cloud data of each target in the surrounding environment of the automobile through various sensors, such as a camera, a laser radar, a millimeter wave radar, an ultrasonic detector and the like, installed on the automobile during driving, wherein the targets comprise static targets and dynamic targets.
Wherein the image data may be a two-dimensional or three-dimensional image of the object. After an image of the surrounding environment is acquired by the camera, the image is input into the target detection model, and the type, position and the like of the target output by the target detection model are obtained. Through laser radar, millimeter wave radar, ultrasonic detector, etc., the point cloud data of each target can be obtained, and the size, speed, acceleration, attitude, etc. corresponding to each target can be obtained according to the point cloud data. And associating each target with the corresponding image data and point cloud data, and calculating to obtain data such as category, size, speed, acceleration, posture, position and the like corresponding to each target as feature data corresponding to the target.
Step S102, acquiring boundary data of a drivable area of a vehicle, and clustering the boundary data to obtain position data of each target;
by detecting the boundary of the travelable area with respect to the image data and the point cloud data acquired by the various sensors, the boundary data of the travelable area can be obtained. For example, by a freespace algorithm, boundary points of the travelable region are obtained. By clustering the boundary points of the travelable region, the boundary point corresponding to each target can be obtained as the position data corresponding to the target.
Step S103, based on the characteristic data and the position data, the attribute information of each target is determined.
And fusing the characteristic data and the position data corresponding to each target to obtain attribute information of each target, wherein the attribute information can comprise attributes such as the category, the size, the speed, the acceleration, the posture, the position and the like of each target.
In the related art, image data and point cloud data acquired by a sensor are fused to serve as final target data, so that the accuracy is not high, and the requirements of parking areas and other scenes needing environment boundary refinement cannot be met. However, only by fusing the boundary data of the travelable area, it is not possible to know which boundary points belong to which target, that is, the target level information of the surrounding environment, and the specific information such as the type, orientation, size, etc. of the target, and the sensing accuracy is poor.
According to the sensor data fusion method provided by the embodiment of the disclosure, the data of a plurality of targets acquired by the sensor are fused, the boundary data of the travelable area is clustered, and the attribute information corresponding to each target can be determined based on the data obtained by fusion and the data obtained by clustering, so that the environment perception accuracy and the scene generalization capability of the intelligent vehicle are improved.
In one possible implementation, determining attribute information of each target based on the feature data and the position data includes:
and for each target, if the feature data comprises position feature data, fusing the position data of the target and the position feature data of the target, and taking the fused data as the position attribute in the attribute information of the target.
In practical application, according to feature data obtained by fusing image data and point cloud data, wherein the feature data comprises position feature data, the position feature data is corrected according to position data obtained by detecting the boundary of the travelable region, and the corrected position is used as a position attribute in attribute information of a target.
In the embodiment of the disclosure, the position of the target is more accurate by fusing the position data of the target obtained by fusing the data of the plurality of sensors and the position characteristic data of the target obtained by clustering.
In one possible implementation, fusing the position data of the target and the position feature data of the target includes:
determining the residual error from each boundary point in the position data of the target to the position point in the corresponding position characteristic data;
determining a residual error of the target based on the residual errors from the boundary points to the corresponding position points;
and performing filtering processing based on the residual error of the target and the variance of the residual error of the target to obtain fused data.
In practical application, the position data of each target is determined, namely each boundary point of the target is obtained, the residual error between the boundary point and the position point closest to the boundary point is calculated for each boundary point, and the residual error of the target is determined according to the residual error corresponding to each boundary point. Optionally, the mean value of the residuals corresponding to each boundary point is calculated to obtain an average residual, and the average residual is used as the target residual. And calculating the variance of the residual of the target, filtering the residual of the target and the variance of the residual, and taking the data output by the filter as fused data. Wherein the filtering process may be implemented by a kalman filter and its derived variants. The data output by the filter is the corrected position of the target.
In the embodiment of the disclosure, the position of the target after correction can be obtained by performing filtering processing based on the residual error of the target and the variance of the residual error of the target, so that the obtained position of the target is more accurate, and the precision of the intelligent vehicle for perceiving the position of the target can be higher.
Fig. 2 is a schematic diagram illustrating fusion of position data of a target and position feature data of the target in an embodiment of the present disclosure, as shown in fig. 2, Ego represents a current vehicle mounted with a plurality of sensors, Car1 and Car2 are targets in a surrounding environment of the current vehicle, each target corresponds to a plurality of boundary points obtained by a freespace algorithm and each position point of a position frame obtained by fusing image data and point cloud data, residual errors between each boundary point and a position point closest to the boundary point are calculated, an average value of residual errors corresponding to each boundary point of Car1 is calculated, an average residual error is obtained, and the average residual error is used as a residual error of Car 1. The variance of the residual is calculated, and the residual of Car1 and the variance of the residual are filtered by a kalman filter to obtain the corrected position of Car1, and similarly, the corrected position of Car2 can also be obtained.
In a possible implementation manner, before performing the filtering process based on the residual of the target and the variance of the residual of the target, the method further includes:
determining the variance of each boundary point in the position data;
based on the variance of each boundary point, the variance of the residual of the target is determined.
In practical applications, the variance of each boundary point may be a device parameter of the sensor, that is, an error value of the sensor when acquiring data of the target, or may be obtained by calculating the variance of the boundary point for a sensor whose accuracy is directly affected by the distance, for example, a camera, etc. Alternatively, for an object, the average of the variances of the boundary points may be calculated as the variance of the residual of the object.
In the embodiment of the disclosure, the variance of the residual error of the target is determined based on the variance of each boundary point, the calculation method is simple, and the result is accurate.
For a specific implementation manner of obtaining the variance of the boundary point through calculation, see the following embodiment:
in one possible implementation, determining the variance of each boundary point in the position data includes:
acquiring the observation height, the observation angle and the standard deviation of the observation angle of the boundary point;
and determining the variance of the boundary point based on the observation height, the observation angle and the standard deviation of the observation angle of the boundary point.
In practical applications, the observation height of the boundary point may be a height from the ground of the sensor, the observation angle may be an angle corresponding to an observation range of the sensor, the standard deviation of the observation angle may be a preset empirical value, and the variance of the boundary point may be obtained based on the observation height of the boundary point, the observation angle, and the standard deviation of the observation angle.
In the embodiment of the disclosure, for the case that the sensor with the accuracy directly affected by the distance cannot directly obtain the variance of the boundary point, the variance of the boundary point can be determined according to the observation parameters of the sensor.
Fig. 3 is a schematic diagram of calculating a standard deviation of boundary points according to an embodiment of the disclosure. As shown in fig. 3, the standard deviation of the boundary points acquired by the camera is calculated. Wherein, H is the observation height, L is the horizontal distance from the boundary point to the camera, and θ is the observation angle, the following formula can be obtained:
L＝H·tanθ (1)
the following formula is obtained according to formula (1):
Lsqm＝H·sec2θ·θsqm (2)
wherein L issqmDenotes the standard deviation, θ, of the boundary pointssqmThe standard deviation of the angle theta is indicated. The variance of the boundary point is LsqmSquare of (d).
After the variance of the boundary points is calculated, for the boundary points with different horizontal distances from the sensor, if the difference value between the horizontal distances of the boundary points and the sensor is smaller than a preset threshold value, the same variance is configured, and a variance lookup table is established, wherein the variance lookup table comprises the horizontal distances between the boundary points and the sensor and the variance of the boundary points. The variance lookup table can be configured in advance, and the variance of the boundary point can be queried through the variance lookup table. The same variance is configured for boundary points different from the sensor horizontal distance, and the calculation efficiency of the variance can be improved.
In a possible implementation manner, the fusing the image data and the point cloud data to obtain feature data of each target includes:
determining first similarity between current image data and point cloud data and historical image data and point cloud data;
and according to the first similarity, correlating the current image data and the point cloud data with the historical image data and the point cloud data to obtain the characteristic data of each target.
In practical application, in the driving process of a vehicle, a sensor continuously acquires image data and point cloud data, first similarity calculation is carried out on current image data and point cloud data acquired each time and historical image data and point cloud data acquired before, specifically, the similarity of relevant characteristics such as size, position, speed and the like can be calculated, historical image data and point cloud data matched with the current image data and the point cloud data are determined in a global similarity or similarity optimal mode, the current image data and the point cloud data are associated with the matched historical image data and the point cloud data, and updating of feature data corresponding to each target is achieved.
In the embodiment of the disclosure, through similarity calculation and data association, the feature data of the target is updated.
In one possible implementation manner, the method further includes:
and determining the feature data of the new target according to the current image data and the point cloud data under the condition that the first similarity is smaller than a preset threshold value.
In practical application, if the similarity between the current image data and the current cloud data are not matched with the current image data and the current cloud.
In the embodiment of the disclosure, through similarity calculation, whether the image data and the point cloud data acquired each time are feature data of a historical target can be determined, so that matching between the feature data and the corresponding target is realized.
In a possible implementation manner, clustering the boundary data to obtain the position data of each target includes:
establishing a grid map according to the boundary data, and determining the probability of the existence of a target in each grid in the grid map;
determining the motion attribute of each target based on the probability of the target existing in each grid;
and clustering the boundary data based on the motion attribute of each target to obtain the position data of each target, wherein the motion attribute comprises motion or stillness.
In practical application, a grid map is created for the boundary data, and the resolution of the grid can be set according to a specific scene.
Fig. 4 is a schematic diagram of a grid map according to an embodiment of the disclosure. As shown in fig. 4, a grid in the grid map includes three states, occupied (occupancy), free (free), Unknown (Unknown).
And (3) carrying out probability estimation updating on grids in a sensor observation range (finished Of View), wherein the grids in the invisible area are the previous probabilities, and the initial values Of the grids are the probabilities corresponding to the unknown states. And on the grid map after initialization, when new data enters, filtering and updating the probability based on a binary Bayesian filter. And judging the occupied or empty state of the current grid according to the updated probability, and counting the state and probability change condition of the corresponding grid by using a sliding window technology. And judging the motion attribute of the target corresponding to the grid, wherein the motion attribute comprises motion or static, and the specific judgment method can be based on a D-S (Dempster-Shafer) evidence theory or other schemes. For Clustering the boundary data of the target with the motion attribute as motion, the Clustering method comprises Density-Based Clustering of Applications with Noise (DBSCAN), K-means and the like.
In the embodiment of the disclosure, the boundary data is processed by establishing the grid map, and the boundary data is clustered, so that the position data corresponding to each target can be obtained, and the determination of the target-level position data is realized.
In a possible implementation manner, clustering the boundary data based on the motion attribute of each target to obtain the position data of each target includes:
respectively taking each target as a current target, and if the current target is determined to be a static target according to the characteristic data, and the characteristic data comprises position characteristic data, mapping the position characteristic data of the current target to a grid map to obtain an interested area in the grid map;
and clustering boundary data corresponding to the region of interest to obtain position data of the current target.
In practical application, each target is respectively used as a current target, the position characteristic data is mapped to a grid map for the target with a static motion attribute to obtain an interested area in the grid map, namely, the initial selection of the interested area is completed, and the clustering of boundary points is performed in the interested area to obtain the position data of the target.
In the embodiment of the disclosure, a clustering mode of boundary data of objects with static motion attributes is provided, so as to obtain position data of each static object.
In one possible implementation, determining attribute information of each target based on the feature data and the position data includes:
correlating the characteristic data and the position data according to corresponding targets to obtain correlation data of each target;
and determining the attribute information of each target according to the associated data of each target.
In practical application, for each target, feature data is obtained by fusing image data and point cloud data, and the feature data and the position data obtained by clustering boundary data are associated to obtain feature data and position data corresponding to each target, and the feature data and the position data are used as associated data, and attribute information such as category, size, position, speed, posture and the like corresponding to each target can be obtained by classifying the associated data.
In the embodiment of the disclosure, the target-level information of each target can be obtained by the association and fusion of the feature data and the position data, and the perception accuracy of the vehicle to the surrounding environment can be improved.
The specific implementation manner of associating the feature data with the position data is shown in the following embodiment:
in a possible implementation manner, associating the feature data and the position data according to corresponding targets to obtain associated data of each target, including:
if the characteristic data comprises position characteristic data, determining second similarity of the position characteristic data and each position data;
and associating the characteristic data of the target with the position data based on the second similarity to obtain associated data of the target.
In practical application, for each target, the position feature data is mapped onto a grid map, the position similarity between the position feature data and the position data, namely the second similarity, is calculated, and the feature data corresponding to the position feature data with the position similarity larger than a preset threshold value is associated with the position data to obtain associated data of each target.
In the embodiment of the disclosure, the correlation between the feature data and the position data of the target is performed by calculating the position similarity, so that the calculation is simple and the result accuracy is high.
In a possible implementation manner, the fusing image data and point cloud data to obtain feature data of each target includes:
under the condition that the number of the sensors is multiple, time synchronization and space synchronization are respectively carried out on the image data and the point cloud data acquired by the multiple sensors, and the image data and the point cloud data after space-time synchronization are obtained;
and fusing the image data and the point cloud data after the time-space synchronization to obtain the characteristic data of each target.
In practical application, a vehicle can be provided with a plurality of sensors and a plurality of types of sensors, when calculation is carried out, data acquired by the sensors need to be subjected to time synchronization according to timestamps, the data are converted into the same coordinate system, space synchronization is achieved, the data subjected to time-space synchronization are fused, and according to the fused data, which data belong to which target is determined, so that characteristic data of each target is obtained.
In the embodiment of the disclosure, the data acquired by the sensors are subjected to space-time synchronization and used as the basis of the subsequent data fusion calculation, so that the space-time consistency of the data can be ensured, and the accuracy of the calculation result is improved.
Optionally, the embodiment of the present disclosure may further include data validity detection, which determines whether data acquired by the sensor is valid according to attributes such as an observation range and a measurement range of the sensor, removes invalid data, saves computing resources, and meanwhile, may also improve accuracy of a computing result.
Fig. 5 is a schematic diagram of a sensor data fusion method according to an embodiment of the disclosure. First, fusion updating of image data and point cloud data of each target acquired by a sensor is introduced. Specifically, image data and point cloud data of a plurality of targets are acquired through a sensor, as shown in fig. 5, and image data is acquired through a camera, as shown in a CameraBbox; point cloud data is acquired by Radar (Radar) and ultrasonic detector (Sonar). Preprocessing image data and point cloud data of each target, and specifically comprises the following steps: and carrying out time synchronization according to the time stamp, converting the time stamp into the same coordinate system to realize space synchronization, carrying out data validity detection on the data subjected to time-space synchronization, and removing invalid data to obtain the corresponding characteristic data of each target. Similarity calculation is carried out on the current image data and the point cloud data, the historical image data and the point cloud data, the currently acquired data are correlated to the matched historical data, and the updated characteristic data (such as 'the track state existing in the updating' shown in the figure) of the target are obtained. If the currently acquired data is not matched with all the historically acquired data, determining the current data as the characteristic data of a new target (such as 'creating a new track' shown in the figure), thereby realizing the update of the characteristic data of the target (such as 'sensor observation and track fusion' shown in the figure). The flight path can include the position, speed, acceleration, orientation, attitude and the like of the target, and the future flight path of the target can be predicted according to the historical data of the target. The process realizes the fusion and updating of the image data and the point cloud data of each target.
Next, a clustering process of boundary data of a drivable region of the vehicle is described. Specifically, data acquired by sensors such as radar and cameras are processed through a freespace algorithm to obtain boundary data of a travelable area, a grid map is created, multi-frame data fusion updating is performed on the basis of the grid map, the probability of a target existing in each grid is determined, the motion attribute of the target corresponding to each grid is determined (such as grid point dynamic and static attribute classification shown in the figure), the boundary data of the target is clustered according to the motion attribute (such as dynamic and static attribute level-based geometric distribution clustering shown in the figure), the boundary data of the target with the motion attribute being motion is clustered, and the clustering method comprises density-based clustering DBSCAN, K-means and the like. And for the target with static motion attribute, mapping the position characteristic data to the grid map to obtain an interested area in the grid map, and clustering boundary points in the interested area to obtain the position data of the target. For each target, feature data is obtained by fusing image data and point cloud data, and the feature data and the position data corresponding to each target can be obtained by correlating with position data obtained by clustering boundary data and serve as correlation data, so that the correlation and fusion of the feature data and the position data are realized, and the type, size, position and speed attribute information of each target is output. In addition, the method also comprises the steps of managing the associated data of each target (such as 'track management' shown in the figure), carrying out space-time synchronization on the characteristic data and the position data, maintaining the life cycle, determining the characteristic data of a new target, and deleting the characteristic data of the target which does not appear in the observation range any more.
Fig. 6 is a flowchart of a sensor data fusion method according to an embodiment of the disclosure. As shown in fig. 6, the method includes:
step S601, acquiring image data and point cloud data corresponding to a plurality of targets in the surrounding environment of the vehicle through a sensor, and fusing the image data and the point cloud data to obtain feature data of each target;
step S602, acquiring boundary data of a travelable area of a vehicle, creating a grid map aiming at the boundary data, and determining the probability of each grid in the grid map having a target;
step S603, determining the motion attribute of each target based on the probability of the target existing in each grid;
step S604, clustering boundary data based on the motion attributes of the targets to obtain position data of the targets;
step S605, for each target, if the feature data comprises position feature data, determining the residual error from each boundary point in the position data of the target to the position point in the corresponding position feature data;
step S606, determining the residual error of the target based on the residual errors from each boundary point to the corresponding position point;
step S607, filtering based on the residual error of the target and the variance of the residual error of the target to obtain fused data, and using the fused data as the position attribute in the attribute information of the target.
According to the sensor data fusion method provided by the disclosure, the data of a plurality of targets acquired by the sensor are fused, the boundary data of the travelable area is clustered, and the attribute information corresponding to each target can be determined based on the fused data and the clustered data, so that the environment perception accuracy and the scene generalization capability of the intelligent vehicle are improved.
Fig. 7 is a schematic diagram of a sensor data fusion apparatus according to an embodiment of the disclosure. As shown in fig. 7, the sensor data fusion apparatus may include:
the fusion module 701 is used for acquiring image data and point cloud data corresponding to a plurality of targets in the surrounding environment of the vehicle through a sensor, and fusing the image data and the point cloud data to obtain feature data of each target;
a clustering module 702, configured to obtain boundary data of a drivable area of a vehicle, and perform clustering on the boundary data to obtain position data of each target;
a determining module 703, configured to determine attribute information of each target based on the feature data and the position data.
According to the sensor data fusion device provided by the embodiment of the disclosure, the data of a plurality of targets acquired by the sensor are fused, the boundary data of the travelable area is clustered, and the attribute information corresponding to each target can be determined based on the data obtained by fusion and the data obtained by clustering, so that the environment perception accuracy and the scene generalization capability of the intelligent vehicle are improved.
In one possible implementation, the determining module 703 is configured to:
and for each target, if the feature data comprises position feature data, fusing the position data of the target and the position feature data of the target, and taking the fused data as the position attribute in the attribute information of the target.
In one possible implementation, the determining module 703 includes a first determining unit, a second determining unit, and a processing unit;
a first determining unit, configured to determine a residual between each boundary point in the target location data and a location point in the corresponding location feature data;
the second determining unit is used for determining the residual error of the target based on the residual errors from the boundary points to the corresponding position points;
and the processing unit is used for carrying out filtering processing based on the residual error of the target and the variance of the residual error of the target to obtain fused data.
In a possible implementation manner, the second determining unit is further configured to:
determining the variance of each boundary point in the position data;
based on the variance of each boundary point, the variance of the residual of the target is determined.
In one possible implementation, the second determining unit, when determining the variance of each boundary point in the position data, is configured to:
acquiring the observation height, the observation angle and the standard deviation of the observation angle of the boundary point;
and determining the variance of the boundary point based on the observation height, the observation angle and the standard deviation of the observation angle of the boundary point.
In a possible implementation manner, the fusion module 701 is configured to:
determining first similarity between current image data and point cloud data and historical image data and point cloud data;
and according to the first similarity, correlating the current image data and the point cloud data with the historical image data and the point cloud data to obtain the feature data of each target.
In a possible implementation manner, the fusion module 701 is further configured to:
and under the condition that the first similarity is smaller than a preset threshold, determining the feature data of a new target according to the current image data and the point cloud data.
In one possible implementation, the clustering module 702 is configured to:
establishing a grid map according to the boundary data, and determining the probability of the existence of a target in each grid in the grid map;
determining the motion attribute of each target based on the probability of the target existing in each grid;
and clustering the boundary data based on the motion attribute of each target to obtain the position data of each target, wherein the motion attribute comprises motion or stillness.
In a possible implementation manner, the clustering module 702 is specifically configured to:
respectively taking each target as a current target, and if the current target is determined to be a static target according to the characteristic data, and the characteristic data comprises position characteristic data, mapping the position characteristic data of the current target to a grid map to obtain an interested area in the grid map;
and clustering the boundary data corresponding to the region of interest to obtain the position data of the current target.
In one possible implementation, the determining module 703 is configured to:
correlating the characteristic data and the position data according to corresponding targets to obtain correlation data of each target;
and determining the attribute information of each target according to the associated data of each target.
In a possible implementation manner, the determining module 703 is specifically configured to:
if the characteristic data comprises position characteristic data, determining second similarity of the position characteristic data and each position data;
and associating the characteristic data of the target with the position data based on the second similarity to obtain associated data of the target.
In a possible implementation manner, the fusion module 701 is specifically configured to:
under the condition that the number of the sensors is multiple, time synchronization and space synchronization are respectively carried out on the image data and the point cloud data acquired by the multiple sensors, and the image data and the point cloud data after space-time synchronization are obtained;
and fusing the image data and the point cloud data after the time-space synchronization to obtain the characteristic data of each target.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method according to any one of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform a method in any of the embodiments of the present disclosure.
According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method in any of the embodiments of the present disclosure.
FIG. 8 shows a schematic block diagram of an example electronic device 800 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and information necessary for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/information with other devices via a computer network such as the internet and/or various telecommunication networks.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving information and instructions from, and transmitting information and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable information processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as an information server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server combining a blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (27)
1. A method of sensor data fusion, the method comprising:
acquiring image data and point cloud data corresponding to a plurality of targets in the surrounding environment of the vehicle through a sensor, and fusing the image data and the point cloud data to obtain feature data of each target;
acquiring boundary data of a drivable area of the vehicle, and clustering the boundary data to obtain position data of each target;
determining attribute information of the respective targets based on the feature data and the position data.
2. The method of claim 1, wherein said determining attribute information for the respective targets based on the feature data and the location data comprises:
and for each target, if the feature data comprises position feature data, fusing the position data of the target and the position feature data of the target, and taking the fused data as the position attribute in the attribute information of the target.
3. The method of claim 2, wherein said fusing the location data of the object and the location characteristic data of the object comprises:
determining the residual error from each boundary point in the position data of the target to the position point in the corresponding position characteristic data;
determining the residual error of the target based on the residual errors from the boundary points to the corresponding position points;
and performing filtering processing based on the residual error of the target and the variance of the residual error of the target to obtain fused data.
4. The method of claim 3, wherein prior to performing filtering processing based on the residual of the object and the variance of the residual of the object, further comprising:
determining a variance of each boundary point in the position data;
and determining the variance of the residual error of the target based on the variances of the boundary points.
5. The method of claim 4, wherein the determining the variance of each boundary point in the location data comprises:
acquiring the observation height, the observation angle and the standard deviation of the observation angle of the boundary point;
and determining the variance of the boundary point based on the observation height, the observation angle and the standard deviation of the observation angle of the boundary point.
6. The method according to claim 1, wherein the fusing the image data and the point cloud data to obtain feature data of each target comprises:
determining first similarity between current image data and point cloud data and between historical image data and point cloud data;
and associating the current image data and the point cloud data with the historical image data and the point cloud data according to the first similarity to obtain the feature data of each target.
7. The method of claim 6, further comprising:
and determining the feature data of a new target according to the current image data and the point cloud data under the condition that the first similarity is smaller than a preset threshold value.
8. The method of claim 1, wherein said clustering said boundary data to obtain location data for said objects comprises:
creating a grid map according to the boundary data, and determining the probability of the existence of a target in each grid in the grid map;
determining the motion attribute of each target based on the probability of each grid existing target;
and clustering the boundary data based on the motion attribute of each target to obtain the position data of each target, wherein the motion attribute comprises motion or stillness.
9. The method of claim 8, wherein the clustering the boundary data based on the motion attributes of the objects to obtain the position data of the objects comprises:
respectively taking each target as a current target, and if the current target is determined to be a static target according to the characteristic data, and the characteristic data comprises position characteristic data, mapping the position characteristic data of the current target to the grid map to obtain an interested area in the grid map;
and clustering the boundary data corresponding to the region of interest to obtain the position data of the current target.
10. The method of claim 1, wherein said determining attribute information for the respective targets based on the feature data and the location data comprises:
correlating the characteristic data and the position data according to corresponding targets to obtain correlation data of each target;
and determining the attribute information of each target according to the associated data of each target.
11. The method of claim 10, wherein the associating the feature data with the position data according to corresponding objects to obtain associated data of each object comprises:
if the characteristic data comprise position characteristic data, determining second similarity of the position characteristic data and each position data;
and associating the characteristic data of the target with the position data based on the second similarity to obtain associated data of the target.
12. The method according to claim 1, wherein the fusing the image data and the point cloud data to obtain feature data of each target comprises:
under the condition that the number of the sensors is multiple, time synchronization and space synchronization are respectively carried out on the image data and the point cloud data acquired by the multiple sensors, and the image data and the point cloud data after space-time synchronization are obtained;
and fusing the image data subjected to the space-time synchronization and the point cloud data to obtain the characteristic data of each target.
13. A sensor data fusion apparatus, the apparatus comprising:
the fusion module is used for acquiring image data and point cloud data corresponding to a plurality of targets in the surrounding environment of the vehicle through a sensor, and fusing the image data and the point cloud data to obtain feature data of each target;
the clustering module is used for acquiring boundary data of a drivable area of the vehicle, clustering the boundary data and obtaining position data of each target;
and the determining module is used for determining the attribute information of each target based on the characteristic data and the position data.
14. The apparatus of claim 13, wherein the means for determining is configured to:
and for each target, if the feature data comprises position feature data, fusing the position data of the target and the position feature data of the target, and taking the fused data as the position attribute in the attribute information of the target.
15. The apparatus of claim 14, wherein the determining means comprises a first determining unit, a second determining unit, and a processing unit;
the first determining unit is used for determining the residual error from each boundary point in the position data of the target to the position point in the corresponding position characteristic data;
the second determining unit is configured to determine a residual error of the target based on residual errors from the boundary points to corresponding position points;
and the processing unit is used for carrying out filtering processing based on the residual error of the target and the variance of the residual error of the target to obtain fused data.
16. The apparatus of claim 15, wherein the second determining unit is further configured to:
determining a variance of each boundary point in the position data;
and determining the variance of the residual error of the target based on the variances of the boundary points.
17. The apparatus of claim 16, wherein the second determining unit, in determining the variance of each boundary point in the location data, is to:
acquiring the observation height, the observation angle and the standard deviation of the observation angle of the boundary point;
and determining the variance of the boundary point based on the observation height, the observation angle and the standard deviation of the observation angle of the boundary point.
18. The apparatus of claim 13, wherein the fusion module is to:
determining first similarity between current image data and point cloud data and historical image data and point cloud data;
and associating the current image data and the point cloud data with the historical image data and the point cloud data according to the first similarity to obtain the feature data of each target.
19. The apparatus of claim 18, the fusion module further to:
and determining the feature data of a new target according to the current image data and the point cloud data under the condition that the first similarity is smaller than a preset threshold value.
20. The apparatus of claim 13, wherein the clustering module is to:
creating a grid map according to the boundary data, and determining the probability of the existence of a target in each grid in the grid map;
determining the motion attribute of each target based on the probability of each grid existing target;
and clustering the boundary data based on the motion attribute of each target to obtain the position data of each target, wherein the motion attribute comprises motion or stillness.
21. The apparatus of claim 20, wherein the clustering module is specifically configured to:
respectively taking each target as a current target, and if the current target is determined to be a static target according to the characteristic data, and the characteristic data comprises position characteristic data, mapping the position characteristic data of the current target to the grid map to obtain an interested area in the grid map;
and clustering the boundary data corresponding to the region of interest to obtain the position data of the current target.
22. The apparatus of claim 13, wherein the means for determining is configured to:
correlating the characteristic data and the position data according to corresponding targets to obtain correlation data of each target;
and determining the attribute information of each target according to the associated data of each target.
23. The apparatus of claim 22, wherein the determining module is specifically configured to:
if the characteristic data comprise position characteristic data, determining second similarity of the position characteristic data and each position data;
and associating the characteristic data of the target with the position data based on the second similarity to obtain associated data of the target.
24. The apparatus according to claim 13, wherein the fusion module is specifically configured to:
under the condition that the number of the sensors is multiple, time synchronization and space synchronization are respectively carried out on the image data and the point cloud data acquired by the multiple sensors, and the image data and the point cloud data after space-time synchronization are obtained;
and fusing the image data subjected to the time-space synchronization and the point cloud data to obtain characteristic data of each target.
25. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.
26. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1-12.
27. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210156864.7A CN114528941A (en) | 2022-02-21 | 2022-02-21 | Sensor data fusion method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210156864.7A CN114528941A (en) | 2022-02-21 | 2022-02-21 | Sensor data fusion method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114528941A true CN114528941A (en) | 2022-05-24 |
Family
ID=81624139
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210156864.7A Withdrawn CN114528941A (en) | 2022-02-21 | 2022-02-21 | Sensor data fusion method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114528941A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115661798A (en) * | 2022-12-23 | 2023-01-31 | 小米汽车科技有限公司 | Method and device for determining target area, vehicle and storage medium |
| CN116844134A (en) * | 2023-06-30 | 2023-10-03 | 北京百度网讯科技有限公司 | Target detection method and device, electronic equipment, storage medium and vehicle |
-
2022
- 2022-02-21 CN CN202210156864.7A patent/CN114528941A/en not_active Withdrawn
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115661798A (en) * | 2022-12-23 | 2023-01-31 | 小米汽车科技有限公司 | Method and device for determining target area, vehicle and storage medium |
| CN115661798B (en) * | 2022-12-23 | 2023-03-21 | 小米汽车科技有限公司 | Method and device for determining target area, vehicle and storage medium |
| CN116844134A (en) * | 2023-06-30 | 2023-10-03 | 北京百度网讯科技有限公司 | Target detection method and device, electronic equipment, storage medium and vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110488234B (en) | External parameter calibration method, device, equipment and medium for vehicle-mounted millimeter wave radar | |
| US8199977B2 (en) | System and method for extraction of features from a 3-D point cloud | |
| CN111324115B (en) | Obstacle position detection fusion method, obstacle position detection fusion device, electronic equipment and storage medium | |
| CN111563450B (en) | Data processing method, device, equipment and storage medium | |
| CN111968229A (en) | High-precision map making method and device | |
| CN114528941A (en) | Sensor data fusion method and device, electronic equipment and storage medium | |
| CN108345823B (en) | Obstacle tracking method and device based on Kalman filtering | |
| CN113378760A (en) | Training target detection model and method and device for detecting target | |
| CN113887400B (en) | Obstacle detection method, model training method and device and automatic driving vehicle | |
| CN111540027B (en) | Detection method, detection device, electronic equipment and storage medium | |
| CN114926549B (en) | Three-dimensional point cloud processing method, device, equipment and storage medium | |
| CN114140759A (en) | High-precision map lane line position determining method and device and automatic driving vehicle | |
| CN114677655A (en) | Multi-sensor target detection method and device, electronic equipment and storage medium | |
| CN112509126A (en) | Method, device, equipment and storage medium for detecting three-dimensional object | |
| CN113688730A (en) | Obstacle ranging method, apparatus, electronic device, storage medium, and program product | |
| CN113706704A (en) | Method and equipment for planning route based on high-precision map and automatic driving vehicle | |
| CN113177980B (en) | Target object speed determining method and device for automatic driving and electronic equipment | |
| CN115685249A (en) | Obstacle detection method and device, electronic equipment and storage medium | |
| CN114018269B (en) | Positioning method, positioning device, electronic equipment, storage medium and automatic driving vehicle | |
| CN111538918B (en) | Recommendation method and device, electronic equipment and storage medium | |
| CN114299242A (en) | Method, device and equipment for processing images in high-precision map and storage medium | |
| CN113436233A (en) | Registration method and device of automatic driving vehicle, electronic equipment and vehicle | |
| CN113378694A (en) | Method and device for generating target detection and positioning system and target detection and positioning | |
| CN116883460A (en) | Visual perception positioning method and device, electronic equipment and storage medium | |
| CN113984072B (en) | Vehicle positioning method, device, equipment, storage medium and automatic driving vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220524 |