CN116685873A - Vehicle-road cooperation-oriented perception information fusion representation and target detection method - Google Patents
Vehicle-road cooperation-oriented perception information fusion representation and target detection method Download PDFInfo
- Publication number
- CN116685873A CN116685873A CN202180011148.3A CN202180011148A CN116685873A CN 116685873 A CN116685873 A CN 116685873A CN 202180011148 A CN202180011148 A CN 202180011148A CN 116685873 A CN116685873 A CN 116685873A
- Authority
- CN
- China
- Prior art keywords
- laser radar
- point cloud
- vehicle
- road side
- voxel level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 129
- 230000008447 perception Effects 0.000 title claims abstract description 20
- 230000004927 fusion Effects 0.000 title claims abstract description 11
- 238000012545 processing Methods 0.000 claims abstract description 35
- 230000002776 aggregation Effects 0.000 claims abstract description 28
- 238000004220 aggregation Methods 0.000 claims abstract description 28
- 238000012512 characterization method Methods 0.000 claims abstract description 10
- 238000000034 method Methods 0.000 claims description 31
- 238000004422 calculation algorithm Methods 0.000 claims description 22
- 238000009434 installation Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000011176 pooling Methods 0.000 claims description 8
- 238000002360 preparation method Methods 0.000 claims description 6
- 239000006185 dispersion Substances 0.000 claims 1
- 239000013598 vector Substances 0.000 description 18
- 230000005540 biological transmission Effects 0.000 description 13
- 238000002310 reflectometry Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 7
- 230000009466 transformation Effects 0.000 description 6
- 238000007906 compression Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000033001 locomotion Effects 0.000 description 5
- 239000013589 supplement Substances 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 241000282372 Panthera onca Species 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000006835 compression Effects 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 230000001502 supplementing effect Effects 0.000 description 3
- 230000004931 aggregating effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000237504 Crassostrea virginica Species 0.000 description 1
- 241000282414 Homo sapiens Species 0.000 description 1
- 241000282373 Panthera pardus Species 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003912 environmental pollution Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
- G01S7/40—Means for monitoring or calibrating
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/87—Combinations of systems using electromagnetic waves other than radio waves
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/003—Transmission of data between radar, sonar or lidar systems and remote stations
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4808—Evaluating distance, position or velocity data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/497—Means for monitoring or calibrating
- G01S7/4972—Alignment of sensor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0108—Measuring and analyzing of parameters relative to traffic conditions based on the source of data
- G08G1/0116—Measuring and analyzing of parameters relative to traffic conditions based on the source of data from roadside infrastructure, e.g. beacons
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
- G08G1/0133—Traffic data processing for classifying traffic situation
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
- G08G1/0141—Measuring and analyzing of parameters relative to traffic conditions for specific applications for traffic information dissemination
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
- G08G1/0145—Measuring and analyzing of parameters relative to traffic conditions for specific applications for active traffic flow control
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/048—Detecting movement of traffic to be counted or controlled with provision for compensation of environmental or other condition, e.g. snow, vehicle stopped at detector
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/164—Centralised systems, e.g. external to vehicles
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Theoretical Computer Science (AREA)
- Electromagnetism (AREA)
- Data Mining & Analysis (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Traffic Control Systems (AREA)
- Radar Systems Or Details Thereof (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
A vehicle-road cooperation-oriented perception information fusion characterization and target detection method comprises the following steps: laying out a roadside laser radar, and configuring corresponding roadside computing equipment for the roadside laser radar; calibrating external parameters of the laser radar at the road side; the road side computing equipment computes the relative position of the automatic driving vehicle relative to the road side laser radar according to the automatic driving vehicle positioning data and the road side laser radar external parameters; the road side computing equipment deflects the road side laser radar point cloud detected by the road side laser radar into an automatic driving vehicle coordinate system according to the relative position to obtain deflection point cloud; and the road side computing equipment performs voxelization processing on the deflection point cloud to obtain the voxelized deflection point cloud. The automatic driving vehicle performs voxelization treatment on the vehicle-mounted laser radar point cloud detected by the vehicle-mounted laser radar to obtain voxelized vehicle-mounted laser radar point cloud; and the road side computing equipment computes voxel level characteristics of the voxelized deflection point cloud to obtain the voxel level characteristics of the deflection point cloud. Calculating voxel-based vehicle-mounted laser radar point cloud voxel level characteristics by an automatic driving vehicle to obtain vehicle-mounted laser radar point cloud voxel level characteristics; and compressing and transmitting the cloud voxel level characteristics of each point to a computing device, wherein the transmitting device can be an automatic driving vehicle, a road side computing device or a cloud. The computing equipment performs data splicing and data aggregation on the vehicle-mounted laser radar point cloud voxel level characteristics and the deflection point cloud voxel level characteristics to obtain aggregate voxel level characteristics; the computing equipment inputs the aggregate voxel level characteristics into a three-dimensional target detection network model based on the voxel level characteristics to obtain a target detection result; and finally, sending the target detection result to the automatic driving vehicle when the computing equipment is the road side computing equipment or the cloud end.
Description
The invention belongs to the technical field of automatic driving vehicle-road cooperation, and relates to a vehicle-road cooperation target detection method using perceptual information fusion characterization.
In the 21 st century, along with the continuous development of urban roads and automobile industry, automobiles become one of the necessary transportation means for people to travel, and great convenience is brought to daily production and life of human beings. However, excessive use of the automobile brings problems of environmental pollution, traffic jam, traffic accidents and the like. In order to relieve the problem of excessive use of the automobile, people are separated from a traffic system, the driving capability of the automobile is improved, meanwhile, the hands of the driver are liberated, and the automobile is automatically driven to become an important direction for the development of the automobile in the future. With the rising of deep learning technology and the great attention of artificial intelligence, autopilot is also stir-fried as an important foothold of great attention in AI.
Autopilot is a complete software and hardware interaction system, and autopilot core technologies comprise hardware (automobile manufacturing technology and autopilot chips), autopilot software, high-precision maps, sensor communication networks and the like. From a software aspect, it can be generally divided into three modules, namely, environmental awareness, behavioral decision-making, and motion control.
The perception is the first ring of autopilot, the tie of vehicle and environment interactions. The overall performance of an autopilot system depends, first of all, on the performance of the perception system. Perception of an autonomous vehicle is achieved by a sensor, wherein a lidar uses a laser to detect and measure. The principle is that pulse laser is emitted to the surrounding, reflected back after encountering an object, and the distance is calculated through the back-and-forth time difference, so that a three-dimensional model is built for the surrounding environment. The laser radar has high detection precision and long distance; since the wavelength of the laser light is short, a very minute target can be detected, and the detection distance is long. The laser radar perceives the point cloud data with large information quantity and higher precision, and is mostly used for detecting and classifying targets in an automatic driving perception ring. On one hand, the laser radar overturns the traditional two-dimensional projection imaging mode, can acquire depth information of a target surface to obtain relatively complete space information of the target, reconstruct a three-dimensional surface of the target through data processing to obtain a three-dimensional figure which can reflect the geometric shape of the target, and can also acquire abundant characteristic information such as reflection characteristics, movement speed and the like of the target surface, so that sufficient information support is provided for data processing such as target detection, identification, tracking and the like, and algorithm difficulty is reduced; on the other hand, the application of the active laser technology ensures that the device has the characteristics of high measurement resolution, strong anti-interference capability, strong stealth resistance, strong penetrating capability and all-weather operation.
Currently, lidars are classified into mechanical lidars and solid-state lidars according to the presence or absence of mechanical components, and although solid-state lidars are considered to be the trend of the future, mechanical lidars still occupy the dominant position in the current lidar battlefield. While the mechanical laser radar is provided with a rotating component for controlling the laser emission angle, the solid-state laser radar does not need a mechanical rotating component and mainly depends on an electronic component to control the laser emission angle.
In the existing automatic driving scheme, the laser radar is basically the most important sensor in the environment sensing module, and takes on most tasks of real-time map establishment, positioning, target detection and the like in environment sensing. For example, five laser radars are added in the sensor configuration scheme of google Waymo, and four side laser radars are respectively distributed on the front, back, left and right sides of the vehicle, are middle-short distance multi-line radars and are used for supplementing blind area vision; the top is provided with a high-line number laser radar for large-scale perception, and the blind area of the visual field is supplemented by four side laser radars.
The scan data of the lidar sensor is recorded in the form of a point cloud. Point cloud data refers to a set of vectors in a three-dimensional coordinate system. These vectors are typically expressed in the form of X, Y, Z three-dimensional coordinates. Each point may contain color information (RGB) or reflectance Intensity information (Intensity) in addition to three-dimensional coordinates.
Wherein, X, Y, Z three columns of data represent three-dimensional positions of point data in a sensor coordinate system or a world coordinate system, and are generally expressed in meters. The Intensity column below represents the reflected Intensity of the laser light at each point, which has no units and is typically normalized to between 0 and 255.
Because the installation height of the vehicle-mounted laser radar is limited by the size of the vehicle type and is about two meters, the detected information is easy to be influenced by shielding objects around the vehicle, for example, a cargo truck running in front of the small-sized vehicle can almost completely shield the front view of the laser radar on the small-sized vehicle, so that the environment sensing capability of the vehicle-mounted laser radar is seriously weakened. In addition, the performance of the radar is limited by the overall cost of the vehicle, and the vehicle end is not provided with a relatively expensive high-line number laser radar. Therefore, the point cloud data obtained by the vehicle-mounted laser radar often has blind areas or sparse conditions, and the automatic driving perception task is difficult to complete only by means of the sensors of the vehicle. Compared with a vehicle-mounted laser radar, the laser radar installed at the road side facility end can be arranged on a higher portal frame or a lamp post, so that the laser radar has a more transparent visual field and is not easy to be blocked. In addition, the laser radar on the road side has higher tolerance to cost, the laser radar with higher line number can be used, and meanwhile, the calculation unit on the road side with higher calculation force can be configured, so that higher detection performance and higher detection speed can be achieved.
At present, the vehicle-road cooperative system is in the hot tide of research and test, and the intelligent vehicle-road cooperative scheme realized based on the V2X technology can enhance the auxiliary driving function realized at the present stage, enhance the vehicle driving safety and the road running efficiency, and can provide data service and technical support for automatic driving in a long term.
The existing laser radar vehicle-road cooperation scheme is that a vehicle and road side facilities respectively detect targets according to laser radar point cloud data, then facility ends send detection results to the vehicle, most of students focus on reliability analysis of transmission data, calculation of relative pose between two ends of a vehicle road or data transmission delay processing of two ends of the vehicle road, and the vehicle-road cooperation process is defaulted to directly send target detection results. Although the data transmission amount is low in this scheme, the detection data at both ends cannot be fully utilized. For example, when the two gears of the vehicle road do not detect the more complete target point cloud, the condition of missed detection and false detection easily occurs, so that the target detection result after cooperation is error. In this regard, some scholars propose to directly send origin cloud data to prevent information loss, for example, cooper framework proposed in 2019 proposes a cooperative sensing scheme of origin cloud data level at the earliest, and the sensing performance is greatly improved by fusing point cloud data of different sources.
However, at the same time, the size of single-frame laser radar point cloud data is often more than ten M and even tens of M, and the existing vehicle-road cooperative communication conditions are difficult to support such a large amount of real-time point cloud data transmission. Therefore, the automatic driving technology is urgent to need a better collaborative detection method using laser radar data at two ends, which not only meets the requirement of target detection precision, but also reduces the data transmission quantity as much as possible.
The existing target identification and classification algorithm based on laser radar point cloud data is based on a deep neural network technology.
Prior Art
Patent document US9562971B2
Patent document US20150187216A1
Patent document CN110989620a
Patent document CN110781927a
Patent document CN111222441a
Patent document CN108010360a
Disclosure of Invention
In order to solve the problems, the invention provides a vehicle-road cooperation-oriented perception information fusion characterization and target detection method, and provides a vehicle-road cooperation scheme based on laser radar point cloud data for balancing the size of transmission data and the degree of information loss, which is used for solving the problems of insufficient vehicle perception capability and insufficient vehicle-road cooperation communication bandwidth of an existing automatic driving vehicle.
The technical problems to be solved include determining a road side laser radar layout scheme, selecting a road side laser radar external parameter calibration method, calculating deflection parameters according to relative pose of an automatic driving vehicle and the road side laser radar, and determining a proper information representation form for vehicle-road coordination.
The invention aims at: and the information transmission quantity is reduced on the premise of ensuring the cooperative sensing capability of the vehicle and the road.
The technical problem to be solved by the invention is divided into a preparation stage and an application stage, wherein the preparation stage comprises the following steps:
A. laying out a roadside laser radar, and configuring corresponding roadside computing equipment for the roadside laser radar;
B. calibrating the external parameters of the laser radar at the road side.
The steps of the application stage are as follows:
C. the road side computing equipment computes the relative pose of the automatic driving vehicle relative to the road side laser radar according to the automatic driving vehicle positioning data and the road side laser radar external parameters;
D. and the road side computing equipment deflects the road side laser radar point cloud detected by the road side laser radar into an automatic driving vehicle coordinate system according to the relative pose, so as to obtain the deflection point cloud.
E. And the road side computing equipment performs voxelization processing on the deflection point cloud to obtain the voxelized deflection point cloud. The automatic driving vehicle performs voxelization treatment on the vehicle-mounted laser radar point cloud detected by the vehicle-mounted laser radar to obtain voxelized vehicle-mounted laser radar point cloud;
F. and the road side computing equipment computes voxel level characteristics of the voxelized deflection point cloud to obtain the voxel level characteristics of the deflection point cloud. Calculating voxel-based vehicle-mounted laser radar point cloud voxel level characteristics by an automatic driving vehicle to obtain vehicle-mounted laser radar point cloud voxel level characteristics;
The subsequent steps are divided into three sub-schemes I, II and III. The sub-scheme I completes the step G at the road side computing equipment 1 、H 1 、I 1 The method comprises the steps of carrying out a first treatment on the surface of the Sub-scenario II completion of step G in an autonomous vehicle 2 、H 2 、I 2 The method comprises the steps of carrying out a first treatment on the surface of the Step G is completed in cloud end by sub-scheme III 3 、H 3 、I 3 。
In sub-scheme I:
G 1 the automatic driving vehicle compresses the vehicle-mounted laser radar point cloud voxel level characteristics to obtain compressed vehicle-mounted laser radar point cloud voxel level characteristics, the compressed vehicle-mounted laser radar point cloud voxel level characteristics are transmitted to the road side computing equipment, the road side computing equipment receives the compressed vehicle-mounted laser radar point cloud voxel level characteristics, and the compressed vehicle-mounted laser radar point cloud voxel level characteristics are restored to vehicle-mounted laser radar point cloud voxel level characteristics;
H 1 the road side computing equipment performs data splicing and data aggregation on the vehicle-mounted laser radar point cloud voxel level characteristics and the deflection point cloud voxel level characteristics to obtain aggregate voxel level characteristics;
I 1 the road side computing equipment inputs the aggregate voxel level characteristics into a three-dimensional target detection network model based on the voxel level characteristics to obtain a target detection result, and transmits the target detection result to an automatic driving vehicle;
in sub-scheme II:
G 2 the road side computing equipment compresses the deflection point cloud voxel level characteristics to obtain compressed deflection point cloud voxel level characteristics, and transmits the compressed deflection point cloud voxel level characteristics to an automatic driving vehicle; autopilot The vehicle receives the compressed deflection point cloud voxel level characteristics and restores the compressed deflection point cloud voxel level characteristics to deflection point cloud voxel level characteristics;
H 2 the automatic driving vehicle performs data splicing and data aggregation on the vehicle-mounted laser radar point cloud voxel level characteristics and the deflection point cloud voxel level characteristics to obtain aggregate voxel level characteristics;
I 2 inputting the aggregate voxel level characteristics into a three-dimensional target detection network model based on the voxel level characteristics by the automatic driving vehicle to obtain a target detection result;
in sub-scheme III:
G 3 and the automatic driving vehicle compresses the vehicle-mounted laser radar point cloud voxel level characteristics to obtain compressed vehicle-mounted laser radar point cloud voxel level characteristics, and the compressed vehicle-mounted laser radar point cloud voxel level characteristics are transmitted to the cloud. And the road side computing equipment compresses the voxel level characteristics of the deflection point cloud to obtain the voxel level characteristics of the compressed deflection point cloud, and transmits the voxel level characteristics to the cloud. The cloud receives the compressed deflection point cloud voxel level characteristics and the compressed vehicle-mounted laser radar point cloud voxel level characteristics, restores the compressed deflection point cloud voxel level characteristics to deflection point cloud voxel level characteristics, and restores the compressed vehicle-mounted laser radar point cloud voxel level characteristics to vehicle-mounted laser radar point cloud voxel level characteristics;
H 3 the cloud end performs data splicing and data aggregation on the vehicle-mounted laser radar point cloud voxel level features and the deflection point cloud voxel level features to obtain aggregate voxel level features;
I 3 And the cloud inputs the aggregate voxel level characteristics into a three-dimensional target detection network model based on the voxel level characteristics to obtain a target detection result, and transmits the target detection result to the automatic driving vehicle.
The specific technical scheme in the steps of the invention is as follows:
A. laying laser radar
The laser radar on the road side is determined according to the existing road side upright post facilities in the cooperative scene of the vehicle and road and the type of the laser radar. The existing road side laser radar is installed in a vertical rod or cross rod mode, and the specific installation positions are on infrastructure columns with power support, such as road side portal frames, street lamps, signal lamp posts and the like.
According to whether rotating parts exist in the laser radar, the laser radar can be divided into a mechanical rotating laser radar, a hybrid laser radar and a solid-state laser radar, wherein the two types of the mechanical rotating laser radar and the solid-state laser radar are commonly used laser radar types at road sides.
And (3) for scenes such as intersections, arranging a road side laser radar with a detection range larger than or equal to the scene range or including a key area in the scene. For long-distance large-range complex scenes such as expressways, parks and the like, the following road side laser radar layout guidelines are recommended to be followed, so that the road side laser radar coverage meets the scene full coverage requirement, namely, a single road side laser radar realizes the supplement of detection blind areas below other road side laser radars in the scene, and a better vehicle-road cooperative target detection effect is achieved.
The road side laser radar layout guideline is divided into a road side mechanical rotation laser radar layout guideline and a road side all-solid-state laser radar layout guideline according to different types of the used road side laser radars.
A 1 ) Roadside mechanical rotary laser radar and roadside hybrid solid-state laser radar layout scheme
The mechanical rotary laser radar realizes laser scanning through mechanical rotation; the laser emission components are arranged into a laser light source linear array in the vertical direction, and can generate light beams with different angle directions in the vertical plane through the lens; the laser beam is driven by a motor to continuously rotate, namely, the light beam in the vertical plane is changed into a 'plane' from a 'line', and a plurality of laser 'planes' are formed through rotary scanning, so that detection in a detection area is realized. The hybrid solid-state laser radar uses a semiconductor micro-motion device (such as an MEMS scanning mirror) to replace a macro-mechanical scanner, and realizes a laser scanning mode of a radar transmitting end on a microscopic scale.
The road side mechanical rotation type laser radar and road side mixed solid-state laser radar layout guideline requires the road side to be horizontally arranged when the mechanical rotation type laser radar and the road side mixed solid-state laser radar are installed, so that the full utilization of the light beam information in all directions is ensured. As shown in fig. 2, the road side mechanical rotary laser radar and the road side hybrid solid-state laser radar should at least meet the following requirements:
Wherein:
H a representing the mounting height of the roadside mechanical rotary laser radar or the roadside hybrid solid-state laser radar;
representing the clamping angle between the highest elevation angle light beam and the horizontal direction of the roadside mechanical rotary laser radar or the roadside hybrid solid-state laser radar;
L a representing the distance between two adjacent road side mechanical rotary laser radar or road side hybrid solid-state laser radar mounting rod positions;
A 2 ) Road side all-solid-state laser radar layout scheme
The all-solid-state laser radar completely cancels a mechanical scanning structure, and the laser scanning in the horizontal direction and the vertical direction is realized in an electronic mode. The phase control laser transmitter is a rectangular array formed by a plurality of transmitting and receiving units, and the aim of adjusting the angle and the direction of the emitted laser can be achieved by changing the phase difference of the light rays emitted by different units in the array. The laser light source enters the optical waveguide array after passing through the optical beam splitter, the phase of the light wave is changed on the waveguide in an externally-controlled mode, and the light wave phase difference between the waveguides is utilized to realize light beam scanning.
As shown in fig. 3, the road-side all-solid-state laser radar layout guideline requires that the road-side all-solid-state laser radar be laid out to at least meet the following requirements:
wherein:
H b representing the mounting height of the road side all-solid-state laser radar;
The view field angle of the road side all-solid-state laser radar in the vertical direction is represented;
the included angle between the highest elevation angle light beam of the road side all-solid-state laser radar and the horizontal direction is shown;
L b representing the distance between two adjacent road side all-solid-state laser radar mounting rod positions;
for the scene of installing the all-solid-state laser radar, the method of installing two reverse laser radars by the same rod can also compensate the road side sensing blind area, reduce the requirement on the number of the road side positions, and meet the requirement as shown in fig. 4 at the moment, namely:
wherein:
H c representing the mounting height of the road side all-solid-state laser radar;
the included angle between the highest elevation angle light beam of the road side all-solid-state laser radar and the horizontal direction is shown;
L c representing the distance between two adjacent road side all-solid-state laser radar mounting rod positions;
for the laser radar vehicle-road cooperative scene which can meet the conditions, the road side mechanical rotary laser radar or the all-solid-state laser radar is arranged according to the requirements, and the laser radar scanning areas are increased when the conditions have allowance. And for the laser radar vehicle road cooperative scene which cannot meet the conditions, the road side laser radar layout conditions meet the road side laser radar layout guidelines by a method for layout new rods and the number of road side laser radars.
B. External parameter calibration
In order to calculate the relative pose of the road side laser radar and the vehicle-mounted laser radar, the installation position and the angle of the road side laser radar need to be calibrated, namely, external parameters are calibrated, and the coordinate position parameter and the angle pose parameter of the laser radar relative to a certain reference coordinate system are obtained. The external parameters of the lidar can be represented by the following vectors:
V 0 ＝[x 0 y 0 z 0 α 0 β 0 γ 0 ] (4)
wherein:
x 0 representing an X coordinate of the roadside laser radar in a reference coordinate system;
y 0 representing Y coordinates of the roadside laser radar in a reference coordinate system;
z 0 the Z coordinate of the roadside laser radar in a reference coordinate system is represented;
α 0 the rotation angle of the road side laser radar around the X axis in a reference coordinate system is represented;
β 0 the rotation angle of the road side laser radar around the Y axis in a reference coordinate system is represented;
γ 0 the rotation angle of the road side laser radar around the Z axis in a reference coordinate system is represented;
the reference coordinate system may be a latitude and longitude coordinate system represented by GCJ02 and WGS84, or a geodetic coordinate system based on a specific geographic point, for example, a beijing 54 coordinate system and a siean 80 coordinate system. Correspondingly, the actual coordinates of a point in the reference coordinate system and the coordinates in the road side laser radar coordinate system obtained after the detection by the laser radar have the following relations:
Wherein:
x lidar an X coordinate of the point in a roadside laser radar coordinate system;
y lidar the Y coordinate of the point in a roadside laser radar coordinate system is defined;
z lidar the Z coordinate of the point in a roadside laser radar coordinate system is obtained;
x real an X coordinate of the point in a reference coordinate system;
y real y coordinates of the point in a reference coordinate system;
z real z coordinate of the point in a reference coordinate system;
R x (α 0 )、R y (β 0 )、R z (γ 0 ) Is based on three angles of external reference alpha 0 、β 0 And gamma 0 A calculated sub-rotation matrix;
the external parameter specific value of the road side laser radar is obtained by measuring the coordinate calculation of the control point in the road side laser radar coordinate system and the reference coordinate system, and the method comprises the following steps:
(1) and selecting at least 4 reflectivity characteristic points as control points in the detection range of the roadside laser radar. The reflectivity characteristic points refer to points with obvious differences between reflectivity and surrounding objects, such as traffic signboards, license plates and the like, and the purpose of selecting the reflectivity characteristic points as control points is to conveniently and rapidly find out corresponding points in point cloud data according to differences between positions and reflection intensities and other points, so that the corresponding relation between the points in the multi-point cloud and one coordinate in a reference coordinate system is rapidly established. The control points should be distributed as discretely as possible. Under the condition that the scene environment allows and the control point is selected to meet the following requirements, the more the control points are, the better the calibration effect is. The control point selection requirements include: should be discretely distributed, and any three control points can not be collinear; in the detection range of the laser radar at the road side, the selected control point is farther away from the laser radar at the road side as far as possible, and the distance is usually greater than 50% of the furthest detection distance of the laser radar. For the situation that it is difficult to select the control points at 50% of the farthest detection distance of the laser radar due to scene limitation, the control points can be selected at less than 50% of the farthest detection distance of the laser radar, but the number of the control points should be increased.
(2) Measuring the accurate coordinates of control points by using high-precision measuring instruments such as a handheld high-precision RTK and the like, and finding corresponding point coordinates in the point cloud of the roadside laser radar; when the high-precision map file of the road side laser radar layout scene is held, the coordinates of the corresponding feature points can be found directly from the high-precision map without using high-precision measuring instruments such as handheld high-precision RTK and the like for measurement.
(3) Calculating laser radar external parameter vector V by using three-dimensional registration algorithm 0 The result is used as the calibration result. Common three-dimensional registration algorithms include an ICP algorithm, an NDT algorithm and the like, wherein the ICP algorithm is mainly used when the three-dimensional registration algorithm is applied to the laser radar external parameter calibration problem. The basic principle of the ICP algorithm is to calculate in a matched target point set P (the coordinate set of a control point in a roadside laser radar coordinate system) and a source point set Q (the coordinate set of a control point in a reference coordinate system)And outputting the optimal matching external parameters so as to minimize an error function. The error function is:
R＝R x (α 0 )R y (β 0 )R z (γ 0 ) (10)
T＝[x 0 y 0 z 0 ] T (11)
wherein:
e (R, T) is a target error function;
r is a rotation transformation matrix;
t is a translation transformation matrix;
n is the number of nearest point pairs in the point set;
p i coordinates of an ith point in the target point set P;
q i for the source point set Q and the point p i The points that make up the nearest pair of points;
C. computing relative poses
And determining the relative pose of the automatic driving vehicle and the road side laser radar according to the automatic driving vehicle positioning data and the road side laser radar external parameter calibration result in the early-stage preparation work. The relative pose is calculated according to the following formula:
V′＝[V′ xyz V′ αβγ ] (12)
V′ αβγ ＝[α′ β′ γ′] T ＝[α′ 1 β′ 1 γ′ 1 ] T -[α′ 0 β′ 0 γ′ 0 ] T (14)
V 1 ＝[x 1 y 1 z 1 α 1 β 1 γ 1 ] T (15)
wherein:
v' is the position and angle vector of the autonomous vehicle relative to the roadside lidar
V′ xyz Position vector for an autonomous vehicle relative to a roadside lidar
V′ αβγ Angular vector for an autonomous vehicle relative to a roadside lidar
V 1 Position and angle vector in a reference coordinate system for an autonomous vehicle
D. Deflection of
The point cloud D of the laser radar on the road side is calculated according to the following formula r Yaw into the autonomous vehicle coordinate system:
R＝R x (α′)R y (β′)R z (γ′) (18)
T＝[x′ y′ z′] (19)
wherein:
H rc a transformation matrix for deflecting the roadside lidar coordinate system to an autonomous vehicle coordinate system;
x ego 、y ego 、z ego for the coordinates of one point in the point cloud of the roadside laser radar after deflecting to the coordinate system of the automatic driving vehicle, the coordinates of the roadside laser radar are correspondingThe coordinates of the points in the system are [ x ] lidar y lidar z lidar ] T ；
O is perspective transformation vector, and O takes 0 0 as no perspective transformation exists in the scene;
E. voxelization of
A voxel is an abbreviation of Volume element (voxel) and is the smallest unit of digital data on three-dimensional space division. Conceptually resembles the smallest unit of two-dimensional space, a pixel. After the point cloud data are segmented by using the voxels, the data features of the point cloud data in each voxel can be calculated once respectively, and the feature of the set formed by the point cloud data in each voxel is called a voxel level feature. One of the major algorithms in the existing three-dimensional target detection algorithm processes laser radar point cloud data based on voxel level characteristics, extracts voxel level characteristics after voxelized point cloud data, and inputs a follow-up three-dimensional target detection network model based on the voxel level characteristics to obtain a target detection result.
The step of voxelization of the point cloud data is as follows:
E 1 ) According to the vehicle-mounted laser radar point cloud D c Size of the spatial dimension where [ DWH ]]The size of the design voxel is [ D ] V W V H V ]. And carrying out voxel division on the vehicle-mounted laser radar according to the designed voxel size.
E 2 ) For a deflection point cloudPoint cloud D of using and vehicle-mounted laser radar c The same voxel division mode is used for division, so that the deflection point cloud is ensured to be dividedIs a space grid and vehicle-mounted laser radar point cloud D c And completely overlap. For example, vehicle-mounted lidar point cloud D c The distribution space of (C) is [ -31m,33m in X-axis direction]Voxel D V 4m, in this case, if the point cloud is deflectedThe distribution space of (C) is [ -32m,34m in X-axis direction]It should be extended to [ -35m,37m]Obtaining an expanded deflection point cloudTo ensure the point cloud D of the vehicle-mounted laser radar c And an expanded deflection point cloudIs consistent with the voxel division grid of (a). The specific calculation formula is as follows:
n 1 ，n 2 ∈N
wherein:
S ego is the vehicle-mounted laser radar point cloud D c The spatial range;
S lidar ' is an extended deflection point cloudSpace in whichA range;
K lidar_start ′、K lidar_end ' is the extended deflection point cloud in the K dimensionA range start value and a range end value of (a);
K lidar_start 、K lidar_end for a deflection point cloud in the K dimensionA range start value and a range end value of (a);
K ego_start 、K ego_end for the vehicle-mounted laser radar point cloud D in the K dimension c A range start value and a range end value of (a);
V K is the size of the voxel in the K dimension;
E 3 ) According to the vehicle-mounted laser radar point cloud D c And an expanded deflection point cloudThe voxels in which the scatter data are located are grouped, and the scatter data in the same voxel are in the same group. Because of the non-uniformity and sparsity of points, the amount of scatter data in each voxel is not necessarily the same, and there may be no scatter data in a portion of the voxels.
E 4 ) In order to reduce the calculation burden and eliminate the discrimination problem caused by inconsistent density, the voxels with the scattered data quantity larger than a certain threshold value in the voxels are randomly sampled, the threshold value recommended to be 35 is obtained, and when the scattered data in the point cloud data is less, the threshold value can be properly reduced. This strategy may save computational resources, reducing the imbalance between voxels.
Through step E 1 ～E 4 Voxelized vehicle-mounted laser radar point cloud D c Obtaining voxelized vehicle-mounted laser radar point cloudVoxel expanded deflection point cloudObtaining voxelized deflection point cloud
F. Computing voxel level features
The method used to calculate the point cloud voxel level features varies according to the target detection model used by the autonomous vehicle. Taking the example of target detection by using a VoxelNet model by an automatic driving vehicle, the steps are as follows:
(1) Firstly, organizing the voxelized point cloud, and for the ith point in the voxel A, acquiring the original data as follows:
a i ＝[x i y i z i r i ] (23)
wherein:
x i 、y i 、z i x, Y, Z coordinates of the i-th point respectively;
r i the reflection intensity for the i-th point;
(2) the mean of the coordinates of all points within the voxel is then calculated and noted as v x v y v z ]。
(3) The information is then supplemented for the i-th point with an offset from the center, namely:
wherein:
information of the ith point after supplementing;
(4) the processed voxelized point cloud is input into a cascade of continuous VFE layers, and a schematic diagram of the VFE layers processing data of the voxelized point cloud is shown in fig. 5. The processing logic of the VFE layer is such that each is first caused toAnd obtaining point-level characteristics of each point through a layer of fully-connected network, carrying out maximum value pooling treatment on the point-level characteristics to obtain voxel-level characteristics, and finally splicing the voxel-level characteristics with the point-level characteristics obtained in the last step to obtain a point-level spliced characteristic result.
(5) After the cascade continuous VFE layer processing, final voxel level characteristics are obtained through full-connection layer integration and maximum value pooling, and each voxel level characteristic is a vector with 1 XC dimension.
Voxel-based vehicle-mounted laser radar point cloudAnd voxelized deflection point cloudThe method can be used for respectively obtaining the vehicle-mounted laser radar point cloud voxel level characteristics And deflection point cloud voxel level features
G. Point cloud voxel level feature transmission
Since the point cloud is sparsely present in space, there are no scatter points within many voxels, and therefore there are no corresponding voxel-level features. The data size can be greatly compressed after the point cloud voxel level characteristics are stored by a special structure, so that the transmission difficulty when the point cloud voxel level characteristics are sent to processing equipment is reduced, namely the point cloud voxel level characteristics are compressed. One of the special structures that can be used is a hash table, which is a data structure that is directly accessed according to a key value. It accesses records by mapping key values to a location in the table to speed up the lookup. The hash key of the hash table is the space coordinate of the voxel, and the corresponding value is the voxel level characteristic.
When using sub-scheme I, the subsequent processing occurs at the roadside computing device.
G 1 ) Automatic driving vehicle-to-vehicle laser radar point cloud voxel level characteristicCompression processing is carried out to obtain the voxel level characteristics of the point cloud of the compressed vehicle-mounted laser radarAnd transmitting the compressed vehicle-mounted laser radar point cloud voxel level characteristics to road side computing equipment, wherein the road side computing equipment receives the compressed vehicle-mounted laser radar point cloud voxel level characteristicsCompressing vehicle-mounted laser radar point cloud voxel level characteristicsRestoring to vehicle-mounted laser radar point cloud voxel level characteristics
When using sub-scenario II, the subsequent processing is performed on an autonomous vehicle.
G 2 ) Deflection point cloud voxel level characteristics by road side computing equipmentPerforming compression processing to obtain the voxel-level characteristics of the compressed deflection point cloudAnd transmitted to the autonomous vehicle; automated driving vehicle receiving compressed deflection point cloud voxel level featuresCompressing the voxel level characteristics of the deflection point cloudReduction to deflection point cloud voxel level features
When using sub-scheme III, the subsequent processing is performed at the cloud.
G 3 ) Automatic driving vehicle-to-vehicle laser radar point cloud voxel level characteristicCompression processing is carried out to obtain the voxel level characteristics of the point cloud of the compressed vehicle-mounted laser radarAnd transmitted to the cloud. Deflection point cloud voxel level characteristics by road side computing equipmentPerforming compression processing to obtain the voxel-level characteristics of the compressed deflection point cloudAnd transmitted to the cloud. Cloud-end receiving compression deflection point cloud voxel level characteristicsAnd compressing vehicle-mounted laser radar point cloud voxel level characteristicsCompressing the voxel level characteristics of the deflection point cloudReduction to deflection point cloud voxel level featuresCompressing vehicle-mounted laser radar point cloud voxel level characteristicsRestoring to vehicle-mounted laser radar point cloud voxel level characteristics
H. Data stitching and data aggregation
Performing data splicing operation, namely, carrying out point cloud voxel level characteristics on the vehicle-mounted laser radar And deflection point cloud voxel level featuresAlignment is performed according to the position of the voxels therein in the autonomous vehicle coordinate system.
Performing data aggregation operation, namely, for the voxel level characteristics of the point cloud of the vehicle-mounted laser radarAnd deflection point cloud voxel level featuresAnd taking the voxel level characteristic of one of the voxels which is not empty as the aggregated voxel level characteristic. And for voxels which are not empty with both sides, calculating the final obtained aggregate voxel level characteristics according to the following formula:
wherein:
for aggregating voxel level features;
f k to aggregate voxel level featuresThe value at position k;
f ego_k is the voxel level characteristic of the point cloud of the vehicle-mounted laser radarThe value at position k;
f lidar_k voxel level characterization for deflection point cloudThe value at position k;
i.e. the features of voxels of the same coordinates are aggregated using a maximum pooling approach.
When using sub-scheme I, the subsequent processing occurs at the roadside computing device.
H 1 ) The road side computing equipment performs voxel level characteristic on the vehicle-mounted laser radar point cloud according to the methodAnd deflection point cloud voxel level featuresPerforming data splicing and data aggregation to obtain aggregate voxel level characteristics
When using sub-scenario II, the subsequent processing is performed on an autonomous vehicle.
H 2 ) The automatic driving vehicle is characterized by the vehicle-mounted laser radar point cloud voxel level according to the method Deflection ofPoint cloud voxel level featuresPerforming data splicing and data aggregation to obtain aggregate voxel level characteristics
When using sub-scheme III, the subsequent processing is performed at the cloud.
H 3 ) The cloud end performs voxel level characteristics on the vehicle-mounted laser radar point cloud according to the methodAnd deflection point cloud voxel level featuresPerforming data splicing and data aggregation to obtain aggregate voxel level characteristics
I. Target detection
And inputting the aggregate voxel level characteristics into a subsequent three-dimensional target detection network model to obtain a detection target. Taking VoxelNet as an example, after the aggregate voxel level feature is obtained, the aggregate voxel level feature is input into a three-dimensional target detection network model based on the voxel level feature to obtain a target detection result.
The target detection result may be represented as U, specifically:
U＝[u 1 ... u n ] (27)
wherein:
u i information of an ith target in the target detection result;
x i an x-axis coordinate in an autonomous vehicle coordinate system for an ith detection target;
y i the y-axis coordinate of the ith detection target in the automatic driving vehicle coordinate system;
z i the z-axis coordinate of the ith detection target in the automatic driving vehicle coordinate system;
C i confidence for the ith detection target;
W i the width of the detection frame corresponding to the ith detection target;
D i the length of the detection frame corresponding to the ith detection target;
H i The height of the detection frame corresponding to the ith detection target;
the direction angle of the detection frame corresponding to the ith detection target;
v xi the projection of the target movement speed in the x-axis direction in the coordinate system of the autonomous vehicle is detected for the ith.
v yi The projection of the target movement speed in the y-axis direction in the coordinate system of the autonomous vehicle is detected for the ith.
v zi The projection of the target movement speed in the z-axis direction in the coordinate system of the autonomous vehicle is detected for the ith.
For any three-dimensional object detection network model based on voxel level characteristics, the object detection result at least comprises the position of an object, namely x i 、y i 、z i . For a high-performance three-dimensional object detection network model based on voxel level characteristics, the object detection result comprises C of a detection object i 、W i 、D i 、H i 、 v xi 、v yi 、v zi Some or all of the attributes. Wherein W is i 、D i 、H i Three attributes may or may not be present in the target detection result at the same time. v xi 、v yi 、v zi Three attributes may or may not be present in the target detection result at the same time.
Using sub-scheme I, target detection occurs at the roadside computing device.
I 1 ) The roadside computing device will aggregate voxel level featuresAnd inputting a three-dimensional target detection network model based on the voxel level characteristics to obtain a target detection result U, and transmitting the target detection result to the automatic driving vehicle.
Using sub-scenario II, the target detection is performed on an autonomous vehicle.
I 2 ) Automated vehicles will aggregate voxel level featuresAnd inputting a three-dimensional target detection network model based on the voxel level characteristics to obtain a target detection result U.
When using sub-scheme III, target detection is performed at the cloud.
I 3 ) Cloud will aggregate voxel level featuresInputting three-dimensional based on voxel level characteristicsThe target detection network model obtains a target detection result and transmits the target detection result U to the automatic driving vehicle.
The invention has the technical key points and advantages that:
the road side laser radar is used as a supplement for sensing the automatic driving vehicle, so that the range and accuracy of the automatic driving vehicle for identifying surrounding objects are improved. Meanwhile, voxel characteristics are used as data transmitted between the roads, so that the original data information is hardly lost, and the requirement on bandwidth in data transmission is reduced.
The above symbols and their meaning are summarized in the following table:
the above terms and their meaning are summarized in the following table:
brief description of the drawings
FIG. 1 is a presented vehicle-road collaboration oriented perception information fusion characterization and target detection method;
FIG. 2 is a schematic diagram of a mechanical rotary lidar for road sides
FIG. 3 is a schematic diagram of an all-solid-state laser radar for road side layout
FIG. 4 is a schematic view of an all-solid-state laser radar (two reverse laser radars mounted on the same pole)
FIG. 5 is a schematic view of the VFE layer processing point cloud data
FIG. 6 is a schematic diagram of voxel feature extraction and aggregation
FIG. 7 is a schematic diagram of merged voxel point cloud target detection
FIG. 8 is a schematic diagram of point cloud coordinate transformation of a roadside lidar
FIG. 9 is a schematic diagram showing the comparison of target detection results (left diagram shows the method for collaborative detection of a vehicle and road according to the present patent, and right diagram shows the result of direct detection of each target with high confidence coefficient)
The invention is described in detail below with reference to the drawings and the detailed description.
The invention relates to a vehicle-road cooperation-oriented perception information fusion characterization and target detection method. The method can be divided into three main steps:
first, the installation and early calibration of the roadside laser radar sensor are performed.
The road side laser radar is arranged according to the existing road side upright post facilities in the cooperative scene of the vehicle and road and the type of the laser radar. The existing road side laser radar is installed in a vertical rod or cross rod mode, and the specific installation positions are on infrastructure columns with power support, such as road side portal frames, street lamps, signal lamp posts and the like.
And (3) for scenes such as intersections, arranging a road side laser radar with a detection range larger than or equal to the scene range or including a key area in the scene. For long-distance large-range complex scenes such as expressways, parks and the like, the proposal follows the road side laser radar layout guidance rules in the invention content, so that the coverage range of the road side laser radar meets the full-coverage requirement of the scene, namely, a single road side laser radar realizes the supplement of detection blind areas under other road side laser radars in the scene, so as to achieve better detection effect of the cooperative targets of the vehicle and the road. In the vehicle-road cooperative scheme, a road-side laser radar is used for improving the perception capability of an automatic driving vehicle, namely the capability of obtaining information of objects around the vehicle relative to the vehicle, the type, the size, the travelling direction and the like of the vehicle. Therefore, the road side laser radar itself should have the highest possible perception capability, and parameters such as the number of radar lines and sampling frequency should not be lower than relevant parameters of the vehicle-mounted laser radar as much as possible. In addition, in order to make up the defect that the vehicle-mounted laser radar is easy to be shielded and realize the perception data redundancy, the perception range of the road side laser radar should ensure to cover all the frequent areas of shielding phenomena, and the road side laser radar is controlled to detect that the sight is transparent and has no obstacle shielding.
After the installation of the road side laser radar sensor is completed, in order to calculate the relative pose of the road side laser radar and the vehicle-mounted laser radar, the installation position and the angle of the road side laser radar are required to be calibrated, namely, external parameters are calibrated, and then the coordinate position parameter and the angle pose parameter of the laser radar relative to a certain reference coordinate system are obtained. Firstly, selecting at least 4 reflectivity characteristic points in a roadside laser radar detection area as control points. The reflectivity characteristic points refer to points with obvious differences between reflectivity and surrounding objects, such as traffic signboards, license plates and the like, and the purpose of selecting the reflectivity characteristic points as control points is to conveniently and rapidly find out corresponding points in point cloud data according to differences between positions and reflection intensities and other points, so that the corresponding relation between the points in the multi-point cloud and one coordinate in a reference coordinate system is rapidly established. The control points should be distributed as discretely as possible. Under the condition that the scene environment allows and the control point is selected to meet the following requirements, the more the control points are, the better the calibration effect is. The control point selection requirements include: should be discretely distributed, and any three control points can not be collinear; in the range of the detection of the laser radar at the road side, the selected control point is farther away from the laser radar at the road side as far as possible, and the distance is usually greater than 50% of the furthest detection distance of the laser radar. For the situation that it is difficult to select the control points at 50% of the farthest detection distance of the laser radar due to scene limitation, the control points can be selected at less than 50% of the farthest detection distance of the laser radar, but the number of the control points should be increased. Then, measuring the accurate coordinates of the control points by using high-precision measuring instruments such as a handheld high-precision RTK and the like, and finding out corresponding point coordinates in the point cloud of the laser radar at the road side; when the high-precision map file of the road side laser radar layout scene is held, the coordinates of the corresponding feature points can be found directly from the high-precision map without using high-precision measuring instruments such as handheld high-precision RTK and the like for measurement. And finally, calculating an optimal value of the laser radar external parameter vector by using a three-dimensional registration algorithm, and taking the result as a calibration result. Common three-dimensional registration algorithms include an ICP algorithm, an NDT algorithm and the like, wherein the ICP algorithm is mainly used when the three-dimensional registration algorithm is applied to the laser radar external parameter calibration problem. The basic principle of the ICP algorithm is that in a matched target point set P (a coordinate set of a control point in a roadside laser radar coordinate system) and a source point set Q (a coordinate set of a control point in a reference coordinate system), optimal matching external parameters are calculated, and an error function is minimized.
The method used to calibrate the roadside lidar external parameters is not limited here, but it should be ensured that the calibration results include the three-dimensional world coordinates of the sensor as well as pitch, yaw and roll angles for point cloud deflection in subsequent steps.
And secondly, processing and extracting characteristics of laser radar point cloud data at a vehicle end.
In the actual vehicle-road cooperative automatic driving process, real-time world coordinates, pitch angle, yaw angle and roll angle of the vehicle are obtained firstly based on an automatic driving self-contained positioning module and the like. Based on the vehicle RTK positioning result and the external parameter calibration result of the road side laser radar, calculating the relative pose of the automatic driving vehicle relative to the road side laser radar, and deflecting the road side laser radar point cloud data into a vehicle coordinate system.
And designing the size of voxels according to the size of the space dimension of the point cloud of the vehicle-mounted laser radar and carrying out voxel division on the vehicle-mounted laser radar. And for the deflection point cloud, the same voxel division mode as the vehicle-mounted laser radar point cloud is used for division, so that the space grid for dividing the deflection point cloud is ensured to completely coincide with the vehicle-mounted laser radar point cloud. Grouping according to voxels where scattered point data in the vehicle-mounted laser radar point cloud and the expanded deflection point cloud are located, wherein the scattered point data in the same voxels are in the same group. Because of the non-uniformity and sparsity of points, the amount of scatter data in each voxel is not necessarily the same, and there may be no scatter data in a portion of the voxels. In order to reduce the calculation burden and eliminate the discrimination problem caused by inconsistent density, the voxels with the scattered data quantity larger than a certain threshold value in the voxels are randomly sampled, the threshold value recommended to be 35 is obtained, and when the scattered data in the point cloud data is less, the threshold value can be properly reduced. This strategy may save computational resources, reducing the imbalance between voxels. Referring to fig. 6, two sets of point cloud data are divided into a plurality of discrete voxels by using a lattice of a fixed size, and the feature vector of each voxel is calculated by using the above-mentioned voxelization method. Taking a more classical VoxelNet network model in a three-dimensional target detection algorithm as an example, a plurality of continuous VFE layers are used for extracting the feature vector of each voxel. That is, the offset of each scattered point data in the voxel relative to the center is used to supplement the system information, and the processed voxelized point cloud is input into a cascade of continuous VFE layers, and a schematic diagram of the VFE layer processing the data of the voxelized point cloud is shown in fig. 5. The processing logic of the VFE layer is that firstly, each expanded scattered point data is enabled to pass through a layer of full-connection network to obtain point level characteristics of each point, then, the point level characteristics are subjected to maximum value pooling processing to obtain voxel level characteristics, and finally, the voxel level characteristics and the point level characteristics obtained in the last step are spliced to obtain point level splicing characteristic results. After the cascade continuous VFE layer treatment, the final voxel level characteristic is obtained through full-connection layer integration and maximum value pooling.
Since the point cloud is sparsely present in space, there are no scatter points within many voxels, and therefore there are no corresponding voxel-level features. The data size can be greatly compressed after the point cloud voxel level characteristics are stored by a special structure, so that the transmission difficulty when the point cloud voxel level characteristics are sent to processing equipment is reduced. One of the special structures that can be used is a hash table, which is a data structure that is directly accessed according to a key value. It accesses records by mapping key values to a location in the table to speed up the lookup. The hash key of the hash table is the space coordinate of the voxel, and the corresponding value is the voxel level characteristic.
And thirdly, carrying out data splicing and data aggregation on the vehicle-mounted laser radar point cloud voxel level characteristics and the deflection point cloud voxel level characteristics to obtain aggregate voxel level characteristics, and carrying out target detection.
Before data aggregation and data stitching, point cloud voxel level features first need to be compressed and transmitted to a computing device. The computing device may be a roadside computing device, an autonomous vehicle, or a cloud. When the sub-scheme I is used, data aggregation and data splicing and subsequent processing are performed on the road side computing equipment; when using sub-scheme II, data aggregation and data stitching and subsequent processing are performed on an autonomous vehicle. When the sub-scheme III is used, data aggregation and data splicing and subsequent processing are carried out on the cloud.
In the data splicing and data aggregation processes, the spatial relative position of the point cloud is not changed by voxelization, so that the vehicle-mounted Lei Dadian cloud voxel level features can be supplemented according to the deflection point cloud voxel level features in the last step, and data splicing operation is carried out, namely the vehicle-mounted laser radar point cloud voxel level features and deflection point cloud voxel level features are aligned according to the positions of the voxels in an automatic driving vehicle coordinate system. And performing data aggregation operation, namely taking the voxel level characteristic of one of the vehicle-mounted laser radar point cloud voxel level characteristic and the deflection point cloud voxel level characteristic as the aggregated voxel level characteristic for the position where any voxel of the vehicle-mounted laser radar point cloud voxel level characteristic is empty. For voxel-level feature vectors with the same space coordinates in the two groups of data, a maximum value pooling method is used for aggregating the feature vectors, and for non-coincident voxel-level feature vectors, feature vector values of a non-empty voxel side are kept.
And inputting the aggregate voxel level characteristics into a subsequent three-dimensional target detection network model to obtain a detection target. Referring to fig. 7, taking the VoxelNet network model as an example, the spliced data is input into a continuous convolution layer in the VoxelNet network model to obtain a space feature map, and finally is input into an RPN (Region Proposal Network, regional generation network) in the VoxelNet network model to obtain a final target detection result.
The invention has the following technical key points and advantages:
the road side laser radar is used as a supplement for sensing the automatic driving vehicle, so that the range and accuracy of the vehicle for identifying surrounding objects are improved. Meanwhile, the point cloud voxelization characteristic is used as data transmitted between the roads, so that the original data information is hardly lost, and the requirement of data transmission on bandwidth is reduced.
Setting an experimental scene at the intersection of the traffic engineering college in the Jia-ding district of the university of the same university, wherein a vertical rod with the height of 6.4m is arranged on each 20m distance in the scene. An Innovusion leopard array type 300-line laser radar and an auster 128-line 360-degree laser radar are used as examples of the roadside laser radar. The angle of view of the Innovuse Jaguar array type 300-line laser radar in the vertical direction is 40 degrees, and the furthest detection distance is 200m. The vertical field angle of the Ouster 128 line 360-degree laser radar is 45 degrees, and the furthest detection distance is 140m. The automatic driving vehicle uses an Ouster 64-line 360-degree radar as a vehicle-mounted laser radar, and is horizontally mounted at a mounting height of 2 m. The vehicle-mounted laser radar is rigidly connected with the vehicle body, the relative posture and displacement between the vehicle-mounted laser radar and the vehicle body are kept unchanged, calibration is completed when the vehicle leaves a factory, and the position and angle of the vehicle-mounted laser radar are corrected in real time according to real-time displacement and deflection of the vehicle measured by the vehicle-mounted RTK when the vehicle moves.
Example 1 is as follows:
(1) Layout and calibration of roadside laser radar sensors
Only using the Ouster 128 line 360-degree laser radar, considering the size of the laser radar, the installation height of the Ouster 128 line 360-degree laser radar is 6.5m, and one Ouster 128 line 360-degree laser radar is installed between every 5 upright posts, and at the moment, the design rules of the road side mechanical rotary laser radar and the road side hybrid solid-state laser radar are met.
Six reflectivity characteristic points are selected as control points in the laser radar area, and the six control points respectively take column feet of the columns at the two sides of the positions 80m, 100m and 120m away from the laser radar installation column. Because the road section has a certain curvature, any three control points meet the non-collineation condition. And measuring the accurate coordinates of the control points by using a handheld RTK, matching the coordinates of the corresponding control points in the laser radar point cloud, and calibrating the laser radar by using an ICP algorithm.
(2) And (5) processing point cloud data and extracting features.
The position of the road side laser radar point cloud in the automatic driving vehicle coordinate system can be obtained through the calibration work of the step (1), and the road side laser radar point cloud is aligned to the automatic driving vehicle coordinate system as shown in fig. 8. Dividing the deflection point cloud into voxels according to an automatic driving vehicle coordinate system and a lattice with a fixed size of [0.4m 0.5m ] and expanding to obtain the voxelized deflection point cloud. And supplementing voxel mean value information for each piece of scattered data in the voxelized deflection point cloud, inputting the voxel level information into the multi-layer VFE to calculate voxel level characteristics, and finally, calculating the voxels which do not contain the scattered data, wherein each voxel is represented by a 128-dimensional characteristic vector. The road side computing equipment stores the computed voxel level characteristics in a hash table, the spatial position of the voxels is used as a hash key, and the corresponding content is the voxel level characteristics of the corresponding voxels, so that the compressed deflection point cloud voxel level characteristics are obtained. And the automatic driving vehicle performs the same processing on the vehicle-mounted laser radar point cloud until the vehicle-mounted laser radar point cloud voxel level characteristics are acquired, namely a hash table is not required to be established for vehicle-end laser radar point cloud data. At this time, the data size is reduced to about 1/10 as compared with the original point cloud data.
(3) Voxel-level feature data stitching, data aggregation, and object detection
And the automatic driving vehicle receives the compressed deflection point cloud voxel level characteristics sent by the road side computing equipment, and decompresses the compressed deflection point cloud voxel level characteristics to restore the deflection point cloud voxel level characteristics. Because the coordinate system of the received deflection point cloud voxel level characteristics is deflected to an automatic driving vehicle coordinate system, the deflection point cloud voxel level characteristics can be directly spliced with the vehicle-mounted laser radar point cloud voxel level characteristic data of the same coordinate system. Data aggregation operations are performed on the same coordinate voxel-level features using a maximum pooling approach, e.g., the aggregate result of the voxel-level features [15,45,90, … …,17] and voxel-level features [8,17,110, … …,43] is [15,45,110, … …,43]. And after data splicing and data aggregation of all voxel level characteristics are completed, inputting the data into a subsequent RPN to obtain a target detection result. The vehicle-road collaborative detection method and the direct fusion are respectively drawn on a point cloud top view as shown in fig. 9 based on target detection results and confidence of the vehicle-mounted laser radar point cloud and the road-side laser radar point cloud. Therefore, the method for sharing the neural network features is used for carrying out the cooperative target detection of the road, so that the target detection precision can be greatly improved, and the data transmission bandwidth requirement can be reduced.
Example 2 is as follows:
(1) Layout and calibration of roadside laser radar sensors
When only Innovuse Jaguar array type 300-line laser radars are used and only one laser radar is arranged on each rod, the laser radar installation height is 6.5m, the depression angle is 7 degrees, and one laser radar is installed between every 8 upright rods, so that the method accords with a road side all-solid-state laser radar layout scheme.
Six reflectivity characteristic points are selected as control points in the laser radar area, and the six control points respectively take column feet of the columns at the positions 100m, 120m and 140m away from the laser radar installation columns. Because the road section has a certain curvature, any three control points meet the non-collineation condition. And measuring the accurate coordinates of the control points by using a handheld RTK, matching the coordinates of the corresponding control points in the laser radar point cloud, and calibrating the laser radar by using an ICP algorithm.
(2) And (5) processing point cloud data and extracting features.
And (3) obtaining the voxel level characteristics of the deflection point cloud and the voxel level characteristics of the vehicle-mounted laser radar point cloud in the same step (2) in the embodiment 1. And the automatic driving automobile stores the calculated voxel level characteristics of the vehicle-mounted laser radar point cloud in a hash table, the spatial position of the voxels is used as a hash key, and the corresponding content is the voxel level characteristics of the corresponding voxels, so that the compressed vehicle-mounted laser radar point cloud voxel level characteristics are obtained.
(3) Voxel-level feature data stitching, data aggregation, and object detection
The road side computing equipment receives the compressed vehicle-mounted laser radar point cloud voxel level characteristics sent by the automatic driving vehicle and decompresses the compressed vehicle-mounted laser radar point cloud voxel level characteristics to restore the vehicle-mounted laser radar point cloud voxel level characteristics. The subsequent steps of data splicing, data aggregation and target detection are the same as in (3) of embodiment 1, until the target detection result is obtained, and the road side computing device sends the target detection result to the automatic driving vehicle.
Example 3 is as follows:
(1) Layout and calibration of roadside laser radar sensors
When only Innovuse Jaguar array type 300-line laser radar is used and two reverse laser radars are arranged on each rod, the installation height of the laser radar is 6.5m, the depression angle is 7 degrees, two laser radars are arranged between every 9 upright rods, and the method meets the guideline of a road side all-solid-state laser radar arrangement scheme.
Six reflectivity characteristic points are selected as control points in the laser radar area, and the six control points respectively take column feet of the columns at the positions 100m, 120m and 140m away from the laser radar installation columns. Because the road section has a certain curvature, any three control points meet the non-collineation condition. And measuring the accurate coordinates of the control points by using a handheld RTK, matching the coordinates of the corresponding control points in the laser radar point cloud, and calibrating the laser radar by using an ICP algorithm.
(2) And (5) processing point cloud data and extracting features.
The compressed deflection point cloud voxel level characteristics are obtained in the same step (2) in the embodiment 1, and the compressed vehicle-mounted laser radar point cloud voxel level characteristics are obtained in the same step (2) in the embodiment 2.
(3) Voxel-level feature data stitching, data aggregation, and object detection
The cloud receives the compressed vehicle-mounted laser radar point cloud voxel level characteristics sent by the automatic driving vehicle, and decompresses the compressed vehicle-mounted laser radar point cloud voxel level characteristics to restore the vehicle-mounted laser radar point cloud voxel level characteristics; and the cloud receives the compressed deflection point cloud voxel level characteristics sent by the road side computing equipment, and decompresses the compressed deflection point cloud voxel level characteristics to restore the compressed deflection point cloud voxel level characteristics to deflection point cloud voxel level characteristics. The subsequent steps of data splicing, data aggregation and target detection are the same as in (3) of embodiment 1, until the target detection result is obtained, and the cloud end sends the target detection result to the automatic driving vehicle.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the claims.
Claims (9)
- A vehicle-road cooperation-oriented perception information fusion characterization and target detection method comprises the following steps:the preparation stage:A. laying out a roadside laser radar, and configuring corresponding roadside computing equipment for the roadside laser radar;B. calibrating external parameters of the laser radar at the road side;the application stage comprises the following steps:C. the road side computing equipment computes the relative pose of the automatic driving vehicle relative to the road side laser radar according to the automatic driving vehicle positioning data and the road side laser radar external parameters;D. the road side computing equipment deflects the road side laser radar point cloud detected by the road side laser radar into an automatic driving vehicle coordinate system according to the relative pose to obtain deflection point cloud;E. and the road side computing equipment performs voxelization processing on the deflection point cloud to obtain the voxelized deflection point cloud. The automatic driving vehicle performs voxelization treatment on the vehicle-mounted laser radar point cloud detected by the vehicle-mounted laser radar to obtain voxelized vehicle-mounted laser radar point cloud;F. the road side computing equipment computes voxel level characteristics of the voxelized deflection point cloud to obtain the voxel level characteristics of the deflection point cloud; calculating voxel-based vehicle-mounted laser radar point cloud voxel level characteristics by an automatic driving vehicle to obtain vehicle-mounted laser radar point cloud voxel level characteristics;G. The automatic driving vehicle compresses the vehicle-mounted laser radar point cloud voxel level characteristics to obtain compressed vehicle-mounted laser radar point cloud voxel level characteristics, the compressed vehicle-mounted laser radar point cloud voxel level characteristics are transmitted to the road side computing equipment, the road side computing equipment receives the compressed vehicle-mounted laser radar point cloud voxel level characteristics, and the compressed vehicle-mounted laser radar point cloud voxel level characteristics are restored to vehicle-mounted laser radar point cloud voxel level characteristics;H. the road side computing equipment performs data splicing and data aggregation on the vehicle-mounted laser radar point cloud voxel level characteristics and the deflection point cloud voxel level characteristics to obtain aggregate voxel level characteristics;I. the road side computing equipment inputs the aggregate voxel level characteristics into a three-dimensional target detection network model based on the voxel level characteristics to obtain a target detection result, and transmits the target detection result to the automatic driving vehicle.
- A vehicle-road cooperation-oriented perception information fusion characterization and target detection method comprises the following steps:the preparation stage:A. laying out a roadside laser radar, and configuring corresponding roadside computing equipment for the roadside laser radar;B. calibrating external parameters of the laser radar at the road side;the application stage comprises the following steps:C. the road side computing equipment computes the relative pose of the automatic driving vehicle relative to the road side laser radar according to the automatic driving vehicle positioning data and the road side laser radar external parameters;D. The road side computing equipment deflects the road side laser radar point cloud detected by the road side laser radar into an automatic driving vehicle coordinate system according to the relative pose to obtain deflection point cloud;E. and the road side computing equipment performs voxelization processing on the deflection point cloud to obtain the voxelized deflection point cloud. The automatic driving vehicle performs voxelization treatment on the vehicle-mounted laser radar point cloud detected by the vehicle-mounted laser radar to obtain voxelized vehicle-mounted laser radar point cloud;F. the road side computing equipment computes voxel level characteristics of the voxelized deflection point cloud to obtain the voxel level characteristics of the deflection point cloud; calculating voxel-based vehicle-mounted laser radar point cloud voxel level characteristics by an automatic driving vehicle to obtain vehicle-mounted laser radar point cloud voxel level characteristics;G. the road side computing equipment compresses the deflection point cloud voxel level characteristics to obtain compressed deflection point cloud voxel level characteristics, and transmits the compressed deflection point cloud voxel level characteristics to an automatic driving vehicle; the automatic driving vehicle receives the compressed deflection point cloud voxel level characteristics and restores the compressed deflection point cloud voxel level characteristics to deflection point cloud voxel level characteristics;H. the automatic driving vehicle performs data splicing and data aggregation on the vehicle-mounted laser radar point cloud voxel level features and the deflection point cloud voxel level features to obtain aggregate voxel level features;I. And the automatic driving vehicle inputs the aggregate voxel level characteristics into a three-dimensional target detection network model based on the voxel level characteristics to obtain a target detection result.
- A vehicle-road cooperation-oriented perception information fusion characterization and target detection method comprises the following steps:the preparation stage:A. laying out a roadside laser radar, and configuring corresponding roadside computing equipment for the roadside laser radar;B. calibrating external parameters of the laser radar at the road side;the application stage comprises the following steps:C. the road side computing equipment computes the relative pose of the automatic driving vehicle relative to the road side laser radar according to the automatic driving vehicle positioning data and the road side laser radar external parameters;D. the road side computing equipment deflects the road side laser radar point cloud detected by the road side laser radar into an automatic driving vehicle coordinate system according to the relative pose to obtain deflection point cloud;E. and the road side computing equipment performs voxelization processing on the deflection point cloud to obtain the voxelized deflection point cloud. The automatic driving vehicle performs voxelization treatment on the vehicle-mounted laser radar point cloud detected by the vehicle-mounted laser radar to obtain voxelized vehicle-mounted laser radar point cloud;F. and the road side computing equipment computes voxel level characteristics of the voxelized deflection point cloud to obtain the voxel level characteristics of the deflection point cloud. Calculating voxel-based vehicle-mounted laser radar point cloud voxel level characteristics by an automatic driving vehicle to obtain vehicle-mounted laser radar point cloud voxel level characteristics;G. The automatic driving vehicle compresses the vehicle-mounted laser radar point cloud voxel level characteristics to obtain compressed vehicle-mounted laser radar point cloud voxel level characteristics, and the compressed vehicle-mounted laser radar point cloud voxel level characteristics are transmitted to a cloud; the road side computing equipment compresses the voxel level characteristics of the deflection point cloud to obtain compressed voxel level characteristics of the deflection point cloud, and transmits the compressed voxel level characteristics to the cloud; the cloud receives the compressed deflection point cloud voxel level characteristics and the compressed vehicle-mounted laser radar point cloud voxel level characteristics, restores the compressed deflection point cloud voxel level characteristics to deflection point cloud voxel level characteristics, and restores the compressed vehicle-mounted laser radar point cloud voxel level characteristics to vehicle-mounted laser radar point cloud voxel level characteristics;H. the cloud end performs data splicing and data aggregation on the vehicle-mounted laser radar point cloud voxel level features and the deflection point cloud voxel level features to obtain aggregate voxel level features;I. the cloud inputs the aggregate voxel level features into a three-dimensional target detection network model based on the voxel level features to obtain a target detection result, and transmits the target detection result to the automatic driving vehicle.
- A method according to any one of claims 1 to 3, wherein the configuration criteria for the roadside lidar are:(1) for the conditions of installing a mechanical rotary laser radar on the road side and installing two reverse all-solid-state laser radars on the same rod, at least the following conditions are satisfied:Wherein:h represents the laser radar mounting height;θ 2 the included angle between the highest elevation angle light beam of the laser radar and the horizontal direction is shown;l represents the distance between two adjacent laser radar mounting rod positions;(2) the following requirements should be met for a roadside-mounted roadside all-solid-state lidar:wherein:H b representing the mounting height of the road side all-solid-state laser radar;the view field angle of the road side all-solid-state laser radar in the vertical direction is represented;the included angle between the highest elevation angle light beam of the road side all-solid-state laser radar and the horizontal direction is shown;L b and the distance between the installation rod positions of the adjacent two road-side all-solid-state laser radars is represented.
- A method according to any one of claims 1 to 3, wherein the number, location dispersion and co-linearity of control points are taken into account when selecting characteristic points as control points in the scanning area of the roadside lidar when calibrating the external parameters of the roadside lidar.
- A method according to any one of claims 1 to 3, wherein the external parameters of the roadside lidar are calibrated by: and taking the coordinates of the control points in a road side laser radar coordinate system and the coordinates in a reference coordinate system measured by the RTK as a target point set P and a source point set Q respectively, and calculating laser radar external parameters by using an ICP algorithm.
- A method according to any one of claims 1 to 3, wherein the deflection point cloud is expanded during the voxelization of the point cloud to ensure the onboard lidar point cloud D c And an expanded deflection point cloudThe voxel division grids of the grid are consistent, and the calculation formula is as follows:wherein:K lidar_start ′、K lidar_end ' asDeflection point cloud expanded in K dimensionA range start value and a range end value of (a);K lidar_start 、K lidar_end for a deflection point cloud in the K dimensionA range start value and a range end value of (a);V K is the size of the voxel in the K dimension.
- A method according to one of claims 1 to 3, characterized in that the point-to-point is supplemented with an offset to the center when extracting the point cloud voxel level features, i.e.:wherein:information of the ith point in the supplemented voxel A;x i 、y i 、z i coordinates of an i-th point in the voxel A;r i the reflection intensity for the i-th point in voxel a;v x 、v y 、v z is the mean of the coordinates of all points within voxel a.
- A method according to any one of claims 1 to 3, wherein the voxel-level feature data aggregation method uses a maximum pooling method to aggregate voxel-level features of the same coordinates, the formula being as follows:f k to aggregate voxel level featuresThe value at position k;f ego_k is the voxel level characteristic of the point cloud of the vehicle-mounted laser radar The value at position k;f lidar_k voxel level characterization for deflection point cloudThe value at position k;
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110000327 | 2021-01-01 | ||
| CN202110228419 | 2021-03-01 | ||
| PCT/CN2021/085148 WO2022141912A1 (en) | 2021-01-01 | 2021-04-01 | Vehicle-road collaboration-oriented sensing information fusion representation and target detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116685873A true CN116685873A (en) | 2023-09-01 |
Family
ID=82260124
Family Applications (5)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202180011148.3A Pending CN116685873A (en) | 2021-01-01 | 2021-04-01 | Vehicle-road cooperation-oriented perception information fusion representation and target detection method |
| CN202280026658.2A Pending CN117441113A (en) | 2021-01-01 | 2022-04-01 | Vehicle-road cooperation-oriented perception information fusion representation and target detection method |
| CN202280026656.3A Pending CN117836667A (en) | 2021-01-01 | 2022-04-01 | Static and non-static object point cloud identification method based on road side sensing unit |
| CN202280026659.7A Pending CN117836653A (en) | 2021-01-01 | 2022-04-01 | Road side millimeter wave radar calibration method based on vehicle-mounted positioning device |
| CN202280026657.8A Pending CN117441197A (en) | 2021-01-01 | 2022-04-01 | Laser radar point cloud dynamic segmentation and fusion method based on driving safety risk field |
Family Applications After (4)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202280026658.2A Pending CN117441113A (en) | 2021-01-01 | 2022-04-01 | Vehicle-road cooperation-oriented perception information fusion representation and target detection method |
| CN202280026656.3A Pending CN117836667A (en) | 2021-01-01 | 2022-04-01 | Static and non-static object point cloud identification method based on road side sensing unit |
| CN202280026659.7A Pending CN117836653A (en) | 2021-01-01 | 2022-04-01 | Road side millimeter wave radar calibration method based on vehicle-mounted positioning device |
| CN202280026657.8A Pending CN117441197A (en) | 2021-01-01 | 2022-04-01 | Laser radar point cloud dynamic segmentation and fusion method based on driving safety risk field |
Country Status (3)
| Country | Link |
|---|---|
| CN (5) | CN116685873A (en) |
| GB (2) | GB2620877B (en) |
| WO (9) | WO2022141912A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117452392A (en) * | 2023-12-26 | 2024-01-26 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Radar data processing system and method for vehicle-mounted auxiliary driving system |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114724362B (en) * | 2022-03-23 | 2022-12-27 | 中交信息技术国家工程实验室有限公司 | Vehicle track data processing method |
| CN115236628B (en) * | 2022-07-26 | 2024-05-31 | 中国矿业大学 | Method for detecting residual cargoes in carriage based on laser radar |
| CN115358530A (en) * | 2022-07-26 | 2022-11-18 | 上海交通大学 | Vehicle-road cooperative sensing roadside test data quality evaluation method |
| CN115113157B (en) * | 2022-08-29 | 2022-11-22 | 成都瑞达物联科技有限公司 | Beam pointing calibration method based on vehicle-road cooperative radar |
| CN115480235A (en) * | 2022-08-30 | 2022-12-16 | 中汽创智科技有限公司 | Road-end laser radar calibration method and device and electronic equipment |
| CN115480243B (en) * | 2022-09-05 | 2024-02-09 | 江苏中科西北星信息科技有限公司 | Multi-millimeter wave radar end-edge cloud fusion calculation integration and application method thereof |
| CN115166721B (en) * | 2022-09-05 | 2023-04-07 | 湖南众天云科技有限公司 | Radar and GNSS information calibration fusion method and device in roadside sensing equipment |
| CN115272493B (en) * | 2022-09-20 | 2022-12-27 | 之江实验室 | Abnormal target detection method and device based on continuous time sequence point cloud superposition |
| CN115235478B (en) * | 2022-09-23 | 2023-04-07 | 武汉理工大学 | Intelligent automobile positioning method and system based on visual label and laser SLAM |
| CN115453545A (en) * | 2022-09-28 | 2022-12-09 | 北京京东乾石科技有限公司 | Target object detection method, apparatus, mobile device and storage medium |
| CN115830860B (en) * | 2022-11-17 | 2023-12-15 | 西部科学城智能网联汽车创新中心(重庆)有限公司 | Traffic accident prediction method and device |
| CN115966084B (en) * | 2023-03-17 | 2023-06-09 | 江西昂然信息技术有限公司 | Holographic intersection millimeter wave radar data processing method and device and computer equipment |
| CN116189116B (en) * | 2023-04-24 | 2024-02-23 | 江西方兴科技股份有限公司 | Traffic state sensing method and system |
| CN117471461B (en) * | 2023-12-26 | 2024-03-08 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Road side radar service device and method for vehicle-mounted auxiliary driving system |
| CN117961915B (en) * | 2024-03-28 | 2024-06-04 | 太原理工大学 | Intelligent auxiliary decision-making method of coal mine tunneling robot |
| CN118226421B (en) * | 2024-05-22 | 2024-08-09 | 山东大学 | Laser radar-camera online calibration method and system based on reflectivity map |
| CN118334263B (en) * | 2024-06-11 | 2024-08-16 | 中国科学技术大学 | High-precision modeling method for fusion laser point cloud based on truncated symbol distance function |
Family Cites Families (47)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6661370B2 (en) * | 2001-12-11 | 2003-12-09 | Fujitsu Ten Limited | Radar data processing apparatus and data processing method |
| WO2014080330A2 (en) * | 2012-11-22 | 2014-05-30 | Geosim Systems Ltd. | Point-cloud fusion |
| KR101655606B1 (en) * | 2014-12-11 | 2016-09-07 | 현대자동차주식회사 | Apparatus for tracking multi object using lidar and method thereof |
| TWI597513B (en) * | 2016-06-02 | 2017-09-01 | 財團法人工業技術研究院 | Positioning system, onboard positioning device and positioning method thereof |
| CN105892471B (en) * | 2016-07-01 | 2019-01-29 | 北京智行者科技有限公司 | Automatic driving method and apparatus |
| WO2018126248A1 (en) * | 2017-01-02 | 2018-07-05 | Okeeffe James | Micromirror array for feedback-based image resolution enhancement |
| KR102056147B1 (en) * | 2016-12-09 | 2019-12-17 | (주)엠아이테크 | Registration method of distance data and 3D scan data for autonomous vehicle and method thereof |
| CN106846494A (en) * | 2017-01-16 | 2017-06-13 | 青岛海大新星软件咨询有限公司 | Oblique photograph three-dimensional building thing model automatic single-body algorithm |
| US10281920B2 (en) * | 2017-03-07 | 2019-05-07 | nuTonomy Inc. | Planning for unknown objects by an autonomous vehicle |
| CN108629231B (en) * | 2017-03-16 | 2021-01-22 | 百度在线网络技术（北京）有限公司 | Obstacle detection method, apparatus, device and storage medium |
| CN107133966B (en) * | 2017-03-30 | 2020-04-14 | 浙江大学 | Three-dimensional sonar image background segmentation method based on sampling consistency algorithm |
| CN108932462B (en) * | 2017-05-27 | 2021-07-16 | 华为技术有限公司 | Driving intention determining method and device |
| FR3067495B1 (en) * | 2017-06-08 | 2019-07-05 | Renault S.A.S | METHOD AND SYSTEM FOR IDENTIFYING AT LEAST ONE MOVING OBJECT |
| CN109509260B (en) * | 2017-09-14 | 2023-05-26 | 阿波罗智能技术(北京)有限公司 | Labeling method, equipment and readable medium of dynamic obstacle point cloud |
| CN107609522B (en) * | 2017-09-19 | 2021-04-13 | 东华大学 | Information fusion vehicle detection system based on laser radar and machine vision |
| CN108152831B (en) * | 2017-12-06 | 2020-02-07 | 中国农业大学 | Laser radar obstacle identification method and system |
| CN108010360A (en) * | 2017-12-27 | 2018-05-08 | 中电海康集团有限公司 | A kind of automatic Pilot context aware systems based on bus or train route collaboration |
| CN108639059B (en) * | 2018-05-08 | 2019-02-19 | 清华大学 | Driver based on least action principle manipulates behavior quantization method and device |
| CN109188379B (en) * | 2018-06-11 | 2023-10-13 | 深圳市保途者科技有限公司 | Automatic calibration method for driving auxiliary radar working angle |
| EP3819668A4 (en) * | 2018-07-02 | 2021-09-08 | Sony Semiconductor Solutions Corporation | Information processing device, information processing method, computer program, and moving body device |
| US10839530B1 (en) * | 2018-09-04 | 2020-11-17 | Apple Inc. | Moving point detection |
| CN109297510B (en) * | 2018-09-27 | 2021-01-01 | 百度在线网络技术（北京）有限公司 | Relative pose calibration method, device, equipment and medium |
| CN111429739A (en) * | 2018-12-20 | 2020-07-17 | 阿里巴巴集团控股有限公司 | Driving assisting method and system |
| JP7217577B2 (en) * | 2019-03-20 | 2023-02-03 | フォルシアクラリオン・エレクトロニクス株式会社 | CALIBRATION DEVICE, CALIBRATION METHOD |
| CN110220529B (en) * | 2019-06-17 | 2023-05-23 | 深圳数翔科技有限公司 | Positioning method for automatic driving vehicle at road side |
| CN110296713B (en) * | 2019-06-17 | 2024-06-04 | 广州卡尔动力科技有限公司 | Roadside automatic driving vehicle positioning navigation system and single/multiple vehicle positioning navigation method |
| CN110532896B (en) * | 2019-08-06 | 2022-04-08 | 北京航空航天大学 | Road vehicle detection method based on fusion of road side millimeter wave radar and machine vision |
| CN110443978B (en) * | 2019-08-08 | 2021-06-18 | 南京联舜科技有限公司 | Tumble alarm device and method |
| CN110458112B (en) * | 2019-08-14 | 2020-11-20 | 上海眼控科技股份有限公司 | Vehicle detection method and device, computer equipment and readable storage medium |
| CN110850378B (en) * | 2019-11-22 | 2021-11-19 | 深圳成谷科技有限公司 | Automatic calibration method and device for roadside radar equipment |
| CN110850431A (en) * | 2019-11-25 | 2020-02-28 | 盟识(上海)科技有限公司 | System and method for measuring trailer deflection angle |
| CN110906939A (en) * | 2019-11-28 | 2020-03-24 | 安徽江淮汽车集团股份有限公司 | Automatic driving positioning method and device, electronic equipment, storage medium and automobile |
| CN111121849B (en) * | 2020-01-02 | 2021-08-20 | 大陆投资(中国)有限公司 | Automatic calibration method for orientation parameters of sensor, edge calculation unit and roadside sensing system |
| CN111999741B (en) * | 2020-01-17 | 2023-03-14 | 青岛慧拓智能机器有限公司 | Method and device for detecting roadside laser radar target |
| CN111157965B (en) * | 2020-02-18 | 2021-11-23 | 北京理工大学重庆创新中心 | Vehicle-mounted millimeter wave radar installation angle self-calibration method and device and storage medium |
| CN111476822B (en) * | 2020-04-08 | 2023-04-18 | 浙江大学 | Laser radar target detection and motion tracking method based on scene flow |
| CN111554088B (en) * | 2020-04-13 | 2022-03-22 | 重庆邮电大学 | Multifunctional V2X intelligent roadside base station system |
| CN111192295B (en) * | 2020-04-14 | 2020-07-03 | 中智行科技有限公司 | Target detection and tracking method, apparatus, and computer-readable storage medium |
| CN111537966B (en) * | 2020-04-28 | 2022-06-10 | 东南大学 | Array antenna error correction method suitable for millimeter wave vehicle-mounted radar field |
| CN111766608A (en) * | 2020-06-12 | 2020-10-13 | 苏州泛像汽车技术有限公司 | Environmental perception system based on laser radar |
| CN111880191B (en) * | 2020-06-16 | 2023-03-28 | 北京大学 | Map generation method based on multi-agent laser radar and visual information fusion |
| CN111880174A (en) * | 2020-07-03 | 2020-11-03 | 芜湖雄狮汽车科技有限公司 | Roadside service system for supporting automatic driving control decision and control method thereof |
| CN111914664A (en) * | 2020-07-06 | 2020-11-10 | 同济大学 | Vehicle multi-target detection and track tracking method based on re-identification |
| CN111985322B (en) * | 2020-07-14 | 2024-02-06 | 西安理工大学 | Road environment element sensing method based on laser radar |
| CN111862157B (en) * | 2020-07-20 | 2023-10-10 | 重庆大学 | Multi-vehicle target tracking method integrating machine vision and millimeter wave radar |
| CN112019997A (en) * | 2020-08-05 | 2020-12-01 | 锐捷网络股份有限公司 | Vehicle positioning method and device |
| CN112509333A (en) * | 2020-10-20 | 2021-03-16 | 智慧互通科技股份有限公司 | Roadside parking vehicle track identification method and system based on multi-sensor sensing |
-
2021
- 2021-04-01 WO PCT/CN2021/085148 patent/WO2022141912A1/en active Application Filing
- 2021-04-01 WO PCT/CN2021/085147 patent/WO2022141911A1/en unknown
- 2021-04-01 WO PCT/CN2021/085150 patent/WO2022141914A1/en unknown
- 2021-04-01 GB GB2316625.9A patent/GB2620877B/en active Active
- 2021-04-01 WO PCT/CN2021/085149 patent/WO2022141913A1/en active Application Filing
- 2021-04-01 GB GB2313215.2A patent/GB2618936B/en active Active
- 2021-04-01 WO PCT/CN2021/085146 patent/WO2022141910A1/en unknown
- 2021-04-01 CN CN202180011148.3A patent/CN116685873A/en active Pending
-
2022
- 2022-04-01 WO PCT/CN2022/084738 patent/WO2022206942A1/en active Application Filing
- 2022-04-01 CN CN202280026658.2A patent/CN117441113A/en active Pending
- 2022-04-01 WO PCT/CN2022/084912 patent/WO2022206974A1/en active Application Filing
- 2022-04-01 WO PCT/CN2022/084929 patent/WO2022206978A1/en active Application Filing
- 2022-04-01 CN CN202280026656.3A patent/CN117836667A/en active Pending
- 2022-04-01 WO PCT/CN2022/084925 patent/WO2022206977A1/en active Application Filing
- 2022-04-01 CN CN202280026659.7A patent/CN117836653A/en active Pending
- 2022-04-01 CN CN202280026657.8A patent/CN117441197A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117452392A (en) * | 2023-12-26 | 2024-01-26 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Radar data processing system and method for vehicle-mounted auxiliary driving system |
| CN117452392B (en) * | 2023-12-26 | 2024-03-08 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Radar data processing system and method for vehicle-mounted auxiliary driving system |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022141912A1 (en) | 2022-07-07 |
| GB2618936B (en) | 2024-10-02 |
| GB2620877A8 (en) | 2024-07-31 |
| WO2022206974A1 (en) | 2022-10-06 |
| GB202313215D0 (en) | 2023-10-11 |
| CN117836667A (en) | 2024-04-05 |
| WO2022141910A1 (en) | 2022-07-07 |
| GB2620877B (en) | 2024-10-02 |
| GB2620877A (en) | 2024-01-24 |
| CN117836653A (en) | 2024-04-05 |
| WO2022141911A1 (en) | 2022-07-07 |
| WO2022206942A1 (en) | 2022-10-06 |
| GB202316625D0 (en) | 2023-12-13 |
| GB2618936A (en) | 2023-11-22 |
| WO2022141914A1 (en) | 2022-07-07 |
| CN117441113A (en) | 2024-01-23 |
| CN117441197A (en) | 2024-01-23 |
| WO2022206978A1 (en) | 2022-10-06 |
| WO2022141913A1 (en) | 2022-07-07 |
| WO2022206977A1 (en) | 2022-10-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116685873A (en) | Vehicle-road cooperation-oriented perception information fusion representation and target detection method | |
| CN111583337B (en) | Omnibearing obstacle detection method based on multi-sensor fusion | |
| Wang | Research on comparison of lidar and camera in autonomous driving | |
| GB2628958A (en) | A method of infrastructure-augmented cooperative perception for autonomous vehicles based on voxel feature aggregation | |
| US12055635B2 (en) | Method and device for adjusting parameters of LiDAR, and LiDAR | |
| CN110531376B (en) | Obstacle detection and tracking method for port unmanned vehicle | |
| Duan et al. | V2I based environment perception for autonomous vehicles at intersections | |
| KR20200126141A (en) | System and method for multiple object detection using multi-LiDAR | |
| CN115273028B (en) | Intelligent parking lot semantic map construction method and system based on global perception | |
| CN115019043B (en) | Cross-attention mechanism-based three-dimensional object detection method based on image point cloud fusion | |
| CN115876198A (en) | Target detection and early warning method, device, system and medium based on data fusion | |
| CN114283394A (en) | Traffic target detection system with integrated vehicle-mounted sensor | |
| CN117237919A (en) | Intelligent driving sensing method for truck through multi-sensor fusion detection under cross-mode supervised learning | |
| CN117111055A (en) | Vehicle state sensing method based on thunder fusion | |
| CN114821526A (en) | Obstacle three-dimensional frame detection method based on 4D millimeter wave radar point cloud | |
| CN116229224A (en) | Fusion perception method and device, electronic equipment and storage medium | |
| Zhu et al. | Design of laser scanning binocular stereo vision imaging system and target measurement | |
| CN117173399A (en) | Traffic target detection method and system of cross-modal cross-attention mechanism | |
| CN113988197B (en) | Multi-camera and multi-laser radar based combined calibration and target fusion detection method | |
| CN113378647B (en) | Real-time track obstacle detection method based on three-dimensional point cloud | |
| CN118570749A (en) | Multi-mode road sensing method, system, terminal equipment and storage medium | |
| CN117501311A (en) | Systems and methods for generating and/or using three-dimensional information with one or more cameras | |
| CN118411517A (en) | Digital twin method and device for traffic road in confluence area | |
| CN115267756A (en) | Monocular real-time distance measurement method based on deep learning target detection | |
| CN208937705U (en) | A kind of device of multi-source heterogeneous sensor characteristics depth integration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |