CN116416802A - Road universe sensing system - Google Patents
Road universe sensing system Download PDFInfo
- Publication number
- CN116416802A CN116416802A CN202111658090.XA CN202111658090A CN116416802A CN 116416802 A CN116416802 A CN 116416802A CN 202111658090 A CN202111658090 A CN 202111658090A CN 116416802 A CN116416802 A CN 116416802A
- Authority
- CN
- China
- Prior art keywords
- millimeter wave
- data
- wave radar
- laser radar
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008447 perception Effects 0.000 claims abstract description 89
- 230000001360 synchronised effect Effects 0.000 claims abstract description 18
- 230000004927 fusion Effects 0.000 claims description 46
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 7
- 238000007499 fusion processing Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 description 17
- 238000000034 method Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
- G08G1/096708—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/04—Detecting movement of traffic to be counted or controlled using optical or ultrasonic detectors
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
- G08G1/096766—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission
- G08G1/096775—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission where the origin of the information is a central station
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The application relates to a road universe perception system, which comprises a laser radar and a millimeter wave radar which are deployed on a target road, and a server. The perception area of the laser radar covers the crossing road section in the target road, the perception area of the millimeter wave radar at least covers the non-crossing road section in the target road, and the laser radar and the millimeter wave radar are connected with the server. The laser radar is used for acquiring laser radar data and sending the laser radar data to the server; the millimeter wave radar is used for acquiring millimeter wave radar data and sending the millimeter wave radar data to the server; the server is used for performing space-time synchronization on the laser radar data and the millimeter wave radar data, obtaining fused data and unfused data based on the space-time synchronized laser radar data and the millimeter wave radar data, and outputting a global perception result of the target road according to the spatial distribution of the fused data and the unfused data. The road universe sensing system can realize universe sensing of the target road.
Description
Technical Field
The application relates to the technical field of intelligent traffic, in particular to a road universe perception system.
Background
With the development of communication technology, autopilot is becoming a research hotspot in the field of intelligent traffic technology.
Automatic driving of a vehicle is generally divided into three parts, namely, environmental awareness, speed planning and path planning. In the conventional technology, the environment sensing, that is, the process of identifying a target object in a driving environment by a vehicle through a sensor of the vehicle.
However, since the vehicle itself is in the running environment and the coverage of the sensor itself is limited, the conventional technology of sensing the environment by using the sensor itself of the vehicle has the technical problem that the vehicle cannot realize the global sensing of the running environment due to the limited sensing of the environment.
Disclosure of Invention
Accordingly, it is desirable to provide a road global sensing system for solving the above-mentioned problems.
The road universe perception system comprises a laser radar and a millimeter wave radar which are deployed on a target road, and a server, wherein a perception area of the laser radar covers a crossing section in the target road, and a perception area of the millimeter wave radar at least covers a non-crossing section in the target road; the laser radar and the millimeter wave radar are connected with the server;
the laser radar is used for acquiring laser radar data and sending the laser radar data to the server;
the millimeter wave radar is used for acquiring millimeter wave radar data and sending the millimeter wave radar data to the server;
the server is used for carrying out space-time synchronization on the laser radar data and the millimeter wave radar data to obtain space-time synchronized laser radar data and millimeter wave radar data; obtaining fusion data and non-fusion data based on the laser radar data and the millimeter wave radar data of the time-space synchronization; and outputting the global perception result of the target road according to the spatial distribution of the fusion data and the non-fusion data.
In one embodiment, the server is configured to convert the lidar data to a world coordinate system according to a first calibration parameter of the lidar, and convert the millimeter wave radar data to the world coordinate system according to the first calibration parameter of the millimeter wave radar to achieve spatial synchronization of the lidar data and the millimeter wave radar data.
In one embodiment, the server comprises a plurality of edge servers and a central server; each edge server is connected with a laser radar and a millimeter wave radar which are installed in a preset area of a target road;
each edge server is used for carrying out space-time synchronization on the received laser radar data and millimeter wave radar data, and obtaining local perception results of corresponding preset areas based on the space-time synchronized laser radar data and millimeter wave radar data; transmitting the local perception result to a central server; the local perception result comprises fusion data and non-fusion data;
the central server is used for projecting the received local perception result to the same space coordinate system to obtain the global perception result of the target road.
In one embodiment, each edge server connects a laser radar set and a millimeter wave radar set; a first overlapping area exists between the sensing areas of the laser radar group and the millimeter wave radar group; the laser radar group comprises one or more laser radars, and the millimeter wave radar group comprises one or more millimeter wave radars;
the edge server is used for performing space-time synchronization on the laser radar data of the laser radar group and the millimeter wave radar data of the millimeter wave radar group by using the second calibration parameters, performing fusion processing on the laser radar data and the millimeter wave radar data after the space-time synchronization to obtain fusion data and non-fusion data, and outputting a local perception result on the basis of the spatial distribution of the fusion data and the non-fusion data; wherein the fusion data corresponds to the first overlapping region; the second calibration parameters are obtained according to the perception of the laser radar group and the millimeter wave radar group to the same target in the first overlapping area.
In one embodiment, a second overlapping region exists between each millimeter wave radar group and an adjacent millimeter wave radar group;
the center server projects the local perception result obtained by each edge server to a target coordinate system by using a third calibration parameter; the third calibration parameter is used for representing a conversion relation between a coordinate system used by the local perception result and a target coordinate system.
In one embodiment, the lidar unit comprises a primary lidar and at least one secondary lidar, and a third overlapping region exists between sensing regions of adjacent lidars in the lidar unit; a fourth overlapping area exists between the sensing area of the laser radar group and the sensing area of the corresponding millimeter wave radar group;
the edge server is used for converting the laser radar data of the auxiliary laser radar into a main laser radar coordinate system by using the fourth calibration parameter, and converting the millimeter wave radar data of the millimeter wave radar group into the main laser radar coordinate system by using the fifth calibration parameter so as to realize the space synchronization of the laser radar data and the millimeter wave radar data in the same edge server; the fourth calibration parameters are obtained according to the perception of the auxiliary laser radar and the main laser radar on the same target in the third overlapping area, and the fifth calibration parameters are obtained according to the perception of the main laser radar and the corresponding millimeter wave radar group on the same target in the fourth overlapping area.
In one embodiment, the millimeter wave radar group comprises a main millimeter wave radar and at least one auxiliary millimeter wave radar, and a fifth overlapping area exists between the sensing areas of adjacent millimeter wave radars in the millimeter wave radar group;
the edge server is used for converting millimeter wave radar data of the auxiliary millimeter wave radar into a main millimeter wave radar coordinate system by utilizing a sixth calibration parameter, and the sixth calibration parameter is obtained according to the perception of the auxiliary millimeter wave radar and the main millimeter wave radar on the same target in the fifth overlapping area.
In one embodiment, the server is configured to perform tracking processing on the target vehicle by using time-space synchronous laser radar data and millimeter wave radar data in multiple times, so as to obtain a global tracking result of the target vehicle on the target road.
In one embodiment, the server comprises a plurality of edge servers and a central server; each road opening of the target road is correspondingly provided with an edge server; each edge server is also used for binding the vehicle with the edge server based on the vehicle end information and the local perception result of the vehicle when the vehicle is driven into the corresponding intersection;
the central server transmits the global perception result to the vehicle based on the binding relation between each edge server and the vehicle.
In one embodiment, the vehicle-end information includes vehicle-end positioning information and/or vehicle attribute information, the local sensing result includes road vehicle positioning information and/or vehicle attribute information, and the content included in the local sensing result at least corresponds to the type of the vehicle-end information.
The road universe perception system comprises a laser radar, a millimeter wave radar and a server, wherein the laser radar and the millimeter wave radar are deployed on a target road. The sensing area of the laser radar covers the crossing section in the target road, the sensing area of the millimeter wave radar at least covers the non-crossing section in the target road, and the laser radar and the millimeter wave radar are connected with the server. The laser radar is used for acquiring laser radar data and sending the laser radar data to the server; the millimeter wave radar is used for acquiring millimeter wave radar data and sending the millimeter wave radar data to the server; the server is used for carrying out space-time synchronization on the laser radar data and the millimeter wave radar data to obtain space-time synchronized laser radar data and millimeter wave radar data, and obtaining fused data and unfused data based on the space-time synchronized laser radar data and the millimeter wave radar data, and further outputting a global perception result of the target road according to the spatial distribution of the fused data and the unfused data. The road universe perception system can realize universe perception of the target road, reduces perception limitation and expands perception range.
Drawings
FIG. 1 is a schematic diagram of a road global sensing system according to an embodiment;
FIG. 2 is a schematic diagram of an application environment of a road global perception system according to an embodiment;
FIG. 3 is a flow chart illustrating the global sensing result of a target road according to one embodiment;
FIG. 4 is a schematic diagram of an application environment of a road global perception system according to another embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
As shown in fig. 1, the road global perception system provided in the present application includes a laser radar 101, a millimeter wave radar 102, and a server 103. Wherein the lidar 101 and the millimeter wave radar are both communicatively connected to the server 103 via a network.
It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
The road universe sensing system can be applied to an application environment shown in fig. 2, wherein a laser radar and a millimeter wave radar are deployed on a target road to be sensed, a sensing area of the laser radar covers a crossing section in the target road, and a sensing area of the millimeter wave radar at least covers a non-crossing section in the target road.
The crossing section is an area where roads cross, such as an intersection. The non-intersecting road section is an area where roads are not intersected, such as a straight road section.
The sensing area of the laser radar and the sensing area of the millimeter wave radar are determined by the setting position and the sensing range of the corresponding radar. Optionally, in this embodiment, the sensing area of the lidar is a circular area formed by taking the setting position O of the lidar as the center of a circle and the sensing range is a distance R from the center of the circle. The sensing area of the millimeter wave radar is a wedge-shaped area formed by taking the setting position O of the millimeter wave radar as a radiation starting point and the sensing range as a distance L from the radiation starting point.
Specifically, the laser radar is used for acquiring laser radar data and sending the laser radar data to the server; the millimeter wave radar is used for millimeter wave radar data and transmits the millimeter wave radar data to the server.
Optionally, the laser radar data may be collected original point cloud data obtained by the laser radar, or may be a target detection result determined by the laser radar based on a target detection algorithm carried by the laser radar. The millimeter wave radar data may be the original point cloud data acquired by the millimeter wave radar, or may be a target detection result determined by the millimeter wave radar based on a target detection algorithm carried by the millimeter wave radar.
Specifically, the server receives laser radar data sent by the laser radar and millimeter wave radar data sent by the millimeter wave radar, and determines a global perception result of the target road according to the received laser radar data and millimeter wave radar data.
As shown in fig. 3, the process of determining the global perception result of the target link by the server includes:
and S310, performing space-time synchronization on the laser radar data and the millimeter wave radar data to obtain space-time synchronized laser radar data and millimeter wave radar data.
Specifically, the server performs time and space synchronization on the received laser radar data and millimeter wave radar data to obtain time-synchronized laser radar data and millimeter wave radar data.
S320, acquiring fusion data and non-fusion data based on the laser radar data and the millimeter wave radar data of the time-space synchronization.
The fusion data and the non-fusion data are used for representing the detection result of the target object in the target road. For example, the detection result may include attribute information of the type, position, motion state, and the like of the target object. The fusion data is a target detection result of the first overlapping area in the target road. The first overlapping region is an overlapping region between the sensing region of the laser radar and the sensing region of the millimeter wave radar. The non-fusion data is the target detection result of a non-first overlapping area in the sensing area of the laser radar and the sensing area of the millimeter wave radar.
Optionally, when the laser radar data and the millimeter wave radar data are both original point cloud data, the server may determine the first coverage area according to a sensing area of the laser radar and a sensing area of the millimeter wave radar, and perform target detection on all point cloud data (including those acquired by the laser radar and those acquired by the millimeter wave radar) after space-time synchronization in the first coverage area, so as to obtain the fused data. And (3) performing target detection on point cloud data (which can be acquired by a laser radar or a millimeter wave radar) in a non-first coverage area to obtain the non-fusion data.
Optionally, when the laser radar data and the millimeter wave radar data are both target detection results, the server may determine the first coverage area according to a sensing area of the laser radar and a sensing area of the millimeter wave radar, determine the laser radar data and the millimeter wave radar data after the same target is synchronized in the first coverage area, and determine a target detection result of the same target according to a data confidence level, so as to obtain the fusion data. And directly acquiring laser radar data and millimeter wave radar data of a non-first coverage area as the non-fusion data.
S330, outputting the global perception result of the target road according to the spatial distribution of the fusion data and the non-fusion data.
Specifically, the server gathers the fused data and the non-fused data, forms spatial distribution of the fused data and the non-fused data according to the positions of the target objects in the fused data and the non-fused data, and outputs the spatial distribution as a global perception result of the target road.
The road universe sensing system provided in the embodiment comprises a laser radar and a millimeter wave radar which are deployed on a target road, and a server. The sensing area of the laser radar covers the crossing section in the target road, the sensing area of the millimeter wave radar at least covers the non-crossing section in the target road, and the laser radar and the millimeter wave radar are connected with the server. The laser radar is used for acquiring laser radar data and sending the laser radar data to the server; the millimeter wave radar is used for acquiring millimeter wave radar data and sending the millimeter wave radar data to the server; the server is used for carrying out space-time synchronization on the laser radar data and the millimeter wave radar data to obtain space-time synchronized laser radar data and millimeter wave radar data, and obtaining fused data and unfused data based on the space-time synchronized laser radar data and the millimeter wave radar data, and further outputting a global perception result of the target road according to the spatial distribution of the fused data and the unfused data. The road universe perception system can realize universe perception of the target road, reduces perception limitation and expands perception range.
In one embodiment, to achieve spatial synchronization of the lidar data and the millimeter wave radar data, the server may convert the lidar data to a world coordinate system according to the first calibration parameters of the lidar, and convert the millimeter wave radar data to the world coordinate system according to the first calibration parameters of the millimeter wave radar to achieve spatial synchronization of the lidar data and the millimeter wave radar data.
The first calibration parameter is a conversion relation between a radar coordinate system used by the laser radar or the millimeter wave radar and a world coordinate system. The laser radar data can be converted into a world coordinate system based on the first calibration parameters of the laser radar; and the millimeter wave radar data can be converted into a world coordinate system based on the first calibration parameters of the millimeter wave radar.
The server in the road universe sensing system provided by the embodiment is used for converting laser radar data into a world coordinate system according to the first calibration parameters of the laser radar and converting millimeter wave radar data into the world coordinate system according to the first calibration parameters of the millimeter wave radar, so that spatial synchronization of the laser radar data and the millimeter wave radar data is realized. The space synchronization process ensures the unification of data, and the practicability and the application range of the road universe sensing system are improved due to the strong universality of the world coordinate system.
In one embodiment, to improve the data processing efficiency, as shown in fig. 4, the server includes a plurality of edge servers and a central server, and each edge server is connected to the lidar and the millimeter wave radar installed in a preset area of the target road.
Alternatively, as shown in fig. 4, the target road may include a plurality of preset areas M, each of which includes a crossing section area and a non-crossing section area, the sensing area of the lidar covering the crossing section area, and the sensing area of the millimeter wave radar covering at least the non-crossing section area.
Specifically, each edge server corresponds to a radar combination formed by a group of laser radars and millimeter wave radars, and correspondingly receives laser radar data and millimeter wave radar data sent by the corresponding radar combination. Each edge server is used for carrying out space-time synchronization on the received laser radar data and millimeter wave radar data, obtaining a local perception result of a corresponding preset area based on the space-time synchronized laser radar data and millimeter wave radar data, and sending the local perception result to the central server. The local perception result comprises fused data and non-fused data. The manner of determining the local sensing result by the edge server according to the received laser radar data and millimeter wave radar data is the same as the process of obtaining the fused data and the unfused data in the foregoing S320, and is not described herein.
Correspondingly, the central server receives the local perception results sent by the edge servers, and further projects the received local perception results to the same space coordinate system, and the global perception results of the target road are obtained after integration and de-duplication.
Alternatively, the global perception result of the target road may be represented by a position distribution of the vehicle on the target road map. The same space coordinate system is the coordinate system used by the target road map.
The server in the road universe sensing system provided by the embodiment comprises a plurality of edge servers and a center server; each edge server is connected with a laser radar and a millimeter wave radar which are installed in a preset area of a target road. Each edge server is used for carrying out space-time synchronization on the received laser radar data and millimeter wave radar data, obtaining local perception results of corresponding preset areas based on the space-time synchronized laser radar data and millimeter wave radar data, and sending the local perception results comprising fused data and unfused data to the central server. And the central server projects the received local perception result to the same space coordinate system to obtain the global perception result of the target road. The local perception results corresponding to the preset areas are determined through the edge servers, then the central service integrates the local perception results sent by the edge servers and the global perception results of the target road are obtained after the local perception results are de-duplicated, so that decentralized simultaneous processing of data is realized, the data processing pressure of a single server is reduced, and the overall data processing efficiency is improved.
In one embodiment, each edge server is connected to the lidar group and the millimeter wave radar group, and a first overlapping area exists between sensing areas of the lidar group and the millimeter wave radar group. Wherein the lidar group comprises one or more lidars and the millimeter wave radar group comprises one or more millimeter wave radars.
Specifically, the road global perception system comprises a plurality of groups of radar combinations of laser radars and millimeter wave radars, and each group of radar combination comprises a laser radar group formed by a plurality of laser radars and a millimeter wave radar group formed by a plurality of millimeter wave thunders. The sensing area of the laser radar group is formed by the sensing areas of a plurality of laser radars together, and can cover the whole road cross section; the sensing area of the millimeter wave radar is formed by jointly using the sensing areas of a plurality of laser radars, and can encircle a road crossing section by 360 degrees.
The edge server is used for performing space-time synchronization on the laser radar data of the laser radar group and the millimeter wave radar data of the millimeter wave radar group by using the second calibration parameters, performing fusion processing on the laser radar data and the millimeter wave radar data after the space-time synchronization to obtain fusion data and non-fusion data, and outputting a local perception result on the basis of the spatial distribution of the fusion data and the non-fusion data.
Wherein the fused data corresponds to the first overlapping region. As described above, the first overlapping region is an overlapping region between the sensing region of the lidar and the sensing region of the millimeter wave radar, for example, the first overlapping region is the region M1 in fig. 4. The second calibration parameters are obtained according to the perception of the laser radar group and the millimeter wave radar group to the same target in the first overlapping area.
It should be noted that, the second calibration parameter is used for representing a conversion relationship between the laser radar coordinate system and the millimeter wave radar coordinate system, and the edge server may determine the conversion relationship between the laser radar and the millimeter wave radar, that is, the second calibration parameter, according to the laser radar data and the millimeter wave radar data corresponding to the same target in the first overlapping area.
Optionally, the edge server may convert the laser radar data in the laser radar coordinate system to the millimeter wave radar coordinate system according to the second calibration parameter, or convert the millimeter wave radar data in the millimeter wave radar coordinate system to the laser radar coordinate system according to the second calibration parameter, so as to achieve coordinate unification of the laser radar data and the millimeter wave radar data, so as to achieve spatial synchronization of the data.
In an alternative embodiment, there is a second overlap region between each millimeter wave radar group and an adjacent millimeter wave radar group.
Specifically, as shown in fig. 4, there is also an overlapping area between the millimeter wave radar sets in the adjacent radar sets of the road global sensing system, that is, the second overlapping area M2.
And the center server projects the local perception result obtained by each edge server to the target coordinate system by using the third calibration parameters. The third calibration parameter is used for representing a conversion relation between a coordinate system used by the local perception result and a target coordinate system.
The coordinate system used by the local sensing result can be a laser radar coordinate system or a millimeter wave radar coordinate system. The target coordinate system may be a world coordinate system.
Optionally, when the coordinate system used by the local sensing result is a lidar coordinate system and the target coordinate system is a world coordinate system, the third calibration parameter is used to represent a conversion relationship between the lidar coordinate system and the world coordinate system; and under the condition that the coordinate system used by the local sensing result is a millimeter wave radar coordinate system and the target coordinate system is a world coordinate system, the third calibration parameter is used for representing the conversion relation between the millimeter wave radar coordinate system and the world coordinate system.
Optionally, after the central server receives the local sensing results sent by each edge server, the local sensing results can be converted into the target coordinate system according to the third calibration parameters, so as to realize data unification of the local sensing results sent by different edge servers.
In the road universe sensing system provided by the embodiment, each edge server is connected with a laser radar group and a millimeter wave radar group, a first overlapping area exists between sensing areas of the laser radar group and the millimeter wave radar group, the laser radar group comprises one or more laser radars, and the millimeter wave radar group comprises one or more millimeter wave radars. Compared with a millimeter wave radar, the millimeter wave radar has the advantages that the sensing range is small, the sensing accuracy is higher, the sensing range is wider, compared with the laser radar, the sensing accuracy of an overlapping area is ensured by adopting a mode of overlapping and deploying multiple radars, the sensing range of the whole sensing system is expanded as much as possible under limited radar equipment, and the utilization rate of the radar equipment is improved.
In one embodiment, the lidar unit includes a primary lidar and at least one secondary lidar, a third overlapping region exists between sensing regions of adjacent lidars in the lidar unit, and a fourth overlapping region exists between sensing regions of the lidar unit and sensing regions of the corresponding millimeter wave radar unit.
The sensing area of the laser radar group is formed by the sensing areas of the main laser radar and the auxiliary laser radar included in the laser radar group. The sensing area of the millimeter wave optical radar group is jointly formed by the sensing areas of the main millimeter wave radar and the auxiliary millimeter wave radar included in the millimeter wave radar group.
Optionally, the millimeter wave radar group comprises a main millimeter wave radar and at least one auxiliary millimeter wave radar, a fifth overlapping area exists between the sensing areas of adjacent millimeter wave radars in the millimeter wave radar group,
as shown in fig. 4, the description will be given taking an example in which the laser radar group includes one main laser radar and one auxiliary laser radar, and the millimeter wave radar group includes one main millimeter wave radar and three auxiliary millimeter wave radars. The sensing area of the laser radar set covers the whole crossroad, and four millimeter wave radars are respectively arranged along four directions of the road towards the crossroad, so that the sensing area of the millimeter wave radar set extends to cover road sections in four directions of the crossroad.
Wherein, there is an overlapping area between the sensing areas of adjacent lidars in the lidar group, namely the third overlapping area M3; an overlapping area, namely the fourth overlapping area M4, exists between the sensing area of the lidar unit and the sensing area of the corresponding millimeter wave radar unit. An overlapping area, namely the fifth overlapping area M5, exists between the sensing areas of adjacent millimeter wave radars in the millimeter wave radar group.
The edge server is used for converting the laser radar data of the auxiliary laser radar into a main laser radar coordinate system by using the fourth calibration parameter, and converting the millimeter wave radar data of the millimeter wave radar group into the main laser radar coordinate system by using the fifth calibration parameter, so that the spatial synchronization of the laser radar data and the millimeter wave radar data in the same edge server is realized.
The fourth calibration parameter is obtained according to the perception of the auxiliary laser radar and the main laser radar to the same target in the third overlapping area. And the fourth calibration parameter is used for representing the conversion relation between the main laser radar coordinate system and the auxiliary laser radar coordinate system.
Specifically, the edge server may determine the conversion relationship between the main lidar coordinate system and the auxiliary lidar coordinate system, that is, the fourth calibration parameter, according to the lidar data of the main lidar and the lidar data of the auxiliary lidar corresponding to the same target in the third overlapping area, and further convert the lidar data of the auxiliary lidar to the lower part of the main lidar coordinate system according to the fourth calibration parameter, so as to implement data unification inside the lidar group.
Optionally, the edge server is further configured to convert millimeter wave radar data of the auxiliary millimeter wave radar to a main millimeter wave radar coordinate system using the sixth calibration parameter.
The sixth calibration parameter is obtained according to the perception of the auxiliary millimeter wave radar and the main millimeter wave radar to the same target in the fifth overlapping area. And the sixth calibration parameter is used for representing the conversion relation between the main laser radar coordinate system and the main millimeter wave radar coordinate system.
Specifically, the edge server may determine the conversion relationship between the main millimeter wave radar coordinate system and the auxiliary millimeter wave radar coordinate system, that is, the sixth calibration parameter, according to the millimeter wave radar data of the main millimeter wave radar and the millimeter wave radar data of the auxiliary millimeter wave radar corresponding to the same target in the fifth overlapping area, and further convert the millimeter wave radar data of the auxiliary millimeter wave radar to the main millimeter wave radar coordinate system according to the sixth calibration parameter, so as to achieve data unification inside the millimeter wave radar group.
The fifth calibration parameter is obtained according to the same target perceived by the main laser radar and the corresponding millimeter wave radar group in the fourth overlapping area. And the fifth calibration parameter is used for representing the conversion relation between the main laser radar coordinate system and the main millimeter wave radar coordinate system.
Specifically, the edge server determines the conversion relationship between the main laser radar coordinate system and the main millimeter wave radar coordinate system, that is, the fifth calibration parameter according to the laser radar data of the main laser radar and the millimeter wave radar data of the main millimeter wave radar corresponding to the same target in the fourth overlapping area, and further converts the millimeter wave radar data of the main millimeter wave radar into the main laser radar coordinate system according to the fifth calibration parameter.
The road universe sensing system provided by the embodiment converts the laser radar data of the auxiliary laser radar in the laser radar group into the main laser radar coordinate system, converts the millimeter wave radar data of the auxiliary millimeter wave radar in the millimeter wave radar group into the main millimeter radar coordinate system, and then converts the main millimeter wave radar data into the main laser radar coordinate system, so that the unification of the internal data of different radars is ensured, the data among different radars is unified, unified standard source data is provided for the subsequent global sensing result of the obtained target road, and the accuracy of the global sensing result is improved.
In one embodiment, in order to track the target vehicle on the target road, the server is further configured to perform tracking processing on the target vehicle by using the time-space synchronous laser radar data and the millimeter wave radar data for a plurality of times, so as to obtain a global tracking result of the target vehicle on the target road.
Optionally, if the laser radar data and the millimeter wave radar data are the original point cloud data, the server performs space-time synchronization on the received laser radar data and millimeter wave radar data to obtain fusion point cloud data of time and space synchronization, and performs vehicle perception detection on the point cloud data to obtain a perception result. The sensing result may include characteristic information such as a position, a type, a traveling direction, or a traveling speed of the vehicle. The server is matched with the sensing result according to the characteristic information of the target vehicle to be tracked so as to determine the target vehicle, and tracks the target vehicle according to the fusion point cloud data of a plurality of times so as to obtain the form track of the target vehicle on the target road as the global tracking result of the target vehicle on the target road.
Optionally, if the laser radar data and the millimeter wave radar data are target detection results, the server performs space-time synchronous fusion processing on the target detection results obtained by different radar data to obtain a fusion result. The fusion result comprises the fusion data and the non-fusion data. The fusion result may include characteristic information such as a position, a type, a traveling direction, or a traveling speed of the vehicle. The server is matched with the fusion result according to the characteristic information of the target vehicle to be tracked so as to determine the target vehicle, and tracks the target vehicle according to the fusion result of a plurality of times so as to obtain the form track of the target vehicle on the target road as the global tracking result of the target vehicle on the target road.
In an alternative embodiment, the server includes a plurality of edge servers and a central server; each road port on the target road is correspondingly provided with an edge server; each edge server is used for binding the corresponding vehicle with the edge server based on the vehicle end information and the local perception result of the vehicle when the vehicle is driven into the corresponding intersection.
The central server transmits the global perception result to the vehicle based on the binding relation between each edge server and the vehicle.
Specifically, a vehicle entering an intersection may communicate with an edge server corresponding to the intersection, and send vehicle-side information to the edge server. The edge server receives the vehicle end information, determines a local perception result according to the laser radar data and the millimeter wave radar data acquired at the moment by the laser radar and the millimeter wave radar, further matches the vehicle end information with the local perception result, and binds a vehicle corresponding to the vehicle end information matched with the local perception result with the corresponding edge server. For example, the local sensing result determined by the edge server a includes target detection results of the vehicle 1, the vehicle 2 and the vehicle 3, and after the local sensing result is matched with the vehicle information, the edge server a and the vehicle 2 can be bound after the local sensing result is determined that the vehicle information is matched with the target detection result of the vehicle 2.
Further, the edge server sends the binding relationship between itself and the vehicle to the center server. Correspondingly, the central server receives the binding relation, and sends the global perception result to the vehicle end for display according to the binding relation after determining the global perception result of the target road.
Optionally, the central server may directly communicate with the vehicle, send the global sensing result to the corresponding vehicle end based on the binding relationship, and may also feed back the global sensing result to the edge server, and then send the global sensing result to the corresponding vehicle by the edge server according to the binding relationship.
Optionally, the vehicle-end information includes vehicle-end positioning information and/or vehicle attribute information, the local sensing result includes vehicle positioning information and/or vehicle attribute information, and the content included in the local sensing result at least corresponds to the type of the vehicle-end information.
The edge server in the road universe sensing system provided by the embodiment is further used for binding the vehicle with the edge server based on the vehicle end information and the local sensing result of the vehicle when the vehicle drives into the corresponding intersection, so that the central server transmits the universe sensing result to the vehicle based on the binding relation between each edge server and the vehicle. The global sensing result is directly fed back to the vehicle end, so that the vehicle end can more conveniently and intuitively know the current running environment.
It should be noted that, the second calibration parameter, the third calibration parameter, the fourth calibration parameter, and the fifth calibration parameter in the embodiment are only used to distinguish the differences between the converted data objects, and the purpose of the method is to convert the corresponding data into the same coordinate system for further operation. Therefore, the coordinate system to which the second calibration parameter, the third calibration parameter, the fourth calibration parameter, and the fifth calibration parameter in the above embodiments are transferred may also be the earth coordinate system. The spatial transformation of the partial data can likewise be simplified if the earth coordinate system is selected.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.
Claims (10)
1. A road universe sensing system comprising a lidar and a millimeter wave radar deployed on a target road, and a server, wherein a sensing area of the lidar covers a crossing section in the target road, and a sensing area of the millimeter wave radar covers at least a non-crossing section in the target road; the laser radar and the millimeter wave radar are connected with the server;
the laser radar is used for acquiring laser radar data and sending the laser radar data to the server;
the millimeter wave radar is used for acquiring millimeter wave radar data and sending the millimeter wave radar data to the server;
the server is used for carrying out space-time synchronization on the laser radar data and the millimeter wave radar data to obtain space-time synchronized laser radar data and millimeter wave radar data; obtaining fusion data and non-fusion data based on the space-time synchronized laser radar data and millimeter wave radar data; and outputting the global perception result of the target road according to the spatial distribution of the fusion data and the non-fusion data.
2. The system of claim 1, wherein the server is configured to convert the lidar data to a world coordinate system based on a first calibration parameter of a lidar and to convert the millimeter-wave radar data to the world coordinate system based on a first calibration parameter of a millimeter-wave radar to achieve spatial synchronization of the lidar data and the millimeter-wave radar data.
3. The system of claim 1, wherein the server comprises a plurality of edge servers and a central server; each edge server is connected with a laser radar and a millimeter wave radar which are installed in a preset area of the target road;
each edge server is used for carrying out space-time synchronization on the received laser radar data and the millimeter wave radar data, and obtaining a local perception result of a corresponding preset area based on the space-time synchronized laser radar data and the millimeter wave radar data; transmitting the local perception result to the central server; the local perception result comprises fusion data and non-fusion data;
the center server is used for projecting the received local perception result to the same space coordinate system to obtain the global perception result of the target road.
4. A system according to claim 3, wherein each edge server is connected to a lidar group and a millimeter wave radar group; a first overlapping area exists between the sensing areas of the laser radar group and the millimeter wave radar group; the laser radar group comprises one or more laser radars, and the millimeter wave radar group comprises one or more millimeter wave radars;
the edge server is used for performing space-time synchronization on the laser radar data of the laser radar group and the millimeter wave radar data of the millimeter wave radar group by using a second calibration parameter, performing fusion processing on the basis of the laser radar data and the millimeter wave radar data after the space-time synchronization to obtain fusion data and non-fusion data, and outputting a local perception result on the basis of the spatial distribution of the fusion data and the non-fusion data; wherein the fusion data corresponds to the first overlap region; and the second calibration parameters are obtained according to the perception of the laser radar group and the millimeter wave radar group to the same target in the first overlapping area.
5. The system of claim 4, wherein a second overlap region exists between each millimeter wave radar group and an adjacent millimeter wave radar group;
the center server projects the local perception results obtained by the edge servers to a target coordinate system by using a third calibration parameter; the third calibration parameter is used for representing a conversion relation between a coordinate system used by the local perception result and the target coordinate system.
6. The system of claim 4, wherein the lidar group comprises a primary lidar and at least one secondary lidar, wherein a third overlapping region exists between sensing regions of adjacent lidars in the lidar group; a fourth overlapping area exists between the sensing area of the laser radar group and the sensing area of the corresponding millimeter wave radar group;
the edge server is used for converting the laser radar data of the auxiliary laser radar into the main laser radar coordinate system by using a fourth calibration parameter, and converting the millimeter wave radar data of the millimeter wave radar group into the main laser radar coordinate system by using a fifth calibration parameter so as to realize the space synchronization of the laser radar data and the millimeter wave radar data in the same edge server; the fourth calibration parameters are obtained according to the perception of the auxiliary laser radar and the main laser radar on the same target in the third overlapping area, and the fifth calibration parameters are obtained according to the perception of the main laser radar and the corresponding millimeter wave radar group on the same target in the fourth overlapping area.
7. The system of claim 6, wherein the millimeter wave radar set includes a primary millimeter wave radar and at least one secondary millimeter wave radar, a fifth overlapping region exists between sensing regions of adjacent millimeter wave radars in the millimeter wave radar set;
the edge server is used for converting millimeter wave radar data of the auxiliary millimeter wave radar into the main millimeter wave radar coordinate system by utilizing a sixth calibration parameter, and the sixth calibration parameter is obtained according to the perception of the auxiliary millimeter wave radar and the main millimeter wave radar on the same target in the fifth overlapping area.
8. The system of claim 1, wherein the server is configured to perform tracking processing on a target vehicle on time-space synchronized laser radar data and millimeter wave radar data of a plurality of times, to obtain a global tracking result of the target vehicle on the target road.
9. The system of claim 8, wherein the server comprises a plurality of edge servers and a central server; each road opening of the target road is correspondingly provided with an edge server; each edge server is further used for binding the vehicle with the edge server based on vehicle end information and local perception results of the vehicle when the vehicle is driven into the corresponding intersection;
the center server transmits the global perception result to the vehicle based on the binding relation between each edge server and the vehicle.
10. The system of claim 9, wherein the vehicle-side information includes vehicle-side positioning information and/or vehicle attribute information, the local perception result includes road vehicle positioning information and/or vehicle attribute information, and the local perception result includes content at least corresponding to a type of the vehicle-side information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111658090.XA CN116416802A (en) | 2021-12-30 | 2021-12-30 | Road universe sensing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111658090.XA CN116416802A (en) | 2021-12-30 | 2021-12-30 | Road universe sensing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116416802A true CN116416802A (en) | 2023-07-11 |
Family
ID=87051636
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111658090.XA Pending CN116416802A (en) | 2021-12-30 | 2021-12-30 | Road universe sensing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116416802A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117452407A (en) * | 2023-12-26 | 2024-01-26 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Radar data service system and method for vehicle-mounted auxiliary driving system |
-
2021
- 2021-12-30 CN CN202111658090.XA patent/CN116416802A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117452407A (en) * | 2023-12-26 | 2024-01-26 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Radar data service system and method for vehicle-mounted auxiliary driving system |
| CN117452407B (en) * | 2023-12-26 | 2024-03-08 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Radar data service system and method for vehicle-mounted auxiliary driving system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10349011B2 (en) | System and method for improved obstacle awareness in using a V2X communications system | |
| US10613547B2 (en) | System and method for improved obstacle awareness in using a V2X communications system | |
| US11630998B2 (en) | Systems and methods for automatically training neural networks | |
| CN109389832B (en) | System and method for improving obstacle awareness using a V2X communication system | |
| US20200074852A1 (en) | Roadside sensing system based on vehicle infrastructure cooperation, and method for controlling vehicle thereof | |
| US20200226794A1 (en) | Three-dimensional data creation method, client device, and server | |
| EP3748299B1 (en) | Information transmitting method, and client device | |
| US9373255B2 (en) | Method and system for producing an up-to-date situation depiction | |
| WO2020263333A1 (en) | Augmented 3d map | |
| CN110441790A (en) | Method and apparatus crosstalk and multipath noise reduction in laser radar system | |
| Liu et al. | Cooperation of V2I/P2I communication and roadside radar perception for the safety of vulnerable road users | |
| CN113587951A (en) | Path planning method, device, system, server, storage medium and product | |
| CN116416802A (en) | Road universe sensing system | |
| US20220234621A1 (en) | Augmented 3d map | |
| US11499833B2 (en) | Inferring lane boundaries via high speed vehicle telemetry | |
| US20210146827A1 (en) | Systems and methods to communicate an intended vehicle maneuver | |
| US11753014B2 (en) | Method and control unit automatically controlling lane change assist | |
| Chehri et al. | Localization for vehicular ad hoc network and autonomous vehicles, are we done yet? | |
| CN112418092A (en) | Fusion method, device, equipment and storage medium for obstacle perception | |
| US20210264689A1 (en) | Vehicle error alerting system | |
| WO2023087248A1 (en) | Information processing method and apparatus | |
| US20230316907A1 (en) | Intersection-based offboard vehicle path generation | |
| US20230316911A1 (en) | Intersection-based map message generation and broadcasting | |
| US20230316921A1 (en) | Collision warning based on intersection information from map messages | |
| US20230316912A1 (en) | Path prediction based on intersection information from map messages |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |