CN115909806A - Collision early warning method and device and road side equipment - Google Patents
Collision early warning method and device and road side equipment Download PDFInfo
- Publication number
- CN115909806A CN115909806A CN202110896048.5A CN202110896048A CN115909806A CN 115909806 A CN115909806 A CN 115909806A CN 202110896048 A CN202110896048 A CN 202110896048A CN 115909806 A CN115909806 A CN 115909806A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- collision
- vulnerable traffic
- vulnerable
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000004927 fusion Effects 0.000 claims description 31
- 238000004422 calculation algorithm Methods 0.000 claims description 28
- 238000012163 sequencing technique Methods 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 13
- 238000007499 fusion processing Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 10
- 238000003062 neural network model Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000694 effects Effects 0.000 abstract description 12
- 238000004891 communication Methods 0.000 description 11
- 230000001133 acceleration Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000008447 perception Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000002592 echocardiography Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000017525 heat dissipation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention provides a collision early warning method, a collision early warning device and roadside equipment, and relates to the technical field of intelligent traffic, wherein the collision early warning method comprises the following steps: acquiring data acquired by at least one roadside sensor aiming at a target object, wherein the target object comprises vulnerable traffic participants and vehicles; determining whether the vulnerable traffic participant and the vehicle have collision danger according to the collected data; and sending early warning information to the vehicle under the condition that the vulnerable traffic participant and the vehicle have collision danger. The embodiment of the invention can improve the collision early warning effect.
Description
Technical Field
The invention relates to the technical field of intelligent traffic, in particular to a collision early warning method and device and road side equipment.
Background
In the practice of road traffic, accidents that vehicles collide with traffic vulnerable participants such as non-motor vehicles and pedestrians frequently occur. The collision early warning system in the prior art mainly carries out collision early warning for vehicles, and researches on collision early warning for vehicles and traffic weak participants are less. When collision early warning is carried out on a vehicle and traffic weakness participants, the vehicle-mounted sensors of the vehicle detect the traffic weakness participants around the vehicle so as to carry out collision early warning, the vehicle-mounted sensors can only detect the traffic weakness participants in a small-range area around the vehicle, and the vehicle-mounted sensors have large blind areas, so that the collision early warning effect is poor.
Disclosure of Invention
The embodiment of the invention provides a collision early warning method, a collision early warning device and roadside equipment, and aims to solve the problems that the existing vehicle-mounted sensor detects traffic weakness participants around a vehicle to perform collision early warning, and the collision early warning effect is poor.
In order to solve the technical problem, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a collision warning method, which is applied to roadside equipment, and includes:
acquiring data acquired by at least one roadside sensor aiming at a target object, wherein the target object comprises vulnerable traffic participants and vehicles;
determining whether the vulnerable traffic participant and the vehicle have collision danger according to the collected data;
and sending early warning information to the vehicle under the condition that the vulnerable traffic participant and the vehicle have collision danger.
Optionally, the at least one roadside sensor includes a camera, a millimeter wave radar, and a laser radar;
the collected data comprises video data output by the camera, first structured data output by the millimeter wave radar and point cloud data output by the laser radar;
the determining whether the vulnerable traffic participant and the vehicle have a collision risk according to the collected data comprises:
sensing and fusing the video data output by the camera, the first structured data output by the millimeter wave radar and the point cloud data output by the laser radar, and determining fused data corresponding to the target object;
and determining whether the vulnerable traffic participant and the vehicle have collision danger or not according to the fusion data corresponding to the target object.
Optionally, the number of the target objects is at least two, and the sensing fusion processing is performed on the video data output by the camera, the first structured data output by the millimeter wave radar, and the point cloud data output by the laser radar, and includes:
processing the video data output by the camera based on a target detection algorithm and a target tracking algorithm to obtain second structured data corresponding to at least two target objects;
processing the point cloud data output by the laser radar based on the trained neural network model to obtain third structured data corresponding to at least two target objects;
and performing association matching on the first structured data, the second structured data and the third structured data by adopting a Hungarian algorithm, and determining fused data corresponding to each target object.
Optionally, the determining whether there is a collision risk between the vulnerable traffic participant and the vehicle according to the collected data includes:
determining the distance between the vulnerable traffic participant and the vehicle from the collected data if the vulnerable traffic participant is located within a motor vehicle lane area;
determining whether there is a collision risk between the vulnerable traffic participant and the vehicle depending on the distance between the vulnerable traffic participant and the vehicle.
Optionally, the determining whether there is a collision risk between the vulnerable traffic participant and the vehicle according to the collected data includes:
under the condition that a traffic signal lamp indicates that the weak traffic participant is not allowed to be located in a lane crossing area, if the weak traffic participant is located in the lane crossing area, predicting a moving path of the weak traffic participant and a moving path of the vehicle according to the collected data;
determining whether the vulnerable traffic participant and the vehicle have collision danger according to the predicted moving path of the vulnerable traffic participant and the moving path of the vehicle.
Optionally, the number of the vulnerable traffic participants is multiple, and the sending of the early warning information to the vehicle when there is a collision risk between the vulnerable traffic participants and the vehicle includes:
determining a collision risk degree of each of the participants of the weak traffic with the vehicle in the case that a plurality of the participants of the weak traffic and the vehicle are in collision risk;
sequencing the early warning sequence of the plurality of weak traffic participants according to the collision danger degree to obtain a sequencing result;
and sequentially sending early warning information to the vehicles according to the sequencing result.
Optionally, the degree of risk of collision of each said participant of vulnerable traffic with said vehicle is determined based on the predicted time of collision of each said participant of vulnerable traffic with said vehicle.
In a second aspect, an embodiment of the present invention provides a collision warning apparatus, which is applied to roadside equipment, and includes:
the system comprises an acquisition module, a data acquisition module and a data acquisition module, wherein the acquisition module is used for acquiring the acquired data of at least one roadside sensor aiming at a target object, and the target object comprises vulnerable traffic participants and vehicles;
the determining module is used for determining whether the vulnerable traffic participant and the vehicle have collision danger according to the collected data;
and the sending module is used for sending early warning information to the vehicle under the condition that the vulnerable traffic participant and the vehicle have collision danger.
Optionally, the at least one roadside sensor includes a camera, a millimeter wave radar, and a laser radar;
the collected data comprises video data output by the camera, first structured data output by the millimeter wave radar and point cloud data output by the laser radar;
the determining module comprises:
the fusion unit is used for performing perception fusion processing on the video data output by the camera, the first structured data output by the millimeter wave radar and the point cloud data output by the laser radar, and determining fusion data corresponding to the target object;
and the determining unit is used for determining whether the vulnerable traffic participant and the vehicle have collision risks according to the fusion data corresponding to the target object.
Optionally, the number of the target objects is at least two, and the fusion unit is specifically configured to:
processing the video data output by the camera based on a target detection algorithm and a target tracking algorithm to obtain second structured data corresponding to at least two target objects;
processing the point cloud data output by the laser radar based on the trained neural network model to obtain third structured data corresponding to at least two target objects;
and performing association matching on the first structured data, the second structured data and the third structured data by adopting a Hungarian algorithm, and determining fused data corresponding to each target object.
Optionally, the determining module is specifically configured to:
determining the distance between the vulnerable traffic participant and the vehicle from the collected data if the vulnerable traffic participant is located within a motor vehicle lane area;
determining whether there is a collision risk between the vulnerable traffic participant and the vehicle depending on the distance between the vulnerable traffic participant and the vehicle.
Optionally, the determining module is specifically configured to:
under the condition that a traffic signal lamp indicates that the weak traffic participant is not allowed to be located in a lane crossing area, if the weak traffic participant is located in the lane crossing area, predicting a moving path of the weak traffic participant and a moving path of the vehicle according to the collected data;
determining whether the vulnerable traffic participant and the vehicle have collision danger according to the predicted moving path of the vulnerable traffic participant and the moving path of the vehicle.
Optionally, the number of the vulnerable traffic participants is multiple, and the sending module is specifically configured to:
determining a collision risk degree of each of the participants of the weak traffic with the vehicle in the case that a plurality of the participants of the weak traffic and the vehicle are in collision risk;
sequencing the early warning sequence of the plurality of weak traffic participants according to the collision danger degree to obtain a sequencing result;
and sequentially sending early warning information to the vehicles according to the sequencing result.
Optionally, the degree of risk of collision of each said participant of vulnerable traffic with said vehicle is determined based on the predicted time of collision of each said participant of vulnerable traffic with said vehicle.
In a third aspect, an embodiment of the present invention provides a roadside apparatus, including: a processor, a memory and a program stored on the memory and executable on the processor, the program implementing the steps of the collision warning method according to the first aspect when executed by the processor.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements the steps of the collision warning method according to the first aspect.
In the embodiment of the invention, the acquisition data of at least one roadside sensor aiming at a target object is acquired, wherein the target object comprises vulnerable traffic participants and vehicles; determining whether the vulnerable traffic participant and the vehicle have collision danger according to the collected data; and sending early warning information to the vehicle under the condition that the vulnerable traffic participant and the vehicle have collision danger. Therefore, data are collected through at least one roadside sensor, the detection range is wide, and the collision early warning effect can be improved during collision early warning.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments of the present invention will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without inventive exercise.
Fig. 1 is a flowchart of a collision warning method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a roadside sensing system according to an embodiment of the present invention;
FIG. 3 is one of the schematic diagrams of road segments provided by embodiments of the present invention;
FIG. 4 is a second schematic diagram of a road segment provided by an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a collision warning apparatus according to an embodiment of the present invention;
fig. 6 is a second schematic structural diagram of a collision warning apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a roadside apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides a collision early warning method, a collision early warning device and roadside equipment, and aims to solve the problems that the existing vehicle-mounted sensors detect traffic weakness participants around a vehicle to perform collision early warning, and the collision early warning effect is poor.
Referring to fig. 1, fig. 1 is a flowchart of a collision warning method provided in an embodiment of the present invention, for roadside equipment, and as shown in fig. 1, the method includes the following steps:
Wherein, at least one roadside sensor may include a camera and/or a radar, etc., and the radar may include a millimeter wave radar and/or a laser radar, etc. The vehicle may be a motor vehicle and the vulnerable traffic participants may include pedestrians and/or non-motor vehicles.
And step 102, determining whether the vulnerable traffic participant and the vehicle have collision danger or not according to the collected data.
The collected data may include information such as position, speed, heading angle, size, longitudinal acceleration, yaw rate, etc. The collected data for the target object may include collected data for the vulnerable traffic participants and collected data for the vehicle. The method comprises the steps that the moving paths of the vulnerable traffic participants and the moving paths of the vehicles can be predicted according to collected data aiming at the vulnerable traffic participants and collected data aiming at the vehicles, and when the moving paths of the vulnerable traffic participants and the moving paths of the vehicles have intersection points, the fact that the vulnerable traffic participants and the vehicles have collision risks is determined; or, the collected data for the vulnerable traffic participant and the collected data for the vehicle may both include location information, and when the distance between the vulnerable traffic participant and the vehicle is less than a preset distance, it may be determined that there is a collision risk between the vulnerable traffic participant and the vehicle; and the like, which is not limited by the present embodiment.
And 103, sending early warning information to the vehicle under the condition that the vulnerable traffic participant and the vehicle have collision danger.
As a specific implementation manner, the roadside apparatus may be a roadside sensing computing apparatus, and a roadside sensing system may be deployed in the intersection area, as shown in fig. 2, the roadside sensing system may include the roadside sensing computing apparatus and at least one roadside sensor. The at least one roadside sensor may include a camera, a millimeter wave radar, and a lidar. The camera, the millimeter wave radar and the laser radar can be accessed to the roadside sensing computing equipment, for example, information such as an IP address, a gateway address, a physical MAC address and a subnet mask of external equipment such as the camera, the millimeter wave radar and the laser radar can be configured on a management platform of the roadside sensing computing equipment, and the roadside sensor can be accessed to the management platform of the roadside sensing computing equipment. The collected data of the roadside sensor can be transmitted to the roadside sensing computing device through a preconfigured rule, taking a camera as an example, the preconfigured rule may include recording time of video data, a stored code stream format and the like, and the stored code stream format may include a main code stream or a sub code stream.
In addition, when the vehicle is warned, the roadside awareness computing device may package the warning information for the vulnerable traffic participant and the specific parameter information of the vulnerable traffic participant into an RSM (Road Safety Message) Message and transmit the RSM Message to the roadside communication unit. The roadside communication Unit may be an RSU (Road Side Unit), and the roadside communication Unit may forward the RSM message to an OBU (On Board Unit). The roadside communication unit may send the RSM message to the Vehicle-mounted communication unit of the Vehicle in the intersection area in a broadcast manner through a PC5 interface of a C-V2X (Vehicle to electrical networking) wireless module. The PC5 interface may be a pass-through communication interface in a C-V2X network. And the vehicle-mounted communication unit receives the RSM message, analyzes the RSM message and sends the RSM message to the vehicle. Exemplarily, the vehicle-mounted communication unit can analyze the RSM message through the C-V2X module, and send the early warning information to a vehicle HMI (Human Machine Interface) for real-time early warning display; or the early warning information aiming at the vulnerable traffic participants and the specific parameter information of the vulnerable traffic participants can be sent to the vehicle-related safety control module, so that the vehicle can conveniently carry out linkage early warning prompt of light, sound and the like.
It should be noted that, the determining whether there is a collision risk between the vulnerable traffic participant and the vehicle according to the collected data may include: sensing and fusing the collected data of the at least one road side sensor for the target object to determine fused data corresponding to the target object; further, the at least one roadside sensor may include a camera, a millimeter wave radar, and a laser radar, and the performing sensing fusion processing on the collected data of the at least one roadside sensor for the target object to determine fusion data corresponding to the target object may include: and performing perception fusion processing on the video data output by the camera, the first structured data output by the millimeter wave radar and the point cloud data output by the laser radar, and determining fusion data corresponding to the target object.
In the embodiment of the invention, the collision early warning and analyzing function of the vulnerable traffic participants is incorporated into the existing traffic monitoring system, so that the intellectualization of traffic facilities is improved, and the intelligent traffic monitoring system can play a role in promoting the construction of intelligent traffic and intelligent cities. Compared with the prior art that pedestrian early warning decision making is carried out only by means of data collected by a vehicle sensor, sometimes the reliability is reduced due to weather factors or visual blind areas, and pedestrians cannot be accurately identified, and meanwhile when the pedestrians are identified and early warned in a sight distance range, sometimes early warning is not timely due to too close distance, the embodiment timely sends event information of regional roads in a certain range to the vehicles in a PC5 mode by constructing a road side sensing system, so that the sight distance range is enlarged, early warning decision making is carried out in advance, and the passing safety of the vehicles and the pedestrians can be effectively improved; the target object can be identified more accurately through multi-source sensing information fusion of road side sensors such as a camera, a millimeter wave radar and a laser radar, so that collision danger possibly occurring to vulnerable traffic participants can be detected and predicted accurately, and road safety is improved; and the roadside communication unit sends the early warning information to the vehicle in a broadcasting mode through a PC5 interface of the C-V2X wireless module, compared with a 4G/5G network, the roadside communication unit is low in time delay of a transmission mode and high in reliability, is beneficial to improving the communication efficiency, and can ensure the traffic safety of motor vehicles and vulnerable traffic participants.
Furthermore, in the embodiment, information acquisition and early warning analysis are completed by a road side end, the vehicle is only used for receiving early warning information, the acquisition burden and the calculation burden of the vehicle are objectively reduced, particularly the endurance burden, the vehicle-mounted power consumption, the heat dissipation and other burdens of a long-distance vehicle are reduced, the vehicle-mounted self-warning is replaced by the further intelligent early warning function of inherent traffic facilities, and the vehicle-mounted pressure is reduced; moreover, the collected information can be fused by the multisource sensor at the roadside end, so that the collected data is more accurate, the problem of visual blind areas caused by the fact that the information is collected by the vehicle end is avoided, the problem of collection angles can be better solved, and the field of view of the collected information is wider; moreover, the acquisition equipment and the early warning equipment at the vehicle end are purchased by respective vehicle owners at present, and are not easy to unify standards when various early warning products exist in the market.
In the embodiment of the invention, the acquisition data of at least one roadside sensor aiming at a target object is acquired, wherein the target object comprises vulnerable traffic participants and vehicles; determining whether the vulnerable traffic participant and the vehicle have collision danger according to the collected data; and sending early warning information to the vehicle under the condition that the vulnerable traffic participant and the vehicle have collision danger. Therefore, data are collected through at least one roadside sensor, the detection range is wide, and the collision early warning effect can be improved during collision early warning.
Optionally, the at least one roadside sensor includes a camera, a millimeter wave radar, and a laser radar;
the collected data comprises video data output by the camera, first structured data output by the millimeter wave radar and point cloud data output by the laser radar;
the determining whether the vulnerable traffic participant and the vehicle have a collision risk according to the collected data comprises:
sensing and fusing video data output by the camera, first structured data output by the millimeter wave radar and point cloud data output by the laser radar to determine fused data corresponding to the target object;
and determining whether the vulnerable traffic participant and the vehicle have collision danger or not according to the fusion data corresponding to the target object.
The video data output by the camera, the first structured data output by the millimeter wave radar and the point cloud data output by the laser radar can be subjected to perception fusion processing by adopting perception fusion processing algorithms such as a Kalman filtering algorithm, a Hungary matching algorithm and the like. The fused data may be structured data, and may include, for example, time, position, speed, heading angle, size, longitudinal acceleration, yaw rate, and other parameters.
In addition, the millimeter wave radar can precisely detect the direction and distance of a target by emitting electromagnetic waves to an obstacle and receiving echoes, and can directly output structured data including data such as the object type, time, speed, XY distance, acceleration, and the like of the target body.
It should be noted that the multi-sensor joint calibration can be performed on the camera, the millimeter wave radar and the laser radar to realize the spatial fusion of the video data output by the camera, the first structured data output by the millimeter wave radar and the point cloud data output by the laser radar. Exemplarily, a coordinate conversion relation among a precise millimeter wave radar coordinate system, a three-dimensional world coordinate system, a camera coordinate system and a laser radar coordinate system can be established, and the camera, the millimeter wave radar and the laser radar are calibrated in a combined manner, so that measured values of different sensor coordinate systems are converted into the same coordinate system, and calibrated internal parameters and external parameters are stored, wherein the internal parameters can comprise a camera lens distortion center, a lens distortion system, an effective focal length of the camera, an image plane horizontal and vertical pixel conversion equivalent ratio system and the like; the external parameters may include a transformation matrix and a translation matrix between the camera coordinate system and the world coordinate system, and the like.
In addition, the camera, the millimeter wave radar and the laser radar can synchronously acquire data so as to realize time fusion of video data output by the camera, first structured data output by the millimeter wave radar and point cloud data output by the laser radar. For example, the sampling rate of the laser radar may be set to 10Hz/s, and in order to ensure the reliability of data, the sampling rate of the laser radar is used as a reference, and the millimeter wave radar and the data cached in a frame on the camera are selected for each frame sampled by the laser radar, so that the data of the camera, the millimeter wave radar and the laser radar, which are fused in time, of one frame can be sampled together.
In this embodiment, the video data output by the camera, the first structured data output by the millimeter wave radar and the point cloud data output by the laser radar are subjected to perception fusion processing, fusion data corresponding to the target object is determined, the target object can be identified more accurately, and the collision risk possibly occurring to the vulnerable traffic participant can be predicted more accurately, so that the collision early warning effect can be further improved.
Optionally, the number of the target objects is at least two, and the sensing fusion processing is performed on the video data output by the camera, the first structured data output by the millimeter wave radar, and the point cloud data output by the laser radar, and includes:
processing the video data output by the camera based on a target detection algorithm and a target tracking algorithm to obtain second structured data corresponding to at least two target objects;
processing the point cloud data output by the laser radar based on the trained neural network model to obtain third structured data corresponding to at least two target objects;
and performing association matching on the first structured data, the second structured data and the third structured data by adopting Hungarian algorithm, and determining fusion data corresponding to each target object.
The video data output by the camera can be processed through a target detection algorithm and a target tracking algorithm to distinguish the vulnerable traffic participants and vehicles, and the second structured data corresponding to the target object can include parameters such as object type, time, position, speed, traveling direction angle, size, longitudinal acceleration, yaw rate and the like of the target object. For example, the YOLO-V4 algorithm may be used as a target detection algorithm, and deppsort may be used as a target tracking algorithm to process video data output by the camera.
It should be noted that the millimeter wave radar can emit electromagnetic waves to an obstacle and receive echoes to accurately detect the direction and distance of a target object. The millimeter wave radar may directly output first structured data, where the first structured data may be data expressed according to a preset rule, and the first structured data may include information of an object type, a time, a speed, a position, an acceleration, and the like of the target object, and may be expressed in the following manner, for example: (object type; time; velocity; position; acceleration).
In addition, the laser radar can detect the target object by using laser to obtain real-time three-dimensional point cloud data, and the point cloud data can be used for representing information such as three-dimensional coordinates, distance, direction angles, intensity of reflected laser, laser codes and time of the target object. The point cloud data output by the laser radar is processed based on the trained neural network model, and the point cloud data output by the laser radar can be segmented, the segmented data can be clustered, the trained neural network model is adopted to reason the clustered data, and third structured data can be output. The third structured data may be data expressed according to a preset rule, and the third structured data may include information of time, object type, speed, position, and the like. By way of example, the third structured data may be represented in the form of: (time; object type; speed; location). The point cloud data of the laser radar is processed in a segmentation and clustering mode, deep learning is carried out, a neural network model is trained by marked data in advance, the point cloud data is reasoned by the trained neural network model, third structured data are output, and accurate third structured data can be obtained.
Further, after the first structured data, the second structured data and the third structured data are subjected to association matching by using the Hungarian algorithm, the first structured data, the second structured data and the third structured data corresponding to each target object can be obtained. The fused data corresponding to each target object may include the first structured data, the second structured data, and the third structured data corresponding to the target object. In order to remove redundant data, the same kind of data in the first structured data, the second structured data and the third structured data corresponding to the target object can be subjected to averaging processing, and the averaged data is used as the data in the fusion data corresponding to the target object; or, the data with the highest accuracy may be selected from the homogeneous data in the first structured data, the second structured data, and the third structured data corresponding to the target object as the data in the fused data corresponding to the target object, and the accuracy may be determined by the characteristics of the roadside sensor.
For example, the plurality of target objects may include a target object a, a target object B, and a target object C, and the first structured data, the second structured data, and the third structured data corresponding to the target object a, the first structured data, the second structured data, and the third structured data corresponding to the target object B, and the first structured data, the second structured data, and the third structured data corresponding to the target object C may be obtained by performing association matching on the first structured data, the second structured data, and the third structured data by using the hungarian algorithm.
Taking target object a as an example, the fused data of target object a includes position information, speed information, and object type information, and the first structured data, the second structured data, and the third structured data corresponding to target object a may all include position information, speed information, and object type information. Because the laser radar has stronger anti-interference capability, high resolution and high ranging accuracy, and the applicability is poorer in rainy and foggy weather, the position information in the fusion data corresponding to the target object A is the position information in the third structured data corresponding to the laser radar; the millimeter wave radar has the characteristics of low ranging precision, high penetrability, high speed perception, all weather and all day time, and is suitable for relatively severe environments, and the speed information in the fusion data corresponding to the target object A is the speed information in the first structured data corresponding to the millimeter wave radar; because the camera has strong recognition capability on the object, the object type information in the fusion data corresponding to the target object A is the object type information in the second structured data corresponding to the camera. The camera, the millimeter wave radar and the laser radar are good and bad, and data redundancy processing is performed by utilizing fusion of the three road side sensors, so that the data accuracy under different environments can be improved.
In the embodiment, the video data output by the camera is processed based on a target detection algorithm and a target tracking algorithm to obtain second structured data corresponding to at least two target objects; processing the point cloud data output by the laser radar based on the trained neural network model to obtain third structured data corresponding to at least two target objects; and performing association matching on the first structured data, the second structured data and the third structured data by adopting a Hungarian algorithm, and determining fused data corresponding to each target object. Therefore, more accurate fusion data corresponding to the target object can be obtained.
Optionally, the determining whether there is a collision risk between the vulnerable traffic participant and the vehicle according to the collected data includes:
determining the distance between the vulnerable traffic participant and the vehicle from the collected data if the vulnerable traffic participant is located within a motor vehicle lane area;
determining whether there is a collision risk between the vulnerable traffic participant and the vehicle depending on the distance between the vulnerable traffic participant and the vehicle.
If the distance between the vulnerable traffic participant and the vehicle is greater than a preset distance, determining that the vulnerable traffic participant and the vehicle have a collision risk; if the distance between the vulnerable traffic participant and the vehicle is less than or equal to a preset distance, it may be determined that there is no collision risk between the vulnerable traffic participant and the vehicle. The preset distance may be 300 meters, 500 meters, or 800 meters, and the like, which is not limited in this embodiment.
In addition, the vulnerable traffic participants can be accurately identified and tracked through the cameras, the millimeter wave radar and the laser radar, as shown in fig. 3, when the vulnerable traffic participants move into the motor vehicle lane area 11, the vulnerable traffic participants can be judged as potential threat vulnerable traffic participants, distances between the vulnerable traffic participants and a plurality of vehicles can be calculated, and if the distances between the vulnerable traffic participants and the vehicles are less than 500 meters, the vulnerable traffic participants and the vehicles can be considered to have collision risks, so that all the vulnerable traffic participants with collision risks can be screened out.
It should be noted that the determining the distance between the vulnerable traffic participant and the vehicle according to the collected data may include performing perceptual fusion processing on the collected data of the target object by the at least one roadside sensor, determining fusion data corresponding to the target object, and determining the distance between the vulnerable traffic participant and the vehicle according to the fusion data corresponding to the target object. The fusion data corresponding to the target object may include positions of the vulnerable traffic participants and the vehicle, so that the distance between the vulnerable traffic participants and the vehicle may be determined according to the positions of the vulnerable traffic participants and the vehicle.
In the embodiment, collision early warning analysis is only carried out on the vulnerable traffic participants running the motor vehicle lane, the problems of excessive early warning and calculation burden for checking intensive pedestrians one by one are properly considered, excessive early warning is not carried out on running vehicles so as to avoid unnecessary interference, and the safety positioning of traffic facilities is met.
In the embodiment, the collision early warning analysis is carried out on the weak traffic participants running the motor vehicle lane, so that the weak traffic participants with collision danger can be screened out relatively quickly, and the collision early warning effect can be improved.
Optionally, the determining whether there is a collision risk between the vulnerable traffic participant and the vehicle according to the collected data includes:
under the condition that a traffic signal lamp indicates that the vulnerable traffic participant is not allowed to be located in a lane crossing area, if the vulnerable traffic participant is located in the lane crossing area, predicting a moving path of the vulnerable traffic participant and a moving path of the vehicle according to the collected data;
determining whether the vulnerable traffic participant and the vehicle have collision danger according to the predicted moving path of the vulnerable traffic participant and the moving path of the vehicle.
Wherein, the traffic signal lamp can be a traffic light. As shown in fig. 4, the traffic light indicates that the vulnerable traffic participant is not allowed to be located within the lane crossing area 12, and the vulnerable traffic participant located within the lane crossing area may be considered as a vulnerable traffic participant running a red light. The lane crossing region may be a region enclosed by a plurality of lane stop lines. The weak traffic participants can be accurately identified and tracked through the cameras, the millimeter wave radar and the laser radar, and can be judged as potential threat weak traffic participants when the weak traffic participants run red light. The historical path can be restored through the information such as the positions, the traveling direction angles, the four-dimensional acceleration and the like of the vulnerable traffic participants and the vehicles at various times, and the predicted moving paths of the vulnerable traffic participants and the moving paths of the vehicles are deduced, so that all the vulnerable traffic participants with collision risks can be screened out. The predicted collision time between the vulnerable traffic participants and the vehicle can be calculated according to the predicted moving path, so that the vulnerable traffic participants with the highest early warning emergency degree can be screened out.
If the predicted movement path of the vulnerable traffic participant and the movement path of the vehicle have an intersection, it may be determined that the vulnerable traffic participant and the vehicle have a collision risk; if the predicted movement path of the vulnerable traffic participant does not have an intersection with the movement path of the vehicle, it may be determined that there is no risk of collision between the vulnerable traffic participant and the vehicle.
The predicting the movement path of the vulnerable traffic participant and the movement path of the vehicle according to the collected data may include performing perceptual fusion processing on the collected data of the target object by the at least one roadside sensor, determining fusion data corresponding to the target object, and predicting the movement path of the vulnerable traffic participant and the movement path of the vehicle according to the fusion data corresponding to the target object. The fusion data corresponding to the target object may include information such as positions, traveling direction angles, four-dimensional accelerations of the vulnerable traffic participants and the vehicle, so that the movement paths of the vulnerable traffic participants and the vehicle may be predicted according to the information such as the positions, the traveling direction angles, and the four-dimensional accelerations of the vulnerable traffic participants and the vehicle.
In the embodiment, collision early warning analysis is only carried out on the vulnerable traffic participants running the red light, the problems of excessive early warning and calculation burden for checking the intensive pedestrians one by one are properly considered, excessive early warning is not carried out on the running vehicles so as to avoid unnecessary interference, and the safety positioning of traffic facilities is met.
In the embodiment, the collision early warning analysis is carried out on the weak traffic participants running the red light, so that the weak traffic participants with collision danger can be screened out relatively quickly, and the collision early warning effect can be improved.
Optionally, the number of the vulnerable traffic participants is multiple, and the sending of the warning information to the vehicle when the vulnerable traffic participants and the vehicle have a collision risk includes:
determining a collision risk degree of each of the participants of the weak traffic with the vehicle in the case that a plurality of the participants of the weak traffic and the vehicle are in collision risk;
sequencing the early warning sequences of the plurality of vulnerable traffic participants according to the collision danger degree to obtain a sequencing result;
and sequentially sending early warning information to the vehicles according to the sequencing result.
Wherein the degree of risk of collision of each of the participants of vulnerable traffic with the vehicle may be determined based on the predicted time of collision of each of the participants of vulnerable traffic with the vehicle; alternatively, the degree of risk of collision of each of the participants of vulnerable traffic with the vehicle may be determined based on the probability value of the expected collision of each of the participants of vulnerable traffic with the vehicle; etc., which are not limited by the present embodiment.
In addition, the early warning sequence of the plurality of weak traffic participants is sequenced according to the collision risk degree, the early warning sequence of the weak traffic participants is sequenced in the front when the collision risk degree of the weak traffic participants is higher, and therefore early warning information of the weak traffic participants with higher collision risk degree can be sent preferentially.
In the embodiment, the early warning sequences of the plurality of vulnerable traffic participants are sequenced according to the collision danger degrees to obtain sequencing results, and early warning information is sequentially sent to the vehicle according to the sequencing results, so that the vehicle can avoid the vulnerable traffic participants in time according to the early warning information, and traffic accidents are avoided.
Optionally, the degree of risk of collision of each of the participants of vulnerable transportation with the vehicle is determined based on the predicted time of collision of each of the participants of vulnerable transportation with the vehicle.
The relative speed can be calculated according to the current speed and the advancing direction angle of each vulnerable traffic participant and the current speed and the advancing direction angle of the vehicle, and the distance between the vulnerable traffic participant and the vehicle is divided by the relative speed to obtain the predicted collision time between the vulnerable traffic participant and the vehicle. The smaller the predicted collision time between the vulnerable traffic participants and the vehicle is, the higher the early warning sequence of the vulnerable traffic participants can be. The closer the predicted time-to-collision is to the current time, the smaller the predicted time-to-collision can be considered.
For example, the relative speed may be calculated according to the current speed and the travel direction angle of the pedestrian or non-motor vehicle A1 and the current speed and the travel direction angle of the vehicle B1, the predicted time to collision may be calculated according to the distance between A1 and B1, and so on, the predicted time to collision of the vehicles A1 and B2, B3, etc. and the predicted time to collision of the pedestrian or non-motor vehicle A2 and the vehicles B1, B2, B3, etc. may be sorted from low to high according to the predicted time to collision. And determining the early warning sequence of the vulnerable traffic participants according to the predicted collision time from small to large, for example, for the vehicle B1, if the predicted collision time of the A1 is less than the predicted collision time of the A2, sending early warning information aiming at the A1 to the vehicle B1, and then sending early warning information aiming at the A2 to the vehicle B1.
In this embodiment, the collision risk level of each of the participants of vulnerable transportation with the vehicle is determined based on the predicted collision time of each of the participants of vulnerable transportation with the vehicle, so that the collision risk level of each of the participants of vulnerable transportation with the vehicle can be determined more accurately.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a collision warning apparatus according to an embodiment of the present invention, and as shown in fig. 5, a collision warning apparatus 200 includes:
an obtaining module 201, configured to obtain data collected by at least one roadside sensor for a target object, where the target object includes a vulnerable traffic participant and a vehicle;
a determining module 202, configured to determine whether there is a collision risk between the vulnerable traffic participant and the vehicle according to the collected data;
the sending module 203 is configured to send early warning information to the vehicle when the vulnerable traffic participant and the vehicle are in a collision risk.
Optionally, the at least one roadside sensor includes a camera, a millimeter wave radar, and a laser radar;
the collected data comprises video data output by the camera, first structured data output by the millimeter wave radar and point cloud data output by the laser radar;
as shown in fig. 6, the determining module 202 includes:
the fusion unit 2021 is configured to perform perceptual fusion processing on the video data output by the camera, the first structured data output by the millimeter wave radar, and the point cloud data output by the laser radar, and determine fusion data corresponding to the target object;
a determining unit 2022, configured to determine whether there is a collision risk between the vulnerable traffic participant and the vehicle according to the fusion data corresponding to the target object.
Optionally, the number of the target objects is at least two, and the fusion unit 2021 is specifically configured to:
processing the video data output by the camera based on a target detection algorithm and a target tracking algorithm to obtain second structured data corresponding to at least two target objects;
processing the point cloud data output by the laser radar based on the trained neural network model to obtain third structured data corresponding to at least two target objects;
and performing association matching on the first structured data, the second structured data and the third structured data by adopting a Hungarian algorithm, and determining fused data corresponding to each target object.
Optionally, the determining module 202 is specifically configured to:
determining the distance between the vulnerable traffic participant and the vehicle from the collected data if the vulnerable traffic participant is located within a motor vehicle lane area;
determining whether there is a collision risk between the vulnerable traffic participant and the vehicle depending on the distance between the vulnerable traffic participant and the vehicle.
Optionally, the determining module 202 is specifically configured to:
under the condition that a traffic signal lamp indicates that the vulnerable traffic participant is not allowed to be located in a lane crossing area, if the vulnerable traffic participant is located in the lane crossing area, predicting a moving path of the vulnerable traffic participant and a moving path of the vehicle according to the collected data;
determining whether the vulnerable traffic participant and the vehicle have collision danger according to the predicted moving path of the vulnerable traffic participant and the moving path of the vehicle.
Optionally, the number of the vulnerable traffic participants is multiple, and the sending module 203 is specifically configured to:
in the case that a plurality of the vulnerable traffic participants and the vehicle are in collision danger, determining the collision danger degree of each of the vulnerable traffic participants and the vehicle;
sequencing the early warning sequences of the plurality of vulnerable traffic participants according to the collision danger degree to obtain a sequencing result;
and sequentially sending early warning information to the vehicles according to the sequencing result.
Optionally, the degree of risk of collision of each said participant of vulnerable traffic with said vehicle is determined based on the predicted time of collision of each said participant of vulnerable traffic with said vehicle.
The collision early warning device can realize each process realized in the method embodiment of fig. 1, and can achieve the same technical effect, and is not described here again to avoid repetition.
As shown in fig. 7, an embodiment of the present invention further provides a roadside apparatus 300, including: the collision warning method includes a processor 301, a memory 302, and a program stored in the memory 302 and capable of running on the processor 301, where the program is executed by the processor 301 to implement the processes of the collision warning method embodiment, and can achieve the same technical effect, and is not described herein again to avoid repetition.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the above-mentioned collision warning method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here. The computer readable storage medium is, for example, ROM, RAM, magnetic disk or optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A collision early warning method is characterized by being applied to roadside equipment and comprising the following steps:
acquiring collected data of at least one roadside sensor for a target object, wherein the target object comprises vulnerable traffic participants and vehicles;
determining whether the vulnerable traffic participant and the vehicle have collision danger according to the collected data;
and sending early warning information to the vehicle under the condition that the vulnerable traffic participant and the vehicle have collision danger.
2. The method of claim 1, wherein the at least one roadside sensor comprises a camera, a millimeter wave radar, and a lidar;
the collected data comprises video data output by the camera, first structured data output by the millimeter wave radar and point cloud data output by the laser radar;
the determining whether the vulnerable traffic participant and the vehicle have a collision risk according to the collected data comprises:
sensing and fusing the video data output by the camera, the first structured data output by the millimeter wave radar and the point cloud data output by the laser radar, and determining fused data corresponding to the target object;
and determining whether the vulnerable traffic participant and the vehicle have collision danger or not according to the fusion data corresponding to the target object.
3. The method according to claim 2, wherein the number of the target objects is at least two, and the perceptual fusion processing of the video data output by the camera, the first structured data output by the millimeter wave radar, and the point cloud data output by the lidar includes:
processing the video data output by the camera based on a target detection algorithm and a target tracking algorithm to obtain second structured data corresponding to at least two target objects;
processing the point cloud data output by the laser radar based on the trained neural network model to obtain third structured data corresponding to at least two target objects;
and performing association matching on the first structured data, the second structured data and the third structured data by adopting Hungarian algorithm, and determining fusion data corresponding to each target object.
4. The method of claim 1, wherein said determining from said collected data whether said vulnerable traffic participant is in danger of colliding with said vehicle comprises:
determining the distance between the vulnerable traffic participant and the vehicle from the collected data if the vulnerable traffic participant is located within a motor vehicle lane area;
determining whether there is a collision risk between the vulnerable traffic participant and the vehicle depending on the distance between the vulnerable traffic participant and the vehicle.
5. The method of claim 1, wherein said determining from said collected data whether said vulnerable traffic participant is in danger of colliding with said vehicle comprises:
under the condition that a traffic signal lamp indicates that the weak traffic participant is not allowed to be located in a lane crossing area, if the weak traffic participant is located in the lane crossing area, predicting a moving path of the weak traffic participant and a moving path of the vehicle according to the collected data;
determining whether the vulnerable traffic participant and the vehicle have collision danger according to the predicted moving path of the vulnerable traffic participant and the moving path of the vehicle.
6. The method according to any one of claims 4 to 5, wherein the number of the vulnerable traffic participants is multiple, and the sending of the early warning information to the vehicle in the case that the vulnerable traffic participants are in collision danger with the vehicle comprises:
in the case that a plurality of the vulnerable traffic participants and the vehicle are in collision danger, determining the collision danger degree of each of the vulnerable traffic participants and the vehicle;
sequencing the early warning sequences of the plurality of vulnerable traffic participants according to the collision danger degree to obtain a sequencing result;
and sequentially sending early warning information to the vehicles according to the sequencing result.
7. The method of claim 6, wherein the degree of risk of collision of each of the participants of vulnerable traffic with the vehicle is determined based on an expected time of collision of each of the participants of vulnerable traffic with the vehicle.
8. A collision early warning device is characterized in that the collision early warning device is applied to roadside equipment, and the device comprises:
the system comprises an acquisition module, a data acquisition module and a data acquisition module, wherein the acquisition module is used for acquiring the acquired data of at least one roadside sensor aiming at a target object, and the target object comprises vulnerable traffic participants and vehicles;
the determining module is used for determining whether the vulnerable traffic participant and the vehicle have collision danger according to the collected data;
and the sending module is used for sending early warning information to the vehicle under the condition that the vulnerable traffic participant and the vehicle have collision danger.
9. A roadside apparatus characterized by comprising: processor, memory and program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the collision warning method according to any one of claims 1 to 7.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the collision warning method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110896048.5A CN115909806A (en) | 2021-08-05 | 2021-08-05 | Collision early warning method and device and road side equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110896048.5A CN115909806A (en) | 2021-08-05 | 2021-08-05 | Collision early warning method and device and road side equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115909806A true CN115909806A (en) | 2023-04-04 |
Family
ID=86482597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110896048.5A Pending CN115909806A (en) | 2021-08-05 | 2021-08-05 | Collision early warning method and device and road side equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115909806A (en) |
-
2021
- 2021-08-05 CN CN202110896048.5A patent/CN115909806A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108417087B (en) | Vehicle safe passing system and method | |
| CN107161141B (en) | Unmanned automobile system and automobile | |
| CN113490863B (en) | Radar-assisted single image three-dimensional depth reconstruction | |
| CN107346612B (en) | Vehicle anti-collision method and system based on Internet of vehicles | |
| CN112558023B (en) | Calibration method and device of sensor | |
| CN112215306B (en) | Target detection method based on fusion of monocular vision and millimeter wave radar | |
| CN111461088B (en) | Rail transit obstacle avoidance system based on image processing and target recognition | |
| CN112216097A (en) | Method and device for detecting blind area of vehicle | |
| CN113706737B (en) | Road surface inspection system and method based on automatic driving vehicle | |
| CN112693466A (en) | System and method for evaluating performance of vehicle environment perception sensor | |
| WO2022184127A1 (en) | Simulation method and apparatus for vehicle and sensor | |
| CN110065494A (en) | A kind of vehicle collision avoidance method based on wheel detection | |
| CN111243274A (en) | Road collision early warning system and method for non-internet traffic individuals | |
| CN110441790A (en) | Method and apparatus crosstalk and multipath noise reduction in laser radar system | |
| CN113012445A (en) | Intelligent traffic control system and control method thereof | |
| CN112379674B (en) | Automatic driving equipment and system | |
| CN111252066A (en) | Emergency braking control method and device, vehicle and storage medium | |
| CN111477010A (en) | Device for intersection holographic sensing and control method thereof | |
| CN114419874B (en) | Target driving safety risk early warning method based on road side sensing equipment data fusion | |
| CN115257784A (en) | Vehicle-road cooperative system based on 4D millimeter wave radar | |
| CN113665570A (en) | Method and device for automatically sensing driving signal and vehicle | |
| RU2763800C1 (en) | Onboard sensor system | |
| CN110816527A (en) | Vehicle-mounted night vision safety method and system | |
| CN112784679A (en) | Vehicle obstacle avoidance method and device | |
| CN111105644A (en) | Vehicle blind area monitoring and driving control method and device and vehicle road cooperative system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |