CN116129641B - Vehicle security situation calculation method and system based on multi-terminal collaborative identification - Google Patents
Vehicle security situation calculation method and system based on multi-terminal collaborative identification Download PDFInfo
- Publication number
- CN116129641B CN116129641B CN202310107673.6A CN202310107673A CN116129641B CN 116129641 B CN116129641 B CN 116129641B CN 202310107673 A CN202310107673 A CN 202310107673A CN 116129641 B CN116129641 B CN 116129641B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- information
- sensing device
- road
- driver
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 73
- 238000000034 method Methods 0.000 claims abstract description 19
- 230000006854 communication Effects 0.000 claims abstract description 9
- 238000004891 communication Methods 0.000 claims abstract description 9
- 230000001133 acceleration Effects 0.000 claims description 17
- 230000008447 perception Effects 0.000 claims description 17
- 230000010365 information processing Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 13
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 238000011156 evaluation Methods 0.000 claims description 8
- 230000036544 posture Effects 0.000 claims description 7
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 claims description 6
- 239000008280 blood Substances 0.000 claims description 6
- 210000004369 blood Anatomy 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 6
- 230000005611 electricity Effects 0.000 claims description 6
- 230000008921 facial expression Effects 0.000 claims description 6
- 229910052760 oxygen Inorganic materials 0.000 claims description 6
- 239000001301 oxygen Substances 0.000 claims description 6
- 238000013468 resource allocation Methods 0.000 claims description 6
- 206010039203 Road traffic accident Diseases 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 230000029058 respiratory gaseous exchange Effects 0.000 claims description 3
- 230000036387 respiratory rate Effects 0.000 claims description 3
- 210000000707 wrist Anatomy 0.000 claims description 3
- 230000008569 process Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 3
- 239000002699 waste material Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000008451 emotion Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000007175 bidirectional communication Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000002864 sequence alignment Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0108—Measuring and analyzing of parameters relative to traffic conditions based on the source of data
- G08G1/0112—Measuring and analyzing of parameters relative to traffic conditions based on the source of data from the vehicle, e.g. floating car data [FCD]
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0108—Measuring and analyzing of parameters relative to traffic conditions based on the source of data
- G08G1/0116—Measuring and analyzing of parameters relative to traffic conditions based on the source of data from roadside infrastructure, e.g. beacons
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0108—Measuring and analyzing of parameters relative to traffic conditions based on the source of data
- G08G1/012—Measuring and analyzing of parameters relative to traffic conditions based on the source of data from other sources than vehicle or roadside beacons, e.g. mobile networks
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/052—Detecting movement of traffic to be counted or controlled with provision for determining speed or overspeed
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a vehicle security situation calculation method and system based on multi-terminal collaborative identification, which are used in a system in which a vehicle-mounted sensing device, a vehicle-mounted security situation calculation device connected with the vehicle-mounted sensing device, a plurality of road side sensing devices connected with the vehicle-mounted sensing device and the vehicle-mounted security situation calculation device in a wireless communication mode and a cloud sensing device connected with the road side sensing devices in a two-way communication mode are mutually cooperated. The method and the system provided by the invention realize the running risk estimation in the intelligent interconnection traffic scene, ensure the perceived real-time performance and the accuracy of the safety situation judgment, and greatly enhance the running safety of the intelligent vehicle.
Description
Technical Field
The invention relates to the field of vehicle safety and intelligent interconnection traffic, in particular to a vehicle safety situation calculation method and system based on multi-terminal collaborative identification.
Background
With the intelligent level of propulsion of vehicles, the perceived demands of vehicles on their own and on the state of the operating environment are increasing. In the existing advanced driving assistance system, the perception of the bad state of a person and the operation environment with a relatively small range outside the vehicle is mainly focused, however, objective judgment on the driving safety cannot be made only by the perception of the sensor arranged on the vehicle; in the existing judging method aiming at the running risk of the vehicle, the fusion degree of various characteristic information of a driver, the vehicle and the running environment is weaker, a plurality of relatively independent systems are often used for judging and early warning respectively, and along with the development of automatic driving, the characteristic information obtained by the vehicle through perception determines the running decision control of the vehicle, so that the improvement of the perception approach richness of the vehicle and the effective fusion of perception parameters are indispensable requirements for the further development of the man-machine co-driving automatic driving. Meanwhile, in the running process of the intelligent vehicle at present, the calculation power resources of the vehicle-mounted calculation equipment and other calculation paths in the intelligent running system of the vehicle are unevenly distributed, single-end calculation power shortage often occurs, and more repeated calculation exists, so that high-accuracy safety situation assessment and driving path decision planning are difficult to achieve in the intelligent running process of the vehicle.
Therefore, the intelligent vehicle perception approach richness is improved, perception parameters are effectively fused, and meanwhile, more accurate security situation assessment and driving path decision planning are carried out by utilizing a calculation mode of combining auxiliary calculation and vehicle-mounted calculation of other approaches.
Disclosure of Invention
The invention provides a vehicle security situation calculation method and system based on multi-terminal collaborative identification, which are used for solving the technical problems of insufficient sensing range capability of the existing intelligent vehicle, uneven distribution of computing resources at each terminal in a vehicle-road-cloud system and low utilization efficiency.
In order to achieve the above object, the present invention provides a vehicle security situation calculating method based on multi-terminal collaborative recognition, which is used in a system in which a vehicle-mounted sensing device, a vehicle-mounted security situation calculating device, a plurality of road side sensing devices and a cloud sensing device are mutually collaborative, and comprises the following steps:
S1, information is collected: the method comprises the steps of collecting multi-mode information of a driver through a vehicle-mounted sensing device, generating characteristic parameters of the driver through calculation, and sending the characteristic parameters of the driver to a vehicle-mounted safety situation calculating device; collecting vehicle external environment information and vehicle operation parameter information through a vehicle-mounted sensing device, and sending the vehicle external environment information and the vehicle operation parameter information to a road side sensing device in a vehicle adjacent range; collecting road network environment information through a road side sensing device and pre-storing road network road information in an area; the cloud sensing device pre-stores and updates the high-precision map information.
S2, processing information: after receiving the external environment information of the vehicle and the running parameter information of the vehicle, the road side sensing device recognizes and ignores repeated information in the road network environment information, the road network road information, the external environment information of the vehicle and the running parameter information of the vehicle; and after the computing resource allocation is carried out, computing is carried out, and data exceeding the computing tolerance are sent to the cloud sensing device.
S3, generating a risk factor package: the cloud sensing device receives information from the road side sensing device and calculates, and the calculated data is returned to the road side sensing device; the road side sensing device receives the information returned by the cloud sensing device, generates a risk factor data packet and sends the risk factor data packet to the vehicle-mounted security situation calculating device.
S4, carrying out security situation assessment and driving decision planning: and after receiving the characteristic parameters and the risk factor data packet of the driver, the vehicle-mounted safety situation calculation device evaluates the safety situation of the vehicle according to the characteristic parameters and the risk factor data packet of the driver, performs driving decision planning according to an evaluation result after the evaluation is finished, and judges whether to give an early warning to the driver or not.
Preferably, in S1, the multimodal information includes facial expression, head pose, body pose, heart rate, blood oxygen saturation, skin electricity, respiration rate, and voice intonation information of the driver; the vehicle external environment information comprises two-dimensional images and three-dimensional Lei Dadian cloud information in a detectable range of the vehicle-mounted sensing device; the vehicle operation parameter information comprises speed, acceleration, steering wheel rotation angle, positioning information, accelerator pedal opening and brake pedal opening information of the vehicle; the road network environment information comprises road network image information and road network three-dimensional radar point cloud information.
The road network and road information in the area pre-stored by the road side sensing device comprises lane information, road connection relation, intersection information and road section information; the high-precision map information pre-stored and updated by the cloud sensing device comprises road congestion conditions, construction conditions, traffic accident conditions, weather conditions, traffic lights, crosswalk and speed limiting conditions.
Preferably, when the road side sensing device calculates the acquired data, the data is input into YOLOv depth neural network, so as to obtain the position, speed and acceleration information of the risk factor vehicles, pedestrians and obstacles in the current road network.
The cloud sensing device calculates the data sent by the road side sensing device to obtain risk factor information in each road network area, and returns coordinate, speed and acceleration parameter information in the risk factor information to the road side sensing device.
Preferably, in S4, when the vehicle-mounted security situation calculation device performs the vehicle security situation assessment, the vehicle running security situation E is:
Wherein, alpha i、βj、γk is a preset parameter larger than 0, which represents the weight of each risk factor, V i represents the risk factor of the external factor, D j represents the risk factors of different states of the driver, and R k represents the risk factor of the road.
When the vehicle-mounted safety situation calculation device performs driving decision planning, a vehicle path planning set is searched according to a safety situation evaluation result, the optimal planning path in the set is continuously adjusted based on a self-learning method and displayed to a driver, and the vehicle planning path set P is as follows:
P＝{δ|εI,EI,θ,εO,EO}
Where δ represents the path planning set of the vehicle, ε I represents the historical track of the vehicle, E I represents the safety situation of the vehicle, θ represents the driving decision of the driver, ε O represents the historical track of the surrounding vehicle, and E O represents the safety situation of the surrounding vehicle.
The invention also provides a vehicle security situation calculation system based on multi-terminal collaborative identification, which comprises: the system comprises a vehicle-mounted sensing device, a vehicle-mounted safety situation calculating device connected with the vehicle-mounted sensing device, a road side sensing device connected with the vehicle-mounted sensing device and the vehicle-mounted safety situation calculating device in a wireless communication mode and a cloud sensing device connected with the road side sensing device in a two-way communication mode.
The vehicle-mounted sensing device is used for collecting multi-mode information of the driver, calculating characteristic parameters of the driver and sending the characteristic parameters of the driver to the vehicle-mounted safety situation calculating device.
The road side sensing device is used for collecting road network environment information, processing self information and information sent by other devices, sending information outside self-calculation bearing capacity to the cloud sensing device and sending a risk factor data packet to the vehicle-mounted security situation calculating device.
The cloud sensing device is used for calculating information from the road side sensing device and returning a result to the road side sensing device.
The vehicle-mounted safety situation calculation device is used for carrying out vehicle safety situation assessment and driving decision planning according to the characteristic parameters of the driver and the risk factor data packet.
Preferably, the road side sensing device is further used for pre-storing road network road information in the area; the cloud sensing device is also used for pre-storing and updating the high-precision map information.
Preferably, the vehicle-mounted sensing device comprises a vehicle-mounted driver sensing module, a vehicle-mounted vehicle sensing module, a vehicle-mounted environment sensing module and a vehicle-mounted information processing module.
The vehicle-mounted driver perception module comprises: the high-definition network cameras are arranged right in front of the rearview mirror in the vehicle and above the rearview mirror in the vehicle, and are used for collecting facial expressions, head postures and body postures of the driver; an audio collector for collecting voiceprint signals and voice tones of a driver; and the physiological signal acquisition bracelet is worn on the wrist of the driver or embedded in the steering wheel and is used for acquiring the heart rate, the respiratory rate, the skin electricity and the blood oxygen saturation of the driver.
The vehicle-mounted vehicle sensing module includes: speed sensors for acquiring the speed of the vehicle, respectively; an acceleration sensor for acquiring acceleration of the vehicle; the steering angle sensor is used for collecting the steering angle of the steering wheel of the vehicle; the GPS locator is used for collecting vehicle positioning information; an accelerator pedal opening sensor for acquiring an opening of an accelerator pedal of a vehicle; and the brake pedal opening sensor is used for acquiring the opening information of the brake pedal of the vehicle.
The vehicle-mounted environment sensing module comprises a high-definition network camera, an infrared camera, a laser radar and a millimeter wave radar, wherein the high-definition network camera, the infrared camera, the laser radar and the millimeter wave radar are arranged on the outer body of the vehicle and used for acquiring two-dimensional images and three-dimensional Lei Dadian cloud information outside the vehicle.
The vehicle-mounted information processing module comprises at least one edge computing device for receiving the multi-mode information from the vehicle-mounted driver perception module, processing the multi-mode information into driver characteristic parameters through a multi-task neural network, sending the driver characteristic parameters to the vehicle-mounted security situation computing device and sending the vehicle external environment information and the vehicle operation parameter information to the road side perception device.
Preferably, the vehicle-mounted security situation calculation device comprises a security situation calculation module and a vehicle path planning module.
The safety situation calculation module is used for receiving the characteristic parameters of the driver from the vehicle-mounted sensing device and the risk factor data packet from the road side sensing device, so as to calculate the safety situation.
The vehicle path planning module is used for receiving the safety situation information from the safety situation calculation module, further carrying out path planning, and displaying and high-risk early warning on the driver.
Preferably, the road side sensing device comprises a road side environment sensing module and a road side information processing module.
The road side environment sensing module comprises a camera and a detection radar, wherein the camera and the detection radar are arranged on the road side sensing device and used for acquiring road network image information and road network three-dimensional radar point cloud information under the current road network.
The road side information processing module comprises a plurality of integrated computing devices which are arranged in the road side sensing device and used for pre-storing road network information in a current area, receiving road network image information and road network three-dimensional radar point cloud information from the road side environment sensing module, receiving vehicle external environment information and vehicle operation parameter information sent by vehicles in a range, performing data check and neglect, performing calculation through YOLOv depth neural network after computing resource allocation, sending data exceeding a computing tolerance to the cloud sensing device, receiving return information of the cloud sensing device, generating a risk factor data packet and sending the risk factor data packet to the vehicle-mounted security situation computing device.
Preferably, the cloud sensing device comprises a cloud information processing module for pre-storing and updating high-precision map information, processing information from the road side sensing device, and integrating the processed data with a plurality of high-performance calculations of the road environment state return road environment state beyond the line-of-sight wide area to the road side sensing device.
The invention has the following beneficial effects:
According to the vehicle security situation calculation method based on multi-terminal collaborative identification, through integrating the sensing capabilities of the vehicle-mounted sensing device, the road side sensing device and the cloud sensing device, the intelligent vehicle can obtain risk factor information in the sight distance and under the beyond sight distance, the sensing route richness of the intelligent vehicle is improved, sensing parameters are effectively fused, risk sensing and risk judging in the driving process are more comprehensive and accurate, and the driving security of the intelligent vehicle is greatly enhanced; the road side sensing device is used for checking and repeating each data and ignoring the repeated data, so that the waste of calculation resources is avoided, and the situation of insufficient calculation capacity is avoided through the dynamic allocation of the calculation capacities of the vehicle-mounted sensing device, the road side sensing device and the cloud sensing device, so that the safety situation assessment and the driving path decision planning in the intelligent driving process have higher accuracy and instantaneity. The vehicle security situation calculation system based on multi-terminal collaborative identification provides a hardware basis for realizing the method, so that the vehicle security situation calculation system has the same beneficial effects as the method.
In addition to the objects, features and advantages described above, the present invention has other objects, features and advantages. The invention will be described in further detail with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
FIG. 1 is a flow chart of a vehicle security situation calculation method based on multi-terminal collaborative recognition according to a preferred embodiment 1 of the present invention;
fig. 2 is a vehicle security situation calculation frame diagram of a preferred embodiment 1 of the present invention;
Fig. 3 is a block diagram of a vehicle security situation calculation system based on multi-terminal collaborative recognition according to a preferred embodiment 2 of the present invention.
Detailed Description
Embodiments of the invention are described in detail below with reference to the attached drawings, but the invention can be implemented in a number of different ways, which are defined and covered by the claims.
Example 1:
Referring to fig. 1 to 2, in a preferred embodiment of the present invention, a vehicle security situation calculating method based on multi-terminal collaborative recognition is provided, which is used in a system in which a vehicle-mounted sensing device, a vehicle-mounted security situation calculating device connected with the vehicle-mounted sensing device, a plurality of road side sensing devices connected with the vehicle-mounted sensing device and the vehicle-mounted security situation calculating device in a wireless communication manner, and a cloud sensing device connected with the road side sensing devices in a bidirectional communication manner are mutually cooperated, and includes the following steps:
S1, information is collected: the method comprises the steps of collecting multi-mode information of a driver through a vehicle-mounted sensing device, generating characteristic parameters of the driver through calculation, and sending the characteristic parameters of the driver to a vehicle-mounted safety situation calculating device; collecting vehicle external environment information and vehicle operation parameter information through a vehicle-mounted sensing device, and sending the vehicle external environment information and the vehicle operation parameter information to a road side sensing device in a vehicle adjacent range; collecting road network environment information through a road side sensing device and pre-storing road network road information in an area; the cloud sensing device pre-stores and updates the high-precision map information.
In S1, the multimodal information includes facial expression, head pose, body pose, heart rate, blood oxygen saturation, skin electricity, respiration rate, and voice intonation information of the driver; the vehicle external environment information comprises two-dimensional images and three-dimensional Lei Dadian cloud information in a detectable range of the vehicle-mounted sensing device; the vehicle operation parameter information comprises speed, acceleration, steering wheel rotation angle, positioning information, accelerator pedal opening and brake pedal opening information of the vehicle; the road network environment information comprises road network image information and road network three-dimensional radar point cloud information.
The road network and road information in the area pre-stored by the road side sensing device comprises lane information, road connection relation, intersection information and road section information; the high-precision map information pre-stored and updated by the cloud sensing device comprises road congestion conditions, construction conditions, traffic accident conditions, weather conditions, traffic lights, crosswalk and speed limiting conditions.
In the preferred embodiment of the invention, a plurality of sensing devices are adopted to collect various physiological information and external expression information of a driver, the state behavior of the driver is identified through a multitask convolutional neural network so as to obtain characteristic parameters of the driver, and the external environment information of the vehicle, the running parameter information of the vehicle and the road network environment information are collected, namely, risk factor information of the driver, the driver in sight distance and beyond sight distance is collected, so that the data collection quality is ensured, and the comprehensiveness of data collection is ensured.
Road network road information in an area is pre-stored in the road side sensing device, and high-precision map information is pre-stored and updated in the cloud sensing device, so that timeliness of existing data is guaranteed, and the method has instantaneity when applied.
S2, processing information: after receiving the external environment information of the vehicle and the running parameter information of the vehicle, the road side sensing device recognizes and ignores repeated information in the road network environment information, the road network road information, the external environment information of the vehicle and the running parameter information of the vehicle; and after the computing resource allocation is carried out, computing is carried out, and data exceeding the computing tolerance are sent to the cloud sensing device.
When the road side sensing device calculates the acquired data, the data is input into YOLOv depth neural network, so as to obtain the position, speed and acceleration information of the risk factor vehicles, pedestrians and obstacles in the current road network.
In the preferred embodiment of the invention, the road side sensing device is used for checking the repetition of each data and ignoring the repeated data, the sensing characteristic information of the vehicle, the road side and the cloud is fused, the advantages of different ends are utilized, the computing resources of different ends are communicated with each other and distributed effectively, the repeated environmental information is prevented from being computed by different vehicle end devices, the waste of computing resources is reduced, and the condition of insufficient computing capacity is avoided.
S3, generating a risk factor package: the cloud sensing device receives information from the road side sensing device and calculates, and the calculated data is returned to the road side sensing device; the road side sensing device receives the information returned by the cloud sensing device, generates a risk factor data packet and sends the risk factor data packet to the vehicle-mounted security situation calculating device.
The cloud sensing device calculates the data sent by the road side sensing device to obtain risk factor information in each road network area, and returns coordinate, speed and acceleration parameter information in the risk factor information to the road side sensing device, so that large-capacity information such as pictures, videos and point cloud images is not transmitted.
In the preferred embodiment of the invention, the cloud sensing device shares the calculation pressure, so that the condition of insufficient calculation capability is avoided, the critical information of the risk factors is preferentially transmitted during data transmission, the data volume is reduced as much as possible, and the real-time performance and the stability of the data interaction between the devices are ensured.
S4, carrying out security situation assessment and driving decision planning: and after receiving the characteristic parameters and the risk factor data packet of the driver, the vehicle-mounted safety situation calculation device evaluates the safety situation of the vehicle according to the characteristic parameters and the risk factor data packet of the driver, performs driving decision planning according to an evaluation result after the evaluation is finished, and judges whether to give an early warning to the driver or not.
In S4, when the vehicle-mounted security situation calculation device performs the vehicle security situation assessment, the vehicle operation security situation E is:
Wherein α i、βj、γk is a preset parameter greater than 0, and represents a weight of each risk factor, V i represents an external factor risk factor such as an adjacent vehicle, a pedestrian, an obstacle, and the like, D j represents a risk factor of different states of the driver such as fatigue, emotion, operation behavior, and the like, and P k represents a road risk factor such as a road curvature, a road gradient, a road visibility, a road flatness, an intersection traffic light, and the like.
When the vehicle-mounted safety situation calculation device performs driving decision planning, searching a vehicle path planning set according to a safety situation evaluation result, continuously adjusting the optimal planning path in the set based on a self-learning method, displaying the optimal planning path to a driver, and performing early warning operation when judging that the risk is large; the vehicle planning path set P is:
P＝{δ|εI,EI,θ,εO,EO}
Where δ represents the path planning set of the vehicle, ε I represents the historical track of the vehicle, E I represents the safety situation of the vehicle, θ represents the driving decision of the driver, ε O represents the historical track of the surrounding vehicle, and E O represents the safety situation of the surrounding vehicle.
In the preferred embodiment of the invention, the 'man-machine-ring' information is fused to more comprehensively and effectively judge the running risk, and the running path of the vehicle is planned according to the judging result, so that the running risk estimation in the intelligent interconnection traffic scene is realized, the perceived real-time performance and the accuracy of safety situation judgment are ensured, and the intelligent running safety is greatly enhanced.
Example 2:
referring to fig. 3, in a preferred embodiment of the present invention, there is provided a vehicle security situation calculating system based on multi-terminal collaborative recognition, including: the system comprises a vehicle-mounted sensing device, a vehicle-mounted safety situation calculating device connected with the vehicle-mounted sensing device, a road side sensing device connected with the vehicle-mounted sensing device and the vehicle-mounted safety situation calculating device in a wireless communication mode and a cloud sensing device connected with the road side sensing device in a two-way communication mode.
The vehicle-mounted sensing device is used for collecting multi-mode information of the driver, calculating characteristic parameters of the driver and sending the characteristic parameters of the driver to the vehicle-mounted safety situation calculating device.
The vehicle-mounted sensing device comprises a vehicle-mounted driver sensing module, a vehicle-mounted vehicle sensing module, a vehicle-mounted environment sensing module and a vehicle-mounted information processing module.
The vehicle-mounted driver perception module comprises: the high-definition network cameras are arranged right in front of the rearview mirror in the vehicle and above the rearview mirror in the vehicle, and are used for collecting facial expressions, head postures and body postures of the driver; an audio collector for collecting voiceprint signals and voice tones of a driver; and the physiological signal acquisition bracelet is worn on the wrist of the driver or embedded in the steering wheel and is used for acquiring the heart rate, the respiratory rate, the skin electricity and the blood oxygen saturation of the driver.
The vehicle-mounted vehicle sensing module includes: speed sensors for acquiring the speed of the vehicle, respectively; an acceleration sensor for acquiring acceleration of the vehicle; the steering angle sensor is used for collecting the steering angle of the steering wheel of the vehicle; the GPS locator is used for collecting vehicle positioning information; an accelerator pedal opening sensor for acquiring an opening of an accelerator pedal of a vehicle; and the brake pedal opening sensor is used for acquiring the opening information of the brake pedal of the vehicle.
The vehicle-mounted environment sensing module comprises a high-definition network camera, an infrared camera, a laser radar and a millimeter wave radar, wherein the high-definition network camera, the infrared camera, the laser radar and the millimeter wave radar are arranged on the outer body of the vehicle and used for acquiring two-dimensional images and three-dimensional Lei Dadian cloud information outside the vehicle.
The vehicle-mounted information processing module comprises at least one edge computing device for receiving the multi-mode information from the vehicle-mounted driver perception module, processing the multi-mode information into driver characteristic parameters through a multi-task neural network, sending the driver characteristic parameters to the vehicle-mounted security situation computing device and sending the vehicle external environment information and the vehicle operation parameter information to the road side perception device.
The vehicle-mounted information processing module receives the characteristic data of the vehicle-mounted driver perception module, performs time sequence alignment and dimension reduction processing, and then inputs the characteristic data into the multi-task neural network to obtain state information such as fatigue, emotion, action behaviors and the like of a driver.
In the preferred embodiment of the invention, through the mutual cooperation of the modules in the vehicle-mounted sensing device, various physiological information and external expression information of a driver, external environment information of a vehicle and operating parameter information of the vehicle are comprehensively acquired, and the reliability and the richness of data acquisition are ensured.
The vehicle-mounted safety situation calculation device is used for carrying out vehicle safety situation assessment and driving decision planning according to the characteristic parameters of the driver and the risk factor data packet.
The vehicle-mounted safety situation calculation device comprises a safety situation calculation module and a vehicle path planning module.
The safety situation calculation module is used for receiving the characteristic parameters of the driver from the vehicle-mounted sensing device and the risk factor data packet from the road side sensing device, so as to calculate the safety situation.
The vehicle path planning module is used for receiving the safety situation information from the safety situation calculation module, further carrying out path planning, and displaying and high-risk early warning on the driver.
In the preferred embodiment of the invention, the comprehensive and effective judgment of the driving risk is ensured according to the combination judgment of the received characteristic parameters of the driver and the risk factor data packet through the cooperation among the modules in the vehicle-mounted safety situation calculation device, and the driving path of the vehicle is planned, so that the judgment of the driving risk in the intelligent interconnection traffic scene is comprehensive and accurate.
The road side sensing device is used for collecting road network environment information, processing self information and information sent by other devices, sending information outside self-calculation bearing capacity to the cloud sensing device and sending a risk factor data packet to the vehicle-mounted security situation calculating device.
The road side sensing device is also used for pre-storing road network road information in the area.
The road side sensing device comprises a road side environment sensing module and a road side information processing module.
The road side environment sensing module comprises a camera and a detection radar, wherein the camera and the detection radar are arranged on the road side sensing device and used for acquiring road network image information and road network three-dimensional radar point cloud information under the current road network.
The road side information processing module comprises a plurality of integrated computing devices which are arranged in the road side sensing device and used for pre-storing road network information in a current area, receiving road network image information and road network three-dimensional radar point cloud information from the road side environment sensing module, receiving vehicle external environment information and vehicle operation parameter information sent by vehicles in a range, performing data check and neglect, performing calculation through YOLOv depth neural network after computing resource allocation, sending data exceeding a computing tolerance to the cloud sensing device, receiving return information of the cloud sensing device, generating a risk factor data packet and sending the risk factor data packet to the vehicle-mounted security situation computing device.
In the preferred embodiment of the invention, the repeated data are ignored through the cooperation among the modules in the road side sensing device, the sensing characteristic information of the vehicle-mounted, road side and cloud is fused, the advantages of different ends are utilized, the computing resources of different ends are communicated with each other and distributed effectively, the repeated environmental information is prevented from being computed by different devices, the waste of computing resources is reduced, and the condition of insufficient computing capacity is avoided.
The cloud sensing device is used for calculating information from the road side sensing device and returning a result to the road side sensing device.
The cloud sensing device is also used for pre-storing and updating the high-precision map information.
The cloud sensing device comprises a cloud information processing module which is used for pre-storing and updating high-precision map information, processing information from the road side sensing device and integrating a plurality of high-performance calculations for returning the processed data to the road side sensing device along with the road environment state under the beyond-view range wide area.
In the preferred embodiment of the invention, the data outside the calculation bearing capacity in the road side sensing device is sent to the cloud sensing device for calculation, so that the calculation pressure of the road side sensing device is effectively reduced, the instantaneity of the data is ensured, and the condition of insufficient calculation capacity is avoided; meanwhile, the high-precision map information provided by the cloud sensing device in the preferred embodiment of the invention also provides necessary data support for intelligent driving.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (8)
1. The vehicle security situation calculation method based on multi-terminal collaborative identification is used in a system of mutual collaboration of a vehicle-mounted sensing device, a vehicle-mounted security situation calculation device, a plurality of road side sensing devices and a cloud sensing device, and is characterized by comprising the following steps:
S1, information is collected: the method comprises the steps of collecting multi-mode information of a driver through a vehicle-mounted sensing device, generating characteristic parameters of the driver through calculation, and sending the characteristic parameters of the driver to a vehicle-mounted security situation calculating device; collecting vehicle external environment information and vehicle operation parameter information through a vehicle-mounted sensing device, and sending the vehicle external environment information and the vehicle operation parameter information to a road side sensing device in a vehicle adjacent range; collecting road network environment information through a road side sensing device and pre-storing road network road information in an area; the cloud sensing device pre-stores and updates high-precision map information;
s2, processing information: after receiving the external environment information of the vehicle and the running parameter information of the vehicle, the road side sensing device recognizes and ignores repeated information in the road network environment information, the road network road information, the external environment information of the vehicle and the running parameter information of the vehicle; after computing resource allocation, computing is executed, and data exceeding the computing tolerance are sent to a cloud sensing device;
S3, generating a risk factor package: the cloud sensing device receives and calculates information from the road side sensing device to obtain risk factor information in each road network area, and returns coordinate, speed and acceleration parameter information in the risk factor information to the road side sensing device; the road side sensing device receives the information returned by the cloud sensing device, generates a risk factor data packet and sends the risk factor data packet to the vehicle-mounted security situation calculating device;
s4, carrying out security situation assessment and driving decision planning: the vehicle-mounted safety situation calculation device receives the driver characteristic parameters and the risk factor data packet, carries out vehicle safety situation assessment according to the driver characteristic parameters and the risk factor data packet, carries out driving decision planning according to assessment results after assessment is finished, and judges whether to pre-warn the driver or not;
In S4, when the vehicle-mounted security situation calculation device performs the vehicle security situation assessment, the vehicle operation security situation E is:
wherein alpha i、βj、γk is a preset parameter greater than 0, the weight of each risk factor is represented, V i represents an external factor risk factor, D j represents a risk factor of different states of a driver, and R k represents a road risk factor;
when the vehicle-mounted safety situation calculation device performs driving decision planning, a vehicle path planning set is searched according to a safety situation evaluation result, the optimal planning path in the set is continuously adjusted based on a self-learning method and displayed to a driver, and the vehicle planning path set P is as follows:
P＝{δ|εI,EI,θ,εO,EO}
Where δ represents the path planning set of the vehicle, ε I represents the historical track of the vehicle, E I represents the safety situation of the vehicle, θ represents the driving decision of the driver, ε o represents the historical track of the surrounding vehicle, and E O represents the safety situation of the surrounding vehicle.
2. The vehicle security posture calculation method based on multi-terminal collaborative recognition according to claim 1, wherein in S1, the multi-modal information includes facial expression, head pose, body pose, heart rate, blood oxygen saturation, skin electricity, respiration rate, and voice intonation information of a driver; the vehicle external environment information comprises two-dimensional images and three-dimensional Lei Dadian cloud information in a detectable range of the vehicle-mounted sensing device; the vehicle operation parameter information comprises speed, acceleration, steering wheel rotation angle, positioning information, accelerator pedal opening and brake pedal opening information of the vehicle; the road network environment information comprises road network image information and road network three-dimensional radar point cloud information;
The road network and road information in the area pre-stored by the road side sensing device comprises lane information, road connection relation, intersection information and road section information; the high-precision map information pre-stored and updated by the cloud sensing device comprises road congestion conditions, construction conditions, traffic accident conditions, weather conditions, traffic lights, crosswalk and speed limiting conditions.
3. The vehicle security situation calculating method based on multi-terminal collaborative recognition according to claim 1, wherein when the road side sensing device calculates the collected data, the data is input into YOLOv depth neural network, so as to obtain the position, speed and acceleration information of risk factor vehicles, pedestrians and obstacles in the current road network;
The cloud sensing device calculates the data sent by the road side sensing device to obtain risk factor information in each road network area, and returns coordinate, speed and acceleration parameter information in the risk factor information to the road side sensing device.
4. A vehicle security situation computing system based on multi-terminal collaborative recognition, comprising: the system comprises a vehicle-mounted sensing device, a vehicle-mounted safety situation calculating device connected with the vehicle-mounted sensing device, a road side sensing device connected with the vehicle-mounted sensing device and the vehicle-mounted safety situation calculating device in a wireless communication mode and a cloud sensing device connected with the road side sensing device in a two-way communication mode;
the vehicle-mounted sensing device is used for collecting multi-mode information of a driver, calculating characteristic parameters of the driver and sending the characteristic parameters of the driver to the vehicle-mounted security situation calculating device;
The road side sensing device is used for collecting road network environment information, processing self information and information sent by other devices, sending information outside self-calculation bearing capacity to the cloud sensing device and sending a risk factor data packet to the vehicle-mounted security situation calculating device;
the cloud sensing device is used for calculating information from the road side sensing device and returning a result to the road side sensing device, and after the cloud sensing device obtains risk factor information in each road network area, coordinate, speed and acceleration parameter information in the risk factor information is returned to the road side sensing device, so that large-capacity information is not transmitted;
the vehicle-mounted safety situation calculation device is used for carrying out vehicle safety situation assessment and driving decision planning according to the characteristic parameters of the driver and the risk factor data packet;
The vehicle-mounted safety situation calculation device comprises a safety situation calculation module and a vehicle path planning module;
the safety situation calculation module is used for receiving the characteristic parameters of the driver from the vehicle-mounted sensing device and the risk factor data packet from the road side sensing device, so as to calculate the safety situation;
The vehicle path planning module is used for receiving the safety situation information from the safety situation calculation module, further carrying out path planning, and carrying out display and high risk early warning on a driver;
When the vehicle-mounted safety situation calculation device evaluates the safety situation of the vehicle, the running safety situation E of the vehicle is as follows:
wherein alpha i、βj、γk is a preset parameter greater than 0, the weight of each risk factor is represented, V i represents an external factor risk factor, D j represents a risk factor of different states of a driver, and R k represents a road risk factor;
when the vehicle-mounted safety situation calculation device performs driving decision planning, a vehicle path planning set is searched according to a safety situation evaluation result, the optimal planning path in the set is continuously adjusted based on a self-learning method and displayed to a driver, and the vehicle planning path set P is as follows:
P＝{δ|εI,EI,θ,εO,EO}
Where δ represents the path planning set of the vehicle, ε I represents the historical track of the vehicle, E I represents the safety situation of the vehicle, θ represents the driving decision of the driver, ε O represents the historical track of the surrounding vehicle, and E O represents the safety situation of the surrounding vehicle.
5. The vehicle security situation calculating system based on multi-terminal collaborative recognition according to claim 4, wherein the road side sensing device is further configured to pre-store road network road information in an area; the cloud sensing device is also used for pre-storing and updating high-precision map information.
6. The vehicle security situation computing system based on multi-terminal collaborative recognition according to claim 4, wherein the vehicle-mounted sensing device comprises a vehicle-mounted driver sensing module, a vehicle-mounted vehicle sensing module, a vehicle-mounted environment sensing module, and a vehicle-mounted information processing module;
The vehicle-mounted driver perception module comprises: the high-definition network cameras are arranged right in front of the rearview mirror in the vehicle and above the rearview mirror in the vehicle, and are used for collecting facial expressions, head postures and body postures of the driver; an audio collector for collecting voiceprint signals and voice tones of a driver; the physiological signal acquisition bracelet is worn on the wrist of the driver or embedded in the steering wheel and is used for acquiring the heart rate, the respiratory rate, the skin electricity and the blood oxygen saturation of the driver;
The vehicle-mounted vehicle sensing module includes: speed sensors for acquiring the speed of the vehicle, respectively; an acceleration sensor for acquiring acceleration of the vehicle; the steering angle sensor is used for collecting the steering angle of the steering wheel of the vehicle; the GPS locator is used for collecting vehicle positioning information; an accelerator pedal opening sensor for acquiring an opening of an accelerator pedal of a vehicle; a brake pedal opening sensor for acquiring vehicle brake pedal opening information;
The vehicle-mounted environment sensing module comprises a high-definition network camera, an infrared camera, a laser radar and a millimeter wave radar, wherein the high-definition network camera, the infrared camera, the laser radar and the millimeter wave radar are arranged on the outer body of a vehicle and used for acquiring two-dimensional images and three-dimensional Lei Dadian cloud information outside the vehicle;
The vehicle-mounted information processing module comprises at least one edge computing device, wherein the at least one edge computing device is used for receiving the multi-mode information from the vehicle-mounted driver perception module, processing the multi-mode information into driver characteristic parameters through a multi-task neural network, sending the driver characteristic parameters to the vehicle-mounted security situation computing device and sending the vehicle external environment information and the vehicle operation parameter information to the road side perception device;
7. The vehicle security situation computing system based on multi-terminal collaborative recognition according to claim 5, wherein the roadside awareness device comprises a roadside environment awareness module and a roadside information processing module;
The road side environment sensing module comprises a camera and a detection radar, wherein the camera and the detection radar are arranged on the road side sensing device and are used for acquiring road network image information and road network three-dimensional radar point cloud information under the current road network;
The road side information processing module comprises a plurality of integrated computing devices which are arranged in the road side sensing device and used for pre-storing road network information in a current area, receiving road network image information and road network three-dimensional radar point cloud information from the road side environment sensing module, receiving vehicle external environment information and vehicle operation parameter information sent by a vehicle in a range, performing data check and neglect, performing calculation through YOLOv depth neural network after calculating resource allocation, sending data exceeding a calculation tolerance to the cloud sensing device, receiving return information of the cloud sensing device, generating a risk factor data packet and sending the risk factor data packet to the vehicle-mounted security situation computing device.
8. The vehicle security situation computing system based on multi-terminal collaborative recognition according to claim 5, wherein the cloud sensing device comprises a cloud information processing module for pre-storing and updating high-precision map information, processing information from a road side sensing device, and integrating a plurality of high-performance computations for returning processed data to the road environment state over a wide area of beyond-line-of-sight to the road side sensing device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310107673.6A CN116129641B (en) | 2023-02-13 | 2023-02-13 | Vehicle security situation calculation method and system based on multi-terminal collaborative identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310107673.6A CN116129641B (en) | 2023-02-13 | 2023-02-13 | Vehicle security situation calculation method and system based on multi-terminal collaborative identification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116129641A CN116129641A (en) | 2023-05-16 |
| CN116129641B true CN116129641B (en) | 2024-06-04 |
Family
ID=86295396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310107673.6A Active CN116129641B (en) | 2023-02-13 | 2023-02-13 | Vehicle security situation calculation method and system based on multi-terminal collaborative identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116129641B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117389937B (en) * | 2023-12-11 | 2024-03-08 | 上海建工一建集团有限公司 | Calculation method of obstacle avoidance data of vehicle, computer and readable storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111540237A (en) * | 2020-05-19 | 2020-08-14 | 河北德冠隆电子科技有限公司 | Method for automatically generating vehicle safety driving guarantee scheme based on multi-data fusion |
| CN113485319A (en) * | 2021-06-08 | 2021-10-08 | 中兴智能汽车有限公司 | Automatic driving system based on 5G vehicle-road cooperation |
| CN113581211A (en) * | 2021-08-30 | 2021-11-02 | 深圳清航智行科技有限公司 | Vehicle driving control method, system and device and readable storage medium |
| CN114084170A (en) * | 2021-04-15 | 2022-02-25 | 上海丰豹商务咨询有限公司 | Vehicle-mounted intelligent unit serving CVCS (continuously variable communication System) and control method thereof |
| CN114241750A (en) * | 2021-11-29 | 2022-03-25 | 重庆文理学院 | Vehicle-road cooperative information interaction system and method based on intelligent road traffic risk early warning method |
| CN114967705A (en) * | 2022-06-17 | 2022-08-30 | 北京航空航天大学 | Road-side guidance-based road-vehicle fusion control method |
| CN114964286A (en) * | 2022-05-10 | 2022-08-30 | 阿波罗智联(北京)科技有限公司 | Trajectory planning information generation method and device, electronic equipment and storage medium |
| CN115512511A (en) * | 2021-06-07 | 2022-12-23 | 中移物联网有限公司 | Early warning method, early warning device, mobile terminal and readable storage medium |
-
2023
- 2023-02-13 CN CN202310107673.6A patent/CN116129641B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111540237A (en) * | 2020-05-19 | 2020-08-14 | 河北德冠隆电子科技有限公司 | Method for automatically generating vehicle safety driving guarantee scheme based on multi-data fusion |
| CN114084170A (en) * | 2021-04-15 | 2022-02-25 | 上海丰豹商务咨询有限公司 | Vehicle-mounted intelligent unit serving CVCS (continuously variable communication System) and control method thereof |
| CN115512511A (en) * | 2021-06-07 | 2022-12-23 | 中移物联网有限公司 | Early warning method, early warning device, mobile terminal and readable storage medium |
| CN113485319A (en) * | 2021-06-08 | 2021-10-08 | 中兴智能汽车有限公司 | Automatic driving system based on 5G vehicle-road cooperation |
| CN113581211A (en) * | 2021-08-30 | 2021-11-02 | 深圳清航智行科技有限公司 | Vehicle driving control method, system and device and readable storage medium |
| CN114241750A (en) * | 2021-11-29 | 2022-03-25 | 重庆文理学院 | Vehicle-road cooperative information interaction system and method based on intelligent road traffic risk early warning method |
| CN114964286A (en) * | 2022-05-10 | 2022-08-30 | 阿波罗智联(北京)科技有限公司 | Trajectory planning information generation method and device, electronic equipment and storage medium |
| CN114967705A (en) * | 2022-06-17 | 2022-08-30 | 北京航空航天大学 | Road-side guidance-based road-vehicle fusion control method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116129641A (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Han et al. | Research on road environmental sense method of intelligent vehicle based on tracking check | |
| CN108572663B (en) | Target tracking | |
| CN110362077B (en) | Unmanned vehicle emergency hedge decision making system, method and medium | |
| KR102674974B1 (en) | Method and apparatus for passenger recognition and boarding support of autonomous vehicle | |
| US11472291B2 (en) | Graphical user interface for display of autonomous vehicle behaviors | |
| US11468574B2 (en) | Image processing apparatus and image processing method | |
| US11884155B2 (en) | Graphical user interface for display of autonomous vehicle behaviors | |
| CN111469838B (en) | Collaborative ACC/AEB decision management system based on Internet of vehicles and vehicle | |
| KR101534056B1 (en) | Traffic signal mapping and detection | |
| KR102043060B1 (en) | Autonomous drive apparatus and vehicle including the same | |
| CN109920246A (en) | It is a kind of that local paths planning method is cooperateed with binocular vision based on V2X communication | |
| CN114442101B (en) | Vehicle navigation method, device, equipment and medium based on imaging millimeter wave radar | |
| KR20170011882A (en) | Radar for vehicle, and vehicle including the same | |
| CN106394555A (en) | Unmanned automobile obstacle avoidance system and method based on 3D camera | |
| KR101698781B1 (en) | Driver assistance apparatus and Vehicle including the same | |
| CN116129641B (en) | Vehicle security situation calculation method and system based on multi-terminal collaborative identification | |
| CN112067013A (en) | AR-HUD-based vehicle-mounted identification system | |
| JP2020204839A (en) | Outside environment recognition device | |
| CN109878445A (en) | A kind of binocular vision DAS (Driver Assistant System) | |
| KR20160144643A (en) | Apparatus for prividing around view and vehicle including the same | |
| KR101872477B1 (en) | Vehicle | |
| KR20160131580A (en) | Apparatus for prividing around view and vehicle including the same | |
| KR20170006753A (en) | Apparatus for detecting navigation and vehicle including the same | |
| KR102568283B1 (en) | Pedestrian communication system for self driving cars | |
| US12087165B2 (en) | Moving body prediction device, traffic safety support system, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |