CN112033425A

CN112033425A - Vehicle driving assistance method and device, computer equipment and storage medium

Info

Publication number: CN112033425A
Application number: CN201910481767.3A
Authority: CN
Inventors: 马潍; 佘咸宁
Original assignee: Changsha Intelligent Driving Research Institute Co Ltd
Current assignee: Changsha Intelligent Driving Research Institute Co Ltd
Priority date: 2019-06-04
Filing date: 2019-06-04
Publication date: 2020-12-04
Anticipated expiration: 2039-06-04
Also published as: CN112033425B

Abstract

The application relates to a vehicle driving assisting method and device running on vehicle-mounted equipment, computer equipment and a storage medium. The method comprises the following steps: acquiring positioning information and destination information based on a vehicle, and sending the positioning information and the destination information to a cloud service platform; receiving global path planning information sent by a cloud service platform based on positioning information and destination information; acquiring local path planning information determined based on global path planning; and fusing the local path planning information and the global path planning information to obtain the target driving plan of the vehicle. By adopting the method, the reliability of the target path planning can be improved, and thus the safety of vehicle driving can be improved. The application relates to a vehicle driving assisting method and device running on roadside equipment, computer equipment and a storage medium, and can also improve the running safety of a vehicle.

Description

Vehicle driving assistance method and device, computer equipment and storage medium

Technical Field

The present application relates to the field of road traffic control technologies, and in particular, to a method and an apparatus for vehicle driving assistance, a computer device, and a storage medium.

Background

With the continuous improvement of living standard and communication technology of people, the gradual acceleration of urbanization process, the rapid increase of urban motor vehicle ownership and other factors, the number of motor vehicles is continuously increased, the vehicle-road cooperation technology based on the internet of vehicles enters a rapid development stage, and the importance of vehicle auxiliary driving is also gradually improved.

According to a traditional vehicle auxiliary driving method, a target driving path is determined according to vehicle-mounted video data acquired by vehicle-mounted equipment. Because the conventional vehicle driving assisting method has single considered factors during driving assisting, potential safety hazards exist, and the driving safety of the vehicle is low and needs to be further improved.

Disclosure of Invention

In view of the above, it is necessary to provide a vehicle driving assistance method, a device, a computer apparatus, and a storage medium capable of improving the vehicle driving safety in view of the above technical problems.

A vehicle assisted driving method, the method comprising:

the method comprises the steps of obtaining positioning information and destination information based on a vehicle, and sending the positioning information and the destination information to a cloud service platform;

receiving global path planning information sent by the cloud service platform based on the positioning information and the destination information;

acquiring local path planning information determined based on the global path planning;

and fusing the local path planning information and the global path planning information to obtain the target driving plan of the vehicle.

In one embodiment, the method further comprises:

when the vehicle is within the transmission range of the target road side device, receiving road side video data sent by the target road side device;

acquiring vehicle-mounted video data based on the vehicle;

and fusing the road side video data and the vehicle-mounted video data to obtain fused video data.

In one embodiment, the fusing the roadside video data and the vehicle-mounted video data to obtain fused video data includes:

analyzing the vehicle-mounted video data to obtain the current visual field distance of the vehicle;

determining a target view distance according to the braking distance of the vehicle;

and adjusting the visual field distance of the fused video data obtained by fusing the road side video data and the vehicle-mounted video data according to the target visual field distance.

In one embodiment, when the vehicle is within the transmission range of the target roadside device, receiving the roadside video data transmitted by the target roadside device includes:

sending a roadside video request to the target roadside device within the transmission range;

and receiving the road side video data sent by the target road side equipment according to the road side video request.

In one embodiment, after the fusing the roadside video data and the vehicle-mounted video data to obtain fused video data, the method further includes:

determining a target driving instruction according to the fusion video data and the target driving plan; the target driving instruction carries a steering parameter and a speed parameter;

and sending the target driving instruction to a central control platform, and controlling the vehicle to run by the central control platform according to the steering parameter and the speed parameter.

In one embodiment, the obtaining local path planning information determined based on the global path planning includes:

and when the vehicle is in the transmission range of the target road side equipment, receiving local path planning information sent by the target road side equipment based on the global path planning.

A vehicle driving assist apparatus, the apparatus comprising:

the information acquisition and forwarding module is used for acquiring positioning information and destination information based on a vehicle and sending the positioning information and the destination information to a cloud service platform;

the global planning receiving module is used for receiving global path planning information sent by the cloud service platform based on the positioning information and the destination information;

the local planning receiving module is used for acquiring local path planning information determined based on the global path planning;

and the target planning fusion module is used for fusing the local path planning information and the global path planning information to obtain the target driving plan of the vehicle.

In one embodiment, the apparatus further includes:

the road side video receiving module is used for receiving road side video data sent by the target road side device when the vehicle is in the transmission range of the target road side device;

the vehicle-mounted video acquisition module is used for acquiring vehicle-mounted video data based on the vehicle;

and the video data fusion module is used for fusing the road side video data and the vehicle-mounted video data to obtain fused video data.

In one embodiment, the video data fusion module includes:

the visual field distance analysis unit is used for analyzing the vehicle-mounted video data to obtain the current visual field distance of the vehicle;

the target visual field determining unit is used for determining a target visual field distance according to the braking distance of the vehicle;

and the visual field distance adjusting unit is used for adjusting the visual field distance of the fused video data obtained by fusing the road side video data and the vehicle-mounted video data according to the target visual field distance.

In one embodiment, the roadside video receiving module includes:

the road side video request unit is used for sending a road side video request to the target road side equipment in the transmission range;

and the road side video receiving unit is used for receiving road side video data sent by the target road side equipment according to the road side video request.

In one embodiment, the apparatus further comprises:

the driving instruction determining module is used for determining a target driving instruction according to the fused video data and the target driving plan after the video data fusing module fuses the road side video data and the vehicle-mounted video data to obtain fused video data; the target driving instruction carries a steering parameter and a speed parameter;

and the driving instruction forwarding module is used for sending the target driving instruction to a central control platform, and the central control platform controls the vehicle to run according to the steering parameter and the speed parameter.

In an embodiment, the local planning acquiring module is configured to receive, when the vehicle is within a transmission range of the target road-side device, local path planning information sent by the target road-side device based on the global path planning.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

According to the vehicle driving assisting method, the vehicle driving assisting device, the computer equipment and the storage medium, firstly, positioning information and destination information based on a vehicle are obtained, and the positioning information and the destination information are sent to a cloud service platform; global path planning information is then received, the global path planning information being determined by the cloud service platform based on the positioning information and the destination information. And acquiring local path planning information determined based on the global path planning. And finally, fusing the local path planning information and the global path planning information to obtain the target driving plan of the vehicle. Therefore, global path planning is carried out through the cloud service platform, the traffic efficiency of the whole traffic network can be improved, traffic jam is reduced, and the decision-making capability of vehicle auxiliary driving is improved; the global path planning information and the local path planning information are fused through the vehicle-mounted equipment, so that the global path planning and the local path planning can be comprehensively considered by the vehicle-mounted equipment during vehicle auxiliary driving, the reliability of target path planning is improved, and the driving safety of the vehicle can be improved.

A vehicle assisted driving method, the method comprising:

the method comprises the steps of obtaining road side video data and sending the road side video data to a cloud service platform;

receiving global path planning information aiming at a target vehicle, which is sent by the cloud service platform based on the roadside video data;

recognizing lanes and obstacles for the roadside video data to obtain lane information and obstacle information;

determining local path planning information of the target vehicle according to the lane information, the obstacle information and the global path planning information;

and when the target vehicle is in the transmission range, sending the local path planning information to the vehicle-mounted equipment of the target vehicle.

In one embodiment, the method further includes at least one of the following two cases:

when the target vehicle is in the transmission range, if a road side video request sent by vehicle-mounted equipment of the target vehicle is received, sending the road side video data to the vehicle-mounted equipment of the target vehicle;

when the target vehicle is in the transmission range, if the road side video data are analyzed to obtain an analysis result of road abnormity, the road side video data are sent to the vehicle-mounted equipment of the target vehicle.

In one embodiment, the sending the local path planning information to the vehicle-mounted device of the target vehicle when the target vehicle is within the transmission range includes:

calibrating the local path planning information through a high-precision map;

and when the target vehicle is in a transmission range, sending the calibrated local path planning information to the vehicle-mounted equipment of the target vehicle.

A vehicle driving assist apparatus, the apparatus comprising:

the video acquisition and forwarding module is used for acquiring road side video data and sending the road side video data to a cloud service platform;

the global planning receiving module is used for receiving global path planning information aiming at a target vehicle, which is sent by the cloud service platform based on the road side video data;

the video data identification module is used for identifying lanes and obstacles to the roadside video data to obtain lane information and obstacle information;

the local path planning module is used for determining local path planning information of the target vehicle according to the lane information, the obstacle information and the global path planning information;

and the local path forwarding module is used for sending the local path planning information to the vehicle-mounted equipment of the target vehicle when the target vehicle is in a transmission range.

In one embodiment, the apparatus further includes at least one of the following two modules:

the video passive sending module is used for sending the road side video data to the vehicle-mounted equipment of the target vehicle if a road side video request sent by the vehicle-mounted equipment of the target vehicle is received when the target vehicle is in a transmission range;

and the video active sending module is used for sending the road side video data to the vehicle-mounted equipment of the target vehicle if the road side video data are analyzed to obtain an analysis result of road abnormity when the target vehicle is in a transmission range.

In one embodiment, the apparatus further includes a path planning calibration module;

the path planning calibration module is used for calibrating the local path planning information through a high-precision map;

and the local path forwarding module is used for sending the calibrated local path planning information to the vehicle-mounted equipment of the target vehicle when the target vehicle is in a transmission range.

According to the vehicle auxiliary driving method, the vehicle auxiliary driving device, the computer equipment and the storage medium, roadside video data are obtained and sent to the cloud service platform; receiving global path planning information aiming at a target vehicle, which is sent by the cloud service platform based on the roadside video data; recognizing lanes and obstacles for the roadside video data to obtain lane information and obstacle information; determining local path planning information of the target vehicle according to the lane information, the obstacle information and the global path planning information; and when the target vehicle is in the transmission range, sending the local path planning information to the vehicle-mounted equipment of the target vehicle. Therefore, global path planning is carried out through the cloud service platform, the traffic efficiency of the whole traffic network can be improved, traffic jam is reduced, and the decision-making capability of vehicle auxiliary driving is improved; the road side equipment determines local path planning information of a target vehicle based on global path planning, and sends the local path planning information to vehicle-mounted equipment of the target vehicle when the target vehicle enters a transmission range. Thus, the global path planning, the local lane information and the obstacle information are comprehensively considered to obtain the local path planning, so that the reliability of the local path planning information can be improved, and the driving safety of the vehicle can be improved.

Drawings

FIG. 1 is a diagram of an exemplary implementation of a vehicle driver assistance method;

FIG. 2 is a schematic flow chart of a vehicle assisted driving method according to one embodiment;

FIG. 3 is a flowchart illustrating a method for assisting driving of a vehicle according to another embodiment;

FIG. 4 is a block diagram showing the construction of a vehicle driving assist apparatus according to an embodiment;

FIG. 5 is a block diagram showing the construction of a vehicle driving assist apparatus according to another embodiment;

FIG. 6 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The vehicle driving assisting method provided by the application can be applied to the application environment shown in fig. 1. A roadside apparatus 104 is provided on a road on which the vehicle 102 travels, and an on-board apparatus communicatively connected to the roadside apparatus 104 is provided on the vehicle 102. Each roadside device 104 is communicatively coupled to a cloud service platform 106. The cloud service platform 106 is in direct communication connection with the vehicle-mounted device of the vehicle 102, or the cloud service platform 106 is in indirect communication connection with the vehicle-mounted device of the vehicle 102 through forwarding of the road side device 104. The vehicle-mounted equipment acquires positioning information and destination information based on the vehicle 102 and sends the positioning information and the destination information to the cloud service platform 106; the vehicle-mounted equipment receives global path planning information sent by the cloud service platform 106 based on the positioning information and the destination information; when the vehicle 102 is in the transmission range of the target road-side device, the vehicle-mounted device receives local path planning information sent by the target road-side device based on global path planning; and the vehicle-mounted equipment fuses the local path planning information and the global path planning information to obtain the target driving plan of the vehicle.

The vehicle-mounted device of the vehicle 102 may be, but is not limited to, a smart phone, a vehicle recorder, various personal computers, a notebook computer, a tablet computer, and a portable wearable device. The roadside apparatus 104 includes a roadside Unit (RSU), which may be a microwave device using DSRC (Dedicated short range Communication) technology. The cloud service platform 106 may be a stand-alone server or a server cluster composed of a plurality of servers. In a preferred embodiment, the vehicle-mounted device is a smartphone. Due to the popularization of the smart phone, the cost of the vehicle-mounted equipment can be reduced by using the smart phone as the vehicle-mounted equipment. Furthermore, the hardware requirement of the smart phone is that the smart phone has an ultrahigh-performance Graphics Processing Unit (GPU) and a Central Processing Unit (CPU), and has the capacity of Processing a large amount of image data; in addition, a global Positioning system (gps) module is required. The ultra-high performance graphics processor may refer to a graphics processor with a processing frequency not lower than 2500MHz (megahertz), and for example, the GPU may be of the type: G. the CPU has a processing frequency not lower than 2.6GHz (gigahertz), and the CPU may be of the type: intel core i 78700. The software can support a common artificial intelligence development framework, a deep learning framework, a scientific calculation framework and an image processing framework.

In one embodiment, as shown in fig. 2, a vehicle driving assistance method is provided, which is described by taking an example of the method applied to an on-board device on a vehicle 102 in fig. 1, and includes the following steps:

s202, positioning information and destination information based on the vehicle are obtained, and the positioning information and the destination information are sent to a cloud service platform.

The in-vehicle apparatus acquires the positioning information and the destination information of the vehicle before the start of the trip. After the driver gets on the vehicle and starts the vehicle, the positioning information of the vehicle is obtained through a GPS module built in the vehicle-mounted device, and the positioning information of the vehicle may include the current longitude and latitude of the vehicle.

The destination information may refer to information of a destination within a trip, which may be a destination manually or by voice input by a driver, or a destination determined by the vehicle-mounted device after analysis based on current location information of the vehicle.

After the positioning information and the destination information based on the vehicle are obtained, the vehicle-mounted equipment sends the positioning information and the destination information to the cloud service platform, and the cloud service platform carries out global path planning on the journey of the vehicle.

And S204, receiving global path planning information sent by the cloud service platform based on the positioning information and the destination information.

The cloud service platform collects positioning information and destination information sent by the vehicle-mounted equipment. The cloud service platform can perform global path planning according to the positioning information and the destination information and by combining with data sent by each road side device, so as to obtain global path planning information.

The cloud service platform can collect road side video data sent by the road side equipment and support data for global path planning of the cloud service platform. The road side video data is video data collected by road side equipment. The cloud service platform may collect traffic data. The traffic data is data of traffic conditions obtained by analyzing the road side traffic data, for example, the traffic data may include at least one of traffic congestion information and traffic flow information. Therefore, the cloud service platform can determine the global path planning information by combining the traffic data, so that the reliability of the global path planning information is improved. The traffic data can be obtained by analyzing roadside video data by roadside equipment and sent to the cloud service platform. The traffic data can be obtained by analyzing the road side video data through the cloud service platform, so that the resource consumption of road side equipment can be reduced, and the cost of the road side equipment is reduced. It should also be noted that the traffic congestion information may also be used to train a traffic congestion model. The congestion level is determined through the traffic congestion model, and the determined traffic congestion level can be stored in a database for later analysis, such as data statistical analysis of the traffic congestion condition.

The cloud service platform also collects road information data, and thus data support is provided for global path planning of the cloud service platform. The road information data may be road information data obtained by analyzing by the roadside device according to the roadside video data and the position information of the roadside device. The road information data can also be obtained by analyzing according to road side video data sent by the road side terminal and vehicle-mounted video data sent by the vehicle-mounted equipment by the cloud service platform. The analysis of the road information data can be realized by a trained neural network model. Therefore, the cloud service platform can fuse the road information data and determine the global path planning information, and therefore the reliability of the global path planning information is improved.

The determination process of the neural network model comprises the following steps: collecting road side video data and vehicle-mounted video data, fusing the vehicle-mounted video data and the road side video data to obtain fused video data, inputting the fused video data into the initialized neural network model, and determining a training result; and when the loss function values of the training result and the target result reach the optimal condition of the model, obtaining the finally trained neural network model. Wherein the target result is manually determined accurate road information data. The training result is the result of the road information data obtained through the neural network model in the training process.

The cloud service platform can also collect real-time traffic situation information. The traffic situation information may be sent by the road side device, and the road side device analyzes the road side video data to obtain the traffic situation information, and sends the traffic situation information to the cloud service platform. The traffic situation information may also be obtained by analyzing received road side video data sent by the road side device by the cloud service platform. The traffic situation information includes status information of each traffic participant. For example, the roadside video data includes a pedestrian, which has a moving direction and a moving speed at a certain time. Therefore, the cloud service platform can fuse historical traffic data and real-time traffic situation to determine global path planning information. Thus, the reliability of the global path planning information is improved.

And S206, acquiring local path planning information determined based on the global path planning.

The local path planning information may be determined by the in-vehicle device. After receiving the road side video data sent by the target road side equipment, the vehicle-mounted equipment identifies the road side video data by lanes and obstacles to obtain lane information and obstacle information. The lane information includes road signs, lane tracks and the like, and one road includes at least one lane. The obstacle information may include obstacle position information, obstacle size, and the like. And then the vehicle-mounted equipment obtains a local path plan according to the lane information, the obstacle information and the global path plan information. Specifically, the information of the starting point and the ending point in the local area may be determined according to the global path planning information and the local area corresponding to the target roadside device; and determining the local path planning information in the local area according to the starting point information and the end point information and by combining the barrier information and the road information.

The local path planning information can also be determined by the target road side equipment or the cloud service platform and then sent to the vehicle-mounted equipment. The target roadside device may perform data transceiving through DSRC, LTE-V (Long Term Evolution-Vehicle communication), or 5G (5th-Generation, fifth Generation mobile communication) technology. The transceiving mode can adopt a C-V2X (Cellular V2X, namely V2X based on Cellular communication technology) and DSRC compatible mode, and ensures that the mode can realize handshaking communication with various modules of vehicle-mounted equipment. Each road side device has a corresponding transmission range, and when a vehicle enters the transmission range of the target road side device, the target road side device sends local path planning information of a local area corresponding to the target road side device to the vehicle-mounted device.

The target road side device or cloud service platform may determine local path planning information based on the global path planning. For example, the target roadside device may perform lane and obstacle recognition on the collected roadside video data to obtain lane information and obstacle information. The lane information includes road signs, lane tracks and the like, and one road includes at least one lane. The obstacle information may include obstacle position information, obstacle size, and the like. The target roadside device may obtain a local path plan by combining the lane information, the obstacle information, and the global path plan information. Specifically, the information of the starting point and the ending point in the local area may be determined according to the global path planning information and the local area corresponding to the target roadside device; and determining the local path planning information in the local area according to the starting point information and the end point information and by combining the barrier information and the road information.

In one preferred embodiment, the obtaining the local path planning information determined based on the global path planning includes: and when the vehicle is in the transmission range of the target road side equipment, receiving local path planning information sent by the target road side equipment based on the global path planning. Therefore, local path planning is carried out through the road side equipment, and the vehicle can be guaranteed to continuously and efficiently run.

And S208, fusing the local path planning information and the global path planning information to obtain the target driving plan of the vehicle.

After receiving the local path planning information and the global path planning information, the vehicle-mounted device may fuse the local path planning and the global path planning to obtain a current driving plan of the vehicle, that is, a target driving plan. The global path planning information is a path planning from a starting point to a destination, and details of the planning may be on which road to drive, that is, the global path planning information includes road information, and the road information may be road identification. The starting point is positioning information of the vehicle acquired when the primary trip is started. The local path planning information is a specific path planning in a local area, and details of the specific path planning may be specific to which lane on which road, that is, the local path planning information includes road information and lane information. The local area refers to an area range corresponding to the target roadside device.

When a vehicle enters a local area corresponding to one piece of road side equipment, a target driving plan of the vehicle in the current driving process can be obtained by fusing local path planning information sent by the road side equipment and global path planning information sent by a cloud service platform. The target driving plan comprises road information of the whole journey and lane information of the current local area.

Based on the vehicle auxiliary driving method of the embodiment, the vehicle-mounted equipment firstly acquires positioning information and destination information based on a vehicle and sends the positioning information and the destination information to the cloud service platform; and then receiving global path planning information, wherein the global path planning information is determined by the cloud service platform based on the positioning information and the destination information. And when the vehicle is in the transmission range of the target road side equipment, receiving local path planning information sent by the target road side equipment based on the global path planning by the vehicle-mounted equipment. And finally, fusing the local path planning information and the global path planning information by the vehicle-mounted equipment to obtain the target driving plan of the vehicle. Therefore, global path planning is carried out through the cloud service platform, the traffic efficiency of the whole traffic network can be improved, traffic jam is reduced, and the decision-making capability of vehicle auxiliary driving is improved; the global path planning information and the local path planning information are fused through the vehicle-mounted equipment, so that the global path planning and the local path planning can be comprehensively considered by the vehicle-mounted equipment during vehicle auxiliary driving, the reliability of target path planning is improved, and the driving safety of the vehicle can be improved.

In one embodiment, the method further comprises: when the vehicle is in the transmission range of the target road side device, receiving road side video data sent by the target road side device; acquiring vehicle-mounted video data based on a vehicle; and fusing the road side video data and the vehicle-mounted video data to obtain fused video data.

When the vehicle needs to be within the transmission range of the target road side device, the vehicle-mounted device of the vehicle can receive the road side video data sent by the target road side device. The roadside video data may be actively transmitted by the target roadside device or passively transmitted by the roadside device. The active sending by the road side device may be that the road side device analyzes the collected road side video data, and when the road is found to be abnormal, the road side device actively sends the road side video data to the vehicle in the transmission range corresponding to the road side device, that is, the vehicle in the local area corresponding to the road side device. Conditions of roadway anomalies include, but are not limited to, the presence of obstacles such as rockfall, sprinkles; potholes on the road surface; the daytime visibility is lower than a preset value; the tunnel is internally provided with flames. The roadside device preprocesses the image data during processing, and the preprocessing processes comprise image enhancement, noise reduction, defogging, snow and rain elimination and the like. After the preprocessing is completed, the roadside device can further transform, reconstruct and synthesize the preprocessing result to realize the reconstruction of the image.

The passive sending by the roadside device may be that, when receiving a roadside video request sent by the vehicle-mounted device, the roadside device passively sends corresponding roadside video data to the vehicle-mounted device according to the roadside video request. The roadside video request may be a request transmitted by the vehicle-mounted device when the driver wants to know the road condition of the forward route. The roadside video request may carry current position information and preset position information in the target driving plan, where the preset position information may be position information in the target driving plan that is a preset distance away from the current position information. For example, 800 meters ahead of the current position in the target driving plan, and thus, road side video data within 800 meters ahead in the target driving plan is requested. The preset position information may also be a position of a road abnormality ahead, and thus, roadside video data at the position of the road abnormality is requested. Therefore, the driver can conveniently know the road condition, and the reliability of driving assistance is further improved.

The mode for acquiring the vehicle-mounted video data based on the vehicle by the vehicle-mounted device can be that the vehicle-mounted video data is acquired by an acquisition device of the vehicle-mounted device, such as a camera. The vehicle-mounted video data refers to video data collected on a vehicle through vehicle-mounted equipment.

The vehicle-mounted equipment receives the road side video data sent by the target road side equipment, and after the vehicle-mounted video data based on the vehicle is obtained, the road side video data and the vehicle-mounted video data are fused to obtain fused video data. The fusing the video data comprises fusing the roadside video data into the vehicle-mounted video data at the view angle of the vehicle-mounted device. The fused video data can be displayed through vehicle-mounted equipment. In this way, the field of view of the in-vehicle video data is expanded, so that a more reliable vehicle driving assist method can be provided.

In one embodiment, fusing the roadside video data and the vehicle-mounted video data to obtain fused video data, including: analyzing the vehicle-mounted video data to obtain the current visual field distance of the vehicle; determining a target view distance according to the braking distance of the vehicle; and adjusting the visual field distance of the fused road side video data and the vehicle-mounted video data according to the target visual field distance to obtain the fused video data.

The current visual field distance of the vehicle is the visual field distance of vehicle-mounted video data acquired by vehicle-mounted equipment of the vehicle, namely the current visual field distance of the vehicle-mounted equipment. The current annual field distance of the vehicle can be obtained by analyzing the vehicle-mounted video data collected by the vehicle-mounted equipment.

The braking distance of the vehicle can be determined according to the running speed of the vehicle and the preset minimum reaction time. The vehicle running speed may be obtained from an on-board Unit (OBU) of the vehicle, or may be determined by analyzing a change condition of the positioning information through map application software on the on-board device. The preset minimum response time is the minimum time required for emergency braking after the preset driver finds an obstacle, and may be 3 seconds, 2 seconds, and the like. The braking distance can be obtained by multiplying the vehicle running speed by the preset minimum reaction time.

The target viewing distance is determined based on the stopping distance, and the stopping distance may be multiplied by a factor greater than or equal to 1 to obtain the target viewing distance. For example, the braking distance may be directly set as the target visual field distance, or a distance 1.5 times the braking distance may be set as the target visual field distance. And finally, adjusting the visual field distance of the fused road side video data and the vehicle-mounted video data according to the target visual field distance to obtain the fused video data. For example, the target visual field distance may be set as the visual field distance of the fused video data, or the distance obtained by multiplying the target visual field distance by a coefficient may be set as the visual field distance of the fused video data. The visual field distance of the fused video data is the distance between the position where the video content that can be displayed in the video data is located and the vehicle, which is farthest from the fused video data. In this way, more reliable vehicle assist driving can be provided.

In one embodiment, when the vehicle is within the transmission range of the target roadside device, receiving roadside video data transmitted by the target roadside device, includes: sending a roadside video request to target roadside equipment within a transmission range; and receiving the road side video data sent by the target road side device according to the road side video request.

In this embodiment, an interface for actively requesting roadside video data is provided for the in-vehicle device. Thus, when the driver needs to know the road side video data 800 meters before the current position in the target path plan, such as in the case of congestion on the road ahead, the road side video request can be sent to the target road side device in the outgoing range, and then the road side video data sent by the target road side device according to the road side video request can be received. The roadside video request may carry current position information and preset position information in the target driving plan, where the preset position information may be position information in the target driving plan that is a preset distance away from the current position information. The preset position information may also be a position of a road abnormality ahead, and thus, roadside video data at the position of the road abnormality is requested. Therefore, fused video data obtained by fusing the road side video data and the vehicle-mounted video data can meet the pointed requirements of users. Therefore, the driver can know the road condition conveniently in a targeted manner, and the reliability of the auxiliary driving is further improved.

In one embodiment, after the road side video data and the vehicle-mounted video data are fused to obtain the fused video data, the method further includes: determining a target driving instruction according to the fused video data and the target driving plan, wherein the target driving instruction carries a steering parameter and a speed parameter; and sending the target driving instruction to a central control platform, and controlling the vehicle to run by the central control platform according to the steering parameter and the speed parameter.

The target driving command carries a steering parameter and a speed parameter. The steering parameter refers to a parameter for controlling the steering of the vehicle, such as a torque parameter for controlling the steering wheel torque. The speed parameter refers to a parameter that controls the running speed of the vehicle. The vehicle-mounted equipment can determine the vehicle running track according to the fused video data and can further correct the vehicle running track according to the target driving plan. For example, when the vehicle running track determined from the fused video data is not consistent with the target driving plan, the vehicle running track may be corrected to a running track consistent with the target driving plan. The vehicle-mounted equipment can determine the steering parameters carried in the target driving instruction according to the vehicle running track or the corrected vehicle running track. The vehicle-mounted equipment can determine the safe distance between the obstacle and the vehicle according to the fused video data, and then determine the speed parameter carried in the target driving instruction according to the safe distance and the preset minimum response time. When the vehicle-mounted device determines the vehicle movement track according to the fused video data, the vehicle movement track is determined according to the visual field distance of the fused video data. Specifically, for example, when the visual field distance of the fused video data is 800 meters, the vehicle travel track within 800 meters can be determined according to the fused video data. Further, the vehicle running track within 800 meters is calibrated by combining with the target driving plan. Therefore, the vehicle running track can be more accurate and reasonable. For example, at the 800 th meter, if the middle lane may be taken according to the fused video data, but if a right turn is required at the 803 th meter according to the target driving plan, the driving track of the vehicle at the 800 th meter may be corrected to the right lane according to the target driving plan at this time.

The central control platform is a vehicle-mounted central control platform and is used for controlling the state of the vehicle, such as controlling what music is played and the volume of a player, and sending an instruction to control the running direction and speed of the vehicle. The central control platform controls the vehicle to run according to the steering parameters and the speed parameters, and the steering parameters can be transmitted to a steering wheel controller of the vehicle through the central control platform, and the speed parameters can be transmitted to a speed controller of the vehicle. The central control platform can also feed back the results of steering control and speed control to the vehicle-mounted equipment, so that a closed-loop control system is formed.

In this embodiment, the vehicle-mounted device determines the target driving instruction according to the fused video data and the target driving plan. In this way, the determined target driving instruction is more reliable. And hiding, sending a more reliable target driving instruction to the central control platform, and controlling the vehicle to run by the central control platform according to the steering parameter and the speed parameter, so that more reliable vehicle auxiliary driving can be provided.

In one embodiment, as shown in fig. 3, a vehicle driving assistance method is provided, which is illustrated by taking the method as an example applied to the roadside apparatus 104 in fig. 1, and comprises the following steps:

s301, road side video data are obtained and sent to a cloud service platform.

The road side video data is video data collected by road side equipment. And the roadside video data collected by each roadside are sent to the cloud service platform, and the cloud service platform performs overall management.

And S303, receiving global path planning information aiming at the target vehicle, which is sent by the cloud service platform based on the road side video data.

The target vehicle is a vehicle for which the cloud service platform performs global path planning. For example, a driver on a target vehicle may send vehicle-based positioning information and destination information to a cloud service platform via an on-board device on the vehicle to request global path planning information for the vehicle. And the road side equipment receives the global path planning information aiming at the target vehicle, which is sent by the cloud service platform based on the road side video data.

The cloud service platform may collect traffic data based on the roadside video data. The traffic data is data of traffic conditions obtained by analyzing the road side traffic data, for example, the traffic data may include at least one of traffic congestion information and traffic flow information. Therefore, the cloud service platform can determine the global path planning information by combining the traffic data, so that the reliability of the global path planning information is improved. The traffic data can be obtained by analyzing roadside video data by roadside equipment and sent to the cloud service platform. The traffic data can be obtained by analyzing the road side video data through the cloud service platform, so that the resource consumption of road side equipment can be reduced, and the cost of the road side equipment is reduced. It should also be noted that the traffic congestion information may also be used to train a traffic congestion model. The congestion level is determined through the traffic congestion model, and the determined traffic congestion level can be stored in a database for later analysis, such as data statistical analysis of the traffic congestion condition.

The cloud service platform can also collect road information data based on the road side video data, so that data support is provided for global path planning of the cloud service platform. The road information data may be road information data obtained by analyzing by the roadside device according to the roadside video data and the position information of the roadside device. The road information data can also be obtained by analyzing according to road side video data sent by the road side terminal and vehicle-mounted video data sent by the vehicle-mounted equipment by the cloud service platform. The analysis of the road information data can be realized by a trained neural network model. Therefore, the cloud service platform can fuse the road information data and determine the global path planning information, and therefore the reliability of the global path planning information is improved.

The cloud service platform can also collect real-time traffic situation information based on roadside video data. The traffic situation information may be sent by the road side device, and the road side device analyzes the road side video data to obtain the traffic situation information, and sends the traffic situation information to the cloud service platform. The traffic situation information may also be obtained by analyzing received road side video data sent by the road side device by the cloud service platform. The traffic situation information includes status information of each traffic participant. For example, the roadside video data includes a pedestrian, which has a moving direction and a moving speed at a certain time. Therefore, the cloud service platform can fuse historical traffic data and real-time traffic situation to determine global path planning information. Thus, the reliability of the global path planning information is improved.

S305, recognizing the lane and the obstacle of the road side video data to obtain lane information and obstacle information.

The roadside device can identify the lane and the obstacle in each frame of image of the roadside video data in an image identification mode. The lane information includes road signs, lane tracks, and the like, wherein one road includes at least one lane. The obstacle information may include obstacle position information, obstacle size, and the like.

And S307, determining local path planning information of the target vehicle according to the lane information, the obstacle information and the global path planning information.

The road side equipment can acquire starting point position information and end point position information of the target vehicle in the target local area according to the global path planning information; then, a local path plan of the target vehicle is determined according to the starting point position information, the end point position information, the lane information and the obstacle information.

Furthermore, the road side equipment can also acquire starting point direction information and end point direction information of the target vehicle in the target local area according to the global path planning information; and then optimizing the local path planning of the target vehicle according to the starting point direction information and the end point direction information. Specifically, if only the positions of the start point and the end point are known, the local path plan is put into the global path plan in consideration of the direction of the start point and the end point, which is not necessarily optimal. Therefore, the direction factors of the starting point and the end point are added, so that the local path planning can still be the optimal path planning on the premise of global path planning.

When the global path planning is optimized and updated, the road side equipment can also optimize and update the local path planning information of the target vehicle obtained before the optimization and update according to the lane information, the obstacle information and the global path planning information again.

And S309, when the target vehicle is in the transmission range, transmitting the local path planning information to the vehicle-mounted equipment of the target vehicle.

Each road side device has a corresponding transmission range, and when a target vehicle enters the transmission range of the road side device, the road side device sends local path planning information of a local area corresponding to the road side device to vehicle-mounted equipment of the target vehicle. Therefore, the vehicle-mounted equipment can obtain the target driving plan by combining the local path planning information and the global path planning information.

Based on the vehicle auxiliary driving method of the embodiment, the road side equipment acquires road side video data and sends the road side video data to the cloud service platform; the method comprises the steps that road side equipment receives global path planning information aiming at a target vehicle, which is sent by a cloud service platform based on road side video data; the road side equipment identifies the lane and the obstacle from the road side video data to obtain lane information and obstacle information; the roadside equipment determines local path planning information of the target vehicle according to the lane information, the obstacle information and the global path planning information; and when the target vehicle is in the transmission range, the road side equipment sends the local path planning information to the vehicle-mounted equipment of the target vehicle. Therefore, global path planning is carried out through the cloud service platform, the traffic efficiency of the whole traffic network can be improved, traffic jam is reduced, and the decision-making capability of vehicle auxiliary driving is improved; the road side equipment determines local path planning information of the target vehicle based on the global path planning, and sends the local path planning information to vehicle-mounted equipment of the target vehicle when the target vehicle enters a transmission range. Therefore, the road side equipment comprehensively considers the global path planning, the local lane information and the obstacle information to obtain the local path planning, so that the reliability of the local path planning information can be improved, and the running safety of the vehicle can be improved.

In one embodiment, in order to enable the vehicle-mounted equipment to know roadside video data collected by the roadside equipment, the view field of the vehicle-mounted equipment is expanded, and the driving safety of a vehicle is improved. The method further comprises at least one of the following two cases:

(1) and when the target vehicle is in the transmission range, if a road side video request sent by the vehicle-mounted equipment of the target vehicle is received, sending road side video data to the vehicle-mounted equipment of the target vehicle.

Thus, a way of passively transmitting roadside video data can be provided for roadside devices. The road side device passively receives the road side video request sent by the vehicle-mounted device and then passively sends the road side video data based on the road side device to the vehicle-mounted device of the target vehicle according to the road side video request, so that the view field range of the vehicle-mounted device can be expanded.

(2) And when the target vehicle is in the transmission range, if the road side video data are analyzed to obtain an analysis result of road abnormity, the road side video data are sent to the vehicle-mounted equipment of the target vehicle.

Therefore, a way of actively transmitting roadside video data is provided for roadside equipment. The roadside device actively analyzes the roadside video data to obtain an analysis result of whether the road is abnormal. And when the analysis result is that the road is abnormal, transmitting the road side video data to the vehicle-mounted equipment of the target vehicle, so that when the road is abnormal, namely a state threatening the driving of the driver is about to occur, reminding the driver on the vehicle that the road in front is abnormal. When the analysis result is that the road is normal, it may not be necessary to transmit the roadside video data to the in-vehicle device of the target vehicle.

When the road side equipment analyzes the road side video data, the video frame image of the road side video data can be identified through the image identification algorithm of the abnormal road condition. The image recognition algorithm of the abnormal road condition mainly comprises the following steps: obstacle and road surface pothole identification, daytime visibility detection, rockfall and shed object detection, flame detection in a tunnel and the like. Conditions of roadway anomalies include, but are not limited to, the presence of obstacles such as rockfall, sprinkles; potholes on the road surface; the daytime visibility is lower than a preset value; the tunnel is internally provided with flames. The roadside device preprocesses the image data during processing, and the preprocessing processes comprise image enhancement, noise reduction, defogging, snow and rain elimination and the like. After the preprocessing is completed, the roadside device can further transform, reconstruct and synthesize the preprocessing result to realize the reconstruction of the image.

In one embodiment, when the target vehicle is within the transmission range, the local path planning information is sent to the vehicle-mounted device of the target vehicle, and the method includes the following steps: calibrating the local path planning information through a high-precision map; and when the target vehicle is in the transmission range, sending the calibrated local path planning information to the vehicle-mounted equipment of the target vehicle.

A high-precision map refers to a map that is at least accurate to each lane. Such as a Baidu map, a Gade map, etc. The roadside device may calibrate the local path plan in conjunction with the high-precision map. For example, when the road information and lane information involved in the local route planning are significantly different from the road information in the high-precision map, the road information and lane information in the local route planning are calibrated in accordance with the high-precision map. In the calibration process, wrong lane information in the local path planning information can be corrected, and the reliability of the local path planning information can be further improved, so that the vehicle-mounted equipment can receive more reliable local path planning information. Thus, the safety of the vehicle running can be further improved.

It should be understood that although the various steps in the flow charts of fig. 2-3 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2-3 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.

In one embodiment, as shown in fig. 4, there is provided a vehicle auxiliary driving apparatus corresponding to the above-described vehicle auxiliary method operated in an in-vehicle device, including:

the information acquisition and forwarding module 402 is configured to acquire positioning information and destination information based on a vehicle, and send the positioning information and the destination information to a cloud service platform;

a global planning receiving module 404, configured to receive global path planning information sent by the cloud service platform based on the positioning information and the destination information;

a local planning obtaining module 406, configured to obtain local path planning information determined based on global path planning;

and a target planning fusion module 408, configured to fuse the local path planning information and the global path planning information to obtain a target driving plan of the vehicle.

The vehicle driving assisting device firstly acquires positioning information and destination information based on a vehicle and sends the positioning information and the destination information to the cloud service platform; and then receiving global path planning information, wherein the global path planning information is determined by the cloud service platform based on the positioning information and the destination information. And when the vehicle is in the transmission range of the target road side equipment, receiving local path planning information sent by the target road side equipment based on the global path planning. And finally, fusing the local path planning information and the global path planning information to obtain the target driving plan of the vehicle. Therefore, global path planning is carried out through the cloud service platform, the traffic efficiency of the whole traffic network can be improved, traffic jam is reduced, and the decision-making capability of vehicle auxiliary driving is improved; the global path planning information and the local path planning information are fused through the vehicle-mounted equipment, so that the global path planning and the local path planning can be comprehensively considered by the vehicle-mounted equipment during vehicle auxiliary driving, the reliability of target path planning is improved, and the driving safety of the vehicle can be improved.

In one embodiment, the local plan obtaining module 406 is configured to receive local path plan information sent by the target road side device based on the global path plan when the vehicle is within the transmission range of the target road side device.

In one embodiment, the apparatus further comprises:

the road side video receiving module is used for receiving road side video data sent by the target road side equipment when the vehicle is in the transmission range of the target road side equipment;

the vehicle-mounted video acquisition module is used for acquiring vehicle-mounted video data based on a vehicle;

In one embodiment, the video data fusion module includes:

and the visual field distance adjusting unit is used for adjusting the visual field distance of the fused road side video data and the vehicle-mounted video data and the obtained fused video data according to the target visual field distance.

In one embodiment, the roadside video receiving module includes:

the road side video request unit is used for sending a road side video request to target road side equipment in a transmission range;

In one embodiment, the apparatus further comprises:

and the driving instruction forwarding module is used for sending the target driving instruction to the central control platform, and the central control platform controls the vehicle to run according to the steering parameter and the speed parameter.

In one embodiment, as shown in fig. 5, there is provided a vehicle driving assistance device corresponding to the above-described vehicle assistance method operating on a roadside apparatus, including:

the video acquiring and forwarding module 501 is configured to acquire road side video data and send the road side video data to a cloud service platform;

the global planning receiving module 503 is configured to receive global path planning information, which is sent by the cloud service platform based on the roadside video data and is for the target vehicle;

the video data identification module 505 is configured to identify lanes and obstacles on the road side video data to obtain lane information and obstacle information;

the local path planning module 507 is used for determining local path planning information of the target vehicle according to the lane information, the obstacle information and the global path planning information;

and a local path forwarding module 509, configured to send the local path planning information to the vehicle-mounted device of the target vehicle when the target vehicle is within the transmission range.

The vehicle auxiliary driving device acquires road side video data and sends the road side video data to the cloud service platform; receiving global path planning information aiming at a target vehicle, which is sent by a cloud service platform based on roadside video data; recognizing lanes and obstacles on the road side video data to obtain lane information and obstacle information; determining local path planning information of the target vehicle according to the lane information, the obstacle information and the global path planning information; and when the target vehicle is in the transmission range, sending the local path planning information to the vehicle-mounted equipment of the target vehicle. Therefore, global path planning is carried out through the cloud service platform, the traffic efficiency of the whole traffic network can be improved, traffic jam is reduced, and the decision-making capability of vehicle auxiliary driving is improved; the road side equipment determines local path planning information of the target vehicle based on the global path planning, and sends the local path planning information to vehicle-mounted equipment of the target vehicle when the target vehicle enters a transmission range. Thus, the global path planning, the local lane information and the obstacle information are comprehensively considered to obtain the local path planning, so that the reliability of the local path planning information can be improved, and the driving safety of the vehicle can be improved.

the video passive sending module is used for sending road side video data to the vehicle-mounted equipment of the target vehicle if a road side video request sent by the vehicle-mounted equipment of the target vehicle is received when the target vehicle is in the transmission range;

and the video active sending module is used for sending the road side video data to the vehicle-mounted equipment of the target vehicle if the road side video data are analyzed to obtain an analysis result of road abnormity when the target vehicle is in the transmission range.

In one embodiment, the apparatus further comprises a path planning calibration module;

the path planning calibration module is used for calibrating the local path planning information through the high-precision map;

and the local path forwarding module is used for sending the calibrated local path planning information to the vehicle-mounted equipment of the target vehicle when the target vehicle is in the transmission range.

In one embodiment, a computer device is provided, which may be a terminal or a server, and its internal structure diagram may be as shown in fig. 6. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a vehicle assisted driving method.

Those skilled in the art will appreciate that the architecture shown in fig. 6 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, there is provided a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

acquiring positioning information and destination information based on a vehicle, and sending the positioning information and the destination information to a cloud service platform;

receiving global path planning information sent by a cloud service platform based on positioning information and destination information;

when the vehicle is in the transmission range of the target road side equipment, receiving local path planning information sent by the target road side equipment based on global path planning;

receiving global path planning information aiming at a target vehicle, which is sent by a cloud service platform based on roadside video data;

recognizing lanes and obstacles on the road side video data to obtain lane information and obstacle information;

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A vehicle assisted driving method, the method comprising:

2. The method of claim 1, further comprising:

acquiring vehicle-mounted video data based on the vehicle;

3. The method according to claim 2, wherein the fusing the roadside video data and the vehicle-mounted video data to obtain fused video data comprises:

4. The method according to claim 2, wherein after fusing the roadside video data and the vehicle-mounted video data to obtain fused video data, further comprising:

5. The method of claim 1, wherein obtaining local path plan information determined based on the global path plan comprises:

6. A vehicle assisted driving method, the method comprising:

7. The method of claim 6, further comprising at least one of:

8. A vehicle driving assist apparatus, the apparatus comprising:

9. A vehicle driving assist apparatus, the apparatus comprising:

10. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.